1.1 --- a/docs/wiki/FilesystemAccess Thu Jun 23 23:05:52 2022 +0200
1.2 +++ b/docs/wiki/FilesystemAccess Sat Jun 25 22:50:02 2022 +0200
1.3 @@ -1,13 +1,11 @@
1.4 = Filesystem Access =
1.5
1.6 -Within the filesystem server library, a number of different abstractions and
1.7 -mechanisms are employed to provide access to filesystem objects.
1.8 +Within the [[ServerLibrary|filesystem server library]], a number of different
1.9 +abstractions and mechanisms are employed to provide access to filesystem
1.10 +objects. An overview of these abstractions is presented below.
1.11
1.12 -<<TableOfContents(2,3)>>
1.13 -
1.14 -The abstractions or components used in the library are organised as follows.
1.15 -
1.16 -######## A graph showing the relationships between library components
1.17 +######## A graph showing the relationships between filesystem access
1.18 +######## components
1.19
1.20 {{{#!graphviz
1.21 #format svg
1.22 @@ -20,281 +18,237 @@
1.23 subgraph {
1.24 rank=same;
1.25
1.26 - Resource -> DirectoryResource -> FilePager
1.27 - [dir=none,style=dotted];
1.28 + Opener_note [shape=note,style=filled,fillcolor=gold,label="Exposes\nfile open\noperation"];
1.29 + Opener;
1.30 +
1.31 + Opener_note -> Opener [dir=none,style=dotted];
1.32 }
1.33
1.34 subgraph {
1.35 rank=same;
1.36
1.37 - Provider -> DirectoryProvider -> FileProvider
1.38 - [dir=none,style=dotted];
1.39 + ResourceRegistry_note [shape=note,style=filled,fillcolor=gold,label="Records\nopen file\nsessions"];
1.40 + ResourceRegistry;
1.41 +
1.42 + ResourceRegistry_note -> ResourceRegistry [dir=none,style=dotted];
1.43 }
1.44
1.45 subgraph {
1.46 rank=same;
1.47
1.48 - Accessor -> Ext2FileAccessor
1.49 - [dir=none,style=dotted];
1.50 + Resource_note [shape=note,style=filled,fillcolor=gold,label="Provides\nfilesystem\ncontent"];
1.51 + Resource [label="Resource\n(Pager)"];
1.52 +
1.53 + Resource_note -> Resource [dir=none,style=dotted];
1.54 + }
1.55 +
1.56 + subgraph {
1.57 + rank=same;
1.58 +
1.59 + Provider_note [shape=note,style=filled,fillcolor=gold,label="Represents\nopen file"];
1.60 + Provider;
1.61 +
1.62 + Provider_note -> Provider [dir=none,style=dotted];
1.63 }
1.64
1.65 subgraph {
1.66 rank=same;
1.67
1.68 - DirectoryAccessor -> Ext2DirectoryAccessor
1.69 - [dir=none,style=dotted];
1.70 + PageMapper_note [shape=note,style=filled,fillcolor=gold,label="Provides\npopulated\nfile pages"];
1.71 + PageMapper;
1.72 +
1.73 + PageMapper_note -> PageMapper [dir=none,style=dotted];
1.74 + }
1.75 +
1.76 + ProviderRegistry [shape=record,label="<head> ProviderRegistry | ... | { file-n | provider-n } | ... "];
1.77 +
1.78 + subgraph {
1.79 + rank=same;
1.80 +
1.81 + FileOpening_note [shape=note,style=filled,fillcolor=gold,label="Exposes\nfilesystem\nfunctionality"];
1.82 + FileOpening;
1.83 +
1.84 + FileOpening_note -> FileOpening [dir=none,style=dotted];
1.85 }
1.86
1.87 subgraph {
1.88 - node [shape=ellipse];
1.89 rank=same;
1.90
1.91 - Directory [label="dir"];
1.92 - File [label="dir/file"];
1.93 + Accessor_note [shape=note,style=filled,fillcolor=gold,label="Populates\nfile pages"];
1.94 + Accessor;
1.95 + Accessor_note -> Accessor [dir=none,style=dotted];
1.96 }
1.97
1.98 - DirectoryResource -> DirectoryProvider -> Ext2DirectoryAccessor -> Directory;
1.99 - FilePager -> FileProvider -> Ext2FileAccessor -> File;
1.100 -}
1.101 -}}}
1.102 -
1.103 -########
1.104 -
1.105 -This document uses C-style interface syntax since the components operating
1.106 -within the described mechanisms will themselves be described using such
1.107 -syntax.
1.108 -
1.109 -== Accountable ==
1.110 -
1.111 -This interface provides the following operations:
1.112 -
1.113 -{{{
1.114 -void attach()
1.115 -unsigned int detach()
1.116 -}}}
1.117 + /* Opening a file. */
1.118
1.119 -Its purpose is to provide reference counting for components, with the `attach`
1.120 -operation incrementing the number of users of a component, and with the
1.121 -`detach` operation decrementing the number of users and returning the
1.122 -resulting number of users. When components no longer have any attached users,
1.123 -they may perform operations to tidy up after themselves and permit their
1.124 -deallocation.
1.125 -
1.126 -== Accessors ==
1.127 -
1.128 -Accessors provide the means of accessing filesystem object data and metadata.
1.129 -Conceptually, files and directories are both exposed by accessors, although
1.130 -the interfaces and mechanisms may differ between these types of objects.
1.131 -
1.132 -=== Directory Accessors ===
1.133 + Opener -> ResourceRegistry -> Resource;
1.134 + ResourceRegistry -> FileOpening;
1.135
1.136 -Currently, directory accessors provide support for traversing directory
1.137 -listings, obtaining the relevant filesystem metadata using the underlying
1.138 -filesystem access library.
1.139 -
1.140 -=== File Accessors ===
1.141 -
1.142 -File content is accessed through an interface with the following operations:
1.143 -
1.144 -{{{
1.145 -void close()
1.146 -void fill(Flexpage *flexpage)
1.147 -void flush(Flexpage *flexpage)
1.148 -offset_t get_size()
1.149 -void set_size(offset_t size)
1.150 -}}}
1.151 + Provider -> PageMapper;
1.152 + FileOpening -> Accessor;
1.153
1.154 -The operations need to be supported with actual filesystem operations. For
1.155 -example, ext2-based filesystems employ a specific abstraction which invokes
1.156 -library functions provided by the `libext2fs` package.
1.157 -
1.158 -== Providers ==
1.159 -
1.160 -Providers encapsulate the essential functionality for accessing filesystem
1.161 -objects. Implementing the `Accountable` interface, they are shared by
1.162 -resources and discarded when no resources are using them.
1.163 + /* Closing a file. */
1.164
1.165 -The following operations are supported by providers:
1.166 -
1.167 -{{{
1.168 -ProviderRegistry *registry()
1.169 -long make_resource(offset_t *size, object_flags_t *object_flags, Resource **resource)
1.170 -bool removal_pending()
1.171 -void remove()
1.172 -void remove_pending(bool remove)
1.173 -}}}
1.174 -
1.175 -Providers are associated with filesystem objects in a registry which can be
1.176 -obtained from each provider using the `registry` operation.
1.177 + Resource -> Provider -> ProviderRegistry:head;
1.178
1.179 -Providers also support the creation of resources through which each user of a
1.180 -provider exercises its use of that provider. The `make_resource` operation
1.181 -performs the creation and returns size, flags and resource instance details.
1.182 -
1.183 -The removal of providers can be directed using the `remove_pending` and
1.184 -`remove` operations. Where `remove_pending` has been called with a true value
1.185 -as its parameter, the `removal_pending` operation will also return a true
1.186 -value, and upon the provider being discarded, the `remove` operation will be
1.187 -invoked. The `remove` operation performs the appropriate underlying operation
1.188 -to remove the object being provided.
1.189 -
1.190 -Typically, removal of providers is managed by the provider registry when
1.191 -resources are closed and detached from providers.
1.192 + /* Open file management. */
1.193
1.194 -######## A graph showing the removal mechanism
1.195 -
1.196 -{{{#!graphviz
1.197 -#format svg
1.198 -#transform notugly
1.199 -digraph provider_removal {
1.200 - node [fontsize="12.0",fontname="sans-serif",shape=box];
1.201 - edge [fontsize="12.0",fontname="sans-serif"];
1.202 - rankdir=LR;
1.203 -
1.204 - ResourceServer -> Resource [label="close"];
1.205 - Resource -> ProviderRegistry [label="detach"];
1.206 - ProviderRegistry -> Provider [label="remove"];
1.207 + ResourceRegistry -> ProviderRegistry:head;
1.208 }
1.209 }}}
1.210
1.211 ########
1.212
1.213 -== Resources ==
1.214 -
1.215 -Resources are objects accessed by clients that support a basic level of
1.216 -accounting and management.
1.217 -
1.218 -The base interface of a resource is as follows:
1.219 -
1.220 -{{{
1.221 -void activate()
1.222 -void close()
1.223 -}}}
1.224 -
1.225 -Activation of a resource is an optional operation that performs any
1.226 -initialisation before a resource is made available to its user.
1.227 -
1.228 -In practice, other operations are required to make resources useful.
1.229 -
1.230 -In some cases, resources provide the mechanism by which each user of a
1.231 -filesystem object may access that object independently. They would then
1.232 -effectively provide a session in which accesses can occur.
1.233 -
1.234 -=== Directory Resources ===
1.235 -
1.236 -Directory resources primarily expose the contents of directories in the
1.237 -filesystem to a user. They employ directory accessors which concern themselves
1.238 -with the actual filesystem content.
1.239 +An `Opener` requests a `Resource` from a `ResourceRegistry`, each `Resource`
1.240 +acting as a [[ServerLibrary#Pager|`Pager`]] and providing the actual access
1.241 +mechanism to file content for a particular program. Since many programs may
1.242 +access the same file simultaneously, a `Provider` object represents a file
1.243 +opened in the system, retaining a `PageMapper` that coordinates the collective
1.244 +access to the file (see the [[Paging|paging documentation]] for more
1.245 +information).
1.246
1.247 -[[Components#Directories|Directory components]] are provided using directory
1.248 -resources.
1.249 -
1.250 -<<Anchor(Pager)>>
1.251 -=== Pagers or File Resources ===
1.252 -
1.253 -Pagers are resources that support dataspace access operations, thus allowing
1.254 -the resources to expose filesystem content in mapped memory regions to a
1.255 -particular user of the object providing the content.
1.256 -
1.257 -[[Components#Files|File components]] and [[Components#Pipes|pipe components]]
1.258 -are provided using pagers.
1.259 +A `ProviderRegistry` maintains the record of opened files, yielding a new
1.260 +`Provider` for an unopened file or an existing `Provider` for a file that is
1.261 +already open. The `ProviderRegistry` monitors the usage of files, discarding
1.262 +any `Provider` no longer in use.
1.263
1.264 -=== Filesystem Resources ===
1.265 +== Opening a File ==
1.266
1.267 -Filesystem resources provide the entry point for access to a filesystem by
1.268 -other components or programs. Since filesystems often enforce identity-based
1.269 -access controls, a filesystem resource will typically support the
1.270 -`open_for_user` operation in various forms, with the result of this operation
1.271 -being the instantiation of an `OpenerResource` configured for the indicated
1.272 -user identity.
1.273 +Opening a file involves the identification of the filesystem object involved,
1.274 +the acquisition of a `Provider` representing the file, and the creation of a
1.275 +`Resource` to deliver file content to the requesting program.
1.276
1.277 -== Registries ==
1.278 -
1.279 -The basic mechanism for obtaining a resource involves a registry, as
1.280 -illustrated by the following diagram.
1.281 -
1.282 -######## A graph showing the use of a registry in obtaining resources
1.283 +######## A graph showing the opening mechanism for an already-open file
1.284
1.285 {{{#!graphviz
1.286 #format svg
1.287 #transform notugly
1.288 -digraph registry {
1.289 +digraph opening_open {
1.290 node [fontsize="12.0",fontname="sans-serif",shape=box];
1.291 edge [fontsize="12.0",fontname="sans-serif"];
1.292 rankdir=LR;
1.293
1.294 subgraph {
1.295 - node [label="OpenerContextResource"];
1.296 rank=same;
1.297
1.298 - OpenerContextResource1 -> OpenerContextResource2 [dir=none,style=dotted];
1.299 + Opener_note [shape=note,style=filled,fillcolor=gold,label="Exposes\nfile open\noperation"];
1.300 + Opener;
1.301 +
1.302 + Opener_note -> Opener [dir=none,style=dotted];
1.303 }
1.304
1.305 subgraph {
1.306 - node [label="OpenerResource"];
1.307 rank=same;
1.308
1.309 - OpenerResource1 -> OpenerResource2 [dir=none,style=dotted];
1.310 + ResourceRegistry_note [shape=note,style=filled,fillcolor=gold,label="Records\nopen file\nsessions"];
1.311 + ResourceRegistry;
1.312 +
1.313 + Resource_returned [label="Resource\n(Pager)"];
1.314 + Resource_returned_note [shape=note,style=filled,fillcolor=gold,label="Provides\nfilesystem\ncontent"];
1.315 +
1.316 + ResourceRegistry_note -> ResourceRegistry [dir=none,style=dotted];
1.317 + Resource_returned -> Resource_returned_note [dir=none,style=dotted];
1.318 }
1.319
1.320 subgraph {
1.321 rank=same;
1.322
1.323 - ResourceRegistry -> Resource;
1.324 + Resource [label="Resource\n(Pager)"];
1.325 +
1.326 + FileOpening_note [shape=note,style=filled,fillcolor=gold,label="Exposes\nfilesystem\nfunctionality"];
1.327 + FileOpening;
1.328 +
1.329 + FileOpening_note -> FileOpening [dir=none,style=dotted];
1.330 }
1.331
1.332 subgraph {
1.333 - rank=same;
1.334 + rank=max;
1.335 +
1.336 + ProviderRegistry_note [shape=note,style=filled,fillcolor=gold,label="Provides\nopen files"];
1.337 + ProviderRegistry;
1.338
1.339 - ProviderRegistry -> Provider;
1.340 + Provider;
1.341 + Provider_note [shape=note,style=filled,fillcolor=gold,label="Represents\nopen file"];
1.342 +
1.343 + ProviderRegistry_note -> ProviderRegistry [dir=none,style=dotted];
1.344 + Provider -> Provider_note [dir=none,style=dotted];
1.345 }
1.346
1.347 - OpenerContextResource1 -> OpenerResource1 [label="open"];
1.348 - OpenerResource1 -> ResourceRegistry [label="get_resource"];
1.349 - ResourceRegistry -> ProviderRegistry [label="get/set"];
1.350 + /* Obtaining a resource from the registry. */
1.351 +
1.352 + Opener -> ResourceRegistry [label="get_resource"];
1.353 + ResourceRegistry -> Resource_returned [dir=none];
1.354 + Resource_returned -> Opener;
1.355 +
1.356 + /* Obtaining a provider. */
1.357
1.358 - Provider -> Resource -> OpenerResource2 -> OpenerContextResource2;
1.359 + ResourceRegistry -> FileOpening [label="get_fileid"];
1.360 + ResourceRegistry -> ProviderRegistry [label="get(fileid)"];
1.361 + ProviderRegistry -> Provider;
1.362 +
1.363 + /* Obtaining the resource from the provider. */
1.364 +
1.365 + ResourceRegistry -> Provider [label="make_resource"];
1.366 + Provider -> Resource [dir=none];
1.367 + Resource -> ResourceRegistry;
1.368 }
1.369 }}}
1.370
1.371 ########
1.372
1.373 -The `ResourceRegistry` coordinates access to filesystem resources and, through
1.374 -synchronisation, prevents conflicting operations from occurring concurrently.
1.375 -For example, a removal operation on a file may not be allowed to occur while
1.376 -an opening operation is in progress.
1.377 +Where a provider does not already exist, with the file not having been opened
1.378 +already, some extra interactions occur:
1.379
1.380 -To achieve this, a common lock is employed to serialise access, with any
1.381 -underlying filesystem operations being initiated from the registry while the
1.382 -lock is held. An object providing the `FileOpening` interface for a filesystem
1.383 -provides such operations, and the following interaction pattern is thereby
1.384 -employed:
1.385 -
1.386 -######## A graph showing the registry coordinating filesystem operations
1.387 +######## A graph showing opening mechanism details for an unopened file
1.388
1.389 {{{#!graphviz
1.390 #format svg
1.391 #transform notugly
1.392 -digraph registry_operations {
1.393 +digraph opening_unopened {
1.394 node [fontsize="12.0",fontname="sans-serif",shape=box];
1.395 edge [fontsize="12.0",fontname="sans-serif"];
1.396 rankdir=LR;
1.397
1.398 - OpenerContextResource -> OpenerResource -> ResourceRegistry -> FileOpening;
1.399 + Opener;
1.400 +
1.401 + ResourceRegistry [label="ResourceRegistry"];
1.402 +
1.403 + subgraph {
1.404 + rank=same;
1.405 +
1.406 + Provider_note [shape=note,style=filled,fillcolor=gold,label="Created\nand set in\nregistry"];
1.407 + Provider [label="Provider"];
1.408 +
1.409 + Accessor [label="Accessor"];
1.410 + Accessor_note [shape=note,style=filled,fillcolor=gold,label="Created\nfor provider"];
1.411 +
1.412 + Provider_note -> Provider [dir=none,style=dotted];
1.413 + Accessor -> Accessor_note [dir=none,style=dotted];
1.414 + }
1.415 +
1.416 + subgraph {
1.417 + rank=max;
1.418 +
1.419 + ProviderRegistry_note [shape=note,style=filled,fillcolor=gold,label="Records\nopen files"];
1.420 + ProviderRegistry [label="ProviderRegistry"];
1.421 + FileOpening [label="FileOpening"];
1.422 + FileOpening_note [shape=note,style=filled,fillcolor=gold,label="Exposes\nfilesystem\nfunctionality"];
1.423 +
1.424 + ProviderRegistry_note -> ProviderRegistry [dir=none,style=dotted];
1.425 + FileOpening -> FileOpening_note [dir=none,style=dotted];
1.426 + }
1.427 +
1.428 + Opener -> ResourceRegistry [dir=none,style=dashed];
1.429 +
1.430 + /* Obtaining a new provider. */
1.431 +
1.432 + ResourceRegistry -> FileOpening [label="make_accessor"];
1.433 + FileOpening -> Accessor [dir=none];
1.434 + Accessor -> ResourceRegistry;
1.435 +
1.436 + ResourceRegistry -> Provider [dir=none];
1.437 + Provider -> ProviderRegistry [label="set(fileid)"];
1.438 }
1.439 }}}
1.440
1.441 ########
1.442 -
1.443 -Since the `ResourceRegistry` functionality is generic, it could be specialised
1.444 -for each filesystem or be configured with an appropriate reference to a
1.445 -`FileOpening` object. The `OpenerResource` would then be generic, invoking the
1.446 -registry which would, in turn, invoke the opening object.
1.447 -
1.448 -However, the chosen approach is to permit `OpenerResource` objects to
1.449 -implement the `FileOpening` interface for each filesystem, meaning that the
1.450 -`ResourceRegistry` will end up being called by the opener and then invoking
1.451 -the opener in return. This is slightly more convenient than the alternative
1.452 -since the opener can be configured with a given user identity, and such
1.453 -identity details will ultimately be employed when accessing the underlying
1.454 -filesystem itself.
3.1 --- a/docs/wiki/Paging Thu Jun 23 23:05:52 2022 +0200
3.2 +++ b/docs/wiki/Paging Sat Jun 25 22:50:02 2022 +0200
3.3 @@ -88,8 +88,8 @@
3.4
3.5 ########
3.6
3.7 -A [[FilesystemAccess#Pager|`Pager`]], being a resource that provides
3.8 -filesystem content to clients for a particular file, requests pages from the
3.9 +A [[ServerLibrary#Pager|`Pager`]], being a resource that provides filesystem
3.10 +content to clients for a particular file, requests pages from the
3.11 `PageMapper`, itself having the role of supplying populated pages of content
3.12 from the given file. The `PageMapper` relies on the `AccessMap` to discover
3.13 existing pages for any accessed region of a file, involving the `Pages` object
3.14 @@ -111,6 +111,15 @@
3.15 the queue are themselves recycled and it is then removed from the queue for
3.16 recycling.
3.17
3.18 +== Satisfying Requests for Pages ==
3.19 +
3.20 +The general procedure for satisfying requests for pages is as follows:
3.21 +
3.22 + 1. Attempt to find an existing page for the affected file region.
3.23 + 1. With an existing page, attempt to reserve the page for issuing.
3.24 + 1. Without an existing or reserved page, obtain a fresh page for populating.
3.25 + 1. Queue the page for eventual reuse.
3.26 +
3.27 Since many files are likely to be in use, and since a fixed amount of memory
3.28 may be shared to provide access to file content, the interaction between
3.29 different `PageMapper` objects, operating on behalf of different files, must
3.30 @@ -131,11 +140,13 @@
3.31 /* First page mapper activities. */
3.32
3.33 subgraph {
3.34 - rank=min;
3.35 + rank=same;
3.36
3.37 PageMapper1_note [shape=note,style=filled,fillcolor=gold,label="Find\nflexpage"];
3.38 PageMapper1 [label="PageMapper"];
3.39
3.40 + Accessor1 [label="Accessor"];
3.41 +
3.42 PageMapper1_reserve_note [shape=note,style=filled,fillcolor=gold,label="Reserve\nflexpage"];
3.43 PageMapper1_reserve [label="PageMapper"];
3.44
3.45 @@ -154,6 +165,7 @@
3.46
3.47 AccessMap_note [shape=note,style=filled,fillcolor=gold,label="Provides\nflexpages\ntentatively"];
3.48 AccessMap1 [label="AccessMap"];
3.49 +
3.50 Flexpage1 [label="Flexpage",shape=note,style=dashed];
3.51 Flexpage1_note [shape=note,style=filled,fillcolor=gold,label="Needs reservation\nif found\nby mapper"];
3.52
3.53 @@ -164,7 +176,6 @@
3.54 Flexpage1_queue [label="Flexpage",shape=note];
3.55
3.56 AccessMap_note -> AccessMap1 [dir=none,style=dotted];
3.57 - Flexpage1 -> Flexpage1_note [dir=none,style=dotted];
3.58 Flexpage1_note -> Flexpage1_reserve [dir=none,style=dotted];
3.59 Flexpage1_reserved -> Flexpage1_reserved_note [dir=none,style=dotted];
3.60
3.61 @@ -228,6 +239,8 @@
3.62 PageMapper2_note [shape=note,style=filled,fillcolor=gold,label="Obtain\nflexpage"];
3.63 PageMapper2 [label="PageMapper"];
3.64
3.65 + Accessor2 [label="Accessor"];
3.66 +
3.67 PageMapper2_record_note [shape=note,style=filled,fillcolor=gold,label="Record\nflexpage"];
3.68 PageMapper2_record [label="PageMapper"];
3.69
3.70 @@ -268,13 +281,16 @@
3.71
3.72 /* Flexpage removal from the access map. */
3.73
3.74 - Pages -> AccessMap1 [color="red",label="remove"];
3.75 + Pages -> AccessMap1 [color="red",label="remove\n(via PageMapper)"];
3.76 + AccessMap1 -> PageMapper1 [color="red",label="flush"];
3.77 + PageMapper1 -> Accessor1 [color="red",label="flush"];
3.78
3.79 /* Second pager dataflow. */
3.80
3.81 PageMapper2 -> Pages [color="red",label="flexpage"];
3.82 Pages -> Flexpage2 [color="red",dir=none];
3.83 Flexpage2 -> PageMapper2 [color="red"];
3.84 + PageMapper2 -> Accessor2 [label="fill"];
3.85
3.86 PageMapper2_record -> Flexpage2_reserved [dir=none,label="insert"];
3.87 Flexpage2_reserved -> AccessMap2;
3.88 @@ -290,16 +306,17 @@
3.89
3.90 When a `PageMapper` requests a page from the `Pages` object and when the
3.91 `Pages` object needs to reclaim such a previously issued page, the page
3.92 -obtained from the head of the queue is removed from the `AccessMap` that
3.93 -currently owns it. The "owner" of such a page is employing the page to satisfy
3.94 -requests for content in a region of a particular file. If modified, the page's
3.95 -contents may be flushed to the underlying file when it is removed from the
3.96 -owner. As a consequence of its removal, the `AccessMap` will no longer be able
3.97 -to offer this page to satisfy a request for data in the affected region to its
3.98 -`PageMapper`. A reclaimed page is then returned to the `PageMapper` requiring
3.99 -it, and since the page will no longer reside in the `PageQueue`, it will be
3.100 -exclusively available for that `PageMapper`, being populated with data from
3.101 -the underlying file and then issued to its client.
3.102 +obtained from the head of the queue is removed from the `AccessMap` employed
3.103 +by the `PageMapper` that currently owns it. The "owner" of such a page is
3.104 +employing the page to satisfy requests for content in a region of a particular
3.105 +file. If modified, the page's contents may be flushed to the underlying file
3.106 +when it is removed from the owner. As a consequence of its removal, the
3.107 +`AccessMap` will no longer be able to offer this page to satisfy a request for
3.108 +data in the affected region to its `PageMapper`. A reclaimed page is then
3.109 +returned to the `PageMapper` requiring it, and since the page will no longer
3.110 +reside in the `PageQueue`, it will be exclusively available for that
3.111 +`PageMapper`, being populated with data from the underlying file and then
3.112 +issued to its client.
3.113
3.114 == Reserving Available Pages ==
3.115
4.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
4.2 +++ b/docs/wiki/ServerLibrary Sat Jun 25 22:50:02 2022 +0200
4.3 @@ -0,0 +1,245 @@
4.4 += Server Library =
4.5 +
4.6 +Within the filesystem server library, a number of different abstractions and
4.7 +mechanisms are employed to provide access to filesystem objects.
4.8 +
4.9 +<<TableOfContents(2,3)>>
4.10 +
4.11 +This document uses C-style interface syntax since the components operating
4.12 +within the described mechanisms will themselves be described using such
4.13 +syntax.
4.14 +
4.15 +== Accountable ==
4.16 +
4.17 +This interface provides the following operations:
4.18 +
4.19 +{{{
4.20 +void attach()
4.21 +unsigned int detach()
4.22 +}}}
4.23 +
4.24 +Its purpose is to provide reference counting for components, with the `attach`
4.25 +operation incrementing the number of users of a component, and with the
4.26 +`detach` operation decrementing the number of users and returning the
4.27 +resulting number of users. When components no longer have any attached users,
4.28 +they may perform operations to tidy up after themselves and permit their
4.29 +deallocation.
4.30 +
4.31 +== Accessors ==
4.32 +
4.33 +Accessors provide the means of accessing filesystem object data and metadata.
4.34 +Conceptually, files and directories are both exposed by accessors, although
4.35 +the interfaces and mechanisms may differ between these types of objects.
4.36 +
4.37 +=== Directory Accessors ===
4.38 +
4.39 +Currently, directory accessors provide support for traversing directory
4.40 +listings, obtaining the relevant filesystem metadata using the underlying
4.41 +filesystem access library.
4.42 +
4.43 +=== File Accessors ===
4.44 +
4.45 +File content is accessed through an interface with the following operations:
4.46 +
4.47 +{{{
4.48 +void close()
4.49 +void fill(Flexpage *flexpage)
4.50 +void flush(Flexpage *flexpage)
4.51 +offset_t get_size()
4.52 +void set_size(offset_t size)
4.53 +}}}
4.54 +
4.55 +The operations need to be supported with actual filesystem operations. For
4.56 +example, ext2-based filesystems employ a specific abstraction which invokes
4.57 +library functions provided by the `libext2fs` package.
4.58 +
4.59 +== Providers ==
4.60 +
4.61 +Providers encapsulate the essential functionality for accessing filesystem
4.62 +objects. Implementing the `Accountable` interface, they are shared by
4.63 +resources and discarded when no resources are using them.
4.64 +
4.65 +The following operations are supported by providers:
4.66 +
4.67 +{{{
4.68 +ProviderRegistry *registry()
4.69 +long make_resource(offset_t *size, object_flags_t *object_flags, Resource **resource)
4.70 +bool removal_pending()
4.71 +void remove()
4.72 +void remove_pending(bool remove)
4.73 +}}}
4.74 +
4.75 +Providers are associated with filesystem objects in a registry which can be
4.76 +obtained from each provider using the `registry` operation.
4.77 +
4.78 +Providers also support the creation of resources through which each user of a
4.79 +provider exercises its use of that provider. The `make_resource` operation
4.80 +performs the creation and returns size, flags and resource instance details.
4.81 +
4.82 +The removal of providers can be directed using the `remove_pending` and
4.83 +`remove` operations. Where `remove_pending` has been called with a true value
4.84 +as its parameter, the `removal_pending` operation will also return a true
4.85 +value, and upon the provider being discarded, the `remove` operation will be
4.86 +invoked. The `remove` operation performs the appropriate underlying operation
4.87 +to remove the object being provided.
4.88 +
4.89 +Typically, removal of providers is managed by the provider registry when
4.90 +resources are closed and detached from providers.
4.91 +
4.92 +######## A graph showing the removal mechanism
4.93 +
4.94 +{{{#!graphviz
4.95 +#format svg
4.96 +#transform notugly
4.97 +digraph provider_removal {
4.98 + node [fontsize="12.0",fontname="sans-serif",shape=box];
4.99 + edge [fontsize="12.0",fontname="sans-serif"];
4.100 + rankdir=LR;
4.101 +
4.102 + ResourceServer -> Resource [label="close"];
4.103 + Resource -> ProviderRegistry [label="detach"];
4.104 + ProviderRegistry -> Provider [label="remove"];
4.105 +}
4.106 +}}}
4.107 +
4.108 +########
4.109 +
4.110 +== Resources ==
4.111 +
4.112 +Resources are objects accessed by clients that support a basic level of
4.113 +accounting and management.
4.114 +
4.115 +The base interface of a resource is as follows:
4.116 +
4.117 +{{{
4.118 +void activate()
4.119 +void close()
4.120 +}}}
4.121 +
4.122 +Activation of a resource is an optional operation that performs any
4.123 +initialisation before a resource is made available to its user.
4.124 +
4.125 +In practice, other operations are required to make resources useful.
4.126 +
4.127 +In some cases, resources provide the mechanism by which each user of a
4.128 +filesystem object may access that object independently. They would then
4.129 +effectively provide a session in which accesses can occur.
4.130 +
4.131 +=== Directory Resources ===
4.132 +
4.133 +Directory resources primarily expose the contents of directories in the
4.134 +filesystem to a user. They employ directory accessors which concern themselves
4.135 +with the actual filesystem content.
4.136 +
4.137 +[[Components#Directories|Directory components]] are provided using directory
4.138 +resources.
4.139 +
4.140 +<<Anchor(Pager)>>
4.141 +=== Pagers or File Resources ===
4.142 +
4.143 +Pagers are resources that support dataspace access operations, thus allowing
4.144 +the resources to expose filesystem content in mapped memory regions to a
4.145 +particular user of the object providing the content.
4.146 +
4.147 +[[Components#Files|File components]] and [[Components#Pipes|pipe components]]
4.148 +are provided using pagers.
4.149 +
4.150 +=== Filesystem Resources ===
4.151 +
4.152 +Filesystem resources provide the entry point for access to a filesystem by
4.153 +other components or programs. Since filesystems often enforce identity-based
4.154 +access controls, a filesystem resource will typically support the
4.155 +`open_for_user` operation in various forms, with the result of this operation
4.156 +being the instantiation of an `OpenerResource` configured for the indicated
4.157 +user identity.
4.158 +
4.159 +== Registries ==
4.160 +
4.161 +The basic mechanism for obtaining a resource involves a registry, as
4.162 +illustrated by the following diagram.
4.163 +
4.164 +######## A graph showing the use of a registry in obtaining resources
4.165 +
4.166 +{{{#!graphviz
4.167 +#format svg
4.168 +#transform notugly
4.169 +digraph registry {
4.170 + node [fontsize="12.0",fontname="sans-serif",shape=box];
4.171 + edge [fontsize="12.0",fontname="sans-serif"];
4.172 + rankdir=LR;
4.173 +
4.174 + subgraph {
4.175 + node [label="OpenerContextResource"];
4.176 + rank=same;
4.177 +
4.178 + OpenerContextResource1 -> OpenerContextResource2 [dir=none,style=dotted];
4.179 + }
4.180 +
4.181 + subgraph {
4.182 + node [label="OpenerResource"];
4.183 + rank=same;
4.184 +
4.185 + OpenerResource1 -> OpenerResource2 [dir=none,style=dotted];
4.186 + }
4.187 +
4.188 + subgraph {
4.189 + rank=same;
4.190 +
4.191 + ResourceRegistry -> Resource;
4.192 + }
4.193 +
4.194 + subgraph {
4.195 + rank=same;
4.196 +
4.197 + ProviderRegistry -> Provider;
4.198 + }
4.199 +
4.200 + OpenerContextResource1 -> OpenerResource1 [label="open"];
4.201 + OpenerResource1 -> ResourceRegistry [label="get_resource"];
4.202 + ResourceRegistry -> ProviderRegistry [label="get/set"];
4.203 +
4.204 + Provider -> Resource -> OpenerResource2 -> OpenerContextResource2;
4.205 +}
4.206 +}}}
4.207 +
4.208 +########
4.209 +
4.210 +The `ResourceRegistry` coordinates access to filesystem resources and, through
4.211 +synchronisation, prevents conflicting operations from occurring concurrently.
4.212 +For example, a removal operation on a file may not be allowed to occur while
4.213 +an opening operation is in progress.
4.214 +
4.215 +To achieve this, a common lock is employed to serialise access, with any
4.216 +underlying filesystem operations being initiated from the registry while the
4.217 +lock is held. An object providing the `FileOpening` interface for a filesystem
4.218 +provides such operations, and the following interaction pattern is thereby
4.219 +employed:
4.220 +
4.221 +######## A graph showing the registry coordinating filesystem operations
4.222 +
4.223 +{{{#!graphviz
4.224 +#format svg
4.225 +#transform notugly
4.226 +digraph registry_operations {
4.227 + node [fontsize="12.0",fontname="sans-serif",shape=box];
4.228 + edge [fontsize="12.0",fontname="sans-serif"];
4.229 + rankdir=LR;
4.230 +
4.231 + OpenerContextResource -> OpenerResource -> ResourceRegistry -> FileOpening;
4.232 +}
4.233 +}}}
4.234 +
4.235 +########
4.236 +
4.237 +Since the `ResourceRegistry` functionality is generic, it could be specialised
4.238 +for each filesystem or be configured with an appropriate reference to a
4.239 +`FileOpening` object. The `OpenerResource` would then be generic, invoking the
4.240 +registry which would, in turn, invoke the opening object.
4.241 +
4.242 +However, the chosen approach is to permit `OpenerResource` objects to
4.243 +implement the `FileOpening` interface for each filesystem, meaning that the
4.244 +`ResourceRegistry` will end up being called by the opener and then invoking
4.245 +the opener in return. This is slightly more convenient than the alternative
4.246 +since the opener can be configured with a given user identity, and such
4.247 +identity details will ultimately be employed when accessing the underlying
4.248 +filesystem itself.