1.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
1.2 +++ b/docs/wiki/FilesystemAccess Sun Jun 12 00:00:12 2022 +0200
1.3 @@ -0,0 +1,300 @@
1.4 += Filesystem Access =
1.5 +
1.6 +Within the filesystem server library, a number of different abstractions and
1.7 +mechanisms are employed to provide access to filesystem objects.
1.8 +
1.9 +<<TableOfContents(2,3)>>
1.10 +
1.11 +The abstractions or components used in the library are organised as follows.
1.12 +
1.13 +######## A graph showing the relationships between library components
1.14 +
1.15 +{{{#!graphviz
1.16 +#format svg
1.17 +#transform notugly
1.18 +digraph components {
1.19 + node [fontsize="12.0",fontname="sans-serif",shape=box];
1.20 + edge [fontsize="12.0",fontname="sans-serif"];
1.21 + rankdir=LR;
1.22 +
1.23 + subgraph {
1.24 + rank=same;
1.25 +
1.26 + Resource -> DirectoryResource -> FilePager
1.27 + [dir=none,style=dotted];
1.28 + }
1.29 +
1.30 + subgraph {
1.31 + rank=same;
1.32 +
1.33 + Provider -> DirectoryProvider -> FileProvider
1.34 + [dir=none,style=dotted];
1.35 + }
1.36 +
1.37 + subgraph {
1.38 + rank=same;
1.39 +
1.40 + Accessor -> Ext2FileAccessor
1.41 + [dir=none,style=dotted];
1.42 + }
1.43 +
1.44 + subgraph {
1.45 + rank=same;
1.46 +
1.47 + DirectoryAccessor -> Ext2DirectoryAccessor
1.48 + [dir=none,style=dotted];
1.49 + }
1.50 +
1.51 + subgraph {
1.52 + node [shape=ellipse];
1.53 + rank=same;
1.54 +
1.55 + Directory [label="dir"];
1.56 + File [label="dir/file"];
1.57 + }
1.58 +
1.59 + DirectoryResource -> DirectoryProvider -> Ext2DirectoryAccessor -> Directory;
1.60 + FilePager -> FileProvider -> Ext2FileAccessor -> File;
1.61 +}
1.62 +}}}
1.63 +
1.64 +########
1.65 +
1.66 +This document uses C-style interface syntax since the components operating
1.67 +within the described mechanisms will themselves be described using such
1.68 +syntax.
1.69 +
1.70 +== Accountable ==
1.71 +
1.72 +This interface provides the following operations:
1.73 +
1.74 +{{{
1.75 +void attach()
1.76 +unsigned int detach()
1.77 +}}}
1.78 +
1.79 +Its purpose is to provide reference counting for components, with the `attach`
1.80 +operation incrementing the number of users of a component, and with the
1.81 +`detach` operation decrementing the number of users and returning the
1.82 +resulting number of users. When components no longer have any attached users,
1.83 +they may perform operations to tidy up after themselves and permit their
1.84 +deallocation.
1.85 +
1.86 +== Accessors ==
1.87 +
1.88 +Accessors provide the means of accessing filesystem object data and metadata.
1.89 +Conceptually, files and directories are both exposed by accessors, although
1.90 +the interfaces and mechanisms may differ between these types of objects.
1.91 +
1.92 +=== Directory Accessors ===
1.93 +
1.94 +Currently, directory accessors provide support for traversing directory
1.95 +listings, obtaining the relevant filesystem metadata using the underlying
1.96 +filesystem access library.
1.97 +
1.98 +=== File Accessors ===
1.99 +
1.100 +File content is accessed through an interface with the following operations:
1.101 +
1.102 +{{{
1.103 +void close()
1.104 +void fill(Flexpage *flexpage)
1.105 +void flush(Flexpage *flexpage)
1.106 +offset_t get_size()
1.107 +void set_size(offset_t size)
1.108 +}}}
1.109 +
1.110 +The operations need to be supported with actual filesystem operations. For
1.111 +example, ext2-based filesystems employ a specific abstraction which invokes
1.112 +library functions provided by the `libext2fs` package.
1.113 +
1.114 +== Providers ==
1.115 +
1.116 +Providers encapsulate the essential functionality for accessing filesystem
1.117 +objects. Implementing the `Accountable` interface, they are shared by
1.118 +resources and discarded when no resources are using them.
1.119 +
1.120 +The following operations are supported by providers:
1.121 +
1.122 +{{{
1.123 +ProviderRegistry *registry()
1.124 +long make_resource(offset_t *size, object_flags_t *object_flags, Resource **resource)
1.125 +bool removal_pending()
1.126 +void remove()
1.127 +void remove_pending(bool remove)
1.128 +}}}
1.129 +
1.130 +Providers are associated with filesystem objects in a registry which can be
1.131 +obtained from each provider using the `registry` operation.
1.132 +
1.133 +Providers also support the creation of resources through which each user of a
1.134 +provider exercises its use of that provider. The `make_resource` operation
1.135 +performs the creation and returns size, flags and resource instance details.
1.136 +
1.137 +The removal of providers can be directed using the `remove_pending` and
1.138 +`remove` operations. Where `remove_pending` has been called with a true value
1.139 +as its parameter, the `removal_pending` operation will also return a true
1.140 +value, and upon the provider being discarded, the `remove` operation will be
1.141 +invoked. The `remove` operation performs the appropriate underlying operation
1.142 +to remove the object being provided.
1.143 +
1.144 +Typically, removal of providers is managed by the provider registry when
1.145 +resources are closed and detached from providers.
1.146 +
1.147 +######## A graph showing the removal mechanism
1.148 +
1.149 +{{{#!graphviz
1.150 +#format svg
1.151 +#transform notugly
1.152 +digraph provider_removal {
1.153 + node [fontsize="12.0",fontname="sans-serif",shape=box];
1.154 + edge [fontsize="12.0",fontname="sans-serif"];
1.155 + rankdir=LR;
1.156 +
1.157 + ResourceServer -> Resource [label="close"];
1.158 + Resource -> ProviderRegistry [label="detach"];
1.159 + ProviderRegistry -> Provider [label="remove"];
1.160 +}
1.161 +}}}
1.162 +
1.163 +########
1.164 +
1.165 +== Resources ==
1.166 +
1.167 +Resources are objects accessed by clients that support a basic level of
1.168 +accounting and management.
1.169 +
1.170 +The base interface of a resource is as follows:
1.171 +
1.172 +{{{
1.173 +void activate()
1.174 +void close()
1.175 +}}}
1.176 +
1.177 +Activation of a resource is an optional operation that performs any
1.178 +initialisation before a resource is made available to its user.
1.179 +
1.180 +In practice, other operations are required to make resources useful.
1.181 +
1.182 +In some cases, resources provide the mechanism by which each user of a
1.183 +filesystem object may access that object independently. They would then
1.184 +effectively provide a session in which accesses can occur.
1.185 +
1.186 +=== Directory Resources ===
1.187 +
1.188 +Directory resources primarily expose the contents of directories in the
1.189 +filesystem to a user. They employ directory accessors which concern themselves
1.190 +with the actual filesystem content.
1.191 +
1.192 +[[Components#Directories|Directory components]] are provided using directory
1.193 +resources.
1.194 +
1.195 +<<Anchor(Pager)>>
1.196 +=== Pagers or File Resources ===
1.197 +
1.198 +Pagers are resources that support dataspace access operations, thus allowing
1.199 +the resources to expose filesystem content in mapped memory regions to a
1.200 +particular user of the object providing the content.
1.201 +
1.202 +[[Components#Files|File components]] and [[Components#Pipes|pipe components]]
1.203 +are provided using pagers.
1.204 +
1.205 +=== Filesystem Resources ===
1.206 +
1.207 +Filesystem resources provide the entry point for access to a filesystem by
1.208 +other components or programs. Since filesystems often enforce identity-based
1.209 +access controls, a filesystem resource will typically support the
1.210 +`open_for_user` operation in various forms, with the result of this operation
1.211 +being the instantiation of an `OpenerResource` configured for the indicated
1.212 +user identity.
1.213 +
1.214 +== Registries ==
1.215 +
1.216 +The basic mechanism for obtaining a resource involves a registry, as
1.217 +illustrated by the following diagram.
1.218 +
1.219 +######## A graph showing the use of a registry in obtaining resources
1.220 +
1.221 +{{{#!graphviz
1.222 +#format svg
1.223 +#transform notugly
1.224 +digraph registry {
1.225 + node [fontsize="12.0",fontname="sans-serif",shape=box];
1.226 + edge [fontsize="12.0",fontname="sans-serif"];
1.227 + rankdir=LR;
1.228 +
1.229 + subgraph {
1.230 + node [label="OpenerContextResource"];
1.231 + rank=same;
1.232 +
1.233 + OpenerContextResource1 -> OpenerContextResource2 [dir=none,style=dotted];
1.234 + }
1.235 +
1.236 + subgraph {
1.237 + node [label="OpenerResource"];
1.238 + rank=same;
1.239 +
1.240 + OpenerResource1 -> OpenerResource2 [dir=none,style=dotted];
1.241 + }
1.242 +
1.243 + subgraph {
1.244 + rank=same;
1.245 +
1.246 + ResourceRegistry -> Resource;
1.247 + }
1.248 +
1.249 + subgraph {
1.250 + rank=same;
1.251 +
1.252 + ProviderRegistry -> Provider;
1.253 + }
1.254 +
1.255 + OpenerContextResource1 -> OpenerResource1 [label="open"];
1.256 + OpenerResource1 -> ResourceRegistry [label="get_resource"];
1.257 + ResourceRegistry -> ProviderRegistry [label="get/set"];
1.258 +
1.259 + Provider -> Resource -> OpenerResource2 -> OpenerContextResource2;
1.260 +}
1.261 +}}}
1.262 +
1.263 +########
1.264 +
1.265 +The `ResourceRegistry` coordinates access to filesystem resources and, through
1.266 +synchronisation, prevents conflicting operations from occurring concurrently.
1.267 +For example, a removal operation on a file may not be allowed to occur while
1.268 +an opening operation is in progress.
1.269 +
1.270 +To achieve this, a common lock is employed to serialise access, with any
1.271 +underlying filesystem operations being initiated from the registry while the
1.272 +lock is held. An object providing the `FileOpening` interface for a filesystem
1.273 +provides such operations, and the following interaction pattern is thereby
1.274 +employed:
1.275 +
1.276 +######## A graph showing the registry coordinating filesystem operations
1.277 +
1.278 +{{{#!graphviz
1.279 +#format svg
1.280 +#transform notugly
1.281 +digraph registry_operations {
1.282 + node [fontsize="12.0",fontname="sans-serif",shape=box];
1.283 + edge [fontsize="12.0",fontname="sans-serif"];
1.284 + rankdir=LR;
1.285 +
1.286 + OpenerContextResource -> OpenerResource -> ResourceRegistry -> FileOpening;
1.287 +}
1.288 +}}}
1.289 +
1.290 +########
1.291 +
1.292 +Since the `ResourceRegistry` functionality is generic, it could be specialised
1.293 +for each filesystem or be configured with an appropriate reference to a
1.294 +`FileOpening` object. The `OpenerResource` would then be generic, invoking the
1.295 +registry which would, in turn, invoke the opening object.
1.296 +
1.297 +However, the chosen approach is to permit `OpenerResource` objects to
1.298 +implement the `FileOpening` interface for each filesystem, meaning that the
1.299 +`ResourceRegistry` will end up being called by the opener and then invoking
1.300 +the opener in return. This is slightly more convenient than the alternative
1.301 +since the opener can be configured with a given user identity, and such
1.302 +identity details will ultimately be employed when accessing the underlying
1.303 +filesystem itself.