# HG changeset patch # User Paul Boddie # Date 1654984812 -7200 # Node ID b596e5cc0db35e4741184d2491091758579f8fcc # Parent 5b2e5ce0f9958f278badfeb094af1ac0e7c8a8c7 Renamed Mechanisms to FilesystemAccess, added Directories and Paging. diff -r 5b2e5ce0f995 -r b596e5cc0db3 docs/wiki/Components --- a/docs/wiki/Components Sat Jun 11 18:29:28 2022 +0200 +++ b/docs/wiki/Components Sun Jun 12 00:00:12 2022 +0200 @@ -251,6 +251,7 @@ {{{ mmap(in offset_t position, in offset_t length, + in offset_t start_visible, in offset_t end_visible, out offset_t start_pos, out offset_t end_pos, out offset_t size) }}} diff -r 5b2e5ce0f995 -r b596e5cc0db3 docs/wiki/Directories --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/docs/wiki/Directories Sun Jun 12 00:00:12 2022 +0200 @@ -0,0 +1,273 @@ += Directories = + +Filesystem directories contain collections of files, other directories, along +with other filesystem objects. Directories can be listed, permitting +filesystem clients to peruse the contents of each directory. However, the act +of listing a directory may impose certain constraints on the operations being +conducted on the directory during the listing process. + +Operations that affect directory contents include the following: + + * Creating a new object + * Removing an object + * Moving an object out of the directory + * Moving an object into the directory + +It is conceivable that for some filesystems, these operations will not be able +to rely on directly modifying the directory since a listing operation may be +in progress. Consequently, a number of rules would be imposed for such +filesystems. It appears that Ext2-based filesystems accessed via libext2fs do +not require such rules to be applied outside the library due to the design of +the filesystem structures and the behaviour of the library itself. + +== Providers == + +Providers of filesystem objects are created when at least one filesystem +client is accessing such an object, with the object regarded as being +'''open'''. By definition, a provider will exist until all clients have closed +or discarded their means of accessing the object. + +An object is regarded as '''accessible''' if it can still be opened by +filesystem clients. Where an object has been removed, it will no longer be +accessible since clients should not be able to see the object in the +filesystem any more. (The actual filesystem residing in storage may not have +been updated for such a removal, this being the eventual outcome of a removal +operation.) + +Providers can be regarded as '''accessible''' if they maintain access to +objects that are open and accessible. Clients opening objects will only gain +access to accessible providers. Providers can become inaccessible if an object +is removed even if the object (and its provider) is still open. + +Initially, providers will correspond to objects resident in the stored +filesystem. Thus, looking up an object using its path will involve the +filesystem, yielding a file identifier that can be used to map to a provider. + +{{{ +def get_provider_from_filesystem(path): + fileid = opening.get_fileid(path) + + if not fileid: + return error + + return find_provider(fileid) +}}} + +However, providers may not always immediately correspond to objects resident +in the stored filesystem. Where an object is created but cannot be immediately +registered in the contents of a directory, it must be registered separately +and attempts to open this object must consult this separate mapping of +filenames to providers. + +{{{ +def get_provider(path): + provider = find_created_provider(path) + + if provider: + return provider + + return get_provider_from_filesystem(path) +}}} + +Providers that are inaccessible need to be retained until they are closed. +However, since they are no longer accessible, they should not be available to +the provider lookup operations. + +When providers are closed, they are removed from any mapping in which they +could be found. Inaccessible providers that have been retained outside the +identifier or filename mappings will represent objects that should be removed +from their parent directory. Accessible providers retained by the mappings +will represent objects that should be created in their parent directory. + +Whether object removal or creation will occur depends on whether the parent +directory is able to safely perform these operations concurrently with other +operations (such as servicing a listing operation) or whether such operations +will need to be deferred until they can be safely performed. + +== Object Removal == + +Filesystem object removal involves obtaining any provider of the object. Where +a provider can be obtained, it will be made inaccessible and removal will be +requested. The actual object will not be removed at least until it is closed +because it may still receive and provide data. (Unlinking open files is a +feature of Unix systems.) + +Where a provider cannot be obtained, an attempt will be made to obtain the +parent directory provider, and if this succeeds, it indicates that the +directory is being accessed and must therefore be notified of the intention to +eventually remove the object concerned. + +Without any provider of the object, and where no directory provider can be +obtained, the object can be immediately removed from the filesystem. + +{{{ +def remove_object(path): + provider = get_provider(path) + + if provider: + return make_provider_inaccessible(provider) + + dirname, basename = split(path) + directory_provider = get_provider(dirname) + + if directory_provider: + return directory_provider.remove_pending(basename) + + return filesystem.remove_object(path) +}}} + +It should be noted that with no accessible provider, any attempt to create a +file with the same name as a removed file should cause a new "version" of the +file to be created. + +== Object Creation == + +Filesystem object creation occurs when no existing provider can be found for a +named object and where creation is requested. Any new object will be +accessible and remain so until or unless it is removed. + +Whether an object is created in the filesystem storage depends on whether the +parent directory is being used. + +If a parent directory is not being used, the object can be registered in its +proper location in the filesystem itself, yielding a file identifier. + +If a parent directory is being used, the object cannot be immediately +registered in its proper location, but since data may still need to be written +to storage, a temporary location is employed, yielding a file identifier. + +For an object created in a temporary location, a temporary mapping from the +path of the object to the provider is established, with the parent directory +being notified of the pending object creation. + +{{{ +def create_object(path): + provider = get_provider(path) + + if provider: + return error + + dirname, basename = split(path) + directory_provider = get_provider(dirname) + + if directory_provider: + provider = make_provider(get_temporary_location(path)) + directory_provider.create_pending(provider) + return register_created_provider(path, provider) + + provider = make_provider(path) + return register_provider(provider) +}}} + +== Created Providers == + +Created providers are retained by the registry until their files can be +committed to the filesystem in the desired location. + +The `register_created_provider` operation establishes a mapping from a path to +a provider that can be queried using the `find_created_provider` operation. + +Created providers are deregistered when they are closed. This will occur when +the parent directory of the file to be committed is closed. At that time, the +created file will be moved from its temporary location to the desired +location. + +== Inaccessible Providers == + +Inaccessible providers are retained by the registry until they are closed. + +Where the provided file resides in a non-temporary location, closure will +occur when the parent directory of the provided file is closed, this having +obstructed the removal of the file. At that time, the provided file will be +removed. + +Where the provided file resides in a temporary location, closure will not be +obstructed by any directory and will cause the file to be removed immediately. + +== Directory Provider Closure == + +A critical event that typically initiates housekeeping work for created and +removed files is the closure of the parent directory of those files. + +Directory providers support the `create_pending` and `remove_pending` +operations that register the details of affected files. When closure occurs, +the provider will contact the registry to initiate the necessary work to +relocate created files and to remove files. + + + + + Where an object provider is notified of pending removal, it will initiate + removal of the actual object upon closure, checking for an active parent + provider. + + Where a parent provider is notified of pending removal, it will initiate + removal of the actual object upon closure, checking for an active object + provider. + + Object opening logic would need to be adjusted. Consider the case where a + file is removed but still open. + + Any attempt to open a file with the same name must ignore the original and + open a new file of that name which would be stored elsewhere until such + time as the original is actually removed from its location. + + This new file might have a different identifier since the original file + would still exist and retain the identifier. + + Any such new, but uncommitted, file must be accessible by other clients + attempting to open a file of that name. However, removal operations must + also be possible, making this version of the open file also unaccessible + to new opening clients. + + Therefore, any providers used by opening clients must only refer to a + version of a file that is pending removal if no other version has been + created. Once a new version has been created, any provider pending removal + must be relegated to a separate collection. + + Storage of new versions of files could be confined to special directories + that are hidden from clients. + + The opening logic would be as follows: + + provider = find_provider(path) + + if provider: + if provider.pending_removal(): + relegate(provider) + provider = create_provider(path, hidden=True) + else: + provider = create_provider(path, hidden=False) + + # Have provider. + + return provider.make_resource() + + A provider would normally be obtained by consulting the filesystem + implementation. However, new versions of files replacing ones pending + removal will reside in locations that are different to the intended + location of the file. Consequently, the registry needs to maintain its own + mapping of paths to providers for such file versions. + + The opportunity to remove a file definitively arises when the following + conditions are satisfied: + + * The parent directory is not open in some way + * The original provider for the file is no longer open + + For new versions of a removed file that have themselves been marked as + pending removal, their closure is sufficient to remove their filesystem + resources. + + However, the registry must maintain a path entry for any active version + of a removed file until that version is closed. Thus, the following + conditions apply: + + * The parent directory (of the original file) is not open in some way + * The provider of the new version is no longer open + + It will not be generally possible to open the hidden directory containing + new file versions. Therefore, the transfer of the file to its intended + location will not risk interfering with any operations on the hidden + directory. + diff -r 5b2e5ce0f995 -r b596e5cc0db3 docs/wiki/FilesystemAccess --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/docs/wiki/FilesystemAccess Sun Jun 12 00:00:12 2022 +0200 @@ -0,0 +1,300 @@ += Filesystem Access = + +Within the filesystem server library, a number of different abstractions and +mechanisms are employed to provide access to filesystem objects. + +<> + +The abstractions or components used in the library are organised as follows. + +######## A graph showing the relationships between library components + +{{{#!graphviz +#format svg +#transform notugly +digraph components { + node [fontsize="12.0",fontname="sans-serif",shape=box]; + edge [fontsize="12.0",fontname="sans-serif"]; + rankdir=LR; + + subgraph { + rank=same; + + Resource -> DirectoryResource -> FilePager + [dir=none,style=dotted]; + } + + subgraph { + rank=same; + + Provider -> DirectoryProvider -> FileProvider + [dir=none,style=dotted]; + } + + subgraph { + rank=same; + + Accessor -> Ext2FileAccessor + [dir=none,style=dotted]; + } + + subgraph { + rank=same; + + DirectoryAccessor -> Ext2DirectoryAccessor + [dir=none,style=dotted]; + } + + subgraph { + node [shape=ellipse]; + rank=same; + + Directory [label="dir"]; + File [label="dir/file"]; + } + + DirectoryResource -> DirectoryProvider -> Ext2DirectoryAccessor -> Directory; + FilePager -> FileProvider -> Ext2FileAccessor -> File; +} +}}} + +######## + +This document uses C-style interface syntax since the components operating +within the described mechanisms will themselves be described using such +syntax. + +== Accountable == + +This interface provides the following operations: + +{{{ +void attach() +unsigned int detach() +}}} + +Its purpose is to provide reference counting for components, with the `attach` +operation incrementing the number of users of a component, and with the +`detach` operation decrementing the number of users and returning the +resulting number of users. When components no longer have any attached users, +they may perform operations to tidy up after themselves and permit their +deallocation. + +== Accessors == + +Accessors provide the means of accessing filesystem object data and metadata. +Conceptually, files and directories are both exposed by accessors, although +the interfaces and mechanisms may differ between these types of objects. + +=== Directory Accessors === + +Currently, directory accessors provide support for traversing directory +listings, obtaining the relevant filesystem metadata using the underlying +filesystem access library. + +=== File Accessors === + +File content is accessed through an interface with the following operations: + +{{{ +void close() +void fill(Flexpage *flexpage) +void flush(Flexpage *flexpage) +offset_t get_size() +void set_size(offset_t size) +}}} + +The operations need to be supported with actual filesystem operations. For +example, ext2-based filesystems employ a specific abstraction which invokes +library functions provided by the `libext2fs` package. + +== Providers == + +Providers encapsulate the essential functionality for accessing filesystem +objects. Implementing the `Accountable` interface, they are shared by +resources and discarded when no resources are using them. + +The following operations are supported by providers: + +{{{ +ProviderRegistry *registry() +long make_resource(offset_t *size, object_flags_t *object_flags, Resource **resource) +bool removal_pending() +void remove() +void remove_pending(bool remove) +}}} + +Providers are associated with filesystem objects in a registry which can be +obtained from each provider using the `registry` operation. + +Providers also support the creation of resources through which each user of a +provider exercises its use of that provider. The `make_resource` operation +performs the creation and returns size, flags and resource instance details. + +The removal of providers can be directed using the `remove_pending` and +`remove` operations. Where `remove_pending` has been called with a true value +as its parameter, the `removal_pending` operation will also return a true +value, and upon the provider being discarded, the `remove` operation will be +invoked. The `remove` operation performs the appropriate underlying operation +to remove the object being provided. + +Typically, removal of providers is managed by the provider registry when +resources are closed and detached from providers. + +######## A graph showing the removal mechanism + +{{{#!graphviz +#format svg +#transform notugly +digraph provider_removal { + node [fontsize="12.0",fontname="sans-serif",shape=box]; + edge [fontsize="12.0",fontname="sans-serif"]; + rankdir=LR; + + ResourceServer -> Resource [label="close"]; + Resource -> ProviderRegistry [label="detach"]; + ProviderRegistry -> Provider [label="remove"]; +} +}}} + +######## + +== Resources == + +Resources are objects accessed by clients that support a basic level of +accounting and management. + +The base interface of a resource is as follows: + +{{{ +void activate() +void close() +}}} + +Activation of a resource is an optional operation that performs any +initialisation before a resource is made available to its user. + +In practice, other operations are required to make resources useful. + +In some cases, resources provide the mechanism by which each user of a +filesystem object may access that object independently. They would then +effectively provide a session in which accesses can occur. + +=== Directory Resources === + +Directory resources primarily expose the contents of directories in the +filesystem to a user. They employ directory accessors which concern themselves +with the actual filesystem content. + +[[Components#Directories|Directory components]] are provided using directory +resources. + +<> +=== Pagers or File Resources === + +Pagers are resources that support dataspace access operations, thus allowing +the resources to expose filesystem content in mapped memory regions to a +particular user of the object providing the content. + +[[Components#Files|File components]] and [[Components#Pipes|pipe components]] +are provided using pagers. + +=== Filesystem Resources === + +Filesystem resources provide the entry point for access to a filesystem by +other components or programs. Since filesystems often enforce identity-based +access controls, a filesystem resource will typically support the +`open_for_user` operation in various forms, with the result of this operation +being the instantiation of an `OpenerResource` configured for the indicated +user identity. + +== Registries == + +The basic mechanism for obtaining a resource involves a registry, as +illustrated by the following diagram. + +######## A graph showing the use of a registry in obtaining resources + +{{{#!graphviz +#format svg +#transform notugly +digraph registry { + node [fontsize="12.0",fontname="sans-serif",shape=box]; + edge [fontsize="12.0",fontname="sans-serif"]; + rankdir=LR; + + subgraph { + node [label="OpenerContextResource"]; + rank=same; + + OpenerContextResource1 -> OpenerContextResource2 [dir=none,style=dotted]; + } + + subgraph { + node [label="OpenerResource"]; + rank=same; + + OpenerResource1 -> OpenerResource2 [dir=none,style=dotted]; + } + + subgraph { + rank=same; + + ResourceRegistry -> Resource; + } + + subgraph { + rank=same; + + ProviderRegistry -> Provider; + } + + OpenerContextResource1 -> OpenerResource1 [label="open"]; + OpenerResource1 -> ResourceRegistry [label="get_resource"]; + ResourceRegistry -> ProviderRegistry [label="get/set"]; + + Provider -> Resource -> OpenerResource2 -> OpenerContextResource2; +} +}}} + +######## + +The `ResourceRegistry` coordinates access to filesystem resources and, through +synchronisation, prevents conflicting operations from occurring concurrently. +For example, a removal operation on a file may not be allowed to occur while +an opening operation is in progress. + +To achieve this, a common lock is employed to serialise access, with any +underlying filesystem operations being initiated from the registry while the +lock is held. An object providing the `FileOpening` interface for a filesystem +provides such operations, and the following interaction pattern is thereby +employed: + +######## A graph showing the registry coordinating filesystem operations + +{{{#!graphviz +#format svg +#transform notugly +digraph registry_operations { + node [fontsize="12.0",fontname="sans-serif",shape=box]; + edge [fontsize="12.0",fontname="sans-serif"]; + rankdir=LR; + + OpenerContextResource -> OpenerResource -> ResourceRegistry -> FileOpening; +} +}}} + +######## + +Since the `ResourceRegistry` functionality is generic, it could be specialised +for each filesystem or be configured with an appropriate reference to a +`FileOpening` object. The `OpenerResource` would then be generic, invoking the +registry which would, in turn, invoke the opening object. + +However, the chosen approach is to permit `OpenerResource` objects to +implement the `FileOpening` interface for each filesystem, meaning that the +`ResourceRegistry` will end up being called by the opener and then invoking +the opener in return. This is slightly more convenient than the alternative +since the opener can be configured with a given user identity, and such +identity details will ultimately be employed when accessing the underlying +filesystem itself. diff -r 5b2e5ce0f995 -r b596e5cc0db3 docs/wiki/Filesystems --- a/docs/wiki/Filesystems Sat Jun 11 18:29:28 2022 +0200 +++ b/docs/wiki/Filesystems Sun Jun 12 00:00:12 2022 +0200 @@ -5,6 +5,12 @@ == Topics == - * [[Components]] and [[Mechanisms]] - * [[ClientLibrary|Client Library]] + * [[ClientLibrary|Client Library]] - convenience functions and structures to + access filesystem objects + * [[Components]] - server objects exposed at the system level that support + filesystem access + * [[FilesystemAccess|Filesystem Access]] - objects within filesystem servers + that manage filesystem resources and content + * [[Paging]] - the mechanism by which filesystem content is provided for use + by client programs * [[Users]] diff -r 5b2e5ce0f995 -r b596e5cc0db3 docs/wiki/Mechanisms --- a/docs/wiki/Mechanisms Sat Jun 11 18:29:28 2022 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,299 +0,0 @@ -= Mechanisms = - -Within the filesystem server library, a number of different abstractions and -mechanisms are employed to provide access to filesystem objects. - -<> - -The abstractions or components used in the library are organised as follows. - -######## A graph showing the relationships between library components - -{{{#!graphviz -#format svg -#transform notugly -digraph components { - node [fontsize="12.0",fontname="sans-serif",shape=box]; - edge [fontsize="12.0",fontname="sans-serif"]; - rankdir=LR; - - subgraph { - rank=same; - - Resource -> DirectoryResource -> FilePager - [dir=none,style=dotted]; - } - - subgraph { - rank=same; - - Provider -> DirectoryProvider -> FileProvider - [dir=none,style=dotted]; - } - - subgraph { - rank=same; - - Accessor -> Ext2FileAccessor - [dir=none,style=dotted]; - } - - subgraph { - rank=same; - - DirectoryAccessor -> Ext2DirectoryAccessor - [dir=none,style=dotted]; - } - - subgraph { - node [shape=ellipse]; - rank=same; - - Directory [label="dir"]; - File [label="dir/file"]; - } - - DirectoryResource -> DirectoryProvider -> Ext2DirectoryAccessor -> Directory; - FilePager -> FileProvider -> Ext2FileAccessor -> File; -} -}}} - -######## - -This document uses C-style interface syntax since the components operating -within the described mechanisms will themselves be described using such -syntax. - -== Accountable == - -This interface provides the following operations: - -{{{ -void attach() -unsigned int detach() -}}} - -Its purpose is to provide reference counting for components, with the `attach` -operation incrementing the number of users of a component, and with the -`detach` operation decrementing the number of users and returning the -resulting number of users. When components no longer have any attached users, -they may perform operations to tidy up after themselves and permit their -deallocation. - -== Accessors == - -Accessors provide the means of accessing filesystem object data and metadata. -Conceptually, files and directories are both exposed by accessors, although -the interfaces and mechanisms may differ between these types of objects. - -=== Directory Accessors === - -Currently, directory accessors provide support for traversing directory -listings, obtaining the relevant filesystem metadata using the underlying -filesystem access library. - -=== File Accessors === - -File content is accessed through an interface with the following operations: - -{{{ -void close() -void fill(Flexpage *flexpage) -void flush(Flexpage *flexpage) -offset_t get_size() -void set_size(offset_t size) -}}} - -The operations need to be supported with actual filesystem operations. For -example, ext2-based filesystems employ a specific abstraction which invokes -library functions provided by the `libext2fs` package. - -== Providers == - -Providers encapsulate the essential functionality for accessing filesystem -objects. Implementing the `Accountable` interface, they are shared by -resources and discarded when no resources are using them. - -The following operations are supported by providers: - -{{{ -ProviderRegistry *registry() -long make_resource(offset_t *size, object_flags_t *object_flags, Resource **resource) -bool removal_pending() -void remove() -void remove_pending(bool remove) -}}} - -Providers are associated with filesystem objects in a registry which can be -obtained from each provider using the `registry` operation. - -Providers also support the creation of resources through which each user of a -provider exercises its use of that provider. The `make_resource` operation -performs the creation and returns size, flags and resource instance details. - -The removal of providers can be directed using the `remove_pending` and -`remove` operations. Where `remove_pending` has been called with a true value -as its parameter, the `removal_pending` operation will also return a true -value, and upon the provider being discarded, the `remove` operation will be -invoked. The `remove` operation performs the appropriate underlying operation -to remove the object being provided. - -Typically, removal of providers is managed by the provider registry when -resources are closed and detached from providers. - -######## A graph showing the removal mechanism - -{{{#!graphviz -#format svg -#transform notugly -digraph provider_removal { - node [fontsize="12.0",fontname="sans-serif",shape=box]; - edge [fontsize="12.0",fontname="sans-serif"]; - rankdir=LR; - - ResourceServer -> Resource [label="close"]; - Resource -> ProviderRegistry [label="detach"]; - ProviderRegistry -> Provider [label="remove"]; -} -}}} - -######## - -== Resources == - -Resources are objects accessed by clients that support a basic level of -accounting and management. - -The base interface of a resource is as follows: - -{{{ -void activate() -void close() -}}} - -Activation of a resource is an optional operation that performs any -initialisation before a resource is made available to its user. - -In practice, other operations are required to make resources useful. - -In some cases, resources provide the mechanism by which each user of a -filesystem object may access that object independently. They would then -effectively provide a session in which accesses can occur. - -=== Directory Resources === - -Directory resources primarily expose the contents of directories in the -filesystem to a user. They employ directory accessors which concern themselves -with the actual filesystem content. - -[[Components#Directories|Directory components]] are provided using directory -resources. - -=== Pagers or File Resources === - -Pagers are resources that support dataspace access operations, thus allowing -the resources to expose filesystem content in mapped memory regions to a -particular user of the object providing the content. - -[[Components#Files|File components]] and [[Components#Pipes|pipe components]] -are provided using pagers. - -=== Filesystem Resources === - -Filesystem resources provide the entry point for access to a filesystem by -other components or programs. Since filesystems often enforce identity-based -access controls, a filesystem resource will typically support the -`open_for_user` operation in various forms, with the result of this operation -being the instantiation of an `OpenerResource` configured for the indicated -user identity. - -== Registries == - -The basic mechanism for obtaining a resource involves a registry, as -illustrated by the following diagram. - -######## A graph showing the use of a registry in obtaining resources - -{{{#!graphviz -#format svg -#transform notugly -digraph registry { - node [fontsize="12.0",fontname="sans-serif",shape=box]; - edge [fontsize="12.0",fontname="sans-serif"]; - rankdir=LR; - - subgraph { - node [label="OpenerContextResource"]; - rank=same; - - OpenerContextResource1 -> OpenerContextResource2 [dir=none,style=dotted]; - } - - subgraph { - node [label="OpenerResource"]; - rank=same; - - OpenerResource1 -> OpenerResource2 [dir=none,style=dotted]; - } - - subgraph { - rank=same; - - ResourceRegistry -> Resource; - } - - subgraph { - rank=same; - - ProviderRegistry -> Provider; - } - - OpenerContextResource1 -> OpenerResource1 [label="open"]; - OpenerResource1 -> ResourceRegistry [label="get_resource"]; - ResourceRegistry -> ProviderRegistry [label="get/set"]; - - Provider -> Resource -> OpenerResource2 -> OpenerContextResource2; -} -}}} - -######## - -The `ResourceRegistry` coordinates access to filesystem resources and, through -synchronisation, prevents conflicting operations from occurring concurrently. -For example, a removal operation on a file may not be allowed to occur while -an opening operation is in progress. - -To achieve this, a common lock is employed to serialise access, with any -underlying filesystem operations being initiated from the registry while the -lock is held. An object providing the `FileOpening` interface for a filesystem -provides such operations, and the following interaction pattern is thereby -employed: - -######## A graph showing the registry coordinating filesystem operations - -{{{#!graphviz -#format svg -#transform notugly -digraph registry_operations { - node [fontsize="12.0",fontname="sans-serif",shape=box]; - edge [fontsize="12.0",fontname="sans-serif"]; - rankdir=LR; - - OpenerContextResource -> OpenerResource -> ResourceRegistry -> FileOpening; -} -}}} - -######## - -Since the `ResourceRegistry` functionality is generic, it could be specialised -for each filesystem or be configured with an appropriate reference to a -`FileOpening` object. The `OpenerResource` would then be generic, invoking the -registry which would, in turn, invoke the opening object. - -However, the chosen approach is to permit `OpenerResource` objects to -implement the `FileOpening` interface for each filesystem, meaning that the -`ResourceRegistry` will end up being called by the opener and then invoking -the opener in return. This is slightly more convenient than the alternative -since the opener can be configured with a given user identity, and such -identity details will ultimately be employed when accessing the underlying -filesystem itself. diff -r 5b2e5ce0f995 -r b596e5cc0db3 docs/wiki/Paging --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/docs/wiki/Paging Sun Jun 12 00:00:12 2022 +0200 @@ -0,0 +1,110 @@ += Paging = + +Within filesystem servers, a number of abstractions are combined to support +[[FilesystemAccess|access to filesystem content]] through paging: the delivery +of content to clients in mapped memory. + +The general paging mechanism used to support paging is depicted below. + +######## A graph showing the general paging mechanism + +{{{#!graphviz +#format svg +#transform notugly +digraph paging { + node [fontsize="12.0",fontname="sans-serif",shape=box]; + edge [fontsize="12.0",fontname="sans-serif"]; + rankdir=LR; + + subgraph { + rank=same; + + Pager_note [shape=note,style=filled,fillcolor=gold,label="Provides\nfilesystem\ncontent"]; + Pager; + + Pager_note -> Pager [dir=none,style=dotted]; + } + + subgraph { + rank=same; + + PageMapper_note [shape=note,style=filled,fillcolor=gold,label="Provides\npopulated\nfile pages"]; + PageMapper; + + PageMapper_note -> PageMapper [dir=none,style=dotted]; + } + + subgraph { + rank=same; + + Accessor_note [shape=note,style=filled,fillcolor=gold,label="Populates\nfile pages"]; + Accessor; + + Accessor_note -> Accessor [dir=none,style=dotted]; + } + + AccessMap [shape=record,label=" AccessMap | + { offset-0 | flexpage-0 } | ... | + { offset-n | flexpage-n } | ..."]; + + subgraph { + rank=same; + + Pages_note [shape=note,style=filled,fillcolor=gold,label="Provides\nmemory\npages"]; + Pages; + + Pages_note -> Pages [dir=none,style=dotted]; + } + + PageQueue [shape=record,label=" PageQueue | + { owner-0 | flexpage-0 } | ... | + { owner-N | flexpage-N } | ..."]; + + Flexpage_offset_n [shape=record,label=" Flexpage | + offset | size | region"]; + + subgraph { + rank=same; + + Memory_note [shape=note,style=filled,fillcolor=gold,label="Allocates\nmemory\nregions"]; + Memory; + + Memory_note -> Memory [dir=none,style=dotted]; + } + + Region; + + /* Relationships. */ + + PageMapper -> Accessor; + PageMapper -> AccessMap:head; + Pager -> PageMapper -> Pages; + Pages -> Memory; + Pages -> PageQueue:head; + AccessMap:fpn -> Flexpage_offset_n:head; + PageQueue:fpN -> Flexpage_offset_n:head; + Flexpage_offset_n:r -> Region; +} +}}} + +######## + +A [[FilesystemAccess#Pager|Pager]], being a resource that provides filesystem +content to clients for a particular file, requests pages from the PageMapper, +itself having the role of supplying populated pages of content from the given +file. The PageMapper relies on the AccessMap to discover existing pages for +any accessed region of a file, involving the Pages object (or page collection) +to obtain new memory pages where existing pages are not available or are no +longer valid, and requesting the population of such pages through use of the +associated Accessor for the file concerned. + +The Pages object may either obtain new memory pages from a Memory object or +reclaim (or recycle) existing pages from a PageQueue, this recording all pages +that have been populated and supplied to clients. Since it may be desirable to +limit the number of pages employed to satisfy file access operations, the +PageQueue provides a kind of lending mechanism: pages are issued to clients +and then added to the end of the queue; where no new page can be supplied by a +Memory object, the issued page at the head of the queue is obtained for +recycling; ultimately, an issued page will remain valid for its user (the +client accessing its contents) until all pages in front of it in the queue are +themselves recycled and it is then removed from the queue for recycling.