paul@392 | 1 | = Program Loading = |
paul@392 | 2 | |
paul@392 | 3 | The provision of [[Paging|paging]] of file content leads to the possibility of |
paul@392 | 4 | demand paging for programs, enabling them to be loaded into memory dynamically |
paul@392 | 5 | and to have only the active portions of those programs resident. To achieve |
paul@392 | 6 | this, programs must be appropriately initialised in new tasks, with a page |
paul@392 | 7 | fault handler configured to provide program file content whenever a region of |
paul@392 | 8 | the program payload is encountered that is not currently resident in memory. |
paul@392 | 9 | |
paul@564 | 10 | <<TableOfContents(2,3)>> |
paul@564 | 11 | |
paul@564 | 12 | == Program Initialisation == |
paul@564 | 13 | |
paul@564 | 14 | To load and initialise a program, a new task must be created, defining a |
paul@564 | 15 | separate address space for the program and allowing it to operate |
paul@564 | 16 | independently of other programs. For a program to actually run, a thread of |
paul@564 | 17 | execution must be created so that the program's instructions can be read and |
paul@573 | 18 | processed. Once running, the program must be able to interact with the system |
paul@573 | 19 | environment and to have its memory resources made available to it upon demand. |
paul@573 | 20 | |
paul@573 | 21 | A number of elements must be prepared to be able to run a program, these being |
paul@573 | 22 | summarised below. |
paul@573 | 23 | |
paul@573 | 24 | === Program Executable File === |
paul@573 | 25 | |
paul@573 | 26 | The program executable file is obtained and the program metadata loaded, |
paul@573 | 27 | identifying the memory regions and entry point (the location of the first |
paul@573 | 28 | instruction). |
paul@573 | 29 | |
paul@573 | 30 | === Task and Threads === |
paul@573 | 31 | |
paul@573 | 32 | A new task is created to hold the program. For this new task, new threads are |
paul@573 | 33 | created to run and support the program. One thread will run the actual |
paul@573 | 34 | program, and another will be used as the region mapper to handle page fault |
paul@573 | 35 | conditions. |
paul@573 | 36 | |
paul@573 | 37 | === Thread Resources === |
paul@573 | 38 | |
paul@573 | 39 | Memory is allocated for the new program's threads, particularly the stack for |
paul@573 | 40 | each thread. |
paul@573 | 41 | |
paul@573 | 42 | Each stack is populated with program argument and environment information. Of |
paul@573 | 43 | particular importance are the capabilities to be made available to the threads |
paul@573 | 44 | since these permit interaction with the rest of the system. |
paul@573 | 45 | |
paul@573 | 46 | The capabilities to be used by the threads must be mapped into the new task. |
paul@573 | 47 | |
paul@573 | 48 | === Thread Configuration === |
paul@573 | 49 | |
paul@573 | 50 | To connect the main thread of the program to its region mapper, an IPC gate |
paul@573 | 51 | must be created. This is mapped to the new task where the region mapper will |
paul@573 | 52 | bind to it, using it to expose its interface. |
paul@573 | 53 | |
paul@573 | 54 | Thread properties such as pagers and exception handlers are defined for the |
paul@573 | 55 | threads, with the main thread employing the newly created IPC gate capability |
paul@573 | 56 | to delegate paging to the region mapper. |
paul@564 | 57 | |
paul@392 | 58 | == Internal Page Fault Handlers == |
paul@392 | 59 | |
paul@392 | 60 | When satisfying page faults for a task, one approach involves situating the |
paul@392 | 61 | page fault handler within the task itself, this managing the available memory |
paul@392 | 62 | regions and employing receive windows when requesting memory pages. |
paul@392 | 63 | |
paul@392 | 64 | The general arrangement involving such internal page fault handlers for a |
paul@392 | 65 | program in a task is as follows: |
paul@392 | 66 | |
paul@392 | 67 | ######## A graph showing the internal paging mechanism |
paul@392 | 68 | |
paul@392 | 69 | {{{#!graphviz |
paul@392 | 70 | #format svg |
paul@392 | 71 | #transform notugly |
paul@392 | 72 | digraph internal_paging { |
paul@392 | 73 | node [fontsize="12.0",fontname="sans-serif",shape=box]; |
paul@392 | 74 | edge [fontsize="12.0",fontname="sans-serif"]; |
paul@392 | 75 | rankdir=LR; |
paul@392 | 76 | |
paul@392 | 77 | TaskMemory [shape=record,label="Task Memory |<s> stack | ... |<d> data | ... |<c> code"]; |
paul@392 | 78 | |
paul@392 | 79 | subgraph { |
paul@392 | 80 | rank=same; |
paul@392 | 81 | |
paul@392 | 82 | InternalPager; |
paul@392 | 83 | InternalPager_note [shape=note,style=filled,fillcolor=gold,label="Resides in\nsame task and\ndefines scope of\nfault resolution"]; |
paul@392 | 84 | |
paul@392 | 85 | InternalPager -> InternalPager_note [dir=none,style=dotted]; |
paul@392 | 86 | } |
paul@392 | 87 | |
paul@392 | 88 | subgraph { |
paul@392 | 89 | rank=same; |
paul@392 | 90 | |
paul@392 | 91 | Regions [shape=record,label="Regions | |
paul@392 | 92 | {<s> stack |<sd> stack-dataspace } | |
paul@392 | 93 | {<d> data | <dd> data-dataspace } | |
paul@392 | 94 | {<c> code |<cd> code-dataspace } |..."]; |
paul@392 | 95 | |
paul@392 | 96 | ReceiveFlexpage [shape=note,label="Flexpage\n(receive window)"]; |
paul@392 | 97 | |
paul@392 | 98 | Flexpage [shape=note]; |
paul@392 | 99 | Flexpage_note [shape=note,style=filled,fillcolor=gold,label="Satisfies\nmemory\naccess"]; |
paul@392 | 100 | |
paul@392 | 101 | Flexpage -> Flexpage_note [dir=none,style=dotted]; |
paul@392 | 102 | } |
paul@392 | 103 | |
paul@392 | 104 | subgraph { |
paul@392 | 105 | rank=same; |
paul@392 | 106 | |
paul@392 | 107 | ResourceD [label="Resource\n(Pager)"]; |
paul@392 | 108 | ResourceD_note [shape=note,style=filled,fillcolor=gold,label="Provides access\nto file content"]; |
paul@392 | 109 | |
paul@392 | 110 | ResourceD -> ResourceD_note [dir=none,style=dotted]; |
paul@392 | 111 | } |
paul@392 | 112 | |
paul@392 | 113 | ProgramFile [shape=record,label="Program File | ... |<d> data |<c> code | ..."]; |
paul@392 | 114 | |
paul@392 | 115 | /* Page fault handling. */ |
paul@392 | 116 | |
paul@392 | 117 | TaskMemory:d -> InternalPager [label="page fault"]; |
paul@392 | 118 | |
paul@392 | 119 | InternalPager -> Regions:d [label="find region"]; |
paul@392 | 120 | |
paul@392 | 121 | Regions:dd -> ResourceD [style=dashed]; |
paul@392 | 122 | |
paul@392 | 123 | InternalPager -> ReceiveFlexpage [dir=none]; |
paul@392 | 124 | ReceiveFlexpage -> ResourceD [label="map"]; |
paul@392 | 125 | |
paul@392 | 126 | ResourceD -> ProgramFile:d [style=dashed]; |
paul@392 | 127 | |
paul@392 | 128 | ResourceD -> Flexpage [dir=none]; |
paul@392 | 129 | Flexpage -> InternalPager; |
paul@392 | 130 | } |
paul@392 | 131 | }}} |
paul@392 | 132 | |
paul@392 | 133 | ######## |
paul@392 | 134 | |
paul@392 | 135 | == External Page Fault Handlers == |
paul@392 | 136 | |
paul@564 | 137 | Another approach is to employ an external page fault handler in the creating |
paul@392 | 138 | task. When a page fault occurs, the external handler ensures that the |
paul@392 | 139 | appropriate content has been brought into its own memory space. It then |
paul@392 | 140 | returns a flexpage from the handler routine to resolve the fault. |
paul@392 | 141 | |
paul@392 | 142 | ######## A graph showing the external paging mechanism |
paul@392 | 143 | |
paul@392 | 144 | {{{#!graphviz |
paul@392 | 145 | #format svg |
paul@392 | 146 | #transform notugly |
paul@392 | 147 | digraph external_paging { |
paul@392 | 148 | node [fontsize="12.0",fontname="sans-serif",shape=box]; |
paul@392 | 149 | edge [fontsize="12.0",fontname="sans-serif"]; |
paul@392 | 150 | rankdir=LR; |
paul@392 | 151 | |
paul@392 | 152 | TaskMemory [shape=record,label="Task Memory |<s> stack | ... |<d> data | ... |<c> code"]; |
paul@392 | 153 | |
paul@392 | 154 | subgraph { |
paul@392 | 155 | rank=same; |
paul@392 | 156 | |
paul@392 | 157 | ExternalPager_note [shape=note,style=filled,fillcolor=gold,label="Resides in\nseparate task"]; |
paul@392 | 158 | ExternalPager; |
paul@392 | 159 | |
paul@392 | 160 | ExternalPager_note -> ExternalPager [dir=none,style=dotted]; |
paul@392 | 161 | |
paul@392 | 162 | MappedFlexpage [shape=note,label="Flexpage\n(positioned)"]; |
paul@392 | 163 | MappedFlexpage_note [shape=note,style=filled,fillcolor=gold,label="Satisfies\nmemory\naccess"]; |
paul@392 | 164 | |
paul@392 | 165 | MappedFlexpage -> MappedFlexpage_note [dir=none,style=dotted]; |
paul@392 | 166 | } |
paul@392 | 167 | |
paul@392 | 168 | subgraph { |
paul@392 | 169 | rank=same; |
paul@392 | 170 | |
paul@392 | 171 | Regions [shape=record,label="Regions | |
paul@392 | 172 | {<s> stack |<sd> stack-dataspace } | |
paul@392 | 173 | {<d> data | <dd> data-dataspace } | |
paul@392 | 174 | {<c> code |<cd> code-dataspace } |..."]; |
paul@392 | 175 | |
paul@392 | 176 | L4Re [shape=ellipse,label="L4Re paging\nfunctionality"]; |
paul@392 | 177 | L4Re_note [shape=note,style=filled,fillcolor=gold,label="Supports normal\naccess to file content"]; |
paul@392 | 178 | |
paul@392 | 179 | L4Re -> L4Re_note [dir=none,style=dotted]; |
paul@392 | 180 | |
paul@392 | 181 | Flexpage [shape=note]; |
paul@392 | 182 | } |
paul@392 | 183 | |
paul@392 | 184 | subgraph { |
paul@392 | 185 | rank=same; |
paul@392 | 186 | |
paul@392 | 187 | ResourceD [label="Resource\n(Pager)"]; |
paul@392 | 188 | ResourceD_note [shape=note,style=filled,fillcolor=gold,label="Provides access\nto file content"]; |
paul@392 | 189 | |
paul@392 | 190 | ResourceD -> ResourceD_note [dir=none,style=dotted]; |
paul@392 | 191 | } |
paul@392 | 192 | |
paul@392 | 193 | ProgramFile [shape=record,label="Program File | ... |<d> data |<c> code | ..."]; |
paul@392 | 194 | |
paul@392 | 195 | /* Page fault handling. */ |
paul@392 | 196 | |
paul@392 | 197 | TaskMemory:d -> ExternalPager [label="page fault"]; |
paul@392 | 198 | |
paul@392 | 199 | ExternalPager -> Regions:d [label="find region"]; |
paul@392 | 200 | |
paul@392 | 201 | Regions:dd -> ResourceD [style=dashed]; |
paul@392 | 202 | |
paul@392 | 203 | ExternalPager -> L4Re; |
paul@392 | 204 | L4Re -> ResourceD [label="map"]; |
paul@392 | 205 | |
paul@392 | 206 | ResourceD -> ProgramFile:d [style=dashed]; |
paul@392 | 207 | |
paul@392 | 208 | ResourceD -> Flexpage [dir=none]; |
paul@392 | 209 | Flexpage -> ExternalPager; |
paul@392 | 210 | |
paul@392 | 211 | ExternalPager -> MappedFlexpage [dir=none]; |
paul@392 | 212 | MappedFlexpage -> TaskMemory:d; |
paul@392 | 213 | } |
paul@392 | 214 | }}} |
paul@392 | 215 | |
paul@392 | 216 | ######## |
paul@392 | 217 | |
paul@564 | 218 | This arrangement may be used to support a program deployed in a task. Since an |
paul@564 | 219 | internal page fault handler is just another kind of program, this external |
paul@564 | 220 | pager arrangement can be constrained to only supporting an internal page fault |
paul@564 | 221 | handler deployed in a task. |
paul@564 | 222 | |
paul@392 | 223 | == Configuring Programs == |
paul@392 | 224 | |
paul@392 | 225 | To provide an internal page fault handler alongside an actual program to be |
paul@392 | 226 | run, the following arrangement is used: |
paul@392 | 227 | |
paul@392 | 228 | ######## A graph showing the configuration arrangement |
paul@392 | 229 | |
paul@392 | 230 | {{{#!graphviz |
paul@392 | 231 | #format svg |
paul@392 | 232 | #transform notugly |
paul@392 | 233 | digraph program_configuration { |
paul@392 | 234 | node [fontsize="12.0",fontname="sans-serif",shape=box]; |
paul@392 | 235 | edge [fontsize="12.0",fontname="sans-serif"]; |
paul@392 | 236 | rankdir=LR; |
paul@392 | 237 | |
paul@392 | 238 | subgraph { |
paul@392 | 239 | rank=min; |
paul@392 | 240 | |
paul@392 | 241 | CreatingTask_note [shape=note,style=filled,fillcolor=gold,label="Responsible for\ncreating the\nnew task"]; |
paul@392 | 242 | CreatingTask [label="Creating task"]; |
paul@392 | 243 | |
paul@392 | 244 | CreatingTask_note -> CreatingTask [dir=none,style=dotted]; |
paul@392 | 245 | } |
paul@392 | 246 | |
paul@392 | 247 | subgraph { |
paul@392 | 248 | rank=max; |
paul@392 | 249 | |
paul@392 | 250 | IPCGate_note [shape=note,style=filled,fillcolor=gold,label="Created for sharing\nbetween the tasks"]; |
paul@392 | 251 | IPCGate [shape=octagon,label="IPC gate"]; |
paul@392 | 252 | |
paul@392 | 253 | IPCGate_note -> IPCGate [dir=none,style=dotted]; |
paul@392 | 254 | } |
paul@392 | 255 | |
paul@568 | 256 | InitCaps [shape=record,label="<head> Initial capabilities | {<s> ENV_INTERNAL_PAGER_NAME |<c> capability }"]; |
paul@392 | 257 | |
paul@392 | 258 | subgraph { |
paul@392 | 259 | rank=same; |
paul@392 | 260 | |
paul@392 | 261 | InternalPager; |
paul@392 | 262 | InternalPager_note [shape=note,style=filled,fillcolor=gold,label="Starts and binds\nto IPC gate"]; |
paul@392 | 263 | |
paul@392 | 264 | InternalPager -> InternalPager_note [dir=none,style=dotted]; |
paul@392 | 265 | |
paul@392 | 266 | Program; |
paul@392 | 267 | Program_note [shape=note,style=filled,fillcolor=gold,label="Starts and references\npager via IPC gate"]; |
paul@392 | 268 | |
paul@392 | 269 | Program -> Program_note [dir=none,style=dotted]; |
paul@392 | 270 | } |
paul@392 | 271 | |
paul@392 | 272 | /* Create and transfer IPC gate for binding. */ |
paul@392 | 273 | |
paul@392 | 274 | CreatingTask -> IPCGate [label="create"]; |
paul@392 | 275 | CreatingTask -> InitCaps:head [label="define"]; |
paul@392 | 276 | InitCaps:c -> IPCGate; |
paul@392 | 277 | |
paul@392 | 278 | /* Thread initiation. */ |
paul@392 | 279 | |
paul@392 | 280 | CreatingTask -> InternalPager [label="start"]; |
paul@392 | 281 | InternalPager -> InitCaps:s [label="bind"]; |
paul@392 | 282 | |
paul@392 | 283 | CreatingTask -> Program [label="start"]; |
paul@392 | 284 | Program -> IPCGate -> InternalPager [style=dashed]; |
paul@392 | 285 | } |
paul@392 | 286 | }}} |
paul@392 | 287 | |
paul@392 | 288 | ######## |
paul@568 | 289 | |
paul@568 | 290 | The creating task performs the following operations: |
paul@568 | 291 | |
paul@568 | 292 | 1. Create an IPC gate for communication between the program and its pager. |
paul@568 | 293 | |
paul@568 | 294 | 1. Map the IPC gate into the created task to be accessible via a given |
paul@568 | 295 | capability slot. |
paul@568 | 296 | |
paul@568 | 297 | 1. Define the IPC gate in the pager's initial capabilities using a |
paul@568 | 298 | well-defined name, allowing the pager to look up the capability in its |
paul@568 | 299 | environment. |
paul@568 | 300 | |
paul@568 | 301 | 1. Start the pager which itself obtains the IPC gate capability and binds to |
paul@568 | 302 | it, making itself available as the pager for the program. |
paul@568 | 303 | |
paul@568 | 304 | 1. Set the pager of the program to the IPC gate as mapped within the task. |
paul@568 | 305 | |
paul@568 | 306 | 1. Start the program which is already configured to send page faults via the |
paul@568 | 307 | IPC gate to the pager. |
paul@568 | 308 | |
paul@568 | 309 | Upon starting, the program will encounter a page fault immediately, unless |
paul@568 | 310 | some additional work was done to map memory pages into the task in advance. |
paul@568 | 311 | The internal pager or page fault handler will attempt to resolve these faults |
paul@568 | 312 | as they occur, being contacted by the kernel on the program's behalf. |