1 = Program Loading = 2 3 The provision of [[Paging|paging]] of file content leads to the possibility of 4 demand paging for programs, enabling them to be loaded into memory dynamically 5 and to have only the active portions of those programs resident. To achieve 6 this, programs must be appropriately initialised in new tasks, with a page 7 fault handler configured to provide program file content whenever a region of 8 the program payload is encountered that is not currently resident in memory. 9 10 <<TableOfContents(2,3)>> 11 12 == Program Initialisation == 13 14 To load and initialise a program, a new task must be created, defining a 15 separate address space for the program and allowing it to operate 16 independently of other programs. For a program to actually run, a thread of 17 execution must be created so that the program's instructions can be read and 18 processed. Once running, the program must be able to interact with the system 19 environment and to have its memory resources made available to it upon demand. 20 21 A number of elements must be prepared to be able to run a program, these being 22 summarised below. 23 24 === Program Executable File === 25 26 The program executable file is obtained and the program metadata loaded, 27 identifying the memory regions and entry point (the location of the first 28 instruction). 29 30 === Task and Threads === 31 32 A new task is created to hold the program. For this new task, new threads are 33 created to run and support the program. One thread will run the actual 34 program, and another will be used as the region mapper to handle page fault 35 conditions. 36 37 === Thread Resources === 38 39 Memory is allocated for the new program's threads, particularly the stack for 40 each thread. 41 42 Each stack is populated with program argument and environment information. Of 43 particular importance are the capabilities to be made available to the threads 44 since these permit interaction with the rest of the system. 45 46 The capabilities to be used by the threads must be mapped into the new task. 47 48 === Thread Configuration === 49 50 To connect the main thread of the program to its region mapper, an IPC gate 51 must be created. This is mapped to the new task where the region mapper will 52 bind to it, using it to expose its interface. 53 54 Thread properties such as pagers and exception handlers are defined for the 55 threads, with the main thread employing the newly created IPC gate capability 56 to delegate paging to the region mapper. 57 58 == Internal Page Fault Handlers == 59 60 When satisfying page faults for a task, one approach involves situating the 61 page fault handler within the task itself, this managing the available memory 62 regions and employing receive windows when requesting memory pages. 63 64 The general arrangement involving such internal page fault handlers for a 65 program in a task is as follows: 66 67 ######## A graph showing the internal paging mechanism 68 69 {{{#!graphviz 70 #format svg 71 #transform notugly 72 digraph internal_paging { 73 node [fontsize="12.0",fontname="sans-serif",shape=box]; 74 edge [fontsize="12.0",fontname="sans-serif"]; 75 rankdir=LR; 76 77 TaskMemory [shape=record,label="Task Memory |<s> stack | ... |<d> data | ... |<c> code"]; 78 79 subgraph { 80 rank=same; 81 82 InternalPager; 83 InternalPager_note [shape=note,style=filled,fillcolor=gold,label="Resides in\nsame task and\ndefines scope of\nfault resolution"]; 84 85 InternalPager -> InternalPager_note [dir=none,style=dotted]; 86 } 87 88 subgraph { 89 rank=same; 90 91 Regions [shape=record,label="Regions | 92 {<s> stack |<sd> stack-dataspace } | 93 {<d> data | <dd> data-dataspace } | 94 {<c> code |<cd> code-dataspace } |..."]; 95 96 ReceiveFlexpage [shape=note,label="Flexpage\n(receive window)"]; 97 98 Flexpage [shape=note]; 99 Flexpage_note [shape=note,style=filled,fillcolor=gold,label="Satisfies\nmemory\naccess"]; 100 101 Flexpage -> Flexpage_note [dir=none,style=dotted]; 102 } 103 104 subgraph { 105 rank=same; 106 107 ResourceD [label="Resource\n(Pager)"]; 108 ResourceD_note [shape=note,style=filled,fillcolor=gold,label="Provides access\nto file content"]; 109 110 ResourceD -> ResourceD_note [dir=none,style=dotted]; 111 } 112 113 ProgramFile [shape=record,label="Program File | ... |<d> data |<c> code | ..."]; 114 115 /* Page fault handling. */ 116 117 TaskMemory:d -> InternalPager [label="page fault"]; 118 119 InternalPager -> Regions:d [label="find region"]; 120 121 Regions:dd -> ResourceD [style=dashed]; 122 123 InternalPager -> ReceiveFlexpage [dir=none]; 124 ReceiveFlexpage -> ResourceD [label="map"]; 125 126 ResourceD -> ProgramFile:d [style=dashed]; 127 128 ResourceD -> Flexpage [dir=none]; 129 Flexpage -> InternalPager; 130 } 131 }}} 132 133 ######## 134 135 == External Page Fault Handlers == 136 137 Another approach is to employ an external page fault handler in the creating 138 task. When a page fault occurs, the external handler ensures that the 139 appropriate content has been brought into its own memory space. It then 140 returns a flexpage from the handler routine to resolve the fault. 141 142 ######## A graph showing the external paging mechanism 143 144 {{{#!graphviz 145 #format svg 146 #transform notugly 147 digraph external_paging { 148 node [fontsize="12.0",fontname="sans-serif",shape=box]; 149 edge [fontsize="12.0",fontname="sans-serif"]; 150 rankdir=LR; 151 152 TaskMemory [shape=record,label="Task Memory |<s> stack | ... |<d> data | ... |<c> code"]; 153 154 subgraph { 155 rank=same; 156 157 ExternalPager_note [shape=note,style=filled,fillcolor=gold,label="Resides in\nseparate task"]; 158 ExternalPager; 159 160 ExternalPager_note -> ExternalPager [dir=none,style=dotted]; 161 162 MappedFlexpage [shape=note,label="Flexpage\n(positioned)"]; 163 MappedFlexpage_note [shape=note,style=filled,fillcolor=gold,label="Satisfies\nmemory\naccess"]; 164 165 MappedFlexpage -> MappedFlexpage_note [dir=none,style=dotted]; 166 } 167 168 subgraph { 169 rank=same; 170 171 Regions [shape=record,label="Regions | 172 {<s> stack |<sd> stack-dataspace } | 173 {<d> data | <dd> data-dataspace } | 174 {<c> code |<cd> code-dataspace } |..."]; 175 176 L4Re [shape=ellipse,label="L4Re paging\nfunctionality"]; 177 L4Re_note [shape=note,style=filled,fillcolor=gold,label="Supports normal\naccess to file content"]; 178 179 L4Re -> L4Re_note [dir=none,style=dotted]; 180 181 Flexpage [shape=note]; 182 } 183 184 subgraph { 185 rank=same; 186 187 ResourceD [label="Resource\n(Pager)"]; 188 ResourceD_note [shape=note,style=filled,fillcolor=gold,label="Provides access\nto file content"]; 189 190 ResourceD -> ResourceD_note [dir=none,style=dotted]; 191 } 192 193 ProgramFile [shape=record,label="Program File | ... |<d> data |<c> code | ..."]; 194 195 /* Page fault handling. */ 196 197 TaskMemory:d -> ExternalPager [label="page fault"]; 198 199 ExternalPager -> Regions:d [label="find region"]; 200 201 Regions:dd -> ResourceD [style=dashed]; 202 203 ExternalPager -> L4Re; 204 L4Re -> ResourceD [label="map"]; 205 206 ResourceD -> ProgramFile:d [style=dashed]; 207 208 ResourceD -> Flexpage [dir=none]; 209 Flexpage -> ExternalPager; 210 211 ExternalPager -> MappedFlexpage [dir=none]; 212 MappedFlexpage -> TaskMemory:d; 213 } 214 }}} 215 216 ######## 217 218 This arrangement may be used to support a program deployed in a task. Since an 219 internal page fault handler is just another kind of program, this external 220 pager arrangement can be constrained to only supporting an internal page fault 221 handler deployed in a task. 222 223 == Configuring Programs == 224 225 To provide an internal page fault handler alongside an actual program to be 226 run, the following arrangement is used: 227 228 ######## A graph showing the configuration arrangement 229 230 {{{#!graphviz 231 #format svg 232 #transform notugly 233 digraph program_configuration { 234 node [fontsize="12.0",fontname="sans-serif",shape=box]; 235 edge [fontsize="12.0",fontname="sans-serif"]; 236 rankdir=LR; 237 238 subgraph { 239 rank=min; 240 241 CreatingTask_note [shape=note,style=filled,fillcolor=gold,label="Responsible for\ncreating the\nnew task"]; 242 CreatingTask [label="Creating task"]; 243 244 CreatingTask_note -> CreatingTask [dir=none,style=dotted]; 245 } 246 247 subgraph { 248 rank=max; 249 250 IPCGate_note [shape=note,style=filled,fillcolor=gold,label="Created for sharing\nbetween the tasks"]; 251 IPCGate [shape=octagon,label="IPC gate"]; 252 253 IPCGate_note -> IPCGate [dir=none,style=dotted]; 254 } 255 256 InitCaps [shape=record,label="<head> Initial capabilities | {<s> ENV_INTERNAL_PAGER_NAME |<c> capability }"]; 257 258 subgraph { 259 rank=same; 260 261 InternalPager; 262 InternalPager_note [shape=note,style=filled,fillcolor=gold,label="Starts and binds\nto IPC gate"]; 263 264 InternalPager -> InternalPager_note [dir=none,style=dotted]; 265 266 Program; 267 Program_note [shape=note,style=filled,fillcolor=gold,label="Starts and references\npager via IPC gate"]; 268 269 Program -> Program_note [dir=none,style=dotted]; 270 } 271 272 /* Create and transfer IPC gate for binding. */ 273 274 CreatingTask -> IPCGate [label="create"]; 275 CreatingTask -> InitCaps:head [label="define"]; 276 InitCaps:c -> IPCGate; 277 278 /* Thread initiation. */ 279 280 CreatingTask -> InternalPager [label="start"]; 281 InternalPager -> InitCaps:s [label="bind"]; 282 283 CreatingTask -> Program [label="start"]; 284 Program -> IPCGate -> InternalPager [style=dashed]; 285 } 286 }}} 287 288 ######## 289 290 The creating task performs the following operations: 291 292 1. Create an IPC gate for communication between the program and its pager. 293 294 1. Map the IPC gate into the created task to be accessible via a given 295 capability slot. 296 297 1. Define the IPC gate in the pager's initial capabilities using a 298 well-defined name, allowing the pager to look up the capability in its 299 environment. 300 301 1. Start the pager which itself obtains the IPC gate capability and binds to 302 it, making itself available as the pager for the program. 303 304 1. Set the pager of the program to the IPC gate as mapped within the task. 305 306 1. Start the program which is already configured to send page faults via the 307 IPC gate to the pager. 308 309 Upon starting, the program will encounter a page fault immediately, unless 310 some additional work was done to map memory pages into the task in advance. 311 The internal pager or page fault handler will attempt to resolve these faults 312 as they occur, being contacted by the kernel on the program's behalf.