1 A Systems Programming Language Target for Micropython
2 =====================================================
3
4 Python-compatible syntax for processing using the compiler module.
5
6 The principal focus is on specific machine code generation and not
7 analysis. Thus, only block generation, address reference generation,
8 temporary storage administration and other code generation tasks are to be
9 left to the systems programming language compiler.
10
11 Special Functions
12 -----------------
13
14 In syspython, the function invocation notation is reserved to specify
15 primitive operations such as attribute access and actual function invocations,
16 with the latter being expressed as follows:
17
18 fn(y) # original Python
19 apply(fn, y) # syspython
20
21 Thus, in syspython, whenever the invocation notation is used, the target of
22 the invocation is always a special function and not a general Python function
23 or method. Note that the apply function resembles the Python function of the
24 same name but is not actually that particular function.
25
26 apply(fn, ...) # general invocation
27
28 A family of special functions for invocations exists, addressing optimisation
29 situations identified by program analysis:
30
31 applyclass(cls, ...) # direct invocation of an instantiator
32 applyfunction(fn, ...) # function-specific invocation
33 applystaticmethod(fn, obj, ...) # specific invocation of a method via a class
34 applymethod(fn, obj, ...) # specific invocation of a method via self
35
36 Where dynamic functions are to be invoked, the context providing the defaults
37 needs to be supplied to the function or method, but this can be done using the
38 above special functions as follows:
39
40 applyclass(cls, __context__, ...)
41 applyfunction(fn, __context__, ...)
42 applystaticmethod(fn, __context__, obj, ...)
43 applymethod(fn, __context__, obj, ...)
44
45 Where optimisation possibilities cannot be identified in advance, the apply
46 function must deal with the following aspects of invocation:
47
48 * Whether a context argument is required
49 * Whether a dynamic function is being invoked, thus requiring a context for
50 access to defaults
51 * Whether an appropriate number of arguments have been provided
52
53 Low-Level Code
54 --------------
55
56 Most Python-level program code should be wrapped in special function
57 invocations, and as a result other syntax features might be used to express
58 low-level concepts. Low-level operations may also be expressed using other
59 special functions. For example:
60
61 storelocal(element, loadobjtable(loadattr(obj, classcode), attrcode))
62
63 Here, element holds the raw data provided by the table access involving a base
64 defined by the classcode of an object and an offset defined by the supplied
65 attrcode.
66
67 Note that all low-level functions deal only with addresses and offsets, not
68 symbols. In the above example, loadattr combines the address of obj with the
69 symbol classcode whose actual value must be substituted by the compiler.
70 However, the loadobjtable function requires a genuine offset value for the
71 classcode (which is why loadattr is being used to obtain it), and a genuine
72 offset for the attrcode (which is provided directly).
73
74 Program Data and Data Structure Definition
75 ------------------------------------------
76
77 Given that micropython has already deduced object and parameter details,
78 such information must be communicated in the systems programming language
79 so that the compiler does not have to deduce it again.
80
81 Explicit constant declaration shall be done at the start of the main
82 module:
83
84 constants(...)
85
86 Each module may feature keyword arguments, and a list of such names is
87 provided as follows:
88
89 keywords(...)
90
91 Explicit structure declaration is still performed using class statements,
92 but base classes are omitted and attributes are declared explicitly as
93 follows:
94
95 class C:
96 instattrs(member...)
97 classattrs(member...)
98
99 Other object table information, such as inherited class attributes and
100 class compatibility (to support isinstance) are also declared explicitly:
101
102 inherited(superclass, member...)
103 descendants(class...)
104
105 Other than function definitions, no other code statements shall appear in
106 class definitions; such statements will appear after classes have been
107 defined.
108
109 For classes in the module namespace or within other classes, the __main__
110 function collects together all "loose" (module-level) statements; class
111 attribute assignments will occur in the __main__ function, and where a name
112 is associated with a function definition and another object, the function will
113 also be explicitly assigned in the __main__ function using its full name.
114
115 For classes in function namespaces, the containing function could contain the
116 "loose" statements at the point at which the class appears. However, such
117 classes are not currently supported in micropython.
118
119 Any class or function defined once in a namespace need not be assigned to that
120 namespace in the __main__ function, but where multiple definitions exist and
121 program logic determines which definition prevails, such definitions must be
122 assigned in the __main__ function.
123
124 For example:
125
126 class C:
127 def method(self, ...):
128 ...
129 if something:
130 method = something
131
132 This is represented as follows:
133
134 class C:
135 ...
136 def method(self, ...):
137 ...
138
139 def __main__():
140 globalnames(...)
141 ...
142 if something:
143 storeattr(module.C, method, something)
144
145 Local class or function definitions are also handled in a similar fashion to
146 those at the module level, although there is no explicit __main__ function
147 within each function.
148
149 For example:
150
151 def outer(x):
152 if something:
153 def inner(...):
154 ...
155 else:
156 def inner(...):
157 ...
158 return inner
159
160 This is represented as follows:
161
162 def outer(x):
163 def inner(...):
164 ...
165 def inner(...):
166 ...
167
168 if something:
169 storelocal(inner, static(module.outer.inner))
170 else:
171 storelocal(inner, static("module.outer.inner#2"))
172 return inner
173
174 Where functions are dynamic - that is they have additional state associated
175 with them, such as defaults (or potentially closures if supported) that are
176 not static (such as constant values) - suitable objects must be created using
177 references to such functions.
178
179 For example:
180
181 def outer(x):
182 def inner(y, z=x):
183 ...
184 return inner
185
186 This is represented as follows:
187
188 def outer(x):
189 def inner(__context__, y, z=x):
190 localnames(__context__, y, z)
191 ...
192 storelocal(inner, makedynamic(static(module.outer.inner), x))
193 return inner
194
195 The special makedynamic invocation creates an object referring to the function
196 and incorporating any specified defaults as attributes of that object. The
197 function itself uses a special __context__ parameter that acts somewhat like
198 the self parameter in methods: when invoked, the __context__ provides access
199 to any default information that needs to be transferred to the local
200 namespace.
201
202 Function defaults that do not require a dynamic function are initialised using
203 the setdefaults special function as follows:
204
205 class C:
206 const = ...
207 def f(a, b=const):
208 ...
209
210 ...
211 setdefaults(module.f, module.C.const)
212
213 Where multiple definitions of a name are involved, setdefaults may also be
214 used in a store operation and can be assumed to return a reference to the
215 function whose defaults are being set.
216
217 Imports
218 -------
219
220 Imports act as invocations of module code and name assignments within a
221 particular scope and are defined as follows:
222
223 # import package
224 package.__main__()
225 storelocal(package, static(package))
226
227 # import package.module
228 package.__main__()
229 package.module.__main__()
230 storelocal(package, static(package))
231
232 # from package.module import cls
233 package.__main__()
234 package.module.__main__()
235 storelocal(cls, loadattribute(package.module, cls)) # see below
236
237 Since import statements can appear in code that may be executed more than
238 once, __main__ functions should test and set a flag indicating whether the
239 function has already been called.
240
241 Python would arguably be more sensible as a language if imports were
242 processed separately, but this would then rule out logic controlling the
243 use of modules.
244
245 Name and Attribute Declarations
246 -------------------------------
247
248 Assignments and name usage involve locals and globals but usage is declared
249 explicitly:
250
251 localnames(...)
252
253 At the function level, locals are genuine local name definitions whereas
254 globals refer to module globals:
255
256 globalnames(...)
257
258 At the module level, locals are effectively equivalent to module globals and
259 are declared as such.
260
261 Each module's __main__ function will declare any referenced module globals as
262 globals. Note that the __main__ function is not a genuine attribute of any
263 module but an internal construct used to initialise modules appropriately.
264
265 Such declarations must appear first in a program unit (module, function).
266 For example:
267
268 def f(a, b):
269 localnames(a, b, x, y)
270 globalnames(f, g)
271
272 storelocal(x, 1)
273 storelocal(y, x)
274 storelocal(a, b)
275 storeattr(module, g, f)
276
277 Assignments
278 -----------
279
280 Since assignments can rebind names used in the value expression, the evaluated
281 expression must be captured and referenced when setting the targets. This is
282 done using the special $expr variable, and so the swap assignment...
283
284 a, b = b, a
285
286 ...would be written (more or less) as...
287
288 $expr = (b, a)
289 storelocal(a, apply(operator.getitem, $expr, 0))
290 storelocal(b, apply(operator.getitem, $expr, 1))
291
292 Names and Attributes
293 --------------------
294
295 Bare names refer to locals or globals according to the localnames and
296 globalnames declarations, or to constants such as None, True, False and
297 NotImplemented. Storage of local or global names is done using explicit
298 functions as follows:
299
300 storelocal(name, value)
301 storeattr(module, name, value) # see below
302
303 No operator usage: all operators are converted to invocations, including
304 all attribute access except static references to modules or particular class
305 or function definitions using the following notation:
306
307 static(package)
308 static(package.module)
309 static(package.module.cls)
310 static(package.module.cls.function)
311
312 A shorthand dot notation could be employed:
313
314 package.module
315 package.module.cls
316 package.module.cls.function
317
318 Where multiple definitions of static objects occur, the dot notation cannot be
319 used, and the full name of such definitions must be quoted. For example:
320
321 static("package.module.cls#1.function")
322
323 In general, attribute access must use an explicit function indicating the
324 kind of access operation being performed. For example:
325
326 # Instance-related operations:
327
328 loadattr(obj, attrname) # preserve retrieved context
329
330 # Constant attribute operations:
331
332 static(value) # see above
333 loadconstant(value, obj) # replace context with obj
334
335 # Static attribute operations:
336
337 loadaddress(parent, attrname) # preserve retrieved context
338 loadaddresscontext(parent, attrname, obj) # replace context with obj
339 loadaddresscontextcond(parent, attrname, obj) # run-time context decision
340
341 # Unoptimised operations:
342
343 loadattrindex(obj, attrname) # preserve retrieved context
344 loadattrindexcontextcond(obj, attrname) # run-time context decision
345
346 # Instance-related operations:
347
348 storeattr(obj, attrname, value) # preserve context for value
349
350 # Static attribute operations:
351
352 storeaddress(parent, attrname, value) # preserve context for value
353 storeaddresscontext(parent, attrname, value, obj) # replace context with obj
354
355 # Unoptimised operations:
356
357 storeattrindex(obj, attrname, value) # preserve context for value
358
359 Recall that for loadattrindex family functions, the location of the attribute
360 is obtained from the object table and the nature of the attribute is
361 determined from the stored context value.
362
363 Temporary variables could employ similar functions:
364
365 loadtemp(0)
366 storetemp(0, value)
367
368 Operators and Invocations
369 -------------------------
370
371 Conventional operators use the operator functions.
372
373 Special operators could also use the operator functions (where available)
374 but might as well be supported directly:
375
376 __is__(a, b)
377 __is_not__(a, b)
378
379 Logical operators involving short-circuit evaluation could be represented
380 as function calls, but the evaluation semantics would be preserved:
381
382 __and__(...) # returns the first non-true value or the final value
383 __not__(obj) # returns the inverse of the boolean interpretation of obj
384 __or__(...) # returns the first true value or the final value
385
386 Comparisons could be rephrased in a verbose fashion:
387
388 a < b < c becomes lt(a, b) and lt(b, c)
389 or __and__(lt(a, b), lt(b, c))
390
391 Advanced Control-Flow
392 ---------------------
393
394 Any statements requiring control-flow definition in terms of blocks must
395 be handled in the language as the notions of labels and blocks are not
396 introduced earlier apart from the special case of jumping to another
397 callable (described below).
398
399 Special functions for low-level operations:
400
401 check(obj, type)
402 jump(callable)
403
404 Function/subroutine definition with entry points for checked and unchecked
405 parameters.
406
407 def fn_checked(self, ...):
408 check(self, Type) # raises a TypeError if not isinstance(self, Type)
409 jump(fn_unchecked) # preserves the frame and return address
410
411 def fn_unchecked(self, ...):
412 ...
413
414 The jump function might also be used for inlining appropriate functions.
415
416 Exceptions must also be handled in the language.
417
418 Object Type Detection
419 ---------------------
420
421 Occasionally, the type of an object (instance of a particular class, class,
422 and so on) needs to be determined at run-time:
423
424 isclass(obj)