1 A Systems Programming Language Target for Micropython
2 =====================================================
3
4 Python-compatible syntax for processing using the compiler module.
5
6 The principal focus is on specific machine code generation and not
7 analysis. Thus, only block generation, address reference generation,
8 temporary storage administration and other code generation tasks are to be
9 left to the systems programming language compiler.
10
11 Special Functions
12 -----------------
13
14 In syspython, the function invocation notation is reserved to specify
15 primitive operations such as attribute access and actual function invocations,
16 with the latter being expressed as follows:
17
18 fn(y) # original Python
19 apply(fn, y) # syspython
20
21 Thus, in syspython, whenever the invocation notation is used, the target of
22 the invocation is always a special function and not a general Python function
23 or method. Note that the apply function resembles the Python function of the
24 same name but is not actually that particular function.
25
26 apply(fn, ...) # general invocation
27
28 A family of special functions for invocations exists, addressing optimisation
29 situations identified by program analysis:
30
31 applyclass(cls, ...) # direct invocation of an instantiator
32 applyfunction(fn, ...) # function-specific invocation
33 applystaticmethod(fn, obj, ...) # specific invocation of a method via a class
34 applymethod(fn, obj, ...) # specific invocation of a method via self
35
36 Where dynamic functions are to be invoked, the context providing the defaults
37 needs to be supplied to the function or method, but this can be done using the
38 above special functions as follows:
39
40 applyclass(cls, __context__, ...)
41 applyfunction(fn, __context__, ...)
42 applystaticmethod(fn, __context__, obj, ...)
43 applymethod(fn, __context__, obj, ...)
44
45 Where optimisation possibilities cannot be identified in advance, the apply
46 function must deal with the following aspects of invocation:
47
48 * Whether a context argument is required
49 * Whether a dynamic function is being invoked, thus requiring a context for
50 access to defaults
51 * Whether an appropriate number of arguments have been provided
52
53 Low-Level Code
54 --------------
55
56 Most Python-level program code should be wrapped in special function
57 invocations, and as a result other syntax features might be used to express
58 low-level concepts. Low-level operations may also be expressed using other
59 special functions. For example:
60
61 storelocal(element, loadobjtable(loadattr(obj, classcode), attrcode))
62
63 Here, element holds the raw data provided by the table access involving a base
64 defined by the classcode of an object and an offset defined by the supplied
65 attrcode.
66
67 Note that all low-level functions deal only with addresses and offsets, not
68 symbols. In the above example, loadattr combines the address of obj with the
69 symbol classcode whose actual value must be substituted by the compiler.
70 However, the loadobjtable function requires a genuine offset value for the
71 classcode (which is why loadattr is being used to obtain it), and a genuine
72 offset for the attrcode (which is provided directly).
73
74 Program Data and Data Structure Definition
75 ------------------------------------------
76
77 Given that micropython has already deduced object and parameter details,
78 such information must be communicated in the systems programming language
79 so that the compiler does not have to deduce it again.
80
81 Explicit constant declaration shall be done at the start of the main
82 module:
83
84 constants(...)
85
86 Each module may feature keyword arguments, and a list of such names is
87 provided as follows:
88
89 keywords(...)
90
91 Explicit structure declaration is still performed using class statements,
92 but base classes are omitted and attributes are declared explicitly as
93 follows:
94
95 class C:
96 instattrs(member...)
97 classattrs(member...)
98
99 Other object table information, such as inherited class attributes and
100 class compatibility (to support isinstance) are also declared explicitly:
101
102 inherited(superclass, member...)
103 descendants(class...)
104
105 Other than function definitions, no other code statements shall appear in
106 class definitions; such statements will appear after classes have been
107 defined.
108
109 For classes in the module namespace or within other classes, the __main__
110 function collects together all "loose" (module-level) statements; class
111 attribute assignments will occur in the __main__ function, and where a name
112 is associated with a function definition and another object, the function will
113 also be explicitly assigned in the __main__ function using its full name.
114
115 For classes in function namespaces, the containing function could contain the
116 "loose" statements at the point at which the class appears. However, such
117 classes are not currently supported in micropython.
118
119 Any class or function defined once in a namespace need not be assigned to that
120 namespace in the __main__ function, but where multiple definitions exist and
121 program logic determines which definition prevails, such definitions must be
122 assigned in the __main__ function.
123
124 For example:
125
126 class C:
127 def method(self, ...):
128 ...
129 if something:
130 method = something
131
132 This is represented as follows:
133
134 class C:
135 ...
136 def method(self, ...):
137 ...
138
139 def __main__():
140 globalnames(...)
141 ...
142 if something:
143 storeattr(module.C, method, something)
144
145 Local class or function definitions are also handled in a similar fashion to
146 those at the module level, although there is no explicit __main__ function
147 within each function.
148
149 For example:
150
151 def outer(x):
152 if something:
153 def inner(...):
154 ...
155 else:
156 def inner(...):
157 ...
158 return inner
159
160 This is represented as follows:
161
162 def outer(x):
163 def inner(...):
164 ...
165 def inner(...):
166 ...
167
168 if something:
169 storelocal(inner, static(outer.inner))
170 else:
171 storelocal(inner, static("outer.inner#2"))
172 return inner
173
174 Where functions are dynamic - that is they have additional state associated
175 with them, such as defaults (or potentially closures if supported) that are
176 not static (such as constant values) - suitable objects must be created using
177 references to such functions.
178
179 For example:
180
181 def outer(x):
182 def inner(y, z=x):
183 ...
184 return inner
185
186 This is represented as follows:
187
188 def outer(x):
189 def inner(__context__, y, z=x):
190 localnames(__context__, y, z)
191 ...
192 storelocal(inner, makedynamic(static(outer.inner), x))
193 return inner
194
195 The special makedynamic invocation creates an object referring to the function
196 and incorporating any specified defaults as attributes of that object. The
197 function itself uses a special __context__ parameter that acts somewhat like
198 the self parameter in methods: when invoked, the __context__ provides access
199 to any default information that needs to be transferred to the local
200 namespace.
201
202 Imports
203 -------
204
205 Imports act as invocations of module code and name assignments within a
206 particular scope and are defined as follows:
207
208 # import package
209 package.__main__()
210 storelocal(package, static(package))
211
212 # import package.module
213 package.__main__()
214 package.module.__main__()
215 storelocal(package, static(package))
216
217 # from package.module import cls
218 package.__main__()
219 package.module.__main__()
220 storelocal(cls, loadattribute(package.module, cls)) # see below
221
222 Since import statements can appear in code that may be executed more than
223 once, __main__ functions should test and set a flag indicating whether the
224 function has already been called.
225
226 Python would arguably be more sensible as a language if imports were
227 processed separately, but this would then rule out logic controlling the
228 use of modules.
229
230 Name and Attribute Declarations
231 -------------------------------
232
233 Assignments and name usage involve locals and globals but usage is declared
234 explicitly:
235
236 localnames(...)
237
238 At the function level, locals are genuine local name definitions whereas
239 globals refer to module globals:
240
241 globalnames(...)
242
243 At the module level, locals are effectively equivalent to module globals and
244 are declared as such.
245
246 Each module's __main__ function will declare any referenced module globals as
247 globals. Note that the __main__ function is not a genuine attribute of any
248 module but an internal construct used to initialise modules appropriately.
249
250 Such declarations must appear first in a program unit (module, function).
251 For example:
252
253 def f(a, b):
254 localnames(a, b, x, y)
255 globalnames(f, g)
256
257 storelocal(x, 1)
258 storelocal(y, x)
259 storelocal(a, b)
260 storeattr(module, g, f)
261
262 Assignments
263 -----------
264
265 Since assignments can rebind names used in the value expression, the evaluated
266 expression must be captured and referenced when setting the targets. This is
267 done using the special $expr variable, and so the swap assignment...
268
269 a, b = b, a
270
271 ...would be written (more or less) as...
272
273 $expr = (b, a)
274 storelocal(a, apply(operator.getitem, $expr, 0))
275 storelocal(b, apply(operator.getitem, $expr, 1))
276
277 Names and Attributes
278 --------------------
279
280 Bare names refer to locals or globals according to the localnames and
281 globalnames declarations, or to constants such as None, True, False and
282 NotImplemented. Storage of local or global names is done using explicit
283 functions as follows:
284
285 storelocal(name, value)
286 storeattr(module, name, value) # see below
287
288 No operator usage: all operators are converted to invocations, including
289 all attribute access except static references to modules or particular class
290 or function definitions using the following notation:
291
292 static(package)
293 static(package.module)
294 static(package.module.cls)
295 static(package.module.cls.function)
296
297 A shorthand dot notation could be employed:
298
299 package.module
300 package.module.cls
301 package.module.cls.function
302
303 Where multiple definitions of static objects occur, the dot notation cannot be
304 used, and the full name of such definitions must be quoted. For example:
305
306 static("package.module.cls#1.function")
307
308 In general, attribute access must use an explicit function indicating the
309 kind of access operation being performed. For example:
310
311 # Instance-related operations:
312
313 loadattr(obj, attrname) # preserve retrieved context
314
315 # Constant attribute operations:
316
317 static(value) # see above
318 loadconstant(value, obj) # replace context with obj
319
320 # Static attribute operations:
321
322 loadaddress(parent, attrname) # preserve retrieved context
323 loadaddresscontext(parent, attrname, obj) # replace context with obj
324 loadaddresscontextcond(parent, attrname, obj) # run-time context decision
325
326 # Unoptimised operations:
327
328 loadattrindex(obj, attrname) # preserve retrieved context
329 loadattrindexcontextcond(obj, attrname) # run-time context decision
330
331 # Instance-related operations:
332
333 storeattr(obj, attrname, value) # preserve context for value
334
335 # Static attribute operations:
336
337 storeaddress(parent, attrname, value) # preserve context for value
338 storeaddresscontext(parent, attrname, value, obj) # replace context with obj
339
340 # Unoptimised operations:
341
342 storeattrindex(obj, attrname, value) # preserve context for value
343
344 Recall that for loadattrindex family functions, the location of the attribute
345 is obtained from the object table and the nature of the attribute is
346 determined from the stored context value.
347
348 Temporary variables could employ similar functions:
349
350 loadtemp(0)
351 storetemp(0, value)
352
353 Operators and Invocations
354 -------------------------
355
356 Conventional operators use the operator functions.
357
358 Special operators could also use the operator functions (where available)
359 but might as well be supported directly:
360
361 __is__(a, b)
362 __is_not__(a, b)
363
364 Logical operators involving short-circuit evaluation could be represented
365 as function calls, but the evaluation semantics would be preserved:
366
367 __and__(...) # returns the first non-true value or the final value
368 __not__(obj) # returns the inverse of the boolean interpretation of obj
369 __or__(...) # returns the first true value or the final value
370
371 Comparisons could be rephrased in a verbose fashion:
372
373 a < b < c becomes lt(a, b) and lt(b, c)
374 or __and__(lt(a, b), lt(b, c))
375
376 Advanced Control-Flow
377 ---------------------
378
379 Any statements requiring control-flow definition in terms of blocks must
380 be handled in the language as the notions of labels and blocks are not
381 introduced earlier apart from the special case of jumping to another
382 callable (described below).
383
384 Special functions for low-level operations:
385
386 check(obj, type)
387 jump(callable)
388
389 Function/subroutine definition with entry points for checked and unchecked
390 parameters.
391
392 def fn_checked(self, ...):
393 check(self, Type) # raises a TypeError if not isinstance(self, Type)
394 jump(fn_unchecked) # preserves the frame and return address
395
396 def fn_unchecked(self, ...):
397 ...
398
399 The jump function might also be used for inlining appropriate functions.
400
401 Exceptions must also be handled in the language.
402
403 Object Type Detection
404 ---------------------
405
406 Occasionally, the type of an object (instance of a particular class, class,
407 and so on) needs to be determined at run-time:
408
409 isclass(obj)