1 A Systems Programming Language Target for Micropython
2 =====================================================
3
4 Python-compatible syntax for processing using the compiler module.
5
6 The principal focus is on specific machine code generation and not
7 analysis. Thus, only block generation, address reference generation,
8 temporary storage administration and other code generation tasks are to be
9 left to the systems programming language compiler.
10
11 Special Functions
12 -----------------
13
14 In syspython, the function invocation notation is reserved to specify
15 primitive operations such as attribute access and actual function invocations,
16 with the latter being expressed as follows:
17
18 fn(y) # original Python
19 apply(fn, y) # syspython
20
21 Thus, in syspython, whenever the invocation notation is used, the target of
22 the invocation is always a special function and not a general Python function
23 or method. Note that the apply function resembles the Python function of the
24 same name but is not actually that particular function.
25
26 apply(fn, ...) # general invocation
27
28 A family of special functions for invocations exists, addressing optimisation
29 situations identified by program analysis:
30
31 applyclass(cls, ...) # direct invocation of an instantiator
32 applyfunction(fn, ...) # function-specific invocation
33 applystaticmethod(fn, obj, ...) # specific invocation of a method via a class
34 applymethod(fn, obj, ...) # specific invocation of a method via self
35
36 Where dynamic functions are to be invoked, the object providing the defaults
37 needs to be supplied to the function or method, but this should be done by the
38 invocation mechanism implemented on the target architecture, exposing only a
39 special __context__ local name providing access to the defaults.
40
41 Where optimisation possibilities cannot be identified in advance, the apply
42 function must deal with the following aspects of invocation:
43
44 * Whether a dynamic function is being invoked, thus requiring an object for
45 access to defaults
46 * Whether a context argument is required
47 * Whether an appropriate number of arguments have been provided
48
49 Low-Level Code
50 --------------
51
52 Most Python-level program code should be wrapped in special function
53 invocations, and as a result other syntax features might be used to express
54 low-level concepts. Low-level operations may also be expressed using other
55 special functions. For example:
56
57 storelocal(element, loadobjtable(loadattr(obj, classcode), attrcode))
58
59 Here, element holds the raw data provided by the table access involving a base
60 defined by the classcode of an object and an offset defined by the supplied
61 attrcode.
62
63 Note that all low-level functions deal only with addresses and offsets, not
64 symbols. In the above example, loadattr combines the address of obj with the
65 symbol classcode whose actual value must be substituted by the compiler.
66 However, the loadobjtable function requires a genuine offset value for the
67 classcode (which is why loadattr is being used to obtain it), and a genuine
68 offset for the attrcode (which is provided directly).
69
70 Program Data and Data Structure Definition
71 ------------------------------------------
72
73 Given that micropython has already deduced object and parameter details,
74 such information must be communicated in the systems programming language
75 so that the compiler does not have to deduce it again.
76
77 Explicit constant declaration shall be done at the start of the main
78 module:
79
80 constants(...)
81
82 Each module may feature keyword arguments, and a list of such names is
83 provided as follows:
84
85 keywords(...)
86
87 Explicit structure declaration is still performed using class statements,
88 but base classes are omitted and attributes are declared explicitly as
89 follows:
90
91 class C:
92 instattrs(member...)
93 classattrs(member...)
94
95 Other object table information, such as inherited class attributes and
96 class compatibility (to support isinstance) are also declared explicitly:
97
98 inherited(superclass, member...)
99 descendants(class...)
100
101 Other than function definitions, no other code statements shall appear in
102 class definitions; such statements will appear after classes have been
103 defined.
104
105 For classes in the module namespace or within other classes, the __main__
106 function collects together all "loose" (module-level) statements; class
107 attribute assignments will occur in the __main__ function, and where a name
108 is associated with a function definition and another object, the function will
109 also be explicitly assigned in the __main__ function using its full name.
110
111 For classes in function namespaces, the containing function could contain the
112 "loose" statements at the point at which the class appears. However, such
113 classes are not currently supported in micropython.
114
115 Any class or function defined once in a namespace need not be assigned to that
116 namespace in the __main__ function, but where multiple definitions exist and
117 program logic determines which definition prevails, such definitions must be
118 assigned in the __main__ function.
119
120 For example:
121
122 class C:
123 def method(self, ...):
124 ...
125 if something:
126 method = something
127
128 This is represented as follows:
129
130 class C:
131 ...
132 def method(self, ...):
133 ...
134
135 def __main__():
136 globalnames(...)
137 ...
138 if something:
139 storeattr(module.C, method, something)
140
141 Local class or function definitions are also handled in a similar fashion to
142 those at the module level, although there is no explicit __main__ function
143 within each function.
144
145 For example:
146
147 def outer(x):
148 if something:
149 def inner(...):
150 ...
151 else:
152 def inner(...):
153 ...
154 return inner
155
156 This is represented as follows:
157
158 def outer(x):
159 def inner(...):
160 ...
161 def inner(...):
162 ...
163
164 if something:
165 storelocal(inner, static(module.outer.inner))
166 else:
167 storelocal(inner, static("module.outer.inner#2"))
168 return inner
169
170 Where functions are dynamic - that is they have additional state associated
171 with them, such as defaults (or potentially closures if supported) that are
172 not static (such as constant values) - suitable objects must be created using
173 references to such functions.
174
175 For example:
176
177 def outer(x):
178 def inner(y, z=x):
179 ...
180 return inner
181
182 This is represented as follows:
183
184 def outer(x):
185 def inner(y, z=x):
186 localnames(y, z, __context__)
187 ...
188 storelocal(inner, makedynamic(static(module.outer.inner), x))
189 return inner
190
191 The special makedynamic invocation creates an object referring to the function
192 and incorporating any specified defaults as attributes of that object. The
193 function itself will use a special __context__ local that acts somewhat like
194 the self parameter in methods: in the invoked function, __context__ provides
195 access to any default information that needs to be transferred to the local
196 namespace.
197
198 Function defaults that do not require a dynamic function are initialised using
199 the setdefaults special function as follows:
200
201 class C:
202 const = ...
203 def f(a, b=const):
204 ...
205
206 ...
207 setdefaults(module.f, module.C.const)
208
209 Where multiple definitions of a name are involved, setdefaults may also be
210 used in a store operation and can be assumed to return a reference to the
211 function whose defaults are being set.
212
213 Imports
214 -------
215
216 Imports act as invocations of module code and name assignments within a
217 particular scope and are defined as follows:
218
219 # import package
220 package.__main__()
221 storelocal(package, static(package))
222
223 # import package.module
224 package.__main__()
225 package.module.__main__()
226 storelocal(package, static(package))
227
228 # from package.module import cls
229 package.__main__()
230 package.module.__main__()
231 storelocal(cls, loadattribute(package.module, cls)) # see below
232
233 Since import statements can appear in code that may be executed more than
234 once, __main__ functions should test and set a flag indicating whether the
235 function has already been called.
236
237 Python would arguably be more sensible as a language if imports were
238 processed separately, but this would then rule out logic controlling the
239 use of modules.
240
241 Name and Attribute Declarations
242 -------------------------------
243
244 Assignments and name usage involve locals and globals but usage is declared
245 explicitly:
246
247 localnames(...)
248
249 At the function level, locals are genuine local name definitions whereas
250 globals refer to module globals:
251
252 globalnames(...)
253
254 At the module level, locals are effectively equivalent to module globals and
255 are declared as such.
256
257 Each module's __main__ function will declare any referenced module globals as
258 globals. Note that the __main__ function is not a genuine attribute of any
259 module but an internal construct used to initialise modules appropriately.
260
261 Such declarations must appear first in a program unit (module, function).
262 For example:
263
264 def f(a, b):
265 localnames(a, b, x, y)
266 globalnames(f, g)
267
268 storelocal(x, 1)
269 storelocal(y, x)
270 storelocal(a, b)
271 storeattr(module, g, f)
272
273 Assignments
274 -----------
275
276 Since assignments can rebind names used in the value expression, the evaluated
277 expression must be captured and referenced when setting the targets. This is
278 done using the special $expr variable, and so the swap assignment...
279
280 a, b = b, a
281
282 ...would be written (more or less) as...
283
284 $expr = (b, a)
285 storelocal(a, apply(operator.getitem, $expr, 0))
286 storelocal(b, apply(operator.getitem, $expr, 1))
287
288 Names and Attributes
289 --------------------
290
291 Bare names refer to locals or globals according to the localnames and
292 globalnames declarations, or to constants such as None, True, False and
293 NotImplemented. Storage of local or global names is done using explicit
294 functions as follows:
295
296 storelocal(name, value)
297 storeattr(module, name, value) # see below
298
299 No operator usage: all operators are converted to invocations, including
300 all attribute access except static references to modules or particular class
301 or function definitions using the following notation:
302
303 static(package)
304 static(package.module)
305 static(package.module.cls)
306 static(package.module.cls.function)
307
308 A shorthand dot notation could be employed:
309
310 package.module
311 package.module.cls
312 package.module.cls.function
313
314 Where multiple definitions of static objects occur, the dot notation cannot be
315 used, and the full name of such definitions must be quoted. For example:
316
317 static("package.module.cls#1.function")
318
319 In general, attribute access must use an explicit function indicating the
320 kind of access operation being performed. For example:
321
322 # Instance-related operations:
323
324 loadattr(obj, attrname) # preserve retrieved context
325
326 # Constant attribute operations:
327
328 static(value) # see above
329 loadconstant(value, obj) # replace context with obj
330
331 # Static attribute operations:
332
333 loadaddress(parent, attrname) # preserve retrieved context
334 loadaddresscontext(parent, attrname, obj) # replace context with obj
335 loadaddresscontextcond(parent, attrname, obj) # run-time context decision
336
337 # Unoptimised operations:
338
339 loadattrindex(obj, attrname) # preserve retrieved context
340 loadattrindexcontextcond(obj, attrname) # run-time context decision
341
342 # Instance-related operations:
343
344 storeattr(obj, attrname, value) # preserve context for value
345
346 # Static attribute operations:
347
348 storeaddress(parent, attrname, value) # preserve context for value
349 storeaddresscontext(parent, attrname, value, obj) # replace context with obj
350
351 # Unoptimised operations:
352
353 storeattrindex(obj, attrname, value) # preserve context for value
354
355 Recall that for loadattrindex family functions, the location of the attribute
356 is obtained from the object table and the nature of the attribute is
357 determined from the stored context value.
358
359 Temporary variables could employ similar functions:
360
361 loadtemp(0)
362 storetemp(0, value)
363
364 Operators and Invocations
365 -------------------------
366
367 Conventional operators use the operator functions.
368
369 Special operators could also use the operator functions (where available)
370 but might as well be supported directly:
371
372 __is__(a, b)
373 __is_not__(a, b)
374
375 Logical operators involving short-circuit evaluation could be represented
376 as function calls, but the evaluation semantics would be preserved:
377
378 __and__(...) # returns the first non-true value or the final value
379 __not__(obj) # returns the inverse of the boolean interpretation of obj
380 __or__(...) # returns the first true value or the final value
381
382 Comparisons could be rephrased in a verbose fashion:
383
384 a < b < c becomes lt(a, b) and lt(b, c)
385 or __and__(lt(a, b), lt(b, c))
386
387 Advanced Control-Flow
388 ---------------------
389
390 Any statements requiring control-flow definition in terms of blocks must
391 be handled in the language as the notions of labels and blocks are not
392 introduced earlier apart from the special case of jumping to another
393 callable (described below).
394
395 Special functions for low-level operations:
396
397 check(obj, type)
398 jump(callable)
399
400 Function/subroutine definition with entry points for checked and unchecked
401 parameters.
402
403 def fn_checked(self, ...):
404 check(self, Type) # raises a TypeError if not isinstance(self, Type)
405 jump(fn_unchecked) # preserves the frame and return address
406
407 def fn_unchecked(self, ...):
408 ...
409
410 The jump function might also be used for inlining appropriate functions.
411
412 Exceptions must also be handled in the language.
413
414 Object Type Detection
415 ---------------------
416
417 Occasionally, the type of an object (instance of a particular class, class,
418 and so on) needs to be determined at run-time:
419
420 isclass(obj)