1 A Systems Programming Language Target for Micropython
2 =====================================================
3
4 Python-compatible syntax for processing using the compiler module.
5
6 The principal focus is on specific machine code generation and not
7 analysis. Thus, only block generation, address reference generation,
8 temporary storage administration and other code generation tasks are to be
9 left to the systems programming language compiler.
10
11 Special Functions
12 -----------------
13
14 In syspython, the function invocation notation is reserved to specify
15 primitive operations such as attribute access and actual function invocations,
16 with the latter being expressed as follows:
17
18 fn(y) # original Python
19 apply(fn, y) # syspython
20
21 Thus, in syspython, whenever the invocation notation is used, the target of
22 the invocation is always a special function and not a general Python function
23 or method. Note that the apply function resembles the Python function of the
24 same name but is not actually that particular function.
25
26 A family of special functions for invocations exists, addressing optimisation
27 situations identified by program analysis:
28
29 apply # general invocation
30 applyclass # direct invocation of an instantiator
31 applyfunction # function-specific invocation
32 applystaticmethod # specific invocation of a method via a class
33 applymethod # specific invocation of a method via self
34
35 Low-Level Code
36 --------------
37
38 Most Python-level program code should be wrapped in special function
39 invocations, and as a result other syntax features might be used to express
40 low-level concepts. Low-level operations may also be expressed using other
41 special functions. For example:
42
43 storelocal(element, loadobjtable(loadattr(obj, classcode), attrcode))
44
45 Here, element holds the raw data provided by the table access involving a base
46 defined by the classcode of an object and an offset defined by the supplied
47 attrcode.
48
49 Note that all low-level functions deal only with addresses and offsets, not
50 symbols. In the above example, loadattr combines the address of obj with the
51 symbol classcode whose actual value must be substituted by the compiler.
52 However, the loadobjtable function requires a genuine offset value for the
53 classcode (which is why loadattr is being used to obtain it), and a genuine
54 offset for the attrcode (which is provided directly).
55
56 Program Data and Data Structure Definition
57 ------------------------------------------
58
59 Given that micropython has already deduced object and parameter details,
60 such information must be communicated in the systems programming language
61 so that the compiler does not have to deduce it again.
62
63 Explicit constant declaration shall be done at the start of the main
64 module:
65
66 constants(...)
67
68 Each module may feature keyword arguments, and a list of such names is
69 provided as follows:
70
71 keywords(...)
72
73 Explicit structure declaration is still performed using class statements,
74 but base classes are omitted and attributes are declared explicitly as
75 follows:
76
77 class C:
78 instattrs(member...)
79 classattrs(member...)
80
81 Other object table information, such as inherited class attributes and
82 class compatibility (to support isinstance) are also declared explicitly:
83
84 inherited(superclass, member...)
85 descendants(class...)
86
87 Other than function definitions, no other code statements shall appear in
88 class definitions; such statements will appear after classes have been
89 defined.
90
91 For classes in the module namespace or within other classes, the __main__
92 function collects together all "loose" (module-level) statements; class
93 attribute assignments will occur in the __main__ function, and where a name
94 is associated with a function definition and another object, the function will
95 also be explicitly assigned in the __main__ function using its full name.
96
97 For classes in function namespaces, the containing function could contain the
98 "loose" statements at the point at which the class appears. However, such
99 classes are not currently supported in micropython.
100
101 Any class or function defined once in a namespace need not be assigned to that
102 namespace in the __main__ function, but where multiple definitions exist and
103 program logic determines which definition prevails, such definitions must be
104 assigned in the __main__ function.
105
106 For example:
107
108 class C:
109 def method(self, ...):
110 ...
111 if something:
112 method = something
113
114 This is represented as follows:
115
116 class C:
117 ...
118 def method(self, ...):
119 ...
120
121 def __main__():
122 globalnames(...)
123 ...
124 if something:
125 storeattr(module.C, method, something)
126
127 Local class or function definitions are also handled in a similar fashion to
128 those at the module level, although there is no explicit __main__ function
129 within each function.
130
131 For example:
132
133 def outer(x):
134 if something:
135 def inner(...):
136 ...
137 else:
138 def inner(...):
139 ...
140 return inner
141
142 This is represented as follows:
143
144 def outer(x):
145 def inner(...):
146 ...
147 def inner(...):
148 ...
149
150 if something:
151 storelocal(inner, static(outer.inner))
152 else:
153 storelocal(inner, static("outer.inner#2"))
154 return inner
155
156 Where functions are dynamic - that is they have additional state associated
157 with them, such as defaults (or potentially closures if supported) that are
158 not static (such as constant values) - suitable objects must be created using
159 references to such functions.
160
161 For example:
162
163 def outer(x):
164 def inner(y, z=x):
165 ...
166 return inner
167
168 This is represented as follows:
169
170 def outer(x):
171 def inner(__context__, y, z=x):
172 localnames(__context__, y, z)
173 ...
174 storelocal(inner, makedynamic(static(outer.inner), x))
175 return inner
176
177 The special makedynamic invocation creates an object referring to the function
178 and incorporating any specified defaults as attributes of that object. The
179 function itself uses a special __context__ parameter that acts somewhat like
180 the self parameter in methods: when invoked, the __context__ provides access
181 to any default information that needs to be transferred to the local
182 namespace.
183
184 Imports
185 -------
186
187 Imports act as invocations of module code and name assignments within a
188 particular scope and are defined as follows:
189
190 # import package
191 package.__main__()
192 storelocal(package, static(package))
193
194 # import package.module
195 package.__main__()
196 package.module.__main__()
197 storelocal(package, static(package))
198
199 # from package.module import cls
200 package.__main__()
201 package.module.__main__()
202 storelocal(cls, loadattribute(package.module, cls)) # see below
203
204 Since import statements can appear in code that may be executed more than
205 once, __main__ functions should test and set a flag indicating whether the
206 function has already been called.
207
208 Python would arguably be more sensible as a language if imports were
209 processed separately, but this would then rule out logic controlling the
210 use of modules.
211
212 Name and Attribute Declarations
213 -------------------------------
214
215 Assignments and name usage involve locals and globals but usage is declared
216 explicitly:
217
218 localnames(...)
219
220 At the function level, locals are genuine local name definitions whereas
221 globals refer to module globals:
222
223 globalnames(...)
224
225 At the module level, locals are effectively equivalent to module globals and
226 are declared as such.
227
228 Each module's __main__ function will declare any referenced module globals as
229 globals. Note that the __main__ function is not a genuine attribute of any
230 module but an internal construct used to initialise modules appropriately.
231
232 Such declarations must appear first in a program unit (module, function).
233 For example:
234
235 def f(a, b):
236 localnames(a, b, x, y)
237 globalnames(f, g)
238
239 storelocal(x, 1)
240 storelocal(y, x)
241 storelocal(a, b)
242 storeattr(module, g, f)
243
244 Assignments
245 -----------
246
247 Since assignments can rebind names used in the value expression, the evaluated
248 expression must be captured and referenced when setting the targets. This is
249 done using the special $expr variable, and so the swap assignment...
250
251 a, b = b, a
252
253 ...would be written (more or less) as...
254
255 $expr = (b, a)
256 storelocal(a, apply(operator.getitem, $expr, 0))
257 storelocal(b, apply(operator.getitem, $expr, 1))
258
259 Names and Attributes
260 --------------------
261
262 Bare names refer to locals or globals according to the localnames and
263 globalnames declarations, or to constants such as None, True, False and
264 NotImplemented. Storage of local or global names is done using explicit
265 functions as follows:
266
267 storelocal(name, value)
268 storeattr(module, name, value) # see below
269
270 No operator usage: all operators are converted to invocations, including
271 all attribute access except static references to modules or particular class
272 or function definitions using the following notation:
273
274 static(package)
275 static(package.module)
276 static(package.module.cls)
277 static(package.module.cls.function)
278
279 A shorthand dot notation could be employed:
280
281 package.module
282 package.module.cls
283 package.module.cls.function
284
285 Where multiple definitions of static objects occur, the dot notation cannot be
286 used, and the full name of such definitions must be quoted. For example:
287
288 static("package.module.cls#1.function")
289
290 In general, attribute access must use an explicit function indicating the
291 kind of access operation being performed. For example:
292
293 # Instance-related operations:
294
295 loadattr(obj, attrname) # preserve retrieved context
296
297 # Constant attribute operations:
298
299 static(value) # see above
300 loadconstant(value, obj) # replace context with obj
301
302 # Static attribute operations:
303
304 loadaddress(parent, attrname) # preserve retrieved context
305 loadaddresscontext(parent, attrname, obj) # replace context with obj
306 loadaddresscontextcond(parent, attrname, obj) # run-time context decision
307
308 # Unoptimised operations:
309
310 loadattrindex(obj, attrname) # preserve retrieved context
311 loadattrindexcontextcond(obj, attrname) # run-time context decision
312
313 # Instance-related operations:
314
315 storeattr(obj, attrname, value) # preserve context for value
316
317 # Static attribute operations:
318
319 storeaddress(parent, attrname, value) # preserve context for value
320 storeaddresscontext(parent, attrname, value, obj) # replace context with obj
321
322 # Unoptimised operations:
323
324 storeattrindex(obj, attrname, value) # preserve context for value
325
326 Recall that for loadattrindex family functions, the location of the attribute
327 is obtained from the object table and the nature of the attribute is
328 determined from the stored context value.
329
330 Temporary variables could employ similar functions:
331
332 loadtemp(0)
333 storetemp(0, value)
334
335 Operators and Invocations
336 -------------------------
337
338 Conventional operators use the operator functions.
339
340 Special operators could also use the operator functions (where available)
341 but might as well be supported directly:
342
343 __is__(a, b)
344 __is_not__(a, b)
345
346 Logical operators involving short-circuit evaluation could be represented
347 as function calls, but the evaluation semantics would be preserved:
348
349 __and__(...) # returns the first non-true value or the final value
350 __not__(obj) # returns the inverse of the boolean interpretation of obj
351 __or__(...) # returns the first true value or the final value
352
353 Comparisons could be rephrased in a verbose fashion:
354
355 a < b < c becomes lt(a, b) and lt(b, c)
356 or __and__(lt(a, b), lt(b, c))
357
358 Advanced Control-Flow
359 ---------------------
360
361 Any statements requiring control-flow definition in terms of blocks must
362 be handled in the language as the notions of labels and blocks are not
363 introduced earlier apart from the special case of jumping to another
364 callable (described below).
365
366 Special functions for low-level operations:
367
368 check(obj, type)
369 jump(callable)
370
371 Function/subroutine definition with entry points for checked and unchecked
372 parameters.
373
374 def fn_checked(self, ...):
375 check(self, Type) # raises a TypeError if not isinstance(self, Type)
376 jump(fn_unchecked) # preserves the frame and return address
377
378 def fn_unchecked(self, ...):
379 ...
380
381 The jump function might also be used for inlining appropriate functions.
382
383 Exceptions must also be handled in the language.
384
385 Object Type Detection
386 ---------------------
387
388 Occasionally, the type of an object (instance of a particular class, class,
389 and so on) needs to be determined at run-time:
390
391 isclass(obj)