micropython

Annotated TO_DO.txt

794:a2ce972280d6
2014-03-17 Paul Boddie Made generator expressions return instances. syspython-as-target
paul@472 1
Name usage types: as parameters, as base classes, as callables. This potentially restricts
paul@472 2
attribute usage effects because names mentioned as base classes are not propagated and
paul@472 3
made freely available for use in attribute accesses.
paul@472 4
paul@779 5
Uninitialised Attributes
paul@779 6
========================
paul@779 7
paul@779 8
Make sure that these are either detected or are initialised in a form that might cause an
paul@779 9
exception. The latter mechanism might then support attribute deletion.
paul@779 10
paul@419 11
Low-Level Instructions and Macro Instructions
paul@419 12
=============================================
paul@419 13
paul@429 14
Have contexts and values stored separately in memory. This involves eliminating DataValue
paul@429 15
and storing attributes using two words.
paul@429 16
paul@419 17
Migrate macro instructions such as the *Index instructions to library code implemented
paul@419 18
using low-level instructions.
paul@419 19
paul@419 20
Consider introducing classic machine level instructions (word addition, subtraction, and
paul@419 21
so on) in order to implement all current RSVP instructions.
paul@419 22
paul@450 23
Move common code sequences to a library routine, such as the context checking that occurs
paul@450 24
in functions and methods.
paul@450 25
paul@464 26
Dataflow Optimisations
paul@464 27
======================
paul@450 28
paul@464 29
Assignments, particularly now that no result register exists, may cause StoreTemp/LoadTemp
paul@464 30
instruction pairs to be produced and these could be eliminated.
paul@450 31
paul@631 32
Ambiguous/Multiple Class/Function Definitions
paul@631 33
=============================================
paul@631 34
paul@631 35
Classes and functions are not supposed to have multiple definitions, where one code path
paul@631 36
may define one form of a class or function with a given name and another code path may
paul@631 37
define another form with that name. Currently, such multiple definitions are treated like
paul@631 38
"unions" in the object table.
paul@631 39
paul@635 40
  Consider functions as well as classes which are supported using "shadow" names for the
paul@635 41
  second and subsequent definitions of classes in the same namespace.
paul@635 42
paul@417 43
Class and Module Attribute Assignment
paul@417 44
=====================================
paul@417 45
paul@557 46
Allow unrestricted class and module assignment (but not new external binding of
paul@557 47
attributes), eliminating run-time checks on object types in instructions like
paul@557 48
StoreAttrIndex. This may involve less specific objects being identified during inspection.
paul@557 49
paul@568 50
  Potentially re-evaluate class bases in order to see if they are non-constant.
paul@568 51
paul@417 52
Verify that the context information is correctly set, particularly for the unoptimised
paul@417 53
cases.
paul@417 54
paul@417 55
  Update docs/assignment.txt.
paul@417 56
paul@431 57
Prevent assignments within classes, such as method aliasing, from causing the source of an
paul@431 58
assignment from being automatically generated. Instead, only external references should be
paul@431 59
registered.
paul@431 60
paul@431 61
Prevent "from <module> import ..." statements from registering references to such local
paul@431 62
aliases such that they cause the source of each alias to be automatically generated.
paul@431 63
paul@419 64
Consider attribute assignment observations, along with the possibility of class and module
paul@419 65
attribute assignment.
paul@419 66
paul@419 67
  (Note direct assignments as usual, indirect assignments via the attribute usage
paul@419 68
  mechanism. During attribute collection and inference, add assigned values to all
paul@419 69
  inferred targets.)
paul@419 70
paul@419 71
  (Since class attributes can be assigned, StoreAttrIndex would no longer need to reject
paul@419 72
  static attributes, although this might still be necessary where attribute usage analysis
paul@419 73
  has not been performed.)
paul@419 74
paul@419 75
  Potentially consider changing static attribute details to use object-relative offsets in
paul@419 76
  order to simplify the instruction implementations. This might allow us to eliminate the
paul@419 77
  static attribute flag for attributes in the object table, at least at run-time.
paul@419 78
paul@413 79
Dynamic Attribute Access
paul@413 80
========================
paul@413 81
paul@425 82
Consider explicit accessor initialisation:
paul@425 83
paul@425 84
  attr = accessor("attr")
paul@425 85
  getattr(C, attr)
paul@413 86
paul@394 87
Attribute Usage
paul@394 88
===============
paul@394 89
paul@498 90
To consider: is it useful to distinguish between attribute name sets when the same names
paul@498 91
are mentioned, but where one path through the code sets different values on attributes
paul@498 92
than another? The _attrtypes collapses observations in order to make a list of object
paul@498 93
types for a name, and the final set of names leading to such type deductions might be a
paul@498 94
useful annotation to be added alongside _attrcombined.
paul@498 95
paul@613 96
  (Update the reports to group identical sets of attribute names.)
paul@613 97
paul@613 98
Attribute usage on attributes might be possible if one can show that the expression of an
paul@613 99
attribute access is constant and that the attribute target is also constant or only refers
paul@613 100
to a single type. For example, in the sys module:
paul@613 101
paul@613 102
  stderr = file()
paul@613 103
paul@613 104
If no work is done to associate the result of the invocation with the stderr name, then
paul@613 105
one could instead at least attempt to determine whether stderr is assigned only once. If
paul@613 106
so, it might be possible to record attribute usage on references to the name. For example:
paul@613 107
paul@613 108
  sys.stderr.write(...) # sys.stderr supports write -> {file, ...}
paul@575 109
paul@498 110
Interface/Type Generalisation
paul@498 111
-----------------------------
paul@498 112
paul@480 113
Consolidate interface observations by taking all cached table accesses and determining
paul@480 114
which usage patterns lead to the same types. For example, if full usage of {a, b} and
paul@480 115
{a, b, c} leads to A and B in both cases, either {a, b} can be considered as partial usage
paul@480 116
of the complete interface {a, b, c}, or the latter can be considered as an
paul@480 117
overspecification of the former.
paul@480 118
paul@364 119
Consider type deduction and its consequences where types belong to the same hierarchy
paul@364 120
and where a guard could be generated for the most general type.
paul@364 121
paul@364 122
Consider permitting multiple class alternatives where the attributes are all identical.
paul@364 123
paul@360 124
Support class attribute positioning similar to instance attribute positioning, potentially
paul@360 125
(for both) based on usage observations. For example, if __iter__ is used on two classes,
paul@360 126
the class attribute could be exposed at a similar relative position to the class (and
paul@360 127
potentially accessible using a LoadAttr-style instruction).
paul@360 128
paul@394 129
**** Constant attribute users need not maintain usage since they are already resolved. ****
paul@394 130
paul@613 131
Self-Related Usage
paul@498 132
------------------
paul@498 133
paul@498 134
Perform attribute usage on attributes of self as names, potentially combining observations
paul@498 135
across methods.
paul@498 136
paul@498 137
Additional Guards
paul@498 138
-----------------
paul@498 139
paul@710 140
Consider handling branches of values within namespaces in order to support more precise
paul@710 141
value usage.
paul@498 142
paul@498 143
Loop entry points and other places where usage becomes more specific might be used as
paul@498 144
places to impose guards. See tests/attribute_access_type_restriction_loop_list.py for an
paul@498 145
example. (Such information is already shown in the reports.)
paul@498 146
paul@498 147
Strict Interfaces/Types
paul@498 148
-----------------------
paul@498 149
paul@498 150
Make the gathering of usage parameterisable according to the optimisation level so that a
paul@498 151
choice can be made between control-flow-dependent observations and the simple collection
paul@498 152
of all attributes used with a name (producing a more static interface observation).
paul@498 153
paul@498 154
AttributeError
paul@498 155
--------------
paul@498 156
paul@504 157
Consider attribute usage observations being suspended or optional inside blocks where
paul@504 158
AttributeError may be caught (although this doesn't anticipate such exceptions being
paul@504 159
caught outside a function altogether). For example:
paul@504 160
paul@504 161
  y = a.y
paul@504 162
  try:
paul@504 163
      z = a.z # z is an optional attribute
paul@504 164
  except AttributeError:
paul@504 165
      z = None
paul@498 166
paul@394 167
Frame Optimisations
paul@394 168
===================
paul@394 169
paul@394 170
Stack frame replacement where a local frame is unused after a call, such as in a tail call
paul@394 171
situation.
paul@394 172
paul@394 173
Local assignment detection plus frame re-use. Example: slice.__init__ calls
paul@394 174
xrange.__init__ with the same arguments which are unchanged in xrange.__init__. There is
paul@419 175
therefore no need to build a new frame for this call, although in some cases the locals
paul@419 176
frame might need expanding.
paul@419 177
paul@456 178
Reference tracking where objects associated with names are assigned to attributes of other
paul@456 179
objects may assist in allocation optimisations. Recording whether an object referenced by
paul@456 180
a name is assigned to an attribute, propagated to another name and assigned to an
paul@456 181
attribute, or passed to another function or method might, if such observations were
paul@456 182
combined, allow frame-based or temporary allocation to occur.
paul@456 183
paul@620 184
Instantiation Deduction
paul@620 185
=======================
paul@620 186
paul@620 187
Consider handling Const, List and Tuple in micropython.inspect in order to produce
paul@620 188
instances of specific classes. Then, consider adding support for guard
paul@620 189
removal/verification where known instances are involved. For example:
paul@620 190
paul@620 191
  l = []
paul@620 192
  l.append(123) # type deductions are filtered using instantiation knowledge
paul@620 193
paul@620 194
Currently, this is done only for Const values in the context of attribute accesses during
paul@620 195
inspection.
paul@620 196
paul@710 197
Handling CallFunc in a similar way is more challenging. Consider the definitions in the
paul@710 198
sys module:
paul@472 199
paul@620 200
  stderr = file()
paul@620 201
paul@620 202
It must first be established that file only ever refers to the built-in file class, and
paul@620 203
only then can the assumption be made that stderr in this case refers to instances of file.
paul@620 204
If file can also refer to other objects, potential filtering operations are more severely
paul@620 205
limited.
paul@620 206
paul@710 207
Propagation of type information can also occur. For example:
paul@710 208
paul@710 209
  DeducedSource(module, program).deduce()
paul@710 210
paul@710 211
The DeducedSource invocation, if yielding an instance of the DeducedSource class, can then
paul@710 212
supply the attribute access operation with type information.
paul@710 213
paul@710 214
A more advanced example involves accesses then invocations:
paul@710 215
paul@710 216
  x = self.__class__()
paul@710 217
paul@710 218
Here, the effect should be the inference that x may refer to an instance of one of a
paul@710 219
number of eligible types of which self is also an instance.
paul@710 220
paul@620 221
Invocation-Related Deduction
paul@620 222
============================
paul@620 223
paul@620 224
Where an attribute access (either in conjunction with usage observations or independently)
paul@620 225
could refer to a number of different targets, but where the resulting attribute is then
paul@620 226
used in an invocation, filtering of the targets could be done to eliminate any targets
paul@620 227
that are not callable. Guards would need introducing to prevent inappropriate operations
paul@620 228
from occurring at run-time.
paul@472 229
paul@419 230
Inlining
paul@419 231
========
paul@419 232
paul@419 233
Where a function or method call can always be determined, the body of the target could be
paul@419 234
inlined - copied into place - within the caller. If the target is only ever called by a
paul@456 235
single caller it could be moved into place. This could enhance deductions based on
paul@456 236
attribute usage since observations from the inlined function would be merged into the
paul@456 237
caller.
paul@394 238
paul@672 239
Distinguish between frame sharing and inlining: where a called function does not rebind
paul@672 240
its names, and where the frame of the caller is compatible, the frame of the caller might
paul@672 241
be shared with the called function even if a branch and return is still involved.
paul@672 242
paul@672 243
Suitable candidates for inlining, frame sharing or enhanced analysis might be lambdas and
paul@672 244
functions containing a single statement.
paul@672 245
paul@779 246
Consider what it would take to support the inlining or analysis of this in the operator
paul@779 247
module...
paul@779 248
paul@779 249
add(x, y) -> binary_op(x, y, lambda a: a.__add__, lambda b: b.__radd__)
paul@779 250
          -> x.__add__(y) ; y.__radd__(x)
paul@779 251
paul@394 252
Function Specialisation
paul@394 253
=======================
paul@394 254
paul@394 255
Specialisation of certain functions, such as isinstance(x, cls) where cls is a known
paul@394 256
constant.
paul@394 257
paul@394 258
Structure and Object Table Optimisations
paul@394 259
========================================
paul@394 260
paul@394 261
Fix object table entries for attributes not provided by any known object, or provide an
paul@394 262
error, potentially overridden by options. For example, the augmented assignment methods
paul@394 263
are not supported by the built-in objects and thus the operator module functions cause
paul@394 264
the compilation to fail. Alternatively, just supply the methods since something has to do
paul@394 265
so in the builtins.
paul@394 266
paul@394 267
Consider attribute merging where many attributes are just aliases for the same underlying
paul@394 268
definition.
paul@394 269
paul@349 270
Consider references to defaults as occurring only within the context of a particular
paul@349 271
function, thus eliminating default value classes if such functions are not themselves
paul@349 272
invoked.
paul@349 273
paul@394 274
Scope Handling
paul@394 275
==============
paul@394 276
paul@394 277
Consider merging the InspectedModule.store tests with the scope conflict handling.
paul@394 278
paul@343 279
Consider labelling _scope on assignments and dealing with the assignment of removed
paul@343 280
attributes, possibly removing the entire assignment, and distinguishing between such cases
paul@343 281
and unknown names.
paul@343 282
paul@342 283
Check name origin where multiple branches could yield multiple scope interpretations:
paul@342 284
paul@504 285
  try:
paul@504 286
      set # built-in name
paul@504 287
  except NameError:
paul@504 288
      from sets import Set as set # local definition of name
paul@342 289
paul@504 290
  set # could be confused by the local definition at run-time
paul@342 291
paul@394 292
Object Coverage
paul@394 293
===============
paul@394 294
paul@332 295
Support __init__ traversal (and other implicit names) more effectively.
paul@332 296
paul@499 297
Importing Modules
paul@499 298
=================
paul@499 299
paul@710 300
(Explicit relative imports are now supported.) Consider supporting relative imports, even
paul@710 301
though this is arguably a misfeature.
paul@499 302
paul@394 303
Other
paul@394 304
=====
paul@394 305
paul@332 306
Check context_value initialisation (avoiding or handling None effectively).
paul@332 307
paul@342 308
Consider better "macro" support where new expressions need to be generated and processed.
paul@402 309
paul@402 310
Detect TestIdentity results involving constants, potentially optimising status-affected
paul@402 311
instructions:
paul@402 312
paul@402 313
  TestIdentity(x, y) # where x is always y
paul@402 314
  JumpIfFalse(...)   # would be removed (never false)
paul@402 315
  JumpIfTrue(...)    # changed to Jump(...)
paul@402 316
paul@402 317
Status-affected blocks could be optimised away for such constant-related results.
paul@672 318
paul@672 319
Caching of structure and attribute usage information for incremental compilation.