micropython

micropython/inspect.py

578:d161d4ac7dc0
2012-07-06 Paul Boddie Added module global modification/invalidation upon imports. Reverted the separate module processing phases introduced in 8824b42d7b71. Established a mechanism for modifying module globals, updating Attr assignment statistics and recording each name. Changed the update method on the Attr class to use a separate assignments update method, fixing the number of assignments recorded. Changed the get_context and get_value methods on the Attr class to test the number of assignments as well as the length of the list of recorded values. Changed one of the tests to more thoroughly demonstrate the effects of global modification on attribute usage and constant value identification.
     1 #!/usr/bin/env python     2      3 """     4 Inspect source files, obtaining details of classes and attributes.     5      6 Copyright (C) 2007, 2008, 2009, 2010, 2011, 2012 Paul Boddie <paul@boddie.org.uk>     7      8 This program is free software; you can redistribute it and/or modify it under     9 the terms of the GNU General Public License as published by the Free Software    10 Foundation; either version 3 of the License, or (at your option) any later    11 version.    12     13 This program is distributed in the hope that it will be useful, but WITHOUT    14 ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS    15 FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more    16 details.    17     18 You should have received a copy of the GNU General Public License along with    19 this program.  If not, see <http://www.gnu.org/licenses/>.    20     21 --------    22     23 The results of inspecting a module are as follows:    24     25 Constants    26 ---------    27     28 All constants identified within the code shall be registered.    29     30 Classes    31 -------    32     33 All global classes shall be registered; local classes (within functions) or    34 nested classes (within classes) are not currently registered.    35     36 Base classes must be detected and constant.    37     38 All classes without bases are made to inherit from __builtins__.object in order    39 to support some standard methods.    40     41 Functions    42 ---------    43     44 All functions and lambda definitions shall be registered.    45     46 Namespaces    47 ----------    48     49 Modules define their own "global" namespace, within which classes, functions    50 and lambda definitions establish a hierarchy of namespaces.    51     52 Only local, global and built-in namespaces are recognised; closures are not    53 supported.    54     55 Assignments    56 -----------    57     58 Name assignment and attribute assignment involving modules and classes cause    59 names to be associated with values within namespaces.    60     61 Any assignments within loops are considered to cause the targets of such    62 assignments to provide non-constant values.    63     64 Assignments to names are only really considered to cause the targets of such    65 assignments to provide constant values if the targets reside in class    66 namespaces, subject to the above conditions.    67     68 Assignments to names within functions are not generally considered to cause the    69 targets of such assignments to provide constant values since functions can be    70 invoked many times with different inputs. This affects particularly the    71 definition of functions or lambdas within functions. However, there may be    72 benefits in considering a local to be constant within a single invocation.    73 """    74     75 from micropython.common import *    76 from micropython.data import *    77 from micropython.errors import *    78 import compiler.ast    79 import sys    80     81 # Program visitors.    82     83 class InspectedModule(ASTVisitor, Module):    84     85     """    86     An inspected module, providing core details via the Module superclass, but    87     capable of being used as an AST visitor.    88     89     A module can be inspected through the invocation of the following methods in    90     order:    91     92      1. parse    93      2. vacuum    94      3. finalise    95     96     A module importer can be expected to perform these invocations.    97     """    98     99     def __init__(self, name, importer):   100    101         """   102         Initialise this visitor with a module 'name' and an 'importer' which is   103         used to provide access to other modules when required.   104         """   105    106         Module.__init__(self, name, importer)   107         self.visitor = self   108    109         # Import machinery links.   110    111         self.optimisations = self.importer.optimisations   112         self.builtins = self.importer.modules.get("__builtins__")   113         self.loaded = 0   114    115         # Current expression state.   116    117         self.expr = None   118         self.in_assignment = 0  # For slice and subscript handling.   119    120         # Namespace state.   121    122         self.in_method = 0      # Find instance attributes in all methods.   123         self.in_function = 0    # Note function presence, affecting definitions.   124         self.in_loop = 0        # Note loop "membership", affecting assignments.   125         self.namespaces = []   126         self.functions = []   127    128     def parse(self, filename):   129    130         "Parse the file having the given 'filename'."   131    132         module = compiler.parseFile(filename)   133         self.process(module)   134    135     def process(self, module):   136    137         """   138         Process the given 'module', visiting only module-level code and only   139         extracting global declarations from functions. In order to support   140         deferred imports, the function code is processed separately after all   141         modules have been initially processed.   142         """   143    144         self.astnode = module   145    146         # Add __name__ to the namespace.   147    148         self.store("__name__", self._visitConst(self.full_name()))   149    150         # Detect and record globals declared in the module.   151    152         self.process_globals(module)   153    154         # Visit module-level code, also recording global names.   155    156         processed = self.dispatch(module)   157    158         self.finalise_attribute_usage()   159    160         # Visit functions.   161    162         self.process_functions()   163    164         # Add references to other modules declared using the __all__ global.   165    166         if self.has_key("__all__"):   167             all = self["__all__"]   168             if isinstance(all, compiler.ast.List):   169                 for n in all.nodes:   170                     self.store(n.value, self.importer.add_module(self.name + "." + n.value))   171    172         return processed   173    174     def process_functions(self):   175    176         """   177         Process all function bodies. Deferred imports may occur during this   178         process.   179         """   180    181         # Then, visit each function, recording other names. This happens to   182         # work for lambda definitions inside functions since they are added to   183         # the end of self.functions and are thus visited as the iteration   184         # reaches the end of the original list.   185    186         for node, namespaces in self.functions:   187             self._visitFunctionBody(node, namespaces)   188             namespaces[-1].finalise_attribute_usage()   189    190     def process_globals(self, node):   191    192         """   193         Within the given 'node', process global declarations, adjusting the   194         module namespace.   195         """   196    197         for n in node.getChildNodes():   198             if isinstance(n, compiler.ast.Global):   199                 for name in n.names:   200    201                     # Each name may potentially be assigned many times.   202                     # We don't try and find out the specifics at this point and   203                     # just indicate that the name cannot be relied upon for   204                     # various observations.   205    206                     module.modify_name(name)   207    208             else:   209                 self.process_globals(n)   210    211     def vacuum(self):   212    213         """   214         Vacuum the module namespace, removing unreferenced objects and unused   215         names.   216         """   217    218         if self.should_optimise_unused_objects():   219             self.vacuum_object(self)   220    221             all_objects = list(self.all_objects)   222    223             for obj in all_objects:   224                 self.vacuum_object(obj)   225    226     def vacuum_object(self, obj, delete_all=0):   227    228         "Vacuum the given object 'obj'."   229    230         # Get all constant objects in apparent use.   231    232         if delete_all:   233             obj_objects = set()   234         else:   235             obj_objects = []   236             for name, attr in obj.items_for_vacuum():   237    238                 # Get constant objects for attributes in use.   239    240                 if self.importer.uses_attribute(obj.full_name(), name) and \   241                     attr is not None and attr.is_constant():   242    243                     value = attr.get_value()   244                     obj_objects.append(value)   245    246         # Now vacuum unused attributes and objects not in use.   247    248         for name, attr in obj.items_for_vacuum():   249    250             # Only consider deleting entire unused objects or things accessible   251             # via names which are never used.   252    253             if delete_all or not self.importer.uses_attribute(obj.full_name(), name):   254                 obj.vacuum_item(name)   255    256                 # Delete any unambiguous attribute value. Such values can only   257                 # have been defined within the object and therefore are not   258                 # redefined by other code regions.   259    260                 if attr is not None and attr.is_constant():   261                     value = attr.get_value()   262    263                     # The value must have this object as a parent.   264                     # However, it must not be shared by several names.   265    266                     if value is not obj and value.parent is obj and \   267                         value in self.all_objects and value not in obj_objects:   268    269                         self.all_objects.remove(value)   270    271                         # Delete class contents and lambdas from functions.   272    273                         self.vacuum_object(value, 1)   274    275     def unfinalise(self):   276    277         "Reset finalised information for the module."   278    279         for obj in self.all_objects:   280             obj.unfinalise_attributes()   281    282     def finalise(self, objtable):   283    284         "Finalise the module."   285    286         for obj in self.all_objects:   287             obj.finalise(objtable)   288    289         self.finalise_users(objtable)   290    291     def add_object(self, obj, any_scope=0):   292    293         """   294         Record 'obj' if non-local or if the optional 'any_scope' is set to a   295         true value.   296         """   297    298         if any_scope or not (self.namespaces and isinstance(self.namespaces[-1], Function)):   299             self.all_objects.add(obj)   300    301     # Optimisation tests.   302    303     def should_optimise_unused_objects(self):   304         return "unused_objects" in self.optimisations   305    306     # Namespace methods.   307    308     def in_class(self, namespaces=None):   309         namespaces = namespaces or self.namespaces   310         return len(namespaces) > 1 and isinstance(namespaces[-2], Class)   311    312     def store(self, name, obj):   313    314         "Record attribute or local 'name', storing 'obj'."   315    316         # Store in the module.   317    318         if not self.namespaces:   319             if self.in_loop and self.used_in_scope(name, "builtins"):   320                 raise InspectError("Name %r already used as a built-in." % name)   321             else:   322                 self.set(name, obj, not self.in_loop)   323    324         # Or store locally.   325    326         else:   327             locals = self.namespaces[-1]   328    329             if self.in_loop and locals.used_in_scope(name, "global") and not name in locals.globals:   330                 raise InspectError("Name %r already used as global." % name)   331             elif self.in_loop and locals.used_in_scope(name, "builtins"):   332                 raise InspectError("Name %r already used as a built-in." % name)   333             else:   334                 locals.set(name, obj, not self.in_loop)   335    336     def store_lambda(self, obj):   337    338         "Store a lambda function 'obj'."   339    340         self.add_object(obj)   341         self.get_namespace().add_lambda(obj)   342    343     def store_module_attr(self, name, module):   344    345         """   346         Record module attribute 'name' in the given 'module' using the current   347         expression.   348         """   349    350         module.set(name, self.expr, 0)   351         self.use_specific_attribute(module.full_name(), name)   352    353     def store_class_attr(self, name, cls):   354    355         """   356         Record class attribute 'name' in the given class 'cls' using the current   357         expression.   358         """   359    360         cls.set(name, self.expr, 0)   361         self.use_specific_attribute(cls.full_name(), name)   362    363     def store_instance_attr(self, name, tentative=False):   364    365         """   366         Record instance attribute 'name' in the current class. If 'tentative' is   367         set to a true value, the instance attribute will be discarded if a class   368         attribute is observed.   369         """   370    371         if self.in_method:   372    373             # Current namespace is the function.   374             # Previous namespace is the class.   375    376             cls = self.namespaces[-2]   377             cls.add_instance_attribute(name, tentative)   378    379             # NOTE: The instance attribute, although defined in a specific   380             # NOTE: class, obviously appears in all descendant classes.   381    382             self.use_specific_attribute(cls.full_name(), name)   383    384     def get_namespace(self):   385    386         "Return the parent (or most recent) namespace currently exposed."   387    388         return (self.namespaces[-1:] or [self])[0]   389    390     def use_name(self, name, node=None, value=None, ns=None):   391    392         """   393         Use the given 'name' within the current namespace/unit, either in   394         conjunction with a particular object (if 'node' is specified and not   395         None) or unconditionally.   396         """   397    398         unit = self.get_namespace()   399    400         # Handle attribute usage situations within the current unit.   401    402         if node is not None and isinstance(node, compiler.ast.Name) and ns is unit:   403             self.use_attribute(node.name, name, value)   404    405         # For general name usage, declare usage of the given name from this   406         # particular unit.   407    408         else:   409             self.importer.use_name(name, unit.full_name(), value)   410    411     def use_constant(self, const):   412    413         "Use the given 'const' within the current namespace/unit."   414    415         unit = self.get_namespace()   416         self.importer.use_constant(const, unit.full_name())   417    418     # Attribute usage methods.   419     # These are convenience methods which refer to the specific namespace's   420     # implementation of these operations.   421    422     def new_branchpoint(self, loop_node=None):   423         self.get_namespace()._new_branchpoint(loop_node)   424    425     def new_branch(self, node):   426         self.get_namespace()._new_branch(node)   427    428     def abandon_branch(self):   429         self.get_namespace()._abandon_branch()   430    431     def suspend_broken_branch(self):   432         self.get_namespace()._suspend_broken_branch()   433    434     def suspend_continuing_branch(self):   435         self.get_namespace()._suspend_continuing_branch()   436    437     def shelve_branch(self):   438         self.get_namespace()._shelve_branch()   439    440     def merge_branches(self):   441         self.get_namespace()._merge_branches()   442    443     def resume_broken_branches(self):   444         self.get_namespace()._resume_broken_branches()   445    446     def resume_continuing_branches(self):   447         self.get_namespace()._resume_continuing_branches()   448    449     def resume_abandoned_branches(self):   450         self.get_namespace()._resume_abandoned_branches()   451    452     def define_attribute_user(self, node):   453    454         """   455         Define 'node' as the user of attributes, indicating the point where the   456         user is defined.   457         """   458    459         self.get_namespace()._define_attribute_user(node)   460    461     def use_attribute(self, name, attrname, value=None):   462    463         """   464         Note usage on the attribute user 'name' of the attribute 'attrname',   465         noting an assignment if 'value' is specified.   466         """   467    468         return self.get_namespace()._use_attribute(name, attrname, value)   469    470     def use_specific_attribute(self, objname, attrname, from_name=None):   471    472         """   473         Note usage on the object having the given 'objname' of the attribute   474         'attrname'. If 'objname' is None, the current namespace is chosen as the   475         object providing the attribute.   476         """   477    478         return self.get_namespace()._use_specific_attribute(objname, attrname, from_name)   479    480     def define_attribute_accessor(self, name, attrname, node, value=None):   481    482         """   483         Note applicable attribute users providing the given 'name' when   484         accessing the given 'attrname' on the specified 'node', with the   485         optional 'value' indicating an assignment.    486         """   487    488         self.get_namespace()._define_attribute_accessor(name, attrname, node, value)   489    490     # Visitor methods.   491    492     def default(self, node, *args):   493         raise InspectError("Node class %r is not supported." % node.__class__)   494    495     def NOP(self, node):   496         for n in node.getChildNodes():   497             self.dispatch(n)   498    499     def NOP_ABANDON(self, node):   500         self.NOP(node)   501         self.abandon_branch()   502    503     def TEST_NOP(self, node):   504         self.use_name("__bool__", node)   505         self.NOP(node)   506    507     def OP(self, node):   508         for n in node.getChildNodes():   509             self.dispatch(n)   510         return make_instance()   511    512     def TEST_OP(self, node):   513         self.use_name("__bool__", node)   514         self.new_branchpoint()   515    516         # Propagate attribute usage to branches.   517         # Each node starts a new conditional region, effectively making a deeply   518         # nested collection of if-like statements.   519    520         for n in node.nodes:   521             self.new_branch(n)   522             self.dispatch(n)   523    524         # The nested regions must be terminated.   525    526         for n in node.nodes:   527             self.shelve_branch()   528    529         self.merge_branches()   530         return make_instance()   531    532     # Generic support for classes of operations.   533    534     def _ensureOperators(self):   535         attr, scope, namespace = self._get_with_scope("$operator")   536         if attr is None:   537             module = self.importer.load("operator")   538             self["$operator"] = module   539         else:   540             module = attr.get_value()   541         return module   542    543     def _visitOperator(self, node, operator_name=None):   544    545         "Accounting method for the operator 'node'."   546    547         operator_module = self._ensureOperators()   548         operator_fn = operator_functions[operator_name or node.__class__.__name__]   549         self.use_specific_attribute(operator_module.full_name(), operator_fn)   550         return self.OP(node)   551    552     _visitBinary = \   553     _visitUnary = _visitOperator   554    555     def _visitAttr(self, expr, attrname, node):   556    557         """   558         Process the attribute provided by the given 'expr' with the given   559         'attrname' and involving the given 'node'.   560         """   561    562         # Attempt to identify the nature of the attribute.   563    564         if isinstance(expr, Attr):   565             value = expr.get_value()   566    567             # Get the attribute and record its usage.   568             # NOTE: Need to provide concrete values for things like base classes   569             # NOTE: while also handling module attribute modification.   570    571             if isinstance(value, (Class, Module)):   572    573                 # Check for class.__class__.   574    575                 if attrname == "__class__" and isinstance(value, Class):   576                     attr = type_class   577                 else:   578                     attr = value.get(attrname) or make_instance()   579                 self.use_specific_attribute(value.full_name(), attrname)   580    581             elif isinstance(value, UnresolvedName):   582                 attr = UnresolvedName(attrname, value.full_name(), self)   583    584             # The actual attribute is not readily identifiable and is assumed   585             # to be an instance.   586    587             else:   588    589                 # Record any instance attributes.   590    591                 if expr.name == "self":   592                     self.store_instance_attr(attrname, tentative=True)   593    594                 attr = make_instance()   595    596                 # Note usage of the attribute where a local is involved.   597    598                 self._visitAttrUser(expr, attrname, node)   599    600         # No particular attribute has been identified, thus a general instance   601         # is assumed.   602    603         else:   604             attr = make_instance()   605             self.use_name(attrname, node)   606    607         return attr   608    609     def _visitAttrUser(self, expr, attrname, node, value=None):   610    611         """   612         Note usage of the attribute provided by 'expr' with the given 'attrname'   613         where a local is involved, annotating the given 'node'. If the optional   614         'value' is given, note an assignment for future effects on attributes   615         where such attributes are inferred from the usage.   616         """   617    618         # Access to attributes via a local in functions, classes or modules.   619         # Since module-level locals are globals that can be modified   620         # independently of the namespace, any attribute usage observations made   621         # here may be revoked later if such modification is thought to occur.   622    623         if expr.parent is self.get_namespace():   624             self.define_attribute_accessor(expr.name, attrname, node, value)   625         else:   626             self.use_name(attrname, node.expr, value, ns=expr.parent)   627    628     def _visitConst(self, value):   629    630         """   631         Register the constant given by 'value', if necessary, returning the   632         resulting object. The type name is noted as being used, thus preserving   633         the class in any generated program.   634         """   635    636         self.use_specific_attribute("__builtins__", self.importer.get_constant_type_name(value))   637         const = self.importer.make_constant(value)   638         self.use_constant(const)   639         return const   640    641     def _visitFunction(self, node, name):   642    643         """   644         Return a function object for the function defined by 'node' with the   645         given 'name'. If a lambda expression is being visited, 'name' should be   646         None.   647         """   648    649         # Define the function object.   650    651         function = Function(   652             name,   653             self.get_namespace(),   654             node.argnames,   655             node.defaults,   656             (node.flags & 4 != 0),   657             (node.flags & 8 != 0),   658             self.in_loop or self.in_function,   659             self,   660             node   661             )   662    663         self.add_object(function, any_scope=1)   664    665         # Make a back reference from the node for code generation.   666    667         node.unit = function   668    669         # Process the defaults.   670    671         for n in node.defaults:   672             self.expr = self.dispatch(n)   673             function.store_default(self.expr)   674    675         # Note attribute usage where tuple parameters are involved.   676    677         if function.tuple_parameters():   678             self.use_name("__getitem__", node)   679    680         # Record the namespace context of the function for later processing.   681    682         self.functions.append((node, self.namespaces + [function]))   683    684         # Store the function.   685    686         if name is not None:   687             self.store(name, function)   688         else:   689             self.store_lambda(function)   690    691         # Test the defaults and assess whether an dynamic object will result.   692    693         function.make_dynamic()   694         return function   695    696     def _visitFunctionBody(self, node, namespaces):   697    698         "Enter the function."   699    700         # Current namespace is the function.   701         # Previous namespace is the class.   702    703         if self.in_class(namespaces):   704             self.in_method = 1   705    706         in_function = self.in_function   707         in_loop = self.in_loop   708         self.in_function = 1   709         self.in_loop = 0   710    711         self.namespaces = namespaces   712         self.dispatch(node.code)   713    714         self.in_loop = in_loop   715         self.in_function = in_function   716         self.in_method = 0   717    718     # Specific handler methods.   719    720     visitAdd = _visitBinary   721    722     visitAnd = TEST_OP   723    724     visitAssert = NOP   725    726     def visitAssign(self, node):   727         self.expr = self.dispatch(node.expr)   728         self.in_assignment = 1   729         for n in node.nodes:   730             self.dispatch(n)   731         self.in_assignment = 0   732    733     def visitAssAttr(self, node):   734         expr = self.dispatch(node.expr)   735         attrname = node.attrname   736    737         # Record the attribute on the presumed target.   738    739         if isinstance(expr, Attr):   740             value = expr.get_value()   741    742             if expr.name == "self":   743                 self.store_instance_attr(attrname)   744                 self.use_attribute(expr.name, attrname, value)   745                 self._visitAttrUser(expr, attrname, node, self.expr)   746    747             elif isinstance(value, Module):   748                 self.store_module_attr(attrname, value)   749                 print >>sys.stderr, "Warning: attribute %r of module %r set outside the module." % (node.attrname, expr.get_value().name)   750    751             elif isinstance(value, Class):   752                 self.store_class_attr(attrname, value)   753    754             # Note usage of the attribute where a local is involved.   755    756             else:   757                 self._visitAttrUser(expr, attrname, node, self.expr)   758    759         else:   760             self.use_name(attrname, node)   761    762     def visitAssList(self, node):   763    764         # Declare names which will be used by generated code.   765    766         self.use_name("__getitem__", node)   767    768         # Process the assignment.   769    770         for i, n in enumerate(node.nodes):   771             self.dispatch(n)   772             self._visitConst(i) # for __getitem__(i) at run-time   773    774     def visitAssName(self, node):   775         if hasattr(node, "flags") and node.flags == "OP_DELETE":   776             print >>sys.stderr, "Warning: deletion of attribute %r in %r is not supported." % (node.name, self.full_name())   777             #raise InspectError("Deletion of attribute %r is not supported." % node.name)   778    779         self.store(node.name, self.expr)   780         self.define_attribute_user(node)   781    782         # Ensure the presence of the given name in this namespace.   783         # NOTE: Consider not registering assignments involving methods, since   784         # NOTE: this is merely creating aliases for such methods.   785    786         if isinstance(self.get_namespace(), (Class, Module)):   787             if not isinstance(self.expr, Attr) or not isinstance(self.expr.get_value(), Function):   788                 self.use_specific_attribute(None, node.name)   789             else:   790                 fn = self.expr.get_value()   791                 ns = self.get_namespace().full_name()   792                 self.use_specific_attribute(fn.parent.full_name(), fn.name, "%s.%s" % (ns, node.name))   793    794     visitAssTuple = visitAssList   795    796     def visitAugAssign(self, node):   797    798         # Accounting.   799    800         operator_fn = operator_functions.get(node.op)   801         operator_module = self._ensureOperators()   802         self.use_specific_attribute(operator_module.full_name(), operator_fn)   803    804         # Process the assignment.   805    806         self.expr = self.dispatch(node.expr)   807    808         # NOTE: Similar to micropython.ast handler code.   809         # NOTE: Slices and subscripts are supported by __setitem__(slice) and   810         # NOTE: not __setslice__.   811    812         if isinstance(node.node, compiler.ast.Name):   813             self.visitAssName(node.node)   814         elif isinstance(node.node, compiler.ast.Getattr):   815             self.visitAssAttr(node.node)   816         else:   817             self.use_specific_attribute("__builtins__", "slice")   818             self.use_name("__setitem__", node)   819    820     visitBackquote = OP   821    822     visitBitand = _visitBinary   823    824     visitBitor = _visitBinary   825    826     visitBitxor = _visitBinary   827    828     def visitBreak(self, node):   829         self.NOP(node)   830         self.suspend_broken_branch()   831    832     visitCallFunc = OP   833    834     def visitClass(self, node):   835    836         """   837         Register the class at the given 'node' subject to the restrictions   838         mentioned in the module docstring.   839         """   840    841         if self.namespaces:   842             print >>sys.stderr, "Warning: class %r in %r is not global: ignored." % (node.name, self.namespaces[-1].full_name())   843             return   844         else:   845             if self.in_loop:   846                 print >>sys.stderr, "Warning: class %r in %r defined in a loop." % (node.name, self.full_name())   847    848             cls = get_class(node.name, self.get_namespace(), self, node)   849    850             # Make a back reference from the node for code generation.   851    852             node.unit = cls   853    854             # Process base classes in the context of the class's namespace.   855             # This confines references to such classes to the class instead of   856             # the namespace in which it is defined.   857    858             self.namespaces.append(cls)   859    860             # Visit the base class expressions, attempting to find concrete   861             # definitions of classes.   862    863             for base in node.bases:   864                 expr = self.dispatch(base)   865    866                 # Each base class must be constant and known at compile-time.   867    868                 if isinstance(expr, Attr):   869                     if expr.assignments != 1:   870                         raise InspectError("Base class %r for %r is not constant: %r" % (base, cls.full_name(), expr))   871                     elif not isinstance(expr.get_value(), Class):   872                         raise InspectError("Base class %r for %r is not a class: %r" % (base, cls.full_name(), expr.get_value()))   873                     else:   874                         cls.add_base(expr.get_value())   875    876                 # Where no expression value is available, the base class is   877                 # not identifiable.   878    879                 else:   880                     raise InspectError("Base class %r for %r is not found: it may be hidden in some way." % (base, cls.full_name()))   881    882             # NOTE: Potentially dubious measure to permit __init__ availability.   883             # If no bases exist, adopt the 'object' class.   884    885             if not node.bases and not (self.name == "__builtins__" and node.name == "object") :   886                 expr = self.dispatch(compiler.ast.Name("object"))   887                 cls.add_base(expr.get_value())   888    889             # Make an entry for the class in the parent namespace.   890    891             self.namespaces.pop()   892             self.store(node.name, cls)   893             self.add_object(cls)   894    895             # Process the class body in its own namespace.   896             # Add __name__ to the namespace.   897    898             self.namespaces.append(cls)   899             self.store("__name__", self._visitConst(node.name))   900             self.dispatch(node.code)   901             self.namespaces.pop()   902    903             cls.finalise_attribute_usage()   904             return cls   905    906     def visitCompare(self, node):   907    908         # Accounting.   909         # NOTE: Replicates some code in micropython.ast.visitCompare.   910    911         self.use_name("__bool__", node)   912    913         this_node = node   914    915         for op in node.ops:   916             op_name, next_node = op   917    918             # Define name/attribute usage.   919             # Get the applicable operation.   920    921             operator_fn = operator_functions.get(op_name)   922    923             # For operators, reference the specific function involved.   924    925             if operator_fn is not None:   926                 operator_module = self._ensureOperators()   927                 self.use_specific_attribute(operator_module.full_name(), operator_fn)   928    929             # Define __contains__ usage on the next node.   930    931             elif op_name.endswith("in"):   932                 self.use_name("__contains__", next_node)   933    934             this_node = next_node   935    936         return self.OP(node)   937    938     def visitConst(self, node):   939         return self._visitConst(node.value)   940    941     def visitContinue(self, node):   942         self.NOP(node)   943         self.suspend_continuing_branch()   944    945     visitDecorators = NOP   946    947     visitDict = OP   948    949     visitDiscard = NOP   950    951     visitDiv = _visitBinary   952    953     visitEllipsis = NOP   954    955     visitExec = NOP   956    957     visitExpression = OP   958    959     visitFloorDiv = _visitBinary   960    961     def visitFor(self, node):   962         self.new_branchpoint(node)   963    964         # Declare names which will be used by generated code.   965    966         self.use_name("__iter__", node.list)   967         self.use_name("next")   968         self.use_name("StopIteration")   969    970         in_loop = self.in_loop   971         self.in_loop = 1   972         self.dispatch(node.list)   973    974         # NOTE: Could generate AST nodes for the actual operations instead of   975         # NOTE: manually generating code in micropython.ast.   976    977         self.expr = make_instance() # each element is a result of a function call   978         self.dispatch(node.assign)   979    980         # Enter the loop.   981         # Propagate attribute usage to branches.   982    983         self.new_branch(node)   984         self.dispatch(node.body)   985    986         self.resume_continuing_branches()   987    988         self.shelve_branch()   989    990         self.in_loop = in_loop   991    992         # A null branch is used to record a path around the loop.   993    994         self.new_branch(node.else_ or NullBranch())   995         self.shelve_branch()   996    997         self.merge_branches()   998    999         # The else clause is evaluated outside any branch.  1000   1001         if node.else_ is not None:  1002             self.dispatch(node.else_)  1003   1004         # Any suspended branches from the loop can now be resumed.  1005   1006         self.resume_broken_branches()  1007   1008     def visitFrom(self, node):  1009         module = self.importer.load(node.modname, 1, importer=node)  1010         if module and not module.loaded:  1011             print >>sys.stderr, "Warning: a circular import of %s is being attempted in %s" % (node.modname, self.full_name())  1012   1013         if module is None and self.importer.verbose:  1014             print >>sys.stderr, "Warning:", node.modname, "not imported."  1015   1016         for name, alias in node.names:  1017   1018             # For specific names, obtain and store referenced objects using  1019             # the name or any alias provided in the current namespace.  1020   1021             if name != "*":  1022                 if module:  1023   1024                     # Missing names may refer to submodules.  1025   1026                     if not module.has_key(name):  1027                         submodule = self.importer.load(node.modname + "." + name, 1, importer=node)  1028                         if submodule:  1029                             if not submodule.loaded:  1030                                 print >>sys.stderr, "Warning: a circular import of %s.%s is being attempted in %s" % (  1031                                     node.modname, name, self.full_name())  1032   1033                             module.store(name, submodule)  1034   1035                     # Complete the import if the name was found.  1036   1037                     if module.has_key(name):  1038                         attr = module[name]  1039                         self.store(alias or name, attr)  1040                         self.use_specific_attribute(module.full_name(), name)  1041                         if isinstance(attr.get_value(), Module) and not attr.get_value().loaded:  1042                             submodule = self.importer.load(attr.get_value().name, importer=node)  1043   1044                             # For circular imports, invalidate attribute usage for  1045                             # all exported names of submodules whose names are  1046                             # specified in the from statement.  1047   1048                             if submodule and not submodule.loaded:  1049                                 for n in submodule.keys():  1050                                     submodule.modify_name(n)  1051   1052                         continue  1053   1054                 # Support the import of names from missing modules.  1055   1056                 self.store(alias or name, UnresolvedName(name, node.modname, self))  1057   1058             # For wildcards, obtain and store all objects from a module in the  1059             # current namespace.  1060   1061             else:  1062                 if module:  1063                     for n in module.keys():  1064                         attr = module[n]  1065                         self.store(n, attr)  1066                         self.use_specific_attribute(module.full_name(), n)  1067                         if isinstance(attr.get_value(), Module) and not attr.get_value().loaded:  1068                             submodule = self.importer.load(attr.get_value().name, importer=node)  1069   1070                             # For circular imports, invalidate attribute usage  1071                             # for all exported names of submodules provided by  1072                             # the module.  1073   1074                             if submodule and not submodule.loaded:  1075                                 for subn in submodule.keys():  1076                                     submodule.modify_name(subn)  1077   1078     def visitFunction(self, node):  1079         return self._visitFunction(node, node.name)  1080   1081     visitGenExpr = OP  1082   1083     visitGenExprFor = NOP  1084   1085     visitGenExprIf = NOP  1086   1087     visitGenExprInner = NOP  1088   1089     def visitGetattr(self, node):  1090         expr = self.dispatch(node.expr)  1091         attrname = node.attrname  1092         node._attr = self._visitAttr(expr, attrname, node)  1093         return node._attr  1094   1095     def visitGlobal(self, node):  1096         if self.namespaces:  1097             for name in node.names:  1098                 ns = self.namespaces[-1]  1099                 if not ns.make_global(name):  1100                     raise InspectError("Name %r is global and local in %r" % (name, ns.full_name()))  1101   1102                 # The name is recorded in an earlier process.  1103   1104     def visitIf(self, node):  1105         self.use_name("__bool__", node)  1106         self.new_branchpoint()  1107   1108         # Propagate attribute usage to branches.  1109   1110         for test, body in node.tests:  1111             self.dispatch(test)  1112   1113             self.new_branch(body)  1114             self.dispatch(body)  1115             self.shelve_branch()  1116   1117         # Maintain a branch for the else clause.  1118   1119         self.new_branch(node.else_ or NullBranch())  1120         if node.else_ is not None:  1121             self.dispatch(node.else_)  1122         self.shelve_branch()  1123   1124         self.merge_branches()  1125   1126     def visitIfExp(self, node):  1127         self.use_name("__bool__", node)  1128         self.new_branchpoint()  1129   1130         # Propagate attribute usage to branches.  1131   1132         self.dispatch(node.test)  1133   1134         self.new_branch(node.then)  1135         self.dispatch(node.then)  1136         self.shelve_branch()  1137   1138         self.new_branch(node.else_)  1139         self.dispatch(node.else_)  1140         self.shelve_branch()  1141   1142         self.merge_branches()  1143         return make_instance() # either outcome is possible  1144   1145     def visitImport(self, node):  1146         for name, alias in node.names:  1147             if alias is not None:  1148                 module = self.importer.load(name, 1, importer=node) or UnresolvedName(None, name, self)  1149                 self.store(alias, module)  1150             else:  1151                 module = self.importer.load(name, importer=node) or UnresolvedName(None, name.split(".")[0], self)  1152                 self.store(name.split(".")[0], module)  1153   1154             circular = module and not module.loaded  1155   1156             # For circular imports, invalidate attribute usage for all exported  1157             # names of modules.  1158   1159             if module and not module.loaded:  1160                 for n in module.keys():  1161                     module.modify_name(n)  1162   1163     visitInvert = _visitUnary  1164   1165     def visitKeyword(self, node):  1166         self.dispatch(node.expr)  1167         self._visitConst(node.name)  1168         self.keyword_names.add(node.name)  1169   1170     def visitLambda(self, node):  1171         fn = self._visitFunction(node, None)  1172         self.use_specific_attribute(None, fn.name)  1173         return fn  1174   1175     visitLeftShift = _visitBinary  1176   1177     def visitList(self, node):  1178         self.use_specific_attribute("__builtins__", "list")  1179         return self.OP(node)  1180   1181     def visitListComp(self, node):  1182   1183         # Note that explicit dispatch is performed.  1184   1185         if node.quals:  1186             self.visitListCompFor(node.quals[0], node.quals[1:], node.expr)  1187         return make_instance()  1188   1189     def visitListCompFor(self, node, following_quals, expr):  1190         self.new_branchpoint()  1191   1192         # Declare names which will be used by generated code.  1193   1194         self.use_name("__iter__", node.list)  1195         self.use_name("next")  1196   1197         in_loop = self.in_loop  1198         self.in_loop = 1  1199         self.dispatch(node.list)  1200   1201         # NOTE: Could generate AST nodes for the actual operations instead of  1202         # NOTE: manually generating code in micropython.ast.  1203   1204         self.expr = make_instance() # each element is a result of a function call  1205         self.dispatch(node.assign)  1206   1207         # Enter the loop.  1208         # Propagate attribute usage to branches.  1209   1210         self.new_branch(node)  1211   1212         # Note that explicit dispatch is performed.  1213   1214         if node.ifs:  1215             self.visitListCompIf(node.ifs[0], node.ifs[1:], following_quals, expr)  1216         elif following_quals:  1217             self.visitListCompFor(following_quals[0], following_quals[1:], expr)  1218         else:  1219             self.dispatch(expr)  1220   1221         self.shelve_branch()  1222         self.in_loop = in_loop  1223   1224         self.merge_branches()  1225   1226     def visitListCompIf(self, node, following_ifs, following_quals, expr):  1227         self.use_name("__bool__", node)  1228         self.new_branchpoint()  1229   1230         # Propagate attribute usage to branches.  1231   1232         self.dispatch(node.test)  1233   1234         # Note that explicit dispatch is performed.  1235   1236         if following_ifs:  1237             self.visitListCompIf(following_ifs[0], following_ifs[1:], following_quals, expr)  1238         elif following_quals:  1239             self.visitListCompFor(following_quals[0], following_quals[1:], expr)  1240         else:  1241             self.new_branch(expr)  1242             self.dispatch(expr)  1243             self.shelve_branch()  1244   1245             # Maintain a branch for the else clause.  1246   1247             self.new_branch(NullBranch())  1248             self.shelve_branch()  1249   1250         self.merge_branches()  1251   1252     visitMod = _visitBinary  1253   1254     def visitModule(self, node):  1255   1256         # Make a back reference from the node for code generation.  1257   1258         node.unit = self  1259         return self.dispatch(node.node)  1260   1261     visitMul = _visitBinary  1262   1263     def visitName(self, node):  1264         return self.get_namespace().get_using_node(node.name, node) or make_instance()  1265   1266     def visitNot(self, node):  1267         self.use_name("__bool__", node)  1268         self.dispatch(node.expr)  1269         return make_instance()  1270   1271     visitOr = TEST_OP  1272   1273     visitPass = NOP  1274   1275     visitPower = _visitBinary  1276   1277     def _visitPrint(self, node, function_name):  1278         self.NOP(node)  1279         self.use_specific_attribute("__builtins__", function_name)  1280   1281     def visitPrint(self, node):  1282         self._visitPrint(node, "_print")  1283   1284     def visitPrintnl(self, node):  1285         self._visitPrint(node, "_printnl")  1286   1287     visitRaise = NOP_ABANDON  1288   1289     visitReturn = NOP_ABANDON  1290   1291     visitRightShift = _visitBinary  1292   1293     def visitSlice(self, node):  1294         return self._visitOperator(node, self.in_assignment and "AssSlice" or "Slice")  1295   1296     visitSliceobj = OP  1297   1298     def visitStmt(self, node):  1299         for n in node.nodes:  1300             self.dispatch(n)  1301   1302     visitSub = _visitBinary  1303   1304     def visitSubscript(self, node):  1305         return self._visitOperator(node, self.in_assignment and "AssSubscript" or "Subscript")  1306   1307     def visitTryExcept(self, node):  1308         self.new_branchpoint()  1309         self.dispatch(node.body)  1310   1311         for name, var, n in node.handlers:  1312             if name is not None:  1313                 self.dispatch(name)  1314   1315             self.new_branch(n)  1316   1317             # Any abandoned branches from the body can now be resumed.  1318   1319             self.resume_abandoned_branches()  1320   1321             # Establish the local for the handler.  1322   1323             if var is not None:  1324                 self.dispatch(var)  1325             if n is not None:  1326                 self.dispatch(n)  1327   1328             self.shelve_branch()  1329   1330         # The else clause maintains the usage from the body but without the  1331         # abandoned branches since they would never lead to the else clause  1332         # being executed.  1333   1334         self.new_branch(node.else_ or NullBranch())  1335         if node.else_ is not None:  1336             self.dispatch(node.else_)  1337         self.shelve_branch()  1338   1339         self.merge_branches()  1340   1341     visitTryFinally = NOP  1342   1343     visitTuple = OP  1344   1345     visitUnaryAdd = _visitUnary  1346   1347     visitUnarySub = _visitUnary  1348   1349     def visitWhile(self, node):  1350         self.use_name("__bool__", node)  1351         self.new_branchpoint(node)  1352   1353         # Propagate attribute usage to branches.  1354   1355         in_loop = self.in_loop  1356         self.in_loop = 1  1357   1358         # The test is evaluated initially and again in the loop.  1359   1360         self.dispatch(node.test)  1361   1362         self.new_branch(node)  1363         self.dispatch(node.body)  1364   1365         self.resume_continuing_branches()  1366   1367         self.dispatch(node.test)  1368         self.shelve_branch()  1369   1370         self.in_loop = in_loop  1371   1372         # A null branch is used to record a path around the loop.  1373   1374         self.new_branch(node.else_ or NullBranch())  1375         self.shelve_branch()  1376   1377         self.merge_branches()  1378   1379         # The else clause is evaluated outside any branch.  1380   1381         if node.else_ is not None:  1382             self.dispatch(node.else_)  1383   1384         # Any suspended branches from the loop can now be resumed.  1385   1386         self.resume_broken_branches()  1387   1388     visitWith = NOP  1389   1390     visitYield = NOP  1391   1392 # vim: tabstop=4 expandtab shiftwidth=4