Transformers & Visitors
Transformers & Visitors provide a convenient interface to process the parse-trees that Lark returns.
They are used by inheriting from the correct class (visitor or transformer), and implementing methods corresponding to the rule you wish to process. Each method accepts the children as an argument. That can be modified using the
v_args decorator, which allows to inline the arguments (akin to
*args), or add the tree
meta property as an argument.
Visitors visit each node of the tree, and run the appropriate method on it according to the node's data.
They work bottom-up, starting with the leaves and ending at the root of the tree.
class IncreaseAllNumbers(Visitor): def number(self, tree): assert tree.data == "number" tree.children += 1 IncreaseAllNumbers().visit(parse_tree)
There are two classes that implement the visitor interface:
Visitor - Visit every node (without recursion)
Visitor_Recursive - Visit every node using recursion. Slightly faster.
The interpreter walks the tree starting at the root (top-down).
For each node, it calls the method corresponding with its
Unlike Transformer and Visitor, the Interpreter doesn't automatically visit its sub-branches.
The user has to explicitly call
visit_children, or use the
This allows the user to implement branching and loops.
class IncreaseSomeOfTheNumbers(Interpreter): def number(self, tree): tree.children += 1 def skip(self, tree): # skip this subtree. don't change any number node inside it. pass IncreaseSomeOfTheNumbers().visit(parse_tree)
Transformers visit each node of the tree, and run the appropriate method on it according to the node's data.
They work bottom-up (or: depth-first), starting with the leaves and ending at the root of the tree.
Transformers can be used to implement map & reduce patterns.
Because nodes are reduced from leaf to root, at any point the callbacks may assume the children have already been transformed (if applicable).
Transformers can be chained into a new transformer by using multiplication.
Transformer can do anything
Visitor can do, but because it reconstructs the tree, it is slightly less efficient.
from lark import Tree, Transformer class EvalExpressions(Transformer): def expr(self, args): return eval(args) t = Tree('a', [Tree('expr', ['1+2'])]) print(EvalExpressions().transform( t )) # Prints: Tree(a, )
All these classes implement the transformer interface:
- Transformer - Recursively transforms the tree. This is the one you probably want.
- Transformer_InPlace - Non-recursive. Changes the tree in-place instead of returning new instances
- Transformer_InPlaceRecursive - Recursive. Changes the tree in-place instead of returning new instances
By default, transformers only visit rules.
visit_tokens=True will tell Transformer to visit tokens as well. This is a slightly slower alternative to
lexer_callbacks, but it's easier to maintain and works for all algorithms (even when there isn't a lexer).
class T(Transformer): INT = int NUMBER = float def NAME(self, name): return lookup_dict.get(name, name) T(visit_tokens=True).transform(tree)
v_args is a decorator.
By default, callback methods of transformers/visitors accept one argument: a list of the node's children.
v_args can modify this behavior.
When used on a transformer/visitor class definition, it applies to all the callback methods inside it.
v_args accepts one of three flags:
inline- Children are provided as
*argsinstead of a list argument (not recommended for very long lists).
meta- Provides two arguments:
meta(instead of just the first)
tree- Provides the entire tree as the argument, instead of the children.
@v_args(inline=True) class SolveArith(Transformer): def add(self, left, right): return left + right class ReverseNotation(Transformer_InPlace): @v_args(tree=True) def tree_node(self, tree): tree.children = tree.children[::-1]
These are the functions that are called on if a function with a corresponding name has not been found.
__default__method has the signature
(data, children, meta), with
databeing the data attribute of the node. It defaults to reconstruct the Tree
__default_token__just takes the
Tokenas an argument. It defaults to just return the argument.
When raising the
Discard exception in a transformer callback, that node is discarded and won't appear in the parent.