Transformers & Visitors

Transformers & Visitors provide a convenient interface to process the parse-trees that Lark returns.

They are used by inheriting from the correct class (visitor or transformer), and implementing methods corresponding to the rule you wish to process. Each method accepts the children as an argument. That can be modified using the v_args decorator, which allows one to inline the arguments (akin to *args), or add the tree meta property as an argument.

See: visitors.py

Visitor

Visitors visit each node of the tree, and run the appropriate method on it according to the node’s data.

They work bottom-up, starting with the leaves and ending at the root of the tree.

There are two classes that implement the visitor interface:

  • Visitor: Visit every node (without recursion)
  • Visitor_Recursive: Visit every node using recursion. Slightly faster.
Example:
class IncreaseAllNumbers(Visitor):
    def number(self, tree):
        assert tree.data == "number"
        tree.children[0] += 1

IncreaseAllNumbers().visit(parse_tree)
class lark.visitors.Visitor

Tree visitor, non-recursive (can handle huge trees).

Visiting a node calls its methods (provided by the user via inheritance) according to tree.data

visit(tree)

Visits the tree, starting with the leaves and finally the root (bottom-up)

visit_topdown(tree)

Visit the tree, starting at the root, and ending at the leaves (top-down)

__default__(tree)

Default function that is called if there is no attribute matching tree.data

Can be overridden. Defaults to doing nothing.

class lark.visitors.Visitor_Recursive

Bottom-up visitor, recursive.

Visiting a node calls its methods (provided by the user via inheritance) according to tree.data

Slightly faster than the non-recursive version.

visit(tree)

Visits the tree, starting with the leaves and finally the root (bottom-up)

visit_topdown(tree)

Visit the tree, starting at the root, and ending at the leaves (top-down)

__default__(tree)

Default function that is called if there is no attribute matching tree.data

Can be overridden. Defaults to doing nothing.

Interpreter

class lark.visitors.Interpreter

Interpreter walks the tree starting at the root.

Visits the tree, starting with the root and finally the leaves (top-down)

For each tree node, it calls its methods (provided by user via inheritance) according to tree.data.

Unlike Transformer and Visitor, the Interpreter doesn’t automatically visit its sub-branches. The user has to explicitly call visit, visit_children, or use the @visit_children_decor. This allows the user to implement branching and loops.

Example:
class IncreaseSomeOfTheNumbers(Interpreter):
    def number(self, tree):
        tree.children[0] += 1

    def skip(self, tree):
        # skip this subtree. don't change any number node inside it.
        pass

    IncreaseSomeOfTheNumbers().visit(parse_tree)

Transformer

class lark.visitors.Transformer(visit_tokens=True)

Transformers visit each node of the tree, and run the appropriate method on it according to the node’s data.

Calls its methods (provided by the user via inheritance) according to tree.data. The returned value replaces the old one in the structure.

They work bottom-up (or depth-first), starting with the leaves and ending at the root of the tree. Transformers can be used to implement map & reduce patterns. Because nodes are reduced from leaf to root, at any point the callbacks may assume the children have already been transformed (if applicable).

Transformer can do anything Visitor can do, but because it reconstructs the tree, it is slightly less efficient. It can be used to implement map or reduce patterns.

All these classes implement the transformer interface:

  • Transformer - Recursively transforms the tree. This is the one you probably want.
  • Transformer_InPlace - Non-recursive. Changes the tree in-place instead of returning new instances
  • Transformer_InPlaceRecursive - Recursive. Changes the tree in-place instead of returning new instances
Parameters:visit_tokens (bool, optional) – Should the transformer visit tokens in addition to rules. Setting this to False is slightly faster. Defaults to True. (For processing ignored tokens, use the lexer_callbacks options)

NOTE: A transformer without methods essentially performs a non-memoized deepcopy.

transform(tree)

Transform the given tree, and return the final result

__mul__(other)

Chain two transformers together, returning a new transformer.

__default__(data, children, meta)

Default function that is called if there is no attribute matching data

Can be overridden. Defaults to creating a new copy of the tree node (i.e. return Tree(data, children, meta))

__default_token__(token)

Default function that is called if there is no attribute matching token.type

Can be overridden. Defaults to returning the token as-is.

Example:
from lark import Tree, Transformer

class EvalExpressions(Transformer):
    def expr(self, args):
            return eval(args[0])

t = Tree('a', [Tree('expr', ['1+2'])])
print(EvalExpressions().transform( t ))

# Prints: Tree(a, [3])
Example:
class T(Transformer):
    INT = int
    NUMBER = float
    def NAME(self, name):
        return lookup_dict.get(name, name)

T(visit_tokens=True).transform(tree)
class lark.visitors.Transformer_NonRecursive(visit_tokens=True)

Same as Transformer but non-recursive.

Like Transformer, it doesn’t change the original tree.

Useful for huge trees.

class lark.visitors.Transformer_InPlace(visit_tokens=True)

Same as Transformer, but non-recursive, and changes the tree in-place instead of returning new instances

Useful for huge trees. Conservative in memory.

class lark.visitors.Transformer_InPlaceRecursive(visit_tokens=True)

Same as Transformer, recursive, but changes the tree in-place instead of returning new instances

v_args

lark.visitors.v_args(inline=False, meta=False, tree=False, wrapper=None)

A convenience decorator factory for modifying the behavior of user-supplied visitor methods.

By default, callback methods of transformers/visitors accept one argument - a list of the node’s children.

v_args can modify this behavior. When used on a transformer/visitor class definition, it applies to all the callback methods inside it.

v_args can be applied to a single method, or to an entire class. When applied to both, the options given to the method take precedence.

Parameters:
  • inline (bool, optional) – Children are provided as *args instead of a list argument (not recommended for very long lists).
  • meta (bool, optional) – Provides two arguments: children and meta (instead of just the first)
  • tree (bool, optional) – Provides the entire tree as the argument, instead of the children.
  • wrapper (function, optional) – Provide a function to decorate all methods.

Example

@v_args(inline=True)
class SolveArith(Transformer):
    def add(self, left, right):
        return left + right


class ReverseNotation(Transformer_InPlace):
    @v_args(tree=True)
    def tree_node(self, tree):
        tree.children = tree.children[::-1]

Discard

class lark.visitors.Discard

When raising the Discard exception in a transformer callback, that node is discarded and won’t appear in the parent.