Unit testing of individual parsers

Spicy comes with tools which given a Spicy grammar can read input from stdin or file and forward to a parser (any public unit in the grammar):

  • spicy-dump additionally prints structured rendering of parsed units
  • spicy-driver runs parser, outputs any print statements
  • use -f to read from file instead of stdin

Often spicy-dump is less intrusive since it requires not grammar changes.

Example

Given a grammar

module foo;
public type X = unit {
    a: uint8;
    b: bytes &until=b"\x00";
};

To parse the input b"\x00foo\x00" to this parser we could feed it data with Bash's printf builtin.

$ printf '\x01foo\x00' | spicy-dump -d foo.spicy
foo::X {
  a: 1
  b: foo
}

If using e.g., BTest we can snapshots this output to make sure it stays the same, e.g.,

# @TEST-EXEC: printf '\x01foo\x00' | spicy-dump -d foo.spicy >output-foo 2>&1
# @TEST-EXEC: btest-diff output-foo

Hint

The default BTest setup generated by the package template sets the environment variable DIST to the root directory of the analyzer.

# ...
[environment]
DIST=%(testbase)s/..
# ...

Use this variable to access the original grammar in tests, e.g.,

# @TEST-EXEC: spicyc -d "${DIST}/analyzer/foo.spicy" -o foo.hlto

General tips

  • often Spicy grammars compile faster in debug mode -d, default to this in tests for faster turnaround
  • make sure to not accidentally append unintended newlines to input, e.g., use echo -n instead of plain echo
  • Bash's printf builtin can be used to create binary data
  • select parser by passing -p module::Unit if the grammar contains multiple entry points

Avoiding repeated analyzer recompilations

Since above spicy-dump invocation needs to compiles the full parser, consider compiling once to an HLTO file and reusing it for multiple checks in the same test, e.g.,

## Compile grammar.
# @TEST-EXEC: spicyc -d "${DIST}/analyzer/foo.spicy" -o foo.hlto
#
## Run tests.
# @TEST-EXEC: printf '\x01foo\x00' | spicy-dump -d foo.hlto >>output-foo 2>&1
# @TEST-EXEC: printf '\x02bar\x00' | spicy-dump -d foo.hlto >>output-foo 2>&1

Adding additional code for test

We might want to add additional code for testing only, e.g., add additional logging, or check state with assert or assert-exception.

We can add testing-only module to the compilation during test, e.g.,

# @TEST-EXEC: spicyc -dj "${DIST}/analyzer/foo.spicy" %INPUT -o http.hlto
#
# @TEST-EXEC: printf '\x01foo\x00' | spicy-dump -d foo.hlto >>output-foo 2>&1
# @TEST-EXEC: printf '\x02bar\x00' | spicy-dump -d foo.hlto >>output-foo 2>&1
#
# @TEST-EXEC: btest-diff output

module test;

import foo;

on foo::Something { print self; }

Since one can implement hooks even for non-public units this is pretty powerful; e.g., we can use this technique to observe data in anonymous fields,

module foo;

public type TcpMessages = unit {
    : Message[]; # Anonymous field since list is unbounded.
};

type Message = unit {
    # Consume and parse input ...
};

# In test code print individual message.

module test;

import foo;

on foo::Message { print self; }