Testing

Whether we like it or not, a Spicy analyzer is a piece of software. To make sure that it currently and it the future operates like intended we should strive to add tests to

  • encode current behavior
  • make sure our changes have the intended effects
  • provide living documentation

When operating a Spicy analyzer as part of Zeek, work tends to fall into somewhat defined layers:

  • parsing of raw bytes with one or more Spicy units
  • interfacing with Zeek, e.g.,
    • mapping of Spicy hooks to Zeek events in EVT files
    • type mappings in EVT file
    • hooks calling functions from the zeek module (template-suggested file analyzer/zeek_ANALYZER.spicy)
  • Zeek scripts consuming Spicy-generated events to create logs

Maintainable tests to cover a mix of

  • single low-level pieces of functionality in isolation (unit tests),
  • integration of different components (integration tests), and
  • end-to-end tests to validate the full stack (system tests)

where lower levels tend to be more exhaustive.