Design compile time api hs_compile_lit() and hs_compile_lit_multi()
to handle pure literal pattern sets. Corresponding option --literal-on
is added for hyperscan testing suites. Extended parameters and part of
flags are not supported for this api.
This commit replaces the ue2::unordered_{set,map} types with their STL
versions, with some new hashing utilities in util/hash.h. The new types
ue2_unordered_set<T> and ue2_unordered_map<Key, T> default to using the
ue2_hasher.
The header util/ue2_containers.h has been removed, and the flat_set/map
containers moved to util/flat_containers.h.
At stream boundaries, we can mark streams as exhausted if there are no
groups active and there are no other ways to report matches. This allows us
to stop maintaining the history buffer on subsequent stream writes.
Previously, streams were only marked as exhausted if a pure highlander case
reported all patterns or the outfix in a sole outfix case died.
Move the hash table used for long literal support in streaming mode from
FDR to Rose, and introduce new instructions CHECK_LONG_LIT and
CHECK_LONG_LIT_NOCASE for doing literal confirm for long literals.
This simplifies FDR confirm, and guarantees that HWLM matchers will only
be used for literals < 256 bytes long.
Add the new Tamarama engine that acts as a container for infix/suffix
engines that can be proven to run exclusively of one another.
This reduces stream state for pattern sets with many exclusive engines.
Unifies all literal match paths so that the Rose program is used for all
of them. This removes the previous specialised "direct report" and
"multi direct report" paths. Some additional REPORT instruction work was
necessary for this.
Reworked literal construction path at compile time in prep for using
program offsets as literal IDs.
Completely removed the anchored log runtime, which is no longer worth
the extra complexity.
Replace the RoseLiteral structure with more program instructions; now,
instead of each literal ID leading to a RoseLiteral, it simply has a
program to run (and a delay rebuild program).
This commit also makes some other improvements:
* CHECK_STATE instruction, for use instead of a sparse iterator over a
single element.
* Elide some checks (CHECK_LIT_EARLY, ANCHORED_DELAY, etc) when not
needed.
* Flatten PUSH_DELAYED behaviour to one instruction per delayed
literal, rather than the mask/index-list approach used before.
* Simple program cache at compile time for deduplication.
