540 Commits

Author SHA1 Message Date
Matthew Barr
f626276271 hsbench: add Hyperscan benchmarker
The hsbench tool provides an easy way to measure Hyperscan's
performance for a particular set of patterns and corpus of data
to be scanned.
2016-12-14 15:26:01 +11:00
Justin Viiret
06cde4c94d ng_literal_analysis: use ue2_graph
This reduces compile time ~10% on a number of large cases.
2016-12-02 11:35:56 +11:00
Justin Viiret
32af5fa794 mcclellan: fix printf format specifiers 2016-12-02 11:35:52 +11:00
Alex Coyte
106667e24e refactor mcclellan runtime
1. use u32 to hold the state during runtime to prevent repeated zero extension
2. have a tight small loop for processing characters which breaks when
   something interesting happens
2016-12-02 11:35:47 +11:00
Justin Viiret
ef99ae108f rose_build_merge: correctly merge NFA outfixes
We were not doing our bookkeeping properly for merges where the number
of NFAs was greater than the batch size of 200.
2016-12-02 11:34:52 +11:00
Alex Coyte
eb7759361d Rework the creation of startDs edges in fillHolderForLockCheck().
This prevents clearing of edges created for virtual starts in late regions.
2016-12-02 11:34:41 +11:00
Alex Coyte
8b94eb86f8 precise dverm fix for 5a32993: handle sign extension correctly 2016-12-02 11:34:20 +11:00
Alex Coyte
f605ca0dc1 check for partial matches at end of dverm 2016-12-02 11:34:04 +11:00
Justin Viiret
16aa22a361 gough: don't dump int data unless it's requested 2016-12-02 11:33:54 +11:00
Matthew Barr
5a842caaf1 shufti: slightly faster short shufti operation
It is better to shift the high lane values in an XMM
before then using insert to combine the high and low lanes.
2016-12-02 11:33:51 +11:00
Matthew Barr
99e14df117 Fix combine2x128 2016-12-02 11:33:48 +11:00
Alex Coyte
8ff7a3cdbb correct dump filenames of som rev engines 2016-12-02 11:33:44 +11:00
Alex Coyte
32c826e9c6 have single dump function per engine 2016-12-02 11:32:36 +11:00
Alex Coyte
71ff480b77 nfa_api: remove subtype from dispatch 2016-12-02 11:32:28 +11:00
Alex Coyte
1614c73eeb Implement some ue2_graph functions using TMP rather than friends
This helps work around issues with some compilers
2016-12-02 11:32:24 +11:00
Alex Coyte
530d84c6f3 allow edge_descriptors to be created from pair<edge_descriptor, bool> 2016-12-02 11:32:20 +11:00
Alex Coyte
e1e9010cac Introduce custom adjacency-list based graph 2016-12-02 11:31:33 +11:00
Alex Coyte
05683655cb remove unused define and old inlining controls 2016-12-02 11:30:27 +11:00
Alex Coyte
2341fe7baa use stable_sort in analysis from 47f53f6; missed review comment 2016-12-02 11:29:30 +11:00
Justin Viiret
29472c7b71 rose_dump: remove stray newline 2016-12-02 11:28:20 +11:00
Justin Viiret
91a7ce1cda getData256(): data needs to be 32-byte aligned 2016-12-02 11:28:16 +11:00
Justin Viiret
054749f9ee smallwrite: minimize DFAs if they have been pruned 2016-12-02 11:28:12 +11:00
Justin Viiret
c67a361080 smallwrite: prune overlong nfa graphs early 2016-12-02 11:26:37 +11:00
Justin Viiret
21a1b47637 ng_limex: add edges in deterministic ordering 2016-12-02 11:26:33 +11:00
Justin Viiret
e108cb841f ng_restructuring: wire start to tops in idx order 2016-12-02 11:25:00 +11:00
Justin Viiret
1a24b0b4db ng_equivalence: don't use ptr_vector
Switch over ptr_vector<T> to vector<unique_ptr<T>>. This works around
some issues we were seeing with MSVC builds, where the contents of the
ptr_vector were being destroyed when it was returned.
2016-12-02 11:24:57 +11:00
Alex Coyte
779bebfd12 fix for analysis in previous commit
Properly distinguish between vertices that will get set after the cyclic
from vertices that may get set alongside the cyclic
2016-12-02 11:24:47 +11:00
Alex Coyte
47f53f63a7 simple pass to pick up paths redundant with those from cyclic's succs 2016-12-02 11:24:30 +11:00
Justin Viiret
8cadba0bdd rose: call loadLongLiteralState() earlier
The ll_buf, ll_buf_nocase buffers must be initialised before anyh path
that could lead to storeLongLiteralState().
2016-12-02 11:23:14 +11:00
Alex Coyte
924089d95e properly consider report behaviour when finding start verts 2016-12-02 11:22:56 +11:00
Justin Viiret
b9650d4fd0 rose: don't unconditionally init ll_buf etc
This is only necessary (and already always done) if there is a long
literal table.
2016-12-02 11:22:27 +11:00
Alex Coyte
592ce06eeb Create combo tops for trigger limexes 2016-12-02 11:22:23 +11:00
Matthew Barr
be8bd41ec4 Merge master into develop 2016-12-02 11:21:29 +11:00
Matthew Barr
0a16e6f78c Bump version number for release v4.3.2 2016-11-15 10:09:59 +11:00
Matthew Barr
c2fcc93fc9 Update changelog for 4.3.2 2016-11-15 10:09:59 +11:00
Matthew Barr
21b44fcbc3 Add the missing degree() in Boost 1.62
This is a workaround that fixes issue #39
2016-11-15 09:52:37 +11:00
Alex Coyte
445cf987a8 remove unused includes 2016-10-28 14:52:56 +11:00
Justin Viiret
f7cc8a618d fdr: reduce confirm size to a u8
Also removes the flexible array member from the LitInfo structure.
2016-10-28 14:52:52 +11:00
Justin Viiret
98c791dc6e noodle: correct history req calculation 2016-10-28 14:52:48 +11:00
Justin Viiret
8869dee643 rose: simplify long lit table, add bloom filter
Replaces the original long lit hash table (used in streaming mode) with a
smaller, simpler linear probing approach. Adds a bloom filter in front
of it to reduce time spent on false positives.

Sizing of both the hash table and bloom filter are done based on max
load.
2016-10-28 14:52:45 +11:00
Justin Viiret
68bf473e2e fdr: move long literal handling into Rose
Move the hash table used for long literal support in streaming mode from
FDR to Rose, and introduce new instructions CHECK_LONG_LIT and
CHECK_LONG_LIT_NOCASE for doing literal confirm for long literals.

This simplifies FDR confirm, and guarantees that HWLM matchers will only
be used for literals < 256 bytes long.
2016-10-28 14:52:26 +11:00
Matthew Barr
6ed30194ce Disable asserts everywhere when we say so 2016-10-28 14:52:15 +11:00
Alex Coyte
648a3c4824 UE-3025: There is no need to prune tops from non-triggered graphs 2016-10-28 14:52:01 +11:00
Matthew Barr
7d3eff8648 extern "C" for mask1bit table 2016-10-28 14:51:49 +11:00
Alex Coyte
c94899dd44 allow sets of tops on edges 2016-10-28 14:51:46 +11:00
Alex Coyte
aca89e66d2 hinted insert operations for flat_set 2016-10-28 14:48:55 +11:00
Matthew Barr
707fe675ea Operator precedence matters 2016-10-28 14:48:46 +11:00
Matthew Barr
7849b9d611 MSVC prefers the attrib at the beginning 2016-10-28 14:48:31 +11:00
Matthew Barr
2e5a2ab2a9 cmake: don't be so heavyhanded with flags 2016-10-28 14:47:11 +11:00
Justin Viiret
6e533589bb rose: move END instruction to start of enum
Stop overloading END as the last Rose interpreter instruction, use new
sentinel LAST_ROSE_INSTRUCTION for that.

This change will also make it easier to add new instructions without
renumbering END and thus changing all generated bytecodes.
2016-10-28 14:47:07 +11:00