120 Commits

Author SHA1 Message Date
Justin Viiret
9cf66b6ac9 util: switch from Boost to std::unordered set/map
This commit replaces the ue2::unordered_{set,map} types with their STL
versions, with some new hashing utilities in util/hash.h. The new types
ue2_unordered_set<T> and ue2_unordered_map<Key, T> default to using the
ue2_hasher.

The header util/ue2_containers.h has been removed, and the flat_set/map
containers moved to util/flat_containers.h.
2017-08-21 11:14:55 +10:00
Wang, Xiang W
86c5f7feb1 FDR: Squash buckets of included literals in FDR confirm
- Change the compile of literal matchers to two passes.
 - Reverse the bucket assignment in FDR, bucket with longer literals has
   smaller bucket id.
 - Squash the buckets of included literals and jump to the the program of
   included literals directly from parent literal program without going
   through FDR confirm for included iterals.
2017-08-21 11:12:36 +10:00
Wang, Xiang W
67a8f43355 literal matchers: change context passed to callback to scratch 2017-08-21 11:12:36 +10:00
Wang, Xiang W
ebb1b0006b remove start argument in literal matcher callbacks 2017-08-21 11:12:36 +10:00
Justin Viiret
1d041b12b7 shufti/truffle tests: silence ubsan warning
The ubsan support in clang warned about us accessing idx-1 of an array here.
2017-08-21 11:12:16 +10:00
Justin Viiret
3bd0c7f6ad unit-hyperscan: pure-literal/smwr coverage 2017-08-21 11:09:35 +10:00
Alex Coyte
d317d75615 character classes: handle \Q\E and utf8 2017-06-21 08:43:44 +10:00
Matthew Barr
dba2470ec9 msvc: use the vectorcall calling convention
This requires declaring external interfaces with the cdecl
calling convention.
2017-06-09 10:12:02 +10:00
Matthew Barr
fb3a03dc69 Disable part of unit test on FreeBSD 10 2017-05-30 14:00:45 +10:00
Matthew Barr
3e345c2567 If we can shift by an immediate, do it. Otherwise, don't. 2017-05-30 14:00:45 +10:00
Matthew Barr
91db20d8eb avx512: CPU detection and platform hints 2017-05-30 13:59:23 +10:00
Matthew Barr
8a56d16d57 avx512: add basic functions to simd_utils
Extends the m512 type to use avx512 and also changes required
for limex.
2017-05-30 13:59:18 +10:00
Matthew Barr
423569ec82 De-multiaccel 2017-05-30 13:59:00 +10:00
Matthew Barr
c510b85bf1 whitespace changes 2017-05-30 13:59:00 +10:00
Matthew Barr
0b8f25a036 Ensure simd types are aligned to what we assume
Turns out Clang is sensitive to the location of the alignment attribute.
2017-05-30 13:48:38 +10:00
Justin Viiret
e8162960fc unit: use bytecode_ptr instead of aligned_unique_ptr 2017-04-26 15:19:36 +10:00
Justin Viiret
a197074c5d nfa: switch to using bytecode_ptr<NFA> 2017-04-26 15:19:36 +10:00
Justin Viiret
cf82924a39 depth: make constructor explicit 2017-04-26 15:19:19 +10:00
Alex Coyte
37cb93e60f rose_build: reduce size/scope of context objects 2017-04-26 15:19:01 +10:00
Matthew Barr
8201183138 Check compiler architecture flags in one place 2017-04-26 15:18:26 +10:00
Justin Viiret
a4df49dd66 unit: better output from expr_info unit tests 2017-04-26 15:18:25 +10:00
Justin Viiret
a871f70c25 ng_extparam: split up work and do per-comp reduce
This change breaks extparam processing up into:

 - propagateExtendedParams: propagates min_length, min_offset and
   max_offset into the reports on the graph
 - reduceExtendedParams: runs graph reductions based on extparams

Then, we apply the reduce pass to the whole graph, and later as well to
each component after calc_components.
2017-04-26 15:18:22 +10:00
Alex Coyte
0cbec2c1c3 mmbBuildInitRangePlan: correct offset if initial block is not block 0 2017-04-26 15:18:13 +10:00
Justin Viiret
560e522457 ng_calc_components: add Grey box control 2017-04-26 15:18:13 +10:00
Justin Viiret
ba867ebaff ng_calc_components: rework to move graphs
Rather than cloning graphs for output, rework calc components so that it
moves them.
2017-04-26 15:18:13 +10:00
Justin Viiret
5dfae12a62 ng: split NGWrapper into NGHolder, ExpressionInfo
We now use NGHolder for all graph information, while other expression
properties (report, flag information, etc) go in new class
ExpressionInfo.
2017-04-26 15:18:09 +10:00
Justin Viiret
d8eb259ac7 serialize: tidy 2017-04-26 15:17:03 +10:00
Justin Viiret
bc7da2807a unit: modernise test_util 2017-04-26 15:17:03 +10:00
Justin Viiret
1376f3849a serialize: parameterize on pattern as well 2017-04-26 15:17:03 +10:00
Justin Viiret
a97ec56aee serialize: add vectored mode 2017-04-26 15:17:03 +10:00
Justin Viiret
7ca81ff530 ng_find_matches: limit how big we're willing to go
Tests which require tracking more than 15K states (including edit
distance states) are very, very slow.
2017-04-26 15:16:22 +10:00
Justin Viiret
5edecbf539 ng: check can_never_match before validate_fuzzy 2017-04-26 15:16:03 +10:00
Anatoly Burakov
9f72dede5c Add support for approximate matching in NFA matcher unit tests 2017-04-26 15:11:54 +10:00
Anatoly Burakov
2de6706df2 Adding support for compiling approximate matching patterns
Adds new "edit_distance" extparam
2017-04-26 15:11:39 +10:00
Justin Viiret
1245156f44 parser: handle "control verbs" without close paren 2017-04-26 14:59:02 +10:00
Justin Viiret
bef6889844 parser: use control_verb parser inline 2017-04-26 14:58:43 +10:00
Justin Viiret
1875d55cf1 parser: add initial parser for control verbs
This more reliably handles control verbs like (*UTF8) that can only
happen at the start of the pattern, and allows them in any ordering.
2017-04-26 14:57:46 +10:00
Alex Coyte
fbaa0a1b25 make expected too large patterns even larger 2017-04-26 14:44:49 +10:00
Justin Viiret
f520599ab7 flat_map: add value_comp() 2017-04-26 14:41:30 +10:00
Justin Viiret
e37fdb240a flat_set/map: add hash_value tests 2017-04-26 14:41:30 +10:00
Justin Viiret
07a6b6510c rose/hwlm: limit literals to eight bytes
Rework HWLM to work over literals of eight bytes ("medium length"),
doing confirm in the Rose interpreter.
2017-04-26 14:41:29 +10:00
Alex Coyte
8f8ba2d591 basic unit test to check correct stream state is being used. 2017-03-01 13:05:10 +11:00
Matthew Barr
46f74300f2 update copyright year 2017-01-17 11:38:09 +11:00
Matthew Barr
981e9896b4 MSVC gets confused by one line of the graph tests 2017-01-06 11:24:16 +11:00
Matthew Barr
0d0e1a5106 api: hs_valid_platform 2016-12-14 15:35:13 +11:00
Matthew Barr
c3a73446ee Fat runtime 2016-12-14 15:34:54 +11:00
Matthew Barr
2214296b7f Convert compile-time code to not require SIMD 2016-12-14 15:29:01 +11:00
Justin Viiret
e271781d95 multibit, fatbit: make _size build-time only
This commit makes mmbit_size() and fatbit_size compile-time only, and
adds a resource limit for very large multibits.
2016-12-14 15:28:54 +11:00
Alex Coyte
e51b6d23b9 introduce Sheng-McClellan hybrid 2016-12-14 15:27:18 +11:00
Alex Coyte
f605ca0dc1 check for partial matches at end of dverm 2016-12-02 11:34:04 +11:00