85 Commits

Author SHA1 Message Date
Justin Viiret
9cf66b6ac9 util: switch from Boost to std::unordered set/map
This commit replaces the ue2::unordered_{set,map} types with their STL
versions, with some new hashing utilities in util/hash.h. The new types
ue2_unordered_set<T> and ue2_unordered_map<Key, T> default to using the
ue2_hasher.

The header util/ue2_containers.h has been removed, and the flat_set/map
containers moved to util/flat_containers.h.
2017-08-21 11:14:55 +10:00
Wang, Xiang W
86c5f7feb1 FDR: Squash buckets of included literals in FDR confirm
- Change the compile of literal matchers to two passes.
 - Reverse the bucket assignment in FDR, bucket with longer literals has
   smaller bucket id.
 - Squash the buckets of included literals and jump to the the program of
   included literals directly from parent literal program without going
   through FDR confirm for included iterals.
2017-08-21 11:12:36 +10:00
Wang, Xiang W
67a8f43355 literal matchers: change context passed to callback to scratch 2017-08-21 11:12:36 +10:00
Wang, Xiang W
ebb1b0006b remove start argument in literal matcher callbacks 2017-08-21 11:12:36 +10:00
Justin Viiret
1d041b12b7 shufti/truffle tests: silence ubsan warning
The ubsan support in clang warned about us accessing idx-1 of an array here.
2017-08-21 11:12:16 +10:00
Matthew Barr
dba2470ec9 msvc: use the vectorcall calling convention
This requires declaring external interfaces with the cdecl
calling convention.
2017-06-09 10:12:02 +10:00
Matthew Barr
fb3a03dc69 Disable part of unit test on FreeBSD 10 2017-05-30 14:00:45 +10:00
Matthew Barr
3e345c2567 If we can shift by an immediate, do it. Otherwise, don't. 2017-05-30 14:00:45 +10:00
Matthew Barr
91db20d8eb avx512: CPU detection and platform hints 2017-05-30 13:59:23 +10:00
Matthew Barr
8a56d16d57 avx512: add basic functions to simd_utils
Extends the m512 type to use avx512 and also changes required
for limex.
2017-05-30 13:59:18 +10:00
Matthew Barr
423569ec82 De-multiaccel 2017-05-30 13:59:00 +10:00
Matthew Barr
c510b85bf1 whitespace changes 2017-05-30 13:59:00 +10:00
Matthew Barr
0b8f25a036 Ensure simd types are aligned to what we assume
Turns out Clang is sensitive to the location of the alignment attribute.
2017-05-30 13:48:38 +10:00
Justin Viiret
e8162960fc unit: use bytecode_ptr instead of aligned_unique_ptr 2017-04-26 15:19:36 +10:00
Justin Viiret
a197074c5d nfa: switch to using bytecode_ptr<NFA> 2017-04-26 15:19:36 +10:00
Justin Viiret
cf82924a39 depth: make constructor explicit 2017-04-26 15:19:19 +10:00
Alex Coyte
37cb93e60f rose_build: reduce size/scope of context objects 2017-04-26 15:19:01 +10:00
Matthew Barr
8201183138 Check compiler architecture flags in one place 2017-04-26 15:18:26 +10:00
Alex Coyte
0cbec2c1c3 mmbBuildInitRangePlan: correct offset if initial block is not block 0 2017-04-26 15:18:13 +10:00
Justin Viiret
560e522457 ng_calc_components: add Grey box control 2017-04-26 15:18:13 +10:00
Justin Viiret
ba867ebaff ng_calc_components: rework to move graphs
Rather than cloning graphs for output, rework calc components so that it
moves them.
2017-04-26 15:18:13 +10:00
Justin Viiret
5dfae12a62 ng: split NGWrapper into NGHolder, ExpressionInfo
We now use NGHolder for all graph information, while other expression
properties (report, flag information, etc) go in new class
ExpressionInfo.
2017-04-26 15:18:09 +10:00
Justin Viiret
7ca81ff530 ng_find_matches: limit how big we're willing to go
Tests which require tracking more than 15K states (including edit
distance states) are very, very slow.
2017-04-26 15:16:22 +10:00
Anatoly Burakov
9f72dede5c Add support for approximate matching in NFA matcher unit tests 2017-04-26 15:11:54 +10:00
Justin Viiret
f520599ab7 flat_map: add value_comp() 2017-04-26 14:41:30 +10:00
Justin Viiret
e37fdb240a flat_set/map: add hash_value tests 2017-04-26 14:41:30 +10:00
Justin Viiret
07a6b6510c rose/hwlm: limit literals to eight bytes
Rework HWLM to work over literals of eight bytes ("medium length"),
doing confirm in the Rose interpreter.
2017-04-26 14:41:29 +10:00
Matthew Barr
46f74300f2 update copyright year 2017-01-17 11:38:09 +11:00
Matthew Barr
981e9896b4 MSVC gets confused by one line of the graph tests 2017-01-06 11:24:16 +11:00
Matthew Barr
2214296b7f Convert compile-time code to not require SIMD 2016-12-14 15:29:01 +11:00
Justin Viiret
e271781d95 multibit, fatbit: make _size build-time only
This commit makes mmbit_size() and fatbit_size compile-time only, and
adds a resource limit for very large multibits.
2016-12-14 15:28:54 +11:00
Alex Coyte
e51b6d23b9 introduce Sheng-McClellan hybrid 2016-12-14 15:27:18 +11:00
Alex Coyte
f605ca0dc1 check for partial matches at end of dverm 2016-12-02 11:34:04 +11:00
Alex Coyte
e1e9010cac Introduce custom adjacency-list based graph 2016-12-02 11:31:33 +11:00
Alex Coyte
445cf987a8 remove unused includes 2016-10-28 14:52:56 +11:00
Justin Viiret
68bf473e2e fdr: move long literal handling into Rose
Move the hash table used for long literal support in streaming mode from
FDR to Rose, and introduce new instructions CHECK_LONG_LIT and
CHECK_LONG_LIT_NOCASE for doing literal confirm for long literals.

This simplifies FDR confirm, and guarantees that HWLM matchers will only
be used for literals < 256 bytes long.
2016-10-28 14:52:26 +11:00
Matthew Barr
c3b5efefb6 Add short avx2 shufti form 2016-10-28 14:46:31 +11:00
Alex Coyte
a08e1dd690 Introduce a 64-bit LimEx model.
On 64-bit platforms, the Limex 64 model is implemented in normal GPRs.
On 32-bit platforms, however, 128-bit SSE registers are used for the
runtime implementation.
2016-10-28 14:44:12 +11:00
Xu, Chi
b96d5c23d1 rose: add new instruction CHECK_MASK_32
This is a specialisation of the "lookaround" code.
2016-10-28 14:43:33 +11:00
Alex Coyte
702f256b39 remove exceptionMap from LimExNFA 2016-08-10 15:11:15 +10:00
Matthew Barr
cbd115f7fe Don't shadow names 2016-08-10 15:06:57 +10:00
Justin Viiret
4ce268af47 ng: ensure that only match states have reports
Ensure (and assert) that vertices without an edge to {accept, acceptEod}
do not have reports set.
2016-08-10 15:05:23 +10:00
Justin Viiret
9eb349a343 rose: expose smwr builder, tidy up engine build 2016-08-10 14:59:10 +10:00
Xu, Chi
4d7469392d rose: add CHECK_BYTE/CHECK_MASK instructions
These instructions are specialisations of the "lookaround" code for
performance.
2016-08-10 14:57:48 +10:00
Alex Coyte
691b08d170 use NGHolder::foo in favour of NFAGraph::foo 2016-08-10 14:52:56 +10:00
Matthew Barr
e3d416a6ea Apply some consistency to the names we give shifts 2016-07-08 11:07:50 +10:00
Matthew Barr
4d6934fc77 Move limex specific shuffle utils and ssse3 funcs 2016-07-08 11:07:50 +10:00
Justin Viiret
cf9e40ae1c nfa: unify NfaCallback and SomNfaCallback
Use just one callback type, with both start and end offsets.
2016-07-08 11:01:56 +10:00
Kirill Rybalchenko
9d2403e8bb limex: implement variable shift NFA engines
Replaces the old LimEx NFA engines, which were specialised for model
size and number of shifts, with a new set of engines that can handle a
variable number of shifts.
2016-07-08 10:57:29 +10:00
Justin Viiret
ea62ba107f unit: make multiaccel tests deterministic
Use a PRNG, not rand().
2016-07-08 10:47:33 +10:00