175 Commits

Author SHA1 Message Date
Hong, Yang A
23e5f06594 add new Literal API for pure literal expressions:
Design compile time api hs_compile_lit() and hs_compile_lit_multi()
to handle pure literal pattern sets. Corresponding option --literal-on
is added for hyperscan testing suites. Extended parameters and part of
flags are not supported for this api.
2019-08-13 14:51:38 +08:00
Derrick Lyndon Pallas
5a1b02bc10 Fix uninitialized use of scatter_unit_uX due to padding
These non-packed structures are placed into a std::vector.  Later, they
contents of the vector are memcpy'd and the CRC of this space is taken.
Some compilers will zero the struct padding but GCC8.2 with -O2 at least
will not.  This means that the CRC is based on uninitialized memory.

Since it is expected that these bytes will be memcpy'd, zero in place once
they're in the std::vector.

Found by Valgrind.

Q.v. Issue #148
2019-08-13 14:49:15 +08:00
Hong, Yang A
f68723a606 literal matching: separate path for pure literal patterns 2019-01-21 09:59:22 +08:00
Lu, Qi
5a0885d235 Windows porting: port hyperscan and chimera tools to windows. 2018-07-09 11:40:43 -04:00
Wang, Xiang W
bf87f8c003 chimera: hybrid of Hyperscan and PCRE 2018-07-09 11:30:35 -04:00
Chang, Harry
8a1c497f44 Logical Combination of patterns. 2018-06-27 14:04:57 +08:00
Justin Viiret
f877f14641 ue2_graph: more direct property_graph defn 2018-06-27 13:40:26 +08:00
Justin Viiret
c7c90c7ab7 graph_undirected: adapt bidi graph to undirected
Introduces an adaptor (like the BGL's reverse_graph) that presents an
undirected view of a bidirectional graph.

Initially used in ng_calc_components.
2018-06-27 13:40:10 +08:00
Justin Viiret
ce7cfbde82 misc: docs, typo fixes, small cleanups 2018-06-27 13:39:05 +08:00
Justin Viiret
08bf909e2b ng_violet: make calcSplitRatio operation faster
Implements count_reachable in a less malloc-happy way, improving
compile performance. Adds a count() function to small_color_map.
2018-06-27 13:38:55 +08:00
Matthew Barr
50ea83cb26 Move cpuid checks inline.
During fat runtime function resolution there was a chance that the PLT
might not be initialised in time for us to call the cpuid check
functions. Moving them inline means there is no PLT to worry about.

Fixes #78
2018-01-19 06:32:18 -05:00
Justin Viiret
ba0bf0c991 rose_build_add_mask: improve findMaskLiteral perf 2017-09-18 13:29:34 +10:00
Justin Viiret
7fe53fec10 partitioned_set: use lower_bound(), not scan 2017-09-18 13:29:34 +10:00
Justin Viiret
012b347284 ue2_literal: define npos in ue2string.cpp 2017-09-18 13:29:33 +10:00
Justin Viiret
fe31b387e8 hash: use std::hash for string hashing 2017-09-18 13:26:18 +10:00
Justin Viiret
25170b32eb ue2_literal: better hash function 2017-09-18 13:26:18 +10:00
Justin Viiret
1aad3b0ed1 ue2_literal: make nocase member a dynamic_bitset
We were previously using vector<bool>, but dynamic_bitset provides a
faster any() impl
2017-09-18 13:26:18 +10:00
Justin Viiret
3ff70d5568 insertion_ordered_{map,set}: add new containers
These are associative map/set structures that are iterable in insertion
order.
2017-08-21 11:25:21 +10:00
Hong, Yang A
205a5bc98f multibit compression support 2017-08-21 11:19:11 +10:00
Justin Viiret
14cf5c3684 small_vector: require boost >= 1.61
We use the small_vector constructors introduced in Boost 1.61 (trac bug
11866, github commit b436c91). If the Boost version is too old, we fall
back to using std::vector.
2017-08-21 11:18:43 +10:00
Chang, Harry
8da2d13baa AVX512 Reinforced FAT teddy. 2017-08-21 11:14:59 +10:00
Justin Viiret
4528485a56 determinise: use find first, rather than emplace
For non-trivial StateSet types, copying to do the emplace if it is
already in the map is more expensive than checking with find() first.
2017-08-21 11:14:59 +10:00
Justin Viiret
33823d60d1 tidy: "ue2::flat_set/map" -> "flat_set/map" 2017-08-21 11:14:59 +10:00
Justin Viiret
9cf66b6ac9 util: switch from Boost to std::unordered set/map
This commit replaces the ue2::unordered_{set,map} types with their STL
versions, with some new hashing utilities in util/hash.h. The new types
ue2_unordered_set<T> and ue2_unordered_map<Key, T> default to using the
ue2_hasher.

The header util/ue2_containers.h has been removed, and the flat_set/map
containers moved to util/flat_containers.h.
2017-08-21 11:14:55 +10:00
Justin Viiret
a425bb9b7c ue2_graph: move descriptors out of graph struct 2017-08-21 11:12:36 +10:00
Chang, Harry
68e08d8e18 AVX512 reinforced teddy. 2017-08-21 11:12:36 +10:00
Justin Viiret
a1ff4d3293 small_color_map: add efficient 2-bit color map 2017-08-21 11:12:26 +10:00
Justin Viiret
f762fb9af6 dump_util: richer StdioFile type 2017-08-21 11:10:11 +10:00
Justin Viiret
4edf1e4195 dump: move openStdioFile() to util/dump_util.h 2017-08-21 11:10:11 +10:00
Justin Viiret
b09e3acd04 ng_misc_opt: improve performance in large cases 2017-08-21 11:10:11 +10:00
Justin Viiret
c83f2ea389 rose_build_matchers: be more careful w/ mixed-case
Overhaul the way fragment literals are added to HWLM and accel, fix
some bugs shaken out by stricter mask use.
2017-08-21 11:09:45 +10:00
Justin Viiret
85f7790a21 dfa: standardise 'using' instead of typedef 2017-08-21 10:40:18 +10:00
Justin Viiret
31141dd35b determinise: use queue, improve api
- Use a queue rather than always building the full vector of state
   sets.
 - Make more use of move, emplace, reserve.
 - Write directly into dstates argument.
 - Return bool rather than int.
2017-08-21 10:40:18 +10:00
Matthew Barr
345897f096 cpuid: exclude AVX512 flag when target not enabled
If we have a fat runtime build without AVX512 enabled, we should not
pick up the AVX512 flag from cpuid.
2017-07-26 15:51:23 +10:00
Justin Viiret
09f5699df7 flat_set/map: workaround for gcc-4.8 C++11 defect
The STL shipped with gcc-4.8 does not provide
vector::erase(const_iterator) for C++11, instead only taking a mutable
iterator.  This causes problems with flat_set/map if we don't have
Boost small_vector available and we fall back to std::vector.

We work around this by providing a function to construct a mutable
iterator given a const_iterator for internal use.
2017-06-13 13:52:33 +10:00
Matthew Barr
3e345c2567 If we can shift by an immediate, do it. Otherwise, don't. 2017-05-30 14:00:45 +10:00
Matthew Barr
f6b688fc06 rename pshufb to pshufb_m128 2017-05-30 13:59:23 +10:00
Matthew Barr
a295c96198 rename vpshufb to pshufb_m256 2017-05-30 13:59:23 +10:00
Matthew Barr
ec7869711f avx512 fat runtime support: experimental 2017-05-30 13:59:23 +10:00
Matthew Barr
91db20d8eb avx512: CPU detection and platform hints 2017-05-30 13:59:23 +10:00
Matthew Barr
8a56d16d57 avx512: add basic functions to simd_utils
Extends the m512 type to use avx512 and also changes required
for limex.
2017-05-30 13:59:18 +10:00
Matthew Barr
fedd48489f Allow the full cpuid flags for fat runtimes 2017-05-30 13:59:00 +10:00
Matthew Barr
c510b85bf1 whitespace changes 2017-05-30 13:59:00 +10:00
Justin Viiret
1a04d1330e verify_types: add type static assertions 2017-05-30 13:59:00 +10:00
Justin Viiret
16a00074c6 verify_types: throw on failure
In release builds, we would like a verify_u32 (etc) failure to be more
than just an assertion.
2017-05-30 13:58:59 +10:00
Justin Viiret
8a7ac432c0 ue2_literal: add hash_value() 2017-05-30 13:58:59 +10:00
Justin Viiret
9258592d0b report_manager: more use of unordered_map 2017-05-30 13:58:32 +10:00
Justin Viiret
1878b9a857 report_manager: use unordered externalIdMap 2017-05-30 13:58:32 +10:00
Justin Viiret
c9be18c7e2 accel_dfa_build_strat: use small_vector for paths 2017-05-30 13:49:49 +10:00
Matthew Barr
0b8f25a036 Ensure simd types are aligned to what we assume
Turns out Clang is sensitive to the location of the alignment attribute.
2017-05-30 13:48:38 +10:00