Multiple changes since last release, this will be the last 100% ABI and
API compatible with Hyperscan release.
Next versions will include major refactors and API extensions, it will
be mostly backwards compatible however.
Without particular order, platform support is now:
* Linux (x86, Arm, Power)
* FreeBSD 14 (x86, Arm, Power)
* MacOS 14+ (x86, Arm)
In total more than 200 configurations in the CI are tested for every PR.
Other features:
- Fat Runtime supported for Arm as well (ASIMD/SVE/SVE2).
- Initial implementations for Arm SVE/SVE2 algorithms added, thanks to
Yoan Picchi from Arm.
- SIMDe support added, used as an alternative backend for existing
platforms, but mostly interesting for allowing Vectorscan to build in
new platforms without a supported SIMD engine.
- Various speedups and optimizations.
- Cppcheck and clang-tidy fixes throughout the code, both have been
added to CI for multiple configurations, but only cppcheck triggers a
build failure for now.
Various bugfixes, most important listed:
- Speed up truffle with 256b TBL instructions (#290)
- Fix Clang Tidy warnings (#295)
- Clang 17+ is more restrictive on rebind<T> on MacOS/Boost, remove
warning (#332)
- partial_load_u64 will fail if buf == NULL/c_len == 0 (#331)
- Bugfix/fix avx512vbmi regressions (#335)
- fix missing hs_version.h header (closes#198)
- hs_valid_platform: Fix check for SSE4.2 (#310)
- Fixed out of bounds read in AVX512VBMI version of fdr_exec_fat_teddy …
(#333)
- Fix noodle SVE2 off by one bug (#313)
- Make vectorscan accept \0 starting pattern (#312)
- Fix 5.4.11's config step regression (#327)
- Fix double shufti's vector end false positive (#325)
Prevents overwriting GNUCC_ARCH with an empty value when parsing output
of gcc -Q --help=target. Ensures robustness if detection fails and
returns an empty string.
Signed-off-by: Ibrahim Kashif <ibrahim.kashif@arm.com>
* Add regression test for double shufti
It tests for false positive at vector edges.
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
* Fix double shufti reporting false positives
Double shufti used to offset one vector, resulting in losing one character
at the end of every vector. This was replaced by a magic value indicating a
match. This meant that if the first char of a pattern fell on the last char of
a vector, double shufti would assume the second character is present and
report a match.
This patch fixes it by keeping the previous vector and feeding its data to the
new one when we shift it, preventing any loss of data.
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
* vshl() will call the correct implementation
* implement missing vshr_512_imm(), simplifies caller x86 code
* Fix x86 case, use alignr instead
* it's the reverse, the avx512 alignr is incorrect, need to fix
* Make shufti's OR reduce size agnostic
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
* Fix test's array size
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
* Fix AVX2/AVX512 alignr implementations and unit tests
* Fix Power VSX alignr
---------
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
Co-authored-by: Konstantinos Margaritis <konstantinos@vectorcamp.gr>
* added static libraries in cmake to fix unit-internal seg fault in freebsd, ppc64le, gcc13 error
* Moved gcc13 flags for freebsd-gcc13 in cmake/cflags-ppc64le.make
Multiple AVX512VBMI-related fixes:
src/nfa/mcsheng_compile.cpp: No need for an assert here, impl_id can be set to 0
src/nfa/nfa_api_queue.h: Make sure this compiles on both C++ and C
src/nfagraph/ng_fuzzy.cpp: Fix compilation error when DEBUG_OUTPUT=on
src/runtime.c: Fix crash when data == NULL
unit/internal/sheng.cpp: Unit test has to enable AVX512VBMI manually as autodetection does not get trigger, this causes test to fail
src/fdr/teddy_fat.cpp: AVX512 loads need to be 64-bit aligned, caused a crash on clang-18
Fixed out of bounds read in AVX512VBMI version of fdr_exec_fat_teddy (#322)
* Replaced the 32 byte read with a properly truncated mapped read
* Added a unit test
Co-authored-by: Rafał Dowgird <rafal.dowgird@rtbhouse.com>
An old commit (24ae1670d) had the side effect of moving cmake defines after
they were being used. This patch move them back to be defined before being used.
Speed hsbench back up by ~ 0.8%
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
Revert the code that produced the regression error in #317
Add the regression error to a unit test regressions.cpp along with the rebar tests
---------
Co-authored-by: gtsoul-tech <gtsoulkanakis@gmail.com>
* Revert "Fix noodle SVE2 off by one bug"
This patch was fixing the bug when it happens at the end of the buffer
but it wasn't fixing it when we do scanDoubleOnce before the main loop
The next patch fix this bug for both case instead
This reverts commit 48dd0e5ff0.
* Fix noodle spurious match with \0 chars for SVE2
When sve2's noodle process a non full vector (before the main loop or
at the end of it), a fake \0 was being parsed, trigerring a match for
pattern that ended with \0. This patch fix this.
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
---------
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
Vectorscan requires SSE4.2 as a minimum on x86_64. For Hyperscan this
used to be SSSE3.
Applications that use the library call hs_valid_platform() to check if
the CPU fulfils this minimum requirement. However, when Vectorscan
upgraded to SSE4.2, the check was not updated. This leads to the library
trying to execute instructions that are not supported, resulting in the
application to crash.
This might not have been noticed as the CPUs that do not support SSE4.2
are rather old and unlikely to run any load where performance is an
issue. However, I believe that the library should not let the
application crash.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Vectorscan used to reject such pattern because they were being compared
to "" and found to be an empty string. We now check the pattern length
instead.
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
By using svmatch on 16 bit lanes with a 8 bit predicate, we end up
including an undefined character in the pattern checks. The inactive
lane after load contains an undefined value, usually \0. Patterns
using \0 as the last character would then match this spurious
character, returning a match beyond the buffer's end. The fix checks
for such matches and rejects them.
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
* maybe fix the hsbench issue (check_ssse3 again) in sse2/simde env
* fix the last failing unit test with fat
---------
Co-authored-by: G.E. <gregory.economou@vectorcamp.gr>
This allows the use of SIMDE library to emulate SSSE3/SSE4.2 instructions on SSE2-only (x86-64-v2) hardware.
---------
Co-authored-by: G.E <gregory.economou@vectorcamp.gr>
Co-authored-by: Konstantinos Margaritis <konstantinos@vectorcamp.gr>
Fixes some of the clang-tidy warnings
clang-analyzer-deadcode.DeadStores
clang-analyzer-cplusplus.NewDelete
clang-analyzer-core.uninitialized.UndefReturn
closes some:#253
ignored in this pr:
/usr/include/boost/smart_ptr/detail/shared_count.hpp:432:24
/usr/include/boost/smart_ptr/detail/shared_count.hpp:443:24
51 in build/src/parser
gtest ones
src/fdr/teddy_compile.cpp:600:5 refactoring on way
src/fdr/fdr_compile.cpp:209:5 refactoring on way
256b wide SVE vectors allow some simplification of truffle. Up to 40%
speedup on graviton3. Going from 12500 MB/s to 17000 MB/s onhe
microbenchmark.
SVE2 also offer this capability for 128b vector with a speedup around
25% compared to normal SVE
Add unit tests and benchmark for this wide variant
256b wide SVE vectors allow some simplification of truffle.
Up to 40% speedup on graviton3. Going from 12500 MB/s to 17000 MB/s
onhe microbenchmark.
SVE2 also offer this capability for 128b vector with a speedup around
25% compared to normal SVE
Add unit tests and benchmark for this wide variant
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
