58 Commits

Author SHA1 Message Date
Felipe Zimmerle
1efd5e460d
Constify Transaction on variable resolution 2020-11-27 23:25:08 -03:00
Felipe Zimmerle
cc699bc794
Reduce the workload on VariableValue
Last compute at the last minute, if needed.
2020-11-27 23:25:08 -03:00
Felipe Zimmerle
a73e08f9ba
Cosmetics: Using VariableValues instead of std::vector<...>
Making the code more readable.
2020-11-27 23:25:07 -03:00
WGH
3a72237d25
Use std::shared_ptr for variable resolution
AnchoredSetVariable::resolve is called for every rule
(see RuleWithOperator::evaluate). The previous implementation allocated
a new copy of every variable, which quickly added up. In my tests,
AnchoredSetVariable::resolve function consumed 7.8% of run time.

AnchoredSetVariable (which is a multimap) values are never changed,
only added. This means it's safe to store them in std::shared_ptr,
and make resolve return shared_ptr pointing to the same object.

Other resolve implementation could also use this optimization by not
allocating new objects, however, they are not hot spots, so this
optimization was not implemented there.

In my benchmark, this raises performance from 117 requests per second to
131 RPS, and overhead is lowered from 7.8% to 2.4%.

As a bonus, replacing plain pointer with smart pointers make code
cleaner, since using smart pointers makes manual deletes no longer necessary.

Additionally, VariableOrigin is now stored in plain std::vector,
since it's wasteful to store structure containing just two integer
values using std::list<std::unique_ptr<T>>.
2020-11-27 23:20:04 -03:00
Felipe Zimmerle
910a1877a9
Makes regular expression selection on collections key case insensitive
This issue was initially reported by @michaelgranzow-avi on #2296.

@airween made an initial attempt to provide a fixed at #2107; As a
consequence of the pull request review - provided by @victorhora,
@zimmerle, and @michaelgranzow-avi - @airween made a second attempt
at #2297. After reviewing by @martinhsv, @zimmerle, I have absorbed
the essential pieces from @airween patch into this one.

This patch differs from @airween's because @airween's patches were
partially working: Key exclusions with regex weren't covered, same
for anchored variables (e.g. ARGS). During the review, I have
highlighted the importance of having elementary test cases. A simple
test case on ARGS could spot the issue. Since that is an important
fix, I don't want to hold this for one more review cycle; therefore,
I am committing the fix myself.

Thank you all involved in the solution of this very own issue.
2020-11-25 09:11:05 -03:00
Felipe Zimmerle
357c140003
Changens copyright year 2020-01-31 10:32:37 -03:00
Felipe Zimmerle
fe98ce4c7d
Cosmetics: address cppcheck warnings 2020-01-30 18:19:34 -03:00
Felipe Zimmerle
86a5f471a9
Cosmetics: fixed static analysis issues. 2020-01-15 20:35:59 -03:00
Felipe Zimmerle
4e76c6adf0
Renames namespace Variables to variables 2019-03-06 15:53:20 -03:00
WGH
37cf60b8d2
Fix use of deleted Regex copy constructor in LMDB code
Bug introduced in ad28de4f. Fixes #2008.
2019-01-28 16:20:02 -03:00
WGH
ad28de4f14 Refactor regex code
This commit fixes quite a few odd things in regex code:
 * Lack of encapsulation.
 * Non-method functions for matching without retrieving all groups.
 * Regex class being copyable without proper copy-constructor (potential UAF
   and double free due to pointer members m_pc and m_pce).
 * Redundant SMatch::m_length, which always equals to match.size() anyway.
 * Weird SMatch::size_ member which is initialized only by one of the three matching
   functions, and equals to the return value of that function anyways.
 * Several places in code having std::string value instead of reference.
2019-01-18 10:34:01 -03:00
Felipe Zimmerle
ee50fea266
Handling key exceptions on the variable itself
This is the first step towords to solve #1697
2018-09-24 16:16:30 -03:00
michaelgranzow-avi
d810de9166
#1818: Variable names must match fully, not partially; also revert to hash table lookup instead of linear search; add test case 2018-06-26 10:47:03 -03:00
Felipe Zimmerle
e51297b436
Improvements on top of #1787 2018-06-12 15:43:08 -03:00
Ervin Hegedus
edb5993d5f
Fixed LMDB collection errors 2018-06-12 14:47:44 -03:00
Felipe Zimmerle
892beb5360
Refactoring on {global,ip,resources,session,tx,user} collections
Now using the same name schema and interface for these "special"
collection.

Fix: #1754, #1778
2018-05-29 23:48:05 -03:00
Andrei Belov
8285a97460
Fix memory leak in Collections
This closes #1729.
2018-04-05 09:48:51 -03:00
Hegedüs Ervin
8d61a3df90
Fix LMDB compile error 2018-02-28 08:52:40 -03:00
Felipe Zimmerle
eeec7efb68
Renames collection::Variable to VariableValue 2018-02-20 13:40:01 -03:00
Felipe Zimmerle
de7c5c89bb
Using shared var for variables names 2018-02-20 13:40:01 -03:00
Felipe Zimmerle
6f7fdd9493
Using direct variable access instead m_collections 2018-02-20 13:40:01 -03:00
Felipe Zimmerle
082a3e3287
Adds support to SecWebAppID 2017-11-08 09:33:14 -03:00
Felipe Zimmerle
2a5085255e
Using multiple threads in reading logs via rule message example 2017-06-03 16:40:47 -03:00
Felipe Zimmerle
8fbb9e8128
Using pthreads to avoid concurrent access to the collection 2017-06-03 16:07:35 -03:00
Felipe Zimmerle
36ab4b952f
Fix lmdb compilation issue 2017-03-06 15:02:00 -03:00
Felipe Zimmerle
e95efa05cc
Fix assorted memory and static analysis errors 2017-03-06 15:02:00 -03:00
Felipe Zimmerle
f2d149fc5f
Extends the direct access model to other collections 2017-03-06 15:02:00 -03:00
Felipe Zimmerle
59114dd598
Refactoring on the operators parsers (2/2)
This is the first step towards remove the memory leaks in the parser
2017-03-06 15:01:50 -03:00
Felipe Zimmerle
e181cb7e0a
Fix lmdb collections backend 2016-12-30 16:43:56 -03:00
Felipe Zimmerle
a7f465cf3a
Avoids string copy by working with pointers while resolving variables 2016-12-28 20:00:14 -03:00
Felipe Zimmerle
168fa22e19
Collections cleanup: removes resolveFirstCopy method 2016-12-28 19:56:27 -03:00
Felipe Zimmerle
6fff8c954a
Performance improvement: makes the collections lookup faster 2016-12-28 19:55:35 -03:00
Felipe Zimmerle
2244e874e2
Moves static methods from class String to the namespace string 2016-11-04 16:00:44 -03:00
Felipe Zimmerle
62a0cb468b
Renames utils/msc_string.[h|cc] to utils/string.[h|cc] 2016-11-04 16:00:42 -03:00
Felipe Zimmerle
4ced1d18e0
Using full path in the header inclusion 2016-11-04 14:45:01 -03:00
Felipe Zimmerle
507ec44cc2
Refactoring on `utils.cc' and adjacents
Completely removed the `utils.cc' by moving residual functions into
sub-classes of `utils/'
2016-11-03 20:26:27 -03:00
Felipe Zimmerle
73c4d69174
Moves string related functions from utils' to utils/string' 2016-11-03 10:47:22 -03:00
Robert Paprocki
049f1abb62
Fix compilation error
lmdb.cc fails to compile following commit c680ddf.
2016-10-07 19:07:14 -03:00
Felipe Zimmerle
ecd3fd0dc1
build: avoids compilation problems due to non existence of the lmdb.h
Based on: 56abe98cb8c791812d46c0902b4e742c8c39620e by @phantom-az
2016-09-22 10:28:45 -03:00
Felipe Zimmerle
a1a1c71d6b
Makes LMDB support optional 2016-09-13 09:51:03 -03:00
Felipe Zimmerle
f723870f18
Fix case sensitive variable resolution in in memory backend
Variables are case insensitive
2016-07-22 13:34:57 -03:00
Felipe Zimmerle
5d64f73817
Makes RULE collection to be resolved inside a macro expansion 2016-07-21 13:09:22 -03:00
Felipe Zimmerle
4cf6c714ac
Cosmetics: Fix coding style 2016-07-12 21:59:17 -03:00
Felipe Zimmerle
4078677b7f
Cosmetic changes: applies changes suggested by static analysis 2016-07-12 00:46:12 -03:00
Felipe Zimmerle
3f38b56682
Renames testdb' to modsec-shared-collections' 2016-07-11 14:00:33 -03:00
Felipe Zimmerle
ad481be09e
lmdb backend: Adds support to select variables using regex 2016-07-11 11:00:06 -03:00
Felipe Zimmerle
de8245d8f9
in-memory backend: Adds support to select variables using regex 2016-07-11 10:59:43 -03:00
Felipe Zimmerle
833089eb70
Adds method resolveFirstCopy to collections
Using the copy whenever it is necessary to avoid memory leak.
2016-07-08 10:22:37 -03:00
Felipe Zimmerle
6e4226ee4d
Adds support to global collections shared among different process
There is a memory leak in the variable resolution that should be
contained by an internal change in the way that the variables
are resolved.
2016-07-07 23:03:47 -03:00
Felipe Zimmerle
5daf4873b5
build: Searching for LMDB during the configuration phase 2016-07-05 11:56:19 -03:00