3652 Commits

Author SHA1 Message Date
Ervin Hegedus
9148668571
Merge pull request #3231 from eduar-hte/remove-copies-transformations
Remove unnecessary heap allocated copies in Transformation actions
2024-08-28 14:33:59 +02:00
Ervin Hegedus
4951702d45
Merge pull request #3233 from eduar-hte/remove-copies-pm-operator
Removed multiple heap-allocated copies in Pm::init & parse_pm_content
2024-08-28 13:31:02 +02:00
Eduardo Arias
3e9d8107a8 Removed multiple heap-allocated copies in parse_pm_content
- The previous version of this function was doing three strdup copies
  to parse the pm content. The updated version only copies the value
  once (in order not to modify the Operator's m_param member variable),
  and then performs the updates inline.
- Binary parsing was broken because digits were not compared as
  characters.
  - Fail parsing when an invalid hex character is found.
- Error message in parse_pm_content would reference freed memory if
  accessed by caller. Removed anyway because it was unused.
2024-08-27 10:43:07 -03:00
Eduardo Arias
a6d64bf615 Replaced VALID_HEX, ISODIGIT & NBSP macros in string.h
- Moved them into modsecurity::utils::string to avoid polluting the
  global namespace.
2024-08-27 10:00:54 -03:00
Eduardo Arias
2f5dac5c4c Simplified initialization of Transformation's action_kind
- Some of the Transformation classes would initialize their Action's
  action_kind using the default (using Transformation constructor
  without an action_kind parameter).
- Others, however, would use that constructor and initialize action_kind
  manually in their constructor, but setting the default value
  (RunTimeBeforeMatchAttemptKind = 1), which was redundant.
- Removed unused Transformation constructor to specify action_kind.
- Converted Action::Kind into an 'enum class' to require using the enum
  constants (instead of integer values, which are difficult to track in
  the codebase and change)
2024-08-27 10:00:54 -03:00
Eduardo Arias
7023c0a8b4 Refactored sha1 & md5 utils to share implementation and reduce code duplication. 2024-08-27 10:00:54 -03:00
Eduardo Arias
fedec96a7e Refactored base64 utils to share implementation and reduce code duplication. 2024-08-27 10:00:38 -03:00
Eduardo Arias
34da8eeeee Pass RuleWithActions::executeTransformation arguments by reference
- This function already expects these arguments not to be null pointers,
  doesn't validate them and just dereference them.
- In order to make this explicit and enforced by the compiler, they're
  now passed as references.
2024-08-27 10:00:38 -03:00
Eduardo Arias
b647dbd905 Remove unnecessary heap-allocation & copy in Transaction::extractArguments
- utils::urldecode_nonstrict_inplace decodes inplace so key & value,
  which are values returned by utils::string::ssplit_pair can be
  just be modified and do not need to be copied.
- Updated signature of utils::urldecode_nonstrict_inplace, as its
  two callers already have std::string values.
2024-08-27 10:00:18 -03:00
Eduardo Arias
021d0caa33 Perform NormalisePath & NormalisePathWin transformations in-place 2024-08-27 10:00:18 -03:00
Eduardo Arias
2c3c228725 Perform Utf8ToUnicode transformation in-place
- Removed inplace helper function from the class, as it's only
  referenced by the implementation.
2024-08-27 10:00:18 -03:00
Eduardo Arias
17a2cbd164 Perform UrlDecodeUni & UrlDecode transformations in-place
- Use std::string in UrlEncode transformation, instead of manually
  memory management. This avoids an additional copy after completing
  encoding by just swapping the encoded value and the input.
- Removed inplace helper function from the class, as it's only
  referenced by the implementation.
2024-08-27 10:00:18 -03:00
Eduardo Arias
8bf4d96e6b Perform HtmlEntityDecode transformation in-place
- Removed inplace helper function from the class, as it's only
  referenced by the implementation.
2024-08-27 10:00:18 -03:00
Eduardo Arias
7d5c9faa43 Perform JsDecode transformation in-place
- Removed inplace helper function from the class, as it's only
  referenced by the implementation.
2024-08-27 10:00:18 -03:00
Eduardo Arias
a520369da0 Perform EscapeSeqDecode transformation in-place
- Removed ansi_c_sequences_decode_inplace helper function from the
  class, as it's only referenced by the implementation.
2024-08-27 10:00:18 -03:00
Eduardo Arias
727f2bf840 Perform CssDecode transformation in-place
- Removed inplace helper function from the class, as it's only
  referenced by the implementation.
2024-08-27 10:00:17 -03:00
Eduardo Arias
e687140d05 Perform HexDecode transformation in-place
- Removed inplace helper function from the class, as it's only
  referenced by the implementation.
2024-08-27 10:00:17 -03:00
Eduardo Arias
4670710376 Perform LowerCase & UpperCase transformations in-place
- Refactored to share implementation and reduce code duplication.
2024-08-27 10:00:17 -03:00
Eduardo Arias
fd8a979463 Perform SqlHexDecode transformation in-place
- Validate buffer size before accessing data. The previous
  implementation would only check that there was a character available
  in the buffer but could continue processing/reading characters from
  an hex representation without checking bounds.
- Removed inplace & mytolower helper functions from the class, as
  they're only referenced by the implementation.
- Removed duplicate VALID_HEX & ISODIGIT macros, already in
  src/utils/string.h.
2024-08-27 10:00:17 -03:00
Eduardo Arias
2915ee60e2 Perform Trim, TrimLeft & TrimRight transformations in-place 2024-08-27 10:00:17 -03:00
Eduardo Arias
74d150c068 Perform RemoveCommentsChar, RemoveComments & ReplaceComments transformations in-place 2024-08-27 10:00:17 -03:00
Eduardo Arias
da775eca81 Perform ReplaceNulls transformation in-place 2024-08-27 10:00:17 -03:00
Eduardo Arias
1505025990 Perform RemoveNulls & RemoveWhitespace transformations in-place
- Refactored to share implementation.
2024-08-27 10:00:17 -03:00
Eduardo Arias
1236d9a7cd Perform CompressWhitespace transformation in-place 2024-08-27 10:00:17 -03:00
Eduardo Arias
13203ae5e7 Perform CmdLine transformation in-place 2024-08-27 10:00:17 -03:00
Eduardo Arias
3ff72fbbc5 Perform ParityEven7bit, ParityOdd7bit & ParityZero7bit transformations in-place
- Refactored to share implementations of ParityEven7bit & ParityOdd7bit.
2024-08-27 10:00:17 -03:00
Eduardo Arias
5d39890783 Updated Transformation::evaluate signature to allow for in-place updates, removing unnecessary heap allocated copies.
- Renamed Transformation::evaluate to Transformation::transform to avoid
  confusion with Action's overload methods.
- Updated Transformation::transform signature to receive the value by
  reference and perform the transformation inline, if possible.
  - Some transformations still need to use a temporary std::string to
    perform their work, and then copy the result back.
- Made Transformation::transform methods const and updated Transaction
  parameter to be const.
  - Transaction parameter could not be removed because it's used by just
    a single transformation, UrlDecodeUni.
- Removed std::string Action::evaluate(const std::string &exp,
  Transaction *transaction); which was only implemented by
  Transformation but was not used from the base class, but only after
  downcasting to Transformation, so it can just be declared there (and
  not pollute other actions with a default member implementation -that
  does nothing- which is never called).
2024-08-27 10:00:17 -03:00
Ervin Hegedus
094143801a
Added PR #3232 2024-08-26 21:48:46 +02:00
Ervin Hegedus
97c8766ef1
Merge pull request #3232 from eduar-hte/failed-unit-tests-automake-output
Unit tests results should not be displayed in 'automake output' mode
2024-08-26 21:47:32 +02:00
David Kirstein
315b3d6e77
Lua::run: Move logging of str parameter to higher log level. 2024-08-26 08:38:48 +02:00
Eduardo Arias
df081af870 Adjusted pthread LDFLAG in examples required for multithreading.
- Some versions of gcc/libc require setting the pthread flag when using
  std::thread, which to implement it.
- This was found compiling the library in a Debian (bullseye) container.
2024-08-18 19:19:29 +00:00
Eduardo Arias
a5f223cb52 Individual test result should not be printed for automake output
- Test results output escape characters to highlight whether the test
  passed or failed. Additionally, the input & output for each test can
  include non-ASCII characters. These characters break parsing of
  results (.log & .trs files) with grep, as the files are interpreted
  to be binary.
2024-08-18 19:19:23 +00:00
Ervin Hegedus
752ab76238
Merge pull request #3229 from eduar-hte/pthread-makefile
Restore pthread LDFLAG in examples
2024-08-14 22:07:24 +02:00
Eduardo Arias
2cb1d032e3 Restore pthread LDFLAG. 2024-08-14 08:46:30 -07:00
Ervin Hegedus
71bea86e91
Added PR #3228 2024-08-14 14:56:42 +02:00
Ervin Hegedus
b4f52325bd
Merge pull request #3228 from eduar-hte/asctime-multithread
Replace usage of std::ctime, which is not safe in multithread contexts
2024-08-14 14:55:53 +02:00
Ervin Hegedus
746f4d7e80
Added PR #3227 2024-08-14 12:37:33 +02:00
Ervin Hegedus
554bd30e74
Merge pull request #3227 from eduar-hte/pm-operator-multithreading
Removed unnecessary lock to call acmp_process_quick in Pm::evaluate
2024-08-14 12:36:54 +02:00
Ervin Hegedus
c9af0c747e
Merge pull request #3221 from eduar-hte/unittest-multithreaded
Add support to run unit tests in a multithreaded context
2024-08-14 12:18:07 +02:00
Ervin Hegedus
c4b2723a4f
Added PR #3225 2024-08-14 09:07:33 +02:00
Ervin Hegedus
a6b287e120
Merge pull request #3225 from airween/v3/mpinvcharreqbody
feat: Check if the MP header contains invalid character
2024-08-14 09:06:14 +02:00
Eduardo Arias
ee5f95eb04 Added support to run unit tests in a multithreaded context
- This is controlled by specifying the 'mtstress' argument when running
  `unit_test`.
- The goal is to detect if the operator/transformation  fails in this
  context.
- In this mode, the test will be executed 5'000 times in 50 threads
  concurrently.
- Allocation & initialization of the operator/transformation is
  performed once in the main thread, while the evaluation is executed in
  the threads.
  - This is consistent with the library's support for multithreading,
    where initialization and loading of rules is expected to run once.
    See issue #3215.
2024-08-13 14:06:23 -07:00
Eduardo Arias
23a341eb6a Calculate sizes of strftime buffers based on format strings
- Leverage std::size to determine buffer size at compile time.
- Simplified 'TimeMon::evaluate' implementation as it was using strftime
  to get the month, convert the string to int, and then decrement it by
  one to make it zero based. This same value is already available in
  the 'struct tm' previously generated with the call to localtime_r (and
  where the month is already zero-based)
2024-08-13 13:36:03 -07:00
Eduardo Arias
5e6fcbc60b Replace usage of std::ctime, which is not safe for use in multithreaded contexts
- std::ctime returns a pointer to a string that "may be shared between
  std::asctime and std::ctime, and may be overwritten on each invocation
  of any of those functions.".
  - https://en.cppreference.com/w/cpp/chrono/c/ctime
- Replaced with call to strftime to generate the same string
  representation (using the format string: %c)
- Leveraged localtime_r (which is thread-safe) to convert time_t to
  struct tm, as required by strftime.
2024-08-13 10:54:06 -07:00
Eduardo Arias
8d6b185856 Removed unnecessary lock to call acmp_process_quick in Pm::evaluate
- This was introduced in commit 119a6fc & 7d786b3 because of a potential
  issue reported in #1573.
- The ACMP tree structure is initialized when the operator is
  initialized.
- During transaction execution the ACMP tree structure is only 'read'
  while traversing the tree (in acmp_process_quick) so this is safe for
  use in a multi-threaded environment.
2024-08-13 10:53:15 -07:00
Ervin Hegedus
f42bc38f4c
Update CHANGES 2024-08-13 19:35:14 +02:00
Ervin Hegedus
718d121ee3
Merge pull request #3216 from eduar-hte/inmemory-collection-shared-mutex
Prevent concurrent access to data in InMemoryPerProcess' resolveXXX methods
2024-08-13 19:30:14 +02:00
Ervin Hegedus
32f6f78e78
Merge pull request #3222 from eduar-hte/remove-copies
Remove several string copies and unnecessary heap allocations
2024-08-13 19:25:11 +02:00
Ervin Hegedus
6388d88f38
Check if the MP header contains invalid character 2024-08-13 18:26:18 +02:00
Eduardo Arias
77adb57524 Avoid std::string copy in ssplit argument
- Other minor changes reported by sonarcloud
2024-08-12 12:59:28 -07:00