Commit Graph

  • 07fb580415 Added PR's #3233 and #3231 Ervin Hegedus 2024-08-28 14:34:58 +02:00
  • 9148668571 Merge pull request #3231 from eduar-hte/remove-copies-transformations Ervin Hegedus 2024-08-28 14:33:59 +02:00
  • 4951702d45 Merge pull request #3233 from eduar-hte/remove-copies-pm-operator Ervin Hegedus 2024-08-28 13:31:02 +02:00
  • 3e9d8107a8 Removed multiple heap-allocated copies in parse_pm_content - The previous version of this function was doing three strdup copies to parse the pm content. The updated version only copies the value once (in order not to modify the Operator's m_param member variable), and then performs the updates inline. - Binary parsing was broken because digits were not compared as characters. - Fail parsing when an invalid hex character is found. - Error message in parse_pm_content would reference freed memory if accessed by caller. Removed anyway because it was unused. Eduardo Arias 2024-08-09 13:01:37 -07:00
  • a6d64bf615 Replaced VALID_HEX, ISODIGIT & NBSP macros in string.h - Moved them into modsecurity::utils::string to avoid polluting the global namespace. Eduardo Arias 2024-08-17 00:53:54 +00:00
  • 2f5dac5c4c Simplified initialization of Transformation's action_kind - Some of the Transformation classes would initialize their Action's action_kind using the default (using Transformation constructor without an action_kind parameter). - Others, however, would use that constructor and initialize action_kind manually in their constructor, but setting the default value (RunTimeBeforeMatchAttemptKind = 1), which was redundant. - Removed unused Transformation constructor to specify action_kind. - Converted Action::Kind into an 'enum class' to require using the enum constants (instead of integer values, which are difficult to track in the codebase and change) Eduardo Arias 2024-08-19 11:26:56 -07:00
  • 7023c0a8b4 Refactored sha1 & md5 utils to share implementation and reduce code duplication. Eduardo Arias 2024-08-19 10:34:00 -07:00
  • fedec96a7e Refactored base64 utils to share implementation and reduce code duplication. Eduardo Arias 2024-08-19 10:25:36 -07:00
  • 34da8eeeee Pass RuleWithActions::executeTransformation arguments by reference - This function already expects these arguments not to be null pointers, doesn't validate them and just dereference them. - In order to make this explicit and enforced by the compiler, they're now passed as references. Eduardo Arias 2024-08-09 13:03:34 -07:00
  • b647dbd905 Remove unnecessary heap-allocation & copy in Transaction::extractArguments - utils::urldecode_nonstrict_inplace decodes inplace so key & value, which are values returned by utils::string::ssplit_pair can be just be modified and do not need to be copied. - Updated signature of utils::urldecode_nonstrict_inplace, as its two callers already have std::string values. Eduardo Arias 2024-06-01 21:59:16 +00:00
  • 021d0caa33 Perform NormalisePath & NormalisePathWin transformations in-place Eduardo Arias 2024-08-19 10:20:12 -07:00
  • 2c3c228725 Perform Utf8ToUnicode transformation in-place - Removed inplace helper function from the class, as it's only referenced by the implementation. Eduardo Arias 2024-08-19 10:15:12 -07:00
  • 17a2cbd164 Perform UrlDecodeUni & UrlDecode transformations in-place - Use std::string in UrlEncode transformation, instead of manually memory management. This avoids an additional copy after completing encoding by just swapping the encoded value and the input. - Removed inplace helper function from the class, as it's only referenced by the implementation. Eduardo Arias 2024-08-19 09:12:12 -07:00
  • 8bf4d96e6b Perform HtmlEntityDecode transformation in-place - Removed inplace helper function from the class, as it's only referenced by the implementation. Eduardo Arias 2024-08-19 09:04:42 -07:00
  • 7d5c9faa43 Perform JsDecode transformation in-place - Removed inplace helper function from the class, as it's only referenced by the implementation. Eduardo Arias 2024-08-19 08:32:27 -07:00
  • a520369da0 Perform EscapeSeqDecode transformation in-place - Removed ansi_c_sequences_decode_inplace helper function from the class, as it's only referenced by the implementation. Eduardo Arias 2024-08-19 08:28:04 -07:00
  • 727f2bf840 Perform CssDecode transformation in-place - Removed inplace helper function from the class, as it's only referenced by the implementation. Eduardo Arias 2024-08-19 08:24:31 -07:00
  • e687140d05 Perform HexDecode transformation in-place - Removed inplace helper function from the class, as it's only referenced by the implementation. Eduardo Arias 2024-08-19 08:12:00 -07:00
  • 4670710376 Perform LowerCase & UpperCase transformations in-place - Refactored to share implementation and reduce code duplication. Eduardo Arias 2024-08-19 08:06:00 -07:00
  • fd8a979463 Perform SqlHexDecode transformation in-place - Validate buffer size before accessing data. The previous implementation would only check that there was a character available in the buffer but could continue processing/reading characters from an hex representation without checking bounds. - Removed inplace & mytolower helper functions from the class, as they're only referenced by the implementation. - Removed duplicate VALID_HEX & ISODIGIT macros, already in src/utils/string.h. Eduardo Arias 2024-08-19 07:55:41 -07:00
  • 2915ee60e2 Perform Trim, TrimLeft & TrimRight transformations in-place Eduardo Arias 2024-08-19 07:45:16 -07:00
  • 74d150c068 Perform RemoveCommentsChar, RemoveComments & ReplaceComments transformations in-place Eduardo Arias 2024-08-19 07:30:11 -07:00
  • da775eca81 Perform ReplaceNulls transformation in-place Eduardo Arias 2024-08-19 08:41:18 -07:00
  • 1505025990 Perform RemoveNulls & RemoveWhitespace transformations in-place - Refactored to share implementation. Eduardo Arias 2024-08-19 07:21:22 -07:00
  • 1236d9a7cd Perform CompressWhitespace transformation in-place Eduardo Arias 2024-08-19 07:14:11 -07:00
  • 13203ae5e7 Perform CmdLine transformation in-place Eduardo Arias 2024-08-19 07:09:26 -07:00
  • 3ff72fbbc5 Perform ParityEven7bit, ParityOdd7bit & ParityZero7bit transformations in-place - Refactored to share implementations of ParityEven7bit & ParityOdd7bit. Eduardo Arias 2024-08-19 01:48:38 +00:00
  • 5d39890783 Updated Transformation::evaluate signature to allow for in-place updates, removing unnecessary heap allocated copies. - Renamed Transformation::evaluate to Transformation::transform to avoid confusion with Action's overload methods. - Updated Transformation::transform signature to receive the value by reference and perform the transformation inline, if possible. - Some transformations still need to use a temporary std::string to perform their work, and then copy the result back. - Made Transformation::transform methods const and updated Transaction parameter to be const. - Transaction parameter could not be removed because it's used by just a single transformation, UrlDecodeUni. - Removed std::string Action::evaluate(const std::string &exp, Transaction *transaction); which was only implemented by Transformation but was not used from the base class, but only after downcasting to Transformation, so it can just be declared there (and not pollute other actions with a default member implementation -that does nothing- which is never called). Eduardo Arias 2024-05-14 01:47:59 +00:00
  • 094143801a Added PR #3232 Ervin Hegedus 2024-08-26 21:48:46 +02:00
  • 97c8766ef1 Merge pull request #3232 from eduar-hte/failed-unit-tests-automake-output Ervin Hegedus 2024-08-26 21:47:32 +02:00
  • d55495e1dc Added PR #3241 Ervin Hegedus 2024-08-26 17:59:11 +02:00
  • 7cc689032f Merge pull request #3241 from marcstern/v2/pr/ap_log_perror Ervin Hegedus 2024-08-26 17:58:25 +02:00
  • 6be2ee534a Fixed ap_log_perror() usage Replaces #3236 Marc Stern 2024-08-26 17:17:36 +02:00
  • 207525e194 Added PR #3191 Ervin Hegedus 2024-08-26 16:38:30 +02:00
  • f65415ae8a Merge pull request #3191 from marcstern/v2/pr/mem_leak_re Ervin Hegedus 2024-08-26 16:37:01 +02:00
  • 315b3d6e77 Lua::run: Move logging of str parameter to higher log level. David Kirstein 2024-08-26 08:38:48 +02:00
  • 6e384d13ab Added PR #3190 Ervin Hegedus 2024-08-20 15:27:44 +02:00
  • fd0e042abc Merge pull request #3190 from marcstern/v2/pr/ci_log Ervin Hegedus 2024-08-20 15:25:13 +02:00
  • 110b61f6d7 Merge branch 'v2/master' into v2/pr/ci_log Ervin Hegedus 2024-08-19 16:57:10 +02:00
  • d996f04e3a Add trailing :: sequence Ervin Hegedus 2024-08-19 16:47:09 +02:00
  • 046d3eb3ec Fixed two error messages Marc Stern 2024-08-19 14:19:05 +02:00
  • 7f4e416fc4 Added PR #3202 Ervin Hegedus 2024-08-18 22:59:22 +02:00
  • e7e11d972f Merge pull request #3202 from marcstern/v2/pr/assert Ervin Hegedus 2024-08-18 22:58:06 +02:00
  • df081af870 Adjusted pthread LDFLAG in examples required for multithreading. - Some versions of gcc/libc require setting the pthread flag when using std::thread, which to implement it. - This was found compiling the library in a Debian (bullseye) container. Eduardo Arias 2024-08-16 05:59:38 -07:00
  • a5f223cb52 Individual test result should not be printed for automake output - Test results output escape characters to highlight whether the test passed or failed. Additionally, the input & output for each test can include non-ASCII characters. These characters break parsing of results (.log & .trs files) with grep, as the files are interpreted to be binary. Eduardo Arias 2024-08-17 22:36:30 +00:00
  • 60d07a5547 added one more NULL check at run-time Marc Stern 2024-08-16 09:23:11 +02:00
  • 752ab76238 Merge pull request #3229 from eduar-hte/pthread-makefile Ervin Hegedus 2024-08-14 22:07:24 +02:00
  • 4edeca70e4 Added "::error" in error message Marc Stern 2024-08-14 19:12:03 +02:00
  • 4b391834ec added more NULL checks at run-time Marc Stern 2024-08-14 19:09:15 +02:00
  • 0066a67911 added more NULL checks at run-time Marc Stern 2024-08-14 19:00:25 +02:00
  • 22a6829690 added more NULL checks at run-time Marc Stern 2024-08-14 18:44:45 +02:00
  • 2cb1d032e3 Restore pthread LDFLAG. Eduardo Arias 2024-08-14 08:46:30 -07:00
  • 71bea86e91 Added PR #3228 Ervin Hegedus 2024-08-14 14:56:42 +02:00
  • b4f52325bd Merge pull request #3228 from eduar-hte/asctime-multithread Ervin Hegedus 2024-08-14 14:55:53 +02:00
  • e5bbd89399 re-added some NULL check at run-time, with an error message on stderr Marc Stern 2024-08-14 13:53:52 +02:00
  • 1680f5be90 removed comment Marc Stern 2024-08-14 12:56:59 +02:00
  • 746f4d7e80 Added PR #3227 Ervin Hegedus 2024-08-14 12:37:33 +02:00
  • 554bd30e74 Merge pull request #3227 from eduar-hte/pm-operator-multithreading Ervin Hegedus 2024-08-14 12:36:54 +02:00
  • c9af0c747e Merge pull request #3221 from eduar-hte/unittest-multithreaded Ervin Hegedus 2024-08-14 12:18:07 +02:00
  • 04dff87623 Added PR #3193 Ervin Hegedus 2024-08-14 10:59:56 +02:00
  • 277e7e2bf6 Merge pull request #3193 from marcstern/v2/pr/useless Ervin Hegedus 2024-08-14 10:59:03 +02:00
  • 0a9e0aa67b Added PR #3226 Ervin Hegedus 2024-08-14 09:32:08 +02:00
  • 8cfb9112fb Merge pull request #3226 from airween/v2/mpinvcharreqbody Ervin Hegedus 2024-08-14 09:31:20 +02:00
  • c4b2723a4f Added PR #3225 Ervin Hegedus 2024-08-14 09:07:33 +02:00
  • a6b287e120 Merge pull request #3225 from airween/v3/mpinvcharreqbody Ervin Hegedus 2024-08-14 09:06:14 +02:00
  • ee5f95eb04 Added support to run unit tests in a multithreaded context - This is controlled by specifying the 'mtstress' argument when running unit_test. - The goal is to detect if the operator/transformation fails in this context. - In this mode, the test will be executed 5'000 times in 50 threads concurrently. - Allocation & initialization of the operator/transformation is performed once in the main thread, while the evaluation is executed in the threads. - This is consistent with the library's support for multithreading, where initialization and loading of rules is expected to run once. See issue #3215. Eduardo Arias 2024-08-09 06:54:35 -07:00
  • 23a341eb6a Calculate sizes of strftime buffers based on format strings - Leverage std::size to determine buffer size at compile time. - Simplified 'TimeMon::evaluate' implementation as it was using strftime to get the month, convert the string to int, and then decrement it by one to make it zero based. This same value is already available in the 'struct tm' previously generated with the call to localtime_r (and where the month is already zero-based) Eduardo Arias 2024-08-12 08:47:04 -07:00
  • 5e6fcbc60b Replace usage of std::ctime, which is not safe for use in multithreaded contexts - std::ctime returns a pointer to a string that "may be shared between std::asctime and std::ctime, and may be overwritten on each invocation of any of those functions.". - https://en.cppreference.com/w/cpp/chrono/c/ctime - Replaced with call to strftime to generate the same string representation (using the format string: %c) - Leveraged localtime_r (which is thread-safe) to convert time_t to struct tm, as required by strftime. Eduardo Arias 2024-08-12 07:58:06 -07:00
  • 8d6b185856 Removed unnecessary lock to call acmp_process_quick in Pm::evaluate - This was introduced in commit 119a6fc & 7d786b3 because of a potential issue reported in #1573. - The ACMP tree structure is initialized when the operator is initialized. - During transaction execution the ACMP tree structure is only 'read' while traversing the tree (in acmp_process_quick) so this is safe for use in a multi-threaded environment. Eduardo Arias 2024-08-09 09:41:37 -07:00
  • ffecae98d3 Update CHANGES Ervin Hegedus 2024-08-13 19:35:17 +02:00
  • f42bc38f4c Update CHANGES Ervin Hegedus 2024-08-13 19:35:14 +02:00
  • 718d121ee3 Merge pull request #3216 from eduar-hte/inmemory-collection-shared-mutex Ervin Hegedus 2024-08-13 19:30:14 +02:00
  • 32f6f78e78 Merge pull request #3222 from eduar-hte/remove-copies Ervin Hegedus 2024-08-13 19:25:11 +02:00
  • 6388d88f38 Check if the MP header contains invalid character Ervin Hegedus 2024-08-13 18:26:18 +02:00
  • e6e3417e9d Remove unnecessary assert() Ervin Hegedus 2024-08-13 11:07:44 +02:00
  • f27c85cf47 Check if the MP header contains invalid character Ervin Hegedus 2024-08-13 11:07:18 +02:00
  • 77adb57524 Avoid std::string copy in ssplit argument - Other minor changes reported by sonarcloud Eduardo Arias 2024-08-12 06:18:20 -07:00
  • 305f33fea1 Merge pull request #3224 from airween/v3/sethostnametestfix Ervin Hegedus 2024-08-12 20:39:30 +02:00
  • eb26b7960c Fix regression test result; Add test to main test-suite list Ervin Hegedus 2024-08-12 18:46:19 +02:00
  • 935e68c816 Merge pull request #3192 from marcstern/v2/pr/errorlog Ervin Hegedus 2024-08-12 17:17:15 +02:00
  • d32c8f1ad8 Fixed invalid logging Marc Stern 2024-08-12 17:06:35 +02:00
  • 914c1a1cb2 Merge pull request #3194 from marcstern/v2/pr/PCRE_ERROR_NOMATCH Ervin Hegedus 2024-08-12 16:40:40 +02:00
  • cc0f893854 Removed unused overload of dash_if_empty that sonarcloud flags as potential buffer overflow Eduardo Arias 2024-08-09 14:02:40 -07:00
  • 8b17f3691f Inline string functions Eduardo Arias 2024-05-24 21:11:21 -03:00
  • 1534ee2448 Removed unnecessary copies Eduardo Arias 2024-05-30 16:32:16 +00:00
  • f8dd09f7c9 Avoid creating a new std::string on the heap to create VariableValue - Introduced helper method addVariableOrigin to reduce code duplication. eduar-hte 2024-05-18 16:55:41 +00:00
  • bb07de9ad7 toupper/tolower is already receiving a copy, so it doesn't need to create a new one to transform it - Make functions inline to improve performance - Introduced helper method toCaseHelper to remove code duplication Eduardo Arias 2024-04-28 21:45:34 -03:00
  • 4bf9616f9e Adding multithreaded example from issue #3054 (by airween) - Rewritten to use C++ libModSecurity API and std::thread (instead of pthreads) Eduardo Arias 2024-08-09 10:56:36 -07:00
  • 293cd214c7 Removed usage of pthreads and replaced with std C++ features - Replaced pthread_mutex_t in modsecurity::operators::Pm with std::mutex - Replaced pthread's thread usage in reading_logs_via_rule_message example with std::thread. - Simplified and modernized C++ code. - Removed unnecessary includes of pthread.h Eduardo Arias 2024-08-07 13:54:58 -07:00
  • 4e15f9ef71 Turn off LMDB by default in Windows build to align with defaults for other platforms - Replaced WITHOUT_XXX build options with WITH_XXX to make it easier to understand and configure. - Updated GitHub workflow to align with these changes and include a build 'with lmdb' (again, analogous to non-Windows configurations) Eduardo Arias 2024-08-07 11:50:55 -07:00
  • e2b3c9594f Prevent concurrent access to data structure in resolve methods - As reported in #3054, the resolve methods in InMemoryPerProcess are not acquiring a lock/mutex to prevent concurrent access to the data structures that may be modified at the same time from other threads, and thus triggering undefined behaviour. - Replace inheritance of std::unordered_multimap data structure with data member to prevent potential clients to use it without acquiring the mutex to protect concurrent access. - Replace pthreads lock with std C++11 std::shared_mutex - Provides exclusive/shared lock access so that multiple readers can access the data at the same time, but only one writer. this is used to favor query performance by allowing more concurrent access to the data until an update needs to be performed. - Simplifies acquisition and disposal of lock/mutex with std::lock_guard, which has RAII semantics. - NOTE: Because std::shared_mutex is not recursive, calls to another function that tries to acquire the lock will fail. Introduced __store & __updateFirst helper methods to workaround this. - Updates to InMemoryPerProcess::resolveFirst - Updated the code to store the expired var in 'expiredVars' to delete them after iterating over the range (and releasing the read lock, as 'delIfExpired' needs to acquire it for exclusive access), as the current call to 'delIfExpired' would invalidate the range triggering undefined behaviour on the following iteration. - Noticed that in commit 118e1b3 the call to 'delIfExpired' in this function is done using 'it->second.getValue()'' instead of 'it->first', which seems incorrect (based on similar code in other resolveXXX functions). - Updated InMemoryPerProcess::delIfExpired to use 'std::find_if' (with a lambda that matches both the key and the 'isExpired' condition) because the data structure is a multimap. The version introduced in commit 118e1b3 could find an entry (not necessarily the first, because the map is unordered) where 'isExpired' is 'false' and exit, while another entry could be expired. Eduardo Arias 2024-08-07 08:33:03 -07:00
  • c575dce3d3 Added PR 3218, 3219, 3220 Ervin Hegedus 2024-08-09 17:40:33 +02:00
  • 7bdc3c825c Merge pull request #3220 from eduar-hte/string-null Ervin Hegedus 2024-08-09 17:37:47 +02:00
  • 3a83196a71 Merge pull request #3219 from eduar-hte/cpp17 Ervin Hegedus 2024-08-09 17:34:02 +02:00
  • 6f0e566f98 Merge pull request #3218 from eduar-hte/remove-dynamic-casts Ervin Hegedus 2024-08-09 17:24:55 +02:00
  • c917d6a2dc Initialize variable in if statement to avoid doing dynamic_cast twice - Refactored duplicate code in RuleWithOperator::getVariablesExceptions - Leveraged auto to simplify declaration of dynamic_cast pointers. Eduardo Arias 2024-08-08 12:49:48 -07:00
  • 18378c10f8 Removed unnecessary dynamic_casts Eduardo Arias 2024-05-18 20:32:21 +00:00
  • 09980324a7 Added PR #3114 Ervin Hegedus 2024-08-08 21:03:10 +02:00
  • a23e88f79f Merge pull request #3114 from airween/v3/sonarmemleakfix Ervin Hegedus 2024-08-08 21:02:15 +02:00
  • 30a68de92d Creating a std::string with a null pointer is undefined behaviour. - cppreference mentions this about the constructor that receives a const char *: - Constructs the string with the contents initialized with a copy of the null-terminated character string pointed to by s. The length of the string is determined by the first null character. The behavior is undefined if [s, s + Traits::length(s)) is not a valid range (for example, if s is a null pointer). - C++23 introduces a deleted constructor to prevent this in static scenarios, which is how this issue was detected. Eduardo Arias 2024-05-21 21:02:25 +00:00