diff --git a/CHANGES b/CHANGES index 6ee1adca..64d63ffe 100644 --- a/CHANGES +++ b/CHANGES @@ -1,3 +1,68 @@ +18 Apr 2011 - 2.6.0-rc1 +------------------- + + * Replaced previous GPLv2 Licento to Apachev2. + + * Added Google Safe Browsing lookups operator and directive. It should be + used to extract and lookup urls from http packets. + + * Added Data Modification operator. It must be used with STREAM_* variables + to replace/add/edit any data from http bodies. + + * Added STREAM_OUPUT_BODY and STREAM_INPUT_BODY variables to work with data + modification operators. + + * Added fast ip address operator. It supports partial ip address, cidr for + IPv4 and IPv6. Thanks Tom Donovan. + + * Added new sensitive data tracking verifyCPF and verifySSN. + + * Added MATCHED_VARS and MATCHED_VARS_NAMES. It is similiar to MATCHED_VAR, + but now we should see all matched variables. + + * Added UNIQUE_ID variable. It holds the data created my mod_unique_id. + + * Added new tranformation cmdline. Thanks Mark Stern. + + * Added new exception handling operators and directives. It should help users + reduce FN and FPs. The directives SecRuleUpdateTargetById, SecRuleRemoveByTag + and its ctl actions were included. + + * Added SecStreamOutBodyInspection and SecStreamInBodyInspection to enable STREAM_* + variables. + + * Added SecGsbLookupDB used to load Google Safe Browsing malware databse into + memory. + + * Added the directive SecInterceptOnError to control what to do if a rule returns + values less than zero. + + * Improvements in DetectionOnly engine mode. Also added SecRequestBodyLimitAction + to control what to do if the engine receive a http request over a hard limit. + Note that there is now many combinations with SecRuleEngine and the limit action + directives for response and request data. Please see the reference manual. + + * Improvements under RBL operator. It now will parse return code values for some + RBL lists. + + * Added new Log Part J. It should log some informations about uploaded files. + + * Added new sanitizeMatchedBytes action. It will give more flexibilty for user to sanitize + logged data, also improving peformance when sanitize big amount of data. + + * Improvements on Logging phase. It is possible now see full chains, distinguish between + simple rules, chain starters and chain nodes. + + * Improvements on AutoTools usage. + + * Improvements on pattern matching operators, pmf, pm and strmatch now supports more flexible + input data allowing any kind of special char. + + * Improvements on SecRuleUpdateActionById to update chain nodes. + + * Many bugs were fixed. Please see the ModSecurity Jira for more details + + 19 Mar 2010 - trunk ------------------- diff --git a/doc/Reference_Manual.html b/doc/Reference_Manual.html index 51522634..424dffa4 100644 --- a/doc/Reference_Manual.html +++ b/doc/Reference_Manual.html @@ -20,15 +20,15 @@ href="http://sourceforge.net/apps/mediawiki/mod-security/index.php?title=Special Atom Feed" href="http://sourceforge.net/apps/mediawiki/mod-security/index.php?title=Special:RecentChanges&feed=atom">
svn
-svn co https://mod-security.svn.sourceforge.net/svnroot/mod-security/m2/trunk modisecurity +svn co https://mod-security.svn.sourceforge.net/svnroot/mod-security/m2/trunk modsecurity+For v2.6.0 and above, the installation process has changed. Follow +these steps: +
+
- cd into the directory -
$cd modsecurity
+- Run autogen.sh script -
$./autogen.sh
+- Run configure script -
$./configure
+- Run make -
$make
+- Run make install -
$make install
+- Copy the new mod_security2.so file into the proper Apache +modules directory -
$cp +/usr/local/modsecurity/lib/mod_security2.so /usr/local/apache/modules/
+Stable Release Download
To download the stable release go to make
Optionally test with:
-make test+
make CFLAGS=-DMSC_TEST test
Description: Selects the cookie format that will be used in @@ -2285,11 +2307,12 @@ programming interface is appreciated.
Description: Updates the action list of the specified rule. -
Syntax: SecRuleUpdateActionById RULEID ACTIONLIST
+
Syntax: SecRuleUpdateActionById RULEID[:offset]
+ACTIONLIST
Example Usage: SecRuleUpdateActionById 12345
"deny,status:403"
Scope: Any -
Version: 2.5.0 +
Version: 2.6.0
This directive will overwrite the action list of the specified rule with the actions provided in the second parameter. It has two limitations: it cannot be used to change the ID or phase of a rule. Only @@ -2392,7 +2415,7 @@ insert.
Description: Configures the ability to use stream inspection -(Apache connection level filter) for inbound request data. +for inbound request data.
Syntax: SecStreamInBodyInspection On|Off
Example Usage: SecStreamInBodyInspection On
Scope: Any @@ -2408,8 +2431,8 @@ REQUEST_HEADER data.
Description: Configures the ability to use stream inspection -(Apache connection level filter) for outbound request data. +
Description: Configures the ability to use stream inspection +for outbound request data.
Syntax: SecStreamOutBodyInspection On|Off
Example Usage: SecStreamOutBodyInspection On
Scope: Any @@ -2552,7 +2575,7 @@ diagram, the 5 ModSecurity processing phases are shown.
In order to select the phase a rule executes during, use the phase action either directly in the rule or in using the @@ -2827,12 +2850,24 @@ class="mw-headline"> HIGHEST_SEVERITY matched so far. Severities are numeric values and thus can be used with comparison operators such as @lt, and so on. A value of 255 indicates that no severity has been set. -
SecRule HIGHEST_SEVERITY "@le 2" \
+
SecRule HIGHEST_SEVERITY "@le 2"
"phase:2,deny,status:500,msg:'severity %{HIGHEST_SEVERITY}'"
This variable will be set to 1 when the request body size is above +the setting configured by SecRequestBodyLimit directive. Your policies +should always contain a rule to check this variable. Depending on the +rate of false positives and your default policy you should decide +whether to block or just warn when the rule is triggered. +
The best way to use this variable is as in the example below: +
SecRule INBOUND_ERROR_DATA "@eq 1"
+"phase:1,t:none,log,pass,msg:'Request Body Larger than
+SecRequestBodyLimit Setting'"
+
This variable holds the value of the most-recently matched variable. @@ -3055,22 +3090,14 @@ information will not be available if the authentication is
handled in the backend web server.
-Contains the name of the currently used request body processor. The -possible values are URLENCODED, MULTIPART, and XML. -
-SecRule REQBODY_PROCESSOR "^XML$ chain - SecRule XML "@validateDTD /opt/apache-frontend/conf/xml.dtd" --
Contains the status of the request body processor used for request body parsing. The values can be 0 (no error) or 1 (error). This variable will be set by request body processors (typically the multipart/request-data parser or the XML parser) when they fail to do their work. -
SecRule REQBODY_PROCESSOR_ERROR "@eq 1" deny,phase:2
+
SecRule REQBODY_ERROR "@eq 1" deny,phase:2
If there’s been an error during request body parsing, the variable will contain the following error message: -
SecRule REQBODY_PROCESSOR_ERROR_MSG "failed to parse"
+
SecRule REQBODY_ERROR_MSG "failed to parse"
Contains the name of the currently used request body processor. The +possible values are URLENCODED, MULTIPART, and XML. +
+SecRule REQBODY_PROCESSOR "^XML$ chain + SecRule XML "@validateDTD /opt/apache-frontend/conf/xml.dtd" +
This variable holds just the filename part of REQUEST_FILENAME (e.g., @@ -3377,9 +3412,8 @@ SESSIONID
This variable is created by a Connection-Level Filter hook in Apache -and give access to the raw request body content. This variable is best -used for two use-cases: +
This variable give access to the raw request body content. This +variable is best used for two use-cases:
This variable is created by a Connection-Level Filter hook in Apache -and give access to the raw response body content. This variable is best - used for two use-cases: +
This variable give access to the raw response body content. This +variable is best used for two use-cases:
SecRule REMOTE_ADDR "@ipMatch 192.168.1.100,192.168.1.50,10.10.50.0/24"-
Description: Performs numerical comparison and returns true if the input value is less than or equal to the operator parameter. Macro @@ -4963,18 +4994,6 @@ SecRule &REQUEST_HEADERS_NAMES "@le 15"
# Detect fewer than 15 headers in a request SecRule &REQUEST_HEADERS_NAMES "@lt 15"-
Description: Performs a string match of the provided word -against the desired input value. The operator uses the pattern matching - Boyer-Moore-Horspool algorithm, which means that it is a single pattern - matching operator. This operator performs much better than a regular -expression. -
Example: -
-# Detect suspicious client by looking at the user agent identification -SecRule REQUEST_HEADERS:User-Agent "@strmatch WebZIP" -
Description: Performs a case-insensitive match of the provided phrases against the desired input value. The operator uses a set-based @@ -5067,14 +5086,18 @@ setvar:tx.%{rule.id}-AUTOMATION/MALICIOUS-%{matched_var_name}=%{matched_var},set
Description: Performs regular expression data substitution when applied to either the STREAM_INPUT_BODY or STREAM_OUTPUT_BODY -variables. This operator also supports macro expasion. -
Syntax: @rsub s/regex/str/[i]
+variables. This operator also supports macro expansion.
+
Syntax: @rsub s/regex/str/[id]
Examples: Removing HTML Comments from response bodies:
SecStreamOutBodyInspection On -SecRule STREAM_OUTPUT_BODY "@rsub s/<!--.*?-->//" "phase:4,t:none,nolog,pass" +SecRule STREAM_OUTPUT_BODY "@rsub s/<!--.*?-->/ /" "phase:4,t:none,nolog,pass"+
Regular expressions are handled by the PCRE library [12]. ModSecurity @@ -5086,7 +5109,9 @@ are newline characters present. case-insensitive matching, you can either use the lowercase transformation function or force case-insensitive matching by prefixing the regular expression pattern with the (?i) modifier (a PCRE feature; -you will find many similar features in the PCRE documentation). +you will find many similar features in the PCRE documentation). Also a +flag [d] should be used if you want to escape the regex string chars +when use macro expansion.
# Detect request parameters "foo" that do not # contain "bar", exactly. SecRule ARGS:foo "!@streq bar"+
Description: Performs a string match of the provided word +against the desired input value. The operator uses the pattern matching + Boyer-Moore-Horspool algorithm, which means that it is a single pattern + matching operator. This operator performs much better than a regular +expression. +
Example: +
+# Detect suspicious client by looking at the user agent identification +SecRule REQUEST_HEADERS:User-Agent "@strmatch WebZIP" +
Description: Validates that the byte values used in input fall @@ -5473,6 +5510,14 @@ SecRuleEngine DetectionOnly # SecRequestBodyAccess On + +# Enable XML request body parser. +# Initiate XML Processor in case of xml content-type +# +SecRule REQUEST_HEADERS:Content-Type "text/xml" \ + "phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML" + + # Maximum request body size we will accept for buffering. If you support # file uploads then the value given on the first line has to be as large # as the largest file you are willing to accept. The second value refers @@ -5488,13 +5533,20 @@ SecRequestBodyNoFilesLimit 131072 # SecRequestBodyInMemoryLimit 131072 +# What do do if the request body size is above our configured limit. +# Keep in mind that this setting will automatically be set to ProcessPartial +# when SecRuleEngine is set to DetectionOnly mode in order to minimize +# disruptions when initially deploying ModSecurity. +# +SecRequestBodyLimitAction Reject + # Verify that we've correctly processed the request body. # As a rule of thumb, when failing to process a request body # you should reject the request (when deployed in blocking mode) # or log a high-severity alert (when deployed in detection-only mode). # -SecRule REQBODY_PROCESSOR_ERROR "!@eq 0" \ -"phase:2,t:none,log,deny,msg:'Failed to parse request body.',severity:2" +SecRule REQBODY_ERROR "!@eq 0" \ +"phase:2,t:none,log,deny,status:400,msg:'Failed to parse request body.',logdata:'%{reqbody_error_msg}',severity:2" # By default be strict with what we accept in the multipart/form-data # request body. If the rule below proves to be too strict for your @@ -5502,7 +5554,7 @@ SecRule REQBODY_PROCESSOR_ERROR "!@eq 0" \ # _not_ to remove it altogether. # SecRule MULTIPART_STRICT_ERROR "!@eq 0" \ -"phase:2,t:none,log,deny,msg:'Multipart request body \ +"phase:2,t:none,log,deny,status:44,msg:'Multipart request body \ failed strict validation: \ PE %{REQBODY_PROCESSOR_ERROR}, \ BQ %{MULTIPART_BOUNDARY_QUOTED}, \ @@ -5519,7 +5571,7 @@ IH %{MULTIPART_FILE_LIMIT_EXCEEDED}'" # Did we see anything that might be a boundary? # SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" \ -"phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'" +"phase:2,t:none,log,deny,status:44,msg:'Multipart parser detected a possible unmatched boundary.'" # PCRE Tuning # We want to avoid a potential RegEx DoS condition @@ -5538,17 +5590,20 @@ SecRule TX:/^MSC_/ "!@streq 0" \ # -- Response body handling -------------------------------------------------- -# Allow ModSecurity to access response bodies. We leave this disabled -# because most deployments want to focus on the incoming threats, and -# leaving this off reduces memory consumption. +# Allow ModSecurity to access response bodies. +# You should have this directive enabled in order to identify errors +# and data leakage issues. +# +# Do keep in mind that enabling this directive does increases both +# memory consumption and response latency. # -SecResponseBodyAccess Off +SecResponseBodyAccess On # Which response MIME types do you want to inspect? You should adjust the # configuration below to catch documents but avoid static files # (e.g., images and archives). # -SecResponseBodyMimeType text/plain text/html +SecResponseBodyMimeType text/plain text/html text/xml # Buffer response bodies of up to 512 KB in length. SecResponseBodyLimit 524288 @@ -5564,16 +5619,17 @@ SecResponseBodyLimitAction ProcessPartial # The location where ModSecurity stores temporary files (for example, when # it needs to handle a file upload that is larger than the configured limit). -# If you don't specify a location here your system's default will be used -# (normally /tmp), but that's less than ideal. It is recommended that you -# specify a location that's private. +# +# This default setting is chosen due to all systems have /tmp available however, +# this is less than ideal. It is recommended that you specify a location that's private. # -SecTmpDir /opt/modsecurity/var/tmp/ +SecTmpDir /tmp/ -# The location where ModSecurity will keep its persistent data. This, -# too, needs to be a place that other users can't access. +# The location where ModSecurity will keep its persistent data. This default setting +# is chosen due to all systems have /tmp available however, it +# too should be updated to a place that other users can't access. # -SecDataDir /opt/modsecurity/var/data/ +SecDataDir /tmp/ # -- File uploads handling configuration ------------------------------------- @@ -5582,19 +5638,19 @@ SecDataDir /opt/modsecurity/var/data/ # location must be private to ModSecurity. You don't want other users on # the server to access the files, do you? # -SecUploadDir /opt/modsecurity/var/upload/ +#SecUploadDir /opt/modsecurity/var/upload/ # By default, only keep the files that were determined to be unusual # in some way (by an external inspection script). For this to work you # will also need at least one file inspection rule. # -SecUploadKeepFiles RelevantOnly +#SecUploadKeepFiles RelevantOnly # Uploaded files are by default created with permissions that do not allow # any other user to access them. You may need to relax that if you want to # interface ModSecurity to an external program (e.g., an anti-virus). # -SecUploadFileMode 0600 +#SecUploadFileMode 0600 # -- Debug log configuration ------------------------------------------------- @@ -5602,34 +5658,35 @@ SecUploadFileMode 0600 # The default debug log configuration is to duplicate the error, warning # and notice messages from the error log. # -SecDebugLog /opt/modsecurity/var/log/debug.log -SecDebugLogLevel 3 +#SecDebugLog /opt/modsecurity/var/log/debug.log +#SecDebugLogLevel 3 # -- Audit log configuration ------------------------------------------------- # Log the transactions that are marked by a rule, as well as those that -# trigger a server error (determined by a 5xx response status code). +# trigger a server error (determined by a 5xx or 4xx, excluding 404, +# level response status codes). # SecAuditEngine RelevantOnly SecAuditLogRelevantStatus "^(?:5|4(?!04))" # Log everything we know about a transaction. -SecAuditLogParts ABCDEFHKZ +SecAuditLogParts ABIJDEFHKZ # Use a single file for logging. This is much easier to look at, but # assumes that you will use the audit log only ocassionally. # SecAuditLogType Serial -SecAuditLog /opt/modsecurity/var/log/audit.log +SecAuditLog /var/log/modsec_audit.log # Specify the path for concurrent audit logging. -SecAuditLogStorageDir /opt/modsecurity/var/audit/ +#SecAuditLogStorageDir /opt/modsecurity/var/audit/ # -- Miscellaneous ----------------------------------------------------------- -# Use the most commonly used application/x-www-form-urlencded parameter +# Use the most commonly used application/x-www-form-urlencoded parameter # separator. There's probably only one application somewhere that uses # something else so don't expect to change this value. # @@ -5644,13 +5701,13 @@ SecCookieFormat 0 - +
@@ -5760,7 +5817,7 @@ pages