Compare commits

..

1 Commits

Author SHA1 Message Date
Daniel Eisenberg
286c017133 fix for crds upload 2024-07-29 15:53:23 +03:00
503 changed files with 20336 additions and 56231 deletions

View File

@@ -1,36 +0,0 @@
---
name: "Bug Report"
about: "Report a bug with open-appsec"
labels: [bug]
---
**Checklist**
- Have you checked the open-appsec troubleshooting guides - https://docs.openappsec.io/troubleshooting/troubleshooting
- Yes / No
- Have you checked the existing issues and discussions in github for the same issue
- Yes / No
- Have you checked the knwon limitations same issue - https://docs.openappsec.io/release-notes#limitations
- Yes / No
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Run '...'
3. See error '...'
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots or Logs**
If applicable, add screenshots or logs to help explain the issue.
**Environment (please complete the following information):**
- open-appsec version:
- Deployment type (Docker, Kubernetes, etc.):
- OS:
**Additional context**
Add any other context about the problem here.

View File

@@ -1,8 +0,0 @@
blank_issues_enabled: false
contact_links:
- name: "Documentation & Troubleshooting"
url: "https://docs.openappsec.io/"
about: "Check the documentation before submitting an issue."
- name: "Feature Requests & Discussions"
url: "https://github.com/openappsec/openappsec/discussions"
about: "Please open a discussion for feature requests."

View File

@@ -1,17 +0,0 @@
---
name: "Nginx Version Support Request"
about: "Request for a specific Nginx version to be supported"
---
**Nginx & OS Version:**
Which Nginx and OS version are you using?
**Output of nginx -V**
Share the output of nginx -v
**Expected Behavior:**
What do you expect to happen with this version?
**Checklist**
- Have you considered a docker based deployment - find more information here https://docs.openappsec.io/getting-started/start-with-docker?
- Yes / No

View File

@@ -6,7 +6,7 @@
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/6629/badge)](https://bestpractices.coreinfrastructure.org/projects/6629)
# About
[open-appsec](https://www.openappsec.io) (openappsec.io) builds on machine learning to provide preemptive web app & API threat protection against OWASP-Top-10 and zero-day attacks. It can be deployed as an add-on to Linux, Docker or K8s deployments, on NGINX, Kong, APISIX, or Envoy.
[open-appsec](https://www.openappsec.io) (openappsec.io) builds on machine learning to provide preemptive web app & API threat protection against OWASP-Top-10 and zero-day attacks. It can be deployed as an add-on to Kubernetes Ingress, NGINX, Envoy (soon), and API Gateways.
The open-appsec engine learns how users normally interact with your web application. It then uses this information to automatically detect requests that fall outside of normal operations, and conducts further analysis to decide whether the request is malicious or not.
@@ -39,13 +39,13 @@ open-appsec can be managed using multiple methods:
* [Using SaaS Web Management](https://docs.openappsec.io/getting-started/using-the-web-ui-saas)
open-appsec Web UI:
<img width="1854" height="775" alt="image" src="https://github.com/user-attachments/assets/4c6f7b0a-14f3-4f02-9ab0-ddadc9979b8d" />
![image](https://github.com/openappsec/openappsec/assets/114033741/22d99379-df52-45c8-984f-1b820635f3b9)
## Deployment Playgrounds (Virtual labs)
You can experiment with open-appsec using [Playgrounds](https://www.openappsec.io/playground)
<img width="781" height="878" alt="image" src="https://github.com/user-attachments/assets/0ddee216-5cdf-4288-8c41-cc28cfbf3297" />
![image](https://github.com/openappsec/openappsec/assets/114033741/14d35d69-4577-48fc-ae87-ea344888e94d)
# Resources
* [Project Website](https://openappsec.io)
@@ -54,21 +54,27 @@ You can experiment with open-appsec using [Playgrounds](https://www.openappsec.i
# Installation
For Kubernetes (NGINX /Kong / APISIX / Istio) using Helm: follow [documentation](https://docs.openappsec.io/getting-started/start-with-kubernetes)
For Kubernetes (NGINX Ingress) using the installer:
For Linux (NGINX / Kong / APISIX) using the installer (list of supported/pre-compiled NGINX attachments is available [here](https://downloads.openappsec.io/packages/supported-nginx.txt)):
```bash
$ wget https://downloads.openappsec.io/open-appsec-k8s-install && chmod +x open-appsec-k8s-install
$ ./open-appsec-k8s-install
```
For Kubernetes (NGINX or Kong) using Helm: follow [documentation](https://docs.openappsec.io/getting-started/start-with-kubernetes/install-using-helm-ingress-nginx-and-kong) use this method if youve built your own containers.
For Linux (NGINX or Kong) using the installer (list of supported/pre-compiled NGINX attachments is available [here](https://downloads.openappsec.io/packages/supported-nginx.txt)):
```bash
$ wget https://downloads.openappsec.io/open-appsec-install && chmod +x open-appsec-install
$ ./open-appsec-install --auto
```
For kong Lua Based plug in follow [documentation](https://docs.openappsec.io/getting-started/start-with-linux)
For Linux, if youve built your own package use the following commands:
```bash
$ install-cp-nano-agent.sh --install --hybrid_mode
$ install-cp-nano-service-http-transaction-handler.sh --install
$ install-cp-nano-service-http-transaction-handler.sh install
$ install-cp-nano-attachment-registration-manager.sh --install
```
You can add the ```--token <token>``` and ```--email <email address>``` options to the first command, to get a token follow [documentation](https://docs.openappsec.io/getting-started/using-the-web-ui-saas/connect-deployed-agents-to-saas-management-k8s-and-linux).
@@ -171,7 +177,7 @@ open-appsec code was audited by an independent third party in September-October
See the [full report](https://github.com/openappsec/openappsec/blob/main/LEXFO-CHP20221014-Report-Code_audit-OPEN-APPSEC-v1.2.pdf).
### Reporting security vulnerabilities
If you've found a vulnerability or a potential vulnerability in open-appsec please let us know at security-alert@openappsec.io. We'll send a confirmation email to acknowledge your report within 24 hours, and we'll send an additional email when we've identified the issue positively or negatively.
If you've found a vulnerability or a potential vulnerability in open-appsec please let us know at securityalert@openappsec.io. We'll send a confirmation email to acknowledge your report within 24 hours, and we'll send an additional email when we've identified the issue positively or negatively.
# License

View File

@@ -95,18 +95,6 @@ getFailOpenHoldTimeout()
return conf_data.getNumericalValue("fail_open_hold_timeout");
}
unsigned int
getHoldVerdictPollingTime()
{
return conf_data.getNumericalValue("hold_verdict_polling_time");
}
unsigned int
getHoldVerdictRetries()
{
return conf_data.getNumericalValue("hold_verdict_retries");
}
unsigned int
getMaxSessionsPerMinute()
{
@@ -185,12 +173,6 @@ getReqBodySizeTrigger()
return conf_data.getNumericalValue("body_size_trigger");
}
unsigned int
getRemoveResServerHeader()
{
return conf_data.getNumericalValue("remove_server_header");
}
int
isIPAddress(c_str ip_str)
{

View File

@@ -66,41 +66,35 @@ TEST_F(HttpAttachmentUtilTest, GetValidAttachmentConfiguration)
"\"static_resources_path\": \"" + static_resources_path + "\",\n"
"\"min_retries_for_verdict\": 1,\n"
"\"max_retries_for_verdict\": 3,\n"
"\"hold_verdict_retries\": 3,\n"
"\"hold_verdict_polling_time\": 1,\n"
"\"body_size_trigger\": 777,\n"
"\"remove_server_header\": 1\n"
"\"body_size_trigger\": 777\n"
"}\n";
ofstream valid_configuration_file(attachment_configuration_file_name);
valid_configuration_file << valid_configuration;
valid_configuration_file.close();
EXPECT_EQ(initAttachmentConfig(attachment_configuration_file_name.c_str()), 1);
EXPECT_EQ(getDbgLevel(), 2u);
EXPECT_EQ(getDbgLevel(), 2);
EXPECT_EQ(getStaticResourcesPath(), static_resources_path);
EXPECT_EQ(isFailOpenMode(), 0);
EXPECT_EQ(getFailOpenTimeout(), 1234u);
EXPECT_EQ(getFailOpenTimeout(), 1234);
EXPECT_EQ(isFailOpenHoldMode(), 1);
EXPECT_EQ(getFailOpenHoldTimeout(), 4321u);
EXPECT_EQ(getFailOpenHoldTimeout(), 4321);
EXPECT_EQ(isFailOpenOnSessionLimit(), 1);
EXPECT_EQ(getMaxSessionsPerMinute(), 0u);
EXPECT_EQ(getNumOfNginxIpcElements(), 200u);
EXPECT_EQ(getKeepAliveIntervalMsec(), 10000u);
EXPECT_EQ(getResProccessingTimeout(), 420u);
EXPECT_EQ(getReqProccessingTimeout(), 42u);
EXPECT_EQ(getRegistrationThreadTimeout(), 101u);
EXPECT_EQ(getReqHeaderThreadTimeout(), 10u);
EXPECT_EQ(getReqBodyThreadTimeout(), 155u);
EXPECT_EQ(getResHeaderThreadTimeout(), 1u);
EXPECT_EQ(getResBodyThreadTimeout(), 0u);
EXPECT_EQ(getMinRetriesForVerdict(), 1u);
EXPECT_EQ(getMaxRetriesForVerdict(), 3u);
EXPECT_EQ(getReqBodySizeTrigger(), 777u);
EXPECT_EQ(getWaitingForVerdictThreadTimeout(), 75u);
EXPECT_EQ(getMaxSessionsPerMinute(), 0);
EXPECT_EQ(getNumOfNginxIpcElements(), 200);
EXPECT_EQ(getKeepAliveIntervalMsec(), 10000);
EXPECT_EQ(getResProccessingTimeout(), 420);
EXPECT_EQ(getReqProccessingTimeout(), 42);
EXPECT_EQ(getRegistrationThreadTimeout(), 101);
EXPECT_EQ(getReqHeaderThreadTimeout(), 10);
EXPECT_EQ(getReqBodyThreadTimeout(), 155);
EXPECT_EQ(getResHeaderThreadTimeout(), 1);
EXPECT_EQ(getResBodyThreadTimeout(), 0);
EXPECT_EQ(getMinRetriesForVerdict(), 1);
EXPECT_EQ(getMaxRetriesForVerdict(), 3);
EXPECT_EQ(getReqBodySizeTrigger(), 777);
EXPECT_EQ(getWaitingForVerdictThreadTimeout(), 75);
EXPECT_EQ(getInspectionMode(), ngx_http_inspection_mode::BLOCKING_THREAD);
EXPECT_EQ(getRemoveResServerHeader(), 1u);
EXPECT_EQ(getHoldVerdictRetries(), 3u);
EXPECT_EQ(getHoldVerdictPollingTime(), 1u);
EXPECT_EQ(isDebugContext("1.2.3.4", "5.6.7.8", 80, "GET", "test", "/abc"), 1);
EXPECT_EQ(isDebugContext("1.2.3.9", "5.6.7.8", 80, "GET", "test", "/abc"), 0);

View File

@@ -1,4 +1,4 @@
install(FILES Dockerfile entry.sh install-cp-agent-intelligence-service.sh install-cp-crowdsec-aux.sh self_managed_openappsec_manifest.json DESTINATION .)
install(FILES Dockerfile entry.sh install-cp-agent-intelligence-service.sh install-cp-crowdsec-aux.sh DESTINATION .)
add_custom_command(
OUTPUT ${CMAKE_INSTALL_PREFIX}/agent-docker.img

View File

@@ -1,7 +1,5 @@
FROM alpine
ENV OPENAPPSEC_NANO_AGENT=TRUE
RUN apk add --no-cache -u busybox
RUN apk add --no-cache -u zlib
RUN apk add --no-cache bash
@@ -13,12 +11,8 @@ RUN apk add --no-cache libunwind
RUN apk add --no-cache gdb
RUN apk add --no-cache libxml2
RUN apk add --no-cache pcre2
RUN apk add --no-cache ca-certificates
RUN apk add --update coreutils
COPY self_managed_openappsec_manifest.json /tmp/self_managed_openappsec_manifest.json
COPY install*.sh /nano-service-installers/
COPY entry.sh /entry.sh

View File

@@ -6,8 +6,6 @@ HTTP_TRANSACTION_HANDLER_SERVICE="install-cp-nano-service-http-transaction-handl
ATTACHMENT_REGISTRATION_SERVICE="install-cp-nano-attachment-registration-manager.sh"
ORCHESTRATION_INSTALLATION_SCRIPT="install-cp-nano-agent.sh"
CACHE_INSTALLATION_SCRIPT="install-cp-nano-agent-cache.sh"
PROMETHEUS_INSTALLATION_SCRIPT="install-cp-nano-service-prometheus.sh"
NGINX_CENTRAL_MANAGER_INSTALLATION_SCRIPT="install-cp-nano-central-nginx-manager.sh"
var_fog_address=
var_proxy=
@@ -46,12 +44,9 @@ while true; do
done
if [ -z $var_token ] && [ $var_mode != "--hybrid_mode" ]; then
var_token=$(env | grep 'AGENT_TOKEN=' | cut -d'=' -f2-)
if [ -z $var_token ]; then
echo "Error: Token was not provided as input argument."
exit 1
fi
fi
orchestration_service_installation_flags="--container_mode --skip_registration"
if [ ! -z $var_token ]; then
@@ -83,14 +78,6 @@ fi
/nano-service-installers/$CACHE_INSTALLATION_SCRIPT --install
/nano-service-installers/$HTTP_TRANSACTION_HANDLER_SERVICE --install
if [ "$PROMETHEUS" == "true" ]; then
/nano-service-installers/$PROMETHEUS_INSTALLATION_SCRIPT --install
fi
if [ "$CENTRAL_NGINX_MANAGER" == "true" ]; then
/nano-service-installers/$NGINX_CENTRAL_MANAGER_INSTALLATION_SCRIPT --install
fi
if [ "$CROWDSEC_ENABLED" == "true" ]; then
/nano-service-installers/$INTELLIGENCE_INSTALLATION_SCRIPT --install
/nano-service-installers/$CROWDSEC_INSTALLATION_SCRIPT --install
@@ -103,16 +90,25 @@ if [ -f "$FILE" ]; then
fi
touch /etc/cp/watchdog/wd.startup
/etc/cp/watchdog/cp-nano-watchdog >/dev/null 2>&1 &
active_watchdog_pid=$!
while true; do
if [ -f /tmp/restart_watchdog ]; then
rm -f /tmp/restart_watchdog
kill -9 ${active_watchdog_pid}
fi
if [ ! "$(ps -f | grep cp-nano-watchdog | grep ${active_watchdog_pid})" ]; then
if [ -z "$init" ]; then
init=true
/etc/cp/watchdog/cp-nano-watchdog >/dev/null 2>&1 &
active_watchdog_pid=$!
sleep 5
active_watchdog_pid=$(pgrep -f -x -o "/bin/bash /etc/cp/watchdog/cp-nano-watchdog")
fi
current_watchdog_pid=$(pgrep -f -x -o "/bin/bash /etc/cp/watchdog/cp-nano-watchdog")
if [ ! -f /tmp/restart_watchdog ] && [ "$current_watchdog_pid" != "$active_watchdog_pid" ]; then
echo "Error: Watchdog exited abnormally"
exit 1
elif [ -f /tmp/restart_watchdog ]; then
rm -f /tmp/restart_watchdog
kill -9 "$(pgrep -f -x -o "/bin/bash /etc/cp/watchdog/cp-nano-watchdog")"
/etc/cp/watchdog/cp-nano-watchdog >/dev/null 2>&1 &
sleep 5
active_watchdog_pid=$(pgrep -f -x -o "/bin/bash /etc/cp/watchdog/cp-nano-watchdog")
fi
sleep 5
done

File diff suppressed because one or more lines are too long

View File

@@ -1,3 +1,4 @@
add_subdirectory(report_messaging)
add_subdirectory(http_manager)
add_subdirectory(signal_handler)
add_subdirectory(gradual_deployment)
@@ -7,4 +8,3 @@ add_subdirectory(pending_key)
add_subdirectory(utils)
add_subdirectory(attachment-intakers)
add_subdirectory(security_apps)
add_subdirectory(nginx_message_reader)

View File

@@ -39,8 +39,6 @@ USE_DEBUG_FLAG(D_ATTACHMENT_REGISTRATION);
using namespace std;
static const AlertInfo alert(AlertTeam::CORE, "attachment registrator");
class AttachmentRegistrator::Impl
{
public:
@@ -165,7 +163,7 @@ private:
break;
}
default:
dbgAssert(false) << alert << "Unsupported Attachment " << static_cast<int>(type);
dbgAssert(false) << "Unsupported Attachment " << static_cast<int>(type);
}
if (!family_id.empty()) handler_path << family_id << "_";
@@ -177,9 +175,7 @@ private:
string
genRegCommand(const string &family_id, const uint num_of_members, const AttachmentType type) const
{
dbgAssert(num_of_members > 0)
<< alert
<< "Failed to generate a registration command for an empty group of attachments";
dbgAssert(num_of_members > 0) << "Failed to generate a registration command for an empty group of attachments";
static const string registration_format = "/etc/cp/watchdog/cp-nano-watchdog --register ";
stringstream registration_command;
@@ -191,7 +187,7 @@ private:
break;
}
default:
dbgAssert(false) << alert << "Unsupported Attachment " << static_cast<int>(type);
dbgAssert(false) << "Unsupported Attachment " << static_cast<int>(type);
}
if (!family_id.empty()) registration_command << " --family " << family_id;
@@ -269,7 +265,7 @@ private:
return -1;
}
dbgAssert(new_socket.unpack() > 0) << alert << "Generated socket is OK yet negative";
dbgAssert(new_socket.unpack() > 0) << "Generated socket is OK yet negative";
return new_socket.unpack();
}
@@ -285,7 +281,7 @@ private:
}
I_Socket::socketFd client_socket = accepted_socket.unpack();
dbgAssert(client_socket > 0) << alert << "Generated client socket is OK yet negative";
dbgAssert(client_socket > 0) << "Generated client socket is OK yet negative";
auto close_socket_on_exit = make_scope_exit([&]() { i_socket->closeSocket(client_socket); });
Maybe<uint8_t> attachment_id = readNumericParam(client_socket);
@@ -379,7 +375,7 @@ private:
}
I_Socket::socketFd client_socket = accepted_socket.unpack();
dbgAssert(client_socket > 0) << alert << "Generated client socket is OK yet negative";
dbgAssert(client_socket > 0) << "Generated client socket is OK yet negative";
auto close_socket_on_exit = make_scope_exit([&]() { i_socket->closeSocket(client_socket); });
Maybe<AttachmentType> attachment_type = readAttachmentType(client_socket);

View File

@@ -31,12 +31,10 @@
#include <stdarg.h>
#include <boost/range/iterator_range.hpp>
#include <boost/algorithm/string.hpp>
#include <boost/regex.hpp>
#include "nginx_attachment_config.h"
#include "nginx_attachment_opaque.h"
#include "generic_rulebase/evaluators/trigger_eval.h"
#include "nginx_parser.h"
#include "i_instance_awareness.h"
#include "common.h"
@@ -78,7 +76,6 @@ using namespace std;
using ChunkType = ngx_http_chunk_type_e;
static const uint32_t corrupted_session_id = CORRUPTED_SESSION_ID;
static const AlertInfo alert(AlertTeam::CORE, "nginx attachment");
class FailopenModeListener : public Listener<FailopenModeEvent>
{
@@ -131,7 +128,6 @@ class NginxAttachment::Impl
Singleton::Provide<I_StaticResourcesHandler>::From<NginxAttachment>
{
static constexpr auto INSPECT = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INSPECT;
static constexpr auto LIMIT_RESPONSE_HEADERS = ngx_http_cp_verdict_e::LIMIT_RESPONSE_HEADERS;
static constexpr auto ACCEPT = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_ACCEPT;
static constexpr auto DROP = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_DROP;
static constexpr auto INJECT = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INJECT;
@@ -263,22 +259,6 @@ public:
);
}
const char* ignored_headers_env = getenv("SAAS_IGNORED_UPSTREAM_HEADERS");
if (ignored_headers_env) {
string ignored_headers_str = ignored_headers_env;
ignored_headers_str = NGEN::Strings::trim(ignored_headers_str);
if (!ignored_headers_str.empty()) {
dbgInfo(D_HTTP_MANAGER)
<< "Ignoring SAAS_IGNORED_UPSTREAM_HEADERS environment variable: "
<< ignored_headers_str;
vector<string> ignored_headers_vec;
boost::split(ignored_headers_vec, ignored_headers_str, boost::is_any_of(";"));
for (const string &header : ignored_headers_vec) ignored_headers.insert(header);
}
}
dbgInfo(D_NGINX_ATTACHMENT) << "Successfully initialized NGINX Attachment";
}
@@ -430,10 +410,7 @@ private:
bool
registerAttachmentProcess(uint32_t nginx_user_id, uint32_t nginx_group_id, I_Socket::socketFd new_socket)
{
dbgAssert(server_sock > 0)
<< alert
<< "Registration attempt occurred while registration socket is uninitialized";
dbgAssert(server_sock > 0) << "Registration attempt occurred while registration socket is uninitialized";
#ifdef FAILURE_TEST
bool did_fail_on_purpose = false;
#endif
@@ -825,10 +802,10 @@ private:
case ChunkType::HOLD_DATA:
return "HOLD_DATA";
case ChunkType::COUNT:
dbgAssert(false) << alert << "Invalid 'COUNT' ChunkType";
dbgAssert(false) << "Invalid 'COUNT' ChunkType";
return "";
}
dbgAssert(false) << alert << "ChunkType was not handled by the switch case";
dbgAssert(false) << "ChunkType was not handled by the switch case";
return "";
}
@@ -1053,11 +1030,7 @@ private:
case ChunkType::REQUEST_START:
return handleStartTransaction(data, opaque);
case ChunkType::REQUEST_HEADER:
return handleMultiModifiableChunks(
NginxParser::parseRequestHeaders(data, ignored_headers),
"request header",
true
);
return handleMultiModifiableChunks(NginxParser::parseRequestHeaders(data), "request header", true);
case ChunkType::REQUEST_BODY:
return handleModifiableChunk(NginxParser::parseRequestBody(data), "request body", true);
case ChunkType::REQUEST_END: {
@@ -1148,29 +1121,17 @@ private:
handleCustomWebResponse(
SharedMemoryIPC *ipc,
vector<const char *> &verdict_data,
vector<uint16_t> &verdict_data_sizes,
string web_user_response_id)
vector<uint16_t> &verdict_data_sizes)
{
ngx_http_cp_web_response_data_t web_response_data;
ScopedContext ctx;
if (web_user_response_id != "") {
dbgTrace(D_NGINX_ATTACHMENT)
<< "web user response ID registered in contex: "
<< web_user_response_id;
set<string> triggers_set{web_user_response_id};
ctx.registerValue<set<GenericConfigId>>(TriggerMatcher::ctx_key, triggers_set);
}
WebTriggerConf web_trigger_conf = getConfigurationWithDefault<WebTriggerConf>(
WebTriggerConf::default_trigger_conf,
"rulebase",
"webUserResponse"
);
bool remove_event_id_param =
getProfileAgentSettingWithDefault<string>("false", "nginxAttachment.removeRedirectEventId") == "true";
string uuid;
string redirectUrl;
if (i_transaction_table->hasState<NginxAttachmentOpaque>()) {
NginxAttachmentOpaque &opaque = i_transaction_table->getState<NginxAttachmentOpaque>();
uuid = opaque.getSessionUUID();
@@ -1180,12 +1141,7 @@ private:
if (web_trigger_conf.getDetailsLevel() == "Redirect") {
web_response_data.response_data.redirect_data.redirect_location_size =
web_trigger_conf.getRedirectURL().size();
bool add_event = web_trigger_conf.getAddEventId();
if (add_event && !remove_event_id_param) {
web_response_data.response_data.redirect_data.redirect_location_size +=
strlen("?event_id=") + uuid.size();
}
web_response_data.response_data.redirect_data.add_event_id = add_event ? 1 : 0;
web_response_data.response_data.redirect_data.add_event_id = web_trigger_conf.getAddEventId() ? 1 : 0;
web_response_data.web_repsonse_type = static_cast<uint8_t>(ngx_web_response_type_e::REDIRECT_WEB_RESPONSE);
} else {
web_response_data.response_data.custom_response_data.title_size =
@@ -1199,13 +1155,8 @@ private:
verdict_data_sizes.push_back(sizeof(ngx_http_cp_web_response_data_t));
if (web_trigger_conf.getDetailsLevel() == "Redirect") {
redirectUrl = web_trigger_conf.getRedirectURL();
if (!remove_event_id_param && web_trigger_conf.getAddEventId()) {
redirectUrl += "?event-id=" + uuid;
}
verdict_data.push_back(reinterpret_cast<const char *>(redirectUrl.data()));
verdict_data_sizes.push_back(redirectUrl.size());
verdict_data.push_back(reinterpret_cast<const char *>(web_trigger_conf.getRedirectURL().data()));
verdict_data_sizes.push_back(web_trigger_conf.getRedirectURL().size());
} else {
verdict_data.push_back(reinterpret_cast<const char *>(web_trigger_conf.getResponseTitle().data()));
verdict_data_sizes.push_back(web_trigger_conf.getResponseTitle().size());
@@ -1281,7 +1232,7 @@ private:
if (verdict.getVerdict() == DROP) {
nginx_attachment_event.addTrafficVerdictCounter(nginxAttachmentEvent::trafficVerdict::DROP);
verdict_to_send.modification_count = 1;
return handleCustomWebResponse(ipc, verdict_fragments, fragments_sizes, verdict.getWebUserResponseID());
return handleCustomWebResponse(ipc, verdict_fragments, fragments_sizes);
}
if (verdict.getVerdict() == ACCEPT) {
@@ -1507,17 +1458,11 @@ private:
opaque.activateContext();
FilterVerdict verdict = handleChunkedData(*chunked_data_type, inspection_data, opaque);
bool is_header =
*chunked_data_type == ChunkType::REQUEST_HEADER ||
*chunked_data_type == ChunkType::RESPONSE_HEADER ||
*chunked_data_type == ChunkType::CONTENT_LENGTH;
if (verdict.getVerdict() == LIMIT_RESPONSE_HEADERS) {
handleVerdictResponse(verdict, attachment_ipc, transaction_data->session_id, is_header);
popData(attachment_ipc);
verdict = FilterVerdict(INSPECT);
}
handleVerdictResponse(verdict, attachment_ipc, transaction_data->session_id, is_header);
bool is_final_verdict = verdict.getVerdict() == ACCEPT ||
@@ -1630,8 +1575,6 @@ private:
return "INJECT";
case INSPECT:
return "INSPECT";
case LIMIT_RESPONSE_HEADERS:
return "LIMIT_RESPONSE_HEADERS";
case IRRELEVANT:
return "IRRELEVANT";
case RECONF:
@@ -1639,7 +1582,7 @@ private:
case WAIT:
return "WAIT";
}
dbgAssert(false) << alert << "Invalid EventVerdict enum: " << static_cast<int>(verdict.getVerdict());
dbgAssert(false) << "Invalid EventVerdict enum: " << static_cast<int>(verdict.getVerdict());
return string();
}
@@ -1690,14 +1633,13 @@ private:
return false;
}
dbgAssert(sock.unpack() > 0) << alert << "The generated server socket is OK, yet negative";
dbgAssert(sock.unpack() > 0) << "The generated server socket is OK, yet negative";
server_sock = sock.unpack();
I_MainLoop::Routine accept_attachment_routine =
[this] ()
{
dbgAssert(inst_awareness->getUniqueID().ok())
<< alert
<< "NGINX attachment Initialized without Instance Awareness";
bool did_fail_on_purpose = false;
@@ -1710,7 +1652,7 @@ private:
<< (did_fail_on_purpose ? "Intentional Failure" : new_sock.getErr());
return;
}
dbgAssert(new_sock.unpack() > 0) << alert << "The generated client socket is OK, yet negative";
dbgAssert(new_sock.unpack() > 0) << "The generated client socket is OK, yet negative";
I_Socket::socketFd new_attachment_socket = new_sock.unpack();
Maybe<string> uid = getUidFromSocket(new_attachment_socket);
@@ -1769,9 +1711,7 @@ private:
Maybe<string>
getUidFromSocket(I_Socket::socketFd new_attachment_socket)
{
dbgAssert(server_sock > 0)
<< alert
<< "Registration attempt occurred while registration socket is uninitialized";
dbgAssert(server_sock > 0) << "Registration attempt occurred while registration socket is uninitialized";
bool did_fail_on_purpose = false;
DELAY_IF_NEEDED(IntentionalFailureHandler::FailureType::ReceiveDataFromSocket);
@@ -1853,7 +1793,6 @@ private:
HttpAttachmentConfig attachment_config;
I_MainLoop::RoutineID attachment_routine_id = 0;
bool traffic_indicator = false;
unordered_set<string> ignored_headers;
// Interfaces
I_Socket *i_socket = nullptr;

View File

@@ -203,13 +203,6 @@ HttpAttachmentConfig::setFailOpenTimeout()
"NGINX wait thread timeout msec"
));
conf_data.setNumericalValue("remove_server_header", getAttachmentConf<uint>(
0,
"agent.removeServerHeader.nginxModule",
"HTTP manager",
"Response server header removal"
));
uint inspection_mode = getAttachmentConf<uint>(
static_cast<uint>(ngx_http_inspection_mode_e::NON_BLOCKING_THREAD),
"agent.inspectionMode.nginxModule",
@@ -240,21 +233,6 @@ HttpAttachmentConfig::setRetriesForVerdict()
"Max retries for verdict"
));
conf_data.setNumericalValue("hold_verdict_retries", getAttachmentConf<uint>(
3,
"agent.retriesForHoldVerdict.nginxModule",
"HTTP manager",
"Retries for hold verdict"
));
conf_data.setNumericalValue("hold_verdict_polling_time", getAttachmentConf<uint>(
1,
"agent.holdVerdictPollingInterval.nginxModule",
"HTTP manager",
"Hold verdict polling interval seconds"
));
conf_data.setNumericalValue("body_size_trigger", getAttachmentConf<uint>(
200000,
"agent.reqBodySizeTrigger.nginxModule",

View File

@@ -19,15 +19,12 @@
#include "config.h"
#include "virtual_modifiers.h"
#include "agent_core_utilities.h"
using namespace std;
using namespace boost::uuids;
USE_DEBUG_FLAG(D_HTTP_MANAGER);
extern bool is_keep_alive_ctx;
NginxAttachmentOpaque::NginxAttachmentOpaque(HttpTransactionData _transaction_data)
:
TableOpaqueSerialize<NginxAttachmentOpaque>(this),
@@ -70,12 +67,6 @@ NginxAttachmentOpaque::NginxAttachmentOpaque(HttpTransactionData _transaction_da
ctx.registerValue(HttpTransactionData::uri_query_decoded, decoded_url.substr(question_mark_location + 1));
}
ctx.registerValue(HttpTransactionData::uri_path_decoded, decoded_url);
// Register waf_tag from transaction data if available
const std::string& waf_tag = transaction_data.getWafTag();
if (!waf_tag.empty()) {
ctx.registerValue(HttpTransactionData::waf_tag_ctx, waf_tag);
}
}
NginxAttachmentOpaque::~NginxAttachmentOpaque()
@@ -128,47 +119,3 @@ NginxAttachmentOpaque::setSavedData(const string &name, const string &data, EnvK
saved_data[name] = data;
ctx.registerValue(name, data, log_ctx);
}
bool
NginxAttachmentOpaque::setKeepAliveCtx(const string &hdr_key, const string &hdr_val)
{
if (!is_keep_alive_ctx) return false;
static pair<string, string> keep_alive_hdr;
static bool keep_alive_hdr_initialized = false;
if (keep_alive_hdr_initialized) {
if (!keep_alive_hdr.first.empty() && hdr_key == keep_alive_hdr.first && hdr_val == keep_alive_hdr.second) {
dbgTrace(D_HTTP_MANAGER) << "Registering keep alive context";
ctx.registerValue("keep_alive_request_ctx", true);
return true;
}
return false;
}
const char* saas_keep_alive_hdr_name_env = getenv("SAAS_KEEP_ALIVE_HDR_NAME");
if (saas_keep_alive_hdr_name_env) {
keep_alive_hdr.first = NGEN::Strings::trim(saas_keep_alive_hdr_name_env);
dbgInfo(D_HTTP_MANAGER) << "Using SAAS_KEEP_ALIVE_HDR_NAME environment variable: " << keep_alive_hdr.first;
}
if (!keep_alive_hdr.first.empty()) {
const char* saas_keep_alive_hdr_value_env = getenv("SAAS_KEEP_ALIVE_HDR_VALUE");
if (saas_keep_alive_hdr_value_env) {
keep_alive_hdr.second = NGEN::Strings::trim(saas_keep_alive_hdr_value_env);
dbgInfo(D_HTTP_MANAGER)
<< "Using SAAS_KEEP_ALIVE_HDR_VALUE environment variable: "
<< keep_alive_hdr.second;
}
if (!keep_alive_hdr.second.empty() && (hdr_key == keep_alive_hdr.first && hdr_val == keep_alive_hdr.second)) {
dbgTrace(D_HTTP_MANAGER) << "Registering keep alive context";
ctx.registerValue("keep_alive_request_ctx", true);
keep_alive_hdr_initialized = true;
return true;
}
}
keep_alive_hdr_initialized = true;
return false;
}

View File

@@ -85,7 +85,6 @@ public:
EnvKeyAttr::LogSection log_ctx = EnvKeyAttr::LogSection::NONE
);
void setApplicationState(const ApplicationState &app_state) { application_state = app_state; }
bool setKeepAliveCtx(const std::string &hdr_key, const std::string &hdr_val);
private:
CompressionStream *response_compression_stream;

View File

@@ -29,7 +29,6 @@ USE_DEBUG_FLAG(D_NGINX_ATTACHMENT_PARSER);
Buffer NginxParser::tenant_header_key = Buffer();
static const Buffer proxy_ip_header_key("X-Forwarded-For", 15, Buffer::MemoryType::STATIC);
static const Buffer source_ip("sourceip", 8, Buffer::MemoryType::STATIC);
bool is_keep_alive_ctx = getenv("SAAS_KEEP_ALIVE_HDR_NAME") != nullptr;
map<Buffer, CompressionType> NginxParser::content_encodings = {
{Buffer("identity"), CompressionType::NO_COMPRESSION},
@@ -178,70 +177,37 @@ getActivetenantAndProfile(const string &str, const string &deli = ",")
}
Maybe<vector<HttpHeader>>
NginxParser::parseRequestHeaders(const Buffer &data, const unordered_set<string> &ignored_headers)
NginxParser::parseRequestHeaders(const Buffer &data)
{
auto maybe_parsed_headers = genHeaders(data);
if (!maybe_parsed_headers.ok()) return maybe_parsed_headers.passErr();
auto parsed_headers = genHeaders(data);
if (!parsed_headers.ok()) return parsed_headers.passErr();
auto i_transaction_table = Singleton::Consume<I_TableSpecific<SessionID>>::by<NginxAttachment>();
auto parsed_headers = maybe_parsed_headers.unpack();
NginxAttachmentOpaque &opaque = i_transaction_table->getState<NginxAttachmentOpaque>();
if (is_keep_alive_ctx || !ignored_headers.empty()) {
bool is_last_header_removed = false;
parsed_headers.erase(
remove_if(
parsed_headers.begin(),
parsed_headers.end(),
[&opaque, &is_last_header_removed, &ignored_headers](const HttpHeader &header)
{
string hdr_key = static_cast<string>(header.getKey());
string hdr_val = static_cast<string>(header.getValue());
if (
opaque.setKeepAliveCtx(hdr_key, hdr_val)
|| ignored_headers.find(hdr_key) != ignored_headers.end()
) {
dbgTrace(D_NGINX_ATTACHMENT_PARSER) << "Header was removed from headers list: " << hdr_key;
if (header.isLastHeader()) {
dbgTrace(D_NGINX_ATTACHMENT_PARSER) << "Last header was removed from headers list";
is_last_header_removed = true;
}
return true;
}
return false;
}
),
parsed_headers.end()
);
if (is_last_header_removed) {
dbgTrace(D_NGINX_ATTACHMENT_PARSER) << "Adjusting last header flag";
if (!parsed_headers.empty()) parsed_headers.back().setIsLastHeader();
}
}
for (const HttpHeader &header : parsed_headers) {
for (const HttpHeader &header : *parsed_headers) {
auto source_identifiers = getConfigurationWithDefault<UsersAllIdentifiersConfig>(
UsersAllIdentifiersConfig(),
"rulebase",
"usersIdentifiers"
);
source_identifiers.parseRequestHeaders(header);
NginxAttachmentOpaque &opaque = i_transaction_table->getState<NginxAttachmentOpaque>();
opaque.addToSavedData(
HttpTransactionData::req_headers,
static_cast<string>(header.getKey()) + ": " + static_cast<string>(header.getValue()) + "\r\n"
);
const auto &header_key = header.getKey();
if (NginxParser::tenant_header_key == header_key) {
if (NginxParser::tenant_header_key == header.getKey()) {
dbgDebug(D_NGINX_ATTACHMENT_PARSER)
<< "Identified active tenant header. Key: "
<< dumpHex(header_key)
<< dumpHex(header.getKey())
<< ", Value: "
<< dumpHex(header.getValue());
auto active_tenant_and_profile = getActivetenantAndProfile(header.getValue());
opaque.setSessionTenantAndProfile(active_tenant_and_profile[0], active_tenant_and_profile[1]);
} else if (proxy_ip_header_key == header_key) {
} else if (proxy_ip_header_key == header.getKey()) {
source_identifiers.setXFFValuesToOpaqueCtx(header, UsersAllIdentifiersConfig::ExtractType::PROXYIP);
}
}
@@ -379,15 +345,12 @@ NginxParser::parseResponseBody(const Buffer &raw_response_body, CompressionStrea
Maybe<CompressionType>
NginxParser::parseContentEncoding(const vector<HttpHeader> &headers)
{
dbgFlow(D_NGINX_ATTACHMENT_PARSER) << "Parsing \"Content-Encoding\" header";
static const Buffer content_encoding_header_key("content-encoding");
static const Buffer content_encoding_header_key("Content-Encoding");
auto it = find_if(
headers.begin(),
headers.end(),
[&] (const HttpHeader &http_header) {
return http_header.getKey().isEqualLowerCase(content_encoding_header_key);
}
[&] (const HttpHeader &http_header) { return http_header.getKey() == content_encoding_header_key; }
);
if (it == headers.end()) {
dbgTrace(D_NGINX_ATTACHMENT_PARSER)

View File

@@ -28,10 +28,7 @@ public:
static Maybe<HttpTransactionData> parseStartTrasaction(const Buffer &data);
static Maybe<ResponseCode> parseResponseCode(const Buffer &data);
static Maybe<uint64_t> parseContentLength(const Buffer &data);
static Maybe<std::vector<HttpHeader>> parseRequestHeaders(
const Buffer &data,
const std::unordered_set<std::string> &ignored_headers
);
static Maybe<std::vector<HttpHeader>> parseRequestHeaders(const Buffer &data);
static Maybe<std::vector<HttpHeader>> parseResponseHeaders(const Buffer &data);
static Maybe<HttpBody> parseRequestBody(const Buffer &data);
static Maybe<HttpBody> parseResponseBody(const Buffer &raw_response_body, CompressionStream *compression_stream);

View File

@@ -282,39 +282,21 @@ isIpTrusted(const string &value, const vector<CIDRSData> &cidr_values)
}
Maybe<string>
UsersAllIdentifiersConfig::parseXForwardedFor(const string &str, ExtractType type) const
UsersAllIdentifiersConfig::parseXForwardedFor(const string &str) const
{
vector<string> header_values = split(str);
if (header_values.empty()) return genError("No IP found in the xff header list");
vector<string> xff_values = getHeaderValuesFromConfig("x-forwarded-for");
vector<CIDRSData> cidr_values(xff_values.begin(), xff_values.end());
string last_valid_ip;
for (auto it = header_values.rbegin(); it != header_values.rend() - 1; ++it) {
if (!IPAddr::createIPAddr(*it).ok()) {
dbgWarning(D_NGINX_ATTACHMENT_PARSER) << "Invalid IP address found in the xff header IPs list: " << *it;
if (last_valid_ip.empty()) {
for (const string &value : header_values) {
if (!IPAddr::createIPAddr(value).ok()) {
dbgWarning(D_NGINX_ATTACHMENT_PARSER) << "Invalid IP address found in the xff header IPs list: " << value;
return genError("Invalid IP address");
}
return last_valid_ip;
}
last_valid_ip = *it;
if (type == ExtractType::PROXYIP) continue;
if (!isIpTrusted(*it, cidr_values)) {
dbgDebug(D_NGINX_ATTACHMENT_PARSER) << "Found untrusted IP in the xff header IPs list: " << *it;
return *it;
}
}
if (!IPAddr::createIPAddr(header_values[0]).ok()) {
dbgWarning(D_NGINX_ATTACHMENT_PARSER)
<< "Invalid IP address found in the xff header IPs list: "
<< header_values[0];
if (last_valid_ip.empty()) {
return genError("No Valid Ip address was found");
}
return last_valid_ip;
if (!isIpTrusted(value, cidr_values)) return genError("Untrusted Ip found");
}
return header_values[0];
@@ -330,7 +312,9 @@ UsersAllIdentifiersConfig::setXFFValuesToOpaqueCtx(const HttpHeader &header, Ext
return;
}
NginxAttachmentOpaque &opaque = i_transaction_table->getState<NginxAttachmentOpaque>();
auto value = parseXForwardedFor(header.getValue(), type);
opaque.setSavedData(HttpTransactionData::xff_vals_ctx, header.getValue());
dbgTrace(D_NGINX_ATTACHMENT_PARSER) << "xff found, value from header: " << static_cast<string>(header.getValue());
auto value = parseXForwardedFor(header.getValue());
if (!value.ok()) {
dbgTrace(D_NGINX_ATTACHMENT_PARSER) << "Could not extract source identifier from X-Forwarded-For header";
return;
@@ -339,13 +323,8 @@ UsersAllIdentifiersConfig::setXFFValuesToOpaqueCtx(const HttpHeader &header, Ext
if (type == ExtractType::SOURCEIDENTIFIER) {
opaque.setSourceIdentifier(header.getKey(), value.unpack());
dbgDebug(D_NGINX_ATTACHMENT_PARSER)
<< "Added source identifier from XFF header"
<< "Added source identifir to XFF "
<< value.unpack();
opaque.setSavedData(HttpTransactionData::xff_vals_ctx, header.getValue());
opaque.setSavedData(HttpTransactionData::source_identifier, value.unpack());
dbgTrace(D_NGINX_ATTACHMENT_PARSER)
<< "XFF found, set ctx with value from header: "
<< static_cast<string>(header.getValue());
} else {
opaque.setSavedData(HttpTransactionData::proxy_ip_ctx, value.unpack());
}
@@ -366,24 +345,6 @@ UsersAllIdentifiersConfig::setCustomHeaderToOpaqueCtx(const HttpHeader &header)
return;
}
void
UsersAllIdentifiersConfig::setWafTagValuesToOpaqueCtx(const HttpHeader &header) const
{
auto i_transaction_table = Singleton::Consume<I_TableSpecific<SessionID>>::by<NginxAttachment>();
if (!i_transaction_table || !i_transaction_table->hasState<NginxAttachmentOpaque>()) {
dbgDebug(D_NGINX_ATTACHMENT_PARSER) << "Can't get the transaction table";
return;
}
NginxAttachmentOpaque &opaque = i_transaction_table->getState<NginxAttachmentOpaque>();
opaque.setSavedData(HttpTransactionData::waf_tag_ctx, static_cast<string>(header.getValue()));
dbgDebug(D_NGINX_ATTACHMENT_PARSER)
<< "Added waf tag to context: "
<< static_cast<string>(header.getValue());
return;
}
Maybe<string>
UsersAllIdentifiersConfig::parseCookieElement(
const string::const_iterator &start,

View File

@@ -128,7 +128,7 @@ private:
break;
}
default:
dbgAssert(false) << AlertInfo(AlertTeam::CORE, "gradual deployment") << "Unsupported IP type";
dbgAssert(false) << "Unsupported IP type";
}
return address;
}
@@ -142,7 +142,7 @@ private:
if (temp_params_list.size() == 1) {
Maybe<IPAddr> maybe_ip = IPAddr::createIPAddr(temp_params_list[0]);
if (!maybe_ip.ok()) return genError("Could not create IP address, " + maybe_ip.getErr());
IpAddress addr = ConvertToIpAddress(maybe_ip.unpackMove());
IpAddress addr = move(ConvertToIpAddress(maybe_ip.unpackMove()));
return move(IPRange{.start = addr, .end = addr});
}
@@ -157,11 +157,11 @@ private:
IPAddr max_addr = maybe_ip_max.unpackMove();
if (min_addr > max_addr) return genError("Could not create ip range - start greater then end");
IpAddress addr_min = ConvertToIpAddress(move(min_addr));
IpAddress addr_max = ConvertToIpAddress(move(max_addr));
IpAddress addr_min = move(ConvertToIpAddress(move(min_addr)));
IpAddress addr_max = move(ConvertToIpAddress(move(max_addr)));
if (addr_max.ip_type != addr_min.ip_type) return genError("Range IP's type does not match");
return IPRange{.start = move(addr_min), .end = move(addr_max)};
return move(IPRange{.start = move(addr_min), .end = move(addr_max)});
}
return genError("Illegal range received: " + range);

View File

@@ -15,18 +15,19 @@
#include <string>
#include <map>
#include <sys/stat.h>
#include <climits>
#include <unordered_map>
#include <unordered_set>
#include <boost/algorithm/string.hpp>
#include <boost/range/iterator_range.hpp>
#include <fstream>
#include <algorithm>
#include "common.h"
#include "config.h"
#include "table_opaque.h"
#include "http_manager_opaque.h"
#include "log_generator.h"
#include "http_inspection_events.h"
#include "agent_core_utilities.h"
USE_DEBUG_FLAG(D_HTTP_MANAGER);
@@ -37,7 +38,6 @@ operator<<(ostream &os, const EventVerdict &event)
{
switch (event.getVerdict()) {
case ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INSPECT: return os << "Inspect";
case ngx_http_cp_verdict_e::LIMIT_RESPONSE_HEADERS: return os << "Limit Response Headers";
case ngx_http_cp_verdict_e::TRAFFIC_VERDICT_ACCEPT: return os << "Accept";
case ngx_http_cp_verdict_e::TRAFFIC_VERDICT_DROP: return os << "Drop";
case ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INJECT: return os << "Inject";
@@ -46,10 +46,7 @@ operator<<(ostream &os, const EventVerdict &event)
case ngx_http_cp_verdict_e::TRAFFIC_VERDICT_WAIT: return os << "Wait";
}
dbgAssert(false)
<< AlertInfo(AlertTeam::CORE, "http manager")
<< "Illegal Event Verdict value: "
<< static_cast<uint>(event.getVerdict());
dbgAssert(false) << "Illegal Event Verdict value: " << static_cast<uint>(event.getVerdict());
return os;
}
@@ -94,14 +91,12 @@ public:
ctx.registerValue(app_sec_marker_key, i_transaction_table->keyToString(), EnvKeyAttr::LogSection::MARKER);
HttpManagerOpaque &state = i_transaction_table->getState<HttpManagerOpaque>();
const auto &custom_header = getProfileAgentSettingWithDefault<string>("", "agent.customHeaderValueLogging");
if (event.getKey().isEqualLowerCase(custom_header)) {
string event_key = static_cast<string>(event.getKey());
if (event_key == getProfileAgentSettingWithDefault<string>("", "agent.customHeaderValueLogging")) {
string event_value = static_cast<string>(event.getValue());
dbgTrace(D_HTTP_MANAGER)
<< "Found header key and value - ("
<< custom_header
<< event_key
<< ": "
<< event_value
<< ") that matched agent settings";
@@ -197,6 +192,7 @@ public:
if (state.getUserDefinedValue().ok()) {
ctx.registerValue("UserDefined", state.getUserDefinedValue().unpack(), EnvKeyAttr::LogSection::DATA);
}
return handleEvent(EndRequestEvent().performNamedQuery());
}
@@ -324,13 +320,9 @@ private:
<< respond.second.getVerdict();
state.setApplicationVerdict(respond.first, respond.second.getVerdict());
state.setApplicationWebResponse(respond.first, respond.second.getWebUserResponseByPractice());
}
FilterVerdict aggregated_verdict(state.getCurrVerdict(), state.getCurrWebUserResponse());
if (aggregated_verdict.getVerdict() == ngx_http_cp_verdict_e::TRAFFIC_VERDICT_DROP) {
SecurityAppsDropEvent(state.getCurrentDropVerdictCausers()).notify();
}
return aggregated_verdict;
return state.getCurrVerdict();
}
static void

View File

@@ -32,13 +32,6 @@ HttpManagerOpaque::setApplicationVerdict(const string &app_name, ngx_http_cp_ver
applications_verdicts[app_name] = verdict;
}
void
HttpManagerOpaque::setApplicationWebResponse(const string &app_name, string web_user_response_id)
{
dbgTrace(D_HTTP_MANAGER) << "Security app: " << app_name << ", has web user response: " << web_user_response_id;
applications_web_user_response[app_name] = web_user_response_id;
}
ngx_http_cp_verdict_e
HttpManagerOpaque::getApplicationsVerdict(const string &app_name) const
{
@@ -58,12 +51,8 @@ HttpManagerOpaque::getCurrVerdict() const
for (const auto &app_verdic_pair : applications_verdicts) {
switch (app_verdic_pair.second) {
case ngx_http_cp_verdict_e::TRAFFIC_VERDICT_DROP:
dbgTrace(D_HTTP_MANAGER) << "Verdict DROP for app: " << app_verdic_pair.first;
current_web_user_response = applications_web_user_response.at(app_verdic_pair.first);
dbgTrace(D_HTTP_MANAGER) << "current_web_user_response=" << current_web_user_response;
return app_verdic_pair.second;
case ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INJECT:
// Sent in ResponseHeaders and ResponseBody.
verdict = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INJECT;
break;
case ngx_http_cp_verdict_e::TRAFFIC_VERDICT_ACCEPT:
@@ -71,21 +60,15 @@ HttpManagerOpaque::getCurrVerdict() const
break;
case ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INSPECT:
break;
case ngx_http_cp_verdict_e::LIMIT_RESPONSE_HEADERS:
// Sent in End Request.
verdict = ngx_http_cp_verdict_e::LIMIT_RESPONSE_HEADERS;
break;
case ngx_http_cp_verdict_e::TRAFFIC_VERDICT_IRRELEVANT:
dbgTrace(D_HTTP_MANAGER) << "Verdict 'Irrelevant' is not yet supported. Returning Accept";
accepted_apps++;
break;
case ngx_http_cp_verdict_e::TRAFFIC_VERDICT_WAIT:
// Sent in Request Headers and Request Body.
verdict = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_WAIT;
break;
default:
dbgAssert(false)
<< AlertInfo(AlertTeam::CORE, "http manager")
<< "Received unknown verdict "
<< static_cast<int>(app_verdic_pair.second);
}
@@ -94,25 +77,6 @@ HttpManagerOpaque::getCurrVerdict() const
return accepted_apps == applications_verdicts.size() ? ngx_http_cp_verdict_e::TRAFFIC_VERDICT_ACCEPT : verdict;
}
std::set<std::string>
HttpManagerOpaque::getCurrentDropVerdictCausers() const
{
std::set<std::string> causers;
if (manager_verdict == ngx_http_cp_verdict_e::TRAFFIC_VERDICT_DROP) {
causers.insert(HTTP_MANAGER_NAME);
}
for (const auto &app_verdic_pair : applications_verdicts) {
bool was_dropped = app_verdic_pair.second == ngx_http_cp_verdict_e::TRAFFIC_VERDICT_DROP;
dbgTrace(D_HTTP_MANAGER)
<< "The verdict from: " << app_verdic_pair.first
<< (was_dropped ? " is \"drop\"" : " is not \"drop\" ");
if (was_dropped) {
causers.insert(app_verdic_pair.first);
}
}
return causers;
}
void
HttpManagerOpaque::saveCurrentDataToCache(const Buffer &full_data)
{

View File

@@ -20,21 +20,16 @@
#include "table_opaque.h"
#include "nginx_attachment_common.h"
static const std::string HTTP_MANAGER_NAME = "HTTP Manager";
class HttpManagerOpaque : public TableOpaqueSerialize<HttpManagerOpaque>
{
public:
HttpManagerOpaque();
void setApplicationVerdict(const std::string &app_name, ngx_http_cp_verdict_e verdict);
void setApplicationWebResponse(const std::string &app_name, std::string web_user_response_id);
ngx_http_cp_verdict_e getApplicationsVerdict(const std::string &app_name) const;
void setManagerVerdict(ngx_http_cp_verdict_e verdict) { manager_verdict = verdict; }
ngx_http_cp_verdict_e getManagerVerdict() const { return manager_verdict; }
ngx_http_cp_verdict_e getCurrVerdict() const;
const std::string & getCurrWebUserResponse() const { return current_web_user_response; };
std::set<std::string> getCurrentDropVerdictCausers() const;
void saveCurrentDataToCache(const Buffer &full_data);
void setUserDefinedValue(const std::string &value) { user_defined_value = value; }
Maybe<std::string> getUserDefinedValue() const { return user_defined_value; }
@@ -54,8 +49,6 @@ public:
private:
std::unordered_map<std::string, ngx_http_cp_verdict_e> applications_verdicts;
std::unordered_map<std::string, std::string> applications_web_user_response;
mutable std::string current_web_user_response;
ngx_http_cp_verdict_e manager_verdict = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INSPECT;
Buffer prev_data_cache;
uint aggregated_payload_size = 0;

View File

@@ -1,45 +0,0 @@
// Copyright (C) 2022 Check Point Software Technologies Ltd. All rights reserved.
// Licensed under the Apache License, Version 2.0 (the "License");
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#ifndef __CENTRAL_NGINX_MANAGER_H__
#define __CENTRAL_NGINX_MANAGER_H__
#include "component.h"
#include "singleton.h"
#include "i_messaging.h"
#include "i_rest_api.h"
#include "i_mainloop.h"
#include "i_agent_details.h"
class CentralNginxManager
:
public Component,
Singleton::Consume<I_RestApi>,
Singleton::Consume<I_Messaging>,
Singleton::Consume<I_MainLoop>,
Singleton::Consume<I_AgentDetails>
{
public:
CentralNginxManager();
~CentralNginxManager();
void preload() override;
void init() override;
void fini() override;
private:
class Impl;
std::unique_ptr<Impl> pimpl;
};
#endif // __CENTRAL_NGINX_MANAGER_H__

View File

@@ -21,29 +21,21 @@
#include "i_env_details.h"
#include "singleton.h"
#include "debug.h"
#include "component.h"
class EnvDetails
:
public Component,
Singleton::Provide<I_EnvDetails>::SelfInterface
class EnvDetails : Singleton::Provide<I_EnvDetails>::SelfInterface
{
public:
EnvDetails();
virtual EnvType getEnvType() override;
virtual std::string getToken() override;
virtual std::string getNameSpace() override;
private:
std::string retrieveToken();
std::string retrieveNamespace();
std::string readFileContent(const std::string &file_path);
bool doesFileExist(const std::string &file_path) const;
std::string token;
std::string agent_namespace;
EnvType env_type = EnvType::LINUX;
EnvType env_type;
};
#endif // __ENV_DETAILS_H__

View File

@@ -24,8 +24,7 @@ class ExternalSdkServer
:
public Component,
Singleton::Provide<I_ExternalSdkServer>,
Singleton::Consume<I_RestApi>,
Singleton::Consume<I_Messaging>
Singleton::Consume<I_RestApi>
{
public:
ExternalSdkServer();

View File

@@ -45,19 +45,6 @@ private:
std::string host;
};
class EqualWafTag : public EnvironmentEvaluator<bool>, Singleton::Consume<I_Environment>
{
public:
EqualWafTag(const std::vector<std::string> &params);
static std::string getName() { return "EqualWafTag"; }
Maybe<bool, Context::Error> evalVariable() const override;
private:
std::string waf_tag;
};
class EqualListeningIP : public EnvironmentEvaluator<bool>, Singleton::Consume<I_Environment>
{
public:

View File

@@ -89,9 +89,7 @@ private:
bool matchAttributesRegEx(const std::set<std::string> &values,
std::set<std::string> &matched_override_keywords) const;
bool matchAttributesString(const std::set<std::string> &values) const;
bool matchAttributesIp(const std::set<std::string> &values) const;
bool isRegEx() const;
void sortAndMergeIpRangesValues();
MatchType type;
Operators operator_type;

View File

@@ -317,12 +317,12 @@ public:
{
return url_for_cef;
}
Flags<ReportIS::StreamType> getStreams(SecurityType security_type, bool is_action_drop_or_prevent) const;
Flags<ReportIS::Enreachments> getEnrechments(SecurityType security_type) const;
private:
ReportIS::Severity getSeverity(bool is_action_drop_or_prevent) const;
ReportIS::Priority getPriority(bool is_action_drop_or_prevent) const;
Flags<ReportIS::StreamType> getStreams(SecurityType security_type, bool is_action_drop_or_prevent) const;
Flags<ReportIS::Enreachments> getEnrechments(SecurityType security_type) const;
std::string name;
std::string verbosity;
@@ -339,32 +339,4 @@ private:
bool should_format_output = false;
};
class ReportTriggerConf
{
public:
/// \brief Default constructor for ReportTriggerConf.
ReportTriggerConf() {}
/// \brief Preload function to register expected configuration.
static void
preload()
{
registerExpectedConfiguration<ReportTriggerConf>("rulebase", "report");
}
/// \brief Load function to deserialize configuration from JSONInputArchive.
/// \param archive_in The JSON input archive.
void load(cereal::JSONInputArchive &archive_in);
/// \brief Get the name.
/// \return The name.
const std::string &
getName() const
{
return name;
}
private:
std::string name;
};
#endif //__TRIGGERS_CONFIG_H__

View File

@@ -21,7 +21,6 @@
#include "i_shell_cmd.h"
#include "i_orchestration_status.h"
#include "component.h"
#include "i_service_controller.h"
class HealthChecker
:
@@ -30,8 +29,7 @@ class HealthChecker
Singleton::Consume<I_Socket>,
Singleton::Consume<I_Health_Check_Manager>,
Singleton::Consume<I_ShellCmd>,
Singleton::Consume<I_OrchestrationStatus>,
Singleton::Consume<I_ServiceController>
Singleton::Consume<I_OrchestrationStatus>
{
public:
HealthChecker();

View File

@@ -27,18 +27,9 @@ public:
verdict(_verdict)
{}
FilterVerdict(
ngx_http_cp_verdict_e _verdict,
const std::string &_web_reponse_id)
:
verdict(_verdict),
web_user_response_id(_web_reponse_id)
{}
FilterVerdict(const EventVerdict &_verdict, ModifiedChunkIndex _event_idx = -1)
:
verdict(_verdict.getVerdict()),
web_user_response_id(_verdict.getWebUserResponseByPractice())
verdict(_verdict.getVerdict())
{
if (verdict == ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INJECT) {
addModifications(_verdict.getModifications(), _event_idx);
@@ -68,12 +59,10 @@ public:
uint getModificationsAmount() const { return total_modifications; }
ngx_http_cp_verdict_e getVerdict() const { return verdict; }
const std::vector<EventModifications> & getModifications() const { return modifications; }
const std::string getWebUserResponseID() const { return web_user_response_id; }
private:
ngx_http_cp_verdict_e verdict = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INSPECT;
std::vector<EventModifications> modifications;
std::string web_user_response_id;
uint total_modifications = 0;
};

View File

@@ -50,11 +50,9 @@ public:
position(mod_position)
{
dbgAssert(mod_type != ModificationType::APPEND || position == injection_pos_irrelevant)
<< AlertInfo(AlertTeam::CORE, "http manager")
<< "Injection position is not applicable to a modification of type \"Append\"";
dbgAssert(mod_type != ModificationType::INJECT || position >= 0)
<< AlertInfo(AlertTeam::CORE, "http manager")
<< "Invalid injection position: must be non-negative. Position: "
<< position;
}
@@ -168,7 +166,6 @@ private:
}
default:
dbgAssert(false)
<< AlertInfo(AlertTeam::CORE, "http manager")
<< "Unknown type of ModificationType: "
<< static_cast<int>(modification_type);
}
@@ -239,7 +236,6 @@ public:
const Buffer & getValue() const { return value; }
bool isLastHeader() const { return is_last_header; }
void setIsLastHeader() { is_last_header = true; }
uint8_t getHeaderIndex() const { return header_index; }
private:
@@ -376,31 +372,16 @@ public:
verdict(event_verdict)
{}
EventVerdict(
const ModificationList &mods,
ngx_http_cp_verdict_e event_verdict,
std::string response_id) :
modifications(mods),
verdict(event_verdict),
webUserResponseByPractice(response_id)
{}
// LCOV_EXCL_START - sync functions, can only be tested once the sync module exists
template <typename T> void serialize(T &ar, uint) { ar(verdict); }
// LCOV_EXCL_STOP
const ModificationList & getModifications() const { return modifications; }
ngx_http_cp_verdict_e getVerdict() const { return verdict; }
const std::string getWebUserResponseByPractice() const { return webUserResponseByPractice; }
void setWebUserResponseByPractice(const std::string id) {
dbgTrace(D_HTTP_MANAGER) << "current verdict web user response set to: " << id;
webUserResponseByPractice = id;
}
private:
ModificationList modifications;
ngx_http_cp_verdict_e verdict = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INSPECT;
std::string webUserResponseByPractice;
};
#endif // __I_HTTP_EVENT_IMPL_H__

View File

@@ -183,16 +183,4 @@ class WaitTransactionEvent : public Event<WaitTransactionEvent, EventVerdict>
{
};
class SecurityAppsDropEvent : public Event<SecurityAppsDropEvent>
{
public:
SecurityAppsDropEvent(
const std::set<std::string> &apps_names)
:
apps_names(apps_names) {}
const std::set<std::string> & getAppsNames() const { return apps_names; }
private:
const std::set<std::string> apps_names;
};
#endif // __HTTP_INSPECTION_EVENTS_H__

View File

@@ -72,8 +72,7 @@ public:
parsed_uri,
client_ip,
client_port,
response_content_encoding,
waf_tag
response_content_encoding
);
}
@@ -92,8 +91,7 @@ public:
parsed_uri,
client_ip,
client_port,
response_content_encoding,
waf_tag
response_content_encoding
);
}
// LCOV_EXCL_STOP
@@ -124,9 +122,6 @@ public:
response_content_encoding = _response_content_encoding;
}
const std::string & getWafTag() const { return waf_tag; }
void setWafTag(const std::string &_waf_tag) { waf_tag = _waf_tag; }
static const std::string http_proto_ctx;
static const std::string method_ctx;
static const std::string host_name_ctx;
@@ -142,7 +137,6 @@ public:
static const std::string source_identifier;
static const std::string proxy_ip_ctx;
static const std::string xff_vals_ctx;
static const std::string waf_tag_ctx;
static const CompressionType default_response_content_encoding;
@@ -159,7 +153,6 @@ private:
uint16_t client_port;
bool is_request;
CompressionType response_content_encoding;
std::string waf_tag;
};
#endif // __HTTP_TRANSACTION_DATA_H__

View File

@@ -26,12 +26,11 @@ public:
virtual Maybe<std::string> getArch() = 0;
virtual std::string getAgentVersion() = 0;
virtual bool isKernelVersion3OrHigher() = 0;
virtual bool isGw() = 0;
virtual bool isGwNotVsx() = 0;
virtual bool isVersionAboveR8110() = 0;
virtual bool isReverseProxy() = 0;
virtual bool isCloudStorageEnabled() = 0;
virtual Maybe<std::tuple<std::string, std::string, std::string, std::string>> parseNginxMetadata() = 0;
virtual Maybe<std::tuple<std::string, std::string, std::string>> parseNginxMetadata() = 0;
virtual Maybe<std::tuple<std::string, std::string, std::string, std::string, std::string>> readCloudMetadata() = 0;
virtual std::map<std::string, std::string> getResolvedDetails() = 0;
#if defined(gaia) || defined(smb)

View File

@@ -66,8 +66,6 @@ public:
virtual std::map<std::string, std::vector<PortNumber>> getServiceToPortMap() = 0;
virtual bool getServicesPolicyStatus() const = 0;
protected:
virtual ~I_ServiceController() {}
};

View File

@@ -27,7 +27,6 @@ struct DecisionTelemetryData
int responseCode;
uint64_t elapsedTime;
std::set<std::string> attackTypes;
bool temperatureDetected;
DecisionTelemetryData() :
blockType(NOT_BLOCKING),
@@ -39,8 +38,7 @@ struct DecisionTelemetryData
method(POST),
responseCode(0),
elapsedTime(0),
attackTypes(),
temperatureDetected(false)
attackTypes()
{
}
};

View File

@@ -28,9 +28,8 @@
// LCOV_EXCL_START Reason: temporary until we add relevant UT until 07/10
bool operator<(const IpAddress &this_ip_addr, const IpAddress &other_ip_addr);
bool operator==(const IpAddress &this_ip_addr, const IpAddress &other_ip_addr);
bool operator<=(const IpAddress &this_ip_addr, const IpAddress &other_ip_addr);
bool operator<(const IPRange &range1, const IPRange &range2);
// LCOV_EXCL_STOP
Maybe<std::pair<std::string, int>> extractAddressAndMaskSize(const std::string &cidr);

View File

@@ -4,7 +4,6 @@
#include "singleton.h"
#include "i_keywords_rule.h"
#include "i_table.h"
#include "i_mainloop.h"
#include "i_http_manager.h"
#include "i_environment.h"
#include "http_inspection_events.h"
@@ -17,8 +16,7 @@ class IPSComp
Singleton::Consume<I_KeywordsRule>,
Singleton::Consume<I_Table>,
Singleton::Consume<I_Environment>,
Singleton::Consume<I_GenericRulebase>,
Singleton::Consume<I_MainLoop>
Singleton::Consume<I_GenericRulebase>
{
public:
IPSComp();

View File

@@ -62,7 +62,6 @@ public:
private:
Maybe<std::string> downloadPackage(const Package &package, bool is_clean_installation);
std::string getCurrentTimestamp();
std::string manifest_file_path;
std::string temp_ext;

View File

@@ -1,28 +0,0 @@
#ifndef __NGINX_MESSAGE_READER_H__
#define __NGINX_MESSAGE_READER_H__
#include "singleton.h"
#include "i_mainloop.h"
#include "i_socket_is.h"
#include "component.h"
class NginxMessageReader
:
public Component,
Singleton::Consume<I_MainLoop>,
Singleton::Consume<I_Socket>
{
public:
NginxMessageReader();
~NginxMessageReader();
void init() override;
void fini() override;
void preload() override;
private:
class Impl;
std::unique_ptr<Impl> pimpl;
};
#endif //__NGINX_MESSAGE_READER_H__

View File

@@ -1,51 +0,0 @@
// Copyright (C) 2022 Check Point Software Technologies Ltd. All rights reserved.
// Licensed under the Apache License, Version 2.0 (the "License");
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#ifndef __NGINX_UTILS_H__
#define __NGINX_UTILS_H__
#include <string>
#include "maybe_res.h"
#include "singleton.h"
#include "i_shell_cmd.h"
class NginxConfCollector
{
public:
NginxConfCollector(const std::string &nginx_conf_input_path, const std::string &nginx_conf_output_path);
Maybe<std::string> generateFullNginxConf() const;
private:
std::vector<std::string> expandIncludes(const std::string &includePattern) const;
void processConfigFile(
const std::string &path,
std::ostringstream &conf_output,
std::vector<std::string> &errors
) const;
std::string main_conf_input_path;
std::string main_conf_output_path;
std::string main_conf_directory_path;
};
class NginxUtils : Singleton::Consume<I_ShellCmd>
{
public:
static std::string getModulesPath();
static std::string getMainNginxConfPath();
static Maybe<void> validateNginxConf(const std::string &nginx_conf_path);
static Maybe<void> reloadNginx(const std::string &nginx_conf_path);
};
#endif // __NGINX_UTILS_H__

View File

@@ -115,7 +115,7 @@ public:
case ResourceFileType::VIRTUAL_SETTINGS: return "virtualSettings";
case ResourceFileType::VIRTUAL_POLICY: return "virtualPolicy";
default:
dbgAssert(false) << AlertInfo(AlertTeam::CORE, "update process") << "Unknown file type";
dbgAssert(false) << "Unknown file type";
}
return std::string();
}

View File

@@ -56,7 +56,7 @@ private:
if (mapped_type.second == type) return mapped_type.first;
}
dbgAssert(false) << AlertInfo(AlertTeam::CORE, "packaging") << "Unsupported type " << static_cast<int>(type);
dbgAssert(false) << "Unsupported type " << static_cast<int>(type);
// Just satisfying the compiler, this return never reached
return std::string();
}

View File

@@ -1,30 +0,0 @@
#ifndef __PROMETHEUS_COMP_H__
#define __PROMETHEUS_COMP_H__
#include <memory>
#include "component.h"
#include "singleton.h"
#include "i_rest_api.h"
#include "i_messaging.h"
#include "generic_metric.h"
class PrometheusComp
:
public Component,
Singleton::Consume<I_RestApi>,
Singleton::Consume<I_Messaging>
{
public:
PrometheusComp();
~PrometheusComp();
void init() override;
private:
class Impl;
std::unique_ptr<Impl> pimpl;
};
#endif // __PROMETHEUS_COMP_H__

View File

@@ -7,21 +7,15 @@
#include "singleton.h"
#include "i_mainloop.h"
#include "i_environment.h"
#include "i_geo_location.h"
#include "i_generic_rulebase.h"
#include "i_shell_cmd.h"
#include "i_env_details.h"
class RateLimit
:
public Component,
Singleton::Consume<I_MainLoop>,
Singleton::Consume<I_TimeGet>,
Singleton::Consume<I_GeoLocation>,
Singleton::Consume<I_Environment>,
Singleton::Consume<I_GenericRulebase>,
Singleton::Consume<I_ShellCmd>,
Singleton::Consume<I_EnvDetails>
Singleton::Consume<I_GenericRulebase>
{
public:
RateLimit();

View File

@@ -7,28 +7,24 @@ static const std::string product_name = getenv("DOCKER_RPM_ENABLED") ? "CloudGua
static const std::string default_cp_cert_file = "/etc/cp/cpCert.pem";
static const std::string default_cp_key_file = "/etc/cp/cpKey.key";
static const std::string default_rpm_conf_path = "/etc/cp/conf/rpmanager/";
static const std::string default_certificate_path = "/etc/cp/rpmanager/certs";
static const std::string default_manual_certs_path = "/etc/cp/rpmanager/manualCerts/";
static const std::string default_config_path = "/etc/cp/conf/rpmanager/servers";
static const std::string default_rpm_prepare_path = "/etc/cp/conf/rpmanager/prepare/servers";
static const std::string default_nginx_log_files_path = "/var/log/nginx/";
static const std::string default_additional_files_path = "/etc/cp/conf/rpmanager/include";
static const std::string default_server_config = "additional_server_config.conf";
static const std::string default_location_config = "additional_location_config.conf";
static const std::string default_trusted_ca_suffix = "_user_ca_bundle.crt";
static const std::string default_nginx_log_files_path = "/var/log/nginx/";
static const std::string default_log_files_host_path = "/var/log/nano_agent/rpmanager/nginx_log/";
static const std::string default_config_path = "/etc/cp/conf/rpmanager/servers";
static const std::string default_template_path = "/etc/cp/conf/rpmanager/nginx-template-clear";
static const std::string default_manual_certs_path = "/etc/cp/rpmanager/manualCerts/";
static const std::string default_server_certificate_path = "/etc/cp/rpmanager/certs/sslCertificate_";
static const std::string default_server_certificate_key_path = "/etc/cp/rpmanager/certs/sslPrivateKey_";
static const std::string default_container_name = "cp_nginx_gaia";
static const std::string default_docker_image = "cp_nginx_gaia";
static const std::string default_nginx_config_file = "/etc/cp/conf/rpmanager/nginx.conf";
static const std::string default_prepare_nginx_config_file = "/etc/cp/conf/rpmanager/nginx_prepare.conf";
static const std::string default_global_conf_template = "/etc/cp/conf/rpmanager/nginx-conf-template";
static const std::string default_nginx_config_include_file =
"/etc/cp/conf/rpmanager/servers/00_nginx_conf_include.conf";
"/etc/cp/conf/rpmanager/servers/nginx_conf_include";
static const std::string default_global_conf_include_template =
"/etc/cp/conf/rpmanager/nginx-conf-include-template";
static const std::string default_global_conf_include_template_no_responses =

View File

@@ -1,39 +0,0 @@
// Copyright (C) 2022 Check Point Software Technologies Ltd. All rights reserved.
// Licensed under the Apache License, Version 2.0 (the "License");
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#ifndef __SERVICE_HEALTH_STATUS_H__
#define __SERVICE_HEALTH_STATUS_H__
#include "singleton.h"
#include "i_rest_api.h"
#include "i_environment.h"
#include "component.h"
class ServiceHealthStatus
:
public Component,
Singleton::Consume<I_RestApi>,
Singleton::Consume<I_Environment>
{
public:
ServiceHealthStatus();
~ServiceHealthStatus();
void init() override;
private:
class Impl;
std::unique_ptr<Impl> pimpl;
};
#endif // __SERVICE_HEALTH_STATUS_H__

View File

@@ -30,7 +30,6 @@
#include "generic_metric.h"
#define LOGGING_INTERVAL_IN_MINUTES 10
USE_DEBUG_FLAG(D_WAAP);
enum class AssetType { API, WEB, ALL, COUNT };
class WaapTelemetryEvent : public Event<WaapTelemetryEvent>
@@ -76,20 +75,6 @@ private:
std::unordered_set<std::string> sources_seen;
};
class WaapAdditionalTrafficTelemetrics : public WaapTelemetryBase
{
public:
void updateMetrics(const std::string &asset_id, const DecisionTelemetryData &data);
void initMetrics();
private:
MetricCalculations::Counter requests{this, "reservedNgenA"};
MetricCalculations::Counter sources{this, "reservedNgenB"};
MetricCalculations::Counter blocked{this, "reservedNgenC"};
MetricCalculations::Counter temperature_count{this, "reservedNgenD"};
std::unordered_set<std::string> sources_seen;
};
class WaapTrafficTelemetrics : public WaapTelemetryBase
{
public:
@@ -138,7 +123,6 @@ private:
std::map<std::string, std::shared_ptr<WaapTrafficTelemetrics>> traffic_telemetries;
std::map<std::string, std::shared_ptr<WaapAttackTypesMetrics>> attack_types;
std::map<std::string, std::shared_ptr<WaapAttackTypesMetrics>> attack_types_telemetries;
std::map<std::string, std::shared_ptr<WaapAdditionalTrafficTelemetrics>> additional_traffic_telemetries;
template <typename T>
void initializeTelemetryData(
@@ -148,7 +132,6 @@ private:
std::map<std::string, std::shared_ptr<T>>& telemetryMap
) {
if (!telemetryMap.count(asset_id)) {
dbgTrace(D_WAAP) << "creating telemetry data for asset: " << data.assetName;
telemetryMap.emplace(asset_id, std::make_shared<T>());
telemetryMap[asset_id]->init(
telemetryName,
@@ -156,9 +139,7 @@ private:
ReportIS::IssuingEngine::AGENT_CORE,
std::chrono::minutes(LOGGING_INTERVAL_IN_MINUTES),
true,
ReportIS::Audience::SECURITY,
false,
asset_id
ReportIS::Audience::SECURITY
);
telemetryMap[asset_id]->template registerContext<std::string>(
@@ -171,10 +152,6 @@ private:
std::string("Web Application"),
EnvKeyAttr::LogSection::SOURCE
);
telemetryMap[asset_id]->registerListener();
}
dbgTrace(D_WAAP) << "updating telemetry data for asset: " << data.assetName;
telemetryMap[asset_id]->template registerContext<std::string>(
"assetId",
asset_id,
@@ -195,6 +172,9 @@ private:
data.practiceName,
EnvKeyAttr::LogSection::SOURCE
);
telemetryMap[asset_id]->registerListener();
}
}
};

View File

@@ -30,7 +30,6 @@ public:
void parseRequestHeaders(const HttpHeader &header) const;
std::vector<std::string> getHeaderValuesFromConfig(const std::string &header_key) const;
void setXFFValuesToOpaqueCtx(const HttpHeader &header, ExtractType type) const;
void setWafTagValuesToOpaqueCtx(const HttpHeader &header) const;
private:
class UsersIdentifiersConfig
@@ -59,7 +58,7 @@ private:
const std::string::const_iterator &end,
const std::string &key) const;
Buffer extractKeyValueFromCookie(const std::string &cookie_value, const std::string &key) const;
Maybe<std::string> parseXForwardedFor(const std::string &str, ExtractType type) const;
Maybe<std::string> parseXForwardedFor(const std::string &str) const;
std::vector<UsersIdentifiersConfig> user_identifiers;
};

View File

@@ -33,9 +33,6 @@ class I_WaapAssetStatesManager;
class I_Messaging;
class I_AgentDetails;
class I_Encryptor;
class I_WaapModelResultLogger;
const std::string WAAP_APPLICATION_NAME = "waap application";
class WaapComponent
:
@@ -51,8 +48,7 @@ class WaapComponent
Singleton::Consume<I_AgentDetails>,
Singleton::Consume<I_Messaging>,
Singleton::Consume<I_Encryptor>,
Singleton::Consume<I_Environment>,
Singleton::Consume<I_WaapModelResultLogger>
Singleton::Consume<I_Environment>
{
public:
WaapComponent();

View File

@@ -1,3 +0,0 @@
link_directories(${BOOST_ROOT}/lib)
add_library(nginx_message_reader nginx_message_reader.cc)

View File

@@ -1,735 +0,0 @@
#include "nginx_message_reader.h"
#include <string>
#include <boost/regex.hpp>
#include <boost/algorithm/string.hpp>
#include <boost/algorithm/string/regex.hpp>
#include "config.h"
#include "singleton.h"
#include "i_mainloop.h"
#include "enum_array.h"
#include "log_generator.h"
#include "maybe_res.h"
#include "http_transaction_data.h"
#include "generic_rulebase/rulebase_config.h"
#include "generic_rulebase/evaluators/asset_eval.h"
#include "generic_rulebase/triggers_config.h"
#include "agent_core_utilities.h"
#include "rate_limit_config.h"
USE_DEBUG_FLAG(D_NGINX_MESSAGE_READER);
using namespace std;
static const string syslog_regex_string = (
"<[0-9]+>([A-Z][a-z][a-z]\\s{1,2}\\d{1,2}\\s\\d{2}"
"[:]\\d{2}[:]\\d{2})\\s([\\w][\\w\\d\\.@-]*)\\s(nginx:)"
);
static const boost::regex socket_address_regex("(\\d+\\.\\d+\\.\\d+\\.\\d+):(\\d+)");
static const boost::regex syslog_regex(syslog_regex_string);
static const boost::regex alert_log_regex(
"("
+ syslog_regex_string + ") "
+ "(.+?\\[alert\\] )(.+?)"
", (client: .+?)"
", (server: .+?)"
", (request: \".+?\")"
", (upstream: \".+?\")"
", (host: \".+?\")$"
);
static const boost::regex error_log_regex(
"("
+ syslog_regex_string + ") "
+ "(.+?\\[error\\] )(.+?)"
", (client: .+?)"
", (server: .+?)"
", (request: \".+?\")"
", (upstream: \".+?\")"
", (host: \".+?\")$"
);
static const boost::regex server_regex("(\\d+\\.\\d+\\.\\d+\\.\\d+)|(\\w+\\.\\w+)");
static const boost::regex uri_regex("^/");
static const boost::regex port_regex("\\d+");
static const boost::regex response_code_regex("[0-9]{3}");
static const boost::regex http_method_regex("[A-Za-z]+");
class NginxMessageReader::Impl
{
public:
void
init()
{
dbgFlow(D_NGINX_MESSAGE_READER);
I_MainLoop *mainloop = Singleton::Consume<I_MainLoop>::by<NginxMessageReader>();
mainloop->addOneTimeRoutine(
I_MainLoop::RoutineType::System,
[this] ()
{
initSyslogServerSocket();
handleNginxLogs();
},
"Initialize nginx syslog",
true
);
}
void
preload()
{
registerConfigLoadCb([this]() { loadNginxMessageReaderConfig(); });
}
void
fini()
{
I_Socket *i_socket = Singleton::Consume<I_Socket>::by<NginxMessageReader>();
i_socket->closeSocket(syslog_server_socket);
}
void
loadNginxMessageReaderConfig()
{
rate_limit_status_code = getProfileAgentSettingWithDefault<string>(
"429",
"accessControl.rateLimit.returnCode"
);
dbgTrace(D_NGINX_MESSAGE_READER) << "Selected rate-limit status code: " << rate_limit_status_code;
}
private:
enum class LogInfo {
HTTP_METHOD,
URI,
RESPONSE_CODE,
HOST,
SOURCE,
DESTINATION_IP,
DESTINATION_PORT,
EVENT_MESSAGE,
ASSET_ID,
ASSET_NAME,
RULE_NAME,
RULE_ID,
COUNT
};
void
initSyslogServerSocket()
{
dbgFlow(D_NGINX_MESSAGE_READER);
I_MainLoop *mainloop = Singleton::Consume<I_MainLoop>::by<NginxMessageReader>();
I_Socket *i_socket = Singleton::Consume<I_Socket>::by<NginxMessageReader>();
string nginx_syslog_server_address = getProfileAgentSettingWithDefault<string>(
"127.0.0.1:1514",
"reverseProxy.nginx.syslogAddress"
);
dbgInfo(D_NGINX_MESSAGE_READER) << "Attempting to open a socket: " << nginx_syslog_server_address;
do {
Maybe<I_Socket::socketFd> new_socket = i_socket->genSocket(
I_Socket::SocketType::UDP,
false,
true,
nginx_syslog_server_address
);
if (!new_socket.ok()) {
dbgError(D_NGINX_MESSAGE_READER) << "Failed to open a socket. Error: " << new_socket.getErr();
mainloop->yield(chrono::milliseconds(500));
continue;
}
if (new_socket.unpack() < 0) {
dbgError(D_NGINX_MESSAGE_READER)<< "Generated socket is OK yet negative";
mainloop->yield(chrono::milliseconds(500));
continue;
}
syslog_server_socket = new_socket.unpack();
dbgInfo(D_NGINX_MESSAGE_READER)
<< "Opened socket for nginx logs over syslog. Socket: "
<< syslog_server_socket;
} while (syslog_server_socket < 0);
}
void
handleNginxLogs()
{
dbgFlow(D_NGINX_MESSAGE_READER);
I_MainLoop::Routine read_logs =
[this] ()
{
Maybe<string> logs = getLogsFromSocket(syslog_server_socket);
if (!logs.ok()) {
dbgWarning(D_NGINX_MESSAGE_READER)
<< "Failed to get NGINX logs from the socket. Error: "
<< logs.getErr();
return;
}
string raw_logs_to_parse = logs.unpackMove();
vector<string> logs_to_parse = separateLogs(raw_logs_to_parse);
for (auto const &log: logs_to_parse) {
bool log_sent;
if (isAccessLog(log)) {
log_sent = sendAccessLog(log);
} else if (isAlertErrorLog(log) || isErrorLog(log)) {
log_sent = sendErrorLog(log);
} else {
dbgWarning(D_NGINX_MESSAGE_READER) << "Unexpected nginx log format";
continue;
}
if (!log_sent) {
dbgWarning(D_NGINX_MESSAGE_READER) << "Failed to send Log to Infinity Portal";
} else {
dbgTrace(D_NGINX_MESSAGE_READER) << "Succesfully sent nginx log to Infinity Portal";
}
}
};
I_MainLoop *mainloop = Singleton::Consume<I_MainLoop>::by<NginxMessageReader>();
mainloop->addFileRoutine(
I_MainLoop::RoutineType::RealTime,
syslog_server_socket,
read_logs,
"Process nginx logs",
true
);
}
bool
sendAccessLog(const string &log)
{
dbgFlow(D_NGINX_MESSAGE_READER) << "Access log" << log;
Maybe<EnumArray<LogInfo, string>> log_info = parseAccessLog(log);
if (!log_info.ok()) {
dbgWarning(D_NGINX_MESSAGE_READER)
<< "Failed parsing the NGINX logs. Error: "
<< log_info.getErr();
return false;
}
auto unpacked_log_info = log_info.unpack();
if (unpacked_log_info[LogInfo::RESPONSE_CODE] == rate_limit_status_code) {
return sendRateLimitLog(unpacked_log_info);
}
return sendLog(unpacked_log_info);
}
bool
sendErrorLog(const string &log)
{
dbgFlow(D_NGINX_MESSAGE_READER) << "Error log" << log;
Maybe<EnumArray<LogInfo, string>> log_info = parseErrorLog(log);
if (!log_info.ok()) {
dbgWarning(D_NGINX_MESSAGE_READER)
<< "Failed parsing the NGINX logs. Error: "
<< log_info.getErr();
return false;
}
return sendLog(log_info.unpack());
}
bool
isAccessLog(const string &log) const
{
dbgFlow(D_NGINX_MESSAGE_READER) << "Chekck if string contains \"accessLog\"" << log;
return log.find("accessLog") != string::npos;
}
bool
isAlertErrorLog(const string &log) const
{
dbgFlow(D_NGINX_MESSAGE_READER) << "Check if log is of type 'error log'. Log: " << log;
return log.find("[alert]") != string::npos;
}
bool
isErrorLog(const string &log) const
{
dbgFlow(D_NGINX_MESSAGE_READER) << "Check if log is of type 'error log'. Log: " << log;
return log.find("[error]") != string::npos;
}
bool
sendLog(const EnumArray<LogInfo, string> &log_info)
{
dbgFlow(D_NGINX_MESSAGE_READER);
string event_name;
switch (log_info[LogInfo::RESPONSE_CODE][0]) {
case '4': {
event_name = "Invalid request or incorrect reverse proxy configuration - Request dropped."
" Please check the reverse proxy configuration of your relevant assets";
break;
}
case '5': {
event_name = "AppSec Gateway reverse proxy error - Request dropped. "
"Please verify the reverse proxy configuration of your relevant assets. "
"If the issue persists please contact Check Point Support";
break;
}
default: {
dbgError(D_NGINX_MESSAGE_READER) << "Irrelevant status code";
return false;
}
}
dbgTrace(D_NGINX_MESSAGE_READER)
<< "Nginx log's event name and response code: "
<< event_name
<< ", "
<< log_info[LogInfo::RESPONSE_CODE];
LogGen log(
event_name,
ReportIS::Audience::SECURITY,
ReportIS::Severity::INFO,
ReportIS::Priority::LOW,
ReportIS::Tags::REVERSE_PROXY
);
log << LogField("eventConfidence", "High");
for (LogInfo field : makeRange<LogInfo>()) {
Maybe<string> string_field = convertLogFieldToString(field);
if (!string_field.ok()) {
dbgDebug(D_NGINX_MESSAGE_READER) << "Enum field was not converted: " << string_field.getErr();
return false;
}
if (field != LogInfo::DESTINATION_PORT) {
log << LogField(string_field.unpack(), log_info[field]);
continue;
}
try {
log << LogField(string_field.unpack(), stoi(log_info[field]));
} catch (const exception &e) {
dbgError(D_NGINX_MESSAGE_READER)
<< "Unable to convert port to numeric value: "
<< e.what();
log << LogField(string_field.unpack(), 0);
}
}
return true;
}
bool
sendRateLimitLog(const EnumArray<LogInfo, string> &log_info)
{
dbgFlow(D_NGINX_MESSAGE_READER) << "Getting rate-limit rules of asset ID: " << log_info[LogInfo::ASSET_ID];
ScopedContext rate_limit_ctx;
rate_limit_ctx.registerValue<GenericConfigId>(AssetMatcher::ctx_key, log_info[LogInfo::ASSET_ID]);
auto rate_limit_config = getConfiguration<RateLimitConfig>("rulebase", "rateLimit");
if (!rate_limit_config.ok()) {
dbgTrace(D_NGINX_MESSAGE_READER)
<< "Rate limit context does not match asset ID: " << log_info[LogInfo::ASSET_ID];
return false;
}
RateLimitConfig unpacked_rate_limit_config = rate_limit_config.unpack();
string nginx_uri = log_info[LogInfo::URI];
const LogTriggerConf &rate_limit_trigger = unpacked_rate_limit_config.getRateLimitTrigger(nginx_uri);
dbgTrace(D_NGINX_MESSAGE_READER)<< "About to generate NGINX rate-limit log";
string event_name = "Rate limit";
string security_action = "Drop";
bool is_log_required = false;
// Prevent events checkbox (in triggers)
if (rate_limit_trigger.isPreventLogActive(LogTriggerConf::SecurityType::AccessControl)) {
is_log_required = true;
}
if (!is_log_required) {
dbgTrace(D_NGINX_MESSAGE_READER) << "Not sending NGINX rate-limit log as it is not required";
return false;
}
ostringstream src_ip;
ostringstream dst_ip;
src_ip << log_info[LogInfo::SOURCE];
dst_ip << log_info[LogInfo::DESTINATION_IP];
ReportIS::Severity log_severity = ReportIS::Severity::MEDIUM;
ReportIS::Priority log_priority = ReportIS::Priority::MEDIUM;
LogGen log = rate_limit_trigger(
event_name,
LogTriggerConf::SecurityType::AccessControl,
log_severity,
log_priority,
true, // is drop
LogField("practiceType", "Rate Limit"),
ReportIS::Tags::RATE_LIMIT
);
for (LogInfo field : makeRange<LogInfo>()) {
Maybe<string> string_field = convertLogFieldToString(field);
if (!string_field.ok()) {
dbgDebug(D_NGINX_MESSAGE_READER) << "Enum field was not converted: " << string_field.getErr();
return false;
}
if (
field == LogInfo::HOST ||
field == LogInfo::URI ||
field == LogInfo::HTTP_METHOD ||
field == LogInfo::SOURCE ||
field == LogInfo::DESTINATION_IP ||
field == LogInfo::ASSET_ID ||
field == LogInfo::ASSET_NAME ||
field == LogInfo::RESPONSE_CODE
) {
if (!log_info[field].empty()) {
log << LogField(string_field.unpack(), log_info[field]);
continue;
}
}
if (field == LogInfo::DESTINATION_PORT) {
try {
int numeric_dst_port = stoi(log_info[field]);
log << LogField(string_field.unpack(), numeric_dst_port);
} catch (const exception &e) {
dbgWarning(D_NGINX_MESSAGE_READER)
<< "Unable to convert dst port: "
<< log_info[field]
<< " to numberic value. Error: "
<< e.what();
}
}
}
return true;
}
Maybe<string>
convertLogFieldToString(LogInfo field)
{
dbgFlow(D_NGINX_MESSAGE_READER);
switch (field) {
case LogInfo::HTTP_METHOD:
return string("httpMethod");
case LogInfo::URI:
return string("httpUriPath");
case LogInfo::RESPONSE_CODE:
return string("httpResponseCode");
case LogInfo::HOST:
return string("httpHostName");
case LogInfo::SOURCE:
return string("httpSourceId");
case LogInfo::DESTINATION_IP:
return string("destinationIp");
case LogInfo::DESTINATION_PORT:
return string("destinationPort");
case LogInfo::ASSET_ID:
return string("assetId");
case LogInfo::ASSET_NAME:
return string("assetName");
case LogInfo::EVENT_MESSAGE:
return string("httpResponseBody");
case LogInfo::RULE_ID:
return string("ruleId");
case LogInfo::RULE_NAME:
return string("ruleName");
case LogInfo::COUNT:
dbgError(D_NGINX_MESSAGE_READER) << "LogInfo::COUNT is not allowed";
return genError("LogInfo::COUNT is not allowed");
}
dbgError(D_NGINX_MESSAGE_READER) << "No Enum found, int value: " << static_cast<int>(field);
return genError("No Enum found");
}
static vector<string>
separateLogs(const string &raw_logs_to_parse)
{
dbgFlow(D_NGINX_MESSAGE_READER) << "separating logs. logs: " << raw_logs_to_parse;
dbgTrace(D_NGINX_MESSAGE_READER) << "separateLogs start of function. Logs to parse: " << raw_logs_to_parse;
boost::smatch matcher;
vector<string> logs;
if (raw_logs_to_parse.empty()) return logs;
size_t pos = 0;
while (NGEN::Regex::regexSearch(__FILE__, __LINE__, raw_logs_to_parse.substr(pos), matcher, syslog_regex)) {
if (pos == 0) {
dbgTrace(D_NGINX_MESSAGE_READER) << "separateLogs pos = 0";
pos++;
continue;
}
auto log_length = matcher.position();
logs.push_back(raw_logs_to_parse.substr(pos - 1, log_length));
pos += log_length + 1;
}
logs.push_back(raw_logs_to_parse.substr(pos - 1));
dbgTrace(D_NGINX_MESSAGE_READER) << "separateLogs end of function";
return logs;
}
static pair<string, string>
parseErrorLogRequestField(const string &request)
{
dbgFlow(D_NGINX_MESSAGE_READER) << "parsing request field. request: " << request;
string formatted_request = request;
vector<string> result;
boost::erase_all(formatted_request, "\"");
boost::erase_all(formatted_request, "\n");
boost::split(result, formatted_request, boost::is_any_of(" "), boost::token_compress_on);
const int http_method_index = 1;
const int uri_index = 2;
return pair<string, string>(result[http_method_index], result[uri_index]);
}
static string
parseErrorLogField(const string &field)
{
dbgFlow(D_NGINX_MESSAGE_READER) << "parsing error log field " << field;
string formatted_field = field;
vector<string> result;
boost::erase_all(formatted_field, "\"");
boost::erase_all(formatted_field, "\n");
boost::split(result, formatted_field, boost::is_any_of(" "), boost::token_compress_on);
const int field_index = 1;
return result[field_index];
}
void
addContextFieldsToLogInfo(EnumArray<LogInfo, string> &log_info)
{
dbgFlow(D_NGINX_MESSAGE_READER);
ScopedContext ctx;
try {
ctx.registerValue<uint16_t>(
HttpTransactionData::listening_port_ctx,
static_cast<uint16_t>(stoi(log_info[LogInfo::DESTINATION_PORT]))
);
} catch (const exception &e) {
dbgError(D_NGINX_MESSAGE_READER) << "Failed register values for context " << e.what();
}
ctx.registerValue<string>(HttpTransactionData::host_name_ctx, log_info[LogInfo::HOST]);
ctx.registerValue<string>(HttpTransactionData::uri_ctx, log_info[LogInfo::URI]);
auto rule_by_ctx = getConfiguration<BasicRuleConfig>("rulebase", "rulesConfig");
if (!rule_by_ctx.ok()) {
dbgWarning(D_NGINX_MESSAGE_READER)
<< "AssetId was not found by the given context. Reason: "
<< rule_by_ctx.getErr();
return;
}
BasicRuleConfig context = rule_by_ctx.unpack();
log_info[LogInfo::ASSET_ID] = context.getAssetId();
log_info[LogInfo::ASSET_NAME] = context.getAssetName();
log_info[LogInfo::RULE_ID] = context.getRuleId();
log_info[LogInfo::RULE_NAME] = context.getRuleName();
}
Maybe<EnumArray<LogInfo, string>>
parseErrorLog(const string &log_line)
{
dbgFlow(D_NGINX_MESSAGE_READER) << "Handling log line:" << log_line;
string port;
EnumArray<LogInfo, string> log_info(EnumArray<LogInfo, string>::Fill(), string(""));
boost::smatch matcher;
vector<string> result;
if (
!NGEN::Regex::regexSearch(
__FILE__,
__LINE__,
log_line,
matcher,
isAlertErrorLog(log_line) ? alert_log_regex : error_log_regex
)
) {
dbgWarning(D_NGINX_MESSAGE_READER) << "Unexpected nginx log format";
return genError("Unexpected nginx log format");
}
const int event_message_index = 6;
const int source_index = 7;
const int request_index = 9;
const int host_index = 11;
string host = string(matcher[host_index].first, matcher[host_index].second);
string source = string(matcher[source_index].first, matcher[source_index].second);
string event_message = string(matcher[event_message_index].first, matcher[event_message_index].second);
string request = string(matcher[request_index].first, matcher[request_index].second);
host = parseErrorLogField(host);
source = parseErrorLogField(source);
pair<string, string> parsed_request = parseErrorLogRequestField(request);
string http_method = parsed_request.first;
string uri = parsed_request.second;
if (NGEN::Regex::regexSearch(__FILE__, __LINE__, host, matcher, socket_address_regex)) {
int host_index = 1;
int port_index = 2;
host = string(matcher[host_index].first, matcher[host_index].second);
port = string(matcher[port_index].first, matcher[port_index].second);
} else if (NGEN::Regex::regexSearch(__FILE__, __LINE__, host, matcher, boost::regex("https://"))) {
port = "443";
} else {
port = "80";
}
log_info[LogInfo::HOST] = host;
log_info[LogInfo::URI] = uri;
log_info[LogInfo::RESPONSE_CODE] = "500";
log_info[LogInfo::HTTP_METHOD] = http_method;
log_info[LogInfo::SOURCE] = source;
log_info[LogInfo::DESTINATION_IP] = host;
log_info[LogInfo::DESTINATION_PORT] = port;
log_info[LogInfo::EVENT_MESSAGE] = event_message;
addContextFieldsToLogInfo(log_info);
if (!validateLog(log_info)) {
dbgWarning(D_NGINX_MESSAGE_READER) << "Unexpected nginx log format";
return genError("Unexpected nginx log format");
}
return log_info;
}
Maybe<EnumArray<LogInfo, string>>
parseAccessLog(const string &log_line)
{
dbgFlow(D_NGINX_MESSAGE_READER) << "Parsing log line: " << log_line;
string formatted_log = log_line;
EnumArray<LogInfo, string> log_info(EnumArray<LogInfo, string>::Fill(), string(""));
vector<string> result;
boost::erase_all(formatted_log, "\"");
boost::erase_all(formatted_log, "\n");
boost::split(result, formatted_log, boost::is_any_of(" "), boost::token_compress_on);
const int valid_log_size = 20;
if (result.size() < valid_log_size) {
dbgWarning(D_NGINX_MESSAGE_READER) << "Unexpected nginx log format";
return genError("Unexpected nginx log format");
}
const int host_index = 6;
const int host_port_index = 7;
const int http_method_index = 13;
const int uri_index = 14;
const int response_cod_index = 16;
const int source_index = 8;
log_info[LogInfo::HOST] = result[host_index];
log_info[LogInfo::URI] = result[uri_index];
log_info[LogInfo::RESPONSE_CODE] = result[response_cod_index];
log_info[LogInfo::HTTP_METHOD] = result[http_method_index];
log_info[LogInfo::SOURCE] = result[source_index];
log_info[LogInfo::DESTINATION_IP] = result[host_index];
log_info[LogInfo::DESTINATION_PORT] = result[host_port_index];
log_info[LogInfo::EVENT_MESSAGE] = "Invalid request or incorrect reverse proxy configuration - "
"Request dropped. Please check the reverse proxy configuration of your relevant assets";
addContextFieldsToLogInfo(log_info);
if (!validateLog(log_info)) {
dbgWarning(D_NGINX_MESSAGE_READER) << "Unexpected nginx log format";
return genError("Unexpected nginx log format");
}
return log_info;
}
static bool
validateLog(const EnumArray<LogInfo, string> &log_info)
{
dbgFlow(D_NGINX_MESSAGE_READER);
boost::smatch matcher;
if (!NGEN::Regex::regexSearch(__FILE__, __LINE__, log_info[LogInfo::HOST], matcher, server_regex)) {
dbgTrace(D_NGINX_MESSAGE_READER) << "Could not validate server (Host): " << log_info[LogInfo::HOST];
return false;
}
if (!NGEN::Regex::regexSearch(__FILE__, __LINE__, log_info[LogInfo::URI], matcher, uri_regex)) {
dbgTrace(D_NGINX_MESSAGE_READER) << "Could not validate Uri: " << log_info[LogInfo::URI];
return false;
}
if (
!NGEN::Regex::regexSearch(
__FILE__,
__LINE__,
log_info[LogInfo::RESPONSE_CODE],
matcher, response_code_regex
)
) {
dbgTrace(D_NGINX_MESSAGE_READER)
<< "Could not validate response code: "
<< log_info[LogInfo::RESPONSE_CODE];
return false;
}
if (
!NGEN::Regex::regexSearch(__FILE__, __LINE__, log_info[LogInfo::HTTP_METHOD], matcher, http_method_regex)
) {
dbgTrace(D_NGINX_MESSAGE_READER) << "Could not validate HTTP method: " << log_info[LogInfo::HTTP_METHOD];
return false;
}
if (!NGEN::Regex::regexSearch(__FILE__, __LINE__, log_info[LogInfo::DESTINATION_PORT], matcher, port_regex)) {
dbgTrace(D_NGINX_MESSAGE_READER)
<< "Could not validate destination port : "
<< log_info[LogInfo::DESTINATION_PORT];
return false;
}
if (!NGEN::Regex::regexSearch(__FILE__, __LINE__, log_info[LogInfo::SOURCE], matcher, server_regex)) {
dbgTrace(D_NGINX_MESSAGE_READER) << "Could not validate source : " << log_info[LogInfo::SOURCE];
return false;
}
return true;
}
Maybe<string>
getLogsFromSocket(const I_Socket::socketFd &client_socket) const
{
dbgFlow(D_NGINX_MESSAGE_READER) << "Reading logs from socket. fd: " << client_socket;
I_Socket *i_socket = Singleton::Consume<I_Socket>::by<NginxMessageReader>();
Maybe<vector<char>> raw_log_data = i_socket->receiveData(client_socket, 0, false);
if (!raw_log_data.ok()) {
dbgWarning(D_NGINX_MESSAGE_READER) << "Error receiving data from socket";
return genError("Error receiving data from socket");
}
string raw_log(raw_log_data.unpack().begin(), raw_log_data.unpack().end());
return move(raw_log);
}
I_Socket::socketFd syslog_server_socket = -1;
string rate_limit_status_code = "429";
};
NginxMessageReader::NginxMessageReader() : Component("NginxMessageReader"), pimpl(make_unique<Impl>()) {}
NginxMessageReader::~NginxMessageReader() {}
void
NginxMessageReader::init()
{
pimpl->init();
}
void
NginxMessageReader::preload()
{
pimpl->preload();
}
void
NginxMessageReader::fini()
{
pimpl->fini();
}

View File

@@ -563,10 +563,7 @@ Packet::parsePacket(PktType type, IPType proto)
return parseFromL3v6();
}
default: {
dbgAssert(false)
<< AlertInfo(AlertTeam::CORE, "packet")
<< "Unknown (neither IPv4, nor IPv6), or uninitialized packet type: "
<< proto;
dbgAssert(false) << "Unknown (neither IPv4, nor IPv6), or uninitialized packet type: " << proto;
}
}

View File

@@ -43,9 +43,7 @@ PendingKey::print(ostream &os) const
size_t
PendingKey::hash() const
{
dbgAssert(src.type != IPType::UNINITIALIZED)
<< AlertInfo(AlertTeam::CORE, "pending key")
<< "PendingKey::hash was called on an uninitialized object";
dbgAssert(src.type != IPType::UNINITIALIZED) << "PendingKey::hash was called on an uninitialized object";
size_t seed = 0;
hashCombine(seed, static_cast<u_char>(src.type));
hashCombine(seed, src.proto);

View File

@@ -1,2 +1,3 @@
add_library(report_messaging report_messaging.cc)
add_subdirectory(report_messaging_ut)

View File

@@ -0,0 +1,3 @@
link_directories(${BOOST_ROOT}/lib)
add_unit_test(report_messaging_ut "report_messaging_ut.cc" "report_messaging;report;messaging;singleton;-lboost_regex")

View File

@@ -3,7 +3,5 @@ add_subdirectory(ips)
add_subdirectory(layer_7_access_control)
add_subdirectory(local_policy_mgmt_gen)
add_subdirectory(orchestration)
add_subdirectory(prometheus)
add_subdirectory(rate_limit)
add_subdirectory(waap)
add_subdirectory(central_nginx_manager)

View File

@@ -1,3 +0,0 @@
include_directories(include)
add_library(central_nginx_manager central_nginx_manager.cc lets_encrypt_listener.cc)

View File

@@ -1,418 +0,0 @@
// Copyright (C) 2022 Check Point Software Technologies Ltd. All rights reserved.
// Licensed under the Apache License, Version 2.0 (the "License");
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "central_nginx_manager.h"
#include "lets_encrypt_listener.h"
#include <string>
#include <vector>
#include <cereal/external/base64.hpp>
#include "debug.h"
#include "config.h"
#include "rest.h"
#include "log_generator.h"
#include "nginx_utils.h"
#include "agent_core_utilities.h"
using namespace std;
USE_DEBUG_FLAG(D_NGINX_MANAGER);
class CentralNginxConfig
{
public:
void load(cereal::JSONInputArchive &ar)
{
try {
string nginx_conf_base64;
ar(cereal::make_nvp("id", file_id));
ar(cereal::make_nvp("name", file_name));
ar(cereal::make_nvp("data", nginx_conf_base64));
nginx_conf_content = cereal::base64::decode(nginx_conf_base64);
central_nginx_conf_path = getCentralNginxConfPath();
shared_config_path = getSharedConfigPath();
if (!nginx_conf_content.empty()) configureCentralNginx();
} catch (const cereal::Exception &e) {
dbgDebug(D_NGINX_MANAGER) << "Could not load Central Management Config JSON. Error: " << e.what();
ar.setNextName(nullptr);
}
}
const string & getFileId() const { return file_id; }
const string & getFileName() const { return file_name; }
const string & getFileContent() const { return nginx_conf_content; }
static string
getCentralNginxConfPath()
{
string central_nginx_conf_path = getProfileAgentSettingWithDefault<string>(
string("/tmp/central_nginx.conf"),
"centralNginxManagement.confDownloadPath"
);
dbgInfo(D_NGINX_MANAGER) << "Central NGINX configuration path: " << central_nginx_conf_path;
return central_nginx_conf_path;
}
static string
getSharedConfigPath()
{
string central_shared_conf_path = getConfigurationWithDefault<string>(
"/etc/cp/conf",
"Config Component",
"configuration path"
);
central_shared_conf_path += "/centralNginxManager/shared/central_nginx_shared.conf";
dbgInfo(D_NGINX_MANAGER) << "Shared NGINX configuration path: " << central_shared_conf_path;
return central_shared_conf_path;
}
private:
void
loadAttachmentModule()
{
string attachment_module_path = NginxUtils::getModulesPath() + "/ngx_cp_attachment_module.so";
if (!NGEN::Filesystem::exists(attachment_module_path)) {
dbgTrace(D_NGINX_MANAGER) << "Attachment module " << attachment_module_path << " does not exist";
return;
}
string attachment_module_conf = "load_module " + attachment_module_path + ";";
if (nginx_conf_content.find(attachment_module_conf) != string::npos) {
dbgTrace(D_NGINX_MANAGER) << "Attachment module " << attachment_module_path << " already loaded";
return;
}
nginx_conf_content = attachment_module_conf + "\n" + nginx_conf_content;
}
Maybe<void>
loadSharedDirective(const string &directive)
{
dbgFlow(D_NGINX_MANAGER) << "Loading shared directive into the servers " << directive;
if (!NGEN::Filesystem::copyFile(shared_config_path, shared_config_path + ".bak", true)) {
return genError("Could not create a backup of the shared NGINX configuration file");
}
ifstream shared_config(shared_config_path);
if (!shared_config.is_open()) {
return genError("Could not open shared NGINX configuration file");
}
string shared_config_content((istreambuf_iterator<char>(shared_config)), istreambuf_iterator<char>());
shared_config.close();
if (shared_config_content.find(directive) != string::npos) {
dbgTrace(D_NGINX_MANAGER) << "Shared directive " << directive << " already loaded";
return {};
}
ofstream new_shared_config(shared_config_path, ios::app);
if (!new_shared_config.is_open()) {
return genError("Could not open shared NGINX configuration file");
}
dbgTrace(D_NGINX_MANAGER) << "Adding shared directive " << directive;
new_shared_config << directive << "\n";
new_shared_config.close();
auto validation = NginxUtils::validateNginxConf(central_nginx_conf_path);
if (!validation.ok()) {
if (!NGEN::Filesystem::copyFile(shared_config_path + ".bak", shared_config_path, true)) {
return genError("Could not restore the shared NGINX configuration file");
}
return genError("Could not validate shared NGINX configuration file. Error: " + validation.getErr());
}
return {};
}
Maybe<void>
loadSharedConfig()
{
dbgFlow(D_NGINX_MANAGER) << "Loading shared configuration into the servers";
ofstream shared_config(shared_config_path);
if (!shared_config.is_open()) {
return genError("Could not create shared NGINX configuration file");
}
shared_config.close();
string shared_config_directive = "include " + shared_config_path + ";\n";
boost::regex server_regex("server\\s*\\{");
nginx_conf_content = NGEN::Regex::regexReplace(
__FILE__,
__LINE__,
nginx_conf_content,
server_regex,
"server {\n" + shared_config_directive
);
ofstream nginx_conf_file(central_nginx_conf_path);
if (!nginx_conf_file.is_open()) {
return genError("Could not open a temporary central NGINX configuration file");
}
nginx_conf_file << nginx_conf_content;
nginx_conf_file.close();
auto validation = NginxUtils::validateNginxConf(central_nginx_conf_path);
if (!validation.ok()) {
return genError("Could not validate central NGINX configuration file. Error: " + validation.getErr());
}
return {};
}
Maybe<void>
configureSyslog()
{
if (!getProfileAgentSettingWithDefault<bool>(false, "centralNginxManagement.syslogEnabled")) {
dbgTrace(D_NGINX_MANAGER) << "Syslog is disabled via settings";
return {};
}
string syslog_directive = "error_log syslog:server=127.0.0.1:1514 warn;";
auto load_shared_directive_result = loadSharedDirective(syslog_directive);
if (!load_shared_directive_result.ok()) {
return genError("Could not configure syslog directive, error: " + load_shared_directive_result.getErr());
}
return {};
}
Maybe<void>
saveBaseCentralNginxConf()
{
ofstream central_nginx_conf_base_file(central_nginx_conf_path + ".base");
if (!central_nginx_conf_base_file.is_open()) {
return genError("Could not open a temporary central NGINX configuration file");
}
central_nginx_conf_base_file << nginx_conf_content;
central_nginx_conf_base_file.close();
return {};
}
void
configureCentralNginx()
{
loadAttachmentModule();
auto save_base_nginx_conf = saveBaseCentralNginxConf();
if (!save_base_nginx_conf.ok()) {
dbgWarning(D_NGINX_MANAGER)
<< "Could not save base NGINX configuration. Error: "
<< save_base_nginx_conf.getErr();
return;
}
string nginx_conf_content_backup = nginx_conf_content;
auto shared_config_result = loadSharedConfig();
if (!shared_config_result.ok()) {
dbgWarning(D_NGINX_MANAGER)
<< "Could not load shared configuration. Error: "
<< shared_config_result.getErr();
nginx_conf_content = nginx_conf_content_backup;
return;
}
auto syslog_result = configureSyslog();
if (!syslog_result.ok()) {
dbgWarning(D_NGINX_MANAGER) << "Could not configure syslog. Error: " << syslog_result.getErr();
}
}
string file_id;
string file_name;
string nginx_conf_content;
string central_nginx_conf_path;
string shared_config_path;
};
class CentralNginxManager::Impl
{
public:
void
init()
{
dbgInfo(D_NGINX_MANAGER) << "Starting Central NGINX Manager";
string main_nginx_conf_path = NginxUtils::getMainNginxConfPath();
if (
NGEN::Filesystem::exists(main_nginx_conf_path)
&& !NGEN::Filesystem::exists(main_nginx_conf_path + ".orig")
) {
dbgInfo(D_NGINX_MANAGER) << "Creating a backup of the original main NGINX configuration file";
NGEN::Filesystem::copyFile(main_nginx_conf_path, main_nginx_conf_path + ".orig", true);
}
i_mainloop = Singleton::Consume<I_MainLoop>::by<CentralNginxManager>();
if (!lets_encrypt_listener.init()) {
dbgWarning(D_NGINX_MANAGER) << "Could not start Lets Encrypt Listener, scheduling retry";
i_mainloop->addOneTimeRoutine(
I_MainLoop::RoutineType::System,
[this] ()
{
while(!lets_encrypt_listener.init()) {
dbgWarning(D_NGINX_MANAGER) << "Could not start Lets Encrypt Listener, will retry";
i_mainloop->yield(chrono::seconds(5));
}
},
"Lets Encrypt Listener initializer",
false
);
}
}
void
loadPolicy()
{
auto central_nginx_config = getSetting<vector<CentralNginxConfig>>("centralNginxManagement");
if (!central_nginx_config.ok() || central_nginx_config.unpack().empty()) {
dbgWarning(D_NGINX_MANAGER)
<< "Could not load Central NGINX Management settings. Error: "
<< central_nginx_config.getErr();
return;
}
auto &config = central_nginx_config.unpack().front();
if (config.getFileContent().empty()) {
dbgWarning(D_NGINX_MANAGER) << "Empty NGINX configuration file";
return;
}
dbgTrace(D_NGINX_MANAGER)
<< "Handling Central NGINX Management settings: "
<< config.getFileId()
<< ", "
<< config.getFileName()
<< ", "
<< config.getFileContent();
string central_nginx_conf_path = config.getCentralNginxConfPath();
ofstream central_nginx_conf_file(central_nginx_conf_path);
if (!central_nginx_conf_file.is_open()) {
dbgWarning(D_NGINX_MANAGER)
<< "Could not open central NGINX configuration file: "
<< central_nginx_conf_path;
return;
}
central_nginx_conf_file << config.getFileContent();
central_nginx_conf_file.close();
auto validation_result = NginxUtils::validateNginxConf(central_nginx_conf_path);
if (!validation_result.ok()) {
dbgWarning(D_NGINX_MANAGER)
<< "Could not validate central NGINX configuration file. Error: "
<< validation_result.getErr();
logError(validation_result.getErr());
return;
}
dbgTrace(D_NGINX_MANAGER) << "Validated central NGINX configuration file";
auto reload_result = NginxUtils::reloadNginx(central_nginx_conf_path);
if (!reload_result.ok()) {
dbgWarning(D_NGINX_MANAGER)
<< "Could not reload central NGINX configuration. Error: "
<< reload_result.getErr();
logError("Could not reload central NGINX configuration. Error: " + reload_result.getErr());
return;
}
logInfo("Central NGINX configuration has been successfully reloaded");
}
void
fini()
{
string central_nginx_base_path = CentralNginxConfig::getCentralNginxConfPath() + ".base";
if (!NGEN::Filesystem::exists(central_nginx_base_path)) {
dbgWarning(D_NGINX_MANAGER) << "Could not find base NGINX configuration file: " << central_nginx_base_path;
return;
}
NginxUtils::reloadNginx(central_nginx_base_path);
}
private:
void
logError(const string &error)
{
LogGen log(
error,
ReportIS::Level::ACTION,
ReportIS::Audience::SECURITY,
ReportIS::Severity::CRITICAL,
ReportIS::Priority::URGENT,
ReportIS::Tags::POLICY_INSTALLATION
);
log.addToOrigin(LogField("eventTopic", "Central NGINX Management"));
log << LogField("notificationId", "4165c3b1-e9bc-44c3-888b-863e204c1bfb");
log << LogField(
"eventRemediation",
"Please verify your NGINX configuration and enforce policy again. "
"Contact Check Point support if the issue persists."
);
}
void
logInfo(const string &info)
{
LogGen log(
info,
ReportIS::Level::ACTION,
ReportIS::Audience::SECURITY,
ReportIS::Severity::INFO,
ReportIS::Priority::LOW,
ReportIS::Tags::POLICY_INSTALLATION
);
log.addToOrigin(LogField("eventTopic", "Central NGINX Management"));
log << LogField("notificationId", "4165c3b1-e9bc-44c3-888b-863e204c1bfb");
log << LogField("eventRemediation", "No action required");
}
I_MainLoop *i_mainloop = nullptr;
LetsEncryptListener lets_encrypt_listener;
};
CentralNginxManager::CentralNginxManager()
:
Component("Central NGINX Manager"),
pimpl(make_unique<CentralNginxManager::Impl>()) {}
CentralNginxManager::~CentralNginxManager() {}
void
CentralNginxManager::init()
{
pimpl->init();
}
void
CentralNginxManager::fini()
{
pimpl->fini();
}
void
CentralNginxManager::preload()
{
registerExpectedSetting<vector<CentralNginxConfig>>("centralNginxManagement");
registerExpectedConfiguration<string>("Config Component", "configuration path");
registerConfigLoadCb([this]() { pimpl->loadPolicy(); });
}

View File

@@ -1,30 +0,0 @@
// Copyright (C) 2022 Check Point Software Technologies Ltd. All rights reserved.
// Licensed under the Apache License, Version 2.0 (the "License");
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#ifndef __LETS_ENCRYPT_HANDLER_H__
#define __LETS_ENCRYPT_HANDLER_H__
#include <string>
#include "maybe_res.h"
class LetsEncryptListener
{
public:
bool init();
private:
Maybe<std::string> getChallengeValue(const std::string &uri) const;
};
#endif // __LETS_ENCRYPT_HANDLER_H__

View File

@@ -1,76 +0,0 @@
// Copyright (C) 2022 Check Point Software Technologies Ltd. All rights reserved.
// Licensed under the Apache License, Version 2.0 (the "License");
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "lets_encrypt_listener.h"
#include <string>
#include "central_nginx_manager.h"
#include "debug.h"
using namespace std;
USE_DEBUG_FLAG(D_NGINX_MANAGER);
bool
LetsEncryptListener::init()
{
dbgInfo(D_NGINX_MANAGER) << "Starting Lets Encrypt Listener";
return Singleton::Consume<I_RestApi>::by<CentralNginxManager>()->addWildcardGetCall(
".well-known/acme-challenge/",
[&] (const string &uri) -> string
{
Maybe<string> maybe_challenge_value = getChallengeValue(uri);
if (!maybe_challenge_value.ok()) {
dbgWarning(D_NGINX_MANAGER)
<< "Could not get challenge value for uri: "
<< uri
<< ", error: "
<< maybe_challenge_value.getErr();
return string{""};
};
dbgTrace(D_NGINX_MANAGER) << "Got challenge value: " << maybe_challenge_value.unpack();
return maybe_challenge_value.unpack();
}
);
}
Maybe<string>
LetsEncryptListener::getChallengeValue(const string &uri) const
{
string challenge_key = uri.substr(uri.find_last_of('/') + 1);
string api_query = "/api/lets-encrypt-challenge?http_challenge_key=" + challenge_key;
dbgInfo(D_NGINX_MANAGER) << "Getting challenge value via: " << api_query;
MessageMetadata md;
md.insertHeader("X-Tenant-Id", Singleton::Consume<I_AgentDetails>::by<CentralNginxManager>()->getTenantId());
Maybe<HTTPResponse, HTTPResponse> maybe_http_challenge_value =
Singleton::Consume<I_Messaging>::by<CentralNginxManager>()->sendSyncMessage(
HTTPMethod::GET,
api_query,
string("{}"),
MessageCategory::GENERIC,
md
);
if (!maybe_http_challenge_value.ok()) return genError(maybe_http_challenge_value.getErr().getBody());
string challenge_value = maybe_http_challenge_value.unpack().getBody();
if (!challenge_value.empty() && challenge_value.front() == '"' && challenge_value.back() == '"') {
challenge_value = challenge_value.substr(1, challenge_value.size() - 2);
}
return challenge_value;
}

View File

@@ -67,18 +67,18 @@ public:
dbgTrace(D_GEO_FILTER) << getListenerName() << " new transaction event";
if (!event.isLastHeader()) return EventVerdict(ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INSPECT);
std::set<std::string> ip_set;
std::set<std::string> xff_set;
auto env = Singleton::Consume<I_Environment>::by<HttpGeoFilter>();
auto maybe_xff = env->get<std::string>(HttpTransactionData::xff_vals_ctx);
if (!maybe_xff.ok()) {
dbgTrace(D_GEO_FILTER) << "failed to get xff vals from env";
} else {
ip_set = split(maybe_xff.unpack(), ',');
xff_set = split(maybe_xff.unpack(), ',');
}
dbgDebug(D_GEO_FILTER) << getListenerName() << " last header, start lookup";
if (ip_set.size() > 0) {
removeTrustedIpsFromXff(ip_set);
if (xff_set.size() > 0) {
removeTrustedIpsFromXff(xff_set);
} else {
dbgDebug(D_GEO_FILTER) << "xff not found in headers";
}
@@ -88,25 +88,16 @@ public:
dbgWarning(D_GEO_FILTER) << "failed to get source ip from env";
return EventVerdict(default_action);
}
auto source_ip = convertIpAddrToString(maybe_source_ip.unpack());
xff_set.insert(source_ip);
// saas profile setting
bool ignore_source_ip =
getProfileAgentSettingWithDefault<bool>(false, "agent.geoProtaction.ignoreSourceIP");
if (ignore_source_ip){
dbgDebug(D_GEO_FILTER) << "Geo protection ignoring source ip: " << source_ip;
} else {
dbgTrace(D_GEO_FILTER) << "Geo protection source ip: " << source_ip;
ip_set.insert(convertIpAddrToString(maybe_source_ip.unpack()));
}
ngx_http_cp_verdict_e exception_verdict = getExceptionVerdict(ip_set);
ngx_http_cp_verdict_e exception_verdict = getExceptionVerdict(xff_set);
if (exception_verdict != ngx_http_cp_verdict_e::TRAFFIC_VERDICT_IRRELEVANT) {
return EventVerdict(exception_verdict);
}
ngx_http_cp_verdict_e geo_lookup_verdict = getGeoLookupVerdict(ip_set);
ngx_http_cp_verdict_e geo_lookup_verdict = getGeoLookupVerdict(xff_set);
if (geo_lookup_verdict != ngx_http_cp_verdict_e::TRAFFIC_VERDICT_IRRELEVANT) {
return EventVerdict(geo_lookup_verdict);
}
@@ -336,14 +327,6 @@ private:
ngx_http_cp_verdict_e verdict = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_IRRELEVANT;
I_GeoLocation *i_geo_location = Singleton::Consume<I_GeoLocation>::by<HttpGeoFilter>();
EnumArray<I_GeoLocation::GeoLocationField, std::string> geo_location_data;
auto env = Singleton::Consume<I_Environment>::by<HttpGeoFilter>();
string source_id;
auto maybe_source_id = env->get<std::string>(HttpTransactionData::source_identifier);
if (!maybe_source_id.ok()) {
dbgTrace(D_GEO_FILTER) << "failed to get source identifier from env";
} else {
source_id = maybe_source_id.unpack();
}
for (const std::string& source : sources) {
@@ -360,7 +343,7 @@ private:
auto asset_location = i_geo_location->lookupLocation(maybe_source_ip.unpack());
if (!asset_location.ok()) {
dbgDebug(D_GEO_FILTER) << "Lookup location failed for source: " <<
dbgWarning(D_GEO_FILTER) << "Lookup location failed for source: " <<
source <<
", Error: " <<
asset_location.getErr();
@@ -375,15 +358,11 @@ private:
<< country_code
<< ", country name: "
<< country_name
<< ", ip address: "
<< source
<< ", source identifier: "
<< source_id;
<< ", source ip address: "
<< source;
unordered_map<string, set<string>> exception_value_country_code = {
{"countryCode", {country_code}},
{"sourceIdentifier", {source_id}}
{"countryCode", {country_code}}
};
auto matched_behavior_maybe = getBehaviorsVerdict(exception_value_country_code, geo_location_data);
if (matched_behavior_maybe.ok()) {
@@ -395,8 +374,7 @@ private:
}
unordered_map<string, set<string>> exception_value_country_name = {
{"countryName", {country_name}},
{"sourceIdentifier", {source_id}}
{"countryName", {country_name}}
};
matched_behavior_maybe = getBehaviorsVerdict(exception_value_country_name, geo_location_data);
if (matched_behavior_maybe.ok()) {
@@ -491,6 +469,5 @@ void
HttpGeoFilter::preload()
{
registerExpectedConfiguration<GeoConfig>("rulebase", "httpGeoFilter");
registerExpectedConfiguration<UsersAllIdentifiersConfig>("rulebase", "usersIdentifiers");
registerConfigLoadCb([this]() { pimpl->loadDefaultAction(); });
}

View File

@@ -43,10 +43,7 @@ CompoundProtection::Impl::getMatch(const set<PMPattern> &matched) const
case Operation::ORDERED_AND: return getMatchOrderedAnd(matched);
}
dbgAssert(false)
<< AlertInfo(AlertTeam::CORE, "ips")
<< "Unknown compound operation: "
<< static_cast<uint>(operation);
dbgAssert(false) << "Unknown compound operation: " << static_cast<uint>(operation);
return MatchType::NO_MATCH;
}

View File

@@ -29,8 +29,6 @@
#include "pm_hook.h"
#include "i_generic_rulebase.h"
#define DEFAULT_IPS_YIELD_COUNT 500
/// \namespace IPSSignatureSubTypes
/// \brief Namespace containing subtypes for IPS signatures.
namespace IPSSignatureSubTypes
@@ -338,23 +336,9 @@ public:
return metadata.getYear();
}
bool
isOk() const
{
return is_loaded;
}
static void
setYieldCounter(int new_yield_cnt)
{
yield_on_load_cnt = new_yield_cnt;
}
private:
IPSSignatureMetaData metadata;
std::shared_ptr<BaseSignature> rule;
bool is_loaded;
static int yield_on_load_cnt;
};
/// \class SignatureAndAction

View File

@@ -98,7 +98,6 @@ public:
registerListener();
table = Singleton::Consume<I_Table>::by<IPSComp>();
env = Singleton::Consume<I_Environment>::by<IPSComp>();
updateSigsYieldCount();
}
void
@@ -308,20 +307,6 @@ public:
EventVerdict respond (const EndTransactionEvent &) override { return ACCEPT; }
void
updateSigsYieldCount()
{
const char *ips_yield_env_str = getenv("CPNANO_IPS_LOAD_YIELD_CNT");
int ips_yield_default = DEFAULT_IPS_YIELD_COUNT;
if (ips_yield_env_str != nullptr) {
dbgDebug(D_IPS) << "CPNANO_IPS_LOAD_YIELD_CNT env variable is set to " << ips_yield_env_str;
ips_yield_default = atoi(ips_yield_env_str);
}
int yield_limit = getProfileAgentSettingWithDefault<int>(ips_yield_default, "ips.sigsYieldCnt");
dbgDebug(D_IPS) << "Setting IPS yield count to " << yield_limit;
IPSSignatureSubTypes::CompleteSignature::setYieldCounter(yield_limit);
}
private:
static void setDrop(IPSEntry &state) { state.setDrop(); }
static bool isDrop(const IPSEntry &state) { return state.isDrop(); }
@@ -388,7 +373,6 @@ IPSComp::preload()
registerExpectedConfigFile("ips", Config::ConfigFileType::Policy);
registerExpectedConfigFile("ips", Config::ConfigFileType::Data);
registerExpectedConfigFile("snort", Config::ConfigFileType::Policy);
registerConfigLoadCb([this]() { pimpl->updateSigsYieldCount(); });
ParameterException::preload();

View File

@@ -8,9 +8,7 @@ IPSConfiguration::Context::Context(ContextType _type, uint history) : type(_type
uint
IPSConfiguration::Context::getHistorySize() const
{
dbgAssert(type == ContextType::HISTORY)
<< AlertInfo(AlertTeam::CORE, "ips")
<< "Try to access history size for non-history context";
dbgAssert(type == ContextType::HISTORY) << "Try to access history size for non-history context";
return history_size;
}
@@ -71,8 +69,6 @@ uint
IPSConfiguration::getHistorySize(const string &name) const
{
auto context = context_config.find(name);
dbgAssert(context != context_config.end())
<< AlertInfo(AlertTeam::CORE, "ips")
<< "Try to access history size for non-exiting context";
dbgAssert(context != context_config.end()) << "Try to access history size for non-exiting context";
return context->second.getHistorySize();
}

View File

@@ -26,8 +26,6 @@ static const map<string, IPSConfiguration::Context> default_conf_mapping = {
};
static const IPSConfiguration default_conf(default_conf_mapping);
static const IPSSignatures default_ips_sigs;
static const SnortSignatures default_snort_sigs;
IPSEntry::IPSEntry() : TableOpaqueSerialize<IPSEntry>(this) {}
@@ -53,9 +51,9 @@ IPSEntry::respond(const ParsedContext &parsed)
ctx.registerValue(name, buf);
ctx.activate();
auto &signatures = getConfigurationWithDefault(default_ips_sigs, "IPS", "IpsProtections");
auto &signatures = getConfigurationWithDefault(IPSSignatures(), "IPS", "IpsProtections");
bool should_drop = signatures.isMatchedPrevent(parsed.getName(), buf);
auto &snort_signatures = getConfigurationWithDefault(default_snort_sigs, "IPSSnortSigs", "SnortProtections");
auto &snort_signatures = getConfigurationWithDefault(SnortSignatures(), "IPSSnortSigs", "SnortProtections");
should_drop |= snort_signatures.isMatchedPrevent(parsed.getName(), buf);
ctx.deactivate();

View File

@@ -45,8 +45,6 @@ static const map<string, IPSLevel> levels = {
{ "Very Low", IPSLevel::VERY_LOW }
};
int CompleteSignature::yield_on_load_cnt = DEFAULT_IPS_YIELD_COUNT;
static IPSLevel
getLevel(const string &level_string, const string &attr_name)
{
@@ -86,7 +84,7 @@ IPSSignatureMetaData::getSeverityString() const
return "Critical";
}
dbgAssert(false) << AlertInfo(AlertTeam::CORE, "ips") << "Illegal severity value: " << static_cast<uint>(severity);
dbgAssert(false) << "Illegal severity value: " << static_cast<uint>(severity);
return "Critical";
}
@@ -118,10 +116,7 @@ IPSSignatureMetaData::getPerformanceString() const
return "Critical";
}
dbgAssert(false)
<< AlertInfo(AlertTeam::CORE, "ips")
<< "Illegal performance value: "
<< static_cast<uint>(performance);
dbgAssert(false) << "Illegal performance value: " << static_cast<uint>(performance);
return "Critical";
}
@@ -221,37 +216,10 @@ IPSSignatureMetaData::getYear() const
void
CompleteSignature::load(cereal::JSONInputArchive &ar)
{
static int sigs_load_counter = 0;
static I_Environment *env = Singleton::Consume<I_Environment>::by<IPSComp>();
static bool post_init = false;
if (!post_init) {
auto routine_id = Singleton::Consume<I_MainLoop>::by<IPSComp>()->getCurrentRoutineId();
if (routine_id.ok()) {
post_init = true;
dbgInfo(D_IPS) << "Loading signatures post init, enabling yield with limit " << yield_on_load_cnt;
}
}
try {
ar(cereal::make_nvp("protectionMetadata", metadata));
RuleDetection rule_detection(metadata.getName());
ar(cereal::make_nvp("detectionRules", rule_detection));
rule = rule_detection.getRule();
is_loaded = true;
} catch (cereal::Exception &e) {
is_loaded = false;
dbgWarning(D_IPS) << "Failed to load signature: " << e.what();
}
if (post_init && (yield_on_load_cnt > 0) && (++sigs_load_counter == yield_on_load_cnt)) {
sigs_load_counter = 0;
auto maybe_is_async = env->get<bool>("Is Async Config Load");
if (maybe_is_async.ok() && *maybe_is_async == true) {
dbgTrace(D_IPS) << "Yielding after " << yield_on_load_cnt << " signatures";
Singleton::Consume<I_MainLoop>::by<IPSComp>()->yield(false);
}
}
}
MatchType
@@ -396,16 +364,7 @@ SignatureAndAction::matchSilent(const Buffer &sample) const
if (method.ok()) log << LogField("httpMethod", method.unpack());
auto path = env->get<Buffer>("HTTP_PATH_DECODED");
if (path.ok()) {
log << LogField("httpUriPath", getSubString(path, 1536), LogFieldOption::XORANDB64);
} else {
auto transaction_path = env->get<string>(HttpTransactionData::uri_path_decoded);
if (transaction_path.ok()) {
auto uri_path = transaction_path.unpack();
auto question_mark = uri_path.find('?');
log << LogField("httpUriPath", uri_path.substr(0, question_mark), LogFieldOption::XORANDB64);
}
}
if (path.ok()) log << LogField("httpUriPath", getSubString(path, 1536), LogFieldOption::XORANDB64);
auto req_header = ips_state.getTransactionData(IPSCommonTypes::requests_header_for_log);
if (req_header.ok()) log << LogField("httpRequestHeaders", getSubString(req_header), LogFieldOption::XORANDB64);
@@ -523,30 +482,13 @@ SignatureAndAction::isMatchedPrevent(const Buffer &context_buffer, const set<PMP
auto method = env->get<string>(HttpTransactionData::method_ctx);
if (method.ok()) log << LogField("httpMethod", method.unpack());
uint max_size = getConfigurationWithDefault<uint>(1536, "IPS", "Max Field Size");
if (trigger.isWebLogFieldActive(url_path)) {
auto path = env->get<Buffer>("HTTP_PATH_DECODED");
if (path.ok()) {
if (path.ok() && trigger.isWebLogFieldActive(url_path)) {
log << LogField("httpUriPath", getSubString(path, max_size), LogFieldOption::XORANDB64);
} else {
auto transaction_path = env->get<string>(HttpTransactionData::uri_path_decoded);
if (transaction_path.ok()) {
auto uri_path = transaction_path.unpack();
auto question_mark = uri_path.find('?');
log << LogField("httpUriPath", uri_path.substr(0, question_mark), LogFieldOption::XORANDB64);
}
}
}
if (trigger.isWebLogFieldActive(url_query)) {
auto query = env->get<Buffer>("HTTP_QUERY_DECODED");
if (query.ok()) {
if (query.ok() && trigger.isWebLogFieldActive(url_query)) {
log << LogField("httpUriQuery", getSubString(query, max_size), LogFieldOption::XORANDB64);
} else {
auto transaction_query = env->get<string>(HttpTransactionData::uri_query_decoded);
if (transaction_query.ok()) {
log << LogField("httpUriQuery", transaction_query.unpack());
}
}
}
auto res_code = env->get<Buffer>("HTTP_RESPONSE_CODE");
@@ -588,11 +530,9 @@ IPSSignaturesResource::load(cereal::JSONInputArchive &ar)
all_signatures.reserve(sigs.size());
for (auto &sig : sigs) {
if (sig.isOk()) {
all_signatures.emplace_back(make_shared<CompleteSignature>(move(sig)));
}
}
}
class CompleteSignatureWrapper
{

View File

@@ -29,8 +29,6 @@ public:
{
comp.preload();
comp.init();
auto err = genError("not coroutine");
EXPECT_CALL(mainloop, getCurrentRoutineId()).WillRepeatedly(Return(Maybe<I_MainLoop::RoutineID>(err)));
}
~ComponentTest()

View File

@@ -7,7 +7,7 @@ TEST(configuration, basic_context)
IPSConfiguration::Context ctx1(IPSConfiguration::ContextType::HISTORY, 254);
EXPECT_EQ(ctx1.getType(), IPSConfiguration::ContextType::HISTORY);
EXPECT_EQ(ctx1.getHistorySize(), 254u);
EXPECT_EQ(ctx1.getHistorySize(), 254);
IPSConfiguration::Context ctx2(IPSConfiguration::ContextType::NORMAL, 0);
EXPECT_EQ(ctx2.getType(), IPSConfiguration::ContextType::NORMAL);
@@ -42,7 +42,7 @@ TEST(configuration, read_configuration)
auto body = conf.getContext("HTTP_REQUEST_BODY");
EXPECT_EQ(body.getType(), IPSConfiguration::ContextType::HISTORY);
EXPECT_EQ(conf.getHistorySize("HTTP_REQUEST_BODY"), 100u);
EXPECT_EQ(conf.getHistorySize("HTTP_REQUEST_BODY"), 100);
auto header = conf.getContext("HTTP_REQUEST_HEADER");
EXPECT_EQ(header.getType(), IPSConfiguration::ContextType::KEEP);

View File

@@ -41,8 +41,6 @@ public:
EntryTest()
{
ON_CALL(table, getState(_)).WillByDefault(Return(ptr));
auto err = genError("not coroutine");
EXPECT_CALL(mock_mainloop, getCurrentRoutineId()).WillRepeatedly(Return(Maybe<I_MainLoop::RoutineID>(err)));
}
void
@@ -139,8 +137,8 @@ private:
TEST_F(EntryTest, basic_inherited_functions)
{
EXPECT_EQ(IPSEntry::name(), "IPS");
EXPECT_EQ(IPSEntry::currVer(), 0u);
EXPECT_EQ(IPSEntry::minVer(), 0u);
EXPECT_EQ(IPSEntry::currVer(), 0);
EXPECT_EQ(IPSEntry::minVer(), 0);
EXPECT_NE(IPSEntry::prototype(), nullptr);
EXPECT_EQ(entry.getListenerName(), IPSEntry::name());

View File

@@ -2,7 +2,6 @@
#include "cptest.h"
#include "environment.h"
#include "config_component.h"
#include "mock/mock_mainloop.h"
using namespace std;
using namespace testing;
@@ -62,9 +61,6 @@ TEST(resources, basic_resource)
{
ConfigComponent conf;
::Environment env;
NiceMock<MockMainLoop> mock_mainloop;
auto err = genError("not coroutine");
EXPECT_CALL(mock_mainloop, getCurrentRoutineId()).WillRepeatedly(Return(Maybe<I_MainLoop::RoutineID>(err)));
conf.preload();
@@ -75,7 +71,7 @@ TEST(resources, basic_resource)
Singleton::Consume<Config::I_Config>::from(conf)->loadConfiguration(resource);
auto loaded_resources = getSettingWithDefault(IPSSignaturesResource(), "IPS", "protections");
EXPECT_EQ(loaded_resources.getSignatures().size(), 2u);
EXPECT_EQ(loaded_resources.getSignatures().size(), 2);
auto version = getSettingWithDefault<string>("", "IPS", "VersionId");
EXPECT_EQ(version, "1234567");
}

View File

@@ -60,12 +60,7 @@ public:
{
IPSHelper::has_deobfuscation = true;
generic_rulebase.preload();
env.preload();
env.init();
EXPECT_CALL(logs, getCurrentLogId()).Times(AnyNumber());
auto err = genError("not coroutine");
EXPECT_CALL(mock_mainloop, getCurrentRoutineId()).WillRepeatedly(Return(Maybe<I_MainLoop::RoutineID>(err)));
ON_CALL(table, getState(_)).WillByDefault(Return(&ips_state));
{
stringstream ss;
@@ -109,12 +104,6 @@ public:
cereal::JSONInputArchive ar(ss);
high_medium_confidance_signatures.load(ar);
}
{
stringstream ss;
ss << "[" << signature_performance_high << ", " << signature_broken << "]";
cereal::JSONInputArchive ar(ss);
single_broken_signature.load(ar);
}
}
~SignatureTest()
@@ -128,6 +117,9 @@ public:
void
loadExceptions()
{
env.preload();
env.init();
BasicRuleConfig::preload();
registerExpectedConfiguration<ParameterException>("rulebase", "exception");
@@ -197,7 +189,6 @@ public:
void
load(const IPSSignaturesResource &policy, const string &severity, const string &confidence)
{
Singleton::Consume<I_Environment>::from(env)->registerValue<bool>("Is Async Config Load", false);
setResource(policy, "IPS", "protections");
stringstream ss;
ss << "{";
@@ -259,7 +250,6 @@ public:
IPSSignaturesResource performance_signatures1;
IPSSignaturesResource performance_signatures2;
IPSSignaturesResource performance_signatures3;
IPSSignaturesResource single_broken_signature;
NiceMock<MockTable> table;
MockAgg mock_agg;
@@ -493,26 +483,6 @@ private:
"\"context\": [\"HTTP_REQUEST_BODY\", \"HTTP_RESPONSE_BODY\"]"
"}"
"}";
string signature_broken =
"{"
"\"protectionMetadata\": {"
"\"protectionName\": \"BrokenTest\","
"\"maintrainId\": \"101\","
"\"severity\": \"Medium High\","
"\"confidenceLevel\": \"Low\","
"\"performanceImpact\": \"High\","
"\"lastUpdate\": \"20210420\","
"\"tags\": [],"
"\"cveList\": []"
"},"
"\"detectionRules\": {"
"\"type\": \"simple\","
"\"SSM\": \"\","
"\"keywosrds\": \"data: \\\"www\\\";\","
"\"context\": [\"HTTP_REQUEST_BODY\", \"HTTP_RESPONSE_BODY\"]"
"}"
"}";
};
TEST_F(SignatureTest, basic_load_of_signatures)
@@ -695,14 +665,3 @@ TEST_F(SignatureTest, high_confidance_signatures_matching)
expectLog("\"protectionId\": \"Test4\"", "\"matchedSignatureConfidence\": \"Medium\"");
EXPECT_FALSE(checkData("mmm"));
}
TEST_F(SignatureTest, broken_signature)
{
load(single_broken_signature, "Low or above", "Low");
EXPECT_FALSE(checkData("ggg"));
expectLog("\"matchedSignaturePerformance\": \"High\"");
EXPECT_TRUE(checkData("fff"));
EXPECT_FALSE(checkData("www"));
}

View File

@@ -131,12 +131,8 @@ public:
EventVerdict
respond(const WaitTransactionEvent &) override
{
if (!isAppEnabled()) {
dbgTrace(D_L7_ACCESS_CONTROL) << "Returning Accept verdict as the Layer-7 Access Control app is disabled";
return ngx_http_cp_verdict_e::TRAFFIC_VERDICT_ACCEPT;
}
dbgFlow(D_L7_ACCESS_CONTROL) << "Handling wait verdict";
dbgTrace(D_L7_ACCESS_CONTROL) << "Handling wait verdict";
return handleEvent();
}
@@ -389,29 +385,8 @@ Layer7AccessControl::Impl::init()
i_intelligence = Singleton::Consume<I_Intelligence_IS_V2>::by<Layer7AccessControl>();
i_mainloop = Singleton::Consume<I_MainLoop>::by<Layer7AccessControl>();
int cache_expiration_in_seconds = 30;
string cache_expiration_env = getenv("CROWDSEC_CACHE_EXPIRATION") ? getenv("CROWDSEC_CACHE_EXPIRATION") : "";
if (!cache_expiration_env.empty()) {
if (
all_of(cache_expiration_env.begin(), cache_expiration_env.end(), ::isdigit)
&& stoi(cache_expiration_env) > 0
) {
cache_expiration_in_seconds = stoi(cache_expiration_env);
dbgInfo(D_L7_ACCESS_CONTROL)
<< "Successfully read cache expiration value from env: "
<< cache_expiration_env;
} else {
dbgWarning(D_L7_ACCESS_CONTROL)
<< "An invalid cache expiration value was provided in env: "
<< cache_expiration_env;
}
}
chrono::seconds expiration(
getProfileAgentSettingWithDefault<uint>(
cache_expiration_in_seconds,
"layer7AccessControl.crowdsec.cacheExpiration"
)
chrono::minutes expiration(
getProfileAgentSettingWithDefault<uint>(60u, "layer7AccessControl.crowdsec.cacheExpiration")
);
ip_reputation_cache.startExpiration(

View File

@@ -247,9 +247,7 @@ Layer7AccessControlTest::verifyReport(
string log = reportToStr(report);
dbgTrace(D_L7_ACCESS_CONTROL) << "Report: " << log;
if (!source_identifier.empty()) {
EXPECT_THAT(log, HasSubstr("\"httpSourceId\": \"" + source_identifier + "\""));
}
if (!source_identifier.empty()) EXPECT_THAT(log, HasSubstr("\"httpSourceId\": \"" + source_identifier + "\""));
EXPECT_THAT(log, HasSubstr("\"securityAction\": \"" + security_action + "\""));
EXPECT_THAT(log, HasSubstr("\"eventName\": \"Access Control External Vendor Reputation\""));
EXPECT_THAT(log, HasSubstr("\"httpHostName\": \"juice-shop.checkpoint.com\""));

View File

@@ -22,5 +22,4 @@ add_library(local_policy_mgmt_gen
access_control_practice.cc
configmaps.cc
reverse_proxy_section.cc
policy_activation_data.cc
)

View File

@@ -228,11 +228,7 @@ AccessControlPracticeSpec::load(cereal::JSONInputArchive &archive_in)
dbgTrace(D_LOCAL_POLICY) << "Loading AppSec practice spec";
parseAppsecJSONKey<string>("name", practice_name, archive_in);
parseAppsecJSONKey<string>("practiceMode", mode, archive_in, "inherited");
if (valid_modes.count(mode) == 0) {
dbgWarning(D_LOCAL_POLICY) << "AppSec Access control practice mode invalid: " << mode;
throw PolicyGenException("AppSec Access control practice mode invalid: " + mode);
}
parseAppsecJSONKey<string>("practiceMode", mode, archive_in);
parseAppsecJSONKey<string>("appsecClassName", appsec_class_name, archive_in);
parseMandatoryAppsecJSONKey<AccessControlRateLimit>("rateLimit", rate_limit, archive_in);
}

View File

@@ -19,14 +19,7 @@ using namespace std;
USE_DEBUG_FLAG(D_LOCAL_POLICY);
// LCOV_EXCL_START Reason: no test exist
static const set<string> valid_modes = {
"prevent-learn",
"detect-learn",
"prevent",
"detect",
"inactive",
"as-top-level"
};
static const set<string> valid_modes = {"prevent-learn", "detect-learn", "prevent", "detect", "inactive"};
static const set<string> valid_confidences = {"medium", "high", "critical"};
void
@@ -145,12 +138,16 @@ AppSecPracticeWebAttacks::load(cereal::JSONInputArchive &archive_in)
dbgWarning(D_LOCAL_POLICY) << "AppSec practice override mode invalid: " << mode;
}
if (getMode() == "Prevent") {
parseAppsecJSONKey<string>("minimum-confidence", minimum_confidence, archive_in, "critical");
if (valid_confidences.count(minimum_confidence) == 0) {
dbgWarning(D_LOCAL_POLICY)
<< "AppSec practice override minimum confidence invalid: "
<< minimum_confidence;
}
} else {
minimum_confidence = "Transparent";
}
parseAppsecJSONKey<int>("max-body-size-kb", max_body_size_kb, archive_in, 1000000);
parseAppsecJSONKey<int>("max-header-size-bytes", max_header_size_bytes, archive_in, 102400);
parseAppsecJSONKey<int>("max-object-depth", max_object_depth, archive_in, 40);
@@ -192,11 +189,8 @@ AppSecPracticeWebAttacks::getMode(const string &default_mode) const
{
if (isModeInherited(mode) || (key_to_practices_val2.find(mode) == key_to_practices_val2.end())) {
dbgError(D_LOCAL_POLICY) << "Couldn't find a value for key: " << mode << ". Returning " << default_mode;
if(key_to_practices_val2.find(default_mode) == key_to_practices_val2.end()) {
return default_mode;
}
return key_to_practices_val2.at(default_mode);
}
return key_to_practices_val2.at(mode);
}
@@ -410,7 +404,6 @@ AppsecPracticeAntiBotSection::save(cereal::JSONOutputArchive &out_ar) const
}
// LCOV_EXCL_START Reason: no test exist
// Used for V1Beta1
WebAppSection::WebAppSection(
const string &_application_urls,
const string &_asset_id,
@@ -424,7 +417,7 @@ WebAppSection::WebAppSection(
const LogTriggerSection &parsed_log_trigger,
const string &default_mode,
const AppSecTrustedSources &parsed_trusted_sources,
const std::map<std::string, std::vector<InnerException>> &exceptions)
const vector<InnerException> &parsed_exceptions)
:
application_urls(_application_urls),
asset_id(_asset_id),
@@ -434,34 +427,21 @@ WebAppSection::WebAppSection(
practice_id(_practice_id),
practice_name(_practice_name),
context(_context),
web_attack_mitigation_severity(parsed_appsec_spec.getWebAttacks().getMinimumConfidence()),
web_attack_mitigation_mode(parsed_appsec_spec.getWebAttacks().getMode(default_mode)),
csrf_protection_mode("Disabled"),
open_redirect_mode("Disabled"),
error_disclosure_mode("Disabled"),
schema_validation_mode("Disabled"),
schema_validation_enforce_level("fullSchema"),
practice_advanced_config(parsed_appsec_spec),
anti_bots(parsed_appsec_spec.getAntiBot()),
trusted_sources({ parsed_trusted_sources })
{
auto mitigation_sevirity = parsed_appsec_spec.getWebAttacks().getMinimumConfidence();
if (key_to_mitigation_severity.find(mitigation_sevirity) == key_to_mitigation_severity.end()) {
dbgWarning(D_LOCAL_POLICY)
<< "web attack mitigation severity invalid: "
<< mitigation_sevirity;
throw PolicyGenException("web attack mitigation severity invalid: " + mitigation_sevirity);
} else {
web_attack_mitigation_severity = key_to_mitigation_severity.at(mitigation_sevirity);
}
web_attack_mitigation = web_attack_mitigation_mode != "Disabled";
web_attack_mitigation_severity =
web_attack_mitigation_mode != "Prevent" ? "Transparent" :
web_attack_mitigation_severity;
web_attack_mitigation_action =
web_attack_mitigation_mode != "Prevent" ? "Transparent" :
web_attack_mitigation_severity == "Critical" ? "Low" :
web_attack_mitigation_severity == "High" ? "Balanced" :
web_attack_mitigation_severity == "Medium" ? "High" :
web_attack_mitigation_severity == "critical" ? "low" :
web_attack_mitigation_severity == "high" ? "balanced" :
web_attack_mitigation_severity == "medium" ? "high" :
"Error";
triggers.push_back(TriggersInWaapSection(parsed_log_trigger));
@@ -469,11 +449,8 @@ WebAppSection::WebAppSection(
overrides.push_back(AppSecOverride(source_ident));
}
for (const auto &exception : exceptions) {
for (const auto &inner_exception : exception.second) {
overrides.push_back(AppSecOverride(inner_exception));
}
for (const InnerException &exception : parsed_exceptions) {
overrides.push_back(AppSecOverride(exception));
}
}
@@ -489,16 +466,11 @@ WebAppSection::WebAppSection(
const string &_context,
const string &_web_attack_mitigation_severity,
const string &_web_attack_mitigation_mode,
const string &_bot_protection,
const string &_schema_validation_mode,
const string &_schema_validation_enforce_level,
const vector<string> &_schema_validation_oas,
const PracticeAdvancedConfig &_practice_advanced_config,
const AppsecPracticeAntiBotSection &_anti_bots,
const LogTriggerSection &parsed_log_trigger,
const AppSecTrustedSources &parsed_trusted_sources,
const NewAppSecWebAttackProtections &protections,
const vector<InnerException> &exceptions)
const NewAppSecWebAttackProtections &protections)
:
application_urls(_application_urls),
asset_id(_asset_id),
@@ -508,29 +480,18 @@ WebAppSection::WebAppSection(
practice_id(_practice_id),
practice_name(_practice_name),
context(_context),
web_attack_mitigation_severity(_web_attack_mitigation_severity),
web_attack_mitigation_mode(_web_attack_mitigation_mode),
bot_protection(_bot_protection),
schema_validation_mode(_schema_validation_mode),
schema_validation_enforce_level(_schema_validation_enforce_level),
schema_validation_oas(_schema_validation_oas),
practice_advanced_config(_practice_advanced_config),
anti_bots(_anti_bots),
trusted_sources({ parsed_trusted_sources })
{
if (key_to_mitigation_severity.find(_web_attack_mitigation_severity) == key_to_mitigation_severity.end()) {
dbgWarning(D_LOCAL_POLICY)
<< "web attack mitigation severity invalid: "
<< _web_attack_mitigation_severity;
throw PolicyGenException("web attack mitigation severity invalid: " + _web_attack_mitigation_severity);
} else {
web_attack_mitigation_severity = key_to_mitigation_severity.at(_web_attack_mitigation_severity);
}
web_attack_mitigation = web_attack_mitigation_mode != "Disabled";
web_attack_mitigation_action =
web_attack_mitigation_mode != "Prevent" ? "Transparent" :
web_attack_mitigation_severity == "Critical" ? "Low" :
web_attack_mitigation_severity == "High" ? "Balanced" :
web_attack_mitigation_severity == "Medium" ? "High" :
web_attack_mitigation_severity == "critical" ? "low" :
web_attack_mitigation_severity == "high" ? "balanced" :
web_attack_mitigation_severity == "medium" ? "high" :
"Error";
csrf_protection_mode = protections.getCsrfProtectionMode(_web_attack_mitigation_mode);
@@ -541,11 +502,6 @@ WebAppSection::WebAppSection(
for (const SourcesIdentifiers &source_ident : parsed_trusted_sources.getSourcesIdentifiers()) {
overrides.push_back(AppSecOverride(source_ident));
}
for (const auto &exception : exceptions) {
overrides.push_back(AppSecOverride(exception));
}
}
// LCOV_EXCL_STOP
@@ -553,6 +509,8 @@ WebAppSection::WebAppSection(
void
WebAppSection::save(cereal::JSONOutputArchive &out_ar) const
{
string disabled_str = "Disabled";
string detect_str = "Detect";
vector<string> empty_list;
out_ar(
cereal::make_nvp("context", context),
@@ -570,10 +528,9 @@ WebAppSection::save(cereal::JSONOutputArchive &out_ar) const
cereal::make_nvp("assetName", asset_name),
cereal::make_nvp("ruleId", rule_id),
cereal::make_nvp("ruleName", rule_name),
cereal::make_nvp("schemaValidation", schema_validation_mode == "Prevent"),
cereal::make_nvp("schemaValidation_v2", schema_validation_mode),
cereal::make_nvp("oas", schema_validation_oas),
cereal::make_nvp("schemaValidationEnforceLevel", schema_validation_enforce_level),
cereal::make_nvp("schemaValidation", false),
cereal::make_nvp("schemaValidation_v2", disabled_str),
cereal::make_nvp("oas", empty_list),
cereal::make_nvp("triggers", triggers),
cereal::make_nvp("applicationUrls", application_urls),
cereal::make_nvp("overrides", overrides),
@@ -581,7 +538,7 @@ WebAppSection::save(cereal::JSONOutputArchive &out_ar) const
cereal::make_nvp("waapParameters", empty_list),
cereal::make_nvp("botProtection", false),
cereal::make_nvp("antiBot", anti_bots),
cereal::make_nvp("botProtection_v2", bot_protection != "" ? bot_protection : string("Detect"))
cereal::make_nvp("botProtection_v2", detect_str)
);
}

View File

@@ -146,9 +146,7 @@ AppsecException::load(cereal::JSONInputArchive &archive_in)
{
dbgTrace(D_LOCAL_POLICY) << "Loading AppSec exception";
parseAppsecJSONKey<string>("name", name, archive_in);
AppsecExceptionSpec single_exception_spec;
single_exception_spec.load(archive_in);
exception_spec.push_back(single_exception_spec);
archive_in(CEREAL_NVP(exception_spec));
}
void
@@ -176,7 +174,7 @@ ExceptionMatch::ExceptionMatch(const AppsecExceptionSpec &parsed_exception)
{
bool single_condition = parsed_exception.isOneCondition();
for (auto &attrib : attributes) {
auto attrib_name = (attrib.first == "sourceIp" ? "sourceIP" : attrib.first);
auto &attrib_name = attrib.first;
auto &attrib_getter = attrib.second;
auto exceptions_value = attrib_getter(parsed_exception);
if (exceptions_value.empty()) continue;

View File

@@ -275,7 +275,7 @@ public:
const LogTriggerSection &parsed_log_trigger,
const std::string &default_mode,
const AppSecTrustedSources &parsed_trusted_sources,
const std::map<std::string, std::vector<InnerException>> &exceptions
const std::vector<InnerException> &parsed_exceptions
);
// used for V1beta2
@@ -290,23 +290,17 @@ public:
const std::string &_context,
const std::string &_web_attack_mitigation_severity,
const std::string &_web_attack_mitigation_mode,
const std::string &_bot_protection,
const std::string &schema_validation_mode,
const std::string &schema_validation_enforce_level,
const std::vector<std::string> &schema_validation_oas,
const PracticeAdvancedConfig &_practice_advanced_config,
const AppsecPracticeAntiBotSection &_anti_bots,
const LogTriggerSection &parsed_log_trigger,
const AppSecTrustedSources &parsed_trusted_sources,
const NewAppSecWebAttackProtections &protections,
const std::vector<InnerException> &exceptions);
const NewAppSecWebAttackProtections &protections);
void save(cereal::JSONOutputArchive &out_ar) const;
bool operator< (const WebAppSection &other) const;
private:
bool web_attack_mitigation;
std::string application_urls;
std::string asset_id;
std::string asset_name;
@@ -321,15 +315,12 @@ private:
std::string csrf_protection_mode;
std::string open_redirect_mode;
std::string error_disclosure_mode;
std::string bot_protection;
std::string schema_validation_mode;
std::string schema_validation_enforce_level;
std::vector<std::string> schema_validation_oas;
bool web_attack_mitigation;
std::vector<TriggersInWaapSection> triggers;
PracticeAdvancedConfig practice_advanced_config;
AppsecPracticeAntiBotSection anti_bots;
std::vector<AppSecOverride> overrides;
std::vector<AppSecTrustedSources> trusted_sources;
std::vector<TriggersInWaapSection> triggers;
std::vector<AppSecOverride> overrides;
};
class WebAPISection
@@ -417,7 +408,7 @@ class ParsedRule
{
public:
ParsedRule() {}
ParsedRule(const std::string &_host, const std::string &_mode) : host(_host), mode(_mode) {}
ParsedRule(const std::string &_host) : host(_host) {}
void load(cereal::JSONInputArchive &archive_in);
const std::vector<std::string> & getExceptions() const;

View File

@@ -44,7 +44,7 @@ public:
bool isOneCondition() const;
private:
int conditions_number = 0;
int conditions_number;
std::string action;
std::vector<std::string> country_code;
std::vector<std::string> country_name;

View File

@@ -24,7 +24,6 @@
#include "maybe_res.h"
#include "i_orchestration_tools.h"
#include "i_shell_cmd.h"
#include "i_encryptor.h"
#include "i_messaging.h"
#include "i_env_details.h"
#include "i_agent_details.h"
@@ -41,14 +40,13 @@ class K8sPolicyUtils
Singleton::Consume<I_Messaging>,
Singleton::Consume<I_ShellCmd>,
Singleton::Consume<I_EnvDetails>,
Singleton::Consume<I_Encryptor>,
Singleton::Consume<I_AgentDetails>
{
public:
void init();
std::tuple<std::map<std::string, AppsecLinuxPolicy>, std::map<std::string, V1beta2AppsecLinuxPolicy>>
createAppsecPolicies();
createAppsecPoliciesFromIngresses();
void getClusterId() const;
private:
@@ -82,8 +80,6 @@ private:
void createSnortFile(std::vector<NewAppSecPracticeSpec> &practices) const;
void createSchemaValidationOas(std::vector<NewAppSecPracticeSpec> &practices) const;
template<class T>
std::vector<T> extractV1Beta2ElementsFromCluster(
const std::string &crd_plural,
@@ -101,18 +97,12 @@ private:
) const;
template<class T, class K>
void createPolicyFromIngress(
void createPolicy(
T &appsec_policy,
std::map<std::string, T> &policies,
std::map<AnnotationKeys, std::string> &annotations_values,
const SingleIngressData &item) const;
template<class T, class K>
void createPolicyFromActivation(
T &appsec_policy,
std::map<std::string, T> &policies,
const EnabledPolicy &policy) const;
std::tuple<Maybe<AppsecLinuxPolicy>, Maybe<V1beta2AppsecLinuxPolicy>> createAppsecPolicyK8s(
const std::string &policy_name,
const std::string &ingress_mode
@@ -122,7 +112,6 @@ private:
I_Messaging* messaging = nullptr;
EnvType env_type;
std::string token;
std::string agent_ns;
};
#endif // __K8S_POLICY_UTILS_H__

View File

@@ -49,13 +49,6 @@ static const std::unordered_map<std::string, TriggerType> string_to_trigger_type
{ "WebUserResponse", TriggerType::WebUserResponse }
};
static const std::unordered_map<std::string, std::string> key_to_mitigation_severity = {
{ "high", "High"},
{ "medium", "Medium"},
{ "critical", "Critical"},
{ "Transparent", "Transparent"}
};
static const std::unordered_map<std::string, std::string> key_to_practices_val = {
{ "prevent-learn", "Prevent"},
{ "detect-learn", "Learn"},
@@ -64,14 +57,6 @@ static const std::unordered_map<std::string, std::string> key_to_practices_val =
{ "inactive", "Inactive"}
};
static const std::unordered_map<std::string, std::string> key_to_practices_mode_val = {
{ "prevent-learn", "Prevent"},
{ "detect-learn", "Detect"},
{ "prevent", "Prevent"},
{ "detect", "Detect"},
{ "inactive", "Disabled"}
};
static const std::unordered_map<std::string, std::string> key_to_practices_val2 = {
{ "prevent-learn", "Prevent"},
{ "detect-learn", "Learn"},
@@ -81,8 +66,6 @@ static const std::unordered_map<std::string, std::string> key_to_practices_val2
};
static const std::string default_appsec_url = "http://*:*";
static const std::string default_appsec_name = "Any";
class PolicyGenException : public std::exception
{
@@ -170,7 +153,6 @@ public:
ss.str(modified_json);
try {
cereal::JSONInputArchive in_ar(ss);
in_ar(cereal::make_nvp("apiVersion", api_version));
in_ar(cereal::make_nvp("spec", spec));
in_ar(cereal::make_nvp("metadata", meta_data));
} catch (cereal::Exception &e) {
@@ -192,18 +174,11 @@ public:
return meta_data;
}
const std::string &
getApiVersion() const
{
return api_version;
}
const T & getSpec() const { return spec; }
private:
T spec;
AppsecSpecParserMetaData meta_data;
std::string api_version;
};
#endif // __LOCAL_POLICY_COMMON_H__

View File

@@ -31,7 +31,7 @@ class NewParsedRule
{
public:
NewParsedRule() {}
NewParsedRule(const std::string &_host, const std::string &_mode) : host(_host), mode(_mode) {}
NewParsedRule(const std::string &_host) : host(_host) {}
void load(cereal::JSONInputArchive &archive_in);

View File

@@ -129,7 +129,7 @@ public:
bool shouldBeautifyLogs() const;
bool getCloud() const;
bool isContainerNeeded() const;
bool isK8SNeeded() const;
bool isCefNeeded() const;
bool isSyslogNeeded() const;
const std::string & getSyslogServerIpv4Address() const;
@@ -140,7 +140,7 @@ private:
const NewLoggingService & getCefServiceData() const;
bool cloud = false;
bool container_service = false;
bool k8s_service = false;
bool agent_local = true;
bool beautify_logs = true;
NewLoggingService syslog_service;

View File

@@ -23,8 +23,6 @@
#include "config.h"
#include "debug.h"
#include "local_policy_common.h"
#include "i_orchestration_tools.h"
#include "i_encryptor.h"
bool isModeInherited(const std::string &mode);
@@ -90,8 +88,6 @@ public:
void save(cereal::JSONOutputArchive &out_ar) const;
bool operator<(const IpsProtectionsSection &other) const;
private:
std::string context;
std::string name;
@@ -109,7 +105,7 @@ public:
// LCOV_EXCL_START Reason: no test exist
IPSSection() {};
IPSSection(const std::vector<IpsProtectionsSection> &_ips);
IPSSection(const std::vector<IpsProtectionsSection> &_ips) : ips(_ips) {};
// LCOV_EXCL_STOP
void save(cereal::JSONOutputArchive &out_ar) const;
@@ -142,12 +138,6 @@ public:
const std::string & getMode(const std::string &default_mode = "inactive") const;
private:
const std::string & getRulesMode(
const std::string &mode,
const std::string &default_mode = "inactive"
) const;
std::string override_mode;
std::string max_performance_impact;
std::string min_severity_level;
@@ -497,16 +487,15 @@ private:
SnortSection snort;
};
class NewSnortSignatures
class NewSnortSignaturesAndOpenSchemaAPI
{
public:
NewSnortSignatures() : is_temporary(false) {};
NewSnortSignaturesAndOpenSchemaAPI() : is_temporary(false) {};
void load(cereal::JSONInputArchive &archive_in);
void addFile(const std::string &file_name);
const std::string & getOverrideMode(const std::string &default_mode = "inactive") const;
const std::string & getEnforceLevel() const;
const std::vector<std::string> & getConfigMap() const;
const std::vector<std::string> & getFiles() const;
bool isTemporary() const;
@@ -514,48 +503,35 @@ public:
private:
std::string override_mode;
std::string enforcement_level;
std::vector<std::string> config_map;
std::vector<std::string> files;
bool is_temporary;
};
class NewOpenApiSchema : Singleton::Consume<I_OrchestrationTools>, Singleton::Consume<I_Encryptor>
class NewAppSecWebBotsURI
{
public:
NewOpenApiSchema() {};
void load(cereal::JSONInputArchive &archive_in);
void addOas(const std::string &file);
const std::string & getOverrideMode(const std::string &default_mode = "inactive") const;
const std::string & getEnforceLevel() const;
const std::vector<std::string> & getConfigMap() const;
const std::vector<std::string> & getFiles() const;
const std::vector<std::string> & getOas() const;
const std::string & getURI() const;
private:
std::string override_mode;
std::string enforcement_level;
std::vector<std::string> config_map;
std::vector<std::string> files;
std::vector<std::string> oas;
std::string uri;
};
class NewAppSecPracticeAntiBot
{
public:
const std::vector<std::string> & getIjectedUris() const;
const std::vector<std::string> & getValidatedUris() const;
const std::string & getMode(const std::string &default_mode = "inactive") const;
std::vector<std::string> getIjectedUris() const;
std::vector<std::string> getValidatedUris() const;
void load(cereal::JSONInputArchive &archive_in);
void save(cereal::JSONOutputArchive &out_ar) const;
private:
std::string override_mode;
std::vector<std::string> injected_uris;
std::vector<std::string> validated_uris;
std::vector<NewAppSecWebBotsURI> injected_uris;
std::vector<NewAppSecWebBotsURI> validated_uris;
};
class NewAppSecWebAttackProtections
@@ -603,8 +579,8 @@ class NewAppSecPracticeSpec
public:
void load(cereal::JSONInputArchive &archive_in);
NewSnortSignatures & getSnortSignatures();
NewOpenApiSchema & getOpenSchemaValidation();
NewSnortSignaturesAndOpenSchemaAPI & getSnortSignatures();
const NewSnortSignaturesAndOpenSchemaAPI & getOpenSchemaValidation() const;
const NewAppSecPracticeWebAttacks & getWebAttacks() const;
const NewAppSecPracticeAntiBot & getAntiBot() const;
const NewIntrusionPrevention & getIntrusionPrevention() const;
@@ -617,8 +593,8 @@ public:
private:
NewFileSecurity file_security;
NewIntrusionPrevention intrusion_prevention;
NewOpenApiSchema openapi_schema_validation;
NewSnortSignatures snort_signatures;
NewSnortSignaturesAndOpenSchemaAPI openapi_schema_validation;
NewSnortSignaturesAndOpenSchemaAPI snort_signatures;
NewAppSecPracticeWebAttacks web_attacks;
NewAppSecPracticeAntiBot anti_bot;
std::string appsec_class_name;

View File

@@ -1,89 +0,0 @@
// Copyright (C) 2022 Check Point Software Technologies Ltd. All rights reserved.
// Licensed under the Apache License, Version 2.0 (the "License");
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#ifndef __POLICY_ACTIVATION_DATA_H__
#define __POLICY_ACTIVATION_DATA_H__
#include <vector>
#include <map>
#include "config.h"
#include "debug.h"
#include "rest.h"
#include "cereal/archives/json.hpp"
#include <cereal/types/map.hpp>
#include "customized_cereal_map.h"
#include "local_policy_common.h"
class PolicyActivationMetadata
{
public:
void load(cereal::JSONInputArchive &archive_in);
private:
std::string name;
};
class EnabledPolicy
{
public:
void load(cereal::JSONInputArchive &archive_in);
const std::string & getName() const;
const std::vector<std::string> & getHosts() const;
private:
std::string name;
std::vector<std::string> hosts;
};
class PolicyActivationSpec
{
public:
void load(cereal::JSONInputArchive &archive_in);
const std::vector<EnabledPolicy> & getPolicies() const;
private:
std::string appsec_class_name;
std::vector<EnabledPolicy> policies;
};
class SinglePolicyActivationData
{
public:
void load(cereal::JSONInputArchive &archive_in);
const PolicyActivationSpec & getSpec() const;
private:
std::string api_version;
std::string kind;
PolicyActivationMetadata metadata;
PolicyActivationSpec spec;
};
class PolicyActivationData : public ClientRest
{
public:
bool loadJson(const std::string &json);
const std::vector<SinglePolicyActivationData> & getItems() const;
private:
std::string api_version;
std::vector<SinglePolicyActivationData> items;
};
#endif // __POLICY_ACTIVATION_DATA_H__

View File

@@ -32,7 +32,6 @@
#include "i_messaging.h"
#include "appsec_practice_section.h"
#include "ingress_data.h"
#include "policy_activation_data.h"
#include "settings_section.h"
#include "triggers_section.h"
#include "local_policy_common.h"
@@ -206,8 +205,7 @@ private:
const RulesConfigRulebase& rule_config,
const std::string &practice_id, const std::string &full_url,
const std::string &default_mode,
std::map<AnnotationTypes, std::string> &rule_annotations,
std::vector<InnerException>
std::map<AnnotationTypes, std::string> &rule_annotations
);
void

View File

@@ -123,7 +123,6 @@ public:
);
const std::string & getIdentifier() const;
const std::string & getIdentifierValue() const;
void save(cereal::JSONOutputArchive &out_ar) const;
@@ -146,7 +145,6 @@ public:
);
const std::string & getIdentifier() const;
const std::string & getIdentifierValue() const;
void save(cereal::JSONOutputArchive &out_ar) const;

View File

@@ -39,7 +39,7 @@ public:
bool _logToAgent,
bool _logToCef,
bool _logToCloud,
bool _logTolocalTuning,
bool _logToK8sService,
bool _logToSyslog,
bool _responseBody,
bool _tpDetect,
@@ -73,7 +73,7 @@ private:
bool logToAgent;
bool logToCef;
bool logToCloud;
bool logTolocalTuning;
bool logToK8sService;
bool logToSyslog;
bool responseBody;
bool tpDetect;
@@ -258,7 +258,7 @@ public:
bool shouldBeautifyLogs() const;
bool getCloud() const;
bool isContainerNeeded() const;
bool isK8SNeeded() const;
bool isCefNeeded() const;
bool isSyslogNeeded() const;
const std::string & getSyslogServerIpv4Address() const;
@@ -269,7 +269,7 @@ private:
const LoggingService & getCefServiceData() const;
bool cloud = false;
bool container_service = false;
bool k8s_service = false;
bool agent_local = true;
bool beautify_logs = true;
LoggingService syslog_service;

View File

@@ -35,14 +35,6 @@ convertAnnotationKeysTostring(const AnnotationKeys &key)
}
}
string
getAppSecScopeType()
{
auto env_res = getenv("CRDS_SCOPE");
if (env_res != nullptr) return env_res;
return "cluster";
}
void
K8sPolicyUtils::init()
{
@@ -50,7 +42,6 @@ K8sPolicyUtils::init()
env_type = env_details->getEnvType();
if (env_type == EnvType::K8S) {
token = env_details->getToken();
agent_ns = getAppSecScopeType() == "namespaced" ? env_details->getNameSpace() + "/" : "";
messaging = Singleton::Consume<I_Messaging>::by<K8sPolicyUtils>();
}
}
@@ -149,12 +140,10 @@ extractElementsFromNewRule(
const NewParsedRule &rule,
map<AnnotationTypes, unordered_set<string>> &policy_elements_names)
{
if (rule.getExceptions().size() > 0) {
policy_elements_names[AnnotationTypes::EXCEPTION].insert(
rule.getExceptions().begin(),
rule.getExceptions().end()
);
}
policy_elements_names[AnnotationTypes::THREAT_PREVENTION_PRACTICE].insert(
rule.getPractices().begin(),
rule.getPractices().end()
@@ -163,25 +152,15 @@ extractElementsFromNewRule(
rule.getAccessControlPractices().begin(),
rule.getAccessControlPractices().end()
);
if (rule.getLogTriggers().size() > 0) {
policy_elements_names[AnnotationTypes::TRIGGER].insert(
rule.getLogTriggers().begin(),
rule.getLogTriggers().end()
);
}
if (rule.getCustomResponse() != "" ) {
policy_elements_names[AnnotationTypes::WEB_USER_RES].insert(rule.getCustomResponse());
}
if (rule.getSourceIdentifiers() != "" ) {
policy_elements_names[AnnotationTypes::SOURCE_IDENTIFIERS].insert(rule.getSourceIdentifiers());
}
if (rule.getTrustedSources() != "" ) {
policy_elements_names[AnnotationTypes::TRUSTED_SOURCES].insert(rule.getTrustedSources());
}
if (rule.getUpgradeSettings() != "" ) {
policy_elements_names[AnnotationTypes::UPGRADE_SETTINGS].insert(rule.getUpgradeSettings());
}
}
map<AnnotationTypes, unordered_set<string>>
K8sPolicyUtils::extractElementsNamesV1beta2(
@@ -280,11 +259,9 @@ K8sPolicyUtils::extractV1Beta2ElementsFromCluster(
dbgTrace(D_LOCAL_POLICY) << "Retrieve AppSec elements. type: " << crd_plural;
vector<T> elements;
for (const string &element_name : elements_names) {
string ns_suffix = getAppSecScopeType() == "namespaced" ? "ns" : "";
string ns = getAppSecScopeType() == "namespaced" ? "namespaces/" : "";
dbgTrace(D_LOCAL_POLICY) << "AppSec element name: " << element_name;
auto maybe_appsec_element = getObjectFromCluster<AppsecSpecParser<T>>(
"/apis/openappsec.io/v1beta2/" + ns + agent_ns + crd_plural + ns_suffix + "/" + element_name
"/apis/openappsec.io/v1beta2/" + crd_plural + "/" + element_name
);
if (!maybe_appsec_element.ok()) {
@@ -385,9 +362,8 @@ K8sPolicyUtils::createSnortFile(vector<NewAppSecPracticeSpec> &practices) const
practice.getSnortSignatures().setTemporary(true);
for (const string &config_map : practice.getSnortSignatures().getConfigMap())
{
string ns = agent_ns == "" ? "default/" : agent_ns;
auto maybe_configmap = getObjectFromCluster<ConfigMaps>(
"/api/v1/namespaces/" + ns + "configmaps/" + config_map
"/api/v1/namespaces/default/configmaps/" + config_map
);
if (!maybe_configmap.ok()) {
dbgWarning(D_LOCAL_POLICY) << "Failed to get configMaps from the cluster.";
@@ -405,28 +381,6 @@ K8sPolicyUtils::createSnortFile(vector<NewAppSecPracticeSpec> &practices) const
}
}
void
K8sPolicyUtils::createSchemaValidationOas(vector<NewAppSecPracticeSpec> &practices) const
{
for (NewAppSecPracticeSpec &practice : practices) {
vector<string> res;
for (const string &config_map : practice.getOpenSchemaValidation().getConfigMap())
{
string ns = agent_ns == "" ? "default/" : agent_ns;
auto maybe_configmap = getObjectFromCluster<ConfigMaps>(
"/api/v1/namespaces/" + ns + "configmaps/" + config_map
);
if (!maybe_configmap.ok()) {
dbgWarning(D_LOCAL_POLICY) << "Failed to get configMaps from the cluster.";
continue;
}
string file_content = maybe_configmap.unpack().getFileContent();
string res = Singleton::Consume<I_Encryptor>::by<K8sPolicyUtils>()->base64Encode(file_content);
practice.getOpenSchemaValidation().addOas(res);
}
}
}
Maybe<V1beta2AppsecLinuxPolicy>
K8sPolicyUtils::createAppsecPolicyK8sFromV1beta2Crds(
const AppsecSpecParser<NewAppsecPolicySpec> &appsec_policy_spec,
@@ -442,7 +396,6 @@ K8sPolicyUtils::createAppsecPolicyK8sFromV1beta2Crds(
}
if (default_rule.getMode().empty() && !ingress_mode.empty()) {
dbgTrace(D_LOCAL_POLICY) << "setting the policy default rule mode to the ingress mode: " << ingress_mode;
default_rule.setMode(ingress_mode);
}
@@ -458,7 +411,6 @@ K8sPolicyUtils::createAppsecPolicyK8sFromV1beta2Crds(
);
createSnortFile(threat_prevention_practices);
createSchemaValidationOas(threat_prevention_practices);
vector<AccessControlPracticeSpec> access_control_practices =
extractV1Beta2ElementsFromCluster<AccessControlPracticeSpec>(
@@ -515,6 +467,17 @@ K8sPolicyUtils::createAppsecPolicyK8sFromV1beta2Crds(
}
// LCOV_EXCL_STOP
bool
doesVersionExist(const map<string, string> &annotations, const string &version)
{
for (auto annotation : annotations) {
if(annotation.second.find(version) != std::string::npos) {
return true;
}
}
return false;
}
std::tuple<Maybe<AppsecLinuxPolicy>, Maybe<V1beta2AppsecLinuxPolicy>>
K8sPolicyUtils::createAppsecPolicyK8s(const string &policy_name, const string &ingress_mode) const
{
@@ -523,19 +486,16 @@ K8sPolicyUtils::createAppsecPolicyK8s(const string &policy_name, const string &i
);
if (!maybe_appsec_policy_spec.ok() ||
maybe_appsec_policy_spec.unpack().getApiVersion().find("v1beta1") == std::string::npos
!doesVersionExist(maybe_appsec_policy_spec.unpack().getMetaData().getAnnotations(), "v1beta1")
) {
try {
std::string v1beta1_error =
maybe_appsec_policy_spec.ok() ? "There is no v1beta1 policy" : maybe_appsec_policy_spec.getErr();
dbgWarning(D_LOCAL_POLICY
) << "Failed to retrieve Appsec policy with crds version: v1beta1, Trying version: v1beta2";
string ns_suffix = getAppSecScopeType() == "namespaced" ? "ns" : "";
string ns = getAppSecScopeType() == "namespaced" ? "namespaces/" : "";
auto maybe_v1beta2_appsec_policy_spec = getObjectFromCluster<AppsecSpecParser<NewAppsecPolicySpec>>(
"/apis/openappsec.io/v1beta2/" + ns + agent_ns + "policies" + ns_suffix + "/" + policy_name
"/apis/openappsec.io/v1beta2/policies/" + policy_name
);
if (!maybe_v1beta2_appsec_policy_spec.ok()) {
dbgWarning(D_LOCAL_POLICY)
<< "Failed to retrieve AppSec policy. Error: " << maybe_v1beta2_appsec_policy_spec.getErr();
@@ -566,7 +526,7 @@ K8sPolicyUtils::createAppsecPolicyK8s(const string &policy_name, const string &i
template<class T, class K>
void
K8sPolicyUtils::createPolicyFromIngress(
K8sPolicyUtils::createPolicy(
T &appsec_policy,
map<std::string, T> &policies,
map<AnnotationKeys, string> &annotations_values,
@@ -575,64 +535,35 @@ K8sPolicyUtils::createPolicyFromIngress(
if (policies.find(annotations_values[AnnotationKeys::PolicyKey]) == policies.end()) {
policies[annotations_values[AnnotationKeys::PolicyKey]] = appsec_policy;
}
auto default_mode = appsec_policy.getAppsecPolicySpec().getDefaultRule().getMode();
if (item.getSpec().doesDefaultBackendExist()) {
dbgTrace(D_LOCAL_POLICY)
<< "Inserting Any host rule to the specific asset set";
K ingress_rule = K("*", default_mode);
K ingress_rule = K("*");
policies[annotations_values[AnnotationKeys::PolicyKey]].addSpecificRule(ingress_rule);
}
for (const IngressDefinedRule &rule : item.getSpec().getRules()) {
string host = rule.getHost();
string url = rule.getHost();
for (const IngressRulePath &uri : rule.getPathsWrapper().getRulePaths()) {
if (uri.getPath() != "/") {
host = host + uri.getPath();
}
if (!appsec_policy.getAppsecPolicySpec().isAssetHostExist(host)) {
if (!appsec_policy.getAppsecPolicySpec().isAssetHostExist(url + uri.getPath())) {
dbgTrace(D_LOCAL_POLICY)
<< "Inserting Host data to the specific asset set:"
<< "URL: '"
<< rule.getHost()
<< url
<< "' uri: '"
<< uri.getPath()
<< "'";
K ingress_rule = K(host, default_mode);
K ingress_rule = K(url + uri.getPath());
policies[annotations_values[AnnotationKeys::PolicyKey]].addSpecificRule(ingress_rule);
}
}
}
}
template<class T, class K>
void
K8sPolicyUtils::createPolicyFromActivation(
T &appsec_policy,
map<std::string, T> &policies,
const EnabledPolicy &policy) const
{
if (policies.find(policy.getName()) == policies.end()) {
policies[policy.getName()] = appsec_policy;
}
auto default_mode = appsec_policy.getAppsecPolicySpec().getDefaultRule().getMode();
for (const string &host : policy.getHosts()) {
if (!appsec_policy.getAppsecPolicySpec().isAssetHostExist(host)) {
dbgTrace(D_LOCAL_POLICY)
<< "Inserting Host data to the specific asset set:"
<< "URL: '"
<< host
<< "'";
K ingress_rule = K(host, default_mode);
policies[policy.getName()].addSpecificRule(ingress_rule);
}
}
}
std::tuple<map<string, AppsecLinuxPolicy>, map<string, V1beta2AppsecLinuxPolicy>>
K8sPolicyUtils::createAppsecPolicies()
K8sPolicyUtils::createAppsecPoliciesFromIngresses()
{
dbgFlow(D_LOCAL_POLICY) << "Getting all policy object from Ingresses and PolicyActivation";
dbgFlow(D_LOCAL_POLICY) << "Getting all policy object from Ingresses";
map<string, AppsecLinuxPolicy> v1bet1_policies;
map<string, V1beta2AppsecLinuxPolicy> v1bet2_policies;
auto maybe_ingress = getObjectFromCluster<IngressData>("/apis/networking.k8s.io/v1/ingresses");
@@ -642,7 +573,7 @@ K8sPolicyUtils::createAppsecPolicies()
dbgWarning(D_LOCAL_POLICY)
<< "Failed to retrieve K8S Ingress configurations. Error: "
<< maybe_ingress.getErr();
maybe_ingress = IngressData{};
return make_tuple(v1bet1_policies, v1bet2_policies);
}
@@ -672,54 +603,19 @@ K8sPolicyUtils::createAppsecPolicies()
if (!std::get<0>(maybe_appsec_policy).ok()) {
auto appsec_policy=std::get<1>(maybe_appsec_policy).unpack();
createPolicyFromIngress<V1beta2AppsecLinuxPolicy, NewParsedRule>(
createPolicy<V1beta2AppsecLinuxPolicy, NewParsedRule>(
appsec_policy,
v1bet2_policies,
annotations_values,
item);
} else {
auto appsec_policy=std::get<0>(maybe_appsec_policy).unpack();
createPolicyFromIngress<AppsecLinuxPolicy, ParsedRule>(
createPolicy<AppsecLinuxPolicy, ParsedRule>(
appsec_policy,
v1bet1_policies,
annotations_values,
item);
}
}
string ns_suffix = getAppSecScopeType() == "namespaced" ? "ns" : "";
string ns = getAppSecScopeType() == "namespaced" ? "namespaces/" : "";
auto maybe_policy_activation = getObjectFromCluster<PolicyActivationData>(
"/apis/openappsec.io/v1beta2/" + ns + agent_ns + "policyactivations" + ns_suffix
);
if (!maybe_policy_activation.ok()) {
dbgWarning(D_LOCAL_POLICY)
<< "Failed to retrieve K8S PolicyActivation configurations. Error: "
<< maybe_policy_activation.getErr();
return make_tuple(v1bet1_policies, v1bet2_policies);
}
PolicyActivationData policy_activation = maybe_policy_activation.unpack();
for (const SinglePolicyActivationData &item : policy_activation.getItems()) {
for (const auto &policy : item.getSpec().getPolicies()) {
auto maybe_appsec_policy = createAppsecPolicyK8s(policy.getName(), "");
if (!std::get<1>(maybe_appsec_policy).ok()) {
dbgWarning(D_LOCAL_POLICY)
<< "Failed to create appsec policy. v1beta2 Error: "
<< std::get<1>(maybe_appsec_policy).getErr();
continue;
} else {
auto appsec_policy=std::get<1>(maybe_appsec_policy).unpack();
createPolicyFromActivation<V1beta2AppsecLinuxPolicy, NewParsedRule>(
appsec_policy,
v1bet2_policies,
policy);
}
}
}
return make_tuple(v1bet1_policies, v1bet2_policies);
}

View File

@@ -36,7 +36,6 @@
#include "customized_cereal_map.h"
#include "include/appsec_practice_section.h"
#include "include/ingress_data.h"
#include "include/policy_activation_data.h"
#include "include/settings_section.h"
#include "include/triggers_section.h"
#include "include/local_policy_common.h"
@@ -86,7 +85,7 @@ public:
K8sPolicyUtils k8s_policy_utils;
k8s_policy_utils.init();
auto appsec_policies = k8s_policy_utils.createAppsecPolicies();
auto appsec_policies = k8s_policy_utils.createAppsecPoliciesFromIngresses();
if (!std::get<0>(appsec_policies).empty()) {
return policy_maker_utils.proccesMultipleAppsecPolicies<AppsecLinuxPolicy, ParsedRule>(
std::get<0>(appsec_policies),

View File

@@ -180,16 +180,10 @@ NewAppsecTriggerLogDestination::load(cereal::JSONInputArchive &archive_in)
} else {
cloud = false;
}
bool local_tuning_default = false;
// check ENV VAR LOCAL_TUNING_ENABLED
char * tuning_enabled = getenv("LOCAL_TUNING_ENABLED");
if (tuning_enabled != NULL) {
for (unsigned int i = 0; i < strlen(tuning_enabled); i++) {
tuning_enabled[i] = tolower(tuning_enabled[i]);
}
local_tuning_default = string(tuning_enabled) == "true";
}
parseAppsecJSONKey<bool>("local-tuning", container_service, archive_in, local_tuning_default);
auto mode = Singleton::Consume<I_AgentDetails>::by<NewAppsecTriggerLogDestination>()->getOrchestrationMode();
auto env_type = Singleton::Consume<I_EnvDetails>::by<NewAppsecTriggerLogDestination>()->getEnvType();
bool k8s_service_default = (mode == OrchestrationMode::HYBRID && env_type == EnvType::K8S);
parseAppsecJSONKey<bool>("k8s-service", k8s_service, archive_in, k8s_service_default);
NewStdoutLogging stdout_log;
parseAppsecJSONKey<NewStdoutLogging>("stdout", stdout_log, archive_in);
@@ -230,9 +224,9 @@ NewAppsecTriggerLogDestination::getCloud() const
}
bool
NewAppsecTriggerLogDestination::isContainerNeeded() const
NewAppsecTriggerLogDestination::isK8SNeeded() const
{
return container_service;
return k8s_service;
}
bool

View File

@@ -22,7 +22,6 @@ static const set<string> performance_impacts = {"low", "medium", "high"};
static const set<string> severity_levels = {"low", "medium", "high", "critical"};
static const set<string> size_unit = {"bytes", "KB", "MB", "GB"};
static const set<string> confidences_actions = {"prevent", "detect", "inactive", "as-top-level", "inherited"};
static const set<string> valied_enforcement_level = {"fullSchema", "endpointOnly"};
static const set<string> valid_modes = {
"prevent",
"detect",
@@ -33,38 +32,31 @@ static const set<string> valid_modes = {
"inherited"
};
static const set<string> valid_confidences = {"medium", "high", "critical"};
static const unordered_map<string, string> key_to_performance_impact_val = {
static const std::unordered_map<std::string, std::string> key_to_performance_impact_val = {
{ "low", "Low or lower"},
{ "medium", "Medium or lower"},
{ "high", "High or lower"}
};
static const unordered_map<string, string> key_to_severity_level_val = {
static const std::unordered_map<std::string, std::string> key_to_severity_level_val = {
{ "low", "Low or above"},
{ "medium", "Medium or above"},
{ "high", "High or above"},
{ "critical", "Critical"}
};
static const unordered_map<string, string> key_to_mode_val = {
static const std::unordered_map<std::string, std::string> key_to_mode_val = {
{ "prevent-learn", "Prevent"},
{ "detect-learn", "Detect"},
{ "prevent", "Prevent"},
{ "detect", "Detect"},
{ "inactive", "Inactive"}
};
static const unordered_map<string, string> anti_bot_key_to_mode_val = {
{ "prevent-learn", "Prevent"},
{ "detect-learn", "Detect"},
{ "prevent", "Prevent"},
{ "detect", "Detect"},
{ "inactive", "Disabled"}
};
static const unordered_map<string, uint64_t> unit_to_int = {
static const std::unordered_map<std::string, uint64_t> unit_to_int = {
{ "bytes", 1},
{ "KB", 1024},
{ "MB", 1048576},
{ "GB", 1073741824}
};
static const string TRANSPARENT_MODE = "Transparent";
static const std::string TRANSPARENT_MODE = "Transparent";
bool
isModeInherited(const string &mode)
@@ -72,11 +64,11 @@ isModeInherited(const string &mode)
return mode == "as-top-level" || mode == "inherited";
}
const string &
const std::string &
getModeWithDefault(
const string &mode,
const string &default_mode,
const unordered_map<string, string> &key_to_val)
const std::string &mode,
const std::string &default_mode,
const std::unordered_map<std::string, std::string> &key_to_val)
{
if (isModeInherited(mode) && (key_to_val.find(default_mode) != key_to_val.end())) {
dbgError(D_LOCAL_POLICY) << "Setting to top-level mode: " << default_mode;
@@ -89,43 +81,57 @@ getModeWithDefault(
return key_to_val.at(mode);
}
const vector<string> &
NewAppSecPracticeAntiBot::getIjectedUris() const
void
NewAppSecWebBotsURI::load(cereal::JSONInputArchive &archive_in)
{
return injected_uris;
}
const vector<string> &
NewAppSecPracticeAntiBot::getValidatedUris() const
{
return validated_uris;
dbgTrace(D_LOCAL_POLICY) << "Loading AppSec Web Bots URI";
parseAppsecJSONKey<string>("uri", uri, archive_in);
}
const string &
NewAppSecPracticeAntiBot::getMode(const string &default_mode) const
NewAppSecWebBotsURI::getURI() const
{
return getModeWithDefault(override_mode, default_mode, anti_bot_key_to_mode_val);
return uri;
}
std::vector<std::string>
NewAppSecPracticeAntiBot::getIjectedUris() const
{
vector<string> injected;
for (const NewAppSecWebBotsURI &uri : injected_uris) injected.push_back(uri.getURI());
return injected;
}
std::vector<std::string>
NewAppSecPracticeAntiBot::getValidatedUris() const
{
vector<string> validated;
for (const NewAppSecWebBotsURI &uri : validated_uris) validated.push_back(uri.getURI());
return validated;
}
void
NewAppSecPracticeAntiBot::load(cereal::JSONInputArchive &archive_in)
{
dbgTrace(D_LOCAL_POLICY) << "Loading AppSec Web Bots";
parseAppsecJSONKey<vector<string>>("injectedUris", injected_uris, archive_in);
parseAppsecJSONKey<vector<string>>("validatedUris", validated_uris, archive_in);
parseAppsecJSONKey<vector<NewAppSecWebBotsURI>>("injectedUris", injected_uris, archive_in);
parseAppsecJSONKey<vector<NewAppSecWebBotsURI>>("validatedUris", validated_uris, archive_in);
parseMandatoryAppsecJSONKey<string>("overrideMode", override_mode, archive_in, "inactive");
if (valid_modes.count(override_mode) == 0) {
dbgWarning(D_LOCAL_POLICY) << "AppSec Web Bots override mode invalid: " << override_mode;
throw PolicyGenException("AppSec Web Bots override mode invalid: " + override_mode);
}
}
void
NewAppSecPracticeAntiBot::save(cereal::JSONOutputArchive &out_ar) const
{
vector<string> injected;
vector<string> validated;
for (const NewAppSecWebBotsURI &uri : injected_uris) injected.push_back(uri.getURI());
for (const NewAppSecWebBotsURI &uri : validated_uris) validated.push_back(uri.getURI());
out_ar(
cereal::make_nvp("injected", injected_uris),
cereal::make_nvp("validated", validated_uris)
cereal::make_nvp("injected", injected),
cereal::make_nvp("validated", validated)
);
}
@@ -242,14 +248,14 @@ NewAppSecPracticeWebAttacks::getProtections() const
}
SnortProtectionsSection::SnortProtectionsSection(
const string &_context,
const string &_asset_name,
const string &_asset_id,
const string &_practice_name,
const string &_practice_id,
const string &_source_identifier,
const string &_mode,
const vector<string> &_files)
const std::string &_context,
const std::string &_asset_name,
const std::string &_asset_id,
const std::string &_practice_name,
const std::string &_practice_id,
const std::string &_source_identifier,
const std::string &_mode,
const std::vector<std::string> &_files)
:
context(_context),
asset_name(_asset_name),
@@ -278,10 +284,10 @@ SnortProtectionsSection::save(cereal::JSONOutputArchive &out_ar) const
}
DetectionRules::DetectionRules(
const string &_type,
const string &_SSM,
const string &_keywords,
const vector<string> &_context)
const std::string &_type,
const std::string &_SSM,
const std::string &_keywords,
const std::vector<std::string> &_context)
:
type(_type),
SSM(_SSM),
@@ -314,14 +320,14 @@ DetectionRules::save(cereal::JSONOutputArchive &out_ar) const
ProtectionMetadata::ProtectionMetadata(
bool _silent,
const string &_protection_name,
const string &_severity,
const string &_confidence_level,
const string &_performance_impact,
const string &_last_update,
const string &_maintrain_id,
const vector<string> &_tags,
const vector<string> &_cve_list)
const std::string &_protection_name,
const std::string &_severity,
const std::string &_confidence_level,
const std::string &_performance_impact,
const std::string &_last_update,
const std::string &_maintrain_id,
const std::vector<std::string> &_tags,
const std::vector<std::string> &_cve_list)
:
silent(_silent),
protection_name(_protection_name),
@@ -394,9 +400,9 @@ ProtectionsProtectionsSection::save(cereal::JSONOutputArchive &out_ar) const
}
ProtectionsSection::ProtectionsSection(
const vector<ProtectionsProtectionsSection> &_protections,
const string &_name,
const string &_modification_time)
const std::vector<ProtectionsProtectionsSection> &_protections,
const std::string &_name,
const std::string &_modification_time)
:
protections(_protections),
name(_name),
@@ -460,16 +466,12 @@ SnortSectionWrapper::save(cereal::JSONOutputArchive &out_ar) const
}
void
NewSnortSignatures::load(cereal::JSONInputArchive &archive_in)
NewSnortSignaturesAndOpenSchemaAPI::load(cereal::JSONInputArchive &archive_in)
{
dbgTrace(D_LOCAL_POLICY) << "Loading AppSec Snort Signatures practice";
parseMandatoryAppsecJSONKey<string>("overrideMode", override_mode, archive_in, "inactive");
parseAppsecJSONKey<vector<string>>("configmap", config_map, archive_in);
parseAppsecJSONKey<vector<string>>("files", files, archive_in);
if (valid_modes.count(override_mode) == 0) {
dbgWarning(D_LOCAL_POLICY) << "AppSec Snort Signatures override mode invalid: " << override_mode;
throw PolicyGenException("AppSec Snort Signatures override mode invalid: " + override_mode);
}
is_temporary = false;
if (valid_modes.count(override_mode) == 0) {
dbgWarning(D_LOCAL_POLICY) << "AppSec Snort Signatures override mode invalid: " << override_mode;
@@ -478,107 +480,42 @@ NewSnortSignatures::load(cereal::JSONInputArchive &archive_in)
}
void
NewSnortSignatures::addFile(const string &file_name)
NewSnortSignaturesAndOpenSchemaAPI::addFile(const string &file_name)
{
files.push_back(file_name);
}
const string &
NewSnortSignatures::getOverrideMode(const string &default_mode) const
NewSnortSignaturesAndOpenSchemaAPI::getOverrideMode(const string &default_mode) const
{
const string &res = getModeWithDefault(override_mode, default_mode, key_to_practices_mode_val);
const string &res = getModeWithDefault(override_mode, default_mode, key_to_practices_val);
return res;
}
const vector<string> &
NewSnortSignatures::getFiles() const
NewSnortSignaturesAndOpenSchemaAPI::getFiles() const
{
return files;
}
const vector<string> &
NewSnortSignatures::getConfigMap() const
NewSnortSignaturesAndOpenSchemaAPI::getConfigMap() const
{
return config_map;
}
bool
NewSnortSignatures::isTemporary() const
NewSnortSignaturesAndOpenSchemaAPI::isTemporary() const
{
return is_temporary;
}
void
NewSnortSignatures::setTemporary(bool val)
NewSnortSignaturesAndOpenSchemaAPI::setTemporary(bool val)
{
is_temporary = val;
}
void
NewOpenApiSchema::load(cereal::JSONInputArchive &archive_in)
{
dbgTrace(D_LOCAL_POLICY) << "Loading AppSec Schema Validation practice";
parseMandatoryAppsecJSONKey<string>("overrideMode", override_mode, archive_in, "inactive");
parseAppsecJSONKey<vector<string>>("configmap", config_map, archive_in);
parseAppsecJSONKey<vector<string>>("files", files, archive_in);
parseAppsecJSONKey<string>("enforcementLevel", enforcement_level, archive_in, "fullSchema");
if (valied_enforcement_level.count(enforcement_level) == 0) {
dbgWarning(D_LOCAL_POLICY) << "AppSec Schema Validation enforcement level invalid: " << enforcement_level;
throw PolicyGenException("AppSec Schema Validation enforcement level invalid: " + enforcement_level);
}
if (valid_modes.count(override_mode) == 0) {
dbgWarning(D_LOCAL_POLICY) << "AppSec Schema Validation override mode invalid: " << override_mode;
throw PolicyGenException("AppSec Schema Validation override mode invalid: " + override_mode);
}
for (const string &file : files)
{
auto i_orchestration_tools = Singleton::Consume<I_OrchestrationTools>::by<NewOpenApiSchema>();
auto file_content = i_orchestration_tools->readFile(file);
if (!file_content.ok()) {
dbgWarning(D_LOCAL_POLICY) << "Couldn't open the schema validation file";
continue;
}
oas.push_back(Singleton::Consume<I_Encryptor>::by<NewOpenApiSchema>()->base64Encode(file_content.unpack()));
}
}
void
NewOpenApiSchema::addOas(const string &file)
{
oas.push_back(file);
}
const string &
NewOpenApiSchema::getOverrideMode(const string &default_mode) const
{
const string &res = getModeWithDefault(override_mode, default_mode, key_to_practices_val2);
return res;
}
const string &
NewOpenApiSchema::getEnforceLevel() const
{
return enforcement_level;
}
const vector<string> &
NewOpenApiSchema::getFiles() const
{
return files;
}
const vector<string> &
NewOpenApiSchema::getConfigMap() const
{
return config_map;
}
const vector<string> &
NewOpenApiSchema::getOas() const
{
return oas;
}
void
IpsProtectionsRulesSection::save(cereal::JSONOutputArchive &out_ar) const
{
@@ -617,7 +554,7 @@ IpsProtectionsSection::IpsProtectionsSection(
{
}
string &
std::string &
IpsProtectionsSection::getMode()
{
return mode;
@@ -639,20 +576,6 @@ IpsProtectionsSection::save(cereal::JSONOutputArchive &out_ar) const
);
}
bool
IpsProtectionsSection::operator<(const IpsProtectionsSection &other) const
{
// for sorting from the most specific to the least specific rule
if (name == default_appsec_name) return false;
if (other.name == default_appsec_name) return true;
return name.size() > other.name.size();
}
IPSSection::IPSSection(const vector<IpsProtectionsSection> &_ips) : ips(_ips)
{
sort(ips.begin(), ips.end());
}
void
IPSSection::save(cereal::JSONOutputArchive &out_ar) const
{
@@ -731,7 +654,7 @@ NewIntrusionPrevention::createIpsRules(const string &default_mode) const
vector<IpsProtectionsRulesSection> ips_rules;
IpsProtectionsRulesSection high_rule(
min_cve_Year,
getRulesMode(high_confidence_event_action, default_mode),
getModeWithDefault(high_confidence_event_action, default_mode, key_to_practices_val),
string("High"),
max_performance_impact,
string(""),
@@ -741,7 +664,7 @@ NewIntrusionPrevention::createIpsRules(const string &default_mode) const
IpsProtectionsRulesSection med_rule(
min_cve_Year,
getRulesMode(medium_confidence_event_action, default_mode),
getModeWithDefault(medium_confidence_event_action, default_mode, key_to_practices_val),
string("Medium"),
max_performance_impact,
string(""),
@@ -751,7 +674,7 @@ NewIntrusionPrevention::createIpsRules(const string &default_mode) const
IpsProtectionsRulesSection low_rule(
min_cve_Year,
getRulesMode(low_confidence_event_action, default_mode),
getModeWithDefault(low_confidence_event_action, default_mode, key_to_practices_val),
string("Low"),
max_performance_impact,
string(""),
@@ -762,45 +685,33 @@ NewIntrusionPrevention::createIpsRules(const string &default_mode) const
return ips_rules;
}
const string &
NewIntrusionPrevention::getMode(const string &default_mode) const
const std::string &
NewIntrusionPrevention::getMode(const std::string &default_mode) const
{
const string &res = getModeWithDefault(override_mode, default_mode, key_to_practices_mode_val);
const string &res = getModeWithDefault(override_mode, default_mode, key_to_practices_val);
return res;
}
const string &
NewIntrusionPrevention::getRulesMode(const string &mode, const string &default_mode) const
{
if (isModeInherited(mode)) return default_mode;
if (key_to_practices_mode_val.find(mode) == key_to_practices_mode_val.end()) {
dbgError(D_LOCAL_POLICY) << "Given mode: " << mode << " or top-level: " << default_mode << " is invalid.";
return key_to_practices_mode_val.at("inactive");
}
return key_to_practices_mode_val.at(mode);
}
FileSecurityProtectionsSection::FileSecurityProtectionsSection(
uint64_t _file_size_limit,
uint64_t _archive_file_size_limit,
bool _allow_files_without_name,
bool _required_file_size_limit,
bool _required_archive_extraction,
const string &_context,
const string &_name,
const string &_asset_id,
const string &_practice_name,
const string &_practice_id,
const string &_action,
const string &_files_without_name_action,
const string &_high_confidence_action,
const string &_medium_confidence_action,
const string &_low_confidence_action,
const string &_severity_level,
const string &_file_size_limit_action,
const string &_multi_level_archive_action,
const string &_unopened_archive_action)
const std::string &_context,
const std::string &_name,
const std::string &_asset_id,
const std::string &_practice_name,
const std::string &_practice_id,
const std::string &_action,
const std::string &_files_without_name_action,
const std::string &_high_confidence_action,
const std::string &_medium_confidence_action,
const std::string &_low_confidence_action,
const std::string &_severity_level,
const std::string &_file_size_limit_action,
const std::string &_multi_level_archive_action,
const std::string &_unopened_archive_action)
:
file_size_limit(_file_size_limit),
archive_file_size_limit(_archive_file_size_limit),
@@ -926,13 +837,13 @@ NewFileSecurityArchiveInspection::getrequiredArchiveExtraction() const
return extract_archive_files;
}
const string &
const std::string &
NewFileSecurityArchiveInspection::getMultiLevelArchiveAction() const
{
return archived_files_within_archived_files;
}
const string &
const std::string &
NewFileSecurityArchiveInspection::getUnopenedArchiveAction() const
{
return archived_files_where_content_extraction_failed;
@@ -981,7 +892,7 @@ NewFileSecurityLargeFileInspection::getFileSizeLimit() const
return (file_size_limit * unit_to_int.at(file_size_limit_unit));
}
const string &
const std::string &
NewFileSecurityLargeFileInspection::getFileSizeLimitAction() const
{
return files_exceeding_size_limit_action;
@@ -1102,7 +1013,7 @@ void
NewAppSecPracticeSpec::load(cereal::JSONInputArchive &archive_in)
{
dbgTrace(D_LOCAL_POLICY) << "Loading AppSec practice spec";
parseAppsecJSONKey<NewOpenApiSchema>(
parseAppsecJSONKey<NewSnortSignaturesAndOpenSchemaAPI>(
"schemaValidation",
openapi_schema_validation,
archive_in
@@ -1110,15 +1021,11 @@ NewAppSecPracticeSpec::load(cereal::JSONInputArchive &archive_in)
parseAppsecJSONKey<string>("appsecClassName", appsec_class_name, archive_in);
parseMandatoryAppsecJSONKey<NewFileSecurity>("fileSecurity", file_security, archive_in);
parseMandatoryAppsecJSONKey<NewIntrusionPrevention>("intrusionPrevention", intrusion_prevention, archive_in);
parseMandatoryAppsecJSONKey<NewSnortSignatures>("snortSignatures", snort_signatures, archive_in);
parseMandatoryAppsecJSONKey<NewSnortSignaturesAndOpenSchemaAPI>("snortSignatures", snort_signatures, archive_in);
parseMandatoryAppsecJSONKey<NewAppSecPracticeWebAttacks>("webAttacks", web_attacks, archive_in);
parseAppsecJSONKey<NewAppSecPracticeAntiBot>("antiBot", anti_bot, archive_in);
parseAppsecJSONKey<string>("name", practice_name, archive_in);
parseAppsecJSONKey<string>("practiceMode", mode, archive_in, "inherited");
if (valid_modes.count(mode) == 0) {
dbgWarning(D_LOCAL_POLICY) << "AppSec Threat prevention practice mode invalid: " << mode;
throw PolicyGenException("AppSec Threat prevention practice mode invalid: " + mode);
}
}
void
@@ -1127,13 +1034,13 @@ NewAppSecPracticeSpec::setName(const string &_name)
practice_name = _name;
}
NewOpenApiSchema &
NewAppSecPracticeSpec::getOpenSchemaValidation()
const NewSnortSignaturesAndOpenSchemaAPI &
NewAppSecPracticeSpec::getOpenSchemaValidation() const
{
return openapi_schema_validation;
}
NewSnortSignatures &
NewSnortSignaturesAndOpenSchemaAPI &
NewAppSecPracticeSpec::getSnortSignatures()
{
return snort_signatures;

View File

@@ -69,7 +69,7 @@ Identifier::load(cereal::JSONInputArchive &archive_in)
dbgWarning(D_LOCAL_POLICY) << "AppSec identifier invalid: " << identifier;
identifier = "sourceip";
}
parseAppsecJSONKey<vector<string>>("value", value, archive_in);
parseMandatoryAppsecJSONKey<vector<string>>("value", value, archive_in);
}
const string &

Some files were not shown because too many files have changed in this diff Show More