Minor fixes

Brought back lines that were accidentally deleted

README.md

@@ -18,25 +18,41 @@ Every request to the application goes through two phases:
 
 2. If the request is identified as a valid and legitimate request the request is allowed, and forwarded to your application. If, however, the request is considered suspicious or high risk, it then gets evaluated by the unsupervised model, which was trained in your specific environment. This model uses information such as the URL and the users involved to create a final confidence score that determines whether the request should be allowed or blocked.
 
-##  Machine Learning models
+![image](https://github.com/openappsec/openappsec/assets/114033741/f32f1c99-9c45-4d21-aa85-61408a16a18e)
 
-open-appsec uses two models:
+
+open-appsec uses two machine learning models:
 
 1. A supervised model that was trained offline based on millions of requests, both malicious and benign.
     
-    * A basic model is provided as part of this repository. It is recommended for use in Monitor-Only and Test environments.
-    * An advanced model which is more accurate and recommended for Production use can be downloaded from the [open-appsec portal](https://my.openappsec.io)->User Menu->Download advanced ML model. This model updates from time to time and you will get an email when these updates happen.
+    * A **basic model** is provided as part of this repository. It is recommended for use in Monitor-Only and Test environments.
+    * An **advanced model** which is more accurate and **recommended for Production** use can be downloaded from the [open-appsec portal](https://my.openappsec.io)->User Menu->Download advanced ML model. This model updates from time to time and you will get an email when these updates happen.
 
 2. An unsupervised model that is being built in real time in the protected environment. This model uses traffic patterns specific to the environment.
 
+
+# Management
+
+open-appsec can be managed using multiple methods:
+* [Declarative configuration files](https://docs.openappsec.io/getting-started/getting-started)
+* [Kubernetes Helm Charts and annotations](https://docs.openappsec.io/getting-started/getting-started)
+* [Using SaaS Web Management](https://docs.openappsec.io/getting-started/using-the-web-ui-saas)
+
+open-appsec Web UI:
+![image](https://github.com/openappsec/openappsec/assets/114033741/22d99379-df52-45c8-984f-1b820635f3b9)
+
+
+## Deployment Playgrounds (Virtual labs)
+You can experiment with open-appsec using [Playgrounds](https://www.openappsec.io/playground)
+
+![image](https://github.com/openappsec/openappsec/assets/114033741/14d35d69-4577-48fc-ae87-ea344888e94d)
+
 # Resources
 * [Project Website](https://openappsec.io)
 * [Offical Documentation](https://docs.openappsec.io/)
 * [Video Tutorials](https://www.openappsec.io/tutorials)
-* [Live Playgrounds](https://www.openappsec.io/playground)
 
-
-# open-appsec Installation
+# Installation
 
 For Kubernetes (NGINX Ingress) using the installer:
 
@@ -63,3 +79,107 @@ $ install-cp-nano-attachment-registration-manager.sh --install
 ```
 You can add the ```--token <token>``` and ```--email <email address>``` options to the first command, to get a token follow [documentation](https://docs.openappsec.io/getting-started/using-the-web-ui-saas/connect-deployed-agents-to-saas-management-k8s-and-linux). 
 
+For Docker: follow [documentation](https://docs.openappsec.io/getting-started/start-with-docker) 
+
+For more information read the [documentation](https://docs.openappsec.io/) or follow the [video tutorials](https://www.openappsec.io/tutorials).
+
+# Repositories
+
+open-appsec GitHub includes four main repositories:
+
+* [openappsec/openappsec](https://github.com/openappsec/openappsec) the main code and logic of open-appsec. Developed in C++.
+* [openappsec/attachment](https://github.com/openappsec/attachment) connects between processes that provide HTTP data (e.g NGINX) and the open-appsec Agent security logic. Developed in C.
+* [openappsec/smartsync](https://github.com/openappsec/smartsync) in charge of correlating learning data from multiple agent instances and delivering a unified learning model for each asset. Developed in Golang.
+* [openappsec/smartsync-shared-files](https://github.com/openappsec/smartsync-shared-files) interface to physical storage used by smartsync service for storing learning data. Developed in Golang.
+
+# Compilation instructions
+
+## Installing external dependencies
+
+Before compiling the services, you'll need to ensure the latest development versions of the following libraries:
+* Boost
+* OpenSSL
+* PCRE2
+* libxml2
+* GTest
+* GMock
+* cURL
+* Hiredis
+
+An example of installing the packages on Alpine:
+
+```bash
+ $ apk update
+ $ apk add boost-dev openssl-dev pcre2-dev libxml2-dev gtest-dev curl-dev hiredis-dev
+```
+
+## Compiling and packaging the agent code
+
+1. Clone this repository
+2. Run CMake command
+3. Run make install command
+
+```bash
+ $ git clone https://github.com/openappsec/openappsec.git
+ $ cd openappsec/
+ $ cmake -DCMAKE_INSTALL_PREFIX=build_out .
+ $ make install
+ $ make package
+```
+
+## Placing the agent code inside an Alpine docker image
+
+Once the agent code has been compiled and packaged, an Alpine image running it can be created. This requires permissions to execute the `docker` command.
+
+```bash
+ $ make docker
+```
+
+This will create a local image for your docker called `agent-docker`.
+
+## Deployment of the agent docker image as a container
+
+To run a Nano-Agent as a container the following steps are required:
+
+1. If you are using a container management system / plan on deploying the container using your CI, add the agent docker image to an accessible registry.
+2. If you are planning to manage the agent using the open-appsec UI, then make sure to obtain an agent token from the Management Portal and Enforce.
+3. Run the agent with the following command (where -e https_proxy parameter is optional):
+
+`docker run -d --name=agent-container --ipc=host -v=<path to persistent location for agent config>:/etc/cp/conf -v=<path to persistent location for agent data files>:/etc/cp/data -v=<path to persistent location for agent debugs and logs>:/var/log/nano_agent -e https_proxy=<user:password@Proxy address:port> -it <agent-image> /cp-nano-agent [--token <token> | --standalone]`
+
+Example:
+```bash
+ $ docker run -d --name=agent-container --ipc=host -v=/home/admin/agent/conf:/etc/cp/conf -v=/home/admin/agent/data:/etc/cp/data -v=/home/admin/agent/logs:/var/log/nano_agent –e https_proxy=user:password@1.2.3.4:8080 -it agent-docker /cp-nano-agent --standalone
+ $ docker ps
+CONTAINER ID        IMAGE               COMMAND                          CREATED             STATUS              PORTS               NAMES
+1e67f2abbfd4        agent-docker        "/cp-nano-agent --hybrid-mode"   1 minute ago        Up 1 minute                             agent-container
+```
+
+ Note that you are not required to use a token from the Management Portal if you are managing your security policy locally. However, you are required to use the --standalone flag in such cases. In addition, the volumes in the command are mandatory only if you wish to have persistency upon restart/upgrade/crash of the agent and its re-execution.
+ Lastly, --ipc=host argument is mandatory in order for the agent to have access to shared memory with a protected attachment (NGINX server).
+
+4. Create or replace the NGINX container using the [Attachment Repository](https://github.com/openappsec/attachment).
+
+This will run a docker container using the agent docker image.
+
+# Contributing
+We welcome everyone that wishes to share their knowledge and expertise to enhance and expand the project.
+
+Please see the [Contributing Guidelines](https://github.com/openappsec/openappsec/blob/main/CONTRIBUTING.md).
+
+# Security
+
+### Security Audit
+open-appsec code was audited by an independent third party in September-October 2022.
+See the [full report](https://github.com/openappsec/openappsec/blob/main/LEXFO-CHP20221014-Report-Code_audit-OPEN-APPSEC-v1.2.pdf).
+
+### Reporting security vulnerabilities
+If you've found a vulnerability or a potential vulnerability in open-appsec please let us know at securityalert@openappsec.io. We'll send a confirmation email to acknowledge your report within 24 hours, and we'll send an additional email when we've identified the issue positively or negatively.
+
+
+# License
+open-appsec is open source and available under Apache 2.0 license.
+
+The basic ML model is open source and available under Apache 2.0 license.
+
+The advanced ML model is open source and available under Machine Learning Model license, available upon download in the tar file.

build_system/docker/entry.sh

@@ -58,6 +58,10 @@ fi
 
 /nano-service-installers/$ORCHESTRATION_INSTALLATION_SCRIPT --install $orchestration_service_installation_flags
 
+if [ -f /var/run/secrets/kubernetes.io/serviceaccount/token ]; then
+    /etc/cp/orchestration/k8s-check-update-listener.sh &
+fi
+
 /nano-service-installers/$ATTACHMENT_REGISTRATION_SERVICE --install
 /nano-service-installers/$HTTP_TRANSACTION_HANDLER_SERVICE --install
 

components/generic_rulebase/evaluators/http_transaction_data_eval.cc

@@ -20,6 +20,9 @@
 #include "environment/evaluator_templates.h"
 #include "i_environment.h"
 #include "singleton.h"
+#include "debug.h"
+
+USE_DEBUG_FLAG(D_RULEBASE_CONFIG);
 
 using namespace std;
 using namespace EnvironmentHelper;
@@ -55,6 +58,51 @@ EqualHost::evalVariable() const
     return lower_host_ctx == lower_host;
 }
 
+WildcardHost::WildcardHost(const vector<string> &params)
+{
+    if (params.size() != 1) reportWrongNumberOfParams("WildcardHost", params.size(), 1, 1);
+    host = params[0];
+}
+
+Maybe<bool, Context::Error>
+WildcardHost::evalVariable() const
+{
+    I_Environment *env = Singleton::Consume<I_Environment>::by<WildcardHost>();
+    auto host_ctx = env->get<string>(HttpTransactionData::host_name_ctx);
+
+    if (!host_ctx.ok())
+    {
+        return false;
+    }
+
+    string lower_host_ctx = host_ctx.unpack();
+    transform(lower_host_ctx.begin(), lower_host_ctx.end(), lower_host_ctx.begin(), ::tolower);
+
+    dbgTrace(D_RULEBASE_CONFIG) << "found host in current context: " << lower_host_ctx;
+
+    size_t pos = lower_host_ctx.find_first_of(".");
+    if (pos == string::npos) {
+        return false;
+    }
+
+    lower_host_ctx = "*" + lower_host_ctx.substr(pos, lower_host_ctx.length());
+
+    string lower_host = host;
+    transform(lower_host.begin(), lower_host.end(), lower_host.begin(), ::tolower);
+
+    dbgTrace(D_RULEBASE_CONFIG)
+        << "trying to match host context with its corresponding wildcard address: "
+        << lower_host_ctx
+        << ". Matcher host: "
+        << lower_host;
+
+    if (lower_host_ctx == lower_host) return true;
+    pos = lower_host_ctx.find_last_of(':');
+    if (pos == string::npos) return false;
+    lower_host_ctx = string(lower_host_ctx.data(), pos);
+    return lower_host_ctx == lower_host;
+}
+
 EqualListeningIP::EqualListeningIP(const vector<string> &params)
 {
     if (params.size() != 1) reportWrongNumberOfParams("EqualListeningIP", params.size(), 1, 1);

components/generic_rulebase/generic_rulebase.cc

@@ -75,6 +75,7 @@ GenericRulebase::Impl::preload()
     addMatcher<IpProtocolMatcher>();
     addMatcher<UrlMatcher>();
     addMatcher<EqualHost>();
+    addMatcher<WildcardHost>();
     addMatcher<EqualListeningIP>();
     addMatcher<EqualListeningPort>();
     addMatcher<BeginWithUri>();

components/include/generic_rulebase/evaluators/http_transaction_data_eval.h

@@ -32,6 +32,19 @@ private:
     std::string host;
 };
 
+class WildcardHost : public EnvironmentEvaluator<bool>, Singleton::Consume<I_Environment>
+{
+public:
+    WildcardHost(const std::vector<std::string> &params);
+
+    static std::string getName() { return "WildcardHost"; }
+
+    Maybe<bool, Context::Error> evalVariable() const override;
+
+private:
+    std::string host;
+};
+
 class EqualListeningIP : public EnvironmentEvaluator<bool>, Singleton::Consume<I_Environment>
 {
 public:

components/include/i_local_policy_mgmt_gen.h

@@ -14,13 +14,15 @@
 #ifndef __I_LOCAL_POLICY_MGMT_GEN_H__
 #define __I_LOCAL_POLICY_MGMT_GEN_H__
 
+#include "i_env_details.h"
+
 class I_LocalPolicyMgmtGen
 {
 public:
-    virtual std::string parsePolicy(const std::string &policy_version) = 0;
-    virtual const std::string & getAgentPolicyPath(void) const = 0;
-    virtual const std::string & getLocalPolicyPath(void) const = 0;
-    virtual void setPolicyPath(const std::string &new_local_policy_path) = 0;
+    virtual std::string generateAppSecLocalPolicy(
+        EnvType env_type,
+        const std::string &policy_version,
+        const std::string &local_policy_path) = 0;
 
 protected:
     ~I_LocalPolicyMgmtGen() {}

components/include/i_orchestration_status.h

@@ -34,6 +34,7 @@ public:
     virtual const std::string & getUpdateTime() const = 0;
     virtual const std::string & getLastManifestUpdate() const = 0;
     virtual const std::string & getPolicyVersion() const = 0;
+    virtual const std::string & getWaapModelVersion() const = 0;
     virtual const std::string & getLastPolicyUpdate() const = 0;
     virtual const std::string & getLastSettingsUpdate() const = 0;
     virtual const std::string & getUpgradeMode() const = 0;

components/include/i_orchestration_tools.h

@@ -106,8 +106,9 @@ public:
         const std::string &profile_id = "") const = 0;
 
     virtual bool isNonEmptyFile(const std::string &path) const = 0;
+    virtual std::shared_ptr<std::ifstream> fileStreamWrapper(const std::string &path) const = 0;
     virtual Maybe<std::string> readFile(const std::string &path) const = 0;
-    virtual bool writeFile(const std::string &text, const std::string &path) const = 0;
+    virtual bool writeFile(const std::string &text, const std::string &path, bool append_mode = false) const = 0;
     virtual bool removeFile(const std::string &path) const = 0;
     virtual bool removeDirectory(const std::string &path, bool delete_content) const = 0;
     virtual void deleteVirtualTenantProfileFiles(
@@ -116,6 +117,7 @@ public:
         const std::string &conf_path) const = 0;
     virtual bool copyFile(const std::string &src_path, const std::string &dst_path) const = 0;
     virtual bool doesFileExist(const std::string &file_path) const = 0;
+    virtual void getClusterId() const = 0;
     virtual void fillKeyInJson(
         const std::string &filename,
         const std::string &_key,

components/include/orchestration_comp.h

@@ -31,6 +31,7 @@
 #include "i_environment.h"
 #include "i_tenant_manager.h"
 #include "i_package_handler.h"
+#include "i_env_details.h"
 #include "component.h"
 
 class OrchestrationComp
@@ -52,7 +53,8 @@ class OrchestrationComp
     Singleton::Consume<I_ServiceController>,
     Singleton::Consume<I_UpdateCommunication>,
     Singleton::Consume<I_Downloader>,
-    Singleton::Consume<I_ManifestController>
+    Singleton::Consume<I_ManifestController>,
+    Singleton::Consume<I_EnvDetails>
 {
 public:
     OrchestrationComp();

components/include/orchestration_status.h

@@ -24,6 +24,7 @@
 #include "i_time_get.h"
 #include "i_mainloop.h"
 #include "i_agent_details.h"
+#include "i_details_resolver.h"
 #include "customized_cereal_map.h"
 
 class OrchestrationStatus
@@ -32,6 +33,7 @@ class OrchestrationStatus
     Singleton::Provide<I_OrchestrationStatus>,
     Singleton::Consume<I_TimeGet>,
     Singleton::Consume<I_AgentDetails>,
+    Singleton::Consume<I_DetailsResolver>,
     Singleton::Consume<I_OrchestrationTools>,
     Singleton::Consume<I_MainLoop>
 {

components/include/orchestration_tools.h

@@ -20,13 +20,23 @@
 #include "i_shell_cmd.h"
 #include "i_tenant_manager.h"
 #include "component.h"
+#include "i_env_details.h"
+#include "i_messaging.h"
+#include "i_environment.h"
+#include "i_agent_details.h"
+#include "i_mainloop.h"
 
 class OrchestrationTools
         :
     public Component,
     Singleton::Provide<I_OrchestrationTools>,
     Singleton::Consume<I_ShellCmd>,
-    Singleton::Consume<I_TenantManager>
+    Singleton::Consume<I_TenantManager>,
+    Singleton::Consume<I_EnvDetails>,
+    Singleton::Consume<I_Messaging>,
+    Singleton::Consume<I_Environment>,
+    Singleton::Consume<I_MainLoop>,
+    Singleton::Consume<I_AgentDetails>
 {
 public:
     OrchestrationTools();

components/include/orchestrator/rest_api/orchestration_check_update.h

@@ -111,6 +111,26 @@ public:
     public:
         UpgradeSchedule() = default;
 
+        UpgradeSchedule(const UpgradeSchedule &other)
+        {
+            mode = other.mode;
+            time = other.time;
+            duration_hours = other.duration_hours;
+            days = other.days;
+        }
+
+        UpgradeSchedule &
+        operator=(const UpgradeSchedule &other)
+        {
+            if (this != &other) {
+                mode = other.mode;
+                time = other.time;
+                duration_hours = other.duration_hours;
+                days = other.days;
+            }
+            return *this;
+        }
+
         void init(const std::string &_upgrade_mode) { mode = _upgrade_mode; }
 
         void
@@ -142,6 +162,22 @@ public:
         C2S_LABEL_OPTIONAL_PARAM(std::vector<std::string>, days, "upgradeDay");
     };
 
+    class LocalConfigurationSettings : public ClientRest
+    {
+    public:
+        LocalConfigurationSettings() = default;
+
+        void
+        setUpgradeSchedule(const UpgradeSchedule &schedule)
+        {
+            upgrade_schedule.setActive(true);
+            upgrade_schedule.get() = schedule;
+        }
+
+    private:
+        C2S_LABEL_OPTIONAL_PARAM(UpgradeSchedule, upgrade_schedule, "upgradeSchedule");
+    };
+
     CheckUpdateRequest(
         const std::string &_manifest,
         const std::string &_policy,
@@ -224,8 +260,10 @@ public:
     void
     setUpgradeFields(const std::string &_upgrade_mode)
     {
-        upgrade_schedule.setActive(true);
-        upgrade_schedule.get().init(_upgrade_mode);
+        UpgradeSchedule upgrade_schedule;
+        upgrade_schedule.init(_upgrade_mode);
+        local_configuration_settings.setActive(true);
+        local_configuration_settings.get().setUpgradeSchedule(upgrade_schedule);
     }
 
     void
@@ -235,12 +273,14 @@ public:
         const uint &_upgrade_duration_hours,
         const std::vector<std::string> &_upgrade_days)
     {
-        upgrade_schedule.setActive(true);
+        UpgradeSchedule upgrade_schedule;
         if (!_upgrade_days.empty()) {
-            upgrade_schedule.get().init(_upgrade_mode, _upgrade_time, _upgrade_duration_hours, _upgrade_days);
-            return;
+            upgrade_schedule.init(_upgrade_mode, _upgrade_time, _upgrade_duration_hours, _upgrade_days);
+        } else {
+            upgrade_schedule.init(_upgrade_mode, _upgrade_time, _upgrade_duration_hours);
         }
-        upgrade_schedule.get().init(_upgrade_mode, _upgrade_time, _upgrade_duration_hours);
+        local_configuration_settings.setActive(true);
+        local_configuration_settings.get().setUpgradeSchedule(upgrade_schedule);
     }
 
 private:
@@ -297,7 +337,7 @@ private:
     C2S_LABEL_PARAM(std::string, checksum_type,  "checksum-type");
     C2S_LABEL_PARAM(std::string, policy_version, "policyVersion");
 
-    C2S_LABEL_OPTIONAL_PARAM(UpgradeSchedule, upgrade_schedule, "upgradeSchedule");
+    C2S_LABEL_OPTIONAL_PARAM(LocalConfigurationSettings, local_configuration_settings, "localConfigurationSettings");
 
     S2C_LABEL_OPTIONAL_PARAM(VirtualConfig, in_virtual_policy,    "virtualPolicy");
     S2C_LABEL_OPTIONAL_PARAM(VirtualConfig, in_virtual_settings,  "virtualSettings");

components/include/rate_limit.h

@@ -0,0 +1,32 @@
+#ifndef __RATE_LIMIT_H_
+#define __RATE_LIMIT_H_
+
+#include <string>
+
+#include "component.h"
+#include "singleton.h"
+#include "i_mainloop.h"
+#include "i_environment.h"
+
+class RateLimit
+    :
+    public Component,
+    Singleton::Consume<I_MainLoop>,
+    Singleton::Consume<I_TimeGet>,
+    Singleton::Consume<I_Environment>
+{
+public:
+    RateLimit();
+    ~RateLimit();
+
+    void preload() override;
+
+    void init() override;
+    void fini() override;
+
+private:
+    class Impl;
+    std::unique_ptr<Impl> pimpl;
+};
+
+#endif // __RATE_LIMIT_H_

components/include/rate_limit_config.h

@@ -0,0 +1,142 @@
+#ifndef __RATE_LIMIT_CONFIG_H__
+#define __RATE_LIMIT_CONFIG_H__
+
+#include <string>
+#include <vector>
+#include <algorithm>
+#include <cereal/archives/json.hpp>
+
+#include "debug.h"
+#include "generic_rulebase/rulebase_config.h"
+#include "generic_rulebase/triggers_config.h"
+#include "generic_rulebase/evaluators/trigger_eval.h"
+
+USE_DEBUG_FLAG(D_REVERSE_PROXY);
+
+class RateLimitTrigger
+{
+public:
+    void
+    load(cereal::JSONInputArchive &ar);
+
+    const std::string & getTriggerId() const { return id; }
+
+private:
+    std::string id;
+};
+
+class RateLimitRule
+{
+public:
+    void load(cereal::JSONInputArchive &ar);
+    void prepare(const std::string &asset_id, int zone_id);
+
+    operator bool() const
+    {
+        if (uri.empty()) {
+            dbgTrace(D_REVERSE_PROXY) << "Recived empty URI in rate-limit rule";
+            return false;
+        }
+
+        if (uri.at(0) != '/') {
+            dbgWarning(D_REVERSE_PROXY)
+                << "Recived invalid rate-limit URI in rate-limit rule: "
+                << uri
+                << " rate-limit URI must start with /";
+            return false;
+        }
+
+        if (limit <= 0) {
+            dbgWarning(D_REVERSE_PROXY)
+                << "Recived invalid rate-limit limit in rate-limit rule: "
+                << limit
+                << " rate-limit rule limit must be positive";
+            return false;
+        }
+
+        return true;
+    }
+
+    friend std::ostream &
+    operator<<(std::ostream &os, const RateLimitRule &rule)
+    {
+        os << "Uri: " << rule.uri << ", Rate scope: " << rule.scope << ", Limit: " << rule.limit;
+
+        return os;
+    }
+
+    int getRateLimit() const { return limit; }
+    const std::string & getRateLimitZone() const { return limit_req_zone_template_value; }
+    const std::string & getRateLimitReq() const { return limit_req_template_value; }
+    const std::string & getRateLimitUri() const { return uri; }
+    const std::string & getRateLimitScope() const { return scope; }
+    const LogTriggerConf & getRateLimitTrigger() const { return trigger; }
+    const std::vector<RateLimitTrigger> & getRateLimitTriggers() const { return rate_limit_triggers; }
+
+    bool isRootLocation() const;
+
+    bool operator==(const RateLimitRule &rhs) { return uri == rhs.uri; }
+    bool operator<(const RateLimitRule &rhs) { return uri < rhs.uri; }
+    bool isExactMatch() const { return exact_match || (!uri.empty() && uri.back() != '/'); }
+    void setExactMatch() { exact_match = true; }
+    void appendSlash() { uri += '/'; }
+
+private:
+    std::string uri;
+    std::string scope;
+    std::string limit_req_template_value;
+    std::string limit_req_zone_template_value;
+    std::string cache_size = "5m";
+    std::vector<RateLimitTrigger> rate_limit_triggers;
+    LogTriggerConf trigger;
+    int limit;
+    bool exact_match = false;
+};
+
+class RateLimitConfig
+{
+public:
+    void load(cereal::JSONInputArchive &ar);
+    void addSiblingRateLimitRule(RateLimitRule &rule);
+    void prepare();
+
+    const std::vector<RateLimitRule> & getRateLimitRules() const { return rate_limit_rules; }
+    const std::string & getRateLimitMode() const { return mode; }
+
+    const LogTriggerConf
+    getRateLimitTrigger(const std::string &nginx_uri) const
+    {
+        const RateLimitRule rule = findLongestMatchingRule(nginx_uri);
+
+        std::set<std::string> rate_limit_triggers_set;
+        for (const RateLimitTrigger &rate_limit_trigger : rule.getRateLimitTriggers()) {
+            dbgTrace(D_REVERSE_PROXY)
+                << "Adding trigger ID: "
+                << rate_limit_trigger.getTriggerId()
+                << " of rule URI: "
+                << rule.getRateLimitUri()
+                << " to the context set";
+            rate_limit_triggers_set.insert(rate_limit_trigger.getTriggerId());
+        }
+
+        ScopedContext ctx;
+        ctx.registerValue<std::set<GenericConfigId>>(TriggerMatcher::ctx_key, rate_limit_triggers_set);
+        return getConfigurationWithDefault(LogTriggerConf(), "rulebase", "log");
+    }
+
+    static void setIsActive(bool _is_active) { is_active |= _is_active; }
+
+    static void resetIsActive() { is_active = false; }
+
+    static bool isActive() { return is_active; }
+
+private:
+    const RateLimitRule
+    findLongestMatchingRule(const std::string &nginx_uri) const;
+
+    static bool is_active;
+    std::string mode;
+    std::vector<RateLimitRule> rate_limit_rules;
+};
+
+#endif // __RATE_LIMIT_CONFIG_H__

components/security_apps/CMakeLists.txt

@@ -1,4 +1,6 @@
 add_subdirectory(ips)
 add_subdirectory(layer_7_access_control)
+add_subdirectory(local_policy_mgmt_gen)
 add_subdirectory(orchestration)
+add_subdirectory(rate_limit)
 add_subdirectory(waap)

components/security_apps/layer_7_access_control/layer_7_access_control.cc

@@ -74,7 +74,7 @@ public:
     getCrowdsecEventId() const
     {
         if (!crowdsec_event_id) return genError("Empty ID");
-        return LogField("externalVendorRecommendationId", crowdsec_event_id);
+        return LogField("externalVendorRecommendationId", to_string(crowdsec_event_id));
     }
 
     bool isMalicious() const { return type == "ban"; }
@@ -280,6 +280,8 @@ Layer7AccessControl::Impl::generateLog(const string &source_ip, const Intelligen
         << LogField("sourceIP", source_ip)
         << LogField("externalVendorName", "CrowdSec")
         << LogField("waapIncidentType", "CrowdSec")
+        << LogField("practiceSubType", "Web Access Control")
+        << LogField("practiceType", "Access Control")
         << ip_reputation.getCrowdsecEventId()
         << ip_reputation.getType()
         << ip_reputation.getOrigin()

components/security_apps/layer_7_access_control/layer_7_access_control_ut/layer_7_access_control_ut.cc

@@ -248,7 +248,7 @@ Layer7AccessControlTest::verifyReport(
     EXPECT_THAT(log, HasSubstr("\"destinationIP\": \"5.6.7.8\""));
     EXPECT_THAT(log, HasSubstr("\"externalVendorName\": \"CrowdSec\""));
     EXPECT_THAT(log, HasSubstr("\"waapIncidentType\": \"CrowdSec\""));
-    EXPECT_THAT(log, HasSubstr("\"externalVendorRecommendationId\": 2253734"));
+    EXPECT_THAT(log, HasSubstr("\"externalVendorRecommendationId\": \"2253734\""));
     EXPECT_THAT(log, HasSubstr("\"externalVendorRecommendedAction\": \"ban\""));
     EXPECT_THAT(log, HasSubstr("\"externalVendorRecommendationOrigin\": \"cscli\""));
     EXPECT_THAT(log, HasSubstr("\"externalVendorRecommendedAffectedScope\": \"1.2.3.4\""));

components/security_apps/local_policy_mgmt_gen/CMakeLists.txt

@@ -0,0 +1,23 @@
+include_directories(include)
+add_library(local_policy_mgmt_gen
+    appsec_practice_section.cc
+    exceptions_section.cc
+    ingress_data.cc
+    rules_config_section.cc
+    settings_section.cc
+    snort_section.cc
+    triggers_section.cc
+    trusted_sources_section.cc
+    policy_maker_utils.cc
+    k8s_policy_utils.cc
+    local_policy_mgmt_gen.cc
+    new_appsec_policy_crd_parser.cc
+    new_appsec_linux_policy.cc
+    new_custom_response.cc
+    new_trusted_sources.cc
+    new_log_trigger.cc
+    new_practice.cc
+    new_exceptions.cc
+    access_control_practice.cc
+    configmaps.cc
+)

components/security_apps/local_policy_mgmt_gen/appsec_practice_section.cc

@@ -34,7 +34,7 @@ AppSecWebBotsURI::getURI() const
     return uri;
 }
 
-std::vector<std::string>
+vector<string>
 AppSecPracticeAntiBot::getIjectedUris() const
 {
     vector<string> injected;
@@ -44,7 +44,7 @@ AppSecPracticeAntiBot::getIjectedUris() const
     return injected;
 }
 
-std::vector<std::string>
+vector<string>
 AppSecPracticeAntiBot::getValidatedUris() const
 {
     vector<string> validated;
@@ -315,18 +315,74 @@ TriggersInWaapSection::save(cereal::JSONOutputArchive &out_ar) const
     );
 }
 
+ParsedMatch::ParsedMatch(const string &_operator, const string &_tag, const string &_value)
+        :
+    operator_type(_operator),
+    tag(_tag),
+    value(_value)
+{
+}
+
+// LCOV_EXCL_START Reason: no test exist
+ParsedMatch::ParsedMatch(const ExceptionMatch &exceptions)
+{
+    if (exceptions.getOperator() == "equals") {
+        operator_type = "basic";
+        tag = exceptions.getKey();
+        value = exceptions.getValue();
+    } else {
+        operator_type = exceptions.getOperator();
+    }
+    for (const ExceptionMatch &exception_match : exceptions.getMatch()) {
+        parsed_match.push_back(ParsedMatch(exception_match));
+    }
+}
+// LCOV_EXCL_STOP
+
+void
+ParsedMatch::save(cereal::JSONOutputArchive &out_ar) const
+{
+    if (parsed_match.size() > 0) {
+        out_ar(cereal::make_nvp("operator", operator_type));
+        int i = 0;
+        for (const ParsedMatch &operand : parsed_match) {
+            i++;
+            out_ar(cereal::make_nvp("operand" + to_string(i), operand));
+        }
+    } else {
+        out_ar(
+            cereal::make_nvp("operator",    operator_type),
+            cereal::make_nvp("tag",         tag),
+            cereal::make_nvp("value",       value)
+        );
+    }
+}
+
 AppSecOverride::AppSecOverride(const SourcesIdentifiers &parsed_trusted_sources)
 {
     string source_ident = parsed_trusted_sources.getSourceIdent();
     map<string, string> behavior = {{"httpSourceId", source_ident}};
     parsed_behavior.push_back(behavior);
-    parsed_match = {{"operator", "BASIC"}, {"tag", "sourceip"}, {"value", "0.0.0.0/0"}};
+    parsed_match = ParsedMatch("BASIC", "sourceip", "0.0.0.0/0");
 }
 
+// LCOV_EXCL_START Reason: no test exist
+AppSecOverride::AppSecOverride(const InnerException &parsed_exceptions)
+        :
+    id(parsed_exceptions.getBehaviorId()),
+    parsed_match(parsed_exceptions.getMatch())
+{
+    map<string, string> behavior = {{parsed_exceptions.getBehaviorKey(), parsed_exceptions.getBehaviorValue()}};
+    parsed_behavior.push_back(behavior);
+}
+// LCOV_EXCL_STOP
+
 void
 AppSecOverride::save(cereal::JSONOutputArchive &out_ar) const
 {
-    string parameter_type = "TrustedSource";
+    if (!id.empty()) {
+        out_ar(cereal::make_nvp("id", id));
+    }
     out_ar(
         cereal::make_nvp("parsedBehavior", parsed_behavior),
         cereal::make_nvp("parsedMatch",    parsed_match)
@@ -355,7 +411,8 @@ WebAppSection::WebAppSection(
     const AppSecPracticeSpec &parsed_appsec_spec,
     const LogTriggerSection &parsed_log_trigger,
     const string &default_mode,
-    const AppSecTrustedSources &parsed_trusted_sources)
+    const AppSecTrustedSources &parsed_trusted_sources,
+    const vector<InnerException> &parsed_exceptions)
         :
     application_urls(_application_urls),
     asset_id(_asset_id),
@@ -382,19 +439,23 @@ WebAppSection::WebAppSection(
     for (const SourcesIdentifiers &source_ident : parsed_trusted_sources.getSourcesIdentifiers()) {
         overrides.push_back(AppSecOverride(source_ident));
     }
+
+    for (const InnerException &exception : parsed_exceptions) {
+        overrides.push_back(AppSecOverride(exception));
+    }
 }
 
 WebAppSection::WebAppSection(
-    const std::string &_application_urls,
-    const std::string &_asset_id,
-    const std::string &_asset_name,
-    const std::string &_rule_id,
-    const std::string &_rule_name,
-    const std::string &_practice_id,
-    const std::string &_practice_name,
+    const string &_application_urls,
+    const string &_asset_id,
+    const string &_asset_name,
+    const string &_rule_id,
+    const string &_rule_name,
+    const string &_practice_id,
+    const string &_practice_name,
     const string &_context,
-    const std::string &_web_attack_mitigation_severity,
-    const std::string &_web_attack_mitigation_mode,
+    const string &_web_attack_mitigation_severity,
+    const string &_web_attack_mitigation_mode,
     const PracticeAdvancedConfig &_practice_advanced_config,
     const AppsecPracticeAntiBotSection &_anti_bots,
     const LogTriggerSection &parsed_log_trigger,
@@ -611,7 +672,7 @@ AppsecPolicySpec::getSpecificRules() const
 }
 
 bool
-AppsecPolicySpec::isAssetHostExist(const std::string &full_url) const
+AppsecPolicySpec::isAssetHostExist(const string &full_url) const
 {
     for (const ParsedRule &rule : specific_rules) {
         if (rule.getHost() == full_url) return true;
@@ -633,7 +694,7 @@ AppsecLinuxPolicy::serialize(cereal::JSONInputArchive &archive_in)
     parseAppsecJSONKey<vector<AppSecPracticeSpec>>("practices", practices, archive_in);
     parseAppsecJSONKey<vector<AppsecTriggerSpec>>("log-triggers", log_triggers, archive_in);
     parseAppsecJSONKey<vector<AppSecCustomResponseSpec>>("custom-responses", custom_responses, archive_in);
-    parseAppsecJSONKey<vector<AppsecExceptionSpec>>("exceptions", exceptions, archive_in);
+    parseAppsecJSONKey<vector<AppsecException>>("exceptions", exceptions, archive_in);
     parseAppsecJSONKey<vector<TrustedSourcesSpec>>("trusted-sources", trusted_sources, archive_in);
     parseAppsecJSONKey<vector<SourceIdentifierSpecWrapper>>(
         "source-identifiers",
@@ -666,8 +727,8 @@ AppsecLinuxPolicy::getAppSecCustomResponseSpecs() const
     return custom_responses;
 }
 
-const vector<AppsecExceptionSpec> &
-AppsecLinuxPolicy::getAppsecExceptionSpecs() const
+const vector<AppsecException> &
+AppsecLinuxPolicy::getAppsecExceptions() const
 {
     return exceptions;
 }

components/security_apps/local_policy_mgmt_gen/configmaps.cc

@@ -0,0 +1,58 @@
+// Copyright (C) 2022 Check Point Software Technologies Ltd. All rights reserved.
+
+// Licensed under the Apache License, Version 2.0 (the "License");
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "configmaps.h"
+
+using namespace std;
+
+USE_DEBUG_FLAG(D_LOCAL_POLICY);
+
+// LCOV_EXCL_START Reason: no test exist
+bool
+ConfigMaps::loadJson(const std::string &json)
+{
+    string modified_json = json;
+    modified_json.pop_back();
+    stringstream in;
+    in.str(modified_json);
+    dbgTrace(D_LOCAL_POLICY) << "Loading ConfigMaps data";
+    try {
+        cereal::JSONInputArchive in_ar(in);
+        in_ar(
+            cereal::make_nvp("data", data)
+        );
+    } catch (cereal::Exception &e) {
+        dbgError(D_LOCAL_POLICY) << "Failed to load ConfigMaps JSON. Error: " << e.what();
+        return false;
+    }
+    return true;
+}
+
+string
+ConfigMaps::getFileContent() const
+{
+    if (data.size()) {
+        return data.begin()->second;
+    }
+    return string();
+}
+
+string
+ConfigMaps::getFileName() const
+{
+    if (data.size()) {
+        return data.begin()->first;
+    }
+    return string();
+}
+// LCOV_EXCL_STOP

components/security_apps/local_policy_mgmt_gen/exceptions_section.cc

@@ -18,39 +18,61 @@ using namespace std;
 USE_DEBUG_FLAG(D_LOCAL_POLICY);
 
 // LCOV_EXCL_START Reason: no test exist
+using AttributeGetter = function<vector<string>(const AppsecExceptionSpec&)>;
+static const vector<pair<string, AttributeGetter>> attributes = {
+    {"countryCode",      [](const AppsecExceptionSpec& e){ return e.getCountryCode(); }},
+    {"countryName",      [](const AppsecExceptionSpec& e){ return e.getCountryName(); }},
+    {"hostName",         [](const AppsecExceptionSpec& e){ return e.getHostName(); }},
+    {"paramName",        [](const AppsecExceptionSpec& e){ return e.getParamName(); }},
+    {"paramValue",       [](const AppsecExceptionSpec& e){ return e.getParamValue(); }},
+    {"protectionName",   [](const AppsecExceptionSpec& e){ return e.getProtectionName(); }},
+    {"sourceIdentifier", [](const AppsecExceptionSpec& e){ return e.getSourceIdentifier(); }},
+    {"sourceIp",         [](const AppsecExceptionSpec& e){ return e.getSourceIp(); }},
+    {"url",              [](const AppsecExceptionSpec& e){ return e.getUrl(); }}
+};
 static const set<string> valid_actions = {"skip", "accept", "drop", "suppressLog"};
+static const unordered_map<string, string> key_to_action = {
+    { "accept", "accept"},
+    { "drop", "reject"},
+    { "skip", "ignore"},
+    { "suppressLog", "ignore"}
+};
 
 void
 AppsecExceptionSpec::load(cereal::JSONInputArchive &archive_in)
 {
     dbgTrace(D_LOCAL_POLICY) << "Loading AppSec exception spec";
-    parseAppsecJSONKey<string>("name", name, archive_in);
-    parseAppsecJSONKey<string>("action", action, archive_in);
+    parseAppsecJSONKey<string>("action", action, archive_in, "skip");
     if (valid_actions.count(action) == 0) {
         dbgWarning(D_LOCAL_POLICY) << "AppSec exception action invalid: " << action;
     }
 
     parseAppsecJSONKey<vector<string>>("countryCode", country_code, archive_in);
+    if (!country_code.empty()) conditions_number++;
+
     parseAppsecJSONKey<vector<string>>("countryName", country_name, archive_in);
+    if (!country_name.empty()) conditions_number++;
+
     parseAppsecJSONKey<vector<string>>("hostName", host_name, archive_in);
+    if (!host_name.empty()) conditions_number++;
+
     parseAppsecJSONKey<vector<string>>("paramName", param_name, archive_in);
+    if (!param_name.empty()) conditions_number++;
+
     parseAppsecJSONKey<vector<string>>("paramValue", param_value, archive_in);
+    if (!param_value.empty()) conditions_number++;
+
     parseAppsecJSONKey<vector<string>>("protectionName", protection_name, archive_in);
+    if (!protection_name.empty()) conditions_number++;
+
     parseAppsecJSONKey<vector<string>>("sourceIdentifier", source_identifier, archive_in);
+    if (!source_identifier.empty()) conditions_number++;
+
     parseAppsecJSONKey<vector<string>>("sourceIp", source_ip, archive_in);
+    if (!source_ip.empty()) conditions_number++;
+
     parseAppsecJSONKey<vector<string>>("url", url, archive_in);
-}
-
-void
-AppsecExceptionSpec::setName(const string &_name)
-{
-    name = _name;
-}
-
-const string &
-AppsecExceptionSpec::getName() const
-{
-    return name;
+    if (!url.empty()) conditions_number++;
 }
 
 const string &
@@ -113,37 +135,82 @@ AppsecExceptionSpec::getUrl() const
     return url;
 }
 
+bool
+AppsecExceptionSpec::isOneCondition() const
+{
+    return conditions_number == 1;
+}
+
+void
+AppsecException::load(cereal::JSONInputArchive &archive_in)
+{
+    dbgTrace(D_LOCAL_POLICY) << "Loading AppSec exception";
+    parseAppsecJSONKey<string>("name", name, archive_in);
+    archive_in(CEREAL_NVP(exception_spec));
+}
+
+void
+AppsecException::setName(const string &_name)
+{
+    name = _name;
+}
+
+const string &
+AppsecException::getName() const
+{
+    return name;
+}
+
+const vector<AppsecExceptionSpec> &
+AppsecException::getExceptions() const
+{
+    return exception_spec;
+}
+
 ExceptionMatch::ExceptionMatch(const AppsecExceptionSpec &parsed_exception)
         :
     match_type(MatchType::Operator),
     op("and")
 {
-    if (!parsed_exception.getCountryCode().empty()) {
-        items.push_back(ExceptionMatch("countryCode", parsed_exception.getCountryCode()));
+    bool single_condition = parsed_exception.isOneCondition();
+    for (auto &attrib : attributes) {
+        auto &attrib_name = attrib.first;
+        auto &attrib_getter = attrib.second;
+        auto exceptions_value = attrib_getter(parsed_exception);
+        if (exceptions_value.empty()) continue;
+        if (single_condition) {
+            if (exceptions_value.size() == 1) {
+                match_type = MatchType::Condition;
+                op = "equals";
+                key = attrib_name;
+                value = exceptions_value;
+                return;
+            } else {
+                match_type = MatchType::Operator;
+                op = "or";
+                for (auto new_value : exceptions_value) {
+                    items.push_back(ExceptionMatch(attrib_name, {new_value}));
+                }
+                return;
+            }
+        }
+        items.push_back(ExceptionMatch(attrib_name, exceptions_value));
     }
-    if (!parsed_exception.getCountryName().empty()) {
-        items.push_back(ExceptionMatch("countryName", parsed_exception.getCountryName()));
-    }
-    if (!parsed_exception.getHostName().empty()) {
-        items.push_back(ExceptionMatch("hostName", parsed_exception.getHostName()));
-    }
-    if (!parsed_exception.getParamName().empty()) {
-        items.push_back(ExceptionMatch("paramName", parsed_exception.getParamName()));
-    }
-    if (!parsed_exception.getParamValue().empty()) {
-        items.push_back(ExceptionMatch("paramValue", parsed_exception.getParamValue()));
-    }
-    if (!parsed_exception.getProtectionName().empty()) {
-        items.push_back(ExceptionMatch("protectionName", parsed_exception.getProtectionName()));
-    }
-    if (!parsed_exception.getSourceIdentifier().empty()) {
-        items.push_back(ExceptionMatch("sourceIdentifier", parsed_exception.getSourceIdentifier()));
-    }
-    if (!parsed_exception.getSourceIp().empty()) {
-        items.push_back(ExceptionMatch("sourceIp", parsed_exception.getSourceIp()));
-    }
-    if (!parsed_exception.getUrl().empty()) {
-        items.push_back(ExceptionMatch("url", parsed_exception.getUrl()));
+}
+
+ExceptionMatch::ExceptionMatch(const std::string &_key, const std::vector<std::string> &values)
+{
+    if (values.size() == 1) {
+        match_type = MatchType::Condition;
+        op = "equals";
+        key = _key;
+        value = values;
+    } else {
+        match_type = MatchType::Operator;
+        op = "or";
+        for (auto new_value : values) {
+            items.push_back(ExceptionMatch(_key, {new_value}));
+        }
     }
 }
 
@@ -210,13 +277,34 @@ ExceptionMatch::save(cereal::JSONOutputArchive &out_ar) const
     }
 }
 
-ExceptionBehavior::ExceptionBehavior(
-    const string &_key,
-    const string &_value)
-        :
-    key(_key),
-    value(_value)
+const string &
+ExceptionMatch::getOperator() const
 {
+    return op;
+}
+
+const string &
+ExceptionMatch::getKey() const
+{
+    return key;
+}
+
+const string &
+ExceptionMatch::getValue() const
+{
+    return value[0];
+}
+
+const vector<ExceptionMatch> &
+ExceptionMatch::getMatch() const
+{
+    return items;
+}
+
+ExceptionBehavior::ExceptionBehavior(const string &_value)
+{
+    key = _value == "suppressLog" ? "log" : "action";
+    value = key_to_action.at(_value);
     try {
         id = to_string(boost::uuids::random_generator()());
     } catch (const boost::uuids::entropy_error &e) {
@@ -234,12 +322,31 @@ ExceptionBehavior::save(cereal::JSONOutputArchive &out_ar) const
     );
 }
 
-const string
+const string &
 ExceptionBehavior::getBehaviorId() const
 {
     return id;
 }
 
+const string &
+ExceptionBehavior::getBehaviorKey() const
+{
+    return key;
+}
+
+const string &
+ExceptionBehavior::getBehaviorValue() const
+{
+    return value;
+}
+
+InnerException::InnerException(ExceptionBehavior _behavior, ExceptionMatch _match)
+        :
+    behavior(_behavior),
+    match(_match)
+{
+}
+
 void
 InnerException::save(cereal::JSONOutputArchive &out_ar) const
 {
@@ -249,12 +356,30 @@ InnerException::save(cereal::JSONOutputArchive &out_ar) const
     );
 }
 
-const string
+const string &
 InnerException::getBehaviorId() const
 {
     return behavior.getBehaviorId();
 }
 
+const string &
+InnerException::getBehaviorKey() const
+{
+    return behavior.getBehaviorKey();
+}
+
+const string &
+InnerException::getBehaviorValue() const
+{
+    return behavior.getBehaviorValue();
+}
+
+const ExceptionMatch &
+InnerException::getMatch() const
+{
+    return match;
+}
+
 ExceptionsRulebase::ExceptionsRulebase(
     vector<InnerException> _exceptions)
         :

components/security_apps/local_policy_mgmt_gen/include/appsec_practice_section.h

@@ -202,16 +202,35 @@ private:
     LogTriggerSection log;
 };
 
-class AppSecOverride
+class ParsedMatch
 {
 public:
-    AppSecOverride(const SourcesIdentifiers &parsed_trusted_sources);
+    ParsedMatch() {}
+    ParsedMatch(const std::string &_operator, const std::string &_tag, const std::string &_value);
+
+    ParsedMatch(const ExceptionMatch &exceptions);
 
     void save(cereal::JSONOutputArchive &out_ar) const;
 
 private:
+    std::string operator_type;
+    std::string tag;
+    std::string value;
+    std::vector<ParsedMatch> parsed_match;
+};
+
+class AppSecOverride
+{
+public:
+    AppSecOverride(const SourcesIdentifiers &parsed_trusted_sources);
+    AppSecOverride(const InnerException &parsed_exceptions);
+
+    void save(cereal::JSONOutputArchive &out_ar) const;
+
+private:
+    std::string id;
     std::vector<std::map<std::string, std::string>> parsed_behavior;
-    std::map<std::string, std::string> parsed_match;
+    ParsedMatch parsed_match;
 };
 
 class AppsecPracticeAntiBotSection
@@ -254,7 +273,8 @@ public:
         const AppSecPracticeSpec &parsed_appsec_spec,
         const LogTriggerSection &parsed_log_trigger,
         const std::string &default_mode,
-        const AppSecTrustedSources &parsed_trusted_sources
+        const AppSecTrustedSources &parsed_trusted_sources,
+        const std::vector<InnerException> &parsed_exceptions
     );
 
     WebAppSection(
@@ -430,7 +450,7 @@ public:
         const std::vector<AppSecPracticeSpec> &_practices,
         const std::vector<AppsecTriggerSpec> &_log_triggers,
         const std::vector<AppSecCustomResponseSpec> &_custom_responses,
-        const std::vector<AppsecExceptionSpec> &_exceptions,
+        const std::vector<AppsecException> &_exceptions,
         const std::vector<TrustedSourcesSpec> &_trusted_sources,
         const std::vector<SourceIdentifierSpecWrapper> &_sources_identifiers)
             :
@@ -448,7 +468,7 @@ public:
     const std::vector<AppSecPracticeSpec> & getAppSecPracticeSpecs() const;
     const std::vector<AppsecTriggerSpec> & getAppsecTriggerSpecs() const;
     const std::vector<AppSecCustomResponseSpec> & getAppSecCustomResponseSpecs() const;
-    const std::vector<AppsecExceptionSpec> & getAppsecExceptionSpecs() const;
+    const std::vector<AppsecException> & getAppsecExceptions() const;
     const std::vector<TrustedSourcesSpec> & getAppsecTrustedSourceSpecs() const;
     const std::vector<SourceIdentifierSpecWrapper> & getAppsecSourceIdentifierSpecs() const;
     void addSpecificRule(const ParsedRule &_rule);
@@ -458,7 +478,7 @@ private:
     std::vector<AppSecPracticeSpec> practices;
     std::vector<AppsecTriggerSpec> log_triggers;
     std::vector<AppSecCustomResponseSpec> custom_responses;
-    std::vector<AppsecExceptionSpec> exceptions;
+    std::vector<AppsecException> exceptions;
     std::vector<TrustedSourcesSpec> trusted_sources;
     std::vector<SourceIdentifierSpecWrapper> sources_identifiers;
 };

components/security_apps/local_policy_mgmt_gen/include/configmaps.h

@@ -0,0 +1,41 @@
+// Copyright (C) 2022 Check Point Software Technologies Ltd. All rights reserved.
+
+// Licensed under the Apache License, Version 2.0 (the "License");
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef __CONFIGMAPS_H__
+#define __CONFIGMAPS_H__
+
+#include <vector>
+#include <map>
+
+#include "config.h"
+#include "debug.h"
+#include "rest.h"
+#include "cereal/archives/json.hpp"
+#include <cereal/types/map.hpp>
+#include "customized_cereal_map.h"
+
+#include "local_policy_common.h"
+
+class ConfigMaps : public ClientRest
+{
+public:
+    bool loadJson(const std::string &json);
+
+    std::string getFileContent() const;
+    std::string getFileName() const;
+
+private:
+    std::map<std::string, std::string> data;
+};
+
+#endif // __CONFIGMAPS_H__

components/security_apps/local_policy_mgmt_gen/include/exceptions_section.h

@@ -31,7 +31,6 @@ class AppsecExceptionSpec
 public:
     void load(cereal::JSONInputArchive &archive_in);
 
-    const std::string & getName() const;
     const std::string & getAction() const;
     const std::vector<std::string> & getCountryCode() const;
     const std::vector<std::string> & getCountryName() const;
@@ -42,10 +41,10 @@ public:
     const std::vector<std::string> & getSourceIdentifier() const;
     const std::vector<std::string> & getSourceIp() const;
     const std::vector<std::string> & getUrl() const;
-    void setName(const std::string &_name);
+    bool isOneCondition() const;
 
 private:
-    std::string name;
+    int conditions_number;
     std::string action;
     std::vector<std::string> country_code;
     std::vector<std::string> country_name;
@@ -58,21 +57,42 @@ private:
     std::vector<std::string> url;
 };
 
+class AppsecException
+{
+public:
+    AppsecException() {};
+
+    // LCOV_EXCL_START Reason: no test exist
+    AppsecException(const std::string &_name, const std::vector<AppsecExceptionSpec> &_exception_spec)
+        :
+    name(_name),
+    exception_spec(_exception_spec) {};
+    // LCOV_EXCL_STOP
+
+    void load(cereal::JSONInputArchive &archive_in);
+
+    const std::string & getName() const;
+    const std::vector<AppsecExceptionSpec> & getExceptions() const;
+    void setName(const std::string &_name);
+
+private:
+    std::string name;
+    std::vector<AppsecExceptionSpec> exception_spec;
+};
+
 class ExceptionMatch
 {
 public:
     ExceptionMatch() {}
     ExceptionMatch(const AppsecExceptionSpec &parsed_exception);
+    ExceptionMatch(const std::string &_key, const std::vector<std::string> &_value);
     ExceptionMatch(const NewAppsecException &parsed_exception);
-    ExceptionMatch(const std::string &_key, const std::vector<std::string> &_value)
-            :
-        match_type(MatchType::Condition),
-        key(_key),
-        op("in"),
-        value(_value)
-    {}
 
     void save(cereal::JSONOutputArchive &out_ar) const;
+    const std::string & getOperator() const;
+    const std::string & getKey() const;
+    const std::string & getValue() const;
+    const std::vector<ExceptionMatch> & getMatch() const;
 
 private:
     MatchType match_type;
@@ -86,13 +106,12 @@ class ExceptionBehavior
 {
 public:
     ExceptionBehavior() {}
-    ExceptionBehavior(
-        const std::string &_key,
-        const std::string &_value
-    );
+    ExceptionBehavior(const std::string &_value);
 
     void save(cereal::JSONOutputArchive &out_ar) const;
-    const std::string getBehaviorId() const;
+    const std::string & getBehaviorId() const;
+    const std::string & getBehaviorKey() const;
+    const std::string & getBehaviorValue() const;
 
 private:
     std::string key;
@@ -104,15 +123,13 @@ class InnerException
 {
 public:
     InnerException() {}
-    InnerException(
-        ExceptionBehavior _behavior,
-        ExceptionMatch _match)
-            :
-        behavior(_behavior),
-        match(_match) {}
+    InnerException(ExceptionBehavior _behavior, ExceptionMatch _match);
 
     void save(cereal::JSONOutputArchive &out_ar) const;
-    const std::string getBehaviorId() const;
+    const std::string & getBehaviorId() const;
+    const std::string & getBehaviorKey() const;
+    const std::string & getBehaviorValue() const;
+    const ExceptionMatch & getMatch() const;
 
 private:
     ExceptionBehavior behavior;

components/security_apps/local_policy_mgmt_gen/include/k8s_policy_utils.h

@@ -47,7 +47,7 @@ public:
 
     std::tuple<std::map<std::string, AppsecLinuxPolicy>, std::map<std::string, V1beta2AppsecLinuxPolicy>>
     createAppsecPoliciesFromIngresses();
-    bool getClusterId() const;
+    void getClusterId() const;
 
 private:
     std::map<AnnotationKeys, std::string> parseIngressAnnotations(
@@ -67,12 +67,19 @@ private:
         const NewParsedRule &default_rule
     ) const;
 
+    std::vector<AppsecException> extractExceptionsFromCluster(
+        const std::string &crd_plural,
+        const std::unordered_set<std::string> &elements_names
+    ) const;
+
     template<class T>
     std::vector<T> extractElementsFromCluster(
         const std::string &crd_plural,
         const std::unordered_set<std::string> &elements_names
     ) const;
 
+    void createSnortFile(std::vector<NewAppSecPracticeSpec> &practices) const;
+
     template<class T>
     std::vector<T> extractV1Beta2ElementsFromCluster(
         const std::string &crd_plural,

components/security_apps/local_policy_mgmt_gen/include/new_appsec_linux_policy.h

@@ -65,7 +65,7 @@ public:
     const std::vector<AccessControlPracticeSpec> & getAccessControlPracticeSpecs() const;
     const std::vector<NewAppsecLogTrigger> & getAppsecTriggerSpecs() const;
     const std::vector<NewAppSecCustomResponse> & getAppSecCustomResponseSpecs() const;
-    const std::vector<NewAppsecException> & getAppsecExceptionSpecs() const;
+    const std::vector<NewAppsecException> & getAppsecExceptions() const;
     const std::vector<NewTrustedSourcesSpec> & getAppsecTrustedSourceSpecs() const;
     const std::vector<NewSourcesIdentifiers> & getAppsecSourceIdentifierSpecs() const;
     void addSpecificRule(const NewParsedRule &_rule);

components/security_apps/local_policy_mgmt_gen/include/new_practice.h

@@ -147,25 +147,25 @@ public:
     // LCOV_EXCL_STOP
 
     FileSecurityProtectionsSection(
-        int                     file_size_limit,
-        int                     archive_file_size_limit,
-        bool                    allow_files_without_name,
-        bool                    required_file_size_limit,
-        bool                    required_archive_extraction,
-        const std::string       &context,
-        const std::string       &name,
-        const std::string       &asset_id,
-        const std::string       &practice_name,
-        const std::string       &practice_id,
-        const std::string       &action,
-        const std::string       &files_without_name_action,
-        const std::string       &high_confidence_action,
-        const std::string       &medium_confidence_action,
-        const std::string       &low_confidence_action,
-        const std::string       &severity_level,
-        const std::string       &fileSize_limit_action,
-        const std::string       &multi_level_archive_action,
-        const std::string       &unopened_archive_actio
+        int                     _file_size_limit,
+        int                     _archive_file_size_limit,
+        bool                    _allow_files_without_name,
+        bool                    _required_file_size_limit,
+        bool                    _required_archive_extraction,
+        const std::string       &_context,
+        const std::string       &_name,
+        const std::string       &_asset_id,
+        const std::string       &_practice_name,
+        const std::string       &_practice_id,
+        const std::string       &_action,
+        const std::string       &_files_without_name_action,
+        const std::string       &_high_confidence_action,
+        const std::string       &_medium_confidence_action,
+        const std::string       &_low_confidence_action,
+        const std::string       &_severity_level,
+        const std::string       &_file_size_limit_action,
+        const std::string       &_multi_level_archive_action,
+        const std::string       &_unopened_archive_action
     );
 
     void save(cereal::JSONOutputArchive &out_ar) const;
@@ -265,6 +265,7 @@ class NewFileSecurity
 public:
     void load(cereal::JSONInputArchive &archive_in);
 
+    const std::string & getOverrideMode() const;
     const NewFileSecurityArchiveInspection & getArchiveInspection() const;
     const NewFileSecurityLargeFileInspection & getLargeFileInspection() const;
     FileSecurityProtectionsSection createFileSecurityProtectionsSection(
@@ -287,17 +288,210 @@ private:
     NewFileSecurityLargeFileInspection  large_file_inspection;
 };
 
+class SnortProtectionsSection
+{
+public:
+    // LCOV_EXCL_START Reason: no test exist
+    SnortProtectionsSection() {};
+    // LCOV_EXCL_STOP
+
+    SnortProtectionsSection(
+        const std::string               &_context,
+        const std::string               &_asset_name,
+        const std::string               &_asset_id,
+        const std::string               &_practice_name,
+        const std::string               &_practice_id,
+        const std::string               &_source_identifier,
+        const std::string               &_mode,
+        const std::vector<std::string>  &_files
+    );
+
+    void save(cereal::JSONOutputArchive &out_ar) const;
+
+private:
+    std::string                                 context;
+    std::string                                 asset_name;
+    std::string                                 asset_id;
+    std::string                                 practice_name;
+    std::string                                 practice_id;
+    std::string                                 source_identifier;
+    std::string                                 mode;
+    std::vector<std::string>                    files;
+};
+
+class DetectionRules
+{
+public:
+    // LCOV_EXCL_START Reason: no test exist
+    DetectionRules() {};
+    // LCOV_EXCL_STOP
+
+    DetectionRules(
+        const std::string                   &_type,
+        const std::string                   &_SSM,
+        const std::string                   &_keywords,
+        const std::vector<std::string>      &_context
+    );
+
+    void load(cereal::JSONInputArchive &archive_in);
+    void save(cereal::JSONOutputArchive &out_ar) const;
+
+private:
+    std::string                 type;
+    std::string                 SSM;
+    std::string                 keywords;
+    std::vector<std::string>    context;
+};
+
+class ProtectionMetadata
+{
+public:
+    // LCOV_EXCL_START Reason: no test exist
+    ProtectionMetadata() {};
+    // LCOV_EXCL_STOP
+
+    ProtectionMetadata(
+        bool                                _silent,
+        const std::string                   &_protection_name,
+        const std::string                   &_severity,
+        const std::string                   &_confidence_level,
+        const std::string                   &_performance_impact,
+        const std::string                   &_last_update,
+        const std::string                   &_maintrain_id,
+        const std::vector<std::string>      &_tags,
+        const std::vector<std::string>      &_cve_list
+    );
+
+    void load(cereal::JSONInputArchive &archive_in);
+    void save(cereal::JSONOutputArchive &out_ar) const;
+
+private:
+    bool                        silent;
+    std::string                 protection_name;
+    std::string                 severity;
+    std::string                 confidence_level;
+    std::string                 performance_impact;
+    std::string                 last_update;
+    std::string                 maintrain_id;
+    std::vector<std::string>    tags;
+    std::vector<std::string>    cve_list;
+};
+
+class ProtectionsProtectionsSection
+{
+public:
+    // LCOV_EXCL_START Reason: no test exist
+    ProtectionsProtectionsSection() {};
+    // LCOV_EXCL_STOP
+
+    ProtectionsProtectionsSection(
+        const ProtectionMetadata    &_protection_metadata,
+        const DetectionRules        &_detection_rules
+    );
+
+    void load(cereal::JSONInputArchive &archive_in);
+    void save(cereal::JSONOutputArchive &out_ar) const;
+
+private:
+    ProtectionMetadata  protection_metadata;
+    DetectionRules      detection_rules;
+};
+
+class ProtectionsSection
+{
+public:
+    // LCOV_EXCL_START Reason: no test exist
+    ProtectionsSection() {};
+    // LCOV_EXCL_STOP
+
+    ProtectionsSection(
+        const std::vector<ProtectionsProtectionsSection>    &_protections,
+        const std::string                                   &_name = "",
+        const std::string                                   &_modification_time = ""
+    );
+
+    void load(cereal::JSONInputArchive &archive_in);
+    void save(cereal::JSONOutputArchive &out_ar) const;
+    const std::vector<ProtectionsProtectionsSection> & getProtections() const;
+
+private:
+    std::vector<ProtectionsProtectionsSection>  protections;
+    std::string                                 name;
+    std::string                                 modification_time;
+};
+
+class ProtectionsSectionWrapper
+{
+public:
+    // LCOV_EXCL_START Reason: no test exist
+    ProtectionsSectionWrapper() {};
+    // LCOV_EXCL_STOP
+
+    void serialize(cereal::JSONInputArchive &archive_in);
+    const std::vector<ProtectionsProtectionsSection> & getProtections() const;
+
+private:
+    ProtectionsSection  protections;
+};
+
+class SnortSection
+{
+public:
+    // LCOV_EXCL_START Reason: no test exist
+    SnortSection() {};
+
+    SnortSection(
+        const std::vector<SnortProtectionsSection> &_snort,
+        const std::vector<ProtectionsSection> &_protections)
+            :
+        snort_protections(_snort),
+        protections(_protections)
+    {};
+    // LCOV_EXCL_STOP
+
+    void load(cereal::JSONInputArchive &archive_in);
+    void save(cereal::JSONOutputArchive &out_ar) const;
+    const std::vector<ProtectionsSection> & getProtections() const;
+
+private:
+    std::vector<SnortProtectionsSection> snort_protections;
+    std::vector<ProtectionsSection> protections;
+};
+
+class SnortSectionWrapper
+{
+public:
+    // LCOV_EXCL_START Reason: no test exist
+    SnortSectionWrapper() {};
+
+    SnortSectionWrapper(
+        const std::vector<SnortProtectionsSection> &_snort,
+        const std::vector<ProtectionsSection> &_protections)
+            :
+        snort(SnortSection(_snort, _protections))
+    {};
+    // LCOV_EXCL_STOP
+
+    void save(cereal::JSONOutputArchive &out_ar) const;
+
+private:
+    SnortSection snort;
+};
+
 class NewSnortSignaturesAndOpenSchemaAPI
 {
 public:
     void load(cereal::JSONInputArchive &archive_in);
 
+    void addFile(const std::string &file_name);
     const std::string & getOverrideMode() const;
     const std::vector<std::string> & getConfigMap() const;
+    const std::vector<std::string> & getFiles() const;
 
 private:
     std::string override_mode;
     std::vector<std::string> config_map;
+    std::vector<std::string> files;
 };
 
 class NewAppSecWebBotsURI
@@ -371,8 +565,8 @@ class NewAppSecPracticeSpec
 public:
     void load(cereal::JSONInputArchive &archive_in);
 
+    NewSnortSignaturesAndOpenSchemaAPI & getSnortSignatures();
     const NewSnortSignaturesAndOpenSchemaAPI & getOpenSchemaValidation() const;
-    const NewSnortSignaturesAndOpenSchemaAPI & getSnortSignatures() const;
     const NewAppSecPracticeWebAttacks & getWebAttacks() const;
     const NewAppSecPracticeAntiBot & getAntiBot() const;
     const NewIntrusionPrevention & getIntrusionPrevention() const;

components/security_apps/local_policy_mgmt_gen/include/policy_maker_utils.h

@@ -57,19 +57,21 @@ class SecurityAppsWrapper
 {
 public:
     SecurityAppsWrapper(
-        const AppSecWrapper &_waap,
-        const TriggersWrapper &_trrigers,
-        const RulesConfigWrapper &_rules,
-        const IntrusionPreventionWrapper &_ips,
-        const AccessControlRulebaseWrapper &_rate_limit,
-        const FileSecurityWrapper &_file_security,
-        const ExceptionsWrapper &_exceptions,
-        const std::string &_policy_version)
+        const AppSecWrapper                 &_waap,
+        const TriggersWrapper               &_trrigers,
+        const RulesConfigWrapper            &_rules,
+        const IntrusionPreventionWrapper    &_ips,
+        const SnortSectionWrapper           &_snort,
+        const AccessControlRulebaseWrapper  &_rate_limit,
+        const FileSecurityWrapper           &_file_security,
+        const ExceptionsWrapper             &_exceptions,
+        const std::string                   &_policy_version)
             :
         waap(_waap),
         trrigers(_trrigers),
         rules(_rules),
         ips(_ips),
+        snort(_snort),
         rate_limit(_rate_limit),
         file_security(_file_security),
         exceptions(_exceptions),
@@ -78,14 +80,15 @@ public:
     void save(cereal::JSONOutputArchive &out_ar) const;
 
 private:
-    AppSecWrapper waap;
-    TriggersWrapper trrigers;
-    RulesConfigWrapper rules;
-    IntrusionPreventionWrapper  ips;
-    AccessControlRulebaseWrapper rate_limit;
-    FileSecurityWrapper file_security;
-    ExceptionsWrapper exceptions;
-    std::string policy_version;
+    AppSecWrapper                   waap;
+    TriggersWrapper                 trrigers;
+    RulesConfigWrapper              rules;
+    IntrusionPreventionWrapper      ips;
+    SnortSectionWrapper             snort;
+    AccessControlRulebaseWrapper    rate_limit;
+    FileSecurityWrapper             file_security;
+    ExceptionsWrapper               exceptions;
+    std::string                     policy_version;
 };
 
 class PolicyWrapper
@@ -129,7 +132,8 @@ public:
 private:
     std::string getPolicyName(const std::string &policy_path);
 
-    Maybe<AppsecLinuxPolicy> openPolicyAsJson(const std::string &policy_path);
+    template<class T>
+    Maybe<T> openFileAsJson(const std::string &path);
 
     void clearElementsMaps();
 
@@ -155,6 +159,20 @@ private:
         std::map<AnnotationTypes, std::string> &rule_annotations
     );
 
+    void createSnortProtecionsSection(const std::string &file_name, const std::string &practic_name);
+
+    void
+    createSnortSections(
+        const std::string & context,
+        const std::string &asset_name,
+        const std::string &asset_id,
+        const std::string &practice_name,
+        const std::string &practice_id,
+        const std::string &source_identifier,
+        const V1beta2AppsecLinuxPolicy &policy,
+        std::map<AnnotationTypes, std::string> &rule_annotations
+    );
+
     void
     createFileSecuritySections(
         const std::string &asset_id,
@@ -215,10 +233,12 @@ private:
 
     std::map<std::string, LogTriggerSection> log_triggers;
     std::map<std::string, WebUserResponseTriggerSection> web_user_res_triggers;
-    std::map<std::string, InnerException> inner_exceptions;
+    std::map<std::string, std::vector<InnerException>> inner_exceptions;
     std::map<std::string, WebAppSection> web_apps;
     std::map<std::string, RulesConfigRulebase> rules_config;
     std::map<std::string, IpsProtectionsSection> ips;
+    std::map<std::string, SnortProtectionsSection> snort;
+    std::map<std::string, ProtectionsSection> snort_protections;
     std::map<std::string, FileSecurityProtectionsSection> file_security;
     std::map<std::string, RateLimitSection> rate_limit;
     std::map<std::string, UsersIdentifiersRulebase> users_identifiers;

components/security_apps/local_policy_mgmt_gen/k8s_policy_utils.cc

@@ -12,7 +12,7 @@
 // limitations under the License.
 
 #include "k8s_policy_utils.h"
-#include "namespace_data.h"
+#include "configmaps.h"
 
 using namespace std;
 
@@ -184,6 +184,36 @@ getAppSecClassNameFromCluster()
 }
 // LCOV_EXCL_STOP
 
+vector<AppsecException>
+K8sPolicyUtils::extractExceptionsFromCluster(
+    const string &crd_plural,
+    const unordered_set<string> &elements_names) const
+{
+    dbgTrace(D_LOCAL_POLICY) << "Retrieve AppSec elements. type: " << crd_plural;
+    vector<AppsecException> elements;
+    for (const string &element_name : elements_names) {
+        dbgTrace(D_LOCAL_POLICY) << "AppSec element name: " << element_name;
+        auto maybe_appsec_element = getObjectFromCluster<AppsecSpecParser<vector<AppsecExceptionSpec>>>(
+            "/apis/openappsec.io/v1beta1/" + crd_plural + "/" + element_name
+        );
+
+        if (!maybe_appsec_element.ok()) {
+            dbgWarning(D_LOCAL_POLICY)
+                << "Failed to retrieve AppSec element. type: "
+                << crd_plural
+                << ", name: "
+                << element_name
+                << ". Error: "
+                << maybe_appsec_element.getErr();
+            continue;
+        }
+
+        AppsecSpecParser<vector<AppsecExceptionSpec>> appsec_element = maybe_appsec_element.unpack();
+        elements.push_back(AppsecException(element_name, appsec_element.getSpec()));
+    }
+    return elements;
+}
+
 template<class T>
 vector<T>
 K8sPolicyUtils::extractElementsFromCluster(
@@ -292,7 +322,8 @@ K8sPolicyUtils::createAppsecPolicyK8sFromV1beta1Crds(
         policy_elements_names[AnnotationTypes::WEB_USER_RES]
     );
 
-    vector<AppsecExceptionSpec> exceptions = extractElementsFromCluster<AppsecExceptionSpec>(
+
+    vector<AppsecException> exceptions = extractExceptionsFromCluster(
         "exceptions",
         policy_elements_names[AnnotationTypes::EXCEPTION]
     );
@@ -320,6 +351,34 @@ K8sPolicyUtils::createAppsecPolicyK8sFromV1beta1Crds(
 }
 
 // LCOV_EXCL_START Reason: no test exist
+void
+K8sPolicyUtils::createSnortFile(vector<NewAppSecPracticeSpec> &practices) const
+{
+    for (NewAppSecPracticeSpec &practice : practices) {
+        auto orchestration_tools = Singleton::Consume<I_OrchestrationTools>::by<K8sPolicyUtils>();
+        auto path = "/etc/cp/conf/snort/snort_k8s_" + practice.getName() + ".rule";
+        bool append_mode = false;
+        for (const string &config_map : practice.getSnortSignatures().getConfigMap())
+        {
+            auto maybe_configmap = getObjectFromCluster<ConfigMaps>(
+                "/api/v1/namespaces/default/configmaps/" + config_map
+            );
+            if (!maybe_configmap.ok())  {
+                dbgWarning(D_LOCAL_POLICY) << "Failed to get configMaps from the cluster.";
+                continue;
+            }
+            string file_content = maybe_configmap.unpack().getFileContent();
+            string file_name = maybe_configmap.unpack().getFileName();
+            if (!orchestration_tools->writeFile(file_content, path, append_mode)) {
+                dbgWarning(D_LOCAL_POLICY) << "Failed to update the snort_k8s_rules file.";
+                continue;
+            }
+            append_mode = true;
+            practice.getSnortSignatures().addFile(file_name);
+        }
+    }
+}
+
 Maybe<V1beta2AppsecLinuxPolicy>
 K8sPolicyUtils::createAppsecPolicyK8sFromV1beta2Crds(
     const AppsecSpecParser<NewAppsecPolicySpec> &appsec_policy_spec,
@@ -349,6 +408,8 @@ K8sPolicyUtils::createAppsecPolicyK8sFromV1beta2Crds(
             policy_elements_names[AnnotationTypes::THREAT_PREVENTION_PRACTICE]
         );
 
+    createSnortFile(threat_prevention_practices);
+
     vector<AccessControlPracticeSpec> access_control_practices =
         extractV1Beta2ElementsFromCluster<AccessControlPracticeSpec>(
             "accesscontrolpractice",
@@ -404,7 +465,7 @@ doesVersionExist(const map<string, string> &annotations, const string &version)
     }
     return false;
 }
-//need to refactor don't forget that
+
 std::tuple<Maybe<AppsecLinuxPolicy>, Maybe<V1beta2AppsecLinuxPolicy>>
 K8sPolicyUtils::createAppsecPolicyK8s(const string &policy_name, const string &ingress_mode) const
 {
@@ -524,50 +585,3 @@ K8sPolicyUtils::createAppsecPoliciesFromIngresses()
     }
     return make_tuple(v1bet1_policies, v1bet2_policies);
 }
-
-bool
-isPlaygroundEnv()
-{
-    const char *env_string = getenv("PLAYGROUND");
-
-    if (env_string == nullptr) return false;
-    string env_value = env_string;
-    transform(env_value.begin(), env_value.end(), env_value.begin(), ::tolower);
-
-    return env_value == "true";
-}
-
-bool
-K8sPolicyUtils::getClusterId() const
-{
-    string playground_uid = isPlaygroundEnv() ? "playground-" : "";
-
-    dbgTrace(D_LOCAL_POLICY) << "Getting cluster UID";
-    auto maybe_namespaces_data = getObjectFromCluster<NamespaceData>("/api/v1/namespaces/");
-
-    if (!maybe_namespaces_data.ok()) {
-        dbgWarning(D_LOCAL_POLICY)
-            << "Failed to retrieve K8S namespace data. Error: "
-            << maybe_namespaces_data.getErr();
-        return false;
-    }
-
-    NamespaceData namespaces_data = maybe_namespaces_data.unpack();
-
-    Maybe<string> maybe_ns_uid = namespaces_data.getNamespaceUidByName("kube-system");
-    if (!maybe_ns_uid.ok()) {
-        dbgWarning(D_LOCAL_POLICY) << maybe_ns_uid.getErr();
-        return false;
-    }
-    string uid = playground_uid + maybe_ns_uid.unpack();
-    dbgTrace(D_LOCAL_POLICY) << "Found k8s cluster UID: " << uid;
-    I_Environment *env = Singleton::Consume<I_Environment>::by<K8sPolicyUtils>();
-    env->getConfigurationContext().registerValue<string>(
-        "k8sClusterId",
-        uid,
-        EnvKeyAttr::LogSection::SOURCE
-    );
-    I_AgentDetails *i_agent_details = Singleton::Consume<I_AgentDetails>::by<K8sPolicyUtils>();
-    i_agent_details->setClusterId(uid);
-    return true;
-}

components/security_apps/local_policy_mgmt_gen/local_policy_mgmt_gen.cc

@@ -43,7 +43,7 @@
 #include "include/rules_config_section.h"
 #include "include/trusted_sources_section.h"
 #include "include/policy_maker_utils.h"
-#include "include/k8s_policy_utils.h"
+#include "k8s_policy_utils.h"
 #include "i_env_details.h"
 
 using namespace std;
@@ -64,31 +64,10 @@ public:
     void
     init()
     {
-        env_details = Singleton::Consume<I_EnvDetails>::by<LocalPolicyMgmtGenerator::Impl>();
-        env_type = env_details->getEnvType();
-        if (env_type == EnvType::LINUX) {
-            dbgInfo(D_LOCAL_POLICY) << "Initializing Linux policy generator";
-            local_policy_path = getFilesystemPathConfig() + default_local_mgmt_policy_path;
-            return;
-        }
-        dbgInfo(D_LOCAL_POLICY) << "Initializing K8S policy generator";
-        k8s_policy_utils.init();
-
-        Singleton::Consume<I_MainLoop>::by<LocalPolicyMgmtGenerator::Impl>()->addOneTimeRoutine(
-            I_MainLoop::RoutineType::Offline,
-            [this] ()
-            {
-                while(!k8s_policy_utils.getClusterId()) {
-                    Singleton::Consume<I_MainLoop>::by<LocalPolicyMgmtGenerator::Impl>()->yield(chrono::seconds(1));
-                }
-                return;
-            },
-            "Get k8s cluster ID"
-        );
     }
 
     string
-    parseLinuxPolicy(const string &policy_version)
+    parseLinuxPolicy(const string &policy_version, const string &local_policy_path)
     {
         dbgFlow(D_LOCAL_POLICY) << "Starting to parse policy - embedded environment";
 
@@ -104,6 +83,10 @@ public:
     {
         dbgFlow(D_LOCAL_POLICY) << "Starting to parse policy - K8S environment";
 
+        dbgInfo(D_LOCAL_POLICY) << "Initializing K8S policy generator";
+        K8sPolicyUtils k8s_policy_utils;
+        k8s_policy_utils.init();
+
         auto appsec_policies = k8s_policy_utils.createAppsecPoliciesFromIngresses();
         if (!std::get<0>(appsec_policies).empty()) {
             return policy_maker_utils.proccesMultipleAppsecPolicies<AppsecLinuxPolicy, ParsedRule>(
@@ -120,27 +103,14 @@ public:
     }
 
     string
-    parsePolicy(const string &policy_version)
+    generateAppSecLocalPolicy(EnvType env_type, const string &policy_version, const string &local_policy_paths)
     {
-        return isK8sEnv() ? parseK8sPolicy(policy_version) : parseLinuxPolicy(policy_version);
+        return env_type == EnvType::K8S ?
+            parseK8sPolicy(policy_version) : parseLinuxPolicy(policy_version, local_policy_paths);
     }
 
-    const string & getAgentPolicyPath(void) const override { return default_local_appsec_policy_path; }
-    const string & getLocalPolicyPath(void) const override { return local_policy_path; }
-    void setPolicyPath(const string &new_local_policy_path) override { local_policy_path = new_local_policy_path; }
-
 private:
-    bool
-    isK8sEnv()
-    {
-        return env_type == EnvType::K8S;
-    }
-
-    I_EnvDetails* env_details = nullptr;
-    EnvType env_type;
     PolicyMakerUtils policy_maker_utils;
-    K8sPolicyUtils k8s_policy_utils;
-    string local_policy_path;
 
 };
 

components/security_apps/local_policy_mgmt_gen/new_appsec_linux_policy.cc

@@ -47,7 +47,7 @@ V1beta2AppsecLinuxPolicy::getAppSecCustomResponseSpecs() const
 }
 
 const vector<NewAppsecException> &
-V1beta2AppsecLinuxPolicy::getAppsecExceptionSpecs() const
+V1beta2AppsecLinuxPolicy::getAppsecExceptions() const
 {
     return exceptions;
 }

components/security_apps/local_policy_mgmt_gen/new_practice.cc

@@ -217,23 +217,253 @@ NewAppSecPracticeWebAttacks::getMode(const string &default_mode) const
     return key_to_practices_val.at(mode);
 }
 
+SnortProtectionsSection::SnortProtectionsSection(
+    const std::string               &_context,
+    const std::string               &_asset_name,
+    const std::string               &_asset_id,
+    const std::string               &_practice_name,
+    const std::string               &_practice_id,
+    const std::string               &_source_identifier,
+    const std::string               &_mode,
+    const std::vector<std::string>  &_files)
+        :
+    context(_context),
+    asset_name(_asset_name),
+    asset_id(_asset_id),
+    practice_name(_practice_name),
+    practice_id(_practice_id),
+    source_identifier(_source_identifier),
+    mode(_mode),
+    files(_files)
+{
+}
+
+void
+SnortProtectionsSection::save(cereal::JSONOutputArchive &out_ar) const
+{
+    out_ar(
+        cereal::make_nvp("context",                         context),
+        cereal::make_nvp("mode",                            key_to_mode_val.at(mode)),
+        cereal::make_nvp("files",                           files),
+        cereal::make_nvp("assetName",                       asset_name),
+        cereal::make_nvp("assetId",                         asset_id),
+        cereal::make_nvp("practiceName",                    practice_name),
+        cereal::make_nvp("practiceId",                      practice_id),
+        cereal::make_nvp("sourceIdentifier",                source_identifier)
+    );
+}
+
+DetectionRules::DetectionRules(
+    const std::string                   &_type,
+    const std::string                   &_SSM,
+    const std::string                   &_keywords,
+    const std::vector<std::string>      &_context)
+        :
+    type(_type),
+    SSM(_SSM),
+    keywords(_keywords),
+    context(_context)
+{
+}
+
+void
+DetectionRules::load(cereal::JSONInputArchive &archive_in)
+{
+    dbgTrace(D_LOCAL_POLICY) << "Loading Snort protections protections detection rules section";
+    parseAppsecJSONKey<string>("type", type, archive_in);
+    parseAppsecJSONKey<string>("SSM", SSM, archive_in);
+    parseAppsecJSONKey<string>("keywords", keywords, archive_in);
+    parseAppsecJSONKey<vector<string>>("context", context, archive_in);
+
+}
+
+void
+DetectionRules::save(cereal::JSONOutputArchive &out_ar) const
+{
+    out_ar(
+        cereal::make_nvp("type",        type),
+        cereal::make_nvp("SSM",         SSM),
+        cereal::make_nvp("keywords",    keywords),
+        cereal::make_nvp("context",     context)
+    );
+}
+
+ProtectionMetadata::ProtectionMetadata(
+    bool                                _silent,
+    const std::string                   &_protection_name,
+    const std::string                   &_severity,
+    const std::string                   &_confidence_level,
+    const std::string                   &_performance_impact,
+    const std::string                   &_last_update,
+    const std::string                   &_maintrain_id,
+    const std::vector<std::string>      &_tags,
+    const std::vector<std::string>      &_cve_list)
+        :
+    silent(_silent),
+    protection_name(_protection_name),
+    severity(_severity),
+    confidence_level(_confidence_level),
+    performance_impact(_performance_impact),
+    last_update(_last_update),
+    maintrain_id(_maintrain_id),
+    tags(_tags),
+    cve_list(_cve_list)
+{
+}
+
+void
+ProtectionMetadata::load(cereal::JSONInputArchive &archive_in)
+{
+    dbgTrace(D_LOCAL_POLICY) << "Loading Snort protections protections metadata section";
+    parseAppsecJSONKey<bool>("silent", silent, archive_in);
+    parseAppsecJSONKey<string>("protectionName", protection_name, archive_in);
+    parseAppsecJSONKey<string>("severity", severity, archive_in);
+    parseAppsecJSONKey<string>("confidenceLevel", confidence_level, archive_in);
+    parseAppsecJSONKey<string>("performanceImpact", performance_impact, archive_in);
+    parseAppsecJSONKey<string>("lastUpdate", last_update, archive_in);
+    parseAppsecJSONKey<string>("maintrainId", maintrain_id, archive_in);
+    parseAppsecJSONKey<vector<string>>("tags", tags, archive_in);
+    parseAppsecJSONKey<vector<string>>("cveList", cve_list, archive_in);
+
+}
+
+void
+ProtectionMetadata::save(cereal::JSONOutputArchive &out_ar) const
+{
+    out_ar(
+        cereal::make_nvp("protectionName",      protection_name),
+        cereal::make_nvp("severity",            severity),
+        cereal::make_nvp("confidenceLevel",     confidence_level),
+        cereal::make_nvp("performanceImpact",   performance_impact),
+        cereal::make_nvp("lastUpdate",          last_update),
+        cereal::make_nvp("maintrainId",         maintrain_id),
+        cereal::make_nvp("tags",                tags),
+        cereal::make_nvp("cveList",             cve_list),
+        cereal::make_nvp("silent",              silent)
+    );
+}
+
+ProtectionsProtectionsSection::ProtectionsProtectionsSection(
+    const ProtectionMetadata    &_protection_metadata,
+    const DetectionRules        &_detection_rules)
+        :
+    protection_metadata(_protection_metadata),
+    detection_rules(_detection_rules)
+{
+}
+
+void
+ProtectionsProtectionsSection::load(cereal::JSONInputArchive &archive_in)
+{
+    dbgTrace(D_LOCAL_POLICY) << "Loading Snort protections protections section";
+    parseAppsecJSONKey<ProtectionMetadata>("protectionMetadata", protection_metadata, archive_in);
+    parseAppsecJSONKey<DetectionRules>("detectionRules", detection_rules, archive_in);
+}
+
+void
+ProtectionsProtectionsSection::save(cereal::JSONOutputArchive &out_ar) const
+{
+    out_ar(
+        cereal::make_nvp("protectionMetadata",      protection_metadata),
+        cereal::make_nvp("detectionRules",          detection_rules)
+    );
+}
+
+ProtectionsSection::ProtectionsSection(
+    const std::vector<ProtectionsProtectionsSection>    &_protections,
+    const std::string                                   &_name,
+    const std::string                                   &_modification_time)
+        :
+    protections(_protections),
+    name(_name),
+    modification_time(_modification_time)
+{
+}
+
+void
+ProtectionsSection::load(cereal::JSONInputArchive &archive_in)
+{
+    dbgTrace(D_LOCAL_POLICY) << "Loading Snort protections section";
+    parseAppsecJSONKey<vector<ProtectionsProtectionsSection>>("protections", protections, archive_in);
+}
+
+void
+ProtectionsSection::save(cereal::JSONOutputArchive &out_ar) const
+{
+    out_ar(
+        cereal::make_nvp("name",                name),
+        cereal::make_nvp("modificationTime",    modification_time),
+        cereal::make_nvp("protections",         protections)
+    );
+}
+
+const vector<ProtectionsProtectionsSection> &
+ProtectionsSection::getProtections() const
+{
+    return protections;
+}
+
+void
+ProtectionsSectionWrapper::serialize(cereal::JSONInputArchive &archive_in)
+{
+    dbgTrace(D_LOCAL_POLICY) << "Loading Snort Section";
+    parseAppsecJSONKey<ProtectionsSection>("IPSSnortSigs", protections, archive_in);
+}
+
+const vector<ProtectionsProtectionsSection> &
+ProtectionsSectionWrapper::getProtections() const
+{
+    return protections.getProtections();
+}
+
+void
+SnortSection::save(cereal::JSONOutputArchive &out_ar) const
+{
+    string version = "LocalVersion";
+    out_ar(
+        cereal::make_nvp("VersionId", version),
+        cereal::make_nvp("SnortProtections", snort_protections),
+        cereal::make_nvp("protections", protections)
+    );
+}
+
+void
+SnortSectionWrapper::save(cereal::JSONOutputArchive &out_ar) const
+{
+    out_ar(
+        cereal::make_nvp("IPSSnortSigs", snort)
+    );
+}
+
 void
 NewSnortSignaturesAndOpenSchemaAPI::load(cereal::JSONInputArchive &archive_in)
 {
     dbgTrace(D_LOCAL_POLICY) << "Loading AppSec Snort Signatures practice";
-    parseAppsecJSONKey<string>("overrideMode", override_mode, archive_in, "Inactive");
+    parseAppsecJSONKey<string>("overrideMode", override_mode, archive_in, "inactive");
     parseAppsecJSONKey<vector<string>>("configmap", config_map, archive_in);
     if (valid_modes.count(override_mode) == 0) {
         dbgWarning(D_LOCAL_POLICY) << "AppSec Snort Signatures override mode invalid: " << override_mode;
     }
 }
 
+void
+NewSnortSignaturesAndOpenSchemaAPI::addFile(const string &file_name)
+{
+    files.push_back(file_name);
+}
+
 const string &
 NewSnortSignaturesAndOpenSchemaAPI::getOverrideMode() const
 {
     return override_mode;
 }
 
+const vector<string> &
+NewSnortSignaturesAndOpenSchemaAPI::getFiles() const
+{
+    return files;
+}
+
 const vector<string> &
 NewSnortSignaturesAndOpenSchemaAPI::getConfigMap() const
 {
@@ -320,7 +550,7 @@ void
 NewIntrusionPrevention::load(cereal::JSONInputArchive &archive_in)
 {
     dbgTrace(D_LOCAL_POLICY) << "Loading AppSec Intrusion Prevention practice";
-    parseAppsecJSONKey<string>("overrideMode", override_mode, archive_in, "Inactive");
+    parseAppsecJSONKey<string>("overrideMode", override_mode, archive_in, "inactive");
     if (valid_modes.count(override_mode) == 0) {
         dbgWarning(D_LOCAL_POLICY) << "AppSec Intrusion Prevention override mode invalid: " << override_mode;
     }
@@ -596,7 +826,7 @@ void
 NewFileSecurity::load(cereal::JSONInputArchive &archive_in)
 {
     dbgTrace(D_LOCAL_POLICY) << "Loading AppSec File Security practice";
-    parseAppsecJSONKey<string>("overrideMode", override_mode, archive_in, "Inactive");
+    parseAppsecJSONKey<string>("overrideMode", override_mode, archive_in, "inactive");
     if (valid_modes.count(override_mode) == 0) {
         dbgWarning(D_LOCAL_POLICY) << "AppSec File Security override mode invalid: " << override_mode;
     }
@@ -633,6 +863,11 @@ NewFileSecurity::load(cereal::JSONInputArchive &archive_in)
     parseAppsecJSONKey<NewFileSecurityLargeFileInspection>("largeFileInspection", large_file_inspection, archive_in);
 }
 
+const string &
+NewFileSecurity::getOverrideMode() const
+{
+    return override_mode;
+}
 
 const NewFileSecurityArchiveInspection &
 NewFileSecurity::getArchiveInspection() const
@@ -707,8 +942,8 @@ NewAppSecPracticeSpec::getOpenSchemaValidation() const
     return openapi_schema_validation;
 }
 
-const NewSnortSignaturesAndOpenSchemaAPI &
-NewAppSecPracticeSpec::getSnortSignatures() const
+NewSnortSignaturesAndOpenSchemaAPI &
+NewAppSecPracticeSpec::getSnortSignatures()
 {
     return snort_signatures;
 }

components/security_apps/local_policy_mgmt_gen/policy_maker_utils.cc

@@ -27,6 +27,7 @@ SecurityAppsWrapper::save(cereal::JSONOutputArchive &out_ar) const
         cereal::make_nvp("rules",               rules),
         cereal::make_nvp("ips",                 ips),
         cereal::make_nvp("exceptions",          exceptions),
+        cereal::make_nvp("snort",               snort),
         cereal::make_nvp("fileSecurity",        file_security),
         cereal::make_nvp("version",             policy_version)
     );
@@ -53,29 +54,30 @@ PolicyMakerUtils::getPolicyName(const string &policy_path)
     return policy_name;
 }
 
-Maybe<AppsecLinuxPolicy>
-PolicyMakerUtils::openPolicyAsJson(const string &policy_path)
+template<class T>
+Maybe<T>
+PolicyMakerUtils::openFileAsJson(const string &path)
 {
-    auto maybe_policy_as_json = Singleton::Consume<I_ShellCmd>::by<PolicyMakerUtils>()->getExecOutput(
-        getFilesystemPathConfig() + "/bin/yq " + policy_path + " -o json"
+    auto maybe_file_as_json = Singleton::Consume<I_ShellCmd>::by<PolicyMakerUtils>()->getExecOutput(
+        getFilesystemPathConfig() + "/bin/yq " + path + " -o json"
     );
 
-    if (!maybe_policy_as_json.ok()) {
+    if (!maybe_file_as_json.ok()) {
         dbgDebug(D_NGINX_POLICY) << "Could not convert policy from yaml to json";
-        return genError("Could not convert policy from yaml to json. Error: " + maybe_policy_as_json.getErr());
+        return genError("Could not convert policy from yaml to json. Error: " + maybe_file_as_json.getErr());
     }
 
     auto i_orchestration_tools = Singleton::Consume<I_OrchestrationTools>::by<PolicyMakerUtils>();
-    auto maybe_policy = i_orchestration_tools->jsonStringToObject<AppsecLinuxPolicy>(
-        maybe_policy_as_json.unpack()
+    auto maybe_file = i_orchestration_tools->jsonStringToObject<T>(
+        maybe_file_as_json.unpack()
     );
 
-    if (!maybe_policy.ok()) {
-        string error = "Policy in path: " + policy_path + " was not loaded. Error: " + maybe_policy.getErr();
+    if (!maybe_file.ok()) {
+        string error = "Policy in path: " + path + " was not loaded. Error: " + maybe_file.getErr();
         dbgDebug(D_NGINX_POLICY) << error;
         return  genError(error);
     }
-    return maybe_policy.unpack();
+    return maybe_file.unpack();
 }
 
 void
@@ -86,6 +88,11 @@ PolicyMakerUtils::clearElementsMaps()
     inner_exceptions.clear();
     web_apps.clear();
     rules_config.clear();
+    ips.clear();
+    snort.clear();
+    snort_protections.clear();
+    file_security.clear();
+    rate_limit.clear();
 }
 
 // LCOV_EXCL_START Reason: no test exist - needed for NGINX config
@@ -351,6 +358,19 @@ convertMapToVector(map<K, V> map)
     return vec;
 }
 
+vector<InnerException>
+convertExceptionsMapToVector(map<string, vector<InnerException>> map)
+{
+    vector<InnerException> vec;
+    if (map.empty()) {
+        return vec;
+    }
+    for (const auto &m : map) {
+        if (!m.first.empty()) vec.insert(vec.end(), m.second.begin(), m.second.end());
+    }
+    return vec;
+}
+
 template<class T, class R>
 R
 getAppsecPracticeSpec(const string &practice_annotation_name, const T &policy)
@@ -404,7 +424,7 @@ template<class T, class R>
 R
 getAppsecExceptionSpec(const string &exception_annotation_name, const T &policy)
 {
-    auto exceptions_vec = policy.getAppsecExceptionSpecs();
+    auto exceptions_vec = policy.getAppsecExceptions();
     auto exception_it = extractElement(exceptions_vec.begin(), exceptions_vec.end(), exception_annotation_name);
 
     if (exception_it == exceptions_vec.end()) {
@@ -710,26 +730,24 @@ createTrustedSourcesSection<V1beta2AppsecLinuxPolicy>(
 }
 
 template<class T>
-InnerException
+vector<InnerException>
 createExceptionSection(
     const string &exception_annotation_name,
     const T &policy)
 {
-    AppsecExceptionSpec exception_spec =
-        getAppsecExceptionSpec<T, AppsecExceptionSpec>(exception_annotation_name, policy);
-    ExceptionMatch exception_match(exception_spec);
-    string behavior =
-        exception_spec.getAction() == "skip" ?
-        "ignore" :
-        exception_spec.getAction();
-
-    ExceptionBehavior exception_behavior("action", behavior);
-    InnerException inner_exception(exception_behavior, exception_match);
-    return inner_exception;
+    AppsecException exception_spec =
+        getAppsecExceptionSpec<T, AppsecException>(exception_annotation_name, policy);
+    vector<InnerException> res;
+    for (auto exception : exception_spec.getExceptions()) {
+        ExceptionMatch exception_match(exception);
+        ExceptionBehavior exception_behavior(exception.getAction());
+        res.push_back(InnerException(exception_behavior, exception_match));
+    }
+    return res;
 }
 
 template<>
-InnerException
+vector<InnerException>
 createExceptionSection<V1beta2AppsecLinuxPolicy>(
     const string &exception_annotation_name,
     const V1beta2AppsecLinuxPolicy &policy)
@@ -737,14 +755,9 @@ createExceptionSection<V1beta2AppsecLinuxPolicy>(
     NewAppsecException exception_spec =
         getAppsecExceptionSpec<V1beta2AppsecLinuxPolicy, NewAppsecException>(exception_annotation_name, policy);
     ExceptionMatch exception_match(exception_spec);
-    string behavior =
-        exception_spec.getAction() == "skip" ?
-        "ignore" :
-        exception_spec.getAction();
-
-    ExceptionBehavior exception_behavior("action", behavior);
+    ExceptionBehavior exception_behavior(exception_spec.getAction());
     InnerException inner_exception(exception_behavior, exception_match);
-    return inner_exception;
+    return {inner_exception};
 }
 
 template<class T>
@@ -842,10 +855,13 @@ createMultiRulesSections(
     const string &web_user_res_vec_type,
     const string &asset_name,
     const string &exception_name,
-    const string &exception_id)
+    const vector<InnerException> &exceptions)
 {
     PracticeSection practice = PracticeSection(practice_id, practice_type, practice_name);
-    ParametersSection exception_param = ParametersSection(exception_id, exception_name);
+    vector<ParametersSection> exceptions_result;
+    for (auto exception : exceptions) {
+        exceptions_result.push_back(ParametersSection(exception.getBehaviorId(), exception_name));
+    }
 
     vector<RulesTriggerSection> triggers;
     if (!log_trigger_id.empty()) {
@@ -864,7 +880,7 @@ createMultiRulesSections(
         url,
         uri,
         {practice},
-        {exception_param},
+        exceptions_result,
         triggers
     );
 
@@ -889,9 +905,9 @@ createMultiRulesSections(
     const string &web_user_res_vec_type,
     const string &asset_name,
     const string &exception_name,
-    const string &exception_id)
+    const vector<InnerException> &exceptions)
 {
-    ParametersSection exception_param = ParametersSection(exception_id, exception_name);
+    ParametersSection exception_param = ParametersSection(exceptions[0].getBehaviorId(), exception_name);
 
     vector<PracticeSection> practices;
     if (!practice_id.empty()) {
@@ -941,6 +957,9 @@ PolicyMakerUtils::createIpsSections(
     auto apssec_practice = getAppsecPracticeSpec<V1beta2AppsecLinuxPolicy, NewAppSecPracticeSpec>(
         rule_annotations[AnnotationTypes::PRACTICE],
         policy);
+
+    if (apssec_practice.getIntrusionPrevention().getMode().empty()) return;
+
     IpsProtectionsSection ips_section = IpsProtectionsSection(
         context,
         asset_name,
@@ -955,6 +974,74 @@ PolicyMakerUtils::createIpsSections(
     ips[asset_name] = ips_section;
 }
 
+void
+PolicyMakerUtils::createSnortProtecionsSection(const string &file_name, const string &practice_name)
+{
+    auto path = getFilesystemPathConfig() + "/conf/snort/snort_k8s_" + practice_name;
+    if (snort_protections.find(path) != snort_protections.end()) return;
+
+    auto snort_scriipt_path = getFilesystemPathConfig() + "/scripts/snort_to_ips_local.py";
+    auto cmd = "python " + snort_scriipt_path + " " + path + ".rule " + path + ".out " + path + ".err";
+
+    auto res = Singleton::Consume<I_ShellCmd>::by<PolicyMakerUtils>()->getExecOutput(cmd);
+
+    if (!res.ok()) {
+        dbgWarning(D_LOCAL_POLICY) << res.getErr();
+        return;
+    }
+
+    Maybe<ProtectionsSectionWrapper> maybe_protections = openFileAsJson<ProtectionsSectionWrapper>(path + ".out");
+    if (!maybe_protections.ok()){
+        dbgWarning(D_LOCAL_POLICY) << maybe_protections.getErr();
+        return;
+    }
+
+    auto i_orchestration_tools = Singleton::Consume<I_OrchestrationTools>::by<PolicyMakerUtils>();
+    i_orchestration_tools->removeFile(path + ".rule");
+    i_orchestration_tools->removeFile(path + ".out");
+    i_orchestration_tools->removeFile(path + ".err");
+
+    snort_protections[path] = ProtectionsSection(
+        maybe_protections.unpack().getProtections(),
+        file_name
+    );
+}
+
+void
+PolicyMakerUtils::createSnortSections(
+    const string & context,
+    const string &asset_name,
+    const string &asset_id,
+    const string &practice_name,
+    const string &practice_id,
+    const string &source_identifier,
+    const V1beta2AppsecLinuxPolicy &policy,
+    map<AnnotationTypes, string> &rule_annotations)
+{
+    auto apssec_practice = getAppsecPracticeSpec<V1beta2AppsecLinuxPolicy, NewAppSecPracticeSpec>(
+        rule_annotations[AnnotationTypes::PRACTICE],
+        policy);
+
+    if (apssec_practice.getSnortSignatures().getOverrideMode() == "inactive" ||
+        apssec_practice.getSnortSignatures().getFiles().size() == 0) {
+        return;
+    }
+    createSnortProtecionsSection(apssec_practice.getSnortSignatures().getFiles()[0], apssec_practice.getName());
+
+    SnortProtectionsSection snort_section = SnortProtectionsSection(
+        context,
+        asset_name,
+        asset_id,
+        practice_name,
+        practice_id,
+        source_identifier,
+        apssec_practice.getSnortSignatures().getOverrideMode(),
+        apssec_practice.getSnortSignatures().getFiles()
+    );
+
+    snort[asset_name] = snort_section;
+}
+
 void
 PolicyMakerUtils::createFileSecuritySections(
     const string &asset_id,
@@ -968,6 +1055,9 @@ PolicyMakerUtils::createFileSecuritySections(
     auto apssec_practice = getAppsecPracticeSpec<V1beta2AppsecLinuxPolicy, NewAppSecPracticeSpec>(
         rule_annotations[AnnotationTypes::PRACTICE],
         policy);
+
+    if (apssec_practice.getFileSecurity().getOverrideMode().empty()) return;
+
     auto file_security_section = apssec_practice.getFileSecurity().createFileSecurityProtectionsSection(
         context,
         asset_name,
@@ -1095,7 +1185,7 @@ PolicyMakerUtils::createThreatPreventionPracticeSections(
         "WebUserResponse",
         asset_name,
         rule_annotations[AnnotationTypes::EXCEPTION],
-        inner_exceptions[rule_annotations[AnnotationTypes::EXCEPTION]].getBehaviorId()
+        inner_exceptions[rule_annotations[AnnotationTypes::EXCEPTION]]
     );
     rules_config[rule_config.getAssetName()] = rule_config;
 
@@ -1121,6 +1211,17 @@ PolicyMakerUtils::createThreatPreventionPracticeSections(
         rule_annotations
     );
 
+    createSnortSections(
+        "practiceId(" + practice_id + ")",
+        rule_config.getAssetName(),
+        rule_config.getAssetId(),
+        rule_annotations[AnnotationTypes::PRACTICE],
+        practice_id,
+        current_identifier,
+        policy,
+        rule_annotations
+    );
+
     createFileSecuritySections(
         rule_config.getAssetId(),
         rule_config.getAssetName(),
@@ -1158,12 +1259,13 @@ PolicyMakerUtils::combineElementsToPolicy(const string &policy_version)
         )
     );
     ExceptionsWrapper exceptions_section({
-        ExceptionsRulebase(convertMapToVector(inner_exceptions))
+        ExceptionsRulebase(convertExceptionsMapToVector(inner_exceptions))
     });
 
     AppSecWrapper appses_section(AppSecRulebase(convertMapToVector(web_apps), {}));
     RulesConfigWrapper rules_config_section(convertMapToVector(rules_config), convertMapToVector(users_identifiers));
     IntrusionPreventionWrapper ips_section(convertMapToVector(ips));
+    SnortSectionWrapper snort_section(convertMapToVector(snort), convertMapToVector(snort_protections));
     FileSecurityWrapper file_security_section(convertMapToVector(file_security));
     AccessControlRulebaseWrapper rate_limit_section(convertMapToVector(rate_limit));
     SecurityAppsWrapper security_app_section = SecurityAppsWrapper(
@@ -1171,6 +1273,7 @@ PolicyMakerUtils::combineElementsToPolicy(const string &policy_version)
         triggers_section,
         rules_config_section,
         ips_section,
+        snort_section,
         rate_limit_section,
         file_security_section,
         exceptions_section,
@@ -1277,7 +1380,7 @@ PolicyMakerUtils::createPolicyElementsByRule(
             "WebUserResponse",
             full_url,
             rule_annotations[AnnotationTypes::EXCEPTION],
-            inner_exceptions[rule_annotations[AnnotationTypes::EXCEPTION]].getBehaviorId()
+            inner_exceptions[rule_annotations[AnnotationTypes::EXCEPTION]]
         );
         rules_config[rule_config.getAssetName()] = rule_config;
 
@@ -1303,7 +1406,8 @@ PolicyMakerUtils::createPolicyElementsByRule(
                 getAppsecPracticeSpec<T, AppSecPracticeSpec>(rule_annotations[AnnotationTypes::PRACTICE], policy),
                 log_triggers[rule_annotations[AnnotationTypes::TRIGGER]],
                 rule.getMode(),
-                trusted_sources[rule_annotations[AnnotationTypes::TRUSTED_SOURCES]]
+                trusted_sources[rule_annotations[AnnotationTypes::TRUSTED_SOURCES]],
+                inner_exceptions[rule_annotations[AnnotationTypes::EXCEPTION]]
             );
             web_apps[rule_config.getAssetName()] = web_app;
         }
@@ -1468,7 +1572,7 @@ PolicyMakerUtils::proccesSingleAppsecPolicy(
     const string &policy_version,
     const string &local_appsec_policy_path)
 {
-    Maybe<AppsecLinuxPolicy> maybe_policy = openPolicyAsJson(policy_path);
+    Maybe<AppsecLinuxPolicy> maybe_policy = openFileAsJson<AppsecLinuxPolicy>(policy_path);
     if (!maybe_policy.ok()){
         dbgWarning(D_LOCAL_POLICY) << maybe_policy.getErr();
         return "";

components/security_apps/local_policy_mgmt_gen/rules_config_section.cc

@@ -98,6 +98,7 @@ PracticeSection::save(cereal::JSONOutputArchive &out_ar) const
     );
 }
 
+// LCOV_EXCL_START Reason: no test exist
 ParametersSection::ParametersSection(
     const string &_id,
     const string &_name)
@@ -120,6 +121,7 @@ ParametersSection::save(cereal::JSONOutputArchive &out_ar) const
         cereal::make_nvp("parameterType", type)
     );
 }
+// LCOV_EXCL_STOP
 
 RulesTriggerSection::RulesTriggerSection(
     const string &_name,

components/security_apps/orchestration/CMakeLists.txt

@@ -12,7 +12,6 @@ add_subdirectory(manifest_controller)
 add_subdirectory(update_communication)
 add_subdirectory(details_resolver)
 add_subdirectory(health_check)
-add_subdirectory(local_policy_mgmt_gen)
 add_subdirectory(env_details)
 
 #add_subdirectory(orchestration_ut)

components/security_apps/orchestration/details_resolver/details_resolver_handlers/checkpoint_product_handlers.h

@@ -42,6 +42,16 @@ checkSamlPortal(const string &command_output)
     return genError("Current host does not have SAML Portal configured");
 }
 
+Maybe<string>
+getIDAGaia(const string &command_output)
+{
+    if (command_output.find("Portal is running") != string::npos) {
+        return string("ida_gaia");
+    }
+
+    return genError("Current host does not have SAML Portal configured");
+}
+
 Maybe<string>
 checkIDP(shared_ptr<istream> file_stream)
 {
@@ -226,58 +236,24 @@ getSmbGWIPSecVPNBlade(const string &command_output)
 {
     return getSmbBlade(command_output, "IPSec VPN Blade was not found");
 }
-
-Maybe<string>
-getMgmtParentObjAttr(shared_ptr<istream> file_stream, const string &parent_obj, const string &attr)
-{
-    string line;
-    bool found_parent_obj = false;
-    while (getline(*file_stream, line)) {
-        size_t parent_obj_pos = line.find(parent_obj);
-        if (parent_obj_pos != string::npos) found_parent_obj = true;
-        if (!found_parent_obj) continue;
-
-        size_t attr_pos = line.find(attr);
-        if (attr_pos == string::npos) continue;
-        line = line.substr(attr_pos + attr.size());
-        return line;
-    }
-    return genError("Parent object attribute was not found. Attr: " + attr);
-}
 #endif // gaia || smb
 
 #if defined(gaia)
 Maybe<string>
-getMgmtParentObjUid(shared_ptr<istream> file_stream)
+getMgmtParentObjUid(const string &command_output)
 {
-    auto maybe_unparsed_uid = getMgmtParentObjAttr(file_stream, "cluster_object", "Uid ");
-    if (!maybe_unparsed_uid.ok()) {
-        return maybe_unparsed_uid;
-    }
-    const string &unparsed_uid = maybe_unparsed_uid.unpack();
-    auto maybe_uid = chopHeadAndTail(unparsed_uid, "(\"{", "}\")");
-    if (!maybe_uid.ok()) {
-        return maybe_uid;
-    }
-    string uid = maybe_uid.unpack();
-    transform(uid.begin(), uid.end(), uid.begin(), ::tolower);
-    return uid;
+    return getAttr(command_output, "Parent object uuid was not found");
 }
 
 Maybe<string>
-getMgmtParentObjName(shared_ptr<istream> file_stream)
+getMgmtParentObjName(const string &command_output)
 {
-    auto maybe_unparsed_name = getMgmtParentObjAttr(file_stream, "cluster_object", "Name ");
-    if (!maybe_unparsed_name.ok()) {
-        return maybe_unparsed_name;
-    }
-    const string &unparsed_name = maybe_unparsed_name.unpack();
-    return chopHeadAndTail(unparsed_name, "(", ")");
+    return getAttr(command_output, "Parent object name was not found");
 }
 
 #elif defined(smb)
 Maybe<string>
-getMgmtParentObjUid(const string &command_output)
+getSmbMgmtParentObjUid(const string &command_output)
 {
     if (!command_output.empty()) {
         return command_output;
@@ -286,7 +262,7 @@ getMgmtParentObjUid(const string &command_output)
 }
 
 Maybe<string>
-getMgmtParentObjName(const string &command_output)
+getSmbMgmtParentObjName(const string &command_output)
 {
     if (!command_output.empty()) {
         return command_output;
@@ -314,6 +290,34 @@ getOsRelease(shared_ptr<istream> file_stream)
     return genError("Os release was not found");
 }
 
+Maybe<string>
+getWaapModelVersion(shared_ptr<istream> file_stream)
+{
+    string line;
+    static const int max_lines = 5;
+    int i = 0;
+    bool found_key = false;
+    while (i < max_lines && getline(*file_stream, line)) {
+        if (!found_key) {
+            size_t index = line.find("\"model_version\":");
+            if (index != string::npos) {
+                found_key = true;
+            }
+        } else {
+            size_t start = line.find_first_of('"');
+            size_t end = line.find_last_of('"');
+            if (start != string::npos && end != string::npos && end > start) {
+                return line.substr(start + 1, end - start - 1);
+            } else {
+                return genError("Model version value unreadable");
+            }
+        }
+        i++;
+    }
+
+    return genError("Model version was not found");
+}
+
 #if defined(alpine)
 string &
 ltrim(string &s)

components/security_apps/orchestration/details_resolver/details_resolver_handlers/details_resolver_impl.h

@@ -55,6 +55,19 @@ SHELL_CMD_HANDLER(
 #if defined(gaia)
 SHELL_CMD_HANDLER("hasSupportedBlade", "enabled_blades", checkHasSupportedBlade)
 SHELL_CMD_HANDLER("hasSamlPortal", "mpclient status saml-vpn", checkSamlPortal)
+SHELL_CMD_HANDLER("requiredNanoServices", "mpclient status saml-vpn", getIDAGaia)
+SHELL_CMD_HANDLER(
+    "cpProductIntegrationMgmtParentObjectName",
+    "cat $FWDIR/database/myself_objects.C "
+    "| awk -F '[:()]' '/:cluster_object/ {found=1; next} found && /:Name/ {print $3; exit}'",
+    getMgmtParentObjName
+)
+SHELL_CMD_HANDLER(
+    "cpProductIntegrationMgmtParentObjectUid",
+    "cat $FWDIR/database/myself_objects.C "
+    "| awk -F'[{}]' '/:cluster_object/ { found=1; next } found && /:Uid/ { uid=tolower($2); print uid; exit }'",
+    getMgmtParentObjUid
+)
 SHELL_CMD_HANDLER(
     "Hardware",
     "cat $FWDIR/database/myself_objects.C | awk -F '[:()]' '/:appliance_type/ {print $3}' | head -n 1",
@@ -81,12 +94,12 @@ SHELL_CMD_HANDLER(
 SHELL_CMD_HANDLER(
     "cpProductIntegrationMgmtParentObjectName",
     "cpsdwan get_data | jq -r .cluster_name",
-    getMgmtParentObjName
+    getSmbMgmtParentObjName
 )
 SHELL_CMD_HANDLER(
     "cpProductIntegrationMgmtParentObjectUid",
     "cpsdwan get_data | jq -r .cluster_uuid",
-    getMgmtParentObjUid
+    getSmbMgmtParentObjUid
 )
 SHELL_CMD_HANDLER(
     "cpProductIntegrationMgmtObjectName",
@@ -143,4 +156,6 @@ FILE_CONTENT_HANDLER(
 FILE_CONTENT_HANDLER("os_release", "/etc/os-release", getOsRelease)
 #endif // gaia || smb
 
+FILE_CONTENT_HANDLER("AppSecModelVersion", "/etc/cp/conf/waap/waap.data", getWaapModelVersion)
+
 #endif // FILE_CONTENT_HANDLER

components/security_apps/orchestration/details_resolver/details_resolving_handler.cc

@@ -22,6 +22,7 @@
 #include "maybe_res.h"
 #include "enum_array.h"
 #include "i_shell_cmd.h"
+#include "i_orchestration_tools.h"
 #include "config.h"
 
 using namespace std;
@@ -77,7 +78,8 @@ DetailsResolvingHanlder::Impl::getResolvedDetails() const
         const string &path = file_handler.second.first;
         FileContentHandler handler = file_handler.second.second;
 
-        shared_ptr<ifstream> in_file = make_shared<ifstream>(path);
+        shared_ptr<ifstream> in_file =
+            Singleton::Consume<I_OrchestrationTools>::by<DetailsResolvingHanlder>()->fileStreamWrapper(path);
         if (!in_file->is_open()) {
             dbgWarning(D_AGENT_DETAILS) << "Could not open file for processing. Path: " << path;
             continue;

components/security_apps/orchestration/details_resolver/details_resolving_handler.h

@@ -18,11 +18,13 @@
 #include <map>
 
 #include "i_shell_cmd.h"
+#include "i_orchestration_tools.h"
 #include "i_agent_details_reporter.h"
 
 class DetailsResolvingHanlder
         :
     Singleton::Consume<I_ShellCmd>,
+    Singleton::Consume<I_OrchestrationTools>,
     Singleton::Consume<I_AgentDetailsReporter>
 {
 public:

components/security_apps/orchestration/downloader/curl_client.cc

@@ -278,6 +278,36 @@ HttpsCurl::HttpsCurl(const HttpsCurl &other) :
     HttpCurl(other),
     ca_path(other.ca_path) {}
 
+bool
+HttpsCurl::downloadOpenAppsecPackages()
+{
+    char errorstr[CURL_ERROR_SIZE];
+    CURL* curl_handle = curl_easy_init();
+    if (!curl_handle) return false;
+
+    curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYPEER, 1);
+    curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYHOST, 2);
+
+    curl_easy_setopt(curl_handle, CURLOPT_URL, ("https://" + curl_url).c_str());
+    curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, writeResponseCallback);
+    curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, &out_file);
+
+    curl_easy_setopt(curl_handle, CURLOPT_VERBOSE, 1L);
+    curl_easy_setopt(curl_handle, CURLOPT_ERRORBUFFER, errorstr);
+
+    CURLcode res = curl_easy_perform(curl_handle);
+    if (res == CURLE_OK) {
+        dbgTrace(D_HTTP_REQUEST) << "CURL HTTP request successfully completed.";
+    } else {
+        dbgWarning(D_HTTP_REQUEST) << "CURL result " + string(curl_easy_strerror(res));
+        curl_easy_cleanup(curl_handle);
+        return false;
+    }
+
+    curl_easy_cleanup(curl_handle);
+    return true;
+}
+
 void
 HttpsCurl::setCurlOpts(long timeout, HTTP_VERSION http_version)
 {
@@ -299,9 +329,9 @@ HttpsCurl::setCurlOpts(long timeout, HTTP_VERSION http_version)
     curl_easy_setopt(curl_handle, CURLOPT_HTTP_VERSION, http_version);
 
     //SSL options
-    if (getProfileAgentSettingWithDefault<bool>(
-        false,
-        "agent.config.message.ignoreSslValidation") == false)
+    if (
+        getProfileAgentSettingWithDefault<bool>(false, "agent.config.message.ignoreSslValidation") == false
+    )
     {
         curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYPEER, 1L);
         curl_easy_setopt(curl_handle, CURLOPT_SSL_CTX_FUNCTION, ssl_ctx_verify_certificate);

components/security_apps/orchestration/downloader/curl_client.h

@@ -105,6 +105,7 @@ public:
     static CURLcode ssl_ctx_verify_certificate(CURL *curl, void *ssl_ctx, void *opq);
     static int verify_certificate(int preverify_ok, X509_STORE_CTX *ctx);
     void setCurlOpts(long timeout = 60L, HTTP_VERSION http_version = HTTP_VERSION::HTTP_VERSION_1_1) override;
+    bool downloadOpenAppsecPackages();
 
 private:
     std::string ca_path;

components/security_apps/orchestration/downloader/downloader_ut/downloader_ut.cc

@@ -51,7 +51,7 @@ TEST_F(DownloaderTest, downloadFileFromFog)
         calculateChecksum(Package::ChecksumTypes::SHA256, "/tmp/virtualSettings.download")
     ).WillOnce(Return(string("123")));
 
-    EXPECT_CALL(mock_orchestration_tools, writeFile(fog_response, "/tmp/virtualSettings.download"))
+    EXPECT_CALL(mock_orchestration_tools, writeFile(fog_response, "/tmp/virtualSettings.download", false))
         .WillOnce(Return(true));
     EXPECT_CALL(mock_orchestration_tools, isNonEmptyFile("/tmp/virtualSettings.download")).WillOnce(Return(true));
 
@@ -183,7 +183,7 @@ TEST_F(DownloaderTest, downloadEmptyFileFromFog)
 
     EXPECT_CALL(mock_communication, downloadAttributeFile(resourse_file)).WillOnce(Return(fog_response));
 
-    EXPECT_CALL(mock_orchestration_tools, writeFile(fog_response, "/tmp/manifest.download"))
+    EXPECT_CALL(mock_orchestration_tools, writeFile(fog_response, "/tmp/manifest.download", false))
         .WillOnce(Return(true));
     EXPECT_CALL(mock_orchestration_tools, isNonEmptyFile("/tmp/manifest.download")).WillOnce(Return(false));
 
@@ -342,13 +342,23 @@ TEST_F(DownloaderTest, download_virtual_policy)
 
     EXPECT_CALL(mock_communication, downloadAttributeFile(resourse_file)).WillOnce(Return(fog_response));
 
-    EXPECT_CALL(mock_orchestration_tools, writeFile(tenant_0000_file, "/tmp/virtualPolicy_0000_profile_1234.download"))
-        .WillOnce(Return(true));
+    EXPECT_CALL(
+        mock_orchestration_tools,
+        writeFile(
+            tenant_0000_file,
+            "/tmp/virtualPolicy_0000_profile_1234.download",
+            false)
+    ).WillOnce(Return(true));
 
     EXPECT_CALL(mock_orchestration_tools, fillKeyInJson(_, _, _)).WillRepeatedly(Return());
 
-    EXPECT_CALL(mock_orchestration_tools, writeFile(tenant_1111_file, "/tmp/virtualPolicy_1111_profile_1235.download"))
-        .WillOnce(Return(true));
+    EXPECT_CALL(
+        mock_orchestration_tools,
+        writeFile(
+            tenant_1111_file,
+            "/tmp/virtualPolicy_1111_profile_1235.download",
+            false)
+    ).WillOnce(Return(true));
 
     map<pair<string, string>, string> expected_downloaded_files =
         {
@@ -427,7 +437,8 @@ TEST_F(DownloaderTest, download_virtual_settings)
         mock_orchestration_tools,
         writeFile(
             tenant_0000_file,
-            tenant_0000_path.str()
+            tenant_0000_path.str(),
+            false
         )
     ).WillOnce(Return(true));
 

components/security_apps/orchestration/downloader/http_client.h

@@ -37,8 +37,8 @@ private:
     std::string loadCAChainDir();
     Maybe<void> getFileSSL(const URLParser &url, std::ofstream &out_file, const std::string &_token);
     Maybe<void> getFileHttp(const URLParser &url, std::ofstream &out_file, const std::string &_token);
-    Maybe<void> curlGetFileOverSSL(const URLParser &url, std::ofstream &out_file, const std::string &_token);
     Maybe<void> curlGetFileOverHttp(const URLParser &url, std::ofstream &out_file, const std::string &_token);
+    Maybe<void> curlGetFileOverSSL(const URLParser &url, std::ofstream &out_file, const std::string &_token);
 };
 // LCOV_EXCL_STOP
 

components/security_apps/orchestration/downloader/https_client.cc

@@ -592,8 +592,13 @@ HTTPClient::curlGetFileOverSSL(const URLParser &url, ofstream &out_file, const s
             proxy_config->getProxyCredentials(ProxyProtocol::HTTPS),
             cert_file_path);
 
-        ssl_curl_client.setCurlOpts();
-        bool connection_ok = ssl_curl_client.connect();
+        bool connection_ok;
+        if (url.getBaseURL().unpack() == "downloads.openappsec.io") {
+            connection_ok = ssl_curl_client.downloadOpenAppsecPackages();
+        } else {
+            ssl_curl_client.setCurlOpts();
+            connection_ok = ssl_curl_client.connect();
+        }
         if (!connection_ok)
         {
             stringstream url_s;

components/security_apps/orchestration/health_check/health_check.cc

@@ -21,6 +21,7 @@
 #include "config.h"
 #include "log_generator.h"
 #include "health_check_manager.h"
+#include "agent_core_utilities.h"
 
 using namespace std;
 using namespace ReportIS;
@@ -145,9 +146,11 @@ private:
     initCloudVendorConfig()
     {
         static const map<string, pair<string, int>> ip_port_defaults_map = {
-            {"Azure", make_pair("168.63.129.16", 8117)},
-            {"Aws", make_pair("", 8117)}
+            {"Azure", make_pair(getenv("DOCKER_RPM_ENABLED") ? "" : "168.63.129.16", 8117)},
+            {"Aws", make_pair("", 8117)},
+            {"Local", make_pair("", 8117)}
         };
+
         auto cloud_vendor_maybe = getSetting<string>("reverseProxy", "cloudVendorName");
         if (cloud_vendor_maybe.ok()) {
             const string cloud_vendor = cloud_vendor_maybe.unpack();
@@ -247,13 +250,36 @@ private:
         );
     }
 
+    HealthCheckStatus
+    getStandaloneHealthStatus()
+    {
+        if (!getenv("DOCKER_RPM_ENABLED")) return HealthCheckStatus::IGNORED;
+
+        static const string standalone_cmd = "/usr/sbin/cpnano -s --docker-rpm; echo $?";
+        dbgTrace(D_HEALTH_CHECK) << "Checking the standalone docker health status with command: " << standalone_cmd;
+
+        auto maybe_result = Singleton::Consume<I_ShellCmd>::by<HealthChecker>()->getExecOutput(standalone_cmd, 1000);
+        if (!maybe_result.ok()) {
+            dbgWarning(D_HEALTH_CHECK) << "Unable to get the standalone docker status. Returning unhealthy status.";
+            return HealthCheckStatus::UNHEALTHY;
+        }
+        dbgTrace(D_HEALTH_CHECK) << "Got response: " << maybe_result.unpack();
+
+        auto response = NGEN::Strings::removeTrailingWhitespaces(maybe_result.unpack());
+
+        if (response.back() == '0') return HealthCheckStatus::HEALTHY;
+        if (response.back() == '1') return HealthCheckStatus::UNHEALTHY;
+
+        return HealthCheckStatus::DEGRADED;
+    }
+
     bool
     nginxContainerIsRunning()
     {
         static const string nginx_container_name = "cp_nginx_gaia";
         static const string cmd_running =
             "docker ps --filter name=" + nginx_container_name + " --filter status=running";
-        dbgTrace(D_HEALTH_CHECK) << "Checking if the container is running with the commmand: " << cmd_running;
+        dbgTrace(D_HEALTH_CHECK) << "Checking if the container is running with the command: " << cmd_running;
 
         auto maybe_result = Singleton::Consume<I_ShellCmd>::by<HealthChecker>()->getExecOutput(cmd_running);
         if (!maybe_result.ok()) {
@@ -263,7 +289,6 @@ private:
         }
 
         return (*maybe_result).find(nginx_container_name) != string::npos;
-
     }
 
     void
@@ -279,7 +304,7 @@ private:
     {
         if (open_connections_counter >= max_connections) {
             dbgDebug(D_HEALTH_CHECK)
-                << "Cannot serve new client, reached maximun open connections bound which is:"
+                << "Cannot serve new client, reached maximum open connections bound which is:"
                 << open_connections_counter
                 << "maximum allowed: "
                 << max_connections;
@@ -331,6 +356,42 @@ private:
                     "health check failed\r\n";
                 static const vector<char> failure_response_buffer(failure_response.begin(), failure_response.end());
 
+                static const string degraded_response =
+                    "HTTP/1.1 202 OK\r\n"
+                    "Content-Length: 22\r\n"
+                    "Content-Type: text/plain\r\n"
+                    "\r\n"
+                    "health check partial\r\n";
+                static const vector<char> degraded_response_buffer(degraded_response.begin(), degraded_response.end());
+
+                HealthCheckStatus standalone_status = getStandaloneHealthStatus();
+                if (standalone_status != HealthCheckStatus::IGNORED) {
+                    if (standalone_status == HealthCheckStatus::HEALTHY) {
+                        dbgDebug(D_HEALTH_CHECK)
+                            << "Standalone status is healthy, returning the following response: "
+                            << success_response;
+                        i_socket->writeData(curr_client_socket, success_response_buffer);
+                        closeCurrentSocket(curr_client_socket, curr_routine_id);
+                        return;
+                    }
+
+                    if (standalone_status == HealthCheckStatus::UNHEALTHY) {
+                        dbgDebug(D_HEALTH_CHECK)
+                            << "Standalone status in unhealthy, returning the following response: "
+                            << failure_response;
+                        i_socket->writeData(curr_client_socket, failure_response_buffer);
+                        closeCurrentSocket(curr_client_socket, curr_routine_id);
+                        return;
+                    }
+
+                    dbgDebug(D_HEALTH_CHECK)
+                        << "Standalone status was partially loaded, returning the following response: "
+                        << degraded_response;
+                    i_socket->writeData(curr_client_socket, degraded_response_buffer);
+                    closeCurrentSocket(curr_client_socket, curr_routine_id);
+                    return;
+                }
+
                 if (nginxContainerIsRunning()) {
                     dbgDebug(D_HEALTH_CHECK)
                     << "nginx conatiner is running, returning the following response: "

components/security_apps/orchestration/health_check/health_check_ut/health_check_ut.cc

@@ -194,7 +194,7 @@ TEST_F(HealthCheckerTest, connectionsLimit)
     connection_handler_routine();
 
     EXPECT_THAT(
-        capture_debug.str(), HasSubstr("Cannot serve new client, reached maximun open connections")
+        capture_debug.str(), HasSubstr("Cannot serve new client, reached maximum open connections")
     );
 }
 

components/security_apps/orchestration/include/declarative_policy_utils.h

@@ -31,6 +31,14 @@ class ApplyPolicyEvent : public Event<ApplyPolicyEvent>
 {
 public:
     ApplyPolicyEvent() {}
+    ApplyPolicyEvent(const std::string &path) : local_policy_path(path) {}
+
+    // LCOV_EXCL_START Reason: no test exist
+    std::string getPolicyPath() const { return local_policy_path; }
+    // LCOV_EXCL_STOP
+
+private:
+    std::string local_policy_path;
 };
 
 class DeclarativePolicyUtils
@@ -40,6 +48,7 @@ class DeclarativePolicyUtils
     Singleton::Consume<I_EnvDetails>,
     Singleton::Consume<I_AgentDetails>,
     Singleton::Consume<I_OrchestrationTools>,
+    public Singleton::Consume<I_MainLoop>,
     Singleton::Consume<I_RestApi>,
     public Listener<ApplyPolicyEvent>
 {
@@ -50,8 +59,7 @@ public:
         void
         doCall() override
         {
-            Singleton::Consume<I_LocalPolicyMgmtGen>::by<DeclarativePolicyUtils>()->setPolicyPath(policy_path.get());
-            ApplyPolicyEvent().notify();
+            ApplyPolicyEvent(policy_path.get()).notify();
         }
 
     private:
@@ -80,6 +88,7 @@ public:
 private:
     std::string getCleanChecksum(const std::string &unclean_checksum);
 
+    std::string local_policy_path;
     std::string curr_version;
     std::string curr_policy;
     bool should_apply_policy;

components/security_apps/orchestration/include/fog_authenticator.h

@@ -142,6 +142,7 @@ protected:
     std::string base64Encode(const std::string &in) const;
     std::string buildBasicAuthHeader(const std::string &username, const std::string &pass) const;
     std::string buildOAuth2Header(const std::string &token) const;
+    std::string getUserEdition() const;
 
     // This apps which the orchestrations requires them from Fog.
     std::vector<std::string> required_security_apps;

components/security_apps/orchestration/include/get_status_rest.h

@@ -54,6 +54,7 @@ public:
         last_update = i_orch_status->getUpdateTime();
         last_update_status = i_orch_status->getUpdateStatus();
         policy_version = i_orch_status->getPolicyVersion();
+        waap_model_version = i_orch_status->getWaapModelVersion();
         last_policy_update = i_orch_status->getLastPolicyUpdate();
         last_manifest_update = i_orch_status->getLastManifestUpdate();
         last_settings_update = i_orch_status->getLastSettingsUpdate();
@@ -72,6 +73,7 @@ private:
     S2C_LABEL_PARAM(std::string, last_update, "Last update");
     S2C_LABEL_PARAM(std::string, last_update_status, "Last update status");
     S2C_LABEL_PARAM(std::string, policy_version, "Policy version");
+    S2C_LABEL_PARAM(std::string, waap_model_version, "AI model version");
     S2C_LABEL_PARAM(std::string, last_policy_update, "Last policy update");
     S2C_LABEL_PARAM(std::string, last_manifest_update, "Last manifest update");
     S2C_LABEL_PARAM(std::string, last_settings_update, "Last settings update");

components/security_apps/orchestration/include/mock/mock_orchestration_status.h

@@ -45,6 +45,7 @@ public:
     MOCK_CONST_METHOD0(getUpdateTime, const std::string&());
     MOCK_CONST_METHOD0(getLastManifestUpdate, const std::string&());
     MOCK_CONST_METHOD0(getPolicyVersion, const std::string&());
+    MOCK_CONST_METHOD0(getWaapModelVersion, const std::string&());
     MOCK_CONST_METHOD0(getLastPolicyUpdate, const std::string&());
     MOCK_CONST_METHOD0(getLastSettingsUpdate, const std::string&());
     MOCK_CONST_METHOD0(getUpgradeMode, const std::string&());

components/security_apps/orchestration/local_policy_mgmt_gen/CMakeLists.txt

@@ -1,3 +0,0 @@
-include_directories(include)
-
-add_library(local_policy_mgmt_gen appsec_practice_section.cc exceptions_section.cc ingress_data.cc local_policy_mgmt_gen.cc policy_maker_utils.cc rules_config_section.cc settings_section.cc snort_section.cc triggers_section.cc trusted_sources_section.cc k8s_policy_utils.cc namespace_data.cc new_appsec_linux_policy.cc new_appsec_policy_crd_parser.cc new_custom_response.cc new_exceptions.cc new_log_trigger.cc new_practice.cc new_trusted_sources.cc access_control_practice.cc)

components/security_apps/orchestration/local_policy_mgmt_gen/include/k8s_policy_common.h

@@ -1,106 +0,0 @@
-// Copyright (C) 2022 Check Point Software Technologies Ltd. All rights reserved.
-
-// Licensed under the Apache License, Version 2.0 (the "License");
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-#ifndef __K8S_POLICY_COMMON_H__
-#define __K8S_POLICY_COMMON_H__
-
-#include <map>
-#include <set>
-#include <string>
-#include <cereal/archives/json.hpp>
-
-#include "config.h"
-#include "debug.h"
-#include "rest.h"
-
-USE_DEBUG_FLAG(D_LOCAL_POLICY);
-// LCOV_EXCL_START Reason: no test exist
-enum class PracticeType { WebApplication, WebAPI };
-enum class TriggerType { Log, WebUserResponse };
-enum class MatchType { Condition, Operator };
-
-static const std::unordered_map<std::string, MatchType> string_to_match_type = {
-    { "condition", MatchType::Condition },
-    { "operator", MatchType::Operator }
-};
-
-static const std::unordered_map<std::string, PracticeType> string_to_practice_type = {
-    { "WebApplication", PracticeType::WebApplication },
-    { "WebAPI", PracticeType::WebAPI }
-};
-
-static const std::unordered_map<std::string, TriggerType> string_to_trigger_type = {
-    { "log", TriggerType::Log },
-    { "WebUserResponse", TriggerType::WebUserResponse }
-};
-
-static const std::unordered_map<std::string, std::string> key_to_practices_val = {
-    { "prevent-learn", "Prevent"},
-    { "detect-learn", "Detect"},
-    { "prevent", "Prevent"},
-    { "detect", "Detect"},
-    { "inactive", "Inactive"}
-};
-
-template <typename T>
-void
-parseAppsecJSONKey(
-    const std::string &key_name,
-    T &value,
-    cereal::JSONInputArchive &archive_in,
-    const T &default_value = T())
-{
-    try {
-        archive_in(cereal::make_nvp(key_name, value));
-    } catch (const cereal::Exception &e) {
-        archive_in.setNextName(nullptr);
-        value = default_value;
-        dbgDebug(D_LOCAL_POLICY)
-            << "Could not parse the required key. Key: "
-            << key_name
-            << ", Error: "
-            << e.what();
-    }
-}
-
-template <typename T>
-class AppsecSpecParser : public ClientRest
-{
-public:
-    AppsecSpecParser() = default;
-    AppsecSpecParser(const T &_spec) : spec(_spec) {}
-
-    bool
-    loadJson(const std::string &json)
-    {
-        std::string modified_json = json;
-        modified_json.pop_back();
-        std::stringstream ss;
-        ss.str(modified_json);
-        try {
-            cereal::JSONInputArchive in_ar(ss);
-            in_ar(cereal::make_nvp("spec", spec));
-        } catch (cereal::Exception &e) {
-            dbgError(D_LOCAL_POLICY) << "Failed to load spec JSON. Error: " << e.what();
-            return false;
-        }
-        return true;
-    }
-
-    const T & getSpec() const { return spec; }
-
-private:
-    T spec;
-};
-// LCOV_EXCL_STOP
-#endif // __K8S_POLICY_COMMON_H__

components/security_apps/orchestration/manifest_controller/manifest_controller.cc

@@ -76,6 +76,7 @@ public:
 
 private:
     bool changeManifestFile(const string &new_manifest_file);
+    bool updateIgnoreListForNSaaS();
 
     bool
     handlePackage(
@@ -155,12 +156,36 @@ ManifestController::Impl::init()
     }
 }
 
+bool
+ManifestController::Impl::updateIgnoreListForNSaaS()
+{
+    if (!getProfileAgentSettingWithDefault<bool>(false, "accessControl.isAwsNSaaS")) return false;
+
+    auto ignore_packages_path = getConfigurationWithDefault<string>(
+        getFilesystemPathConfig() + "/conf/ignore-packages.txt",
+        "orchestration",
+        "Ignore packages list file path"
+    );
+    ofstream ignore_file(ignore_packages_path);
+    if (!ignore_file.is_open()) {
+        dbgWarning(D_ORCHESTRATOR) << "Unable to open file " << ignore_packages_path << " for writing";
+        return false;
+    }
+
+    ignore_file << "all";
+    ignore_file.close();
+    dbgInfo(D_ORCHESTRATOR) << "Updated " << ignore_packages_path << " to ignore all packages";
+
+    return true;
+}
+
 bool
 ManifestController::Impl::updateManifest(const string &new_manifest_file)
 {
     auto i_env = Singleton::Consume<I_Environment>::by<ManifestController>();
     auto span_scope = i_env->startNewSpanScope(Span::ContextType::CHILD_OF);
     auto orchestration_tools = Singleton::Consume<I_OrchestrationTools>::by<ManifestController>();
+    static bool ignore_packages_update = false;
 
     if (isIgnoreFile(new_manifest_file)) {
         if (!orchestration_tools->copyFile(new_manifest_file, manifest_file_path)) {
@@ -173,9 +198,12 @@ ManifestController::Impl::updateManifest(const string &new_manifest_file)
     dbgDebug(D_ORCHESTRATOR) << "Starting to update manifest file";
     auto ignored_settings_packages = getProfileAgentSetting<IgnoredPackages>("orchestration.IgnoredPackagesList");
     set<string> packages_to_ignore = ignore_packages;
-    if (ignored_settings_packages.ok()) packages_to_ignore = *(*ignored_settings_packages);
+    if (ignored_settings_packages.ok()) {
+        packages_to_ignore = *(*ignored_settings_packages);
+        ignore_packages_update = false;
+    }
 
-    if (packages_to_ignore.count("all") > 0) {
+    if (ignore_packages_update || packages_to_ignore.count("all") > 0) {
         dbgTrace(D_ORCHESTRATOR) << "Nothing to update (\"ignore all\" turned on)";
 
         if (!orchestration_tools->copyFile(new_manifest_file, manifest_file_path)) {
@@ -315,6 +343,8 @@ ManifestController::Impl::updateManifest(const string &new_manifest_file)
 
     if (all_installed && (any_installed || no_change) && no_corrupted_package) {
         manifest_file_update = changeManifestFile(new_manifest_file);
+        // In NSaaS - set ignore packages to any
+        ignore_packages_update = updateIgnoreListForNSaaS();
     } else if (any_installed) {
         manifest_file_update = orchestration_tools->packagesToJsonFile(current_packages, manifest_file_path);
     }

components/security_apps/orchestration/modules/modules_ut/orchestration_status_ut.cc

@@ -11,6 +11,7 @@
 #include "mock/mock_time_get.h"
 #include "mock/mock_orchestration_tools.h"
 #include "mock/mock_agent_details.h"
+#include "mock/mock_details_resolver.h"
 #include "mock/mock_mainloop.h"
 #include "mock/mock_rest_api.h"
 
@@ -38,9 +39,17 @@ public:
             .WillOnce(DoAll(SaveArg<2>(&routine), Return(1))
         );
         EXPECT_CALL(mock_tools, readFile(file_path)).WillOnce(Return(start_file_content));
+        prepareResolvedDetails();
         orchestration_status.init();
     }
 
+    void
+    prepareResolvedDetails()
+    {
+        map<string, string> resolved_details({{"AppSecModelVersion", waap_model}});
+        EXPECT_CALL(mock_details_resolver, getResolvedDetails()).WillRepeatedly(Return(resolved_details));
+    }
+
     string
     orchestrationStatusFileToString()
     {
@@ -82,7 +91,8 @@ public:
         const string &registration_details_architecture = "",
         const string &agent_id = "None",
         const string &profile_id = "None",
-        const string &tenant_id = "None"
+        const string &tenant_id = "None",
+        const string &waap_model_version = "Advanced model"
     )
     {
         return  "{\n"
@@ -91,6 +101,7 @@ public:
                 "    \"Last update\": \"" + last_update + "\",\n"
                 "    \"Last manifest update\": \"" + last_manifest_update + "\",\n"
                 "    \"Policy version\": \"" + policy_version + "\",\n"
+                "    \"AI model version\": \"" + waap_model_version + "\",\n"
                 "    \"Last policy update\": \"" + last_policy_update + "\",\n"
                 "    \"Last settings update\": \"" + last_settings_update + "\",\n"
                 "    \"Upgrade mode\": \"" + upgrade_mode + "\",\n"
@@ -118,12 +129,14 @@ public:
     ostringstream capture_debug;
     StrictMock<MockOrchestrationTools> mock_tools;
     StrictMock<MockAgentDetails> mock_agent_details;
+    StrictMock<MockDetailsResolver> mock_details_resolver;
     OrchestrationStatus orchestration_status;
     I_OrchestrationStatus * i_orchestration_status =
         Singleton::Consume<I_OrchestrationStatus>::from(orchestration_status);
     string file_path;
     Maybe<string> start_file_content = genError("No file");
     I_MainLoop::Routine routine;
+    string waap_model = "Advanced model";
 };
 
 TEST_F(OrchestrationStatusTest, doNothing)
@@ -147,6 +160,7 @@ TEST_F(OrchestrationStatusTest, recoverFields)
 
 TEST_F(OrchestrationStatusTest, loadFromFile)
 {
+    prepareResolvedDetails();
     Maybe<string> status = genError("No file");;
     CPTestTempfile status_file;
     file_path = status_file.fname;
@@ -214,12 +228,14 @@ TEST_F(OrchestrationStatusTest, recoveryFields)
     const string agent_id = "AgentId";
     const string profile_id = "ProfileId";
     const string tenant_id = "TenantId";
+
     auto fog_addr = Maybe<string>(string("FogDomain"));
 
     EXPECT_CALL(mock_agent_details, getAgentId()).WillOnce(Return(agent_id));
     EXPECT_CALL(mock_agent_details, getProfileId()).WillOnce(Return(profile_id));
     EXPECT_CALL(mock_agent_details, getTenantId()).WillOnce(Return(tenant_id));
     EXPECT_CALL(mock_agent_details, getFogDomain()).WillOnce(Return(fog_addr));
+
     i_orchestration_status->writeStatusToFile();
     EXPECT_THAT(capture_debug.str(), HasSubstr("Repairing status fields"));
 
@@ -227,6 +243,7 @@ TEST_F(OrchestrationStatusTest, recoveryFields)
     EXPECT_EQ(i_orchestration_status->getProfileId(), profile_id);
     EXPECT_EQ(i_orchestration_status->getTenantId(), tenant_id);
     EXPECT_EQ(i_orchestration_status->getFogAddress(), fog_addr.unpack());
+    EXPECT_EQ(i_orchestration_status->getWaapModelVersion(), waap_model);
 }
 
 TEST_F(OrchestrationStatusTest, updateAllLastUpdatesTypes)
@@ -419,6 +436,7 @@ TEST_F(OrchestrationStatusTest, setAllFields)
                         "    \"Last update\": \"current time\",\n"
                         "    \"Last manifest update\": \"current time\",\n"
                         "    \"Policy version\": \"12\",\n"
+                        "    \"AI model version\": \"Advanced model\",\n"
                         "    \"Last policy update\": \"current time\",\n"
                         "    \"Last settings update\": \"current time\",\n"
                         "    \"Upgrade mode\": \"Test Mode\",\n"

components/security_apps/orchestration/modules/orchestration_status.cc

@@ -108,6 +108,7 @@ public:
         last_update_attempt = from.last_update_attempt;
         last_manifest_update = from.last_manifest_update;
         policy_version = from.policy_version;
+        waap_model_version = from.waap_model_version;
         last_policy_update = from.last_policy_update;
         last_settings_update = from.last_settings_update;
         upgrade_mode = from.upgrade_mode;
@@ -128,6 +129,7 @@ public:
     const string & getUpdateTime() const                             { return last_update_time; }
     const string & getLastManifestUpdate() const                     { return last_manifest_update; }
     const string & getPolicyVersion() const                          { return policy_version; }
+    const string & getWaapModelVersion() const                       { return waap_model_version; }
     const string & getLastPolicyUpdate() const                       { return last_policy_update; }
     const string & getLastSettingsUpdate() const                     { return last_settings_update; }
     const string & getUpgradeMode() const                            { return upgrade_mode; }
@@ -142,6 +144,16 @@ public:
     const map<string, string> & getServicePolicies() const { return service_policies; }
     const map<string, string> & getServiceSettings() const { return service_settings; }
 
+    void updateWaapModelVersion() {
+        map<string, string> details_resolver =
+            Singleton::Consume<I_DetailsResolver>::by<OrchestrationStatus>()->getResolvedDetails();
+        if (details_resolver.find("AppSecModelVersion") != details_resolver.end()) {
+            waap_model_version =  details_resolver["AppSecModelVersion"];
+        } else {
+            waap_model_version = "None";
+        }
+    }
+
     void
     insertServicePolicy(const string &key, const string &value)
     {
@@ -267,12 +279,13 @@ public:
         last_manifest_update = "None";
         last_policy_update = "None";
         last_settings_update = "None";
+        waap_model_version = "None";
         fog_address = "None";
         agent_id = "None";
         profile_id = "None";
         tenant_id = "None";
         registration_status = "None";
-        manifest_status = "None";
+        manifest_status = getenv("CLOUDGUARD_APPSEC_STANDALONE") ? "Succeeded" : "None";
         upgrade_mode = "None";
     }
 
@@ -292,6 +305,7 @@ public:
             } else {
                 fog_address = "None";
             }
+            updateWaapModelVersion();
         }
     }
 
@@ -304,6 +318,7 @@ public:
         archive(cereal::make_nvp("Last update", last_update_time));
         archive(cereal::make_nvp("Last manifest update", last_manifest_update));
         archive(cereal::make_nvp("Policy version", policy_version));
+        archive(cereal::make_nvp("AI model version", waap_model_version));
         archive(cereal::make_nvp("Last policy update", last_policy_update));
         archive(cereal::make_nvp("Last settings update", last_settings_update));
         archive(cereal::make_nvp("Upgrade mode", upgrade_mode));
@@ -331,6 +346,7 @@ public:
             archive.setNextName(nullptr);
         }
 
+        archive(cereal::make_nvp("AI model version",        waap_model_version));
         archive(cereal::make_nvp("Last policy update",      last_policy_update));
         archive(cereal::make_nvp("Last settings update",    last_settings_update));
 
@@ -368,6 +384,7 @@ private:
     string last_update_attempt;
     string last_manifest_update;
     string policy_version;
+    string waap_model_version;
     string last_policy_update;
     string last_settings_update;
     string upgrade_mode;
@@ -387,13 +404,14 @@ class OrchestrationStatus::Impl : Singleton::Provide<I_OrchestrationStatus>::Fro
 {
 public:
     void
-    writeStatusToFile()
+    writeStatusToFile() override
     {
         auto orchestration_status_path = getConfigurationWithDefault<string>(
             filesystem_prefix + "/conf/orchestration_status.json",
             "orchestration",
             "Orchestration status path"
         );
+        status.updateWaapModelVersion();
         auto write_result =
             orchestration_tools->objectToJsonFile<Status>(status, orchestration_status_path);
         if (!write_result) {
@@ -497,6 +515,7 @@ private:
     const string & getUpdateTime() const override                     { return status.getUpdateTime(); }
     const string & getLastManifestUpdate() const override             { return status.getLastManifestUpdate(); }
     const string & getPolicyVersion() const override                  { return status.getPolicyVersion(); }
+    const string & getWaapModelVersion() const override               { return status.getWaapModelVersion(); }
     const string & getLastPolicyUpdate() const override               { return status.getLastPolicyUpdate(); }
     const string & getLastSettingsUpdate() const override             { return status.getLastSettingsUpdate(); }
     const string & getUpgradeMode() const override                    { return status.getUpgradeMode(); }

components/security_apps/orchestration/orchestration_comp.cc

@@ -189,6 +189,10 @@ public:
             "Orchestration runner",
             true
         );
+
+        auto orchestration_tools = Singleton::Consume<I_OrchestrationTools>::by<OrchestrationComp>();
+        orchestration_tools->getClusterId();
+
         hybrid_mode_metric.init(
             "Watchdog Metrics",
             ReportIS::AudienceTeam::AGENT_CORE,
@@ -198,7 +202,6 @@ public:
             ReportIS::Audience::INTERNAL
         );
         hybrid_mode_metric.registerListener();
-        auto orchestration_tools = Singleton::Consume<I_OrchestrationTools>::by<OrchestrationComp>();
         orchestration_tools->loadTenantsFromDir(
             getConfigurationWithDefault<string>(getFilesystemPathConfig() + "/conf/", "orchestration", "Conf dir")
         );
@@ -1485,6 +1488,9 @@ private:
         if (i_details_resolver->compareCheckpointVersion(8100, greater_equal<int>())) {
             agent_data_report << AgentReportFieldWithLabel("isCheckpointVersionGER81", "true");
         }
+        if (i_details_resolver->compareCheckpointVersion(8200, greater_equal<int>())) {
+            agent_data_report << AgentReportFieldWithLabel("isCheckpointVersionGER82", "true");
+        }
 #endif // gaia || smb
 
         if (agent_data_report == curr_agent_data_report) {

components/security_apps/orchestration/orchestration_tools/CMakeLists.txt

@@ -1,5 +1,5 @@
 ADD_DEFINITIONS(-Wno-deprecated-declarations)
 
-add_library(orchestration_tools orchestration_tools.cc)
+add_library(orchestration_tools orchestration_tools.cc namespace_data.cc)
 
 #add_subdirectory(orchestration_tools_ut)

components/security_apps/orchestration/orchestration_tools/namespace_data.cc

@@ -12,11 +12,31 @@
 // limitations under the License.
 
 #include "namespace_data.h"
-#include "local_policy_common.h"
 
 using namespace std;
 
-USE_DEBUG_FLAG(D_LOCAL_POLICY);
+USE_DEBUG_FLAG(D_ORCHESTRATOR);
+
+template <typename T>
+void
+parseNameSpaceJSONKey(
+    const string &key_name,
+    T &value,
+    cereal::JSONInputArchive &archive_in,
+    const T &default_value = T())
+{
+    try {
+        archive_in(cereal::make_nvp(key_name, value));
+    } catch (const cereal::Exception &e) {
+        archive_in.setNextName(nullptr);
+        value = default_value;
+        dbgDebug(D_ORCHESTRATOR)
+            << "Could not parse the required key. Key: "
+            << key_name
+            << ", Error: "
+            << e.what();
+    }
+}
 
 class NamespaceMetadata
 {
@@ -24,9 +44,9 @@ public:
     void
     load(cereal::JSONInputArchive &archive_in)
     {
-        dbgFlow(D_LOCAL_POLICY);
-        parseAppsecJSONKey<string>("name", name, archive_in);
-        parseAppsecJSONKey<string>("uid", uid, archive_in);
+        dbgFlow(D_ORCHESTRATOR);
+        parseNameSpaceJSONKey<string>("name", name, archive_in);
+        parseNameSpaceJSONKey<string>("uid", uid, archive_in);
     }
 
     const string &
@@ -52,7 +72,7 @@ public:
     void
     load(cereal::JSONInputArchive &archive_in)
     {
-        parseAppsecJSONKey<NamespaceMetadata>("metadata", metadata, archive_in);
+        parseNameSpaceJSONKey<NamespaceMetadata>("metadata", metadata, archive_in);
     }
 
     const NamespaceMetadata &
@@ -68,7 +88,7 @@ private:
 bool
 NamespaceData::loadJson(const string &json)
 {
-    dbgFlow(D_LOCAL_POLICY);
+    dbgFlow(D_ORCHESTRATOR);
     string modified_json = json;
     modified_json.pop_back();
     stringstream in;
@@ -81,7 +101,7 @@ NamespaceData::loadJson(const string &json)
             ns_name_to_uid[single_ns_data.getMetadata().getName()] = single_ns_data.getMetadata().getUID();
         }
     } catch (cereal::Exception &e) {
-        dbgWarning(D_LOCAL_POLICY) << "Failed to load namespace data JSON. Error: " << e.what();
+        dbgWarning(D_ORCHESTRATOR) << "Failed to load namespace data JSON. Error: " << e.what();
         return false;
     }
     return true;

components/security_apps/orchestration/orchestration_tools/orchestration_tools.cc

@@ -19,6 +19,7 @@
 #include "cereal/types/vector.hpp"
 #include "cereal/types/set.hpp"
 #include "agent_core_utilities.h"
+#include "namespace_data.h"
 
 #include <netdb.h>
 #include <arpa/inet.h>
@@ -47,11 +48,13 @@ public:
         const string &tenant_id,
         const string &profile_id) const override;
 
+    shared_ptr<ifstream> fileStreamWrapper(const std::string &path) const override;
     Maybe<string> readFile(const string &path) const override;
-    bool writeFile(const string &text, const string &path) const override;
+    bool writeFile(const string &text, const string &path, bool append_mode = false) const override;
     bool removeFile(const string &path) const override;
     bool copyFile(const string &src_path, const string &dst_path) const override;
     bool doesFileExist(const string &file_path) const override;
+    void getClusterId() const override;
     void fillKeyInJson(const string &filename, const string &_key, const string &_val) const override;
     bool createDirectory(const string &directory_path) const override;
     bool doesDirectoryExist(const string &dir_path) const override;
@@ -127,6 +130,98 @@ OrchestrationTools::Impl::fillKeyInJson(const string &filename, const string &_k
 }
 // LCOV_EXCL_STOP
 
+bool
+isPlaygroundEnv()
+{
+    const char *env_string = getenv("PLAYGROUND");
+
+    if (env_string == nullptr) return false;
+    string env_value = env_string;
+    transform(env_value.begin(), env_value.end(), env_value.begin(), ::tolower);
+
+    return env_value == "true";
+}
+
+Maybe<NamespaceData, string>
+getNamespaceDataFromCluster(const string &path)
+{
+    NamespaceData name_space;
+    string token = Singleton::Consume<I_EnvDetails>::by<OrchestrationTools>()->getToken();
+    Flags<MessageConnConfig> conn_flags;
+    conn_flags.setFlag(MessageConnConfig::SECURE_CONN);
+    conn_flags.setFlag(MessageConnConfig::IGNORE_SSL_VALIDATION);
+    auto messaging = Singleton::Consume<I_Messaging>::by<OrchestrationTools>();
+    bool res = messaging->sendObject(
+        name_space,
+        I_Messaging::Method::GET,
+        "kubernetes.default.svc",
+        443,
+        conn_flags,
+        path,
+        "Authorization: Bearer " + token + "\nConnection: close"
+    );
+
+    if (res) return name_space;
+
+    return genError(string("Was not able to get object form k8s cluser in path: " + path));
+}
+
+bool
+doesClusterIdExists()
+{
+    string playground_uid = isPlaygroundEnv() ? "playground-" : "";
+
+    dbgTrace(D_ORCHESTRATOR) << "Getting cluster UID";
+
+    auto maybe_namespaces_data = getNamespaceDataFromCluster("/api/v1/namespaces/");
+
+    if (!maybe_namespaces_data.ok()) {
+        dbgWarning(D_ORCHESTRATOR)
+            << "Failed to retrieve K8S namespace data. Error: "
+            << maybe_namespaces_data.getErr();
+        return false;
+    }
+
+    NamespaceData namespaces_data = maybe_namespaces_data.unpack();
+
+    Maybe<string> maybe_ns_uid = namespaces_data.getNamespaceUidByName("kube-system");
+    if (!maybe_ns_uid.ok()) {
+        dbgWarning(D_ORCHESTRATOR) << maybe_ns_uid.getErr();
+        return false;
+    }
+    string uid = playground_uid + maybe_ns_uid.unpack();
+    dbgTrace(D_ORCHESTRATOR) << "Found k8s cluster UID: " << uid;
+    I_Environment *env = Singleton::Consume<I_Environment>::by<OrchestrationTools>();
+    env->getConfigurationContext().registerValue<string>(
+        "k8sClusterId",
+        uid,
+        EnvKeyAttr::LogSection::SOURCE
+    );
+    I_AgentDetails *i_agent_details = Singleton::Consume<I_AgentDetails>::by<OrchestrationTools>();
+    i_agent_details->setClusterId(uid);
+    return true;
+}
+
+void
+OrchestrationTools::Impl::getClusterId() const
+{
+    auto env_type = Singleton::Consume<I_EnvDetails>::by<OrchestrationTools>()->getEnvType();
+
+    if (env_type == EnvType::K8S) {
+        Singleton::Consume<I_MainLoop>::by<OrchestrationTools>()->addOneTimeRoutine(
+            I_MainLoop::RoutineType::Offline,
+            [this] ()
+            {
+                while(!doesClusterIdExists()) {
+                    Singleton::Consume<I_MainLoop>::by<OrchestrationTools>()->yield(chrono::seconds(1));
+                }
+                return;
+            },
+            "Get k8s cluster ID"
+        );
+    }
+}
+
 bool
 OrchestrationTools::Impl::doesFileExist(const string &file_path) const
 {
@@ -140,7 +235,7 @@ OrchestrationTools::Impl::doesDirectoryExist(const string &dir_path) const
 }
 
 bool
-OrchestrationTools::Impl::writeFile(const string &text, const string &path) const
+OrchestrationTools::Impl::writeFile(const string &text, const string &path, bool append_mode) const
 {
     dbgDebug(D_ORCHESTRATOR) << "Writing file: text = " << text << ", path = " << path;
     if (path.find('/') != string::npos) {
@@ -151,8 +246,15 @@ OrchestrationTools::Impl::writeFile(const string &text, const string &path) cons
             return false;
         }
     }
+
+    ofstream fout;
+
+    if (append_mode) {
+        fout.open(path, std::ios::app);
+    } else {
+        fout.open(path);
+    }
     try {
-        ofstream fout(path);
         fout << text;
         return true;
     } catch (const ofstream::failure &e) {
@@ -186,6 +288,12 @@ OrchestrationTools::Impl::isNonEmptyFile(const string &path) const
     return false;
 }
 
+shared_ptr<ifstream>
+OrchestrationTools::Impl::fileStreamWrapper(const std::string &path) const
+{
+    return make_shared<ifstream>(path);
+}
+
 Maybe<string>
 OrchestrationTools::Impl::readFile(const string &path) const
 {

components/security_apps/orchestration/orchestration_tools/orchestration_tools_ut/orchestration_tools_ut.cc

@@ -1,8 +1,13 @@
 #include "orchestration_tools.h"
 
 #include "cptest.h"
+#include "config_component.h"
 #include "mock/mock_tenant_manager.h"
 #include "mock/mock_shell_cmd.h"
+#include "mock/mock_messaging.h"
+#include "mock/mock_env_details.h"
+#include "mock/mock_agent_details.h"
+#include "mock/mock_mainloop.h"
 
 using namespace std;
 using namespace testing;
@@ -14,6 +19,17 @@ public:
     {
     }
 
+    string
+    getResource(const string &path)
+    {
+        string resource_path = cptestFnameInSrcDir(path);
+        ifstream resource_file(resource_path);
+        EXPECT_TRUE(resource_file.is_open());
+        stringstream resource_file_content;
+        resource_file_content << resource_file.rdbuf();
+        return resource_file_content.str();
+    }
+
     void
     cleanSpaces(string &str)
     {
@@ -47,27 +63,74 @@ public:
 
     OrchestrationTools orchestration_tools;
     I_OrchestrationTools *i_orchestration_tools = Singleton::Consume<I_OrchestrationTools>::from(orchestration_tools);
-    StrictMock<MockTenantManager> mock_tenant_manager;
+    NiceMock<MockMessaging> mock_messaging;
+    NiceMock<MockAgentDetails> mock_agent_details;
+    NiceMock<MockMainLoop> mock_mainloop;
     StrictMock<MockShellCmd> mock_shell_cmd;
+    StrictMock<EnvDetailsMocker> mock_env_details;
+    StrictMock<MockTenantManager> mock_tenant_manager;
+    ::Environment env;
+
 };
 
 TEST_F(OrchestrationToolsTest, doNothing)
 {
 }
 
+TEST_F(OrchestrationToolsTest, getClusterId)
+{
+    EXPECT_CALL(mock_env_details, getToken()).WillOnce(Return("123"));
+    EXPECT_CALL(mock_env_details, getEnvType()).WillOnce(Return(EnvType::K8S));
+    I_MainLoop::Routine routine;
+    EXPECT_CALL(
+        mock_mainloop,
+        addOneTimeRoutine(I_MainLoop::RoutineType::Offline, _, "Get k8s cluster ID", _)
+    ).WillOnce(DoAll(SaveArg<1>(&routine), Return(1)));
+
+    string namespaces = getResource("k8s_namespaces.json");
+    EXPECT_CALL(
+        mock_messaging,
+        sendMessage(
+            true,
+            "",
+            I_Messaging::Method::GET,
+            "kubernetes.default.svc",
+            443,
+            _,
+            "/api/v1/namespaces/",
+            "Authorization: Bearer 123\nConnection: close",
+            _,
+            _
+        )
+    ).WillRepeatedly(Return(Maybe<string>(namespaces)));
+    i_orchestration_tools->getClusterId();
+    routine();
+}
+
 TEST_F(OrchestrationToolsTest, writeReadTextToFile)
 {
-    EXPECT_TRUE(i_orchestration_tools->writeFile(manifest_text, manifest_file));
+    EXPECT_TRUE(i_orchestration_tools->writeFile(manifest_text, manifest_file, false));
     EXPECT_TRUE(i_orchestration_tools->doesFileExist(manifest_file));
     EXPECT_TRUE(i_orchestration_tools->isNonEmptyFile(manifest_file));
+    EXPECT_TRUE(i_orchestration_tools->fileStreamWrapper(manifest_file)->is_open());
     EXPECT_EQ(manifest_text, i_orchestration_tools->readFile(manifest_file).unpack());
 
     EXPECT_FALSE(i_orchestration_tools->isNonEmptyFile("no_such_file"));
 }
 
+TEST_F(OrchestrationToolsTest, writeAndAppendToFile)
+{
+    EXPECT_TRUE(i_orchestration_tools->writeFile("blabla", "in_test.json", false));
+    EXPECT_TRUE(i_orchestration_tools->doesFileExist("in_test.json"));
+    EXPECT_TRUE(i_orchestration_tools->isNonEmptyFile("in_test.json"));
+    EXPECT_TRUE(i_orchestration_tools->writeFile(" Appending Text", "in_test.json", true));
+
+    EXPECT_EQ("blabla Appending Text", i_orchestration_tools->readFile("in_test.json").unpack());;
+}
+
 TEST_F(OrchestrationToolsTest, loadPackagesFromJsonTest)
 {
-    EXPECT_TRUE(i_orchestration_tools->writeFile("blabla", "in_test.json"));
+    EXPECT_TRUE(i_orchestration_tools->writeFile("blabla", "in_test.json", false));
     string file_name = "in_test.json";
     Maybe<map<string, Package>> packages = i_orchestration_tools->loadPackagesFromJson(file_name);
     EXPECT_FALSE(packages.ok());
@@ -83,7 +146,7 @@ TEST_F(OrchestrationToolsTest, loadPackagesFromJsonTest)
 
 TEST_F(OrchestrationToolsTest, copyFile)
 {
-    EXPECT_TRUE(i_orchestration_tools->writeFile("blabla", "in_test.json"));
+    EXPECT_TRUE(i_orchestration_tools->writeFile("blabla", "in_test.json", false));
     EXPECT_TRUE(i_orchestration_tools->copyFile("in_test.json", "cpy_test.json"));
     EXPECT_EQ("blabla", i_orchestration_tools->readFile("cpy_test.json").unpack());
     EXPECT_FALSE(i_orchestration_tools->copyFile("NOT_EXISTS_FILE", "cpy2_test.json"));
@@ -199,7 +262,7 @@ TEST_F(OrchestrationToolsTest, jsonFileToPackages)
                         "        }"
                         "    ]"
                         "}";
-    i_orchestration_tools->writeFile(string_stream.str(), "packages_tmp.json");
+    i_orchestration_tools->writeFile(string_stream.str(), "packages_tmp.json", false);
     Maybe<map<string, Package>> packages = i_orchestration_tools->loadPackagesFromJson("packages_tmp.json");
     EXPECT_TRUE(packages.ok());
     EXPECT_TRUE(packages.unpack().find("nano-agent") != packages.unpack().end());
@@ -222,7 +285,7 @@ TEST_F(OrchestrationToolsTest, packagesToJsonFile)
                         "       }"
                         "   ]"
                         "}";
-    i_orchestration_tools->writeFile(string_stream.str(), "packages.json");
+    i_orchestration_tools->writeFile(string_stream.str(), "packages.json", false);
     Maybe<map<string, Package>> packages = i_orchestration_tools->loadPackagesFromJson("packages.json");
     EXPECT_TRUE(packages.ok());
     EXPECT_TRUE(i_orchestration_tools->packagesToJsonFile(packages.unpack(), "packages.json"));
@@ -277,8 +340,8 @@ TEST_F(OrchestrationToolsTest, deleteVirtualTenantFiles)
     EXPECT_TRUE(i_orchestration_tools->createDirectory(policy_folder_path));
 
     string settings_file_path = conf_path + "/tenant_3fdbdd33_profile_c4c498d8_settings.json";
-    i_orchestration_tools->writeFile(string_stream.str(), settings_file_path);
-    i_orchestration_tools->writeFile(string_stream.str(), policy_file_path);
+    i_orchestration_tools->writeFile(string_stream.str(), settings_file_path, false);
+    i_orchestration_tools->writeFile(string_stream.str(), policy_file_path, false);
 
     EXPECT_TRUE(i_orchestration_tools->doesFileExist(settings_file_path));
     EXPECT_TRUE(i_orchestration_tools->doesFileExist(policy_file_path));
@@ -301,16 +364,16 @@ TEST_F(OrchestrationToolsTest, loadTenants)
     EXPECT_TRUE(i_orchestration_tools->createDirectory(policy_folder_path2));
 
     string settings_file_path1 = conf_path + "/tenant_3fdbdd33_profile_c4c498d8_settings.json";
-    i_orchestration_tools->writeFile(string_stream.str(), settings_file_path1);
+    i_orchestration_tools->writeFile(string_stream.str(), settings_file_path1, false);
 
     string settings_file_path2 = conf_path + "/tenant_123456_profile_654321_settings.json";
-    i_orchestration_tools->writeFile(string_stream.str(), settings_file_path2);
+    i_orchestration_tools->writeFile(string_stream.str(), settings_file_path2, false);
 
     string policy_file_path1 = policy_folder_path1 + "/policy.json";
-    i_orchestration_tools->writeFile(string_stream.str(), policy_file_path1);
+    i_orchestration_tools->writeFile(string_stream.str(), policy_file_path1, false);
 
     string policy_file_path2 = policy_folder_path2 + "/policy.json";
-    i_orchestration_tools->writeFile(string_stream.str(), policy_file_path2);
+    i_orchestration_tools->writeFile(string_stream.str(), policy_file_path2, false);
 
     EXPECT_TRUE(i_orchestration_tools->doesFileExist(settings_file_path1));
     EXPECT_TRUE(i_orchestration_tools->doesFileExist(settings_file_path2));

components/security_apps/orchestration/orchestration_ut/orchestration_multitenant_ut.cc

@@ -62,6 +62,8 @@ public:
             addOneTimeRoutine(I_MainLoop::RoutineType::RealTime, _, "Orchestration runner", true)
         ).WillOnce(DoAll(SaveArg<1>(&routine), Return(1)));
 
+        EXPECT_CALL(mock_orchestration_tools, getClusterId());
+
         EXPECT_CALL(mock_shell_cmd, getExecOutput("openssl version -d | cut -d\" \" -f2 | cut -d\"\\\"\" -f2", _, _))
             .WillOnce(Return(string("OpenSSL certificates Directory")));
 
@@ -91,11 +93,11 @@ public:
         Maybe<string> err = genError("No file exist");
         EXPECT_CALL(mock_orchestration_tools, readFile("/etc/cp/conf/user-cred.json")).WillOnce(Return(err));
 
-        EXPECT_CALL(mock_orchestration_tools, writeFile("This is fake", "/etc/cp/data/data1.a")).WillOnce(
+        EXPECT_CALL(mock_orchestration_tools, writeFile("This is fake", "/etc/cp/data/data1.a", false)).WillOnce(
             Return(true));
-        EXPECT_CALL(mock_orchestration_tools, writeFile("0000 is fake", "/etc/cp/data/data4.a")).WillOnce(
+        EXPECT_CALL(mock_orchestration_tools, writeFile("0000 is fake", "/etc/cp/data/data4.a", false)).WillOnce(
             Return(true));
-        EXPECT_CALL(mock_orchestration_tools, writeFile("This is 3333", "/etc/cp/data/data6.a")).WillOnce(
+        EXPECT_CALL(mock_orchestration_tools, writeFile("This is 3333", "/etc/cp/data/data6.a", false)).WillOnce(
             Return(true));
     }
 

components/security_apps/orchestration/orchestration_ut/orchestration_ut.cc

@@ -54,6 +54,8 @@ public:
             addOneTimeRoutine(I_MainLoop::RoutineType::RealTime, _, "Orchestration runner", true)
         ).WillOnce(DoAll(SaveArg<1>(&routine), Return(1)));
 
+        EXPECT_CALL(mock_orchestration_tools, getClusterId());
+
         EXPECT_CALL(
             mock_shell_cmd,
             getExecOutput("openssl version -d | cut -d\" \" -f2 | cut -d\"\\\"\" -f2", _, _)
@@ -118,11 +120,11 @@ public:
         Maybe<string> err = genError("No file exist");
         EXPECT_CALL(mock_orchestration_tools, readFile("/etc/cp/conf/user-cred.json")).WillOnce(Return(err));
 
-        EXPECT_CALL(mock_orchestration_tools, writeFile("This is fake", "/etc/cp/data/data1.a")).WillOnce(
+        EXPECT_CALL(mock_orchestration_tools, writeFile("This is fake", "/etc/cp/data/data1.a", false)).WillOnce(
             Return(true));
-        EXPECT_CALL(mock_orchestration_tools, writeFile("0000 is fake", "/etc/cp/data/data4.a")).WillOnce(
+        EXPECT_CALL(mock_orchestration_tools, writeFile("0000 is fake", "/etc/cp/data/data4.a", false)).WillOnce(
             Return(true));
-        EXPECT_CALL(mock_orchestration_tools, writeFile("This is 3333", "/etc/cp/data/data6.a")).WillOnce(
+        EXPECT_CALL(mock_orchestration_tools, writeFile("This is 3333", "/etc/cp/data/data6.a", false)).WillOnce(
             Return(true));
     }
 
@@ -1333,26 +1335,6 @@ TEST_F(OrchestrationTest, manifestUpdate)
     } catch (const invalid_argument& e) {}
 }
 
-TEST_F(OrchestrationTest, loadFromOrchestrationPolicy)
-{
-    EXPECT_CALL(
-        rest,
-        mockRestCall(RestAction::ADD, "proxy", _)
-    ).WillOnce(WithArg<2>(Invoke(this, &OrchestrationTest::restHandler)));
-    waitForRestCall();
-    init();
-}
-
-TEST_F(OrchestrationTest, loadFromOrchestrationBackupPolicy)
-{
-    EXPECT_CALL(
-        rest,
-        mockRestCall(RestAction::ADD, "proxy", _)
-    ).WillOnce(WithArg<2>(Invoke(this, &OrchestrationTest::restHandler)));
-    waitForRestCall();
-    init();
-}
-
 TEST_F(OrchestrationTest, getBadPolicyUpdate)
 {
     EXPECT_CALL(
@@ -1815,6 +1797,7 @@ TEST_F(OrchestrationTest, GetRestOrchStatus)
                     "    \"Last update\": \"" + test_str + "\",\n"
                     "    \"Last update status\": \"" + test_str + "\",\n"
                     "    \"Policy version\": \"" + test_str + "\",\n"
+                    "    \"AI model version\": \"" + test_str + "\",\n"
                     "    \"Last policy update\": \"" + test_str + "\",\n"
                     "    \"Last manifest update\": \"" + test_str + "\",\n"
                     "    \"Last settings update\": \"" + test_str + "\",\n"
@@ -1841,6 +1824,7 @@ TEST_F(OrchestrationTest, GetRestOrchStatus)
     EXPECT_CALL(mock_status, getUpdateTime()).WillOnce(ReturnRef(test_str));
     EXPECT_CALL(mock_status, getLastManifestUpdate()).WillOnce(ReturnRef(test_str));
     EXPECT_CALL(mock_status, getPolicyVersion()).WillOnce(ReturnRef(test_str));
+    EXPECT_CALL(mock_status, getWaapModelVersion()).WillOnce(ReturnRef(test_str));
     EXPECT_CALL(mock_status, getLastPolicyUpdate()).WillOnce(ReturnRef(test_str));
     EXPECT_CALL(mock_status, getLastSettingsUpdate()).WillOnce(ReturnRef(test_str));
     EXPECT_CALL(mock_status, getUpgradeMode()).WillOnce(ReturnRef(test_str));

components/security_apps/orchestration/service_controller/service_controller_ut/service_controller_ut.cc

@@ -246,7 +246,8 @@ TEST_F(ServiceControllerTest, UpdateConfiguration)
     EXPECT_CALL(mock_orchestration_tools, jsonObjectSplitter(new_configuration, _, _))
         .WillOnce(Return(json_parser_return));
     EXPECT_CALL(mock_orchestration_tools, doesFileExist(l4_firewall_policy_path)).WillOnce(Return(false));
-    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path)).WillOnce(Return(true));
+    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path, false))
+        .WillOnce(Return(true));
     EXPECT_CALL(mock_orchestration_status,
         setServiceConfiguration("l4_firewall", l4_firewall_policy_path, OrchestrationStatusConfigType::POLICY));
 
@@ -357,8 +358,9 @@ TEST_F(ServiceControllerTest, supportVersions)
         .WillOnce(Return(json_parser_return));
     EXPECT_CALL(mock_orchestration_tools, doesFileExist(policy_versions_path)).WillOnce(Return(false));
     EXPECT_CALL(mock_orchestration_tools, doesFileExist(l4_firewall_policy_path)).WillOnce(Return(false));
-    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path)).WillOnce(Return(true));
-    EXPECT_CALL(mock_orchestration_tools, writeFile(versions, policy_versions_path)).WillOnce(Return(true));
+    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path, false))
+        .WillOnce(Return(true));
+    EXPECT_CALL(mock_orchestration_tools, writeFile(versions, policy_versions_path, false)).WillOnce(Return(true));
     EXPECT_CALL(mock_orchestration_status,
         setServiceConfiguration("versions", policy_versions_path, OrchestrationStatusConfigType::POLICY));
     EXPECT_CALL(mock_orchestration_status,
@@ -455,7 +457,8 @@ TEST_F(ServiceControllerTest, TimeOutUpdateConfiguration)
     EXPECT_CALL(mock_orchestration_tools, jsonObjectSplitter(new_configuration, _, _))
         .WillOnce(Return(json_parser_return));
     EXPECT_CALL(mock_orchestration_tools, doesFileExist(l4_firewall_policy_path)).WillOnce(Return(false));
-    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path)).WillOnce(Return(true));
+    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path, false))
+        .WillOnce(Return(true));
     EXPECT_CALL(mock_orchestration_status,
         setServiceConfiguration("l4_firewall", l4_firewall_policy_path, OrchestrationStatusConfigType::POLICY));
 
@@ -575,7 +578,8 @@ TEST_F(ServiceControllerTest, writeRegisteredServicesFromFile)
     EXPECT_CALL(mock_orchestration_tools, jsonObjectSplitter(new_configuration, _, _))
         .WillOnce(Return(json_parser_return));
     EXPECT_CALL(mock_orchestration_tools, doesFileExist(l4_firewall_policy_path)).WillOnce(Return(false));
-    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path)).WillOnce(Return(true));
+    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path, false))
+        .WillOnce(Return(true));
     EXPECT_CALL(mock_orchestration_status,
         setServiceConfiguration("l4_firewall", l4_firewall_policy_path, OrchestrationStatusConfigType::POLICY));
 
@@ -807,7 +811,8 @@ TEST_F(ServiceControllerTest, SettingsAndPolicyUpdateCombinations)
     EXPECT_CALL(mock_orchestration_tools, jsonObjectSplitter(new_configuration, _, _))
         .WillOnce(Return(json_parser_return));
     EXPECT_CALL(mock_orchestration_tools, doesFileExist(l4_firewall_policy_path)).WillOnce(Return(false));
-    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path)).WillOnce(Return(true));
+    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path, false))
+        .WillOnce(Return(true));
     EXPECT_CALL(mock_orchestration_status,
         setServiceConfiguration("l4_firewall", l4_firewall_policy_path, OrchestrationStatusConfigType::POLICY));
 
@@ -965,7 +970,7 @@ TEST_F(ServiceControllerTest, backup)
     ).WillOnce(Return(true));
     EXPECT_CALL(
         mock_orchestration_tools,
-        writeFile(l4_firewall, l4_firewall_policy_path)).WillOnce(Return(true)
+        writeFile(l4_firewall, l4_firewall_policy_path, false)).WillOnce(Return(true)
     );
     EXPECT_CALL(mock_orchestration_tools, copyFile(policy_file_path, policy_file_path + backup_extension))
         .WillOnce(Return(true));
@@ -1078,7 +1083,7 @@ TEST_F(ServiceControllerTest, backup_file_doesnt_exist)
     ).WillOnce(Return(true));
     EXPECT_CALL(
         mock_orchestration_tools,
-        writeFile(l4_firewall, l4_firewall_policy_path)).WillOnce(Return(true)
+        writeFile(l4_firewall, l4_firewall_policy_path, false)).WillOnce(Return(true)
     );
 
     // backup file doesn't exist so the copyFile function should be called 0 times
@@ -1194,7 +1199,7 @@ TEST_F(ServiceControllerTest, backupAttempts)
 
     EXPECT_CALL(
         mock_orchestration_tools,
-        writeFile(l4_firewall, l4_firewall_policy_path)).WillOnce(Return(true)
+        writeFile(l4_firewall, l4_firewall_policy_path, false)).WillOnce(Return(true)
     );
 
     EXPECT_CALL(mock_orchestration_tools, copyFile(policy_file_path, policy_file_path + backup_extension))
@@ -1311,8 +1316,10 @@ TEST_F(ServiceControllerTest, MultiUpdateConfiguration)
     EXPECT_CALL(mock_orchestration_status,
         setServiceConfiguration("orchestration", orchestration_policy_path, OrchestrationStatusConfigType::POLICY));
 
-    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path)).WillOnce(Return(true));
-    EXPECT_CALL(mock_orchestration_tools, writeFile(orchestration, orchestration_policy_path)).WillOnce(Return(true));
+    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path, false))
+        .WillOnce(Return(true));
+    EXPECT_CALL(mock_orchestration_tools, writeFile(orchestration, orchestration_policy_path, false))
+        .WillOnce(Return(true));
     EXPECT_CALL(mock_orchestration_tools, copyFile(policy_file_path, policy_file_path + backup_extension))
         .WillOnce(Return(true));
     EXPECT_CALL(mock_orchestration_tools, copyFile(file_name, policy_file_path)).WillOnce(Return(true));
@@ -1560,7 +1567,12 @@ TEST_F(ServiceControllerTest, ErrorUpdateConfigurationRest)
     EXPECT_CALL(mock_orchestration_tools, jsonObjectSplitter(new_configuration, _, _))
         .WillOnce(Return(json_parser_return));
     EXPECT_CALL(mock_orchestration_tools, doesFileExist(l4_firewall_policy_path)).WillOnce(Return(false));
-    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path)).WillOnce(Return(true));
+    EXPECT_CALL(
+        mock_orchestration_tools,
+        writeFile(
+            l4_firewall,
+            l4_firewall_policy_path,
+            false)).WillOnce(Return(true));
     EXPECT_CALL(
         mock_orchestration_status,
         setServiceConfiguration("l4_firewall", l4_firewall_policy_path, OrchestrationStatusConfigType::POLICY)
@@ -1667,7 +1679,7 @@ TEST_F(ServiceControllerTest, errorWhileWrtingNewConfiguration)
 
     EXPECT_CALL(
         mock_orchestration_tools,
-        writeFile(l4_firewall, l4_firewall_policy_path)).WillOnce(Return(false)
+        writeFile(l4_firewall, l4_firewall_policy_path, false)).WillOnce(Return(false)
     );
 
     EXPECT_FALSE(i_service_controller->updateServiceConfiguration(file_name, "").ok());
@@ -1782,7 +1794,7 @@ TEST_F(ServiceControllerTest, testMultitenantConfFiles)
 
         EXPECT_CALL(mock_orchestration_tools, doesFileExist(l4_firewall_policy_path_new)).WillOnce(Return(false));
 
-        EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path_new))
+        EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path_new, false))
             .WillOnce(Return(true));
 
         EXPECT_CALL(mock_orchestration_status, setServiceConfiguration(
@@ -1889,7 +1901,7 @@ TEST_F(ServiceControllerTest, test_delayed_reconf)
     EXPECT_CALL(mock_orchestration_tools, jsonObjectSplitter(new_configuration, _, _))
         .WillOnce(Return(json_parser_return));
     EXPECT_CALL(mock_orchestration_tools, doesFileExist(l4_firewall_policy_path)).WillOnce(Return(false));
-    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path)).
+    EXPECT_CALL(mock_orchestration_tools, writeFile(l4_firewall, l4_firewall_policy_path, false)).
         WillOnce(Return(true));
     EXPECT_CALL(mock_orchestration_status,
         setServiceConfiguration("l4_firewall", l4_firewall_policy_path, OrchestrationStatusConfigType::POLICY));

components/security_apps/orchestration/update_communication/CMakeLists.txt

@@ -1,2 +1,2 @@
-add_library(update_communication update_communication.cc hybrid_communication.cc fog_communication.cc fog_authenticator.cc local_communication.cc declarative_policy_utils.cc)
+add_library(update_communication update_communication.cc hybrid_communication.cc fog_communication.cc fog_authenticator.cc local_communication.cc declarative_policy_utils.cc fog_helper_open_source.cc)
 #add_subdirectory(update_communication_ut)

components/security_apps/orchestration/update_communication/declarative_policy_utils.cc

@@ -16,6 +16,7 @@ USE_DEBUG_FLAG(D_ORCHESTRATOR);
 void
 DeclarativePolicyUtils::init()
 {
+    local_policy_path = getFilesystemPathConfig() + "/conf/local_policy.yaml";
     should_apply_policy = true;
     Singleton::Consume<I_RestApi>::by<DeclarativePolicyUtils>()->addRestCall<ApplyPolicyRest>(
         RestAction::SET, "apply-policy"
@@ -25,9 +26,10 @@ DeclarativePolicyUtils::init()
 
 // LCOV_EXCL_START Reason: no test exist
 void
-DeclarativePolicyUtils::upon(const ApplyPolicyEvent &)
+DeclarativePolicyUtils::upon(const ApplyPolicyEvent &event)
 {
     dbgTrace(D_ORCHESTRATOR) << "Apply policy event";
+    local_policy_path = event.getPolicyPath();
     should_apply_policy = true;
 }
 // LCOV_EXCL_STOP
@@ -54,11 +56,9 @@ DeclarativePolicyUtils::getLocalPolicyChecksum()
         return orchestration_tools->readFile("/etc/cp/conf/k8s-policy-check.trigger");
     }
 
-    string policy_path = Singleton::Consume<I_LocalPolicyMgmtGen>::by<DeclarativePolicyUtils>()->getLocalPolicyPath();
-
     Maybe<string> file_checksum = orchestration_tools->calculateChecksum(
         I_OrchestrationTools::SELECTED_CHECKSUM_TYPE,
-        policy_path
+        local_policy_path
     );
 
     if (!file_checksum.ok()) {
@@ -83,8 +83,11 @@ void
 DeclarativePolicyUtils::updateCurrentPolicy(const string &policy_checksum)
 {
     string clean_policy_checksum = getCleanChecksum(policy_checksum);
-    curr_policy = Singleton::Consume<I_LocalPolicyMgmtGen>::by<DeclarativePolicyUtils>()->parsePolicy(
-        clean_policy_checksum
+    auto env = Singleton::Consume<I_EnvDetails>::by<DeclarativePolicyUtils>()->getEnvType();
+    curr_policy = Singleton::Consume<I_LocalPolicyMgmtGen>::by<DeclarativePolicyUtils>()->generateAppSecLocalPolicy(
+        env,
+        clean_policy_checksum,
+        local_policy_path
     );
 }
 
@@ -94,7 +97,7 @@ DeclarativePolicyUtils::getPolicyChecksum()
     I_OrchestrationTools *orchestration_tools = Singleton::Consume<I_OrchestrationTools>::by<DeclarativePolicyUtils>();
     Maybe<string> file_checksum = orchestration_tools->calculateChecksum(
         I_OrchestrationTools::SELECTED_CHECKSUM_TYPE,
-        Singleton::Consume<I_LocalPolicyMgmtGen>::by<DeclarativePolicyUtils>()->getAgentPolicyPath()
+        "/tmp/local_appsec.policy"
     );
 
     if (!file_checksum.ok()) {

components/security_apps/orchestration/update_communication/fog_authenticator.cc

@@ -187,6 +187,8 @@ FogAuthenticator::registerAgent(
         request << make_pair("managedMode", "management");
     }
 
+    request << make_pair("userEdition", getUserEdition());
+
     if (details_resolver->isReverseProxy()) {
         request << make_pair("reverse_proxy", "true");
     }
@@ -207,6 +209,9 @@ FogAuthenticator::registerAgent(
     if (details_resolver->compareCheckpointVersion(8100, std::greater_equal<int>())) {
         request << make_pair("isCheckpointVersionGER81", "true");
     }
+    if (details_resolver->compareCheckpointVersion(8200, std::greater_equal<int>())) {
+        request << make_pair("isCheckpointVersionGER82", "true");
+    }
 #endif // gaia || smb
 
     auto fog_messaging = Singleton::Consume<I_Messaging>::by<FogAuthenticator>();

components/security_apps/orchestration/update_communication/fog_helper_open_source.cc

@@ -0,0 +1,9 @@
+#include "fog_authenticator.h"
+
+#include <string>
+
+std::string
+FogAuthenticator::getUserEdition() const
+{
+    return "community";
+}

components/security_apps/rate_limit/CMakeLists.txt

@@ -0,0 +1,6 @@
+include_directories(../waap/waap_clib)
+include_directories(../waap/include)
+
+add_library(rate_limit_comp rate_limit.cc)
+
+add_library(rate_limit_config rate_limit_config.cc)

components/security_apps/rate_limit/rate_limit.cc

@@ -0,0 +1,535 @@
+#include "rate_limit.h"
+
+#include <memory>
+#include <string>
+#include <vector>
+
+#include "debug.h"
+#include "i_environment.h"
+#include "i_mainloop.h"
+#include "i_time_get.h"
+#include "rate_limit_config.h"
+#include "nginx_attachment_common.h"
+#include "http_inspection_events.h"
+#include "Waf2Util.h"
+#include "generic_rulebase/evaluators/asset_eval.h"
+#include "WaapConfigApi.h"
+#include "WaapConfigApplication.h"
+#include "PatternMatcher.h"
+#include "i_waapConfig.h"
+
+#include <iostream>
+#include <unordered_map>
+#include <string>
+#include <chrono>
+#include <ctime>
+#include <sys/types.h>
+#include <sys/ipc.h>
+#include <sys/shm.h>
+#include <sys/sem.h>
+#include <unistd.h>
+#include <arpa/inet.h>
+
+#include "hiredis/hiredis.h"
+
+USE_DEBUG_FLAG(D_RATE_LIMIT);
+
+using namespace std;
+
+enum class RateLimitVedict { ACCEPT, DROP, DROP_AND_LOG };
+
+class RateLimit::Impl
+    :
+    public Listener<HttpRequestHeaderEvent>
+{
+public:
+    Impl() = default;
+    ~Impl() = default;
+
+    Maybe<string>
+    extractUri(const string &address)
+    {
+        size_t protocolPos = address.find("://");
+        if (protocolPos == std::string::npos) return genError("Invalid URI format: " + address);
+
+        size_t domainPos = address.find('/', protocolPos + 3);
+        if (domainPos == std::string::npos) return string("");
+
+        return address.substr(domainPos);
+    }
+
+    bool
+    isRuleMatchingUri(const string &rule_uri, const string &request_uri, bool should_rule_be_exact_match)
+    {
+        if (rule_uri.find("*") != string::npos) {
+            // first condition is for 'exact match with wildcard'
+            // second is for when the rule serves as a prefix
+            bool wildcard_match =
+                !should_rule_be_exact_match && PatternMatcherWildcard(rule_uri + "*").match(request_uri + "/");
+            wildcard_match |=  PatternMatcherWildcard(rule_uri).match(request_uri);
+            return wildcard_match;
+        }
+
+        return !should_rule_be_exact_match && str_starts_with(request_uri, rule_uri);
+    }
+
+    Maybe<RateLimitRule>
+    findRateLimitRule(const string &matched_uri, string &asset_id)
+    {
+        WaapConfigAPI api_config;
+        WaapConfigApplication application_config;
+        IWaapConfig* site_config = nullptr;
+
+        if (WaapConfigAPI::getWaapAPIConfig(api_config)) {
+            site_config = &api_config;
+        } else if (WaapConfigApplication::getWaapSiteConfig(application_config)) {
+            site_config = &application_config;
+        }
+
+        if (site_config == nullptr) return genError("Failed to get asset configuration. Skipping rate limit check.");
+
+        asset_id = site_config->get_AssetId();
+        ScopedContext rate_limit_ctx;
+        rate_limit_ctx.registerValue<GenericConfigId>(AssetMatcher::ctx_key, site_config->get_AssetId());
+        auto maybe_rate_limit_config = getConfiguration<RateLimitConfig>("rulebase", "rateLimit");
+        if (!maybe_rate_limit_config.ok())
+            return genError("Failed to get rate limit configuration. Skipping rate limit check.");
+
+        const auto &rate_limit_config = maybe_rate_limit_config.unpack();
+        mode = rate_limit_config.getRateLimitMode();
+
+        if (mode == "Inactive") return genError("Rate limit mode is Inactive in policy");
+
+        set<string> rule_set;
+        Maybe<RateLimitRule> matched_rule = genError("URI did not match any rate limit rule.");
+        int rate_limit_longest_match = 0;
+        for (const auto &application_url : site_config->get_applicationUrls()) {
+            dbgTrace(D_RATE_LIMIT) << "Application URL: " << application_url;
+
+            auto maybe_uri = extractUri(application_url);
+            if (!maybe_uri.ok()) {
+                dbgWarning(D_RATE_LIMIT) << "Failed to extract URI from application URL: " << maybe_uri.getErr();
+                continue;
+            }
+
+            string application_uri = maybe_uri.unpack();
+            if (application_uri.back() == '/') application_uri.pop_back();
+
+            for (const auto &rule : rate_limit_config.getRateLimitRules()) {
+                string full_rule_uri = application_uri + rule.getRateLimitUri();
+                int full_rule_uri_length = full_rule_uri.length();
+
+                // avoiding duplicates
+                if (!rule_set.insert(full_rule_uri).second) continue;
+
+                dbgTrace(D_RATE_LIMIT)
+                    << "Trying to match rule uri: "
+                    << full_rule_uri
+                    << " with request uri: "
+                    << matched_uri;
+
+                if (full_rule_uri_length < rate_limit_longest_match) {
+                    dbgDebug(D_RATE_LIMIT)
+                        << "rule is shorter then already matched rule. current rule length: "
+                        << full_rule_uri_length
+                        << ", previously longest matched rule length: "
+                        << rate_limit_longest_match;
+                    continue;
+                }
+
+                if (full_rule_uri == matched_uri ||
+                    full_rule_uri == matched_uri + "/" ||
+                    full_rule_uri + "/" == matched_uri) {
+                    dbgDebug(D_RATE_LIMIT)
+                        << "Found Exact match to request uri: "
+                        << matched_uri
+                        << ", rule uri: "
+                        << full_rule_uri;
+                    return rule;
+                }
+
+                if (rule.getRateLimitUri() == "/") {
+                    dbgDebug(D_RATE_LIMIT)
+                        << "Matched new longest rule, request uri: "
+                        << matched_uri
+                        << ", rule uri: "
+                        << full_rule_uri;
+                    matched_rule = rule;
+                    rate_limit_longest_match = full_rule_uri_length;
+                    continue;
+                }
+
+                if (isRuleMatchingUri(full_rule_uri, matched_uri, rule.isExactMatch())) {
+                    dbgDebug(D_RATE_LIMIT)
+                        << "Matched new longest rule, request uri: "
+                        << matched_uri
+                        << ", rule uri: "
+                        << full_rule_uri;
+                    matched_rule = rule;
+                    rate_limit_longest_match = full_rule_uri_length;
+                }
+            }
+        }
+
+        return matched_rule;
+    }
+
+    EventVerdict
+    respond(const HttpRequestHeaderEvent &event) override
+    {
+        if (!event.isLastHeader()) return INSPECT;
+
+        auto uri_ctx = Singleton::Consume<I_Environment>::by<RateLimit>()->get<string>(HttpTransactionData::uri_ctx);
+        if (!uri_ctx.ok()) {
+            dbgWarning(D_RATE_LIMIT) << "Unable to get URL from context, Not enforcing rate limit";
+            return ACCEPT;
+        }
+
+        string asset_id;
+        auto uri = uri_ctx.unpack();
+        transform(uri.begin(), uri.end(), uri.begin(), [](unsigned char c) { return tolower(c); });
+        auto maybe_rule = findRateLimitRule(uri, asset_id);
+        if (!maybe_rule.ok()) {
+            dbgDebug(D_RATE_LIMIT) << "Not Enforcing Rate Limit: " << maybe_rule.getErr();
+            return ACCEPT;
+        }
+
+        const auto &rule = maybe_rule.unpack();
+        burst = rule.getRateLimit();
+        limit = static_cast<float>(rule.getRateLimit()) / (rule.getRateLimitScope() == "Minute" ? 60 : 1);
+
+        dbgTrace(D_RATE_LIMIT)
+            << "found rate limit rule with: "
+            << rule.getRateLimit()
+            << " per "
+            << (rule.getRateLimitScope() == "Minute" ? 60 : 1)
+            << " seconds";
+
+        auto maybe_source_identifier =
+            Singleton::Consume<I_Environment>::by<RateLimit>()->get<string>(HttpTransactionData::source_identifier);
+        if (!maybe_source_identifier.ok()) {
+            dbgWarning(D_RATE_LIMIT) << "Unable to get source identifier from context, not enforcing rate limit";
+            return ACCEPT;
+        }
+
+        auto &source_identifier = maybe_source_identifier.unpack();
+        dbgDebug(D_RATE_LIMIT) << "source identifier value: " << source_identifier;
+
+        string unique_key = asset_id + ":" + source_identifier + ":" + uri;
+        if (unique_key.back() == '/') unique_key.pop_back();
+
+        auto verdict = decide(unique_key);
+        if (verdict == RateLimitVedict::ACCEPT) {
+            dbgTrace(D_RATE_LIMIT) << "Received ACCEPT verdict.";
+            return ACCEPT;
+        }
+
+        if (verdict == RateLimitVedict::DROP_AND_LOG) sendLog(uri, source_identifier, rule);
+
+        if (mode == "Active") {
+            dbgTrace(D_RATE_LIMIT) << "Received DROP verdict, this request will be blocked by rate limit";
+            return DROP;
+        }
+
+        dbgTrace(D_RATE_LIMIT) << "Received DROP in detect mode, will not block.";
+        return ACCEPT;
+    }
+
+    string
+    getListenerName() const override
+    {
+        return "rate limit";
+    }
+
+    RateLimitVedict
+    decide(const std::string &key) {
+        if (redis == nullptr) {
+            dbgDebug(D_RATE_LIMIT)
+                << "there is no connection to the redis at the moment, unable to enforce rate limit";
+            reconnectRedis();
+            return RateLimitVedict::ACCEPT;
+        }
+
+        redisReply* reply = static_cast<redisReply*>(redisCommand(redis, "EVALSHA %s 1 %s %f %d",
+        rate_limit_lua_script_hash.c_str(), key.c_str(), limit, burst));
+
+        if (reply == NULL || redis->err) {
+            dbgDebug(D_RATE_LIMIT)
+                << "Error executing Redis command: No reply received, unable to enforce rate limit";
+            reconnectRedis();
+            return RateLimitVedict::ACCEPT;
+        }
+
+        // redis's lua script returned true - accept
+        if (reply->type == REDIS_REPLY_INTEGER) {
+            freeReplyObject(reply);
+            return RateLimitVedict::ACCEPT;
+        }
+
+        // redis's lua script returned false - drop, no need to log
+        if (reply->type == REDIS_REPLY_NIL) {
+            freeReplyObject(reply);
+            return RateLimitVedict::DROP;
+        }
+
+        // redis's lua script returned string - drop and send log
+        const char* log_str = "BLOCK AND LOG";
+        if (reply->type == REDIS_REPLY_STRING && strncmp(reply->str, log_str, strlen(log_str)) == 0) {
+            freeReplyObject(reply);
+            return RateLimitVedict::DROP_AND_LOG;
+        }
+
+        dbgDebug(D_RATE_LIMIT)
+            << "Got unexected reply from redis. reply type: "
+            << reply->type
+            << ". not enforcing rate limit for this request.";
+        freeReplyObject(reply);
+        return RateLimitVedict::ACCEPT;
+    }
+
+    void
+    sendLog(const string &uri, const string &source_identifier, const RateLimitRule& rule)
+    {
+        set<string> rate_limit_triggers_set;
+        for (const auto &trigger : rule.getRateLimitTriggers()) {
+            rate_limit_triggers_set.insert(trigger.getTriggerId());
+        }
+
+        ScopedContext ctx;
+        ctx.registerValue<std::set<GenericConfigId>>(TriggerMatcher::ctx_key, rate_limit_triggers_set);
+        auto log_trigger = getConfigurationWithDefault(LogTriggerConf(), "rulebase", "log");
+
+        if (!log_trigger.isPreventLogActive(LogTriggerConf::SecurityType::AccessControl)) {
+            dbgTrace(D_RATE_LIMIT) << "Not sending rate-limit log as it is not required";
+            return;
+        }
+
+        auto maybe_rule_by_ctx = getConfiguration<BasicRuleConfig>("rulebase", "rulesConfig");
+        if (!maybe_rule_by_ctx.ok()) {
+            dbgWarning(D_RATE_LIMIT)
+                << "rule was not found by the given context. Reason: "
+                << maybe_rule_by_ctx.getErr();
+            return;
+        }
+
+        string event_name = "Rate limit";
+
+        LogGen log = log_trigger(
+            event_name,
+            LogTriggerConf::SecurityType::AccessControl,
+            ReportIS::Severity::HIGH,
+            ReportIS::Priority::HIGH,
+            true,
+            LogField("practiceType", "Rate Limit"),
+            ReportIS::Tags::RATE_LIMIT
+        );
+
+        const auto &rule_by_ctx = maybe_rule_by_ctx.unpack();
+
+        log
+            << LogField("assetId", rule_by_ctx.getAssetId())
+            << LogField("assetName", rule_by_ctx.getAssetName())
+            << LogField("ruleId", rule_by_ctx.getRuleId())
+            << LogField("ruleName", rule_by_ctx.getRuleName())
+            << LogField("httpUriPath", uri)
+            << LogField("httpSourceId", source_identifier)
+            << LogField("securityAction", (mode == "Active" ? "Prevent" : "Detect"))
+            << LogField("waapIncidentType", "Rate Limit");
+
+        auto http_method =
+            Singleton::Consume<I_Environment>::by<RateLimit>()->get<string>(HttpTransactionData::method_ctx);
+        if (http_method.ok()) log << LogField("httpMethod", http_method.unpack());
+
+        auto http_host =
+            Singleton::Consume<I_Environment>::by<RateLimit>()->get<string>(HttpTransactionData::host_name_ctx);
+        if (http_host.ok()) log << LogField("httpHostName", http_host.unpack());
+
+        auto source_ip =
+            Singleton::Consume<I_Environment>::by<RateLimit>()->get<IPAddr>(HttpTransactionData::client_ip_ctx);
+        if (source_ip.ok()) log << LogField("sourceIP", ipAddrToStr(source_ip.unpack()));
+
+        auto proxy_ip =
+            Singleton::Consume<I_Environment>::by<RateLimit>()->get<std::string>(HttpTransactionData::proxy_ip_ctx);
+        if (proxy_ip.ok() && source_ip.ok() && ipAddrToStr(source_ip.unpack()) != proxy_ip.unpack()) {
+            log << LogField("proxyIP", static_cast<std::string>(proxy_ip.unpack()));
+        }
+    }
+
+    string
+    ipAddrToStr(const IPAddr& ip_address) const
+    {
+        char str[INET_ADDRSTRLEN];
+        inet_ntop(AF_INET, &(ip_address), str, INET_ADDRSTRLEN);
+        return string(str);
+    }
+
+    Maybe<void>
+    connectRedis()
+    {
+        disconnectRedis();
+
+        redisOptions options;
+        memset(&options, 0, sizeof(redisOptions));
+        REDIS_OPTIONS_SET_TCP(
+            &options,
+            "127.0.0.1",
+            getConfigurationWithDefault<int>(6379, "connection", "Redis Port")
+        );
+
+        timeval timeout;
+        timeout.tv_sec = 0;
+        timeout.tv_usec = getConfigurationWithDefault<int>(30000, "connection", "Redis Timeout");
+        options.connect_timeout = &timeout;
+        options.command_timeout = &timeout;
+
+        redisContext* context = redisConnectWithOptions(&options);
+        if (context != nullptr && context->err) {
+            dbgDebug(D_RATE_LIMIT)
+                << "Error connecting to Redis: "
+                << context->errstr;
+            redisFree(context);
+            return genError("");
+        }
+
+        if (context == nullptr) return genError("");
+
+        redis = context;
+        static string luaScript = R"(
+            local key = KEYS[1]
+            local rateLimit = tonumber(ARGV[1])
+            local burstLimit = tonumber(ARGV[2])
+            local currentTimeSeconds = tonumber(redis.call('time')[1])
+            local lastRequestTimeSeconds = tonumber(redis.call('get', key .. ':lastRequestTime') or "0")
+            local elapsedTimeSeconds = currentTimeSeconds - lastRequestTimeSeconds
+            local tokens = tonumber(redis.call('get', key .. ':tokens') or burstLimit)
+            local was_blocked = tonumber(redis.call('get', key .. ':block') or "0")
+
+            tokens = math.min(tokens + (elapsedTimeSeconds * rateLimit), burstLimit)
+
+            if tokens >= 1 then
+                tokens = tokens - 1
+                redis.call('set', key .. ':tokens', tokens)
+                redis.call('set', key .. ':lastRequestTime', currentTimeSeconds)
+                redis.call('expire', key .. ':tokens', 60)
+                redis.call('expire', key .. ':lastRequestTime', 60)
+                return true
+            elseif was_blocked == 1 then
+                redis.call('set', key .. ':block', 1)
+                redis.call('expire', key .. ':block', 60)
+                return false
+            else
+                redis.call('set', key .. ':block', 1)
+                redis.call('expire', key .. ':block', 60)
+                return "BLOCK AND LOG"
+            end
+        )";
+
+        // Load the Lua script in Redis and retrieve its SHA1 hash
+        redisReply* loadReply =
+            static_cast<redisReply*>(redisCommand(redis, "SCRIPT LOAD %s", luaScript.c_str()));
+        if (loadReply != nullptr && loadReply->type == REDIS_REPLY_STRING) {
+            rate_limit_lua_script_hash = loadReply->str;
+            freeReplyObject(loadReply);
+        }
+
+        return Maybe<void>();
+    }
+
+    void
+    reconnectRedis()
+    {
+        dbgFlow(D_RATE_LIMIT) << "Trying to reconnect to redis after failure to invoke a redis command";
+        static bool is_reconnecting = false;
+        if (!is_reconnecting) {
+            is_reconnecting = true;
+            Singleton::Consume<I_MainLoop>::by<RateLimit>()->addOneTimeRoutine(
+                I_MainLoop::RoutineType::System,
+                [this] ()
+                {
+                    connectRedis();
+                    is_reconnecting = false;
+                },
+                "Reconnect redis",
+                false
+            );
+        }
+
+    }
+
+    void
+    handleNewPolicy()
+    {
+        if (RateLimitConfig::isActive() && !redis) {
+            connectRedis();
+            registerListener();
+            return;
+        }
+
+        if (!RateLimitConfig::isActive()) {
+            disconnectRedis();
+            unregisterListener();
+        }
+    }
+
+    void
+    disconnectRedis()
+    {
+        if (redis) {
+            redisFree(redis);
+            redis = nullptr;
+        }
+    }
+
+    void
+    init()
+    {
+        Singleton::Consume<I_MainLoop>::by<RateLimit>()->addOneTimeRoutine(
+            I_MainLoop::RoutineType::System,
+            [this] ()
+            {
+                handleNewPolicy();
+                registerConfigLoadCb([this]() { handleNewPolicy(); });
+            },
+            "Initialize rate limit component",
+            false
+        );
+    }
+
+    void
+    fini()
+    {
+        disconnectRedis();
+    }
+
+private:
+    static constexpr auto DROP = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_DROP;
+    static constexpr auto ACCEPT = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_ACCEPT;
+    static constexpr auto INSPECT = ngx_http_cp_verdict_e::TRAFFIC_VERDICT_INSPECT;
+
+    string mode;
+    string rate_limit_lua_script_hash;
+    int burst;
+    float limit;
+    redisContext* redis = nullptr;
+};
+
+RateLimit::RateLimit() : Component("RateLimit"), pimpl(make_unique<Impl>()) {}
+
+RateLimit::~RateLimit() = default;
+
+void
+RateLimit::preload()
+{
+    registerExpectedConfiguration<WaapConfigApplication>("WAAP", "WebApplicationSecurity");
+    registerExpectedConfiguration<WaapConfigAPI>("WAAP", "WebAPISecurity");
+    registerExpectedConfigFile("waap", Config::ConfigFileType::Policy);
+    registerExpectedConfiguration<RateLimitConfig>("rulebase", "rateLimit");
+    registerExpectedConfigFile("accessControlV2", Config::ConfigFileType::Policy);
+    registerConfigPrepareCb([]() { RateLimitConfig::resetIsActive(); });
+}
+
+void
+RateLimit::init() { pimpl->init(); }
+
+void
+RateLimit::fini() { pimpl->fini(); }

components/security_apps/rate_limit/rate_limit_config.cc

@@ -0,0 +1,157 @@
+#include "rate_limit_config.h"
+
+bool RateLimitConfig::is_active = false;
+
+void
+RateLimitTrigger::load(cereal::JSONInputArchive &ar)
+{
+    dbgTrace(D_REVERSE_PROXY) << "Serializing single Rate Limit rule's triggers";
+    try {
+        ar(cereal::make_nvp("id", id));
+    } catch (const cereal::Exception &e) {
+        dbgWarning(D_REVERSE_PROXY)
+            << "Failed to load single Rate Limit JSON rule's triggers. Error: " << e.what();
+        ar.setNextName(nullptr);
+    }
+}
+
+void
+RateLimitRule::load(cereal::JSONInputArchive &ar)
+{
+    dbgTrace(D_REVERSE_PROXY) << "Serializing single Rate Limit rule";
+    try {
+        ar(cereal::make_nvp("URI", uri));
+        ar(cereal::make_nvp("scope", scope));
+        ar(cereal::make_nvp("limit", limit));
+        ar(cereal::make_nvp("triggers", rate_limit_triggers));
+    } catch (const cereal::Exception &e) {
+        dbgWarning(D_REVERSE_PROXY) << "Failed to load single Rate Limit JSON rule. Error: " << e.what();
+        ar.setNextName(nullptr);
+    }
+}
+
+void
+RateLimitRule::prepare(const std::string &asset_id, int zone_id)
+{
+    std::string zone_id_s = std::to_string(zone_id);
+    std::string zone;
+    if (isRootLocation()) {
+        zone = "root_zone_" + asset_id + "_" + zone_id_s;
+    } else {
+        std::string zone_name_suffix = uri;
+        std::replace(zone_name_suffix.begin(), zone_name_suffix.end(), '/', '_');
+        zone = "zone" + zone_name_suffix + "_" + zone_id_s;
+    }
+
+    limit_req_template_value = "zone=" + zone + " burst=" + std::to_string(limit) + " nodelay";
+
+    // nginx conf will look like: limit_req_zone <sourceIdentifier> zone=<location>_<id>:10m rate=<limit>r/<scope>;
+    std::string rate_unit = scope == "Minute" ? "r/m" : "r/s";
+    limit_req_zone_template_value =
+        "zone=" + zone + ":" + cache_size + " rate=" + std::to_string(limit) + rate_unit;
+
+    dbgTrace(D_REVERSE_PROXY)
+        << "limit_req_zone nginx template value: "
+        << limit_req_zone_template_value
+        << ", limit_req nginx template value: "
+        << limit_req_template_value;
+}
+
+bool
+RateLimitRule::isRootLocation() const
+{
+    if (uri.empty()) {
+        return false;
+    }
+
+    auto non_root = uri.find_first_not_of("/");
+    if (non_root != std::string::npos) {
+        return false;
+    }
+    return true;
+}
+
+void
+RateLimitConfig::load(cereal::JSONInputArchive &ar)
+{
+    dbgTrace(D_REVERSE_PROXY) << "Serializing Rate Limit config";
+    try {
+        ar(cereal::make_nvp("rules", rate_limit_rules));
+        ar(cereal::make_nvp("mode", mode));
+        prepare();
+    } catch (const cereal::Exception &e) {
+        dbgWarning(D_REVERSE_PROXY) << "Failed to load single Rate Limit JSON config. Error: " << e.what();
+        ar.setNextName(nullptr);
+    }
+}
+
+void
+RateLimitConfig::addSiblingRateLimitRule(RateLimitRule &rule) {
+    rule.setExactMatch();
+    RateLimitRule sibling_rule(rule);
+    sibling_rule.appendSlash();
+    sibling_rule.setExactMatch();
+    rate_limit_rules.push_back(sibling_rule);
+}
+
+void
+RateLimitConfig::prepare()
+{
+    // Removes invalid rules
+    auto last_valid_rule =
+        std::remove_if(
+            rate_limit_rules.begin(),
+            rate_limit_rules.end(),
+            [](const RateLimitRule &rule) { return !rule; }
+        );
+
+    rate_limit_rules.erase(last_valid_rule, rate_limit_rules.end());
+
+    // Removes duplicates
+    sort(rate_limit_rules.begin(), rate_limit_rules.end());
+    rate_limit_rules.erase(std::unique(rate_limit_rules.begin(), rate_limit_rules.end()), rate_limit_rules.end());
+
+    std::for_each(
+        rate_limit_rules.begin(),
+        rate_limit_rules.end(),
+        [this](RateLimitRule &rule) { if (rule.isExactMatch()) { addSiblingRateLimitRule(rule); } }
+    );
+
+    dbgTrace(D_REVERSE_PROXY)
+        << "Final rate-limit rules: "
+        << makeSeparatedStr(rate_limit_rules, "; ")
+        << "; Mode: "
+        << mode;
+
+    setIsActive(mode != "Inactive");
+}
+
+const RateLimitRule
+RateLimitConfig::findLongestMatchingRule(const std::string &nginx_uri) const
+{
+    dbgFlow(D_REVERSE_PROXY) << "Trying to find a matching rat-limit rule for NGINX URI: " << nginx_uri;
+
+    size_t longest_len = 0;
+    RateLimitRule longest_matching_rule;
+    for (const RateLimitRule &rule : rate_limit_rules) {
+        if (rule.getRateLimitUri() == nginx_uri) {
+            dbgTrace(D_REVERSE_PROXY) << "Found exact rate-limit match: " << rule;
+            return rule;
+        }
+
+        if (nginx_uri.size() < rule.getRateLimitUri().size()) {
+            continue;
+        }
+
+        if (std::equal(rule.getRateLimitUri().rbegin(), rule.getRateLimitUri().rend(), nginx_uri.rbegin())) {
+            if (rule.getRateLimitUri().size() > longest_len) {
+                longest_matching_rule = rule;
+                longest_len = rule.getRateLimitUri().size();
+                dbgTrace(D_REVERSE_PROXY) << "Longest matching rate-limit rule so far: " << rule;
+            }
+        }
+    }
+
+    dbgTrace(D_REVERSE_PROXY) << "Longest matching rate-limit rule: " << longest_matching_rule;
+    return longest_matching_rule;
+}

components/security_apps/waap/include/i_waapConfig.h

@@ -52,6 +52,7 @@ public:
     virtual const std::string&   get_RuleName() const = 0;
     virtual const bool&          get_WebAttackMitigation() const = 0;
     virtual const std::string&   get_WebAttackMitigationAction() const = 0;
+    virtual const std::vector<std::string> & get_applicationUrls() const = 0;
 
     virtual const std::shared_ptr<Waap::Override::Policy>& get_OverridePolicy() const = 0;
     virtual const std::shared_ptr<Waap::Trigger::Policy>& get_TriggerPolicy() const = 0;

components/security_apps/waap/waap_clib/WaapConfigBase.cc

@@ -253,6 +253,12 @@ void WaapConfigBase::loadOpenRedirectPolicy(cereal::JSONInputArchive& ar)
 }
 
 
+const std::vector<std::string> &
+WaapConfigBase::get_applicationUrls() const
+{
+    return m_applicationUrls;
+}
+
 void WaapConfigBase::loadErrorDisclosurePolicy(cereal::JSONInputArchive& ar)
 {
     std::string failMessage = "Failed to load the WAAP Information Disclosure policy";

components/security_apps/waap/waap_clib/WaapConfigBase.h

@@ -45,6 +45,7 @@ public:
     virtual const std::string&   get_RuleName() const;
     virtual const bool&          get_WebAttackMitigation() const;
     virtual const std::string&   get_WebAttackMitigationAction() const;
+    virtual const std::vector<std::string> & get_applicationUrls() const;
 
     virtual const std::shared_ptr<Waap::Override::Policy>& get_OverridePolicy() const;
     virtual const std::shared_ptr<Waap::Trigger::Policy>& get_TriggerPolicy() const;

core/agent_core_utilities/agent_core_utilities.cc

@@ -21,6 +21,7 @@
 #include <sys/syscall.h>
 #include <dirent.h>
 #include <sstream>
+#include <algorithm>
 
 #include "debug.h"
 
@@ -49,6 +50,24 @@ exists(const string &path)
     return false;
 }
 
+bool
+isDirectory(const string &path)
+{
+    dbgFlow(D_INFRA_UTILS) << "Checking if path is a directory. Path: " << path;
+    struct stat buffer;
+    if (stat(path.c_str(), &buffer) != 0) {
+        dbgTrace(D_INFRA_UTILS) << "Path does not exists. Path: " << path;
+        return false;
+    }
+
+    if (buffer.st_mode & S_IFDIR) {
+        dbgTrace(D_INFRA_UTILS) << "Path is a directory. Path: " << path;
+        return true;
+    }
+
+    return false;
+}
+
 bool
 makeDir(const string &path, mode_t permission)
 {
@@ -356,4 +375,20 @@ regexReplace(const char *file, int line, const string &sample, const regex &rege
 
 }// namespace Regex
 
+namespace Strings
+{
+
+string
+removeTrailingWhitespaces(string str)
+{
+    str.erase(
+        find_if(str.rbegin(), str.rend(), [] (char c) { return !isspace(c); }).base(),
+        str.end()
+    );
+
+    return str;
+}
+
+} // namespace Strings
+
 } // namespace NGEN

core/agent_core_utilities/agent_core_utilities_ut/agent_core_utilities_ut.cc

@@ -98,8 +98,20 @@ TEST_F(AgentCoreUtilUT, printTest)
     EXPECT_EQ(NGEN::Filesystem::convertToHumanReadable(1024*gigabyte), "1024.00 GB");
 }
 
-
 TEST_F(AgentCoreUtilUT, fileBasenameTest)
 {
     EXPECT_EQ(NGEN::Filesystem::getFileName("/test/base/file/name"), "name");
 }
+
+TEST_F(AgentCoreUtilUT, isDirectoryTest)
+{
+    mkdir("./test", 0400);
+    EXPECT_EQ(NGEN::Filesystem::isDirectory("/test/base/file/name"), false);
+    EXPECT_EQ(NGEN::Filesystem::isDirectory("./test"), true);
+}
+
+TEST_F(AgentCoreUtilUT, removeTrailingWhitespacesTest)
+{
+    string str_with_trailing_whitespace = "str_with_trailing_whitespace\n\n\n\r    \n\n\r";
+    EXPECT_EQ(NGEN::Strings::removeTrailingWhitespaces(str_with_trailing_whitespace), "str_with_trailing_whitespace");
+}