Decrypting SentinelOne Cloud Detection | The Behavioral AI Engine in Real-Time CWPP

In October, the first blog post in this series discussed the Static AI Engine. In this, the second installment of the Detection Engine blog series, we examine the SentinelOne Behavioral AI Engine. Although AI, especially GenAI, are very hot topics right now, SentinelOne has been using AI as a keystone of our technology since our founding in 2013. We hope that this blog series conveys to our customers, prospects, and stakeholders how our AI-powered agent in Singularity Cloud Workload Security is uniquely equipped to create substantial value in delivering real-time cloud workload protection.

Our real-time CWPP solution uses five detection engines, each working to complement the other, to detect runtime threats impacting cloud workloads.

Behavioral AI Engine 101

SentinelOne’s Behavioral AI Engine detects and mitigates previously unknown threats by monitoring kernel process actions and memory usage. This form of AI is not bypassed by malicious countermeasures, and readily identifies sophisticated threats including:

  • Fileless attacks
  • Ransomware, including polymorphic ransomware
  • Zero-day exploits
  • Credential theft
  • Privilege escalation
  • Malicious scripts
  • MITRE tactics and techniques
  • And more.

The Behavioral AI Engine has several characteristics:

  • Autonomous operation: It functions fully with or without an internet connection to the SaaS management console. The intelligence of the engine is built within the agent itself such that there is no round-trip latency to the cloud for analysis.
  • Real-time: The agent monitors all kernel-level processes as they are launched.
  • Post-execution: Unlike the Static AI Engine, which examines files before they are executed, the Behavioral AI Engine is constantly observing all processes as they execute.
  • Storyline™: Our patented visualization technology that tracks what’s happening inside of each cloud workload.

What is Storyline™?

To truly appreciate the inner workings of the Behavioral AI Engine, one must understand the role of SentinelOne’s Storyline technology. Storyline accelerates attack responses, reduces atomic alert noise, and surfaces actionable context for security analysts. Here is how it works.

SentinelOne’s CWPP supports 14 (and counting) Linux distributions and 20 years of Windows Servers. Storyline observes all concurrent kernel processes, malicious and benign. It automatically “connects the dots” (i.e., identifies relationships) between related processes and preserves context such as process metadata. The AI monitors each thread against probabilistic thresholds of normalcy which, when crossed, trigger instantaneous protection against machine-speed attacks.

Because the CWPP agent has the Behavioral AI Engine’s intelligence built-in, the AI makes this judgment autonomously. There is no round-trip latency to the cloud for processing or for human analysis. The AI identifies and stops the spread of machine-speed evil in real-time, at the edge. Process threads, or storylines, deemed suspicious or malicious may be remediated (e.g., process kill and file quarantine) according to policies which are owned and governed by the customer, and which are easily modified via the management console.

Every thread is preserved in the SentinelOne Singularity Data Lake according to the data retention period the customer has selected. Therein, the workload telemetry may be queried, inspected, and used for threat hunting and/or further analysis. The following example walks through a representative behavioral detection and subsequent analysis.

Example: Behavioral Detection of a Python Script

In this example, we have a Kubernetes cluster running in Amazon EKS (Elastic Kubernetes Service), with Singularity Cloud Workload Security for Kubernetes deployed for real-time cloud workload protection.

For illustration purposes, we will launch a shell script via command injection, which will in turn download a python script and initiate a sequence of events and trigger the Behavioral AI Engine.

Additionally, the CWPP response policies are set to Detect Mode for suspicious threats (ie, those which the engine detects with reasonable confidence), and Protect Mode for malicious threats (ie, those which the engine assesses with high confidence). Recall that the algorithms in our detection engines have been trained over the course of several years and hundreds of millions (nearly a billion) of malware samples.


In the following figure, we see that the Behavioral AI Engine was triggered by what it deemed to be a suspicious threat. The details captured include the path to the source process (ie, python), all the command line arguments, which points to a base64-encoded script, the process user, and the originating process, which is containerd.

Behavioral detection triggered by a suspicious threat
Behavioral detection triggered by a suspicious threat

Additional details include:

  • information about the AWS EC2 instance running the k8s cluster (e.g., account ID, region, instance ID, network, tags, etc) on the CLOUD tab,
  • info on the cluster itself (e.g., namespace, pod, and container image) on the KUBERNETES tab, and
  • mapping of telemetry to MITRE TTPs on the THREAT INDICATORS pane.

All of this information can be used to accelerate an incident response investigation.

Rich details in the console for incident response
Rich details in the console for incident response


By clicking on EXPLORE along the top of the management console, the security analyst can look at the process tree which Storyline has automatically assembled. The originating process, the containerd runtime, is shown in blue, indicating that the container runtime itself is not suspicious.

In itself, this is unsurprising, though it does indicate that the suspicious process is containerized. The Behavioral AI Engine first triggers a suspicious alert on the child bash process, which in turn spawns other child processes.

Note that the attack sequence is allowed to continue because the policy is set to Detect Mode. Each event in the sequence has process details recorded by the agent and shown in the right panel. It is worth noting that only an agent can deliver real-time threat detection and kernel process-level visibility, key for subsequent investigation, minimizing dwell time, and balancing risk management.

Exploring the execution chain
Exploring the execution chain

By simply following the visual sequence of events in the chain, each with its forensic details automatically recorded to the Singularity Data Lake by SentinelOne’s real-time CWPP, we quickly come to the python process itself. Here, the command line details shown in the right panel are much more telling. This is the malicious base64-encoded shell.

The security analyst now understands that a threat actor has accessed a specific k8s worker node (name, label, EC2 instance ID, region), installed a containerized web server, and, via command injection, downloaded a shell script which kicked-off a python script that launched base64-encoded web shell.

The explorer displays the base64-encoded command line used by the attacker
The explorer displays the base64-encoded command line used by the attacker

If arriving at that understanding seems like a lot of work, consider that CWPP and Storyline automatically assembled it all in an easy-to-follow sequence, compressing potentially hours of analysis into just a few minutes. And if those events seem like a lot of noise, consider that there is only a single alert per storyline, suppressing noise so that the analyst can focus on root cause analysis.


Now that the attack is understood, the security analyst can initiate a 1-click remediation action in the management console, such as process kill – which stops all processes related to the threat sequence – and file quarantine, to encrypt the threat file and its executables. Recall, for this example, the customer policy was set to Detect Mode. Had the policy been set to Protect Mode, the CWPP solution would have initiated remediation actions (again, governed by policy) when the threat was detected.

The analyst can also open a JIRA or ServiceNow ticket, using the integrations available in Singularity Marketplace, conveniently accessed via the SentinelOne management console. Knowing that this incident impacted a containerized workload running on a managed Kubernetes service (ie, Amazon EKS), it is a good bet that this customer has a solution such as Synk Container to manage vulnerabilities in the workload source code.

By virtue of our CWPP integration with Synk, the runtime threat detection is automatically enriched with details from Snyk about known vulnerabilities it found in the source code. This information can be used to enrich the ticket and route to the appropriate DevOps owner, to investigate and resolve exploited vulnerabilities at the source.

Simple one-click remediation in the console mitigates the threat
Simple one-click remediation in the console mitigates the threat

Now that the suspicious threat has been mitigated, the analyst may wish to query the Singularity Data Lake to see what network activities are associated with this specific storyline. Doing so is as simple as a 1-click pivot on the “Storyline” field in the console, as shown here.

Such a query may reveal a pattern of communication back to specific IP addresses, which can then in turn, be hunted across the rest of the workload telemetry in the data lake, to understand what other activity, if any, the threat actor may have initiated.


If the Static AI Engine is the workhorse of our real-time CWPP solution, then the Behavioral AI Engine is the fusion reactor. By concurrently monitoring hundreds, even thousands, of concurrent kernel process threads, our Behavioral AI Engine is able to recognize when a sequence of related events exceed statistical norms.

In this way, the Behavioral AI Engine detects even the most sophisticated or as yet unknown threats in real time and records extensive attack details so that incident response is streamlined and an in-depth understanding achieved.

To learn more about the value of real-time CWPP in your cloud security stack, head over to the solution homepage, or see how Singularity Cloud Workload Security works with a 2-minute guided walk-through here. And of course, whenever you are ready, you may connect with one of our cloud security experts for a personalized demo.