|
|
It’s that time once again. MITRE Engenuity ATT&CK Evaluations 2021 testing has drawn to a conclusion, and technologists worldwide await the results. At SentinelOne, we continue to be enthusiastic supporters for the work MITRE Engenuity is doing to painstakingly define and continually expand a common cybersecurity language that describes how adversaries operate. This matters to you because ATT&CK Evaluations is a unifier and a force multiplier for the people on security’s front line who work tirelessly defending their infrastructure and assets from unscrupulous adversaries looking to turn a quick buck, wreak havoc, or steal a life’s work. When vendors use ATT&CK Evaluations…No wait… when vendors fully adopt ATT&CK Evaluations, their offerings have the potential to make defense and response easier, faster, and more effective.
CISOs, SOC analysts, and architects, this post is for you. It’s SentinelOne’s take on ATT&CK Evaluations 2021, what it means to your organization, and how you can implement it to better understand and use the security tools at your disposal.
What Is the ATT&CK Framework?
In chess, there are three tactical game phases: the opening, middlegame, and endgame. Within each game phase, multiple moves are employed to progress the game from one phase to the next. Players of different skill levels will employ techniques at varying sophistication levels as they work through their strategy to get to checkmate.
In the real world, we deal with adversaries, and each one plays their chess game a little differently. They all use tools. They develop methodologies and approaches toward objectives. They weave legitimate and atypical behaviors into different attack tapestries. And they all know what they’re after.
ATT&CK Evaluations is a way to describe how and why they do what they do. ATT&CK Evaluations framework is “a curated knowledge base and model for cyber adversary behavior, reflecting the various phases of an adversary’s attack lifecycle and the platforms they are known to target.” Its purpose is to be a common language whose components are used in endless combinations to describe how threat actors operate.
Let’s explain. The first key concept here is phases. An adversary performs multiple phases to achieve an objective. Generic example:
Initial access → discovery → lateral movement → collection → exfiltration
In this linear example, the adversary’s ultimate strategy—his objective—is to exfiltrate your data. We can describe his attack methodology in 5 tactical phases, step 1 being initial access through to step 5, the exfiltration. The MITRE Engenuity ATT&CK Evaluations framework currently consists of 14 tactics as seen on the X-axis of the Enterprise navigator tool (hint: click “create new layer” then “enterprise”).
The second key concept from the quote above is behaviors. Behaviors are the moves bad guys utilize against you each step of the way. Behaviors are the techniques they employ within each tactical phase. For example, to achieve initial access (Tactic #1 above), the adversary may send a phishing email with a link to a compromised website that takes advantage of an unpatched browser flaw. The ATT&CK framework currently consists of 200+ techniques (Y-axis of the navigator tool) organized under the 14 tactics.
The next level is the procedures adversaries use. Procedures are the end mechanics within each technique.
Therefore, the end goal requires an initial tactic with one or more techniques, followed by another tactic with its techniques, and so on until the adversary’s objective is met. This layering of general tactics down to specific procedures is where we get TTP: Tactic, Technique, Procedure.
Why Do MITRE Engenuity ATT&CK Evaluations Matter?
MITRE Engenuity ATT&CK Evaluations emulations are constructed to mimic an adversary’s known TTPs and are conducted in a controlled lab environment to determine each participating vendor’s product efficacy. According to MITRE Engenuity,
“The (ATT&CK) evaluations use adversary emulation, which is a way of testing ‘in the style of’ a specific adversary. This allows us to select a relevant subset of ATT&CK techniques to test. To generate our emulation plans, we use public threat intel reporting, map it to ATT&CK, and then determine a way to replicate the behaviors.”
The aim is to put together a complete, logical attack that moves through all the stages of a comprehensive, successful attack from initial compromise to persistence, lateral movement, data exfiltration, and so on.
The ATT&CK framework brings a common lexicon to stakeholders, cyber defenders, and vendors helping us to apply intelligence to cybersecurity operations.
Three benefits ensue:
- We gain insight into the adversary’s game plan in terms of combinations of tactics and techniques.
- We can clearly communicate the exact nature of a threat and respond faster with greater insight.
- When we understand who our typical adversaries are and how they attack us, we can proactively design defenses to blunt them.
MITRE Engenuity points out that it is a “mid-level adversary model”, meaning that it is not too generalized and not too specific. High-level models like the Lockheed Martin Cyber Kill Chain® illustrate adversary goals but aren’t specific about how the goals are achieved. Conversely, exploit and malware databases specifically define IoC “jigsaw pieces” in a giant puzzle but aren’t necessarily connected to how the bad guys use them, nor do they typically identify who the bad guys are. MITRE Engenuity’s TTP model is that happy medium where tactics are the stepwise intermediate goals and the techniques represent how each tactic is achieved.
2021 ATT&CK Evaluations
Since MITRE Engenuity collaborates with vendors during the evaluations, MITRE Engenuity is effectively the red team, while the vendor providing detection and response to MITRE Engenuity is the
blue team
. The result is a “purple team” that helps test security controls in real time by emulating the type of approach that intruders are likely to use in an actual attack based on their known TTPs observed in the wild.
While ATT&CK Evaluation 2019 (the first year of testing) was based on APT3 (Gothic Panda), and ATT&CK Evaluation 2020 focused on TTPs associated with APT29 (Cozy Bear), this year’s evaluations focus on emulating financial threat groups Carbanak and FIN7.
Both Carbanak and FIN7 have a well-documented history of widespread impact. Carbanak is cited with the theft of a cumulative $900M from banks and more than a thousand private customers. FIN7 is said to be responsible for the theft of more than 15 million customer credit card records from victims spanning the globe. The main goal behind its malicious activities is to steal financial assets from companies, such as debit card information, or to get access to financial data through the computers of finance department employees to conduct wire transfers to offshore accounts.
These numbers are what we know. Many incidents go unreported.
Detection Quality
ATT&CK does not score vendors on performance. Instead, the evaluation focuses on how detections occurred as each test moves through its steps. For several years, SentinelOne has underscored what MITRE Engenuity depicts in their evaluation guide—not every detection is of the same quality. It’s pretty clear that whereas a “Telemetry” detection is minimally processed data related to an adversary behavior, at the other end of the quality spectrum, a “Technique” detection is information-rich and orients the analyst at a glance. Consistent technique-driven detections are ideal for organizations that want more out of their tools.
More about detection types:
- Tactic & Techniques: These are the highest quality tool-produced detections. Tactics provide the analyst with “intent of the activity” (why are they doing this? what they are trying to accomplish?). Techniques provide the analyst with “information on how the action was performed or helped answer the question ‘what was done’.”
- General & Telemetry: These are detections further down the quality scale and more simplistic in nature. By themselves, General detections and Telemetry detections provide less context to the analyst and can be thought of as raw data. Note that when vendors are awarded a Technique, they are often also awarded a Telemetry. However, when they are only awarded a simplistic Telemetry (due to the tool’s inability to correlate enough data points), this is not accompanied by the more sophisticated Technique.
- Config Change and Delayed: These are test modifiers. Config Change indicates when a vendor “tweaked” their configuration in the middle of the test. Delayed indicates when a detection was not immediately available to the test proctors due to some delay in processing.
2021 ATT&CK Evaluations also introduced two significant evolutions: Testing on Linux environments, as well as the addition of Protection testing.
The final results are due to be released on April 20, 2021. Until then we wait. A game of chess, anyone?
How Can CISOs Navigate Through Vendor Positioning to Interpret and Understand the Results?
As a CISO, navigating through various vendors’ positions can be a real challenge. Here are a few critical pointers:
- Be wary of excessive misses, delays, and config changes
Vendors that miss a lot of detections…Enough said. Vendors that have lots of delays are getting credit for detections using means typically outside of the tool’s normal workflow, which means your people will have to do the same thing. Vendors with lots of config changes felt the need to modify their detection capabilities in the middle of the test. Try to understand whether these changes are understandable or if the test was being gamed. - Be wary of high Telemetry numbers and low Techniques numbers
Vendors that trumpet their big Telemetry numbers without many Techniques have a tool that does not automate the correlation of events, which means your people will have to do it manually or there may be significant delays and accuracy in connecting the dots. Delays here lead to delays in response, and that leads to more risk. - Be wary of vendors that invent their own scoring systems
We’ve seen many vendors obfuscating poor results with stats and numbers that make them look good but are, in actuality, complete nonsense. Stats like “Context per alert” and “100% Detection” (when there clearly were missed detections) are silly. Read the fine print.
And when it comes to product architectures, CISOs will find these product-centric tenets to be compatible with the spirit of MITRE Engenuity’s objectives:
- EDR Visibility & Coverage Are Table Stakes
The foundation of a superior EDR solution lies in its ability to consume and correlate data at scale in an economic way by harnessing the power of the cloud. Every piece of pertinent data should be captured—with few to no misses—to provide breadth of visibility for the SecOps team. Data, specifically capturing all events, is the building block of EDR and should be considered table stakes and a key MITRE Engenuity metric. - Machine-built Context and Correlation is Indispensable
Correlation is the process of building relationships among atomic data points. Preferably, correlation is performed by the machine and at machine speed, so an analyst doesn’t have to manually stitch data together and waste precious time. Furthermore, this correlation should be accessible in its original context for long periods of time in case it’s needed. - Console Alert Consolidation Is Critical
More signal, less noise is a challenge for the SOC and modern IR teams who face information overload. Rather than getting alerted on every piece of telemetry within an incident and fatiguing the already-burdened SOC team, ensure that the solution automatically groups data points into consolidated alerts. Ideally, a solution can correlate related activity into unified alerts to provide campaign level insight. This reduces the amount of manual effort needed, helps with alert fatigue, and significantly lowers the skillset barrier of responding to alerts. All of this leads to better outcomes for the SOC in the form of shorter containment times and an overall reduction in response times.
How Can CISOs Leverage the ATT&CK Framework In Their Organization?
CISOs and security teams can use the following best practices to improve their security posture:
- Plan a cyber security strategy: Use ATT&CK to plan your cyber security strategy. Build your defenses to counter the techniques known to be used against your type of organization and equip yourself with security monitoring to detect evidence of ATT&CK techniques in your network.
- Run adversary emulation plans: Use ATT&CK for Adversary Emulation Plans to improve Red team performance. Red teams can develop and deploy a consistent and highly organized approach to defining the tactics and techniques of specific threats, then logically assess their environment to see if the defenses work as expected.
- Identify gaps in defenses: ATT&CK matrices can help Blue teams better understand the components of a potential or ongoing cyber attack to identify gaps in defenses and implement solutions for those gaps. ATT&CK documents suggested remediations and compensating controls for the techniques to which you are more prone.
- Integrate threat intelligence: ATT&CK can effectively integrate your threat intelligence into cyber defense operations. Threats can be mapped to the specific attacker techniques to understand if gaps exist, determine risk, and develop an implementation plan to address them.
Conclusion
MITRE Engenuity ATT&CK Evaluations continues its stellar record in pushing the security industry forward and brings much-needed visibility and independent testing to the EDR space. As a security leader or practitioner, it’s important that you move beyond just the numbers game to look holistically at which vendors can provide high visibility and high quality detections while reducing the burden on your security team.
In the short term, we are excited to announce the details of SentinelOne’s participation in the MITRE Engenuity ATT&CK Evaluations 2021, and we will be posting the results when available. In the meantime, if you’d like to learn more about how the SentinelOne Singularity platform can help your organization achieve these goals, contact us for more information or request a free demo.
