Identifying Advanced Persistent Threat Activity
Through Threat-Informed Detection Engineering:
Enhancing Alert Visibility in Enterprises
Author: Eric LeBlanc,
[email protected] Advisor: Lenny Zeltser
Accepted: February 7th, 2025
Abstract
Advanced Persistent Threats (APTs) are among the most challenging to detect in
enterprise environments, often mimicking authorized privileged access prior to their
actions on objectives. Moving within the environment slowly and quietly, APTs can often
persist within the environment for months before detection. There are several approaches
to detecting these adversaries, with many mature enterprises utilizing some combination
of User-Entity Behavior Analytics (UEBA), Risk-Based Alerting (RBA), and traditional
detection engineering practices. However, even these advanced approaches can have
gaps. While they may show anomalous behavior, they can result in false positives,
leading to wasted analyst cycles and potential alert fatigue. To combat this, the question
is asked: does threat modeling prior to detection engineering generate more robust
detections than traditional detection engineering alone? By leveraging the threat
modeling process, enterprises can leverage their existing detection strategies differently,
using information gained from the threat modeling process to alert them with detections
aligning to Tactics, Techniques, and Procedures (TTPs) commonly used together as part
of an intrusion.
Identifying Advanced Persistent Threat Activity through Threat-Informed 2
Detection Engineering: Enhancing Alert Visibility in Enterprises
1. Introduction
Advanced Persistent Threat actors (APTs) have long been difficult to engineer
detection for, requiring an entirely new lexicon to describe their tactics, techniques, and
procedures (TTPs) (Strom, et al., 2020). Leveraging this new lexicon, MITRE’s
ATT&CK Framework for Enterprise (Strom, et al., 2020) allows cyber defense
professionals to have a common language for communicating between offensive,
defensive, and cyber threat intelligence (CTI) professionals. This has allowed for several
new mechanisms to detect the otherwise difficult-to-surface logs generated by
sophisticated, patient attackers.
One of these mechanisms is posed as the thesis of this paper: does threat modeling
prior to detection engineering generate more robust detections than traditional detection
engineering alone? In this instance, threat modeling refers to identifying potential threats
and the hypothetical TTPs they might use against an enterprise environment, as well as
analyzing previously identified intrusion sets that might be relevant to the enterprise.
These intrusion sets and the information required for threat modeling might come from
sources such as MITRE’s ATT&CK program, an industry-based Information Sharing and
Analysis Center (ISAC), enterprise internal threat intelligence teams, or open reporting.
Traditional detection engineering has often focused on specific, signature-based
detections that might be created in a Security Event and Information Management
(SIEM) system. These detections are designed to recognize distinct behaviors, such as a
brute force logon attempt or cross-site scripting attempt on a web server. Each resulting
alert is assigned a criticality rating and a sensitivity level based on the system that
triggered the alert. By surfacing alerts based on the criticality of the detection or the
sensitivity of the entity generating the alert and other historic alerts seen within the
environment, an enterprise might better detect APT-style “low and slow” attacks.
By operationalizing threat modeling from a detection engineering perspective,
using the shared lexicon developed in the MITRE ATT&CK Framework for Enterprise,
detections can be developed based on already existing detections, a type of detection best
categorized as a “meta-detection.” Such detections search for previously logged alerts
within an environment, searching against metadata stored as part of the historic detection
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 3
Detection Engineering: Enhancing Alert Visibility in Enterprises
information, such as what tactic, technique, or procedure a given detection is looking for.
In turn, meta-detections correlate previous detections with each other, seeking to use the
Lockheed Martin Cyber Kill-Chain (Lockheed Martin, 2024) to identify potential follow-
on activity from a known threat actor within an enterprise environment. Using these
meta-detections allows for surfacing alerts based not on just the criticality of the
detection or the sensitivity of the entity generating the alert but also based on other
historic alerts seen within the environment, enabling an enterprise to better detect APT-
style “low and slow” attacks.
Attacker TTPs previously identified in breaches at other similar enterprises,
industries, or government organizations can be used to identify potential adversary
activity and – if a match is found – disrupt, deny, or degrade that activity before the
adversary can achieve their actions on objectives.
2. Research Method
As part of an experimental study, a series of meta-detections have been developed
based on the tactics, techniques, and procedures (TTPs) associated with the group
designated by MITRE as G0016, commonly referred to in the literature as APT29
(MITRE Corporation, 2024). Meta-detections are a form of detection mechanism that
analyzes previously logged alerts within an environment by examining metadata
associated with historical detection records. This includes information such as the
specific TTPs targeted by a given detection or the phase of the attack chain to which
those TTPs correspond. An example of the type of metadata meta-detections look at can
be seen in Figure 1 below, along with an example of the detection logic used in a
“traditional” detection; in this case, the traditional detection is looking for brute force
access behavior, while the metadata we might search for using a meta-detection includes
the MITRE ATT&CK Technique ID. These meta-detections were compared against
using traditionally developed detections in conjunction with Risk-Based Alerting, which
focuses on specific individual TTPs and assigns them a specific “risk value” to quantify
accumulated risk but has no underlying associations between detections aside from the
entity (user or system) that triggered the alert.
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 4
Detection Engineering: Enhancing Alert Visibility in Enterprises
Figure 1: Example of Operationalizing MITRE ATT&CK Within Detections
2.1. Experimental Environment
The meta-detections and traditional signature-based detections were tested using
AttackIQ’s Breach and Attack Simulation (BAS) software within an enterprise
environment on representative (non-production) systems, using a full suite of security
tooling. BAS software was chosen for its ability to emulate threat actor behavior,
including scripted content, should a specific test or scenario fail to function due to
security tooling, but is not specific to AttackIQ’s product; similar functionality exists
within the open-source community (MITRE Caldera) or from other vendors (Cymulate,
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 5
Detection Engineering: Enhancing Alert Visibility in Enterprises
SafeBreach). The full security tooling stack was chosen to simulate an enterprise
environment’s security controls best and to provide multiple telemetry sources for
detections to alert against. Telemetry sources used for alerting include Endpoint
Detection and Response, Operating System, Network Intrusion Detection, Firewall, and
HTTPS Proxy logs. These logs are all normalized to the Splunk Common Information
Model (CIM) data model standard, allowing for detections to be developed using
standardized Splunk Processing Language (SPL) for the enterprise Security Information
and Event Management (SIEM) environment, Splunk Enterprise Security. While Splunk
specifically was used within the experimental environment, the capabilities demonstrated
are not unique to the tooling used and can be developed using other products such as
ELK-SOF or other SIEM products.
2.2. Experimental Design
Meta-detection efficacy was tested by running a scenario using a common
progression of TTPs from APT 29 using BAS software. The scenario was run first using
only traditionally developed detections in conjunction with existing risk scoring within
the enterprise SIEM for the test environment. Following the first run of the scenario, a
second run was conducted using the same detections as the first time, but also with the
meta-detections active. From there, false positive rates for both methods were compared
to the existing baseline rate within the environment, along with the time to detection
between the two scenario runs to find what difference, if any, using meta-detections
produced.
3. Findings and Discussion (Exposition of the Data)
The representative systems identified in the experimental environment were tested
using Breach and Attack Simulation (BAS) software, configured to emulate an intrusion
from the group tracked as APT 29. This scenario configuration overview is below in
Figure 2, and additional information on the individual tests within each tactic can be
found in the Appendix. This set both a baseline level of detection within the environment
and allowed a direct comparison of detections with Risk-Based Alerting versus
detections, Risk-Based Alerting, and meta-detections.
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 6
Detection Engineering: Enhancing Alert Visibility in Enterprises
Figure 2: AttackIQ APT 29 Scenario Configuration
3.1. Baseline Detection Rate
Detection rates for the standard detections in conjunction with Risk-Based
alerting were approximately 33% of all tests, with 75% of exfiltration, 63% of
persistence, and 60% of privilege escalation detected with the baseline detections, and
can be seen in the screenshot from the BAS software in Figure 3, below.
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 7
Detection Engineering: Enhancing Alert Visibility in Enterprises
Figure 3: Baseline Detections by MITRE ATT&CK Tactic
3.2. Meta-Detection Enhanced Detection Rate
Surprisingly, in this experiment, neither the detection speed nor the accuracy of the test
incident was affected compared to the default detections in conjunction with Risk-Based
Alerting. Both were detected within approximately 10 minutes of beginning. However,
the meta-detection did have the benefit of accurately attributing the activity to APT 29 or
emulation thereof. In contrast, the traditional Risk-Based Alerting had initial attribution
to the group APT 34, also referred to as OilRig (MITRE Corporation, 2024). An example
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 8
Detection Engineering: Enhancing Alert Visibility in Enterprises
the meta-detection and results generated by it can be seen below in Figure 4.
Figure 4: Splunk Processing Language and Example Results from Meta-
Detection
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 9
Detection Engineering: Enhancing Alert Visibility in Enterprises
Compared to traditional detections with Risk-Based Alerting, there was no substantive
difference in false positive or false negative rate between the two methodologies (see data
in Appendix 6.2); Risk-Based Alerting continues to have a strong resistance to false
positives and false negatives once the threshold is set at the correct level for the
enterprise based on previous baselining conducted as part of the original implementation,
while the meta-detection does not alert unless multiple TTPs matching the threat model
are seen. A confounding variable for the lack of difference between the meta-detections
and traditional detections could be that this experiment took place over the course of
days, rather than the weeks or even months that some APT groups operate at. APT
groups are known for their long dwell times, patient behavior, and ability to blend into
the background noise of an enterprise (Strom, et al., 2020)
While there was no substantive difference found between Risk-Based Alerting
and the proposed meta-detection methodology in time to detect or in false-positive rates,
it should be noted that the meta-detection methodology does allow for better positive
identification of previously seen threat actors within an environment. This could be a
useful methodology for enterprises having recurrent intrusions from the same threat actor,
or for differentiating multiple separate intrusions within an environment, assuming there
was minimal overlap in the groups’ TTPs.
4. Recommendations and Implications for Future
Research
The following recommendations are specifically for enterprises with a robust
cyber security program, with mature implementations of the various controls required for
their industry. These should not be implemented prior to having a mature environment
complete with an existing Security Operations Center (SOC) and Incident Response (IR)
team to screen and respond to high-criticality alerts, an existing Cyber Threat Intelligence
(CTI) program necessary to conduct the threat modeling required, and a mature Detection
Engineering (DE) practice capable of covering baseline gaps in alert coverage required
prior to building out extensive meta-detections called for from threat modeling exercises.
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 10
Detection Engineering: Enhancing Alert Visibility in Enterprises
4.1. Recommendations for Implementation
Meta-detections should be implemented with a holistic approach based on the
enterprise’s risk posture. This requires multiple components of cyber security
departments to work in conjunction with one another at all implementation phases;
components must have strong communication lines between them. The following
sections contain steps developed in conducting the experiment within the enterprise
environment in order to implement functional meta-detections in conjunction with Risk-
Based Alerting.
4.1.1. Threat Modeling
The first phase of implementation requires the Cyber Threat Intelligence group
within the enterprise to conduct a thorough threat modeling exercise, modeling the
groups most likely to attempt an intrusion into the enterprise. Depending on the industry,
this may be as few as five or as many as twenty different actors. Once those groups have
been identified, the groups’ known and suspected tradecraft should be decomposed into
MITRE ATT&CK TTPs per actor. Following this, DE must collaborate with CTI to
identify existing detections that map to the various TTPs identified by CTI and any gaps
that may exist in the current detection environment. Once the required detections have
been identified, DE should confirm that all identified detections are annotated within the
Security Information and Event Management system with the respective MITRE
ATT&CK tactics, techniques, and procedures corresponding to the detected behavior.
Additional alert fidelity can also be gained by mapping TTPs to the Lockheed Martin
Cyber Kill-Chain at this phase. Following this, the groups shift into the second
implementation phase.
4.1.2. Meta-Detection Engineering
Meta-detection engineering begins with the threat model developed in the first
phase and the identified detections for the TTPs contained within the threat model. In
conjunction with the Security Operations Center and the Cyber Threat Intelligence
groups, Detection Engineering creates the initial meta-detections for the enterprise using
the existing detections identified in the first implementation phase. This can be done
based on a few different mechanisms, whether a progression of MITRE ATT&CK TTPs
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 11
Detection Engineering: Enhancing Alert Visibility in Enterprises
commonly seen in an intrusion, the Lockheed Martin Cyber Kill-Chain phases
corresponding to TTPs, or a combination of the two. The meta-detection engineering
portion should be informed based on the feedback from the SOC and the input from CTI
that engineering started with.
4.2. Implications for Future Research
While the meta-detections did not accelerate detection speed within the enterprise
environment, they correctly identified that intrusion activity was occurring. In addition,
due to the time constraints of this experiment, the benefits of meta-detections might not
be readily apparent since the tests were conducted over a matter of days rather than the
usual weeks and months that APT adversaries often employ in enterprise breaches
(Strom, et al., 2020). Future research should consider this and take a more longitudinal
approach to testing, preferably closer to the average dwell time in other enterprise
breaches.
5. Conclusion
Based on the data from this experiment, performing threat-modeling and
developing meta-detections may not provide more robust detections than traditional
detection engineering alone. While no regressions were introduced (measured utilizing
false positive/false negative rate), the meta-detections did not surface any additional
alerts or surface alerts faster than Risk-Based Alerting did. Despite unexpected results,
this research has shown that while detection speed has not improved, there may still be
merit in exploring different mechanisms to surface alerts suspected of APT activity. The
short timeframe of the experiment may be a confounding factor as to why the traditional
detections plus Risk-Based Alerting were performed to the same level as the meta-
detections.
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 12
Detection Engineering: Enhancing Alert Visibility in Enterprises
References
Bhattacharjee, J., Sengupta, A., & Mazumdar, C. (2014). A Formal Methodology for
Modeling Threats to Enterprise Assets. Information Systems Security, 149-
166.
Kuppa, A., Aouad, L., & Le-Khac, N. (2021). Linking CVEs to MITRE ATT&CK
Techniques. Proceedings of the 16th Internal Conference on Availability,
Reliability, and Security.
Lockheed Martin. (2024, 12 2). Lockheed Martin Cyber Kill-Chain. Retrieved from
Lockheed Martin Corporate Website: https://www.lockheedmartin.com/en-
us/capabilities/cyber/cyber-kill-chain.html
MITRE Corporation. (2024, 12 2). APT 29. Retrieved from MITRE ATT&CK:
https://attack.mitre.org/groups/G0016/
MITRE Corporation. (2024, 12 04). OilRig. Retrieved from MITRE ATT&CK:
https://attack.mitre.org/groups/G0049/
Straub, J. (2020). Modeling Attack, Defense, and Threat Trees and the Cyber Kill
Chain, ATT&CK, and STRIDE Frameworks as Blackboard Architecture
Networks. 2020 IEEE International Conference on Smart Cloud.
Strom, B., Applebaum, A., Miller, D., Nickels, K., Pennington, A., & Thomas, C. (2020).
MITRE ATT&CK: Design and Philosophy. MITRE Corp.
Williams, L., McGraw, G., & Migues, S. (2018). Engineering Security Vulnerability
Prevention, Detection, and Response. IEEE Software, 35(5), 76-80.
Xiong, W., & Lagerström, R. (2019). Threat Modeling -- A Systematic Literature
Review. Computers & Security, 84, 53-69.
Xiong, W., Åberg, O., & Lagerström, R. (2021). Cyber Security Threat Modeling Based
on the MITRE Enterprise ATT&CK Matrix. Software and Systems Modeling,
21(1), 157-177.
Zhang, J., Zheng, J., Chen, T., Tan, Y., Zhang, Q., & Li, Y. (2024). ATT&CK-based
Advanced Persistent Threat Attacks Risk Propagation Assessment Model for
Zero Trust Networks. Computer Networks.
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 13
Detection Engineering: Enhancing Alert Visibility in Enterprises
Appendix: BAS Execution Tests Used by MITRE
ATT&CK Tactic
Figure 5: Individual Execution Tests Used
Figure 6: Individual Persistence Tests Used
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 14
Detection Engineering: Enhancing Alert Visibility in Enterprises
Figure 7: Individual Privilege Escalation Tests Used
Figure 8: Individual Defense Evasion Tests Used
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 15
Detection Engineering: Enhancing Alert Visibility in Enterprises
Figure 9: Individual Credential Access Tests Used
Figure 10: Individual Discovery Tests Used
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 16
Detection Engineering: Enhancing Alert Visibility in Enterprises
Figure 11: Individual Lateral Movement Tests Used
Figure 12: Individual Collection Tests Used
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 17
Detection Engineering: Enhancing Alert Visibility in Enterprises
Figure 13: Individual C2 and Exfiltration Tests Used, Pt 1
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 18
Detection Engineering: Enhancing Alert Visibility in Enterprises
Figure 14: Individual C2 and Exfiltration Tests Used, Pt 2
Figure 15: Individual C2 and Exfiltration Tests Used, Pt 3
Author Name, email@address
Identifying Advanced Persistent Threat Activity through Threat-Informed 19
Detection Engineering: Enhancing Alert Visibility in Enterprises
Experimental Data
Simulation Run Time to Detect False Positive False Negative
Rate Rate
Baseline 601 seconds 5% 11.5%
Risk-Based Alerting 610 seconds 2% 0%
Meta-Detections 611 seconds 2% 0%
Author Name, email@address
