Detection Engineering Program - Part 4 - Detection Engineering Design Philosophies
Detection Engineering Design Philosophies
At this point in the course, it is useful to take a quick beyond the detection development process and go beyond detection engineering form and process. It is critical to examine the underlying philosophies that make detections more effective, resilient, and operationally valuable.
Some detection design requirements are straightforward. As discussed previously, frameworks such as Palantir’s Alerting and Detection Strategy (ADS) establish a consistent structure for documenting essential detection information, including:
Goal
Categorization
Strategy abstract
Technical context
Blind spots and assumptions
Known false positives
Validation procedures
Alert priority
Response guidance
Additional resources
These elements ensure that a detection is consistently documented, understandable, testable, and actionable. However, a complete template does not automatically produce a good detection.
Effective detection design also requires deeper philosophical decisions about:
Whether to detect temporary artifacts or durable attacker behaviors
How an adversary could modify activity to evade the logic
Which assumptions are supported by evidence
How environmental and attack context should influence an alert
How to balance coverage, precision, analyst workload, and attacker disruption
This section explores these broader detection design philosophies and how they can be used to create detections that are more durable, context-rich, difficult to evade, and valuable to security operations.
Detection Engineering Philosophy: Five Guiding Principles
These principles describe what makes detections effective, independent of any particular development framework, platform, or query language.
1. Detect Durable Adversary Behavior
Focus detections on the capabilities, procedures, and behaviors attackers must perform, rather than only on tools, filenames, hashes, IP addresses, or individual attack instances. Tools and indicators change quickly; underlying procedures are fewer, more stable, and more costly for an attacker to abandon.
2. Think Like an Adversary
Treat every detection as something an attacker will attempt to evade. Ask what alternate commands, APIs, paths, formats, timing, thresholds, or execution methods could produce the same outcome without triggering the rule. A detection should be evaluated not only by what it catches, but also by its predictable false negatives.
3. Replace Assumptions with Evidence
Begin with a clear, testable hypothesis and validate it against the organization’s actual telemetry. Understand both the attack and normal system and user behavior before deciding what is suspicious. Thresholds, exclusions, and conclusions should be supported by measurable environmental evidence, not intuition alone.
4. Detect the Story, Not Just the Event
A single event may be ambiguous, especially when attackers use legitimate tools. Strong detections combine identity, asset, timing, sequence, prevalence, and related activity to explain who did what, where, when, and why it matters. Context should improve confidence and actionability without making the logic unnecessarily brittle.
5. Optimize for Defensive Value
A good detection balances:
Coverage and false-negative risk
Precision and analyst workload
Durability and maintenance cost
Response value and alert volume
Attacker disruption and defender effort
Drilling into Optimizing Defensive Value in detection Engineering
Coverage and False-Negative Risk
Coverage describes the range of malicious behaviors, procedures, systems, and attack variations a detection can identify. A false negative occurs when relevant malicious activity is present but the detection fails to identify it.
A narrowly written rule may detect one known command or tool with high accuracy while missing alternative methods that achieve the same result. Broadening the logic can increase coverage, but it may also capture more legitimate activity.
Design philosophies:
Detect the attacker’s underlying behavior or procedure rather than only one tool or implementation.
Identify alternate commands, APIs, paths, utilities, and execution methods that could accomplish the same objective.
Document what the detection intentionally covers and what remains outside its scope.
Use complementary detections when one rule cannot safely cover every variation.
Treat false-negative analysis as a deliberate engineering activity rather than waiting for an incident to reveal missed activity.
Example: A rule that detects only procdump.exe accessing LSASS has limited coverage. A behavioral detection focused on unusual processes accessing LSASS memory may detect multiple credential-dumping tools and custom implementations.
Guiding question: How could an attacker achieve the same objective without triggering this detection?
Precision and Analyst Workload
Precision is the proportion of generated alerts that represent meaningful suspicious or malicious activity. Low-precision detections create false positives, consume analyst time, and contribute to alert fatigue.
However, making a rule extremely specific can reduce false positives while also making it brittle and easier to evade. The goal is therefore not zero false positives, it is an acceptable balance between detection sensitivity and operational burden.
Design philosophies:
Understand normal system, user, service-account, and administrative behavior before defining suspicious activity.
Add relevant context such as identity, asset criticality, prevalence, location, timing, and process ancestry.
Avoid exclusions that eliminate noise by creating large, unmonitored blind spots.
Use lower-confidence signals as inputs to correlation, risk scoring, or threat hunting instead of sending every signal directly to the SOC.
Evaluate precision according to organizational risk and available analyst capacity.
Example: Alerting on every PowerShell execution would create excessive workload. A more useful detection might consider encoded content, unusual parent processes, rare users, external downloads, or execution on high-value systems.
Guiding question: Does this alert provide enough evidence to justify the analyst time required to investigate it?
Durability and Maintenance Cost
Durability describes how long a detection remains effective as attackers, technologies, applications, and organizational environments change. Detections based on temporary artifacts usually require more maintenance than detections based on stable behaviors.
High durability is desirable, but durable behavioral detections can be more difficult to develop and may require richer telemetry and context.
Design philosophies:
Prefer stable attacker procedures and necessary behaviors over volatile indicators.
Use hashes, IP addresses, domains, filenames, and tool-specific strings when they provide immediate value, but recognize their short operational life.
Avoid unnecessarily hard-coded values, paths, versions, and thresholds.
Build logic that tolerates expected variations without becoming so broad that it loses meaning.
Compare the expected security value of a detection with the cost of testing, tuning, and updating it.
Example: A detection for one malware hash may be precise but becomes obsolete when the malware is recompiled. Detecting the malware’s persistence behavior may remain effective across many variants.
Guiding question:How easily can an attacker or environmental change make this detection obsolete?
Response Value and Alert Volume
A detection has response value when it provides enough reliable information for an analyst or automated system to take a meaningful next step. High alert volume is not evidence of strong coverage if the alerts do not support decisions or action.
Some low-confidence signals still have value, but they may be better retained for correlation, risk scoring, threat hunting, or investigation enrichment rather than presented as independent incidents.
Design philosophies:
Define the expected response before placing a detection into production.
Include the affected identity, host, resource, activity, supporting evidence, and recommended investigation steps.
Match alert priority to the confidence, potential impact, and urgency of the observed behavior.
Aggregate related events into an attack story when individual signals lack sufficient meaning.
Reserve direct SOC alerts for activity that warrants timely human or automated action.
Route lower-value signals into supporting analytic layers instead of discarding them.
Example: Ten separate low-severity alerts may describe one compromised administrator creating a persistence account and assigning it elevated privileges. A consolidated alert presenting the sequence provides greater response value than ten isolated events.
Guiding question: If this detection fires, what decision or action should follow?
Attacker Disruption and Defender Effort
The strongest detections do more than identify one attack instance, they force attackers to change tools, procedures, timing, infrastructure, or objectives. This imposes attacker cost.
Defenders also operate with limited time, staff, telemetry, storage, and computing resources. Detection engineering should therefore favor high-leverage detections that create substantial difficulty for adversaries without imposing unsustainable costs on defenders.
Design philosophies:
Target behaviors attackers must perform to accomplish their objectives.
Prioritize detections that cover multiple tools, procedures, or attack paths.
Consider how an attacker will adapt after encountering the detection.
Build complementary coverage so that evading one detection moves the attacker into another monitored area.
Protect critical assets and high-risk attack paths before pursuing low-impact edge cases.
Compare the attacker effort created with the defender effort required to build and operate the detection.
Example: Detecting a specific credential-dumping utility may cause the attacker to rename or replace the tool. Detecting the underlying credential-access procedure may force the attacker to adopt a more difficult method, accept greater operational risk, or abandon that path.
Guiding question: Does this detection meaningfully increase attacker cost without creating disproportionate cost for the defender?
Overall Principle
Detection quality is not determined by a single metric. A high-value detection achieves an appropriate balance across all five areas:
Detect meaningful attacker behavior, minimize preventable misses, control analyst burden, remain effective over time, support decisive response, and make successful attacks more difficult and expensive.
Review: Detection Engineering Design Philosophies
Five Guiding Principles
Detect durable adversary behavior — Focus on the underlying behaviors and procedures attackers must perform, rather than relying only on temporary indicators, tools, or filenames.
Think like an adversary — Anticipate how attackers could change commands, APIs, paths, timing, thresholds, or execution methods to evade the detection.
Replace assumptions with evidence — Use testable hypotheses, organizational telemetry, and an understanding of normal behavior to support detection logic and thresholds.
Detect the story, not just the event — Combine identity, asset, timing, sequence, prevalence, and related activity to provide meaningful context and distinguish malicious behavior from legitimate operations.
Optimize for defensive value — Balance:
Coverage against false-negative risk
Precision against analyst workload
Durability against maintenance cost
Response value against alert volume
Attacker disruption against defender effort
Essential Design Questions
Before deploying a detection, ask:
What attacker behavior is the detection intended to identify?
How could an attacker accomplish the same objective without triggering it?
What evidence supports the detection’s assumptions and thresholds?
Does the alert provide sufficient context for investigation and response?
Is its defensive value greater than its operational cost?
Will evading it require the attacker to accept greater cost, risk, or constraints?
Detection Engineering Design Philosophies Quiz
Question 1
Which detection is generally more durable?
A. A rule matching one known malware hash
B. A rule matching one command-line string
C. A rule detecting the underlying attacker procedure
D. A rule matching one known malicious IP address
Question 2
What question best demonstrates an adversarial detection-engineering mindset?
A. How many alerts did the rule generate?
B. How could an attacker accomplish the same objective without triggering this rule?
C. Which analyst should own the alert?
D. Which color should represent the alert severity?
Question 3
Why should detection engineers understand normal organizational behavior?
A. To eliminate the need for threat intelligence
B. To identify meaningful deviations and reduce false positives
C. To ensure every unusual event becomes an incident
D. To replace detection testing with analyst experience
Question 4
What does it mean to “detect the story, not just the event”?
A. Alert on every security event independently
B. Replace telemetry with threat-intelligence reports
C. Combine related activity and environmental context into a meaningful attack narrative
D. Only generate alerts after an attack is completed
Question 5
Which option best describes optimizing a detection for defensive value?
A. Maximizing alert volume regardless of analyst capacity
B. Eliminating every false positive, even if important threats are missed
C. Creating the most technically complex detection possible
D. Balancing coverage, precision, durability, response value, and operational cost
Answer Key
C — Attacker procedures and required behaviors are generally more stable than hashes, IP addresses, or individual command strings.
B — Adversarial reasoning examines how an attacker could modify an activity to create a false negative.
B — Understanding normal behavior helps distinguish malicious activity from legitimate but unusual operations.
C — Context-rich detections connect identities, assets, timing, sequence, and related events to explain why activity may be malicious.
D — A high-value detection balances security coverage with accuracy, durability, analyst workload, response value, and maintenance cost.
Comments