Detection Engineering Program - Part 3 - Detection Tracking

Identifying and Tracking Detections

An essential part of improving your detection capabilities is establishing a structured approach to inventory and track your detection rules. The first step is determining where and how detections will be tracked. However, several key considerations must be taken into account:

1. Visibility into Detections – Many cybersecurity tools operate proprietary detection engines, which are not publicly exposed. Tracking every individual rule within tools like Endpoint Detection and Response (EDR) or Intrusion Prevention Systems (IPS) is impractical. Instead, organizations should track these tools as broad detection mechanisms rather than attempting to document each hidden detection rule. However, when testing detections, it is crucial to validate whether these tools successfully detect specific threats. If a tool fails to detect a known attack, engaging with the vendor for insights into their detection logic is recommended.

   
   

2. Tracking Custom Detection Rules – For tools such as SIEM platforms, where detection rules are custom-built, it is crucial to establish a structured tracking mechanism. This involves defining:

   
   

   • Detection format – What query language or framework will be used?

   • Storage location – Where will detection rules be maintained?

   • Additional attributes – What metadata will be associated with each detection?

Defining Detection Formats

Detection tracking typically evolves as an organization matures. The foundational form of detection tracking is the query itself, which varies by platform:

• Splunk: SPL (Search Processing Language)

• Elastic Stack: Query DSL, ES|QL, Kibana queries

• Microsoft Security Tools: KQL (Kusto Query Language)

Example Splunk SPL:

index=sample_data_index | stats sum(bytes) BY host | addinfo

Example Elastic query:

????

Example Microsoft KQL query:

??????

The Rise of Generic Query Syntax

Security analysts often spend significant time crafting detection queries specific to individual SIEM or logging platforms. To streamline this process and promote interoperability, the Sigma project was created (Sigma GitHub).

Sigma provides a vendor-agnostic detection rule format that abstracts query logic into a standardized syntax. This enables easy translation of detection rules across multiple SIEM platforms, reducing duplication of effort and accelerating threat detection development.

???examples of sigma?????

???image??

???example conversions???

Converting rules:

https://detection.studio/

Example Sigma Repo for Rules

https://github.com/SigmaHQ/sigma

???AIs role in conversions???

Microsoft BICEP

Microsoft also support a syntax called BICEP. Bicep is a domain-specific language, streamlines the management of analytic rules in Microsoft Sentinel by providing a declarative syntax for deploying Azure resources, including those related to security monitoring and alerts. This approach simplifies the configuration and deployment of analytic rules, promoting efficiency and consistency.

???[BICEP image]???

sentinel rules : ARM templates:

https://www.infernux.no/MicrosoftSentinel-TemplateAnalyticRules/

Storage and Management of Detection Rules

As organizations mature, their approach to tracking detection rules also evolves:

1. Basic Approach: No formal tracking beyond storing detections within the SIEM itself. This limits visibility and makes rule management inefficient.

2. Manual Tracking: Detections are documented in spreadsheets or wikis, allowing teams to add metadata (e.g., response procedures or threat context). While helpful, manual tracking is prone to inefficiencies and can become difficult to maintain over time.

3. Version-Controlled Tracking: Mature organizations leverage repositories such as GitHub to store detection rules, enabling version control, collaboration, and automation. This approach treats detections as code (Detection-as-Code), allowing for Continuous Integration/Continuous Deployment (CI/CD) practices for managing detection rule lifecycles.

???example of tracking in tool???

???example of tracking in github????

Key Attributes for Detection Tracking

Effective detection tracking involves adding contextual attributes to improve visibility and response effectiveness. Common attributes include:

• Attack Techniques: Mapping detections to specific threats, such as Kerberoasting, and aligning them with the MITRE ATT&CK Framework (e.g., Kerberoasting falls under Credential Access).

• Data Sources: Identifying which logs and telemetry are required for each detection rule to function.

• Detection intervals:  Detection strategy must balance system performance and alert relevance. Running intensive detection queries too frequently can degrade system performance, especially when analyzing large volumes of data. Equally important is understanding the log analysis versus ingestion window. This is the time interval the analytic query is run against the lookback period of the query.

• Detection efficacy: A critical distinction also exists between types of monitors as they relate to response.  Some key concepts here include:

  • Alerting Monitors: These require immediate action. Think of them as the "wake the analyst up at 2 AM" alerts.

  • Analyst Monitors: These are lower-priority signals that suggest unusual behavior or patterns worth reviewing but not urgent. These are better suited for dashboards or periodic reviews and don’t need to run every few minutes. Analyst monitors are closely aligned with anomaly detections, they flag unusual but not necessarily malicious behavior. These detections are designed to surface weak signals or early indicators that warrant further investigation, rather than trigger immediate response.

  • Detection actionability: How clearly and easily an alert guides the next steps. Actionable alerts should provide sufficient context and confidence for analysts to take prompt, informed action.

  • False Positives:  Documenting common false positives is essential to refining detection accuracy. Reducing false positives requires a strategic approach that includes:

    • Enhancing rule logic: Add components to improve specificity and increase the true positive rate.

    • Pattern recognition: Identify and exclude known benign patterns that consistently trigger false alerts.

    • Back-end suppression: Use filtering or indexing to suppress expected false positives without affecting detection coverage.

    • Refactor or retire: If a rule consistently generates excessive noise and cannot be tuned effectively, consider breaking it into smaller rules or deprecating it altogether.

• Priority & Response Procedures: Defining actions to take when a detection is triggered, ensuring quick and effective incident response.

?????examples????

Microsoft example: https://techcommunity.microsoft.com/blog/microsoftsentinelblog/automate-extraction-of-microsoft-sentinel-analytical-rules-from-github-solutions/4411498?utm_source=substack&utm_medium=email