There is a cliché business term that references the success of an information technology goal, business, function, etc comes down to “People, Processes and Technology”. This series is going to look at the People, Processes and Technology that make up a successful Security Operations Center (SOC).
SOC Technology
In SOCs there can be a wide range of technologies. To simplify understanding of SOC technologies it is important to start with the SOC Triad.
The heart of the SOC Triad is a logging system commonly referred to as a SIEM “Security Information & Event Management” system. This centralized system allows for the centralized monitoring and search of the systems telemetry from instrumentation in the environment.
SIEMs are notoriously finicky information technology system. Just do a google search of terms like “why SIEMs fail” and “hate SIEM” and you will see a common theme of SIEMs promising too much and not delivering value to the businesses that deploy them. Yet SIEMs still stand at their position at the top of the SOC Triad, so what is the deal with SIEM?
The breakdown of SIEMs is too complex to cover in a single blog post. In a nutshell SIEMs are not magical boxes. They can be expensive, complex to setup and administer, and provide lower cost-to-value cyber detection than other tools in your cyber security stack.
For a deeper dive into the SIEMs see my blog series, The Log and Pony Show - Security Logging and Information & Event Management (SIEM) systems. https://www.croninity.com/post/the-log-and-pony-show-security-logging-and-information-event-management-siem-systems This blog series does a deep dive into SIEM deployments.
SOC Triad – Network Detection Response (NDR), and Endpoint Detection Response (EDR)
One of the things that is important for SIEMs to be successful is the instrumentation and telemetry feeding it. At a high level there are two high level types of instrumentation systems that produce telemetry data for cyber security. They are Network Detection Response (NDR), and Endpoint Detection Response (EDR). NDR and EDR are the lower corners of the SOC Triad. In the industry NDR is often referred to as NSM "Network Security Monitoring". For more information on the topic of NDR and NSM see the blog series, Network Security Monitoring (NSM) & Network Detection Response (NDR) https://www.croninity.com/post/data-exfiltration-simplified-overview
Other Technologies that Accelerate the SOC Triad
In SOCs there are a few other technologies that are critical to the success of SOCs. These technologies are:
Asset Management
Cyber Threat Intelligence (CTI)
Security Orchestration & Response (SOAR)
Asset Management is foundational to any cyber security program. CTI and SOAR serve as accelerator functions that allow the organizations SOC Triad to function more effectively.
The instrumentation and SIEM within your environment should be connected to CTI which allows for more accurate alerting and better response. SOAR systems allow for the automatic interaction between system that allows pre-defined steps called Playbooks to be dynamically executed (Note: This diagram displays SOAR as integrating connectivity between SOC Triad components. SOAR can also be used to integrate with components outside the SOC Triad including CTI).
For more information related to CTI see my CTI blog series, Cyber Threat Intelligence (CTI) - Overview https://www.croninity.com/post/cyber-threat-intelligence-cti-overview
For more information related to SOAR see my article, Part 7 – Integrating centralized logging/SIEM into work flows – alerting, threat hunting, correlation and SOAR, <link here> that is part of the The Log and Pony Show - Security Logging and Information & Event Management (SIEM) systems blog series <link here>
SOC Processes
Tiers - Do they really matter?
Life is always divided into hierarchical levels and Operations Centers are no different. One framework commonly used in operations centers is 3-Tiered system.
Tier 1 – The least experienced analysts
Tier 2 – Mid level analysts
Tier 3 – Supposed to be expert analysts
How Tiers relate to SOC processes?
An extremely simplified view of cyber security SOC is, something bad happens, the system alerts, and then someone does something about it.
In the real-world things are many dynamics in SOCs that make this more complex than just manning someone in a SOC to provide the proper response action. Two SOC dynamics that have a direct impact to the Tiering system are:
The instrumentation and telemetry can produce alerts or data that indicates alerts when they don’t exist. These are referred to as False Positives.
False positives need additional checks to be performed to determine false positive.
Valid alerts or data producing alerts need action which can often be automated.
From a business perspective you would not want your more experienced people taking their work productivity away from other tasks to do some of this 1st level tracking and verification work. This is where the Tier-1 analyst comes in. Some of the Tier-1 more junior analyst responsibilities include:
Basic triage steps that can’t be automated
Ticketing and communications that can’t be automated
Response steps that can’t be automated
Some level of analysis
This model allows the more experienced Tier-2 and Tier-3 analysts to focus on more high value tasks such as tuning detection rules, creating detection rules, implementing and improving playbooks, performing more advanced analysis, and Threat Hunting (TH).
Without automation all of these things can take a lot of time which is why the goal of the 1st triage level (Pre-Tier-1) of a SOC should always be automation. This is why SOAR is such a big topic area of SOCs. The goal of SOAR is to automate much of the manual work of a SOC:
Populating tickets
Performing verification checks
Collecting additional information from systems
Performing basic response actions
In the diagram below the processes are automated as indicated by the gear and the human analyst is supervising the process rather than performing the process.
Tierless SOCs
One of the issues with Tiers is that you are defining a title that in some senses is locking someone into a pre-defined role. If Tier-1 analyst can do more than what they are tasked to do you are not helping your organization or helping them. In order to keep developing the Tier-1 analyst has to move to an entirely different position. If your organization doesn't have that position they will then move to another organization. The concept of Tierless SOCs is let analysts handle everything from the alert coming into the SOC to more advanced investigations. Tierless SOCs also have the added benefit of removing some of the stratification of work that happens in engineering organizations that is often de-moralizing to coworkers. Stratification of work is where certain analysts/engineers take attitudes that certain work like putting an update in a ticket is beneath them. It is generally found that Tierless SOCs have less turnover and higher employee morale.
SOC People
“Culture guides countless decisions each day by establishing what the right answer looks and feels like in ambiguous situations which are plentiful in a SOC.” (Microsoft security, CISO Series: lessons learned from the Microsoft SOC – Part 1: Organization)
SOC Culture
A few years back researchers at Kansas State University embedded researchers in a SOC to do some research into why SOC burnout is so prevalent. What the researchers found out is that SOC burnout revolved around the virtuous cycle of the Human Capital Cycle.
In the Human Capital Cycle skills leads to empowerment which leads to creativity which leads to growth. It is in stable cycle where positive functions/actions within the SOC can keep analysts on the cycle but negative functions/actions within the SOC can bump analysts off the cycle and leave them without movement in the cycle. The start of the cycle is skills and the SOC has to have opportunities for analysts to develop their skills. many people think skills development is just sending people to expensive cyber security training but there are many other more important factors related to SOC skills development. These factors include:
Working together with teammates and learning from co-workers
Automating routine tasks to have time to develop skills
Empowering analysts with additional levels of responsibility
The full article A Human Capital Model for Mitigating Security Analyst
Burnout can be found here https://www.usenix.org/system/files/conference/soups2015/soups15-paper-sundaramurthy.pdf
SOC Training
People like to learn it provides a sense of accomplishment and leads to empowerment. SOC Training involves:
Self-Learning
See blog article Top Incident Response Books for Cyber Security Professionals https://www.croninity.com/post/top-incident-response-books-for-cyber-security-professionals
See blog article Top Cyber Threat Intelligence (CTI) Books for Cyber Security Professionals https://www.croninity.com/post/top-cyber-threat-intelligence-cti-books-for-cyber-security-professionals
Training Classes
See blog article <affordable cyber security classes>
Testing your SOC
See blog article <testing your SOC with Mitre ATT&CK>
See blog article <Continuous learning in Operations Center through quizzing>
SOC Schedules
One difficult part of running a SOC is manning. The SOC job role is one that is highly controlled which is different than many business jobs that re focused around getting the work done. The SOC analyst has set schedules where they always have to be on the job. If the SOC is 24/7 these schedules often include non-standard business times including nights and weekends. To learn more about operations center scheduling see the blog post
Building and Managing Security Operations Centers (SOCs) - So you want to run 24/7? https://www.croninity.com/post/building-and-managing-security-operations-centers-socs-so-you-want-to-run-24-7
Types of skills needed for SOCs
Some skills that are important for SOC workers are:
A curious mindset to ask the why questions and continue to learn about information technology and cyber security.
Be researched oriented and hypothesis driven in investigations
Be calm in stressful situations
Organizational structures and career pathing
There are some common career paths for SOC analysts.
Becomes a SOC analyst with a higher level of expertise (e.g., Tier-3). Work doing more advanced investigations and digital forensics. Also do proactive Threat Hunting.
Become skilled in the engineering and administration of major SOC tooling like SIEMs, NSM/NDR, and EDR. Take a role in engineering and/or administering those systems.
Supervising and managing other SOC analysts
Any combination of all of the above
There are other career paths including for SOC analysts depending upon the organizations structure and work performed. Some of these other paths include CTI analyst and Governance Risk Compliance (GRC) work related to the SOC and/or vulnerability management.
Comments