With the dwell time of threat actors still hovering between 49 and 99 days, organizations are struggling to detect and contain threats in a timely fashion, which impacts their ability to mitigate the impact of a breach.
Passive techniques that rely on signatures and even behavioural analysis require an attack signature be quantified in advance or sufficient baseline data and deviation from it, in order to automate detection. This is like placing a security guard at the entrance of a building and hoping criminals are wearing a mask. A good security program instead would have the guard patrolling the premises, checking doors and challenging people for identification. This second approach is the cyber security equivalent of proactive threat hunting.
Threat hunting assumes that prevention and detection are not reliable, that breaches are inevitable and threat actors are already inside the perimeter. Instead of waiting for a threat to be detected, threat hunting proactively investigates and analyses activity, looking for indicators of compromise (IoC) that may have been overlooked due to gaps in visibility, detection mechanisms or an attack’s advanced obfuscation and evasion techniques.
Theorize and Hypothesise
Threat Hunting begins with a hypothesis, which may be based on a specific threat actor and their associated Tactics, Techniques and Procedures (TTPs). For example, if an organization operates in the Aerospace or energy sector and is based in the USA, it may be a prime target for the APT33 group. They generally initiate attacks via spear-phishing emails that reference job postings and contain malicious HTML (.hta) files.
Alternatively, a hypothesis may be more general. If an organization stores and processes credit card information, threat actors will target these systems.
The Right Tools for the Job
Threat hunting requires data and access to it. This means that logs and events must be generated, collected, stored and made available in a way that permits searching, correlation, pivoting and analysis. Ideally, data is collected from multiple channels – access, authentication, network, endpoints, etc. – for a multi-dimensional view of activity, which can be correlated for greater visibility.
In the simplest terms, an organization needs a detection stack that covers a variety of data channels, and a way to consolidate, aggregate and analyze the resulting telemetry. The combination of Security Information and Event Management, Endpoint Detection & Response, User & Entity Behaviour Analysis and Network Traffic Analysis can provide these capabilities.
Meanwhile, Security Incident Response Platforms, Threat Intelligence Platforms, Security Automation and Orchestration solutions and dedicated Cyber Threat Hunting tools are emerging technologies that can provide dedicated capabilities to support threat hunting.
Visualization, Statistical Analysis and Machine Learning
Armed with knowledge of a threat or threat actor’s TTPs, an organization can search through its security telemetry data for IoC, related artifacts and anomalous and suspicious activity. While being able to drill down and pivot into data, and perform text searches are important and powerful tools, applying entity relationship mapping and graph visualization techniques can quickly identify activity that would be cumbersome to analyze using the former, more manual approaches.
In addition to visualization, Statistical Analysis and Machine Learning are powerful weapons in the threat hunter’s arsenal. For example, Cluster Analysis, whereby sets of objects are grouped by similar or related attributes and the distance between them can be determined, can help to identify even small divergences from a baseline and show degrees of relationships.
Absence of Evidence Is not Evidence of Absence
Threat Hunting is often compared to unravelling a knot. Once the correct end of the string is identified, the knot will soon come loose. In threat hunting, discovering one IoC can expose related threat crumbs and avenues for investigation.
For example, some threats are not associated with an easily quantifiable IoC, or the malicious activity may appear benign. A sales executive accessing customer data would appear the same whether the authorized user or a cybercriminal who had hijacked the account were performing the activity. However, if the access originates from a different IP-Address than usual, or out of hours, it becomes suspicious.
Art, not Science
Despite the fact that organizations have more and better threat hunting tools and technologies than ever before, investigations still require human analysis to be effective. This is primarily due to the fact that even though threat hunting utilizes scientific methods and approaches, it is more art, than science.
A skilled threat hunter has an intuitive understanding of how attackers work and think, which is constantly enhanced by access to reliable threat intelligence and situational awareness. A hypothesis is often based on a hunch – a skill that has not and may never be codified into machine algorithms. Currently, machines can still only be used to verify hunches.
About Oliver Rochford
Oliver Rochford is the Vice President of Security Evangelism at DFLabs. He previously worked as research director for Gartner, and is a recognized expert on threat and vulnerability management, cyber security monitoring and operations management.