Despite implementing multiple solutions to create a defense in depth and following all industry best practices, some of the very best IT security directors will confess that they have a very fundamental problem: They don’t really know how well their security is working.
Until recently, enterprises have had limited capabilities for assessing the damage that a cyberattack could do to their systems. They might have conducted penetration testing, vulnerability assessments, security audits, red team testing or threat hunting, but each of these approaches have limitations that prevent it from providing a comprehensive, ongoing picture of an organization’s overall security posture.
To fill this need, vendors have stepped up with a new type of tool called breach and attack simulation (BAS). This terminology first started gaining attention around 2017, so it is still in the early stages of development. In its 2018 Hype Cycle for Threat-Facing Technologies, Gartner places BAS in the beginning, “on the rise” portion of the cycle.
See our list of top breach and attack simulation vendors and top vulnerability management solutions.
No research has yet been published on the current size of the BAS market. However, because the need for this type of service is so great, the market for BAS tools could become huge very quickly. In its report on the automated breach simulation market, Cyber Research Databank predicted, “The market for automated breach and penetration and simulation [could] reach the size of $1B by 2020. This includes internal developments and open-source tools.” Organizations of any size in any industry could benefit from the software — who couldn’t benefit from finding and plugging as many vulnerabilities as possible?
With that sort of potential, breach simulation is clearly something that IT security managers need to learn more about.
What is breach and attack simulation (BAS)?
Breach and attack simulation (BAS) technology pretends to be an attacker in order to test a network’s cyber defenses. These automated tools run simulated attacks to measure the effectiveness of a company’s prevention, detection and mitigation capabilities. For example, the software might simulate a phishing attack on a company’s email systems, a cyberattack on the company’s web application firewall (WAF), attempted data exfiltration, lateral movement within networks or a malware attack on an endpoint. Most of the tools can run 24×7 in order to provide alerts whenever a change to the network results in potential vulnerabilities or risk. Some provide the ability to run scheduled mock attacks, and some can run surprise mock attacks in order to gauge a security operations center’s capabilities. Some also incorporate artificial intelligence and machine learning capabilities to launch more sophisticated attacks over time or to analyze data on a company’s cybersecurity posture.
Because this technology is very new, the solutions on the market vary widely. Some focus more on breach simulation, primarily attempting to break through network defenses. Others offer more comprehensive attack simulation with the ability to measure an organization’s responses in the exploitation and post-exploitation stages of a cyberattack.
Deployment also varies. Some run in the cloud; some run on-premises. Some require agents, others are agentless. Some of the software as a service (SaaS) solutions can deploy in just minutes, while others may take hours or days. However, most of them begin returning reports in less than an hour after deployment, and some can return reports in just a couple of minutes.
In short, the market is still working to define what exactly constitutes a BAS solution and how it differs from other products on the market.
Breach simulation vs. penetration testing
Some of the confusion around the definition of BAS concerns its relationship to penetration testing. Breach simulation and penetration testing are very similar, but they aren’t exactly the same. Usually penetration testing is conducted by a security expert, a “white hat hacker,” who applies his or her knowledge of how to breach defenses to the task of penetrating an organization’s networks. This approach is very effective because it relies on people who have the same set of skills as the criminals who are conducting cyberattacks.
However, on the downside, penetration testing tends to be very expensive and it only offers a snapshot of an organization’s defenses at a particular point in time. When organizations improve their defenses or as new attacks are seen in the wild, security managers often have no idea how the changes actually affect their security posture unless they pay for more testing. While some very large enterprises have gotten around these disadvantages by hiring in-house cybersecurity experts who can perform pen testing on a regular basis, this isn’t an option for most organizations.
Breach simulation automates the testing process and performs it continuously. While these tools may not have the same creativity and ingenuity as human white hats, they can test all the time across a broad spectrum of different kinds of attacks.
Gartner has noted that BAS tools also have a slightly different scope than pen tests. It wrote, “Penetration testing helps answer the question ‘can they get in?’; BAS tools answer the question ‘does my security work?’”
Complicating matters, some tools combine breach simulation with penetration testing capabilities. Time will tell whether these remain distinct categories or if the two types of solutions conflate.
Why deploy breach and attack simulation technology?
The primary reason organizations would want to deploy BAS is because they want an answer to the fundamental question of whether their systems are secure. Even careful security teams sometimes find that some of their security tools have been inadvertently shut down or are not performing as desired because of configuration errors. Breach and attack simulation can help organizations identify these problems early as part of their proactive security efforts.
In addition, most large organizations use many different security tools that may or may not be operating as intended or working together properly. According to Gartner, “Large enterprises report having 30 to 70 security vendors.” In that type of environment, security tools will be changing almost constantly, as vendors update their tools to adapt to the evolving threat landscape. The only way to know with any level of certainty that a company’s networks and systems are secure as they go through those changes is with some sort of cyberattack simulation testing. And BAS tools represent the most cost-effective way to do that testing on a continual basis.
Of course, it’s not just security tools that are changing constantly. Thanks to the rise of cloud computing and the Internet of Things (IoT), enterprise networks themselves are evolving all the time. Large enterprises may be using computing resources that are scattered all over the globe, and testing this type of environment on a regular basis is difficult with other types of security tools.
Gartner concluded, “BAS tools are the best option when consistent, systematic and frequent tests of production security controls are required.”
Potential disadvantages of breach and attack simulation technology
While BAS tools offer a lot of benefits, they also carry some potential risks. They include the following:
- Prioritization difficulties: In some cases, BAS tools identify so many different vulnerabilities and problems with the security systems that it can be difficult for security teams to know where to begin. Some BAS tools can help by prioritizing fixes.
- Lack of support for zero-day attacks: While human pen testers may come up with a novel approach that no one has ever tried before, automated systems are limited to known attack and threat simulations.
- Potential system disruption: Although BAS tools are meant to simulate an attack, it is sometimes difficult to distinguish a simulated attack from a real one. Organizations face the very real possibility that actions taken by their breach and attack simulation solution could knock production systems offline or slow performance.
- Alert overload: Overworked security personal are already struggling to sort through the deluge of security alerts they receive on a daily basis. Adding BAS to the mix could turn up the volume on this noise, making it more difficult to distinguish the really important alerts from those that can be safely ignored.
- Difficulty in choosing a vendor: Because BAS offerings on the market vary so widely, it can be very difficult for organizations to compare products and services. They need to be very clear about their needs and carefully vet each product they consider, without making assumptions about what the products will be able to deliver.
- Costs: Some of the BAS solutions can be fairly expensive, although some free or low-cost options are available. Most vendors make it difficult or impossible to find out the price for the products without contacting the company.
