OT Pen Testing: Worth the Risk?

Introduction

In the rapidly evolving world of cybersecurity, it's widely understood that many Operational Technology (OT) cybersecurity strategies have been inherited from the Information Technology (IT) domain. This transference, while logical, brings with it a host of assumptions and practices that may not fully align with the unique demands of OT systems. Particularly in the realm of penetration testing, we often see this crossover in the form of IT tools and methodologies being applied, although cautiously, to OT environments. However, this approach, while not inherently flawed, raises some critical concerns.

From my research and observations of industry practices, it becomes evident that the standard IT-centric penetration testing methods, when applied to OT, often fall short of addressing the full spectrum of OT-specific challenges. The reliance on IT tools and methods, despite being implemented with added caution, may not sufficiently grasp the potential impacts unique to OT systems. This isn't to say that the adoption of IT methodologies in OT is misplaced; rather, it highlights the need for a more tailored approach.

In evaluating OT penetration testing, it is essential to distinguish between critical and less-critical OT systems. This differentiation calls for a categorization that tailors approaches and techniques to each environment's specific needs. But beyond categorization, we must also question the very objectives of penetration testing in OT contexts. Is the goal solely to identify vulnerabilities, or does it extend to ensuring the continued safe and reliable operation of critical infrastructure? And in pursuit of these objectives, could there be safer methodologies?

In this article, we delve deep into the critical concerns surrounding penetration testing in OT environments. We explore the unique challenges that OT systems present, questioning the suitability and safety of current penetration testing practices. Instead of advocating for distinct approaches for different OT system categories, we emphasize the need for extreme caution, examining whether the goals of penetration testing in OT are achievable without compromising system integrity and operational continuity. 

OT Pen Test is a field that demands a high degree of specialization and an acute understanding of the operational, safety, and regulatory aspects of OT environments. The challenges stem from a mix of misconceptions, a lack of specialized knowledge, and the application of IT-centric approaches in a domain where they may not be wholly applicable.

However, if OT Pen Test is mandatory, it is ideally not a starting point but rather a choice that should be made, at a more mature stage of the organization’s cybersecurity posture.

Have you completed the Foundational Security Assessment?

Before conducting penetration testing, it's crucial for an OT environment to undergo a comprehensive security assessment. This assessment should identify existing vulnerabilities, evaluate current security measures, and understand the specific operational context. It lays the groundwork for any advanced testing like penetration testing.

Have you developed a Cybersecurity Maturity Model?

Penetration testing is more effective when an organization has a certain level of maturity in its cybersecurity practices. This maturity is not just about having advanced technologies in place but also includes having well-established policies, procedures, incident response plans, and a culture of security awareness.

Asking this question for the following reasons:

1. Mature organizations are better equipped to understand and manage the risks associated with penetration testing in OT environments. These risks include potential disruptions to operations, safety implications, and regulatory compliance issues.

2. Mature organizations are more likely to have the resources and processes to act on a penetration test's findings effectively. This includes remediating identified vulnerabilities, enhancing security controls, and implementing strategic improvements.

3. Penetration testing can be resource-intensive and costly. Organizations higher on the maturity model are more likely to weigh the costs against the potential benefits effectively, ensuring that penetration testing provides value commensurate with the investment.

4. Organizations with a higher maturity level are usually better prepared to simulate and defend against advanced threats during penetration testing. This is crucial in OT environments where the stakes are high.

In my experience, I have noticed that discussions around OT Pen Test are not as frequent as they should be. As a result, I get the feeling that sometimes the objective of conducting an OT Pen Test is the test itself, rather than the identification of vulnerabilities and potential security threats. However, I could be wrong in my assessment and I am open to hearing differing perspectives on this topic, but bear with me as I share my concerns:

1. How the service is being promoted?

There is a significant concern regarding OT Pen Test, as most of the available information on this topic is provided by service providers who are selling these services. Consequently, the information provided often presents penetration testing in a way that is biased towards making the service more attractive to customers, without fully addressing the complexities and risks involved. Although some service providers may not do this intentionally, there is a tendency to overlook the limitations and potential downsides of such testing, which can create misconceptions about what OT Pen Test implicates and its true value. I'm sure you come across claims such as zero downtime during testing, guaranteed vulnerability detection, OT-tailored pen tests, non-intrusive pen tests, and more.

2. Testing IT components with OT systems

In many cases, the penetration testing offered focuses mainly on the IT elements of OT systems. At best, it involves a superficial scan of vulnerabilities specific to OT. This approach fails to fully investigate the unique aspects of OT, such as the specific protocols used and the operational complexities of industrial control systems. As a result, this can lead to:

• Overlooking industry-specific protocols, such as Modbus, DNP3, BACnet, etc., could result in vulnerabilities being missed.

• Failure to consider operational context may result in undetected critical dependencies between IT and OT and vulnerabilities affecting physical processes.

• Insufficient depth in security scans may result in generic and superficial results, which can miss deeper weaknesses specific to the organization's technology.

3. Pen Test on Production

A concerning practice in the field of OT cybersecurity is the promotion of testing on live, production OT systems. Few service providers and even internal security teams might advocate for live OT testing, believing that it provides a more "realistic" picture of the system's vulnerabilities. However, this is often done without fully understanding or acknowledging the potential impacts. Testing in a live OT environment, such as a power plant or manufacturing facility, can have serious consequences, including operational disruptions, safety incidents, and even physical damage. The risks involved are much higher compared to testing in a typical IT setting.

4. OT Cybersecurity ownership

In some organizations, especially those that do not have specialized personnel for OT security, IT decision-makers may be responsible for making choices regarding OT pen testing due to their broader knowledge of security. While they bring valuable expertise, there is a lack of understanding regarding the operational and safety implications that are unique to OT systems. This gap can result in:

• The risks associated with OT may be underestimated, leading to inadequate testing frameworks and insufficient risk mitigation strategies.

• Testing methodologies may fail to address OT-specific vulnerabilities and security concerns.

• Not understanding the impact of OT can have serious consequences, including operational disruptions, safety incidents, and even physical damage.

5. OT Pen Test Tools

Penetration testing tools like Metasploit and Nmap, which excel in IT environments, encounter limitations in Operational Technology (OT) contexts. These tools are primarily designed for IT systems, focusing on standard IT protocols and scenarios. Consequently, they often fall short in fully supporting OT-specific protocols and scenarios, leading to an incomplete vulnerability assessment in OT environments.

While some penetration testing tools offer limited support for OT protocols, their functionality remains basic. For instance, initiatives like the Industrial Security Exploitation Framework (ISF) represent significant steps towards OT-specific scanning and exploitation tools. However, even such specialized tools offer limited protocol coverage and are often constrained to certain PLC models (ISF: https://github.com/w3h/isf).

This limitation is inherent in the nature of OT systems. Unlike IT environments, where a handful of common protocols prevail, OT landscapes vary significantly across industries and depend on the specific products and system integrators involved. This diversity results in the use of a wide array of protocols from a vast pool, making it impractical to anticipate and prepare tools for every possible scenario. This diversity is also why a universal, generic framework capable of scanning and exploiting any OT environment does not exist.

Effectively exploiting OT assets and navigating OT protocol communications demands a deep understanding of both the assets and the protocols, as well as a fundamental knowledge of PLC logic and HMI design. For OT protocols, it is crucial to comprehend their structure, encoding, encryption, communication transport, and the nature of their communication models (such as Master/Slave, Publisher/Subscriber, Peer-to-Peer, Producer/Consumer, Broadcast/Multicast, etc.).

Moreover, the complexity of OT protocols varies widely. While some Layer 2 (L2) protocols may have straightforward payloads, others, particularly ISO protocols, feature complex headers and payloads. Additionally, concepts such as routing and switching redundancy, precise time synchronization, and various network topologies add layers of complexity. Therefore, conducting penetration testing in OT networks without a comprehensive understanding of these elements can be challenging and potentially ineffective.

Furthermore, OT systems are often complex, with a mix of new and legacy technologies and complicated interdependencies. This complexity is not always well understood or addressed in standard penetration testing tools.

6. IT Pen Test Frameworks

When discussing the application of penetration testing techniques and tools outlined by frameworks such as NIST, PTES, OWASP, OSSTMM, CREST, and others in OT environments, it's crucial to understand both the limitations and the applicable aspects of these methodologies. Here's a detailed perspective:

• Frameworks Limitations:

  • Those penetration testing frameworks developed primarily for IT systems. These frameworks emphasize protocols, systems, and threat models that are prevalent in IT environments.

  • Tools recommended by these frameworks, such as Metasploit, Nmap, or OWASP ZAP, …etc are primarily designed to exploit vulnerabilities in common IT infrastructure and applications. They are less effective in OT environments, which use different protocols and have different operational requirements.

  • OT systems often involve specialized hardware and software, like Programmable Logic Controllers (PLCs) and SCADA systems, operating on protocols like Modbus, DNP3, or proprietary industrial protocols, which are not typically the focus of IT-centric penetration testing tools.

• Applicability in OT:

  • Despite these limitations, certain aspects of these frameworks can be adapted for use in OT environments. The structured approach to planning and reporting, as emphasized by all these frameworks, is universally applicable and beneficial. This includes defining the scope of a penetration test, threat modeling, and detailed reporting.

  • Discovery and information gathering techniques can be adapted to some extent. For instance, network mapping tools might be used to identify devices on an OT network, though the interpretation of results will differ.

  • The focus on legal and ethical considerations, as well as the emphasis on minimizing operational impact during testing, are especially pertinent in OT environments, where systems control physical processes that can have real-world consequences.

• OT-Specific Considerations:

  • Penetration testing in OT requires a detailed understanding of industrial processes and the operational constraints of these systems. For example, testing must often be done without interrupting live processes, which is less of a concern in most IT environments.

  • There is a need for specialized tools that understand and can interact with OT-specific protocols and devices. While initiatives like the Industrial Security Exploitation Framework (ISF) are steps in the right direction, comprehensive toolsets for OT are still developing.

  • The security of OT environments is not just about data confidentiality and integrity, as is often the case in IT, but also about ensuring availability and safety. This shifts the focus of penetration testing to include aspects like physical safety and process reliability.

7. Real Attacks

Lastly, simulating sophisticated real-world cyberattacks poses a significant challenge. High-profile incidents like Stuxnet and Triton highlight the advanced nature of threats targeting OT systems. These attacks are not just technically complex but are also specifically designed to exploit unique vulnerabilities in OT environments.

Stuxnet, for instance, was a highly sophisticated worm that targeted SCADA systems and was specifically engineered to disrupt Iran's nuclear program. Its complexity lay in its ability to remain undetected while manipulating industrial processes. Triton, another advanced threat, targeted safety instrumented systems (SIS), aiming to cause physical damage by manipulating industrial control systems. These attacks stand as a testament to the level of customization and deep understanding of industrial systems required to execute such operations.

I strongly believe that the offered Pen Test for OT lacks the depth and specialization needed to prepare OT systems against such advanced threats fully. For example, the knowledge of industrial processes and specific PLC programming required to design an attack like Stuxnet is generally outside the scope of conventional penetration testing tools and techniques.

Conclusion

Considering the specific challenges and risks associated with OT systems, this article strongly advises against routine penetration testing in OT environments under current circumstances. The unique sophistications of these systems, particularly when linked to critical infrastructure, make standard penetration testing methods potentially hazardous and less beneficial.

However, if penetration testing is absolutely necessary, it must be conducted under extremely controlled conditions, with adherence to the following guidelines:

Knowledgeable Testing Team: Ensure that the team conducting the penetration test possesses in-depth knowledge of OT systems. Their expertise in OT-specific protocols, processes, and safety considerations is crucial for conducting safe and effective testing.

White Box Testing Approach: Opt for a white box testing method where the testing team is fully informed about the system's internals. This approach allows for more precise and safe testing.

Critically Defined Scope: The scope of the penetration test must be accurately defined, focusing exclusively on relevant areas to minimize potential disruptions to essential operations.

Scheduled Testing: Conduct penetration testing according to a predetermined schedule known to all relevant stakeholders. This scheduling helps in preparing for potential contingencies and minimizes the impact on regular operations.

Clear and Specific Objectives: Set explicit and well-understood objectives for the penetration test. This ensures alignment with the OT environment's security needs and clarity on expected outcomes.

Avoidance of Production Systems: Refrain from testing on live, production OT systems to avoid operational disruptions and safety incidents.

Utilization of Test Beds or Digital Twins: Where possible, use test beds or digital twins for testing. These simulated environments allow comprehensive vulnerability assessments without impacting real-world operations.

In conclusion, while penetration testing remains a vital component in IT security, its application in the OT domain should be approached with exceptional caution and preparation. The distinct nature of OT systems, characterized by their critical operational role and sensitivity to interruptions, demands a careful, informed, and strategic approach. By adhering to these stringent guidelines, organizations can ensure a safer and more effective method of assessing and enhancing the security posture of their OT environments.