Feature story from the Netherlands: Pentesting in IT-audit: the proof of the pudding


In early 2020, a serious security threat was identified in a Citrix-software component widely used by government agencies in the Netherlands for remote access to IT-systems. In resolving the crisis followed by the discovery of the threat many organizations turned off this component, making it impossible for civil servants to work remotely. Forced to work on premise, the country had to deal with ‘Citrix traffic-jams’ for a week.

Numerous similar incidents in the past years have pointed out the direct impact on business continuity that information security threats can have, and the possible real-life consequences for society. Supreme Audit Institutions (SAIs) have become aware that they have a role in protecting public interest by supervising cybersecurity spending, policies and effectiveness of IT-security measures in the public domain.

Security tests such as vulnerability assessments and pentests (‘ethical hacking’) play an important role in safeguarding digital security. In IT-research at the Netherlands Court of Audit (NCA) we have noted that although security policies were often in place on paper, the necessary measures were not always actually enforced in practice. In a recent audit for instance, we encountered lists of passwords that were written down in regular Word-documents: something that was clearly not compliant with regulations set by the auditee. It is thus even more important to put effort in testing the effectiveness of security measures. If an auditee itself cannot provide this evidence, a test can be part of an independent audit. In our audit on the cybersecurity of critical waterstructures for instance we tested the physical security of IT-operating rooms and the system to detect network intruders to check the effectiveness of these important security measures.

IT-security measures are often related to technical configuration of IT-systems. Practical expertise in the area of testing is still under development at many SAIs. In the 2020 audit compendium on cybersecurity by the Contact Committee of EU SAIs, the NCA was the only SAI to have used security tests in a cybersecurity audit. We stress the importance of building knowledge as a SAI on IT-security testing. Proving the possible consequences of lacking security measures with a test can greatly increase the impact of an audit report. In our audit on cybersecurity of border controls at Schiphol airport for instance, ethical hackers proved they could send emails on behalf of random Ministry of Defence officials. A screenshot of a ready-to-send fraud email signed by the Chief of the Defence Staff immediately made clear the potential risks to the auditee.

In the last ITWG newsletter our Norwegian colleagues shared their experience in pentesting in IT-audits. In this contribution we want to set out our approach and experience in incorporating security tests in IT-audit. This approach is based on three lines of action. Although these are not necessarily ‘maturity levels’ we suggest SAIs start with the first line of action and move on with second and third after gaining some basic knowledge.

1. Involving security test reports from the auditee in research

SAIs should strive to involve the results of security test conducted by or on behalf of the auditee in audit. Most widely used IT-standards prescribe regular security testing of IT-systems and processes. Using such norms in audit will make clear how the auditee is meeting standards to perform the ‘check’ in a plan-do-check-act cycle. Although we found several security test reports in our cybersecurity audits we also observed that security tests are sometimes not performed at all, even at critical infrastructure.

Learning to read test reports will increase the knowledge of SAIs about IT-security testing and how to translate the technical vulnerabilities to business risks. 

SAI researchers can get familiar with different test types and approaches. For instance the classification of ‘boxes’, that tell you how much technical information was given to a test team when performing a security test.

The Netherlands pic1 boxes


Also they will learn how the scope of a security test can be defined and what methods can be used. This will help to grow understanding of possible test types such as vulnerability scans or pentests, the different purposes, and common used methods such as port scanning, network sniffing and social engineering.

 Lastly, by reading test reports SAI researchers will learn how security test results are interpreted and described in a report. Usually a standardized ‘scoring method’ is used to assess the seriousness of IT-risk found in a test. Common methods take into account multiple elements of likelihood and impact: such as the knowledge required to perform an attack and the nature of the data that would be effected.

 2. Designing and executing a security test in audit with a third-party

In case the auditee has not yet performed a security test, a test can be executed in the context of the audit. SAIs do not need to be able to perform such a test themselves: a third party can be hired for the actual testing. However it is important to have some knowledge to act as a client for the third party and interpret their findings.

In close cooperation with the auditee we let a third party perform security tests in two cybersecurity audits. Based on the situation and possible threats we observed, we developed specific scenario’s to test. In the audit on crital waterstructures this included gaining physical access to areas using social engineering. In the audit on cybersecurity of border control at Schiphol airport we argued that an insider threat (a malicious employee with basic access to the network) could pose a significant risk.

Close corporation with the auditee is essential when performing a security test in audit. Security testing is itself not without risk, for instance data can be lost or damaged. Clear arrangements should be made about the exact set up of the test and the way risks as result of the test are minimized. To keep responsibility and liability clear we chose for a model where the auditee was the legal client of the third party. We as researchers determined what was tested and received the unfiltered test results first.

3. Building own security test capacity as a SAI

A third line of action we are exploring is building experience in performing security test ourselves. Although the primary responsibility for performing tests lies with the auditee, gaining hands on experience as a SAI can help in IT-security audits. Inspired by our Norwegian colleagues who already have experience with their own security tests in audits, we are now also looking into this line of action. It is not our ambition to turn researchers into professional ethical hackers. But building knowledge by actually performing security tests yourself greatly improves understanding of the possibilities, limitations and interpretation of results when it comes to testing.

Security testing is not magic exclusively performed by technical experts. The internet provides accessible information and fairly easy-to-use software to perform simple security tests. For instance the free application Zenmap, available for Windows and Apple computers, allows every beginner to perform a port scan like a professional.

Newsletter esipilt

A Zenmap port scan of the home network of one of the NCA-researchers:
red circles are machines with many open ports; a possible security risk

In conclusion

We described three lines of action in which the NCA incorporates security testing in its IT-research. We believe building knowledge and experience in this area is crucial for every SAI, given the ever growing digitization of our world and the risks involved. We hope to inspire other SAI’s to set of their journey in the world of security testing, underlining that the secret to getting ahead is getting started.


By Christiaan Luteijn MA and Xander Bordeaux MSc CISSP CEH, researchers at the Netherlands Court of Audit