Penetration testing guide: why manual penetration testing is better than automated tests?

penetration testing for web application

We have previously discussed why it is crucial for organisations to test their effective security, allowing them to gain an accurate assessment of how they would fare against outside threats.

Among the suggested measures for the successful execution of such an assessment includes the hiring of specialist companies that can conduct penetration testing with the ultimate aim of identifying and eliminating weaknesses in an organisation’s internal system.

But what exactly is penetration testing and what are the best practices and solutions when it comes to penetration testing?

What is penetration testing?

Penetration testing, informally also known as pen testing or ethical hacking, is the equivalent of a military exercise but in information security, an imitation of a battle to improve one’s expertise and knowledge. It involves a legitimate and planned-in-advance cybersecurity attack on your organisation’s system in order to gauge the overall level of security, finding weak points and vulnerabilities in the process, in order to have a better understanding of what needs to be upgraded, remedied or otherwise strengthened.

While your organisation may have already put a range of security measures in place, those may be now out-of-date, poorly configured, or not specialised enough to cope with the demands of your business, sector or client requirements, particularly in security-critical fields such as finance, law or healthcare. Moreover, just as cybersecurity is not a one-time project, but rather a continuous process of assessment and improvement, so too is the case with penetration testing. Threats evolve and change while entirely new sources of danger emerge at an ever-quickening pace. Commissioning tests to ensure that your defence system is robust enough to handle outside attacks should be a recurring exercise.

The fundamentals of penetration testing go all the way back to the early 1970s when James P. Anderson developed an approach centred around two key aspects: exploiting the weakness in a system and then exploiting the weakness in the attack originally designed to breach the system. “A major point is that with no recognized principles of design for security, the ad hoc protection mechanisms of most contemporary systems are insufficient to defend against a dedicated penetrator,” Anderson said in a 1972 paper. In 1980, Anderson would go on to contribute massively to the development of audit trail-based intrusion detection.

Are all penetration tests the same?

While the general principles of penetration testing are shared and well-established, the people who you choose to go with definitely matter for two key reasons. One is related to the result of the penetration test, and the second is related to what happens while the penetration test is taking place.

Regarding the former, you have to remember that part of the desired result is as detailed a report as possible about what data the testers could extract or view and what points in the system allowed them to gain entry or otherwise exploit. The testers should have both the technical and ethical skills to produce high-quality results.

This is also related to the second point. The quality of the testers you opt to task with carrying out the penetration test will also affect the level of disruption that your organisation and its customers will experience during the testing process.

Although it is best practice that your in-house team of information technology professionals already possess a high enough level of understanding of the organisation’s system that a penetration test mostly confirms their suspicions, this is not always the case.

What are the phases and tools used in penetration testing?

The penetration testing process can be broadly broken down into five key phases.

First comes the reconnaissance, where information gathering takes place so that a profile of the desired target system can be created. The more information is gathered the more thorough the attack can become. An example of a tool used in this phase is a commonplace open-source search engine that can help the team perform a phishing attack or other attacks of a social engineering nature.

This is followed by the scanning phase, where a range of technical tools is used to increase the testing team’s pre-existing level of knowledge of the target system. In this instance, a network mapping tool can reveal network vulnerabilities such as open ports.

The third phase involves gaining access to the target system by utilising all the information collected during the first two phases of the testing process. In this phase, a tool such as Cobalt Strike or Metasploit can be used to perform an automated attack on the target system’s weak points.

Another tool that can be used during this phase is Nessus, a piece of software that acts as a vulnerability scanner. Nessus, and other similar pieces of software, can reveal weaknesses that can be used to gain unauthorised control or access to a system’s data, reveal misconfigurations such as missing security or software patches, reveal default or commonly used passwords (including attacking a system with a Hydra attack, where a dictionary is used to find the password), as well as bring to light any Denial of Service (DDoS) vulnerabilities.

After the gaining access phase comes the part where the penetration testing team attempts to maintain their access into the successfully attacked system. This mirrors a hacker or malicious entity’s desire to retain access in order to either maximise their damage on the victim or extract as much data as possible before the victim retaliates and shuts them out.

Finally, the penetration testing team will attempt to cover their tracks, meaning they will try to remove any traces of them having compromised the target system. This includes the clearing of any signs that data has been tampered with or extracted and the deletion or manipulation of event logs to ensure that the attacker’s identity is not disclosed.

Manual vs Automated Penetration Testing

Penetration testing can be divided into broad two types of approach: manual penetration and automated penetration testing.

Each type of penetration test approach has different attributes which we will examine below, although the crucial point to bear in mind here is that a manual penetration test, despite its higher cost and additional time necessary for its completion, is much more thorough and multifaceted than an automated penetration test.

What is manual penetration testing and what does it involve?

As mentioned above, manual penetration testing is much more rigorous than automated testing and can unveil issues that may be missed by an automated test. The manual penetration test will be carried out by penetration testers, sometimes referred to as the red team, who will not only use an array of tools to detect security gaps and flaws, but also apply real human insight and contextualisation to enhance the level of detail and insight your organisation will receive at the end of the testing process.

Moreover, manual penetration testing allows the organisation to get the best of both worlds, as the testing team can combine the manual aspects of the process with automated tools for improved results.

The manual penetration testing process starts in a similar fashion to the general penetration testing procedure discussed above, with the testing team gathering information and collecting data. This can include the version of the database, software and hardware the victim system is using, as well as information on third-party software or plug-ins. All of this can prove useful during the simulated attack involved in the test.

Again, this is followed by a vulnerability assessment to unearth potential flaws in the attack system that can be exploited during the next part of the process, the simulated attack, where manual techniques mixed with real human instinct and perception are used to identify and take advantage of any weak points.

The final two steps of the manual penetration testing process including the preparation of a detailed report, as well as the addressing of the breach and the weaknesses that facilitated it. The former step involves a thorough outlining of everything that took place, such as the discovery of flaws, bugs and vulnerabilities, the testing scope deployed, and more; while the latter step involves an analysis of the findings, the potential damage they would cause and how they will be resolved and fixed after the penetration testing process has finished.

Manual Penetration Testing Guide: Pros of Manual Penetration Testing

As mentioned above, there are numerous benefits to manual penetration testing over an automated approach.

First and foremost, it is conducted by a qualified expert or a team of specialists who have both the industry experience and technical knowledge to carry out the test in a bespoke manner, adjusted to the structure and field of your organisation. This will help deliver the optimal results and thus lead to the best remediation actions down the line.

In addition, the results of a manual penetration are validated by the process itself, since the testers perform it manually and can check each step in detail. Conversely, an automated test is more ill-defined and opaque, making the results of the process harder to verify.

As previously mentioned, manual penetration tests are more bespoke than automated tests and can thus be customised accordingly, including in terms of access control and session handling. While it does take a serious amount of work by the testing team, it does allow for a more thorough inspection of the entire setup.

One of the most prominent benefits of manual penetration tests is their ability to spot logic flaws in a way that automated penetration tests are not equipped to handle. Logic flaws occur during the creation of a web application or other piece of software designed to perform certain functions. While there may not be a bug per se, there may be something inherently broken with the structure of the application that an automated test will simply gloss over entirely.

Finally, the results of a manual penetration test are applied to the individual profile of the customer. This makes the remediation process more effective and more likely to provide the organisation with the desired outcome, which is to eliminate vulnerabilities and enhance the level of their cybersecurity.

Automated Penetration Testing Guide: Cons of Automated Penetration Testing

While automated penetration testing has a few benefits, such as a somewhat lower cost when compared to manual penetration testing, it does have a number of drawbacks.

To begin with, an automated penetration test can perform an evaluation of the cases it has been programmed to test. This is the result of a lack of precision that automated penetration tests tend to have. Something that many penetration tests do not disclose when performing automated tests is that they are executing out-of-the-box, segmented, packaged tests that may not deliver the required results.

“Many penetration testing organisations today offer a packaged service comprising of penetration testing aimed at the network, web application, social engineering or wireless and mobility layers,” information security recruiter Ricki Burke wrote.

“More often than not, these packaged engagements are not customisable and offer the client no flexibility when determining the scope and return on equity (RoE),” Burke added.

Another negative side-effect of automated penetration testing is the high number of false positives, which is linked to one of the benefits of manual testing, validated results. A large batch of false positives, meaning wrongly flagged vulnerabilities, can result in either additional time verifying them or degradation of the entire testing process since it negatively affects its trustworthiness.

The generic nature of automated penetration testing is reflected in the results it produces, which lack the level of detail and analysis a manual test can produce. In fact, its broad but imprecise scope is also one of the reasons why automated penetration tests are not usually considered acceptable by independent bodies wishing to assess an organisation’s level of cybersecurity protection.

Finally, automated tests are also bereft of the intuition, experience-derived specialist knowledge, insight and ability to place within a specific context that human beings inherently have, especially in fields, industries or situations where these attributes are critical for the production of truly usable and beneficial results.

How Boltonshield can help

Boltonshield can help you assess your current security levels through a variety of methods, including penetration testing and security audits. This can help you identify problems and potential risks in a proactive manner and address them before they can be exploited.

You can get in touch with us to find out exactly how Boltonshield can help you by clicking here.

If you want to get updated about our recent publications about cybersecurity related topics, subscribe to our newsletter!

Subscribe to our Newsletter

Sign up for our content, including blog articles, news, tips and more