Running penetration tests of a mature web application is always a great challenge. Systems are usually well hardened, and scanners fall short of flagging anything interesting, requiring an experienced security engineer to identify vulnerabilities using advanced exploitation methods.
On the other side, some applications are going for their first release ever or release after a major code change. That usually involves issues not related to security – for example, functional bugs, which the penetration testing team must address before they can provide valuable results.
Following are some of the key takeaways from a web application penetration test engagement we did recently, which involved web and mobile application sharing a common backend.
Not starting on time
Software projects are almost always late for various reasons. And for the same reasons, the scheduled penetration tests will not start on time because critical functional issues will have precedence. Or some bugs may be exposed only when the testing environment is set up for the testing.
According to the 2019 Gartner survey, 45% of products aren’t launched on time.
From those that do, 42% suffer from launch failures.
With hard go-live deadlines, the penetration test must be executed in a shorter time period, requiring last-minute availability arrangements of the workforce and may require additional help of subcontractors to cover the agreed number of hours.
Test platform issues
Ideally, the test platform will be ready on time and with code-freeze for the duration of the penetration test. In a not-so-ideal scenario, the platform will have to be shared with other teams, causing extra effort to rule out false positives because both teams can perform the same changes, causing confusion on each side.
If there is no code freeze, the development team will roll out releases with fixes. It is critical to agree on a testing plan so already tested parts of the system don’t have to be re-tested. This can be further enforced by having a changelog accompanying each code release.
Functional bugs
Errors in the tested applications should usually be already discovered by the client’s internal team testing the functionality before the penetration test. But as quick changes and last-minute hotfixes are applied, new bugs can be introduced to the code that can either hinder further penetration testing or must be reported to the client – eating away time of security experts that are not looking for security vulnerabilities but are writing functionality bug reports.
To address this issue, the estimated effort needs to account for handling functional bugs that will be encountered during the penetration testing.
Too many trivial vulnerabilities to report
Simple vulnerabilities take little time to detect, but they require detailed reporting with the same quality and effort as issues with higher severity. If there are too many simple issues, the security engineers will deplete the allocated time on them, leaving the serious ones untouched. The fact that low-impact findings are not the highest priority during the pre-release period is decreasing the value client is getting out of the penetration test even more, as they will be remediated later – usually after the release.
If daily updates about new findings are provided to the client, having too many findings will again take time allocated to testing for reporting these.
Basic vulnerability scanning should be integrated into the software development process to weed out the easy-to-spot issues using automated tools. The penetration test report should include more serious issues worth manual exploitation.
No time to test remediations
So eventually, security issues that were reported by the penetration testing team will be remediated by the developers and scheduled for a retest. But with the product already live, there will be untested remediations and potentially vulnerable code exposed to the Internet for some time.
In the past, there were a lot of examples where remediations were not done correctly, and just by slightly altering the payload, the original vulnerability would be again exploitable.
To be safe, plan the engagement a little ahead when high severity vulnerabilities are to be expected.
Below are some more lessons we learned that are more generic and are not related only to newly released web apps.
Do static code analysis first. It’s cheaper
Some vulnerabilities can be detected by static code analyzers or dynamic scanners (SAST and DAST), which can spot usual suspects quite easily in large codebases. Something that would be very tedious and not effective when done manually. Testers will spot and report some of the vulnerabilities, but the code coverage and number of findings will be significantly lower due to time constraints.
Penetration testers (VS1-3) have much lower code coverage than
the tested static analyzers (SA1-3)
Why spend effort on something that can be done automatically, right?
Whitebox vs. Blackbox approach
Penetration tests are usually performed as black-box engagement (which means no access to sources, system, logs, etc.), which should simulate a real-world scenario where attackers try to exploit systems without additional details. Nice idea, but there is one very limiting factor: the timeframe for the engagement. Let’s say both pen-testers and attackers find a loophole, where the application sends contents of a local file based on the filename provided as user input. To figure out all filenames on the system, a tool needs to be built that will try all possible letter/number combinations to disclose sensitive files and their content – and as there are a lot of combinations to try, it will eventually make the testers run out of time. They will report the issue but miss further exploitation steps that the attackers would leverage in real-life – maybe by using a different vulnerability.
So, in the end, the black-box approach will benefit mainly the attackers.
Getting the value on time
This post covered some caveats of performing a penetration test on a newly released web application. The main risk here is the large number of findings that must be expected and a delayed start of the testing. Using static code analysis and preliminary tests to cover the most common vulnerabilities will vastly increase the value that a penetration test can provide. Planning for an earlier start would take off the pressure on both sides but is not always possible. Therefore, expect at least high impact findings that will need to be remediated and retested before the release so that security incidents can be prevented.
How SecurityScorecard can help
SecurityScorecard’s Penetration Testing service tests the effectiveness of your security controls while achieving compliance and protecting your brand. Thinking offensively by using simulated cyberattack strategies and tools designed to access or exploit your computer systems, networks, websites, and applications is the approach to building stronger defenses.