Most often during my pre-sales work, I am asked to derive the estimation effort for DAST (Dynamic Application Security Testing) and SAST (Static Application Security Testing). These two testing methodologies are not new in a Software Development life cycle and are almost always done if a web application is internet facing. Read on if you are a customer (software owner) who requires DAST and SAST services for your application or a service provider who wishes to provide this service.
DAST – Dynamic Application Security Testing. This is also called as Black Box testing or Application Vulnerability Assessment. It involves navigating a web application during run time and mimicking a hacker by providing malicious attack vectors of all possibilities (using popular tools like Appscan, WebInspect, Acunetix or WhiteHat), doing false positive analysis and then providing a report.
SAST – Static Application Security Testing. Otherwise known as White Box testing or Secure Code Analysis. It involves scanning the code using tools like fortify, appscan, veracode, checkmarx etc for known weaknesses and code flaws and listing out the security weaknesses.
Both the methodologies involve tools, scan time, going over the results to reduce false positives, re-categorize the issues based on severity, documentation and reporting. All the steps involve a certain amount of time and would have to be done if you really want to weed out security issues before deploying your software. Customers most often try to bargain for lesser price and lesser time (why not give me the results in 1 hour ?) and service providers (just the deal in the mind) may ask people like me to reduce effort as much as possible by possibly employing more staff. Well, quoting a good friend here, “You cannot ask a woman to deliver a baby in 1 month by providing her 10 men”.
Some steps involved in both these methodologies are sequential and some are parallel. Some need an human element and some can be automated. I will list out the activities and the effort part first and in the next series mention ways to refine the effort.
- Scan Time For DAST – This is dependent on the scan tool’s efficiency, infrastructure (RAM, CPU core, Hard disk (where logs get saved)), size of the application (number of pages, input parameters per page, number of workflows and complexity), security posture of the application and also the application deployment infrastructure. From my experience, I have seen application with 20-30 pages getting scanned in 2 hours’ time. I have also seen website with 1000 urls’ taking 2 weeks.
Always do 3 preliminary scans with applications of varying size to find out the probable scan time that your tool takes on a given infrastructure and then use that as baseline.
Scan Time for SAST – This is dependent on executable lines of code of application, code flow, infrastructure (again your RAM, CPU core etc comes into picture) and the choice of tool. Here also, do a base line scan to determine the time taken. Most simple applications with 10,000 LOC have taken 30-45 minutes to scan in my experience but that may vary based on your infrastructure and tool.
- Issue Analysis Time For DAST and SAST – The most important factor here is the number of total issues to analyze. The second factor is the expertise level of personnel analyzing the issue. While it could be true that the total number of issues may increase linearly based on the size of the application, it needn’t be true always. A 50 web page application can give out 25 issues but a 20 web page application that had no security in mind or intention of being secure may give 100 issues. Likewise a 30K LOC code may give 2000 issues and 50K LOC may give only 300 issues.
So, instead of blindly putting an effort saying that it will take 1 day to analyze issues, go by the parameter ‘Vulnerability Density’. What is the expected density say in 1K LOC? Accordingly the resultant total issues will be for an application. An expert looking at a given issue may weed out false positive in 5-10 minutes or if you are giving to a fresher/not-interested personnel, he may take 30 minutes per issue. For example, I can check whether the XSS issue is real or not just by looking at the response (10 – 20 seconds) but to check if a padding oracle attack really succeeded and giving rise to DOS, I may want to execute and see (probably 30 minutes).
- Gathering Evidence – This is directly proportional to the number of total real issues you have got and you just need to replicate the steps manually and take screenshots for further use in documentation.
- Documentation and Reporting – I would usually give 4-5 hours for this considering that you have already put enough time in gathering evidence.
As you would notice, I didn’t really give you a thumb rule that says estimation effort is 1:3:5 for Simple, Medium, Complex of DAST or 2:4:6 for a SAST. Even if I give, that would be wrong as what may be right for my software and infra may not be true for you. So, always do a base-line and then get started. I will however provide some tips to refine these activities in next article.
A Security Consultant's Diary 