Treating Security Vulnerabilities As Software Defects
SANS What Works In AppSec 2010
Friday February 5th, 2010
1
Agenda
• Something Most Security Vendors Won’t Tell You
• The Wrong Way to Do It
• A More Excellent Way
• Strategies
• Demo
• Questions?
Something Most Security Vendors Won’t Tell You
2
Something Most Security Vendors Won’t Tell You
Finding Vulnerabilities is Easy
Fixing Vulnerabilities is Valuable
3
The Wrong Way to Do It
4
The Wrong Way to Do It
Dan: What is your application security strategy
A: We bought Scanner XYZ
Dan: Cool! Have you started using it?
A: Yes. The analyst who wanted us to buy it ran a bunch of scans when we got
the license key.
Dan: All right! Did you find anything?
A: Oh yeah! We found all sorts of scary stuff.
Dan: Well what did you do about it?
A: We sent the PDF report to the development team and told them to fix the
problems.
Dan: Were they successful?
A: I don’t know. I guess I should check in on that…
5
Why Is This Bad?
• PDFs are blobs
• Email is infinitely ignorable
• Lumps all vulnerabilities together
• No guidance for developers
• Just plain rude
6
A More Excellent Way
7
A More Excellent Way
• Treat (Application) Security Vulnerabilities as Software Defects
• Why?
– Developers have to fix the issues eventually
– Developers understand defects
– Even most “loosely-structured” development teams have defect tracking systems
8
What Makes a Good Security Defect?
9
What Makes a Good Security Defect?
• Why do I care?
• Scope
• Where is it?
• How do I fix it?
10
Why Do I Care?
11
Why Do I Care?
• Do not rely solely on the defect to communicate this
– Simply pumping defects into the defect tracking system is unlikely to be effective
• Provide context
• Provide steps to reproduce
– Automated if possible
• Transparency!
12
Scope
13
Scope
• Defects that take 5 minutes to fix take far longer to administer
– Especially with mature (elaborate) QA processes
• Maximum time: 16 hours– http://www.joelonsoftware.com/items/2007/10/26.html
• Target: 1-16 hours
– Long enough to be an actual task, short enough to be predictable
– Defects for technical vulnerabilities should be shorter
– Defects for logical vulnerabilities can be longer
14
Where Is It?
15
Where Is It?
• Providing location information removes a “barrier” to fixing
• Better location information leads to quicker fix times
• Dynamic analysis: attack surface location
– Vulnerability type, URL, possibly parameter
– (For web applications)
• Static analysis: code location
– Filename
– Line (and hopefully column)
– Include actual code if possible in case underlying codebase has changed
16
How Do I Fix It?
17
How Do I Fix It?
• Prescriptive guidance is required here
– Removes a reason not to fix
– Leads to consistency
• Does your organization have an ESAPI? Does it address this issue?
18
Why Is This Approach Better?
• Defects are structured data
• Defects are durable
• Vulnerabilities have been portioned out into tractable chunks of “work”
• We have provided prescriptive guidance
• Communicates with developers via systems they already use
19
Strategies
20
Strategies
• Group by location
• Group by type
• Group by severity
21
Grouping By Location
• By file/URL or by directory
• Pros:
– Helpful if there is one “owner” for that area of the code
– Can help to minimize requirements for QA regression testing
• Cons:
– Different vulnerability types require different fixes
– Can be hard to keep things straight
22
Grouping By Type
• By vulnerability type (XSS, SQL injection, authorization issue, etc)
• Pros:
– Similar vulnerabilities often have very similar fixes
– Economies of assembly lines – get Henry Ford on vulnerabilities
– Approach with a “punchlist” mentality
• Cons:
– There can be LOTS of vulnerabilities of a given type if bad coding idioms are in use
23
Grouping By Severity
• High, medium, low
• Pros:
– Can help you game certain metric programs
• Cons:
– Least tied to how developers work
– Different types of vulnerabilities
– Cutting across functional areas
24
Strategies (continued)
• Combine more than one
– Group by type or severity and then by location
25
What About BIG Issues?
• Serious issues can map to multiple defects
• REALLY serious issues can map to enterprise change management
initiatives
26
What About Non-Software Vulnerabilities?
• Transition to change management systems rather than defect tracking
systems
27
Demo
28
Contact
Dan Cornell
(210) 572-4400
@danielcornell
Web: www.denimgroup.com
Blog: blog.denimgroup.com
Vuln Mgr: vulnerabilitymanager.denimgroup.com
29
Recommended
