As an information technology professional, information security is of huge practical importance to me, influencing pretty much every project I am involved in. While security is not my key focus I thankfully have some colleagues in the industry who I can turn to for advice and analysis: Adam Shostack and Dave Piscitello have been particularly valuable in directing my thinking on security issues.
Here is a transcript of an e-mail exchange Dave and I had about human factor engineering and vulnerability reporting:
Ian:
I read a book called the Human Factor awhile back and it got me thinking about problems in security, particularly reporting of incidents and the difficulty of getting accurate actionable data. Unfortunately I never got much of an answer from anyone as to whether it might be worthwhile to implement a system in the genre of the Aviation Safety Reporting System, which is voluntary, confidential, non-punitive and independent.
Dave:
I agree that our senses are dulled when it comes to security, to a far greater extent than air and auto safety, and several orders of magnitude *above* medical safety. But comparing computer incidents to either of these is an odd comparison. By and large, consumer and enterprise security incidents don’t result in death. Moreover, when human injury and death are identified as risks (space shuttles, ATC software, and yes, medical systems that govern life support systems and imaging), hardware and software systems undergo more stringent code review and testing, commonly by 3rd parties or government agencies. So I don’t see the risk profiles as easily compared and as a result, I think incident handling and reporting have different characteristics.
Before we consider a computer safety reporting system, we have to consider society’s understanding of the state of and need for security. Our society claims to worry over identity theft and related security incidents, but does society really worry more today over identity theft than it did when dumpster divers hunted for carbon paper used in the original credit card charge forms, or is it that financial institutions see a much greater revenue loss than they factor into their fees? Online transactions continue to grow exponentially, not because everyone believes SSL protects the transaction, or that identity theft is being thwarted, but because nearly everyone now trusts that their personal liability is bounded should someone use their credit card fraudulently.
Conclusions: Financial institutions will always take the loss because they have no choice. If they put the liability on individuals, individuals won’t use credit and debit, online or in person. Security is secondary to sustaining the revenue model. The result is that individuals don’t take security seriously because someone else assumes accountability and absorbs the loss.
Many security measures are compromised in the same way. What incentive does anyone have in reporting identity theft? Will credit card interest rates go down? Will the cost of products be reduced?
Next rant. What is a security incident? Only “technology wizards” know the answer, which throws us back to the issue of human factor engineering. The majority of users can’t distinguish between a security incident and an operating system, application or network failure. We expect computers and networks to break, we have little faith that software will work, and we have less faith that anyone is actually motivated to improve the situation. Everyone believes any system and application will eventually fail. People fall back on what they know: restart the machine, reinstall the OEM disks, lose data and cry about it, call a computer geek. How do you educate users to distinguish between a security incident and buggy software, and what incentive do they have to learn this? Imagine if you put a CSRS in place and you *don’t* educate them. Who fields all the incident reports and sorts incident from user/configuration error and buggy software?
Given how low (and accurately) the standard is set, I don’t see how a CSRS would help any more than a reporting system where drivers notify law enforcement when a fellow driver exceeds the speed limit by one mile.
Before we implement CSRS, we need to recalibrate our expectations of quality assurance and software integrity. I’d rather see society spend more time developing software reliability assurances; like SLAs, these could be used by government agencies and F1000s to hold system and software developers accountable to a set of minimum performance, stability, vulnerability, availability, and other quality metrics. Once you hold vendors accountable and threaten revenue, they’ll more seriously consider secure code review and more quality assurance cycles and we’ll see less flawed code.
One last point. I think the claims that incidents go unreported because those who report are penalized and embarrassed is rather suspect. The savvy companies I’ve consulted with work directly with their vendor when they identify an exploitable bug. They don’t make a lot of noise and fuss. The vendor assumes the responsibility for disclosure and this is probably fine; after all, how does DuPont or Goldman Sachs benefit from publishing a CVE? I worry that the notion that everyone needs to know every exploit at the zero hour is a bit overworked, and that it serves market push more than security.
Ian:
So here were my assumptions, which are important since security is not my primary line of work:
Assumption #1
As I understand it many security breaches in the corporate world go unreported:
-security staff afraid of reporting to management in certain cases
-corporate management afraid of reporting it to the industry (painting a target on their heads or shaking investor confidence)
Dave:
I think this is part Urban Myth. The rise to prominence of CSOs/ISOs gives officer/board level credibility to security in F1000s and there’s a trickle-down effect. The increased obligation to comply with regulatory guidelines that are incomprehensible to any but security professionals has made them much more valuable, and in turn, security professionals are more outspoken and demanding. In regulatory situations, reporting is mandatory; in other situations, I imagine reporting continues to vary per-incident, per-organization. I’d have to understand more how reporting leads to improved security to be convinced that unreported incidents are a root cause.
I have a more basic issue with this assumption, since it suggests that security incidents are all breaches, and all of equal and serious weight, which I know you understand is not true. However, absent a taxonomy and classification of breaches, I think it’s damned hard to determine what to report, to whom, and how to manage incidents (unless regulatory compliance demands reporting).
Examples:
- A configuration error at a firewall allows possible unauthorized disclosure of sensitive information for an undetermined period. When is this a reportable incident? To whom? How do you measure damage? Whether the organization can accurately determine what’s been touched, copied or altered and whether will they invest in the forensics to complete such an investigation and “go public” varies wildly and is hugely influenced by the regulatory environment in which the organization operates.
- Spyware infestations. An estimated 80-90% of PCs have spyware of one form or another. Even if this number is wrong by a factor of two, that’s lots more PCs than jets. Which spyware infestations are justifiably incidents? Again, no clear taxonomy and a constantly evolving threat.
- Use of unauthorized applications, unlicensed software/warez. Some say these are security incidents, others say no. Some companies spend lots of time and $ on this, others are entirely lax.
- Credential sharing/disclosure. Most security folks consider this an incident. I audited a company and found that 8 of 10 managers routinely shared credentials with secretaries and staff.
My point is that the realm of reportable security incidents is entirely too broad. Every company must decide what’s reportable, to whom, etc. according to the assets they value, regulatory environment, operating environment, etc.
This is a *very* different realm than the “bug tracking” world. BugTraq and others like them are basically external quality assurance. The model in place is entirely broken and would benefit from an operating environment that has the characteristics you list later: voluntary/confidential/non-punitive/independent.
Ian:
Assumption #2
As a result of there being penalties for reporting it would seem that the industry does not have good data on the quantity and scale of breaches which makes it difficult to band our collective intelligence together to create products (components of the architecture) which combine in a synergistic fashion so that better security emerges from the architecture not worse security due to added complexity.
Dave:
I think penalties are a small contributor to general lack of data. We have too little data for many reasons, including the ones I cited above, but ones I think are most important are (1) many companies don’t know the extent to which incidents occur (they remain undetected), (2) there’s no clear incentive to investigate incidents and hence no easy way to justify the cost (where’s the ROI in dedicating 2 engineers to a month-long analysis of disparate and non-synchronized logs to construct the exact anatomy of a successful attack?), and (3) the expertise to collect, assimilate the data is a limited resource that is almost always fighting other fires.
Ian:
Assumption #3
People might not die as a result of security issues but there are very large sums of money at stake so one would assume there could be an incentive to have this data available so both the product designers and security teams can focus on the top sources of pain and suffering.
Dave:
IMO, many calculations of loss due to attacks are pure alchemy. However, punitive damages by injured parties (e.g., the party whose HIV infection is disclosed) and fines for non-compliance to regulations *are* certainly incentives not to collect data, but to improve security deployment. Ask a CSO whether he thinks his security budget will yield better results:
(a) post-mortem analysis of security incidents for the sake of building reliable industry data.
(b) hardening servers and implementing admission control
BTW, product engineers, esp software engineers, should know exactly where the top sources of pain and suffering are: shoddy programming practices. If every data access in every program were tested to assure that buffers could not be overrun, we’d eliminate LOTS of the pain and suffering.
Ian:
Assumption #4
There is no such system/organization which allows easy reporting which is:
1) voluntary
2) confidential
3) non-punitive
4) independent
Dave:
Your points are excellent:
-incident handling and reporting have different characteristics since “death isn’t on the line”
-individual liability is low
-end users are not well trained
-revenue isn’t threatened
-disclosure of vulnerabilities happen through the vendors
So I am going to try to get more specific, say some crackers threatening (credibly) to bring down a large enterprise business, so lets pretend:
-huge sums of money that can threaten the revenue model of the company ARE on the line
-the only people involved in reporting to this system are the security savvy IT personnel
-the system will inform other IT personnel at other companies of this new risk
-that specific vendor disclosures of vulnerabilities is unlikely to prevent or help this
Now I have heard of a new kind of insurance – basically insurance against online extortion – and this does concern me since if companies can externalize the risk of having their network held ransom or having 100,000 customer records stolen THEN there really is no way to improve things and the burden of this activity will be carried by society.
I have to admit that it does seem that it all comes down to the economics and practical aspects of security which are damn complicated and simply aren’t as well controlled as, say, elements of risk in the aviation industry. But we have to strive to improve things somehow, no?
Oh I honestly believe we have to improve things. I think reporting is the tail wagging the dog. We need to fundamentally change the way code is developed and qualified. We don’t put code through exhaustive checks in the same way a manufacturer must to have his materials used in a space shuttle. The problem is that we must convince people that doing this up front, which clearly increases software costs, ultimately reduces the lifecycle cost of computing and networking.
We then have to improve how we configure and operate clients, servers and networking equipment, and how we monitor and assimilate the information we collect as we monitor.
Ian:
Dave,
Thanks so much for taking the time to enrich my understanding on this topic
Dave:
Always enjoyable chatting with you.