tag:blogger.com,1999:blog-13756280.post1351966142729193977..comments2024-02-08T03:44:23.780-08:00Comments on Jeremiah Grossman: Mythbusting, Secure code is less expensive to developJeremiah Grossmanhttp://www.blogger.com/profile/05017778127841311186noreply@blogger.comBlogger20125tag:blogger.com,1999:blog-13756280.post-35049953152254365562016-11-14T10:26:48.180-08:002016-11-14T10:26:48.180-08:00foreign education consultants in hyderabaddo you w...<br /><br /><br /><br /><a title="usa study abroad consultancies in hyderabad" href="http://rakabroad.in/usa-study-abroad-education-consultants/" rel="nofollow">foreign education consultants in hyderabad</a>do you want to study in abroad today or in the next intake. we are the best and top rated study abroad consultancies in india with good visa assurance.we help you in filing the f1 visa for you in very less time. we are also help you with information needed to apply for the college university. <br /><br />taiseerhttps://www.blogger.com/profile/06640706068951962972noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-84610718684606642772009-05-04T15:33:00.000-07:002009-05-04T15:33:00.000-07:00I am not so sure that the calculation for the cost...I am not so sure that the calculation for the cost of fixing the bug is really accurate either, though no one has really pointed that out.<br /><br />Fixing a bug costs more than just the time to find and fix it. Most large systems require extensive testing and the involvement of many people outside of the programmer.<br /><br />This factor can drastically increase the overall cost of the identified flaw. I would probably multiple the values by 3 to 5, at least, to have even a tiny bit of comfort in the amount.<br /><br />Of course, this still makes it not cost effective, but you cannot consider the cost of fixing a single vulnerability, even if you can identify all the instances of that single vulnerability at one time, which is highly doubtful.<br /><br />Comparing that cost against a process designed to prevent a wide range of vulnerabilities is not a valid comparison. It is like comparing the cost of a bag of oranges to the cost of the output of an entire orchard. (The cost of a QA check and the cost of preventative pest control.)<br /><br />The pest control may cost a whole lot more than a few bad bags, but can also eliminate the depredation that is less visible and may not be caught until much later, much like those security flaws that have not been found yet.<br /><br />The numbers would be dubious no matter what, but you need to add a whole bunch of costs if you really want to say whether an SDL is cheaper than the alternative, in the long run. This has many more factors that are indicated here.Brad Andrewshttps://www.blogger.com/profile/02449947300802682625noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-40655360738514333602009-05-04T07:45:00.000-07:002009-05-04T07:45:00.000-07:00I like the idea to "get the order of magnitude rig...I like the idea to "get the order of magnitude right", even if it is just a coarse approximation. However, the the calculation of $4000 per vulnerability is valid for applications only: The fix has to be applied on the few server sites, and that's it.<br /><br />However, even today not <I>all</I> software is web application software.<br /><br />On the other end of the spectrum there is software like e.g. Internet Explorer or Adobe Reader. I'm mentioning these two because they are pretty popular, and because both Microsoft and Adobe were taking part in the BSIMM interviews.<br /><br />Now consider the cost of a rollout of a security fix for this type of software, throughout the world. Even if we say that for the vendor it's still $4000 per vulnerability (neglecting the difficult-to-estimate effect of vulnerabilities on reputation): As a customer, I do feel a difference between a vulnerability which has been removed in the development cycle (invisible to me) and a vulnerability in my local software (forcing me to apply a fix).<br /><br />For the same type of software, I'd rather live with a difficult-to-exploit functional defect than with a difficult-to-exploit vulnerability. I have control over what <I>I</I> am doing, but not over what the bad guys will do.<br /><br />So I'd rather rewrite the initial statement:<br /><br />“If <I>we</I> (vendor) spend $X dollars implementing an SDL <I>you</I> (customer) will save $Y dollars.”<br /><br />The vendor might want to silently add $X dollars to the license fee, or to find other ways to turn this into a competitive advantage.Harald Jörgnoreply@blogger.comtag:blogger.com,1999:blog-13756280.post-61471723553503646972009-05-03T14:55:00.000-07:002009-05-03T14:55:00.000-07:00Hi, I found the original post quite interesting an...Hi, I found the original post quite interesting and I think we are talking about the different risk models used in projects and in production. <br /><br />I have a short post on my response here <br /><br />http://lukenotricks.blogspot.com/2009/05/28000-question-project-vs-production.html<br /><br />LukeAnonymoushttps://www.blogger.com/profile/16153635896554944056noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-84081706732603807962009-05-03T12:57:00.000-07:002009-05-03T12:57:00.000-07:00@anonymous, you don't have to hide your name to ma...@anonymous, you don't have to hide your name to make your opinion heard here. Misguided and incorrect as it may be.Jeremiah Grossmanhttps://www.blogger.com/profile/05017778127841311186noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-7911534354400551012009-05-03T12:55:00.000-07:002009-05-03T12:55:00.000-07:00@Jim, very well stated, thank you very much for sh...@Jim, very well stated, thank you very much for sharing. Great perspective and makes a lot of sense. I guess the only thing I'd add is if "security" is not important to the stakeholders from the beginning, which is not to say it absolutely should be, then that might complicates matters significantly later. Thanks again, you've added a lot of value to the discussion.Jeremiah Grossmanhttps://www.blogger.com/profile/05017778127841311186noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-56297864781164376642009-05-03T11:21:00.000-07:002009-05-03T11:21:00.000-07:00Wow...your apps must really suck if WhiteHat can c...Wow...your apps must really suck if WhiteHat can come in with a one size fits all scanner and some teenagers and bust it up. No wonder you're defensive.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-13756280.post-73401492054866041792009-05-03T10:32:00.000-07:002009-05-03T10:32:00.000-07:00Jeremiah, I'm a software development manager, not ...Jeremiah, I'm a software development manager, not a security expert, so I may come at this from a different perspective. I agree with your point that a vulnerability (a potential problem) is not the same as a bug (a manifested problem, found by the test team or the customer). Most bugs need to be fixed. But so do potential problems found in design and code reviews and by the static analysis tools - bugs that nobody has seen yet but are waiting to happen. <br /><br />Since I work in financial services, I have to satisfy my customer's (and the regulator's) requirements for performance and reliability and security in addition to the functional requirements of the system. So we manage functional bugs and feature requests, performance and capacity requirements, operations needs and security vulnerabilities through the same risk management and triage process, deciding where to focus resources, deciding what to do next.<br /><br />If I didn't have to consider security, things would be easier of course. But since I do, I am not going to leave it to the end: that would be risky and as Dan noted, difficult to predict and manage. To me the problem is the same as reliability and performance: if you know you need to hit an aggressive performance target from the start for example, then you need to set the stage and make performance a key driver in architecture and design, construction, testing, and infrastructure; take it into account in everything that you do. Trying to fix a major performance problem after the fact, if you got the requirements or architecture wrong, can be expensive, unpredictable and uncertain. Security is no different. Again, agreeing with Dan, it's easier to get and manage the resources and budget earlier, establishing performance and reliability and security needs and costs upfront, than to have to fight this fight later in the project - or, worse, wait until the system is in production and leave the problem to the maintenance and support team.<br /><br />So it just makes sense to me to build security in from the beginning, as one of the demands on the project and the team. The tradeoff between features, and security (and performance and reliability and so on), where you are going to spend your money and time, has to be made by the project stakeholders. If it's important to my stakeholders, I am going to manage it from the beginning. It's all the same to me: I gotta do what I gotta do to make the project a success, and starting off right gives me a better chance of success.Jim Birdhttps://www.blogger.com/profile/17371102366836131341noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-53675225462881936812009-05-02T09:55:00.000-07:002009-05-02T09:55:00.000-07:00As had been previously mentioned, your analysis do...As had been previously mentioned, your analysis doesn't take into account architectural flaws, and that is where a holistic SDL really pays for itself. XSS and SQLI are implementation flaws, and individually they can be very easy to fix (if you had pervasive XSS and wanted a consistent and unified solution for the product that still provided field specific validation, that might be a bit more expensive), but implementation flaws are easy to avoid from an SDL perspective anyway- you can simply provide a prescriptive list of coding standards (use prepared statements, use whitelist input validation and HTML character encoding, etc). Ideally you would also have some form of source analysis to make sure that the prescriptive standards are followed. Either way, the cost is cheaper than retroactively finding and fixing, even if it isn't the markup asserted.<br /><br />The really cost is when you need to do more than tweak an implementation. How much would it cost google or microsoft to change the way their web SSO to their services works because of a flaw? It isn't going to take 40 hours of work but hundreds of hours and a crap ton of testing. Design changes and architectural changes are expensive to make and it doesn't really matter what sort of software you write. The only thing that web apps have going for them is much cheaper deployment of fixes.Joshbwhttps://www.blogger.com/profile/01597375484321975241noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-68351432733422991802009-05-02T06:58:00.000-07:002009-05-02T06:58:00.000-07:00@Jim, and that's the weird dilemma in this kind of...@Jim, and that's the weird dilemma in this kind of calculation. The more maintainable you make the code (ie via ESAPI), the cheaper it is to fix security issues after the fact. However, one must take into consideration that you may not need to fix as many issues after the fact as well.Jeremiah Grossmanhttps://www.blogger.com/profile/05017778127841311186noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-69261812625416579002009-05-02T03:02:00.000-07:002009-05-02T03:02:00.000-07:00Wow, you really read this far. You MUST be thinki...Wow, you really read this far. You MUST be thinking, what WAS that podcast that Andre mentioned in the post above? I'm here to help.<br /><br />OWASP Podcast 18, an interview with Jeremiah Grossman can be found at http://www.owasp.org/index.php/Podcast_18<br /><br />But you're thinking, one at a time? How can I just subscribe to the entire OWASP Podcast Series?<br /><br />Great Question!<br /><br />Our RSS Feed is here: http://www.owasp.org/download/jmanico/podcast.xml<br /><br />And for your Steve Job's loving iTunes weenies: here you go: http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewPodcast?id=300769012<br /><br />PS: I use OWASP ESAPI in my codebase; my cost to fix XSS is approximately 50 cents/1 dollar per XSS. =)Jim Manicohttps://www.blogger.com/profile/12382834501997208557noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-61833425848284724022009-05-01T14:37:00.000-07:002009-05-01T14:37:00.000-07:00So in case you read my comments and got the wrong ...So in case you read my comments and got the wrong idea, I'm saying the average cost is reduced <B>only when the pieces of the SDL puzzle are already there</B>.Arshan Dabirsiaghihttps://www.blogger.com/profile/17228728745073712711noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-61253124327699346182009-05-01T14:26:00.001-07:002009-05-01T14:26:00.001-07:00I agree with other posters - their insight is grea...I agree with other posters - their insight is great and there are lots of statistically unsound assumptions here. The formula for vulnerability fixing cost does not consider the following facts:<br /><br />1) The 'average cost' figure hides the variance. The average cost across vulnerability type costs does not accurately reflect real typical cost. For example, if you have 9 XSS vulnerabilities and 1 massive authorization problem, your real mean vulnerability fix cost will be way less than you've shown.<br /><br />2) The 'average cost' to fix depends directly on how much security is in the lifecycle. If developers are trained, the right security mechanism/SME is already available then the cost will be much less per vulnerability.<br /><br />3) The text here assumes that also all functionality bugs are immediately fixed (false) and that security bugs must be fixed regardless of priority (false).<br /><br />4) The text here doesn't recognize and can't measure the fact that security problems are often built on top in ways that make it a "feature", aggravating cost even more exponentially in the application's downstream life.<br /><br />5) Many vulnerabilities can be addressed with a single fix. That murders your 'average cost' rate. Imagine an freshly installed AOP pointcut that installs an authorization check on all business logic - 30 lines of code just fixed 300 security vulnerabilities. Of course, I don't know exactly how you're counting vulnerabilities so this is arguable.<br /><br />Why do you make me fight about SDL? This shit is so boring.Arshan Dabirsiaghihttps://www.blogger.com/profile/17228728745073712711noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-43579646550606471062009-05-01T14:26:00.000-07:002009-05-01T14:26:00.000-07:00@Alex, and that's the exact calculations people do...@Alex, and that's the exact calculations people do! I've seen it personally, know other people have countless times as well.Jeremiah Grossmanhttps://www.blogger.com/profile/05017778127841311186noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-74379532946870477342009-05-01T14:19:00.000-07:002009-05-01T14:19:00.000-07:00whoo boy. Love the post.
It occurs to me that th...whoo boy. Love the post.<br /><br />It occurs to me that there's a missing element in these sorts of calculations - the opportunity cost of delaying web app production by "building/baking security in".<br /><br />So (this is coarse, I know - just bear with me) if I can earn $28,001 by rushing the app out early, it makes economic sense to do so.Alexhttp://newschoolsecurity.comnoreply@blogger.comtag:blogger.com,1999:blog-13756280.post-53018469804098638392009-05-01T13:38:00.000-07:002009-05-01T13:38:00.000-07:00@shrdlu, code proliferation! That's a really good ...@shrdlu, code proliferation! That's a really good point. Don't know how to projects those costs on anything close to generically, but hadn't considered it before.<br /><br />I'd also agree that the Myth is absolutely not busted. Still valuable to raise the discussion since the costs involved are not always straight forward.<br /><br />@sintixerr, agreed. Architectural / Business Logic Flaw issues can indeed be much more difficult and costly to fix after that fact. I mentioned that in the text, but perhaps it was clear enough. Still these issues could be just as cheap to fix as XSS/SQLi issues, it really all depends on the instance. Couldn't make for reasonable generic calculations.<br /><br />On the "often lead to fundamental cost-reducing changes", its not that I don't agree, but would be really interested in an case studies written you know of speaking to that. Would link to them in the future.<br /><br />@Dan, definitely needs more vuln types explored. I'll probably write a follow-up once I get a chance to digest all the feedback. It is time to start assigning some numbers to our (royal we) wisdom to see if the results actually achieve what we think they can.<br /><br />@Andrew, would love a link to that paper and would link to accordingly. For whatever reason this research has been hard to source.Jeremiah Grossmanhttps://www.blogger.com/profile/05017778127841311186noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-71081357426763041992009-05-01T12:36:00.000-07:002009-05-01T12:36:00.000-07:00At the risk of causing the flashing lights of the ...At the risk of causing the flashing lights of the Wayback Machine to produce seizures in rats, Kevin Soo Hoo, Andy Sudbury and I did a report in ~2001 on just this subject. ("Tangible ROI Through Secure Software Engineering", an @stake publication). It was the original ROSI paper, as far as I can tell.<br /><br />We found that most customers fixed nearly all of the quick hit items (easy to fix, high risk) and a lesser proportion of the gnarlier ones (hard to fix, high risk), for a total of about half of the defects we found -- the remainder being lower risk items that customers couldn't or wouldn't fix. We also used the 100x multiplier that was first famously floated in the ~1981 IBM System Sciences app research paper Jeremiah alludes to. Even so -- after accounting for the defect-fix rate we were seeing in the field -- we modeled that the ROI of secure software engineering was between 12-21%. Not bad.<br /><br />I've got the original paper, and would be happy to shoot it to you for posting.Unknownhttps://www.blogger.com/profile/14781280521548124865noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-47963599909716184352009-05-01T12:34:00.000-07:002009-05-01T12:34:00.000-07:00This is interesting analysis, but should probably ...This is interesting analysis, but should probably be expanded to address more types of vulnerabilities and to look at how vulnerabilities cluster in applications.<br /><br />If you get your fundamental coding idioms wrong, you're going to end up with a whole bunch of XSS or SQLi vulnerabilities. Fixing the first one might take a couple of hours (environment set up, tracking down the issue, testing the fix), but fixing vulnerability 1+N is pretty cheap because you just have to go to the next vulnerability and change the code. Rinse and repeat. It isn't fun work, but at least it is predictable to schedule. Project managers love this.<br /><br />Business logic issues are a lot more variable to fix. It could be as simple as adding a permissions check before allowing data access or it could require you to change the application requirements and then those changes have to flow down through the development process. Project managers like these less because they could take a lot or a little. Uncertainty and variability are not the project manager's friends.<br /><br />Architectural issues are the most variable and tend to start large and get enormous with regard to the level of effort required for the fix. I worked on one system where a single vulnerability (weakness in the way authentication worked) required a multi-year change control effort. Yikes! Needless to say, project managers (and line of business managers, executives, boards of directors...) like these the least.<br /><br />I like the idea of economically modeling remediation efforts, and I think the model should be expanded to look at the mix of vulnerabilities found in applications as well as the mix of applications that organizations have.<br /><br />I gave a talk about <A HREF="http://video.google.com/videoplay?docid=3200887090385342211&ei=FU37SdSiNomM-QH4gt3RAw&q=owasp+minneapolis" REL="nofollow">Vulnerability Management in an Application Security World</A> at OWASP Minneapolis/St. Paul a while back that is up on <A HREF="http://video.google.com/videoplay?docid=3200887090385342211&ei=FU37SdSiNomM-QH4gt3RAw&q=owasp+minneapolis" REL="nofollow">Google Video</A>. We talk about some of the economics surrounding fixing identified vulnerabilities in there a bit.<br /><br />--DanDan Cornellhttp://denimgroup.typepad.com/noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-68034659966029187332009-05-01T11:26:00.000-07:002009-05-01T11:26:00.000-07:00I'd like to add to what shrdlu said:
First, archi...I'd like to add to what shrdlu said:<br /><br />First, architectural flaws weren't accounted for here. Sometimes, fundamental mistakes are made at the design phase that, once implemented, are much more difficult to undo than a simple XSS vulnerability. This can occur not just in the code, but in assumptions made about the security of the surrounding environment. This is often where a lot of problems go unresolved due to direct and indirect costs (time). <br /><br />Second: You mention that <I>Frequently these activities require fundamental changes to the way software is built and business conducted.</I> I would suggest that, if properly implemented, an SDL and a Business Security Architecture may often lead to fundamental cost-reducing changes above and beyond security-specific or development-specific changes.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-13756280.post-3058491506231274872009-05-01T11:01:00.000-07:002009-05-01T11:01:00.000-07:00Interesting. I'd agree with your assertion that i...Interesting. I'd agree with your assertion that if it costs $4,000 to fix an XSS vulnerability, it'll cost the same regardless of whether it's during the design phase or after deployment -- IFF there's only the one instance. Catching the problem early may well prevent it from proliferating, especially if the code is reused elsewhere. If catching it early improves awareness among the developers, it may have a preventive effect. Also, fixing it in production requires another iteration of testing and code release, which adds marginally to the cost (QA people usually don't get paid as much as developers ;-). And finally, fixing it earlier guarantees that there is still money in the budget to do so; project managers tend to use up all available dollars when they believe they've "finished" an application, and in a lot of organizations that same $4,000 is impossible to come by later on; it skews the risk analysis because it's a separate, visible cost. Managers decide that the XSS vulnerability isn't so bad, compared to the risk of having to go back to ask for more money.<br /><br />So there are still plenty of good reasons to fix things early and often; I don't think the myth is busted just yet.<br /><br />/jamie wants big boomshrdluhttp://layer8.itsecuritygeek.comnoreply@blogger.com