tag:blogger.com,1999:blog-13756280.post4861129583303768690..comments2024-02-08T03:44:23.780-08:00Comments on Jeremiah Grossman: Battle of the Colored Boxes (part 1 of 2)Jeremiah Grossmanhttp://www.blogger.com/profile/05017778127841311186noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-13756280.post-64574777270276274792007-04-27T07:39:00.000-07:002007-04-27T07:39:00.000-07:00Thank you all for the comments, some excellent poi...Thank you all for the comments, some excellent points have been made that lend context to the discussion. I'm going to have to revisit my definitions and amend my post a little to align with the descriptions that I originally had in mind. Fortunately for me the feedback seems limited to that aspect and my "highlight" material seemed to make sense. Otherwise Im sure everyone would have said something instant.Jeremiah Grossmanhttps://www.blogger.com/profile/05017778127841311186noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-7531998438737283072007-04-26T18:54:00.000-07:002007-04-26T18:54:00.000-07:00@ntp:As example of black box testing, you talk abo...@ntp:<BR/>As example of black box testing, you talk about binary analysis; but it's not true. Binary Analysis can be white or black box:<BR/>- white box if you try to get information from your binary file (try to get x86 patterns etc.), when you read in that file.<BR/>- black box if you use your binary only as executable ie basically if you try to detect a buffer overflow with a fuzzer...Unknownhttps://www.blogger.com/profile/00365924018304230086noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-21457860700844137992007-04-26T15:47:00.000-07:002007-04-26T15:47:00.000-07:00The definitions of white box and black box testing...The definitions of white box and black box testing that this discussion is based on are incorrect. You link "black box" to wikipedia's "dynamic analysis" definition. Then you link "white box" to wikipedia's "static analysis" definition. No where in wikipedia's definitions are the terms "white box" or "black box" used.<BR/><BR/>I cover what these terms mean in Chapter 5, "Shades of Analysis: White, Grey, and Black Box Testing" in my book, "The Art of Software Security Testing". <BR/><BR/>Black box refers to the tester not being informed about the test subject. In black box testing, if the subject it is a web site, the only information the tester has is what he can gather by probing the site like an attacker. You can certainly white box test a web site dynamically. This would mean the tester had access to design documentation and source code. <BR/><BR/>The converse is true. You can statically black box test software. If the software is COTS the tester has the same information an attacker has. He must disassemble it with a tool such as IDA Pro.<BR/><BR/>These terms have been around for ages in the QA community. It is best to not redefine them. <BR/><BR/>-ChrisChris Wysopalhttps://www.blogger.com/profile/01919049944772664611noreply@blogger.comtag:blogger.com,1999:blog-13756280.post-59513996477715722032007-04-26T09:09:00.000-07:002007-04-26T09:09:00.000-07:00there are actually three types (i call them, "shad...there are actually three types (i call them, "shades") of greybox testing from my perspective:<BR/>1) whitebox side of greybox testing. this is my favorite method. the idea is to be able to fuzz and do fault injection on every input at full code coverage. the only tool i know that does this is jCUTE<BR/>2) blackbox side of greybox testing, which could be numerous things - but normally i think of binary analysis (e.g. BinAudit, BugScam, etc - both requiring IDA Pro)<BR/>3) the middle-ground, which could also be numerous things. i tend to think of this as runtime, memory, or core file debugging along with custom fuzz testing and/or fault injectors that attempt to get full code coverage. the primary tools in this area are PaiMei and BinNavi (which both require IDA Pro), or possibly OllyDbg with the hit-trace plugin. on the WAVA side, combining Firebug with a fault-injector would count<BR/><BR/>speaking of web application vulnerability assessment tools, normally these are very black or white. even FortifySoftware Tracer and SPI Dynamics DevInspect have a lot to learn from the non-webapp world of vulnerability assessment.<BR/><BR/>when i think of web application fault-injectors/scanners, i normally think of forced browsing / directory traversal (owasp t10 2007, section a10) - which isn't really "black-box" bug/flaw finding in my mind. sure, they also find real bugs/flaws such as xss or sql injection, but there are usually better tools and methods that do this manually - let alone business logic flaws such as csrf.<BR/><BR/>in all cases (white/black/grey, the 3 shades of grey, webapp vs. non-webapp), there is no universal answer to bug/flaw finding. each approach should be unique to the person/team (i.e. talent) working on discoveries and should be tailored to the specific application under test.<BR/><BR/>there are also many other methods (e.g. using virtualization, build tools, system analysis tools, etc) to speed up the process of finding bugs/flaws. finally, process and methodology can provide huge gains in time-to-finding, especially when dealing with a complex application.Anonymousnoreply@blogger.com