Basic Training
James A. Whittaker,
[email protected] Application
Penetration Testing
I
f you’ve ever had an account on a Unix or Linux machine, IBM—developers shifted the way
they were building software. The
chances are good that you’ve used the Pine email program. largest vendors began pushing secu-
rity into all stages of the development
In an early version, Pine had what vigilant administrators life cycle, and one of the most inter-
esting changes emerged in the way we
first thought was a minor performance issue: it created a test for bugs that might impact secu-
rity. This move ignited an industry-
temporary file when a user was editing an email message. Yet, wide awareness of security, which had
immediate impact in rapid produc-
H ERBERT H. what seemed like a pretty insignifi- The move to tion environments such as e-com-
THOMPSON cant issue from a quality assurance software security merce sites. We can expect this trend
Security perspective, turned out to be a gaping Many software vendors have now to continue as old products are re-
Innovation security hole when researchers dis- pushed quality assurance checks ear- placed with newer versions that have
covered that anyone on the system lier in the software development life benefited from this paradigm shift.
could read these temporary files (see cycle rather than just at the end. In In theory, tests that expose secu-
www.securityfocus.com/archive/1/ the past 10 years, we’ve also seen the rity flaws should easily merge into
150150 for more details). An attacker emergence of processes and tools existing quality practices and testing.
could read any message from any user that help translate requirements into In practice, however, the techniques
while it was being composed. code faster than ever before. These and skills needed to find security
Is this a bug though? It’s clearly a processes have allowed vendors to problems are drastically different
security issue of the highest severity, manage larger and more complex from those for locating functional
but it doesn’t fit the model of a tradi- software-development efforts with flaws. To understand why, consider
tional functional flaw. Users could tighter ship schedules and drool- how tests for functional problems
successfully compose and send mail, inspiring profit margins. As an in- are typically constructed. Tradi-
and developers likely executed such dustry we’ve made amazing progress, tional quality assurance has been
test cases successfully thousands of but we somehow overlooked some- about verifying a given system’s
times. The testers and developers just thing critical along the way: security. functionality—checking that a
failed to notice the side-effect of In the past five years, we’ve specification is implemented cor-
writing to temporary, unprotected learned that the processes, tools, and rectly in code and behavior. Model-
storage; typical software quality as- techniques that let vendors make ing languages such as the Unified
surance procedures aren’t well suited complex, specification-compliant Modeling Language (UML) make
to exposing this type of problem. software haven’t produced signifi- moving from requirements to test
Security bugs’ hidden nature is cantly more secure software. Even as cases a fairly seamless process. This
why we need specific, focused appli- new development paradigms like type of testing is good at verifying
cation-security testing techniques, extreme programming and the Ra- the presence of specified behavior,
testing that defies the traditional tional Unified Process gained trac- but it isn’t suited to finding the ab-
model of verifying an application’s tion (www.cert.org/stats/cert_stats. sence of unspecified behavior. Ex-
specification and instead identifies html), the number of new vulnera- posing such additional behavior de-
the unspecified and insecure side- bilities reported in software doubled mands more creativity from testers.
effects of “correct” application func- each year between 1999 and 2002. It requires out-of-the-box thinking
tionality. In this issue’s installment of The number of new software vul- and a process that tests against risks
Basic Training, I’ll examine applica- nerabilities then started to shrink in rather than traditional testing goals,
tion penetration testing—software 2003. What changed? Led by the such as verifying that the majority of
testing that’s specifically designed to biggest names in the industry— source code is executed by test cases
hunt down security vulnerabilities. companies like Microsoft and (code coverage).
66 PUBLISHED BY THE IEEE COMPUTER SOCIETY ■ 1540-7993/05/$20.00 © 2005 IEEE ■ IEEE SECURITY & PRIVACY
Basic Training
Creating a
threat model Threat 1:
Bypass authentication and gain access to a user's account
The first step in launching a security
testing project is to create a threat and
model, a detailed, written description
of the key risks to your application.
Developing this model is a creative,
collaborative process in which you 1.a 1.b 1.c
try to think like a bad guy in how to Site allows a large or unlimited Possible password combinations Site has an exploitable
number of password guesses are small (like a 4-digit PIN) command injection
use an application’s features, archi- before locking out and thus can be brute-forced (SQL, CGI, and so on)
tecture, and potential attack vectors. an account or IP address in a reasonable number of tries vulnerability
Once you flesh out these general
attack goals, you can build a hierar- and
chical tree of the preconditions to
render the threats realizable. 1.c.b.
1.c.a
User data is placed
As an example, let’s use a simple User data is not securely
insecurely into a command
Web application that lets users man- validated on the server
or shell on the server
age their bank accounts online. Be-
fore allowing access to an account, the
application prompts for a username Figure 1. Partial threat tree. This tree describes the conditions necessary for a particular
and personal identification number threat to be realizable.
(PIN). There are many potential
threats here, including gaining con-
trol over the bank’s Web server, creat- • tampering with data, actly when we’ll perform the testing
ing an arbitrary account, or changing • repudiation, (time of day, week, or month), the
an account’s balance illegitimately, • information disclosure, test accounts we will use, IP address
but let’s consider just one: accessing an • denial of service, and ranges from which the tests will
account for which we don’t have the • elevation of privilege. originate, the tests’ potential impact
PIN. At a high level, the threat is that on the production system, and so on.
an attacker will bypass the authentica- James Whittaker and I also present This information is key to enabling
tion or brute-force the password field several examples in our book of how system administrators to discrimi-
for a known user. to assess threats on a diverse set of ap- nate between testing activity and po-
Figure 1 shows a diagram of the plications,2 and Frank Swiderski and tential attacks.
conditions necessary to realize this Window Snyder examine the prob- The test plan should also include
threat. This type of model is known lem in depth in their recent book.3 a timetable of activities, along with a
as a threat tree. In this figure we see list and description of deliverables
several potential ways that attackers Building a test plan (such as problem reports and a final
could attain Threat 1. For example, The test plan is a roadmap for a secu- report). This helps the development
if both conditions 1.a. and 1.b. hold, rity testing effort, including a high- organization integrate results into
they can then gain unauthorized ac- level overview of the test cases, how the project timeline.
cess to a user’s account. Threat trees exploratory testing will be con- The most important part of any
and the threat-modeling process ducted, and which components will test plan is an overview of the test
help security testers break an exploit be tested. The plan must address sev- cases we will construct and execute.
end-goal into testable subgoals that eral key issues: Based on the threat model we con-
they can assess more easily. structed for the application, the test
Several techniques help software • logistics, cases are tied directly to application
testers, developers, and security audi- • deliverables and timeline, and risks. The test plan should also out-
tors create meaningful threat models. • test cases and tools. line the tools needed to conduct the
One of the best and most widely used tests, as well as any opportunities for
is the STRIDE method, developed We must include logistical issues automated testing and which tools
by Michael Howard and David such as the project schedule and the and techniques this testing requires.
LeBlanc.1 STRIDE is a useful people, resources, and equipment
acronym for general classes of threats: we will need. When testing produc- Executing test cases
tion systems, we need even finer- Conducting security tests is as much
• spoofing identity, grained detail on issues such as ex- an art as a science. The problem is
www.computer.org/security/ ■ IEEE SECURITY & PRIVACY 67
Basic Training
dow, text fields, buttons, and so on. failure that can’t be pinned down to
Testing resources Tests in this category span techniques a specific set of inputs and condi-
like parameter tampering, command tions generally requires extensive
injection, buffer overflow testing, developer effort to track down,
M any public vulnerability databases and mailing
lists are available online. The following sites are
invaluable resources for security testers.
and other methods of generating
corrupt and unanticipated inputs.
which makes it unlikely to be fixed.
Severity
• The Bugtraq mailing list (www.securityfocus. Design testing In contrast to functional bugs, we
com/archive/1) is one of the best repositories of secu- Designing secure software is a com- can’t measure a security bug’s sever-
rity vulnerabilities on the Internet. plex task. Software architects and de- ity by how many users are likely to
• The Computer Emergency Readiness Team (CERT) at velopers often overlook the implica- run into the failure through typical
Carnegie Mellon University is a center of Internet secu- tions a design choice has on an use cases. For example, an applica-
rity expertise and a good source of information and application’s security and the system tion crash that results when a user
statistics on security vulnerabilities (www.cert.org). that executes it. This category of at- goes through an unlikely input se-
• Common Vulnerabilities and Exposures (CVE; http:// tacks is designed to ferret out design quence will probably get a much
cve.mitre.org) is also an excellent source for vulnera- errors such as unsecured ports and lower severity rating than a bug
bilities reported in the field. default accounts. that’s present in a common user sce-
nario. For security bugs, however,
Implementation testing severity is based on the potential re-
that insecure behavior is often sub- Even if an application’s design is se- sult of the failure, regardless of how
tle—the creation of a temporary file cure, many bad things can happen obscure the input sequence that
or confidential information being when developers make specific im- found it. An accurate severity rank-
stored in a cookie, for example. plementation choices or errors. This ing is crucial because it heavily im-
Identifying when a test case “fails” category of tests focuses on exposing pacts vendors’ decisions regarding
requires an understanding of the some common implementation remediation steps—issuing an ur-
often obscure symptoms of insecu- vulnerabilities such as time-of- gent patch to a released product, for
rity. In How to Break Software Secu- check-to-time-of-use (TOCTOU) example, can incur tremendous cost
rity, James Whittaker and I con- issues, in which a time gap exists be- and potential reputation loss.
ducted an extensive study of these tween when an application checks
symptoms and distilled them and security on a particular function or Exploit scenarios
the techniques to find them into piece of data and when that privi- Convincing test and development
four groups of issues: those con- lege is used. managers of a bug’s exploitability
cerning dependency, user interface, presents one of the biggest chal-
design, and implementation. The problem report lenges in getting it fixed. For well-
The problem report is a critical out- understood problems, such as
Dependency testing put of any testing process. For secu- buffer overflows, the rule of thumb
Applications often rely on external rity problems, report formats can in most software development
resources to behave securely. In this vary, but they must at least include houses is that fixing the problem is
respect, most software development reproduction steps, severity, and ex- cheaper than determining whether
is done in a utopian environment, in ploit scenarios. it’s exploitable. Still, many organi-
which developer machines have zations need to see a direct path to
plenty of RAM and hard-disk space, Reproduction steps exploitability to justify the cost of
necessary external files are always For security bugs, the problem re- fixing the bug. While some secu-
available, and the registry is a safe port must clearly and unambigu- rity issues are relatively cheap to
haven. Tests in this section expose ously list the steps that another fix—replacing a dangerous func-
insecurities through the file system, tester or developer can follow to re- tion with a more secure alternative,
the registry, and external libraries by produce the failure. At the very for example—others require devel-
either denying the application access least, developers need to know opers to rewrite extensive portions
to these resources or tampering with what versions of the operating sys- of code. Exploit scenarios—the
and corrupting data. tem, application, relevant proto- specific sequence of things an at-
cols, and client are used and how tacker can do to take advantage of a
User interface testing they’re configured; ideally, they security flaw and the consequences
User interface testing delivers input should also have example code of doing so—thus play a critical
to the software the way developers showing the attack. These repro- role in describing a flaw’s impact to
anticipated: via the command win- duction steps are critical because a decision makers.
68 IEEE SECURITY & PRIVACY ■ JANUARY/FEBRUARY 2005
Basic Training
Postmortems or deployed products, particularly Acknowledgments
Bugs are corporate assets that should when bugs are uncovered in the Thanks to James Whittaker, Florence Mottay,
be treasured and studied. There’s no field. Many organizations have also Scott Chase, Fabien Casteran, Aditya
better way to understand what your found it valuable to study their com- Kakrania, and Scott Bradley for their contri-
organization is doing wrong than to petitors’ bugs to ensure that they butions to the application penetration testing
thoughtfully analyze the flaws that es- have methods in place to keep similar methodology at Security Innovation.
cape your normal development and bugs out of current projects.
testing processes. Postmortem evalua- References
tions help refine the testing process to 1. M. Howard and D. LeBlanc, Writ-
ensure that you can find bugs sooner
in future security testing endeavors.
The focus should be on what the bug
F unctional, specification-driven
testing is an important part of the
software development process, but it
ing Secure Code, 2nd ed., Microsoft
Press, 2002.
2. J. Whittaker and H. Thompson,
was, why it was missed, and how to isn’t enough. In this installment of How to Break Software Security, Pear-
improve the process to prevent or iso- Basic Training, I talked about some son Education, 2003.
late such issues. They can also help a strategies for application penetration 3. F. Swiderski and W. Snyder, Threat
team highlight holes in its testing testing that we at Security Innovation Modeling, Microsoft Press, 2004.
process and sharpen techniques to (www.securityinnovation.com) have
find certain classes of security failures. developed and successfully applied at Herbert H. Thompson is director of secu-
Typically, a postmortem entails Fortune 1000 companies to secure rity technology and research at Security
getting the test team into a room their applications. We’ve seen a defi- Innovation. His interests include software
after a project is complete to dissect nite, abrupt move in the software in- security, application security testing, and
secure development techniques. Thomp-
the bugs it found. Such evaluations dustry toward increased security over son received a PhD in applied mathemat-
are best done soon after the testing the past three years, but to make sus- ics from the Florida Institute of Technology.
project ends, when bugs are still fresh tained progress, corporations must He is the coauthor of several books on soft-
in the testers’ minds. Postmortems integrate application penetration ware security including How to Break
Software Security (Addison-Wesley
are also extremely valuable when testing and security training into their 2003). Contact him at hthompson@
performed periodically for released software development processes. securityinnovation.com.
The magazine that helps scientists to
apply high-end software
in their research!
Speed up and improve
your research
Specific tips
Focuses not on how computers work, from one scientist
but how scientists can use computers to another
more effectively in their research.
Top-Flight
Departments in $42 print subscription
Peer- Reviewed Theme & Feature Articles
Each Issue! Save 42% 2005 Jan/Feb New Directions
• Visualization Corner off the non-
• Computer Simulations Mar/Apr Cluster Computing
member price!
• Book & Web Reviews May/Jun Multiphysics Modeling
• Scientific
Programming Jul/Aug Earth System Research
• Technology Reviews Editor-in-Chief: Norman Chonacky, Sep/Oct Grid Computing
• CSE Education Yale University
• Your Homework Nov/Dec Monte Carlo Methods
Assignment
Subscribe to CiSE online at http://cise.aip.org and www.computer.org/cise
www.computer.org/security/ ■ IEEE SECURITY & PRIVACY 69
