Computers & Security (2004) 23, 22e28

www.elsevier.com/locate/cose

TBSEdan engineering approach to thedesign of accurate and reliable security systems

John Leach)

John Leach Information Security Limited, Innisfree, Stoke Road, Smannell Andover,Hants SP11 6JL, England, UK

Received 11 November 2003; revised 11 November 2003; accepted 11 November 2003

–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

Abstract Formany years, the IT Security industry has been trying to devise away toquantify risk and the benefits provided by security countermeasures in a form mean-ingful to senior business management. Threat-Based Security Engineering (TBSE) isa fresh approach to modelling and forecasting information security risk. TBSE takesa non-deterministic approach to modelling how security threats interact with coun-termeasures enabling quantitative forecasts of the likelihood and characteristics ofsecurity incidents as a direct function of the security measures employed. Prelimi-nary results are encouraging and there appears to be no reason why the TBSE tech-niques could not be applied to a wide range of threats and countermeasures.Assuming they can, these techniques could become the foundation for a greatlyneeded disciplined engineering approach to the design of accurate and reliable se-curity systems. Amongst the many other benefits, this would give senior businessmanagement themuch sought after tools with which to oversee and direct corporatesecurity expenditures. This article describes the TBSE approach and what it can do.ª 2004 Elsevier Ltd. All rights reserved.

KEYWORDSRisk modelling;Risk forecasting;Risk quantification;Metrics;Security engineering;Security benefits;Security RoI

–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

Information Security practitioners have longsought, without success, methods which would en-able them to forecast in a reliable and measurableway the effects of different security measures onthe level and nature of security incidents experi-enced. Without such methods, practitioners donot have satisfactory ways to quantify security riskand security benefits in a form which businessmanagement finds helpful, or to design securitysystems which can be proven to address accurately

) Tel.: D44-1264-332-477.E-mail address: john.leach@johnleachis.com.

0167-4048/04/$ - see front matter ª 2004 Elsevier Ltd. All rights redoi:10.1016/S0167-4048(04)00069-0

the needs of the business and provide a reliablelevel of risk reduction.

As a consequence, senior business managementis not able to:

� evaluate whether a given security programmewill provide adequate protection to the business;

� measure the outcomes or benefits attributableto any security programme, or evaluate thereturn on investment achieved by expenditureon a given security programme;

� demonstrate to stakeholders, including share-holders and regulators, that the senior man-agement of the company has an appropriate

served.

TBSEdan engineering approach to the design 23

and cost-efficient security programme in placegiven the security needs of the business.

These are not small shortcomings. TBSE hasbeen created as a way to overcome the centralproblem behind these shortcomings, the problemof modelling and forecasting security outcomes.

The problem with deterministicmodelling approaches

Much effort has been spent over many years at-tempting to quantify risk. The overwhelmingmajority of these attempts have tried to deal withrisk in a deterministic way. Security incidents arethought of as being caused by well-defined pre-dictable and repeatable interactions and events.Put a certain set of circumstances together, in-cluding the full details of a specific threat and spe-cific countermeasures, and the outcome could befully determined as either certain success or cer-tain defeat.

The belief underpinning a deterministic ap-proach is that if one knew everything which oneneeded to know about the threat and the securitymeasures employed, one could predict exactlywhatthe various outcomes would be and the attendantlikelihood of each outcome. However, whether ornot this belief is sound in principle, it is of no usein practice. Much of what, within a deterministicmodel, one would need to know in order to deter-mine the success of an attack simply cannot beknown. The form in which a security incident occursis determined by a huge number of details many ofwhichcannotbeknown.Forexample,withahackingattack, one cannot know the motivation or state ofmind that day of the hacker, the suitability of theirindividual experience and skill at addressing theparticular challenge the attacked security defencesput before them, the amount of time they will keepgoing before they get bored, and so forth.

The problem with deterministic approachescomes from the fact that each security incidentis essentially unique. An incident can be seen asthe particular outcome of a particular set of cir-cumstances which just happened to come togetherin a particular way at the time the incident oc-curred. It is always possible to substantiate thatan incident which happened to one organisationwould not happen like that to another organisa-tion, because the details of how they run their ITservice, or the controls they have in place, or thestrength of their security culture, or the way theirstaff are trained, will be different. It is thisuniqueness of incidents which ensures that any

deterministic approach will inevitably fail due tothe impossibility of ever being able to capture suf-ficient data to support the analyses required.

An analogy will serve to substantiate this claim.Consider medieval siege warfare and an attack-

ing army firing a continuous shower of arrowsagainst defenders hiding behind a castle’s stoneperimeter wall. If an arrow strikes solid stone, itwill fall uselessly to the ground. If it just happensto find an arrow loop, the slit in the stone wallthrough which the defenders fire arrows out, thearrow will go through the wall and will have a goodchance of wounding or killing a defending soldier.If one knew everything there was to know aboutthe lie of the land, the weather, the time of day,the light, the bowman, his health and his state ofmind, the castle and its defenders, maybe onewould be able to determine the precise trajectoryof an arrow and determine whether that arrowwould find an occupied arrow slit or hit solid stone.However, such a deterministic analysis is clearlycompletely impractical as so many of the detailsessential to the calculation cannot ever be known.

Consider now the siege from the point of view ofthe defender firing arrows out through an arrowloop. The defender does not need to know whicharrow from which bowman is going to be the onewhich finds his arrow loop. He is interested inknowing just what the probability is of any arrowfrom any bowman finding his arrow loop. Hence,he does not need to know every detail about everyarrow fired. He needs to know only the density ofarrows in the arrow storm and the size of thehole his arrow loop makes in the stone wall in or-der to calculate the chance of any arrow strikinghis arrow loop. With the right description of theshower of arrows, a description which requiresjust a fraction of the total amount of informationwhich would be required for the deterministicanalysis of even a single arrow’s trajectory, the de-fender could calculate the probability of an arrowcoming through his arrow loop and wounding him.

Quickly consider a second analogy, water pres-sure against the hull of a submarine. An engineercan work out the rate at which water will leakthrough the hull by knowing just the external wa-ter pressure and how watertight the hull is. Theengineer does not need to know which water mol-ecules will be the ones to get through, just therate at which water will get through as a functionof the water pressure against the hull and howwatertight the hull has been made.

Each security incident is essentially unique, justas are the individual arrows which make it throughthe arrow loops or the individual water moleculeswhich need to be pumped out by the submarine’s

24 J. Leach

bilge pumps. Provided the engineer does not needto know exactly which instance of the threatwill be the one which defeats the defences, hedoes not need to try the clearly impractical task ofconducting a deterministic analysis. He can modelthe threat and the fraction of the threat that willget through his defences with a tractable non-deterministic analysis based on a verymuch simplerdescription of the threat. A non-deterministic anal-ysis of the security threat interacting with securitycountermeasures will allow a security engineer tocalculate the rate of security incidents as a functionof the effectiveness of the security measures inmuch the sameway that a castle engineer could cal-culate the likelihood of an arrow passing through anarrow loop as a function of the surface area of thearrow loop or a submarine engineer could calculatethe rate of leakage of water as a function of howwatertight the hull had been made.

The TBSE approach

TBSE takes a non-deterministic approach to model-ling risk. It models the interactions between threatsand security measures in such a way that the likeli-hood and characteristics of security incidents canbe forecast as a function of the security measuresemployed. This is, in outline, how TBSE works.

Security measures are classified into three clas-ses. The first class brings together those measureswhich resist a threat and work to prevent attacksfrom being successful. Anti-virus, patching, fire-walls, access control, are all examples of this type.The second class brings together those measureswhich do nothing to stop an attack being success-ful, but which work instead to make the attack’seffects less severe. Developing an incident re-sponse capability is an example of this type. An in-cident fire-fighting team does not come into playuntil after an incident has occurred, so its primarygoal is not prevention but rather to arrest the at-tack and contain the severity of its effects. Thethird class contains those security measures whichdo nothing to make the incident less likely or lesssevere but which work instead to reduce theimpact, the resultant business pain, caused bythe incident. Insurance and fall-back processesare examples of this type.

TBSE is careful to distinguish between the se-verity and the impact of an incident. Severity is ameasure of the intrinsic magnitude of the disrup-tion caused by an incident. Impact, which dependson severity as well as on other factors, is a measureof the business pain which the induced disruption

causes, i.e. the actual loss of revenue or valuecaused to the business by that disruption. Considerthe example of an outage. The severity of an out-age is measured by its duration. It is clear that theimpact of an outage will vary according to the se-verity (duration) of the outage. In addition, the im-pact of an outage of a given severity will varyaccording to the sensitivity of the business to theprocesses affected by the outage. A 2-hour outageof a bank’s trading floor will doubtless have a largerimpact on the bank’s business than a 2-hour outageof one of its rural branches.

The likelihood of an incident occurring is con-trolled by the interaction between the threat andthe first of the three classes of security measures.Within TBSE, this first class of security measures isreferred to as ‘‘Interdictive’’ security measuresas these work to resist the success of the threat.Interdictive interactions are modelled, in a non-deterministic way, as if they were a contest be-tween a threat with a certain ability to penetratedefences and a countermeasure with a certainability to resist that penetration. Clearly, thismodelling technique will work only if it is possibleto quantify the penetration ability of a threat andthe resistance ability of a countermeasure andthen to derive the function which describes howthe two interact. It turns out that this is quite sim-ple to do within the TBSE approach.

A sketched-out example will help to illustratehow this is done.

Example of applying the TBSEmodelling technique

Consider the simple scenario of a desktop PC ona broadband connection to the Internet protectedagainst e-mail-borne viruses by Anti-Virus (AV)software. The ability of the AV software to preventany virus exposure (the threat) causing an infec-tion of the desktop (an incident) is determinedby the promptness with which the user updatestheir desktop’s AV signature files. (This presumes,which is a fair approximation in today’s matureAV marketplace, that an AV product will havea 100% rate of success at resisting any virus whichis recorded in its signature file.)

In this case, the variable which describes theresistive strength of this particular Interdictivecountermeasure is the number of hours betweensuccessive signature file updates. Let us call thisvariable b. The corresponding variable whichdescribes the virus threat’s penetrability is thenthe age of the virus in hours since it was first

TBSEdan engineering approach to the design 25

Figure 1 The resistance function, R(a; b), as a function of a for a given Tupdate and b.

Figure 2 A possible number density for the virus threat, n(a), as a function of a.

released. Let us call this variable a. The resistancefunction, R(a; b), which describes whether a virusof age a hours defeats an AV countermeasureupdated every b hours is then a simple functionof a and b and a third parameter, Tupdate, whichdescribes how long, typically, it takes the AV ven-dor to make a new virus’ signature available forusers to download. It might look something likethe curve shown in Fig. 1.

The probability of virus infection at the desktopis determined from the resistance function,R(a; b), and the virus threat number density,n(a), the proportion of e-mail messages carryingviruses of age a. That number density could bemeasured readily by managed e-mail service pro-viders. They could record over a period of time,say a month, the average proportion of e-mailmessages carrying a virus as a function of that vi-rus’ age in hours. A virus threat number densitymight then look like the example shown in Fig. 2.

The probability, per Internet e-mail message re-ceived, of the desktop getting a virus infection canbe calculated as a function of the update period b,where it should be expected that the probability of

virus infection will grow with the update period b.This is indeed what the results predict, as is shownin Fig. 3 for the above example data.

A user can then decide what probability of infec-tion they are willing to tolerate for a given numberof Internet e-mails received and determine howfrequently they need to update their AV signaturefiles in order to achieve that desired level of protec-tion. This is shown, for the example data, in Fig. 4.

Fig. 4 can be read as follows.1 The ‘‘300e-mails’’ line shows that, for the user to have nomore than a 1% probability of becoming infectedwith an e-mail virus after receiving 300 Internete-mails, they need to update their AV signaturefile within 55 hours of their AV vendor posting virussignature updates. For that probability to fall to0.3%, the user’s updates need to happen within32 hours. For it to fall to 0.1%, the user’s updatesneed to happen within 7 hours.

The ‘‘10,000 e-mails’’ line shows that, for theuser to have no more than a 10% probability of

1 Do bear in mind that the results given here are for theexample data which might not reflect accurately the actualnumber density of the virus threat on the Internet.

26 J. Leach

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

2.5E-05

1 5 9 13 17 21 25 29 33 37 41 45

Update Period (Hours)

Prob

abili

ty

Figure 3 The probability of virus infection per Internet e-mail received as a function of the user’s signature fileupdate period, b.

becoming infected with an e-mail virus after re-ceiving 10,000 Internet e-mails, they need to up-date their AV signature file within 33 hours oftheir AV vendor posting signature updates. Forthat probability to fall to 3%, the user’s updatesneed to happen within 5 hours. The user cannotachieve a lower than 2.5% probability of becominginfected from 10,000 e-mails because, in this ex-ample, the AV vendor updates are not postedpromptly enough. This example has the AV vendorupdates posted on average 20 hours after the virus

is released into the wild. This shows that, no mat-ter how promptly the user applies available AV sig-nature file updates, there is always a chance theuser will be infected before their vendor makesthe virus’ signature available for detection. Thisis an expected result which TBSE can quantify.

One can also calculate how trends in the natureof the virus threat would change the curves in theabove figures. For example, if there were a trendfor e-mail viruses to propagate more rapidly, per-haps if e-mail viruses increasingly carried with

Figure 4 AV update periodicity in hours required to achieve a given level of protection for a given volume of e-mails.

TBSEdan engineering approach to the design 27

them their own mail code so that they couldpropagate without user action, the peak of themeasured n(a) curve in Fig. 2 would move to theleft, to lower values of a. This would causethe curve in Fig. 3 to move up the chart to showhigher probabilities of infection for all updateperiodicities across the range. All the lines inFig. 4 would then move to the right. As a conse-quence, if a user wanted to maintain a constantlevel of AV protection despite this trend, theywould need to apply AV signature updates morefrequently. This is the result we would expect.

Other more complex scenarios

The above worked AV example serves to illustratethe type of results which TBSE can generate. Forthat purpose, the example needed to be a simpleone. However, this is not to suggest that TBSE ap-plies only to simple scenarios.

In the example, it was implicitly assumed thatthe eventual infection of the desktop would notchange the subsequent number density of the virusthreat. This is clearly a suitable approximation forthe Internet virus threat for which the magnitudeof the threat felt at any time is the aggregated ef-fect of many thousands of on-line infected hostsaround the world. It is clearly not a valid assump-tion for the internal virus threat where, once onedesktop on an office LAN has become infected,that desktop then contributes to the threat envi-ronment for all the other desktops on the LAN. Inthese situations, the threat modelling needs tomodel the threat coming simultaneously frommore than one threat environment, and to modelthe evolution of the threat number density func-tion over time. The TBSE technique models thisquite naturally.

Multiple security measures, each contributingresistance to a given threat, can be included with-in the analysis by incorporating the resistancefunctions for each countermeasure. For example,if we take the above virus example and convertit into a worm example, a worm can be blockedin either of two ways: by its signature being inthe AV signature files or by software patching toremove the software vulnerability the worm ex-ploits. The two Interdictive measures, AV softwareand software patching, can be treated as workingin series, i.e. the threat has to dominate boththe AV countermeasure and the patching counter-measure to be successful. The TBSE techniquemodels this quite naturally, too.

Early results show that this modelling techniquecan be applied effectively to a variety of Interdic-

tive security measures, including anti-virus, patch-ing and firewalls. The TBSE technique can also beapplied to modelling internal threats where thethreat comes primarily from the different typesof inappropriate behaviour of staff. TBSE also mod-els the interactions between the threat numberdensity and the other two classes of security mea-sure, those which make incidents less severe andthose which make the impact less painful. Themodelling of these interactions is conducted ina different way (though still non-deterministically)from the modelling of the Interdictive interactions,reflecting the different nature of the interactions.

There appears to be no reason why this type ofanalysis cannot be extended to accommodatea wide variety of countermeasures, permittingthe modelling and forecasting technique to be ap-plied to a corporate IT infrastructure where hostcomputers might be exposed to a combination ofthreat environments, where threats are resistedby a combination of security measures, includingtechnical and management measures, and wherethe threat number density varies with time. Someof the countermeasure resistance functions willinitially be harder to derive than others. In suchsituations, the resistance function might need tobe approximated until, with the analysis of gath-ered data, it can be estimated more accuratelyand then modelled analytically.

What TBSE can offer

As was noted at the beginning of this article, theInformation Security industry has in the past beenunable to devise adequate methods for quantifyingrisk and forecasting security outcomes. TBSE isa process and set of techniques for forecastingthe quantified security outcomes resulting fromthe use of various amounts of different securitymeasures. It offers us the ability to design andbuild security measures or security programmeswhich are:

� Accuratedby enabling the designer to tailorand scale the design according to known andmeasured threat environments.

� Reliabledby enabling the designer to base thesecurity design on the known effectiveness ofeach security measure at resisting, mitigatingor alleviating relevant threats, thereby re-ducing the risk of the security design beingeither under- or over-engineered.

� Provabledby having the security design meetobjective and measurable security targets

28 J. Leach

derived from a suitable analysis of the busi-ness, need for protection.

� Effective and cost-efficientdby enabling thedesigner to optimise the security design inorder to minimise, in accordance with statedbusiness priorities, the cost of the design orthe impairment of other operational require-ments such as performance, ease of use,reliability.

The return on investment expected from a secu-rity design can be calculated by costing the designand comparing that with the forecast benefitsachievable as a direct result of the proposedsecurity expenditure. Calculations can be verifiedindependently to give management assurance. Ex-penditure on security measures can be justified interms which relate to the ensuing business benefit,making it easier for management to authorise ap-propriate security expenditures. The effectivenessof an implemented security programme can bemeasured and its success or failure at satisfyingthe security targets set can be established objec-tively according to agreed criteria.

TBSE will bring improved transparency to theway in which Information Security supports thebusiness, and senior business management will beable to strengthen its oversight and supervisionof its Information Security arrangements. Manage-ment will then be able to demonstrate to stake-holders (including shareholders and regulators)that they have good governance of their Informa-tion Security risks and an appropriate and cost-effective security programme in place given theneeds of the business.

In a wider context, TBSE can facilitate the de-velopment of a standard framework for quantify-ing and calibrating security solutions, facilitatingopenness and interoperability. It can provide op-portunities for service providers and product ven-dors to develop new services which quantify andforecast threat levels in useful terms. And it canfacilitate the development of a fully functioningdigital risk insurance market around productswhich can be reliably priced, offer effective coverand are simple to administer.

Conclusion

TBSE is a fresh approach to modelling threatdynamics which offers an attractive opportunityfor the Information Security industry to developa disciplined engineering approach to the designof accurate and reliable security systems and whichwill allow security designers to show business man-agement the benefits and return on investmentthey can achieve from their security expenditures.

Its applicationhas already shown thatmeaningfulresults can be generated, and indicates that thesetechniques should be applicable to a wide range ofthreats and security measures in a practical way. Aprogrammeofwork is being developedwhichwill al-low these early results to be built upon and themer-its of theTBSEapproachand techniquesproven. Theopportunity exists within this programme for com-panies toexpress their interest in theTBSEapproachand to contribute to or collaborate in TBSE projects.Any such expressions of interest, or requests fora more detailed description of the TBSE approach,should be directed to the author.

John Leach has been an Information Security professional forover 17 years. His career started in 1986 when he created andmanaged a small internal IT security team for the NatWestBank. In 1991, he joined Zergo and rose to become one of thethree Principal Consultants that headed up that consultancyteam during its most successful years. Since then he has createdand headed IT Security consultancy teams for Trusted Informa-tion Systems, Network Associates (by its acquisition of TIS),Global Integrity and Predictive Systems (by its acquisition ofGlobal Integrity).

He set up his own independent consultancy inDecember 2002.His principal areas of expertise are in risk and threat modelling,the creation of IT Security Improvement Programmes and Strate-gies, Network Security Strategies, and the design of ElectronicCommerce security systems. He specialises in innovative solu-tions to complex problems, bringing together his academic re-search grounding and his long experience working withcommercial organisations. He has worked for clients across mostsectors of industry including Financial Services, Oil and Petro-chemicals,Manufacturing, ITandTelecommunications,CivilGov-ernment and Defence, and the service sector. He has developedand delivered numerous training courses and workshops for cli-ents, and presented papers at public conferences on avariety of topics, most often on the subjects of Electronic Com-merce and Network Security.