The Economics of Security and Privacy

Ross Anderson

Cambridge University

Background

• Economics and security diverged after WW2; started coming back together recently

• Economists started thinking about crime and policing in late 60s, about privacy in late 70s

• Information security economics started growing five years ago

• Many new ideas in last couple of years• Workshop on Economics and Infosec every spring

Privacy - First Wave

• ‘Right to be left alone’, Brandeis 1890• Privacy violation as a tort - false light,

misappropriation, intrusion (Prosser 1960)• Westin, 1967 - data shadow, privacy as

informational self-determination• Inspiration for European data protection

movement

Privacy - Second Wave

• Becker 1968 - economic analysis of crime• Hirshleifer, 70s - conflict theory• Stigler, 1980 - free exchange of information brings

Pareto improvement regardless of ownership (bad debtors pay more regardless)

• Posner - poor employees want to hide data, good ones to reveal it; privacy inefficient, redistributive

• Noam - PETs may change who pays but not what happens - they just redistribute (poor to rich)

• Price discrimination is efficient (albeit unpopular)

Economics of Information Security

• Over the last four years, we have started to apply economic analysis to information security

• Economic analysis often explains security failure better then technical analysis!

• Information security mechanisms are used increasingly to support business models rather than to manage risk

• Economic analysis is also vital for the public policy aspects of security

Traditional View of Infosec

• People used to think that the Internet was insecure because of lack of features – crypto, authentication, filtering

• So engineers worked on providing better, cheaper security features – AES, PKI, firewalls …

• About 1999, we started to realize that this is not enough

New View of Infosec

• Systems are often insecure because the people who could fix them have no incentive to

• Bank customers suffer when bank systems allow fraud; patients suffer when hospital systems break privacy; Amazon’s website suffers when infected PCs attack it

• Security is often what economists call an ‘externality’ – like environmental pollution

• Provides an excuse for government intervention

New Uses of Infosec

• Xerox started using authentication in ink cartridges to tie them to the printer

• Followed by HP, Lexmark … and Lexmark’s case against SCC, and EU Parliament Directives

• Motorola started authenticating mobile phone batteries to the phone

• BMW now has a car prototype that authenticates its major components

IT Economics (1)

• The first distinguishing characteristic of many IT product and service markets is network effects

• Metcalfe’s law – the value of a network is the square of the number of users

• Real networks – phones, fax, email• Virtual networks – PC architecture versus MAC,

or Symbian versus WinCE• Network effects tend to lead to dominant firm

markets where the winner takes all

IT Economics (2)

• Second common feature of IT product and service markets is high fixed costs and low marginal costs

• Competition can drive down prices to marginal cost of production

• This can make it hard to recover capital investment, unless stopped by patent, brand, compatibility …

• These effects can also lead to dominant-firm market structures

IT Economics (3)

• Third common feature of IT markets is that switching from one product or service to another is expensive

• E.g. switching from Windows to Linux means retraining staff, rewriting apps

• Shapiro-Varian theorem: the net present value of a software company is the total switching costs

• This is why so much effort is starting to go into accessory control – manage the switching costs in your favour

IT Economics and Security

• High fixed/low marginal costs, network effects and switching costs all tend to lead to dominant-firm markets with big first-mover advantage

• So time-to-market is critical• Microsoft philosophy of ‘we’ll ship it Tuesday

and get it right by version 3’ is not perverse behaviour by Bill Gates but driven by economics

• Whichever company had won in the PC OS business would have done the same

IT Economics and Security 2

• When building a network monopoly, it is also critical to appeal to the vendors of complementary products

• E.g., application software developers in the case of PC versus Apple, or now of Symbian versus CE

• Lack of security in earlier versions of Windows makes it easier to develop applications

• Similarly, motive for choice of security technologies that dump the support costs on the user (e.g. SSL, PKI, …)

Why are many security products ineffective?

• Akerlof’s Nobel-prizewinning paper, ‘The Market for Lemons’ provides key insight – asymmetric information

• Suppose a town has 100 used cars for sale: 50 good ones worth $2000 and 50 lemons worth $1000

• What is the equilibrium price of used cars in this town?

• If $1500, no good cars will be offered for sale …• Usual fix: brands (e.g. ‘Volvo certified used car’)

Security and Liability

• Why did digital signatures not take off (e.g. SET protocol)?

• Industry thought: legal uncertainty. So EU passed electronic signature law

• Recent research: customers and merchants resist transfer of liability by bankers for disputed transactions

• Best to stick with credit cards, as any fraud is the bank’s problem

• Similar resistance to phone-based payment – people prefer prepayment plans because of uncertainty

Why Bill wasn’t interested in security

• While Microsoft was growing, the two critical factors were speed, and appeal to application developers

• Security markets were over-hyped and driven by artificial factors

• Issues like privacy and liability were more complex than they seemed

• The public couldn’t tell good security from bad anyway

Why is Bill changing his mind?

• ‘Trusted Computing’ initiative ranges from TCG and NGSCB to the IRM mechanisms in Office 2003

• IRM – Information Rights Management – changes ownership of a file from the machine owner to the file creator

• Files are encrypted and associated with rights management information

• The file creator can specify that a file can only be read by Mr. X, and only till date Y

• What will be the effect on the typical business that uses PCs?

Why is Bill changing his mind? (2)

• At present, a company with 100 PCs pays maybe $500 per seat for Office

• Remember – value of software company = total switching costs

• So – cost of retraining everyone to use Linux, converting files etc is maybe $50,000

• But once many of the documents can’t be converted without the creators’ permission, the switching cost is much higher

• Lock-in is the key!

Open or Closed?

• Free/open source view - easier for defenders to find and fix bugs (‘to many eyes, all bugs are shallow’)

• NSA view - easier for attackers to find and exploit bugs

• Under standard reliability growth model assumptions, openness helps attackers and defenders equally

• Whether open or closed is better will depend on how your system departs from the ideal

How often should we patch?

• Big topic at WEIS 2004, two weeks ago• Rescorla: bugs independent, most exploits follow

patching - so we should never disclose vulnerabilities or ship patches

• Arora, Telang, Xu: under different assumptions, we should cut disclosure delay

• Arora, Telang et al: some empirical evidence - disclosure increases attacks, patching cuts

• Ozment - auction theory may give some ideas

How are Incentives Skewed?

• If you are DirNSA and have a nice new hack on NT, do you tell Bill?

• Tell – protect 300m Americans• Don’t tell – be able to hack 400m Europeans,

1000m Chinese,…• If the Chinese hack US systems, they keep quiet.

If you hack their systems, you can brag about it to the President and get more budget

Skewed Incentives (2)

• Within corporate sector, large companies spend too much on security - small companies too little

• Adverse selection effect: the most risk-averse people end up as corporate security managers

• More risk-loving people may be sales or engineering staff, or small business entrepreneurs

• Also: due-diligence effects, government regulation, insurance market issues

• We tolerate attacks on stuff we already know to be useful (smartphone viruses worse than PC viruses)

How Much to Spend?

• How much should the average company spend on information security?

• Governments, vendors: much much more than at present

• They’ve been saying this for 20 years!• Security ROI may be about 20% p.a.• So current expenditure maybe about right (but too

little in small firms and too much in governments, big companies)

Privacy - Third Wave

• Varian 96 - privacy as the right not to be annoyed by direct marketers - define rights better

• When sending marketing pitches was expensive and evaluating them was cheap, we got too few messages and bought magazines. Now it’s the other way round and we buy spam filters

• Huang 98 - regulation helps construct privacy preferences by steering people to one of many equilibria, which then stick

Privacy (cont’d) - Social Level

• Odlyzko 2001 - pressure to price-discriminate is the main threat to privacy, and technology is making it steadily worse

• End of bubble: privacy technology ventures had mostly failed - yet privacy costs billions, to business and consumers (Gellman 2002)

• Taylor 2002: if data trading covert, firms gain more; otherwise high-value customers back off

• Chellapa 2002: perceived security, privacy separate but correlated; it’s better for a firm to be trusted with privacy rather than just trusted

Privacy Themes - WEIS 2003

• Privacy paradox - most people say they value privacy, but act otherwise

• May be due to myopic consumers (Syverson)• Lemons market for retailers (Vila, Greenstadt,

Molnar)• Need a concrete solution to a clear threat

(Shostack)• Shoppers care about privacy when buying clothes,

but not cameras! Sensitivity focuses on items relating to personal image (Acquisti, Grossklags)

Privacy (cont’d) - social level

• Varian / Wallenberg / Woloch, WEIS 2004 - privacy as ‘do not call’ strongly correlated with income - large study with DNC records

• Mialon & Mialon 2004 - privacy as 4th amendment rights which cut intrusion directly but increase it indirectly (more crime). Technology lowers search costs -> society moves to exterior equilibrium of Swiss or Afghan type, depending on police accountability

Privacy - mechanism level

• What sort of incentives will make people participate in remailer / P2P networks etc?

• Acquisti / Dingledine / Syverson - free-rider problems in mix-nets, and options for clubs, reputation systems, preferential service etc

• Danezis / Anderson - discretion is better• There’s now a whole workshop for P2P

economics - many issues go across to privacy

Conclusions

• Security and privacy spending seems to be determined in complex ways by assorted market failures

• Firms, and governments, generally spend too much on security - they are risk-averse

• Too little gets spent on privacy - consumers don’t care as much

• To say much more, you have to be more specific about the type of security or privacy! Ultimately it’s all about power

More …

• Economics and Security Resource Page – www.cl.cam.ac.uk/~rja14/econsec.html (or follow link from my home page

• Economics of Privacy Page – www.heinz.cmu.edu/~acquisti/economics-privacy.htm