FIDLAFaultInjectionDescriptionLanguage

forCompiler-basedTools

MaryamRaiyatAliabadiKarthikPattabiraman

UniversityofBritishColumbia(UBC)Canada

Introduction

2

FI:Asystematicwaytomodelfaults

Testingthecodeagainstfaultsisa‘must’

Background-LLVM

3

LLFI

❖Widelyusedinindustry[Lattner’05]

WeiJ,ThomasA,LiG,Pattabiraman K.Quantifyingtheaccuracyofhigh-levelfaultinjectiontechniquesforhardwarefaults.InDSN’14.

GenLLFIIndexPass

Profiling Pass FaultInjectionPass

FIInstSelector

InstTypeFIInstSelector

ControllerFICustomSelectorManager

FIRegSelecto

ProfilingLib Fault InjectionLib TraceInstructionLib

Fault Injector Manager

Fault Injector HardFaultInjectors SoftFaultInjectors

Fault-free Outcome

Profiling executable

Fault injection executable

Trace executable

Faulty outcomeTrace diff

Com

pile

Tim

eRu

n Ti

me

TracePass

Traces

API Data MemoryI/O Timing Messaging

SoftFIRegSelectors

Background- LLFI[ISSRE’14 ]

4

Auto-discoveryPass

PrimaryMotivation❖ HowtomakeLLFIprogrammable?

❖ ComplexcodingVS.Simplescripts

5

#include "llvm/IR/Instructions.h"#include "llvm/IR/Constants.h"#include "llvm/IR/IntrinsicInst.h"#include <fstream>using namespace std;namespace llfi {

class FuncArgRegSelector: publicSoftwareFIRegSelector {

public:FuncArgRegSelector(int target_arg) :

pos_argument(target_arg), specified_arg(true) {};FuncArgRegSelector():pos_argument(0),

specified_arg(false) {};private:

int pos_argument;bool specified_arg;virtual bool isRegofInstFITarget(Value *reg,

Instruction *inst);virtual bool isRegofInstFITarget(Value* reg,

Instruction* inst, int pos);};class FuncDestRegSelector: public

SoftwareFIRegSelector {private:

virtual bool isRegofInstFITarget(Value *reg, Instruction *inst);

boolFuncArgRegSelector::isRegofInstFITarget(Value *reg, Instruction *inst){

if(isa<CallInst>(inst) == false){return false }else{CallInst* CI = dyn_cast<CallInst>(inst);if(this->specified_arg == true){

Challenges- Whyanewlanguage?

6

Extensibility

distinctroles

combinatorialexplosion

Solution:FIDL

❖ FIDL: FaultInjectionDescriptionLanguage

❖ Efficiently makesSFItoolprogrammable

❖Simplified and accelerated faultmodeldesign

❖ Dynamically extendsSFItool

7

FIDL- Testmanagement

❖ Asystematicwaytofaultscenariodesign❖ Flexiblymanagescombinatorialexplosion

❖ Combinesheuristicswithfaultmodel❖ Securityanalysis

8

FIDL- Extensibility

❖FIDL: anAspect-OrientedProgramming(AOP)Language

❖ AspectsinFIDLscripts❖WeavesaspectsintothesourcecodeofSFItool❖AutomaticallyextendsSFItool

9

FIDL- Highlevelabstraction

❖Encapsulation❖Hidescompilerdetails❖Simplifiesfaultmodeldesignprocess❖SeparatesrolesoftesterandSFItooldeveloper

❖ Acceleration❖ Plug&playdesign

10

FIDLFI:Aprogrammableframework

❖FIDLFI=FIDL+Compiler-basedSFItool

11

FIDL Compiler(SFIdriver)

ApplicationUnderTest(AUT)FIDLscripts

(Faultscenarios)

Compiler-basedSFIEngineFaultmodels

FIDLScript

LLFI

FIDLCompiler Extended

LLFI

{Action, [Trigger, Target]}

AspectWeaver

{Advice, Point-Cut}Aspectspecification

Codebase

FIDLFIworkflow

12

Modifiedcode

Failure_Class:Failure_Mode:New_Failure_Mode:{

Trigger:<IRinstructions>

Trigger*:<RefinedIRinstructions>

Target:<IRregisters>

Action:<Corrupt/SetValue/Freeze/Delay/Perturb>

}

13

Faultinjectormodule

FIDLScriptSpecification

Faulttriggermodule

FIDLscriptExample- Heartbleed

1 Failure_Class: Memory2 Failure_Mode: Heartbleed3 New_Failure_Mode:4 Trigger: call: [memcpy]

6 Target: memcpy :: src[2]7 Action: Perturb :: CustomInjector 8 Custom_Injector:9 *Target = *Target + 10000;

14

ApplicationinjectedbyHeartbleed

FIDLscriptExample- Heartbleed

1 Failure_Class: Memory2 Failure_Mode: Heartbleed3 New_Failure_Mode:4 Trigger: call: [memcpy]5 Trigger*: [14,44] 6 Target: memcpy :: src[2]7 Action: Perturb :: CustomInjector 8 Custom_Injector:9 *Target = *Target + 10000;

16

ExperimentalEvaluation

• Experimentalsetup– Fivesamplefaultmodels– Fourstandardbenchmarks,andtheNull-httpdwebserver

– 2000-runSFIcampaignforeveryfaultmodel

• Evaluationmetrics– Timeoverhead– Implementationoverhead– Complexity

17

ExperimentalResults:Complexity

18

• 10Xcomplexityreduction

FaultModelFIDLScript

(LOC)FFM(LOC)

OFM(LOC)

BufferOverflow 9 96 68

Memoryleak 11 71 68

DataCorruption 8 64 61

WrongAPI 11 111 109G-Heartbleed 10 112 81

OFM (OriginalFaultModel):primarilydevelopedinLLFIinC++languageFFM(FIDL-generatedFaultModel):translatedfromFIDLscripttoC++code

ExperimentalResults:Spaceoverhead

19

• 4-18%spaceoverhead

FaultModelFIDLScript(LOC)

FFM(LOC)

OFM(LOC)

BufferOverflow 9 96 68

Memoryleak 11 71 68

DataCorruption 8 64 61

WrongAPI 11 111 109G-Heartbleed 10 112 81

OFM (OriginalFaultModel):primarilydevelopedinLLFIinC++languageFFM(FIDL-generatedFaultModel):translatedfromFIDLscripttoC++code

FFM(C++)

LLFIRecompilation

FIDLCompiler

4-5millisecond

2-3second

2-3second

OFM(C++)

FIDLscript

FaultinjectionintoApplicationUnderTest

OverallPerformance

20

LLFIRecompilation

• Maximumtimeoverhead:6.7%• Averagetimeoverhead:3.9%

21

Figure1.ComparingTimeoverhead(%)ofmemoryleak’sfaultmodelacrossbenchmarks.

TF(OFM) :AverageTimeoverheadforanoriginalfaultmodel

TF(FFM):AverageTimeoverheadforaFIDL-generatedfaultmodel

Experimentalresults:Timeoverhead

0

50

100

150

200

250

300

350

mcf sad cutcp blackscholes nullhttpd

TF(OFM) TF(FFM)

PreviousworkVsFIDLFIFITool Programmability Highlevel

AbstractionExtensibility

FIG DomainSpecificLanguage(DSL)

Yes No

LFI XML-based No No

FAIL* DSL(FAIL) Yes No

PREFAIL Policy-based Yes No

EDFI CommandLineArguments

No No

FIDLFI FIDL Yes Yes

22

Summary• FIDL:Fault Injection Description Language

– Drivescompiler-basedSFItools

– DevelopsfaultmodelsinaPlug&playfashion

– Reducesthecomplexityoffaultmodelsby10timeswithnegligibletimeoverhead.

• IntegratedintoLLFIFramework(BSDlicense)

https://github.com/DependableSystemsLab/LLFIraiyat@ece.ubc.ca

23