functional testing of digital systems

8/8/2019 Functional Testing of Digital Systems

1/7

Functional Testing of Digital SystemsK w o k - W o o n L a iBell LaboratoriesMurray Hill, New Jersey 07974

D a n i e l P . S i e w i o r e kCarnegie-Mellon UniversityPittsburgh, Pennsylvania 15213

A B S T R A C T

F u n c t i o n a l t e s t i n g i s t e s ti n g a i m e d a t v a l i d a t i n g t h e c o r r e c to p e r a t i o n o f a d i g i t a l s y s t e m w i t h r e s p e c t t o i t s funct ionalspecif icat ion. W e h a v e designed and implemented a p r a c t i c a l t e s tg e n e r a t i o n m e t h o d o l o g y t h a t c a n g e n e r a t e t e s t s d i r e ct l y f r o m as y s t e m ' s h i g h - l e v e l s p e c i f ic a t i o n . S o l u t i o n s a d o p t e d i n c l u d em u l t i - le v e l f a u l t m o d e l s a n d m u l t i -s t a g e t e s t g e n e r a t i o n . T e s t sg e n e r a t e d f r o m t h e m e t h o d o l o g y w e r e c o m p a r e d a g a i n s t t e s tp r o g r a m s s u p p l i e d b y a c o m p u t e r m a n u f a c t u r e r a n d w e r e f o u n dt o d e t e c t m o r e fa u l t s w i t h m u c h b e t t e r e f f i d e n c y . T h ee x p e r i m e n t d e m o n s t r a t e d t h a t f u n c t i o n a l te s t in g c a n b e b o t hpract i cal and eJ~cient . A u t o m a t i c g e n e r a t i o n o f d e s i g nv a l i d a t i o n t e s t s i s n o w c l o s e r t o r e a l i t y .

1 . I n t r o d u c t i o nF u n c t i o n a l t e s t i n g is testing aimed at validating the correctoperation of a digital system with respect to its functionalspecification. Conv ent iona l testing methodologie s, bycontrast, generate tests so le ly based on the physical ~structure(e.g. interc onne ction s, comp onen ts used) of a system. Theiraim is to diagnose hardware failures and they offer littleassurance about whether a system has been designedcorrectly in the first place.

1 . 1 D e s i g n F a u l t sDesign faults are non- physical faults caused by imperfectionsintroduced into a system during its design stage. Our originalmotivation for research into automatic test generation (ATG)at the functional level was the problem of architecturevalidation - validating that a prototype correctly implementsthe instr uction set of a specified comput er architecture. Asecond motivation is the high cost of design errors in VLSIcircuits, where field engineer ing changes are no longerpossible.Currently, the functional correctness of most new systemsare being checked by ad hoc methods. Test programme rs are

* R~ear.da eonduetes:l while principal autho r was a graduate s tudent art...amegae-Mellon University. Th is research was spon sored by theD e f e n s e A d v a n c , ~ a ~ e a r e ~ . r oj ec t' s s e n c y ( D O D ) , A R P A O r d e r N o .~ / , tuomturen oy me A~r F6rce AvaonicsLaboratory u n d e r contrac~F 3 3 6 1 5 - 7 8 - C - I 5 1 .The views and conclusionscontained n this document are those of theauthors and thould . n o =b.e inte]!pr.exed~ representing he official polities,earner expressen or map lien, ot me /Jefense Advanced Researchvroject~Agency or the U.S. Government .

often given the tedious task of writing validation programswith vague guidelines such as "exercise each instruction atleast once" and left to their own ingenuity. For a newmember of an existing computer family, these programs maybe supplemented by operating systems and applicationprograms which have not been written with testing in mind.It is hardly surprising that such ad hoc approaches often donot give satisfactory quality assurance. Good e videncedemonstrating the inadequacies of present testing techniquesis expensive design errors discovered in the field for manycomput er families. A functiona l testing methodology is verymuch needed.1 . 2 P h y s i c a l F a u l t D e t e c t i o nThe usefulness of functional testing, however, goes beyonddesign validation. It may even be able to supplant traditionalcircuit testing in detect ing physical faults. With the numb er ofcomponents per chip doubling every one to two years,traditional test generation methods are apparently coming toa dead end. Many ATG systems based on circuit testingtechniques have become unusable due to combinatorialexplosions in their computer run times. When usersgenerate tests for their systems piecemeal, they find that thecomplexity has lowered fault coverage while requiring everlarger numbers of test vectors.This growing complexity can be made more manageable byspecifying a system at the functional level in a hierarchicalmanner and generating tests at this level using efficientheuristics. Functi onal tests may actually be more efficientthan tests generated from a circuit diagram using traditionalapproaches. Whereas implemen tation-spec ific tests are morethorough in detecting individual physical faults, functionaltests exercise a system at a higher level, perhaps activatingmany physically distant componen ts simultaneously, and canpotentially detect mo re faults in shorter time than tests thatfocus on one spot of the hardware at a time. As in manyother areas of computer science, global efficiency may proveto be far more important than local efficiency.1 . 3 A d v a n t a g e s o f F u n c t i o n a l T e s t in gAutomatic test generation at the functional level offers thefollowing advantages over ad hoc approaches:

Be t t e r q u a l i t y - tests would be generated according toscientifically established fault mo dels rather than thepersonal judgm ent of test programmers. The tests shouldbe much more thorough. A u t o m a t i o n - test generation time and cost can bedrastically reduced, freeing skilled manpower. Tests can

20th DesignAutom ation Conference0738-100X/83/0000/020751.00 1983 IEEE Paper 15.3207


2/7

be generated as soon as the functional specification of thesystem becomes available, even before logic design isunderwa y, and keep up with any changes. Such anautomatic bridge between specification andimplementation is extremely useful in development anddebugging and can significantly shorten the developmentcycle. Project managers can count on tests of guara nteedqualities being generated on schedule, rather than live inthe constant fear of unsatisfactory tests and scheduledelays.

1 .4 Rev iew o f L i t era tu reAkers Ill have pr oposed the us e of binary decision diagrams ingenerating "experiments" for simple digital devices such asflip-flops and gates. His work is amon g the first attempts togenerat e tests based on formal func tional descriptions otherthan boole an equations and flow tables. Binary decisiondiagrams, however, are not adequate for the description ofmore complicated digital systems. Breuer and Frie dman [21have extended traditional circuit testing approaches to handle"functional level primitives" such as adders and shiftregisters. Their m ain concern is hardware fault diagnosis andtests are still generated from the circuit diagram of thehardware to be tested. Because the structure of theiralgorithms are the same as in the original methods, theirextension inherits circuit testing's problem of combinatorialexplosion. Thatte and Abraham TM have proposed amethodology of testing microprocessors. Their chief concernis again hardware fault detection. Their model describes datatransfer among registers and the main memory but does notinclude control and data transformation functions. Based onthis model they have proposed a number of test generationalgorithms. Other researchers interested in hardware faultdetection have also looked into the problem of ATG formicroprocessors. In the intere st of space, their work will notbe discussed here.

2 . Met h od o logyOur methodo logy is an ambitious one. It generates testsdirectly from the functional specification of a digital systemusing multi-level fault models and multi-stage testgeneration. Major compone nts of the methodology areshown in Figure 1. Functional analyzer is the heart of the testgeneration system. Its inputs are the digital system's graphdescription and graph level fault models chosen by the user.For each fault model and each graph primitive covered bythe model, it generates a parameterized test that detects thefault at that primitive. The test case synthesizer thensubstitutes the formal parameters by bit patterns obtainedfrom the test pattern database. These test vectors can then bemapped by the test program synthesizer into test programsegments that would actually carry out the tests.Any practical functional testing methodology must necessarilybe a compromise between generality and practicality.Functional approaches that assume nothing about how asystem is implemented would require astronomical numbersof tests to cover every possibility. For test gene ration to befeasible, one has to make assumptions about theimplementation. Since the choice of fault models directlydetermines the cost of test generation and the effectivenessof the tests generated, our approach is to let users decide fo r

l " . . . . . . . . . . . . . . . . " - I, F u n c t i o n a l S p e c i f i c a t i o n ,I !, (Grap h Description) ,I . . . . . . . . :

. . . . Fa_~_ M_od_ els_ . . . "

P a r a m e t e r i z e d T e s t s

T e s t C a s e S y n t h e s i z e r

T e s t C a s e s

T e s t P r o g r a mS y n t h e s i z e r

T e s t Program

P r i m i t i v e L e v e lF a u l t M o d e l s1

l ~ a t a B a s e

F i g u r e 1. C o m p o n e n t s o f the Methodologythemselves - they can choose predefined fault models ordefine their own.

2 . 1 G r a p h D e s c r i p ti o n L a n g u a g eA functional specification language suitable for testgeneration was needed, so we designed an extensible graphdescription language called s t a t e t r a n s f o r m a t i o n g r a p h s(STG). STG can describe digital systems at levels of detailranging from user-defined primitives of arbitrary complexitydown to logic gates. While an STG graph serves as asystem's functional specification, it also facilitates theexpression of structural assumptions about the system whichare vital for practical test generat ion. This is a tradeoffbetwee n generality and practicality. By allowing the u ser toinclude as few impl emen tati on assu mption s as is practical, asmuch generality as possible in the tests generated ispreserved.A user interested in physical fault detection is free toconstruct a description the nodes and paths of which have aone to one correspondence with physical components in thesystem. With such a description and appropriate fault models,tests generated by our methodology would actually beidentical to those generated by a traditional method from thesystem's circuit diagrams. In other words, our methodologyactually contains traditional circuit testing as a special case.We should emphasize, however, that STG is intended to be afunctional specification language used by designers to specifytheir systems.Associated with each STG graph is a set of state variableswhich can be read and written by read/write operators,together they represent a finite state machine. Even thoughSTG is intended to be a functional specification languageused by designers, it can also describe arbitrary

Paper 15.3208


3/7

I f r i n g e 1 I ~ l l < "D a t a O p e r a t o r D e c i d e r W r i t e C e l l

I I . . . I I . . . Ic e U - > I l a r r a y t 1 - > I J ~ a y [ l < - II I iR e a d C e l l R e a d M e m o r y W r i t e M e m o r y

I iI i . . . i i i . . . iD e c o d e D e m u l t i p l e x e r

J u n c t io n D a t a F l o w G a t e sF i g u r e 2 . S o m e S T G P r i m i t i v e s

c o m b i n a t i o n a l a n d s e q u e n t i a l c i r cu i t s . A n S T G g r a p h i s ad i r e c t e d g r a p h i n w h i c h p a t h s r e p r e s e n t ( l o g ic a l ) d a t a / c o n t r o lp a t h s w h i l e n o d e s r e p r e s e n t d a t a t r a n s f o r m a t i o n o p e r a t o r s .D a t a i s p a s s e d a l o n g i n t h e f o r m o f tokens, w i t h a t m o s t o n et o k e n p e r p a t h . A n o d e c a n .f ire w h e n i t h a s t h e r e q u i r e dt o k e n s o n i t s in p u t p a t h s . T h e f i ri n g o f a n o d e r e s u l t s i n t h er e m o v a l o f i n p u t t o k e n s a n d t h e p l a c e m e n t o f n ew t o k e n s o ni t s o u t p u t s . E v e r y g r a p h h a s t w o s p e c ia l n o d e s : b e g i n a n de n d . A t t h e b e g i n n i n g o f e ac h c y c l e , a t o k e n i s f i r e d f r o mt h e b e g i n n o d e w h i c h s u b s e q u e n t l y a c t i v a t e s o t h e r n o d e sd e p e n d i n g o n t h e c u r r e n t s t a t e . E v e n t u a l l y a t o k e n a r r i v e s a tt h e e n d n o d e s i g n a l l in g t h e e n d o f t h e c y cl e . S o m e b a s i cS T G p r i m i t i v e s a r e s h o w n i n F i g u r e 2 . A s a n e x a m p l e , t h et o p l e v e l g r a p h d e s c r ip t i o n o f t h e P D P - 8 m i n i c o m p u t e r i sg i v e n i n F i g u r e 3 a n d t h e g r a p h s f o r t w o o f i t s i n s t r u c t i o n s ,A N D ( lo g i ca l a n d ) a n d J M S ( j u m p s u b r o u t i n e ) , a r e s h o w n i nF i g u r e 4 . D e t a i l e d d e f i n i t io n o f S T G c a n b e f o u n d i n [ 6 ].

2.2 Graph Level Fau lt ModelsG r a p h l e v e l f a u l t m o d e l s c a n b e v i e w e d a s " m a c ro " f a u l tm o d e l s t h a t m o d e l f a u l t s a t th e g r a p h l e v e l a n d a r e t h e r e f o r ef u n c t i o n a l f a u l t m o d e l s . E a c h m o d e l s p ec i f ie s t h e k i n d o fg r a p h p r i m i t i v e s c o v e r e d b y t h e m o d e l , a n d g i v e n s u c h ap r i m i t i v e , w h e r e t h e t e s t d a t a s h o u l d b e a p p l i e d a n d w h e r et h e t e s t r e s u l t s c a n b e f o u n d .T h e s i n g l e - p a t h m o d e l m o d e l s f a u l ts i n t h e d a t a p a t h s o f ag r a p h d e s c r i p t i o n w i t h t h e a s s u m p t i o n t h a t a t m o s t o n e d a t ap a t h c a n b e f a u l t y at a n y o n e t i m e ( n o t e t h a t a p a t h i s al o g i ca l e n t i t y a n d m a y n o t c o r r e s p o n d t o a n y p h y s i c a l w i r e s ) .I t c o v e r s a n y f a u l t t h a t a f f e c t s a p a t h ' s i n t e g r i t y . A p a t h c a nb e t e s t e d b y a p p l y i n g t e s t d a ta t o i t s i n p u t a n d e x a m i n i n g t h er e s u l t a t i t s o u t p u t . D u r i n g t e s t g e n e r a t i o n , s y m b o l sr e p r e s e n t i n g t e s t d a t a a n d t e s t r e s u l t a r e p r o p a g a t e d b a c k w a r da n d f o r w a r d r e s p e c t i v e l y . T h e m o d e l a l s o s p e c i fi e s f a u l t

It I1 1 . 9 > Ip -I I i

]-> I P C + I pageJ i II i i i8 > I

bit bit addr

/ 0 1i ia n d t a dL _ _ _ L _ _ _

ii

2 3 4 5 6 7 ~i i i i i idea itm imp lotL _ _ _ I . . . . I . . . I . . . . I _ _ _ J _ _ .

Im t ruc dm a Exe c ut ion

" - 7 7 . - , ; 7 . .I

I ~ 1F i g u r e 3 . S T G g r a p h o f P D P - 8 I n s t r u c ti o n D e c o d i n g

e f f e c t i v ea n d

I ~ c ' ~ I I M t l " ~ I' - . , /i ~ D II ^ e l < " I

I

e f f e c t i v e P C + I

I + 1 I I M t ] < - I

i

I

F i g u r e 4 . P D P - 8 I n s t r u c t i o n s : A N D , J M Sd o m i n a n c e a n d f a u l t e q u i va l e n c e r e l at i o n s h i ps a m o n g i n p u t sa n d o u t p u t s o f e a c h n o d e t y p e . S i m i l a r t o t h e s i n g l e - p a t hm o d e l , t h e s i n g l e -n o d e m o d e l m o d e l s f a u l t s i n t h e n o d es o f ag r a p h . T h i s m o d e l s p e c i fi e s t h a t t e s t p a r a m e t e r s a r e t o b ea p p l i e d t o a l l i n p u t s o f a n o d e w i t h t h e n o d e ' s o u t p u t s t h e nc a r r y i n g t h e t e s t r e s u l t s t o b e i n s p e c t e d .T h e d o u b l e - p a t h m o d e l a s s u m e s t h a t u p t o t w o p a t h s m a y b ef a u l t y a t th e s a m e t i m e . F o r e a c h p a i r o f p a t h s s e l e c t e d f o rt e s t g e n e r a t i o n , a t e st p a r a m e t e r i s a p p l i e d t o t h e b e g i n n i n go f e a c h p a t h w i th t h e c o r r e s p o n d i n g t e s t r e s u l t c o m i n g o u t a tt h e e n d o f t h e p a th . B o t h p a r a m e t e r s a re b a c k w a r dp r o p a g a t e d w h i l e b o t h r e s u l t s a r e f o r w a r d p r o p a g a t e d . T h e

P ap er 15.3209


4/7

d o u b l e - p a t h m o d e l i s u s e f u l f o r d e t e c t i n g i n t e r f e r e n c eb e t w e e n t w o d a t ~ p a t h s . T h e n u m b e r o f t e s t s r e q u i r e d i sp r o p o r t i o n a l t o n , w h e r e n i s t h e n u m b e r o f p a t h s in t h ed e s c r i p t i o n . O n e c a n g o e v e n f u r t h e r a n d a l l o w a n y n u m b e ro f p a th s t o b e fa u l t y a t th e s a m e t i m e . T h e n u m b e r o f t es t st o b e g e n e r a t e d t h e n b e c o m e s o ( 2 n ) . A g r a p h l e v el f a u l tm o d e l c a n s p e c if y a n y c o m b i n a t i o n o f p a t h s a n d n o d e s - t h ep o s s i b i l i ti e s a r e e n d l e s s .T h e m o r e p a r a m e t e r s a n d r e s u l t s to p r o p a g a t e , t h e h i g h e r t h ep r o b a b i l i t y t h a t a t e s t e i t h e r d o e s n o t e x i s t o r c a n n o t b eg e n e r a t e d b y t h e h e u r i s t i c s . P r a c ti c a l e x p e r i e n c e i n o t h e rf i e ld s o f t e s t i n g h a s s h o w n t h a t m u l t i p l e f a u l t s w h i c h c a n n o tb e d e t e c t e d b y t e s t s g e n e r a t e d f o r s i n g l e f a u l t s o c c u ri n f r e q u e n tl y . W i t h c o s t s m u c h h i g h e r th a n t h e s i n g l e - f au l tm o d e l s , t h e p o t e n t i a l b e n ef i t o f m u l t i p l e - f a u l t m o d e l s i sp r o b a b l y m a r g i n a l.

2 . 3 P r i m i t i v e L e v e l F a u l t M o d e l sPrimi t ive leve l fau l t mode ls m o d e l f a u l t s a t t h e l e v e l o fi n d i v i d u a l g r a p h p r i m i t i v e s . A t t h i s l e v e l , a s s u m p t i o n s a b o u tt h e i m p l e m e n t a t i o n a r e i n e v i t a b l e u n l e s s e x h a u s t i v e o rr a n d o m t e s ti n g i s t o b e u s e d . F o r e a c h l i ke l y i m p l e m e n t a t i o no f t h e s a m e f u n c t i o n a l p r i m i t i v e , a d i f f er e n t m o d e l m a y b ed e v e l o p e d a n d e n t e r e d i n t o t h e t es t p a t te r n d a t a b as e . A u s e rc a n c h o o s e a s u i t a b l e f a u l t m o d e l o r s i m p l y u s e a l l a v a i l a b l em o d e l s t o c o v e r a s w i d e a r a n g e o f i m p l e m e n t a t i o n s a sp o s s i b l e . T h i s i s a t r a d e o f f b e t w e e n c o s t a n d g e n e r a l i t y .I f a m o d e l a s s u m e s a s p e c i fi c c i r c u i t r e a l i z a t i o n , t r a d i t i o n a lc i r c u i t t e s t i n g m e t h o d s c a n b e c a l l ed u p o n t o g e n e r a t e t e s tp a t t e r n s b a s e d o n t h a t r e a l i z a t i o n . I f t h e g r a p h p r i m i t i v e i s i nt u r n d e f i n e d i n t e r m s o f l o w e r - l e v e l p r i m i t i v e s , t h ef u n c t i o n a l t e s t g e n e r a t i o n s y s t e m c a n b e ca l l e d u p o nr e c u r s i ve l y t o g e n e r a t e ' t e s t s f o r t h e h i g h e r - l e v el p ri m i t i v e.T h i s i s m a d e p o s s i b l e b y t h e e x t e n s i b l e g r a p h l a n g u a g e . I ne a c h c a s e , p a t t e r n s g e n e r a t e d a r e e n t e r e d i n t o t h e t e s t p a t t e r nd a t a b a s e u n d e r t h e n a m e o f t h e p r i m i t i ve al o n g w i th t h ei m p l e m e n t a t i o n s a s s u m e d a n d t h e n u m b e r o f t e st v e c t o r sr e q u i r e d .E x a m p l e : T h e s i n g l e - p a t h m o d e l m a k e s n o a s s u m p t i o n a b o u tw h a t t h e a c t u a l fa u l t in t h e d a t a p a t h m a y b e . T h e t a s k o fs e l e c t i n g t e s t d a t a f r o m t h e 2 w p o s s i b i l i ti e s ( w h e r e w i s t h ew i d t h o f t h e d a t a p a t h ) i s l e f t to t h e p r i m i t i v e l e v e l f a u l tm o d e l . T o t e s t f o r si n g l e - b i t s t u c k - a t f a u l t s i n t h e d a t a p a th ,f o r e x a m p l e , i t i s o n l y n e c e s s a r y t o u s e a p a i r o f t e s t v e c t o r st h a t t u r n e a c h b i t o n a n d o f f a t l e a s t o n c e e . g0 0 . . 0 0 & 1 1 .. 1 1 , 0 1 .. 0 1 & 1 0 . . 1 0 . T o t e s t f o r s h o r t s b e t w e e na d j a c e n t b i t s , h o w e v e r , w o u l d r e q u i r e m o r e t h a n t w o t e s tv e c t o r s

2 . 4 F u n c t i o n a l A n a l y z e rT h e f u n c t i o n a l a n a l y z e r f i r s t c o n s u l t s t h e g r a p h l e v e l f a u l tm o d e l c h o s e n t o s e l e c t a s e t o f p r im i t i v e s . F o r e a c h c a s e ,t e s t g e n e r a ti o n i s p e r f o r m e d i n f o u r s e q u e n t ia l s t ag e s ( F i g u r e5 ) : p a r a m e t e r i n t r o d u c t i o n , b a c k w a r d p r o p ag a t i o n , fo r w a r dp r o p a g a t i o n , a n d ju s t i f i c a t i o n . E a c h s t a g e h a s i ts o w no b j e c t iv e a n d c o n d i t i o n s a r e i m p o s e d o n t h e g r a p h a sn e c e s s a r y t o m e e t t h a t o b j e c t i v e . B y s u c c e s s i v e l yc o n s t r a i n i n g t h e g r a p h , a p a r a m e t e r i z e d t e s t w h i c h d e t e c t st h e f a u l t i s g e n e r a t e d a t t h e e n d .

E a c h s t a g e i s f u r t h e r s u b d i v i d e d i n t o t e s t g e n e r a t i o n r o u t i n e st h a t w o r k o n o n e p r i m i t i v e a t a t i m e . T h e s e r o u t i n e s a r ec a l l ed r e c u r s i v e l y u n t i l s u c c e s s . I f a s t e p f a i ls , t h e s t a t e o ft h e g r a p h i s r o l l e d b a c k t o t h e p o i n t w h e r e a cho ice w a s l a s tm a d e a n d t h e n e x t c h o i c e i s a t t e m p t e d . L o c a l i ze d h e u r i s t i c sw h i ch o n l y c o n s i d e r a p r i m i t i v e ' s i m m e d i a t e n e i g h b o r s i nm a k i n g a d e c i s i o n a r e u s e d w h e n e v e r p o s s i b le , so t h en u m b e r o f p r i m i t i v e s c o n s i d e r e d a t e a c h s t e p is no t a f f e c t e db y t h e s iz e o f th e g r a p h , a l t h o u g h t h e n u m b e r o f s t e ps

, , , l . , r a p n - L e v e l, G r a p h D e s c r i p t i o n , ,, , , F a u l t M o d e l s ,. _ I I _ . . . . . . . . . . . . . - J

i In ] P a r a m e t e r I n t r o d u c t i o n I' t' _ ! B a c k w a r d P r o p a g a t i o n I' , 1 I:

G r a p h i I F o r w a r d P r o p a g a t i o n IP r i t iv e : ID a t a b a s e

,

" - ! J u s t i f i c a t i o n [!

.. J P a r a m e t e r i z e d T e s t s

F i g u r e 5 . O p e r a t i o n o f t h e F u n c t i o n a l A n a l y z e rn e e d e d i n e a c h s t a g e m a y g r o w p r o p o r t i o n a l l y w i t h t h en u m b e r o f p a t h s i n a g r a p h . A t p r e s e n t , o n l y l o o p - f r e e g r a p hd e s c r i p t i o n s c a n b e h a n d l e d b y t h e f u n c t i o n a l a n a l y z e r .A c o n s t r a i n t i s a p r e d i c a t e f u n c t i o n a s s o c i a t e d w i t h a p a th .A p a t h c a n o n l y c a r r y t o k e n s t h a t s a t i s f y t h e c o n s t r a i n t . Ap a t h c a n h a v e a n y n u m b e r o f c o n s t r ai n t s a s lo n g a s th e y a r em u t u a l l y c o n s i s t e n t . N o n - o v e r l a p p i n g b i t f i e l d s w i t h i n a p a t he a c h c a n h a v e i t s o w n s e t o f c o n s t r a i n t s . C e l l s i n t h em a c h i n e s t a te a n d s y m b o l s c r e a t e d d u r i n g t e s t g e n e r a t i o nm a y a l s o h a v e c o n s t r a in t s . S y m b o l s a r e u s e d t o r e p r e s e n t t e s td a t a , t e s t r e s u l t s , b i t f i e ld s , a n d a n u m b e r o f o t h e r t h i n g s .C o n s t r a i n t s i n v o l v i n g s y m b o l s a n d t h e i r m a n i p u l a t i o n p l a y ac e n t r a l r o l e in t h e t e s t g e n e r a t i o n p r o c e s s . D u r i n g t e s tg e n e r a t i o n , c o n s t r a i n t s a r e a d d e d t o p a t h s , c e l ls , a n d s y m b o l st o c r e a t e t h e c o n d i t i o n s n e e d e d t o g e n e r a t e a s u c c e s s f u l t e s t .C o n s t r a i n t r e s o l u t i o n i s th e p r o c e s s o f d e t e r m i n i n g w h e t h e ra s e t o f c o n s t r a i n t s i s c o n s i s t e n t a n d t h e n p o s s i b l y s i m p l i f y i n gt h e s e t . C o n s t r a i n t s a r e u s u a l l y a d d e d o n e a t a t i m e t o a s e to f c o n s t r a i n ts t h a t h a s a l r e a d y b e e n d e t e r m i n e d t o b ec o n s i s te n t . C o n s t r a i n t r e s o l u t i o n ca n b e a n e x t r e m e l yi n v o l v e d p r o b l e m i f a r b i t r a r y p r e d i c a t e f u n c t i o n s a r e t o b ec o n s i d e r e d . F o r t u n a t e l y , t h e d e s c r i p ti o n s o f m o s t c o m p u t e r sr e q u i r e o n l y s i m p l e p r e d i c at e f u n c t i o n s su c h a s = , ~ , < , ~ , > / . A s a r e s u l t , m o s t c o n s t r a i n t s t h a t a r i s e i n p ra c t i c e a r ev e r y s i m p l e a n d c a n b e r e s o l v e d u s i n g r e l at i v e ly si m p l ea l g o r i t h m s .

Paper 15.3210


5/7

P a r a m e t e r I n t r o d u c t i o n - t h i s p h a s e i n t r o d u c e s s y m b o l sr e p r e s e n t i n g t e s t d a t a p a r a m e t e r s a n d t e s t r e s u l t s i n t o t h e g r a p ha n d s e t s t h e s t a g e f o r s u b s e q u e n t p h a s e s . T h e g r a p h l e v e l f a u l tm o d e l s e l e c t e d i s c o n s u l t e d t o i d e n t i f y t h e n e x t s e t o f p r i m i t i v e sf o r w h i c h a te s t i s t o b e g e n e r a t e d . S y m b o l s r e p r e s e n t i n g t h er e q u i r e d p a r a m e t e r s a n d r e s u l ts a r e t h en a d d e d t o t h e g r a p h .I m p l i c a t i o n - e a c h t i m e a p a t h r e c e i v e s a n e w c o n s t r a i n t , i t m a yh a v e i m p l i c a t i o n s o n o t h e r p a t h s c o n n e c t e d t o t h e s a m e n o d e s .A t e a c h t e s t g e n e r a t i o n s t e p , p a t h s t h a t h a v e r e c e i v e d n e wc o n s t r a i n t s a r e n o t e d a n d t h e i m p l i c a t i o n r o u t i n e i s c a l l e d a t t h ee n d o f t h e s t e p t o c h e c k i f a n y o f t h e c h a n g e s r e s u l t s i n ac o n t r a d i c t i o n , i n d i c a t i n g f a i l u r e o f t h a t s t e p .2.4.1 Backward PropagationF o r t e s t d a t a t o b e a p p l i e d t o t h e r e q u i r e d p l a c e s , t h e m a c h i n es t a t e p r i o r t o t h e t e s t c y c l e m u s t c o n t a i n t h e t e s t d a t a o r s o m et r a n s f o r m a t i o n o f t he m . T h e o b j e c t iv e o f th e b a c k w a r dp r o p a g a t i o n p h a s e i s t o s e t u p t h e c o n d i t i o n s necessary f o r t h ea p p l i c a t i o n o f t e s t d a t a p a r a m e t e r s t o r e q u i r e d p la c e s . T h i s i sd o n e b y backward propagating e a c h p a r a m e t e r t o a r e a d n o d e i nt h e g r a p h . T h e n b y i n i ti a l i z i n g t h e m a c h i n e s t a t e a c c o r d i n g l yp r i o r t o t h e t e s t c y c l e , i t i s g u a r a n t e e d t h a t t h e p a r a m e t e r sw o u l d b e a p p l i e d t o t h e c o r r e c t s p o t s .E a c h s t e p w i t h i n t h e b a c k w a r d p r o p a g a t i o n p h a s e c o n s i d e r so n l y o n e p a t h a t a t i m e . I t t r i e s t o i m p o s e t h e n e c e s s a r yc o n d i t i o n s o n t h e i n p u t s o f t h e p a t h ' s s o u r c e n o d e s o t h a t t h ec u r r e n t v a l u e o f t h e p a t h i s a s s u r e d . S i n c e t h e v a l u e t o b eb a c k w a r d p r o p a g a t e d i s a lm o s t a l w a y s a s y m b o l i c p a r a m e t e r o ra s y m b o l i c e x p r e s s i o n , a o n e t o o n e m a p p i n g o f t h a t v a l u e m u s ta p p e a r s o m e w h e r e o n t h e i n p u t s o f t he s o u r c e c o d e . T h i sc o n c e p t i s f o r m a l i z e d i n t h e f o l l o w i n g d e f i n i t i o n .

D e f i n i t i o n : A c o n s t a n t v e c t o r(Co,C1 . . . . c i . l , c i + 1 . . . C n .1 )i s a b a c k w a r d p r o p a g a t i o n v e c t o r o f t h e n - i n p u t f u n c t i o n Fi f fF ( c 0 . . . . . c i .1 , g(x ) , c i+ 1 . . . . Cn.1) = x

-1f o r a ll x in t h e o u t p u t d o m a i n o f F a n d i f f g e x i s ts . T h ef u n c t i o n g i s ca l l e d t h e t r a n s f o r m a t i o n f u n c t i o n o f t h ev e c t o r .B y i m p o s i n g a b a c k w a r d p r o p a g a t i o n v e c t o r o n t h e i n p u t s o f t h es o u r c e n o d e , t h e o b j e c t i v e o f t h e c u r r e n t b a c k w a r d p r o p a g a t i o ns t e p i s s a t i s f i e d . T h e b a c k w a r d p r o p a g a t i o n r [] ~ t in e is t h e nr e c u r s i v e l y c a l l e d f o r t h e p a t h c o n n e c t e d to t h e i i n p u t o f t h en o d e F . T h e p r o c e s s i s s u c c e s s f u l i f a r e a d n o d e i s e v e n t u a l l yr e a c h e d .

2.4.2 Forward PropagationT e s t r e s u l t s m u s t b e s a v e d i n t h e m a c h i n e s t a t e a t t h e e n d o ft h e t e s t c y c l e s o t h a t t h e y c a n b e c h e c k e d f o r c o r r e c t n e s s i ns u b s e q u e n t c y c l e s . T h e y m u s t t h e r e f o r e e i t h e r a p p e a r i n t h ec o n t e n t s o r t h e a d d r e ss e s o f th e n e w m a c h i n e s t a te . T h eo b j e c t iv e o f t h e f o r w a r d p r o p a g a t i o n p h a s e i s t o b r i n g t h e s er e s u l t s t o " w r i t e " n o d e s i n t h e g r a p h t o e n s u r e t h e i ro b s e r v a b i l i t y a f t e r t h e t e s t c y c l e .F o r w a r d p r o p a g a t i o n i s s i m i l ar t o b a c k w a r d p r o p a g a t i o n i no p e r a t i o n e x c e p t f o r i t s f o r w a r d d i r e c t i o n . T h e c o n c e p t o fp r o p a g a t i o n v e c t o r s c a n s i m i l a r l y b e d e f i n e d .

D e f i n i t i o n : A c o n s t a n t v e c t o r( c 0 , c I . . . . . C i . l ,C i+ 1 . . . . Cn.1)i s a f o r w a r d p r o p a p t l o n v e c t o r f o r t h e i h i n p u t o f t h e n -i n p u t f u n c t i o n F i f fF ( c 0 , c 1 . . . . C i . l , x , c i+ 1 . . . . Cn.1) = g(x)f o r a l l x i n t h e i t h i n p u t d o m a i n o f F a n d i f f g ' l e x i s t s . T h ef u n c t i o n g i s c a l l e d t h e t r a n s f o r m a t i o n f u n c t i o n o f t h ev e c t o r .

P r o p a g a t i o n i s s u c c e s s f u l i f t h e v a l u e i s p a s s e d a l o n g t o t h eo u t p u t w i t h n o l o ss o f i n f o r m a t i o n . F o r a m u l t i p le - o u t p u t n o d er e p r e s e n t e d b y t h e s e t o f f u n c t io n s { F 0 . , F . I , , . . . , F -1} w h e re mi s t h e n u m b e r o f o u r 0 u t s , t h e a b o v e d e f t m t io . n t hc a n ~ e e x t e n d e d -a v e c t o r i s a p r o p a g a t i o n t l ~ e c t o r f o r t h e i i n p u t i f i t c a np r o p a g a t e t h e v a lu e at th e i i n p u t t o any one o f t h e m o u t p u t s .

g e t n a m e s o f a l ln o d e s c o n n e c t e d t o p a t hn o )

a n y n o d e l e f t? ~ . .

s e l ec t n i ~ n o d e ~

o b t a i n p r o p a g a t i o n v e c t o rs ]f o r n o d e f r o m d a t a b a s e /n o /

a n y v e c t o r l e f t ? . ~ , , , ,y e s

n o d .e l e c t n e x t v e c t o rp r o p a g a t e s y m b o l t h r ui m p l i c a t i o n o k ? n o /y e sP r o p a g a t i o n S u c c e s s f u lP r o p a g a t i o nF a i l e d

F i g u r e 6. F l o w C h a r t f o r F o r w a r d P r o p a g a t i o nP r o p a g a t i o n v e c t o r s f o r a f u n c t i o n c a n u s u a l l y b e d e t e r m i n e ds i m p l y b y i n s p e c t i o n . T h e m o s t o b v i o u s a n d s im p l e o n e s c a n b ee n t e r e d i n t o t h e g r a p h p r i m i t i v e d a t a b a s e w h ic h c o n t a i n s a l ld e f i n i t i o n s a n d h e u r i s t i c s a s s o c i a t e d w i t h e a c h t y p e o f g r a p hp r i m i t i v e . F i g u r e 6 s h o w s a f l o w c h a r t o f t h e f o r w a r dp r o p a g a t i o n p h a se . T h e b a c k w a r d p r o p a g a t i o n p h a s e w o r k s i ne s s e n ti a l ly t h e s a m e m a n n e r .J u s t i f i c a t io n - D u r i n g t h e c o u r s e o f b a c k w a r d a n d f o r w a r dp r o p a g a t i o n , w h e n necessary c o n d i t i o n s f o r t h e t e s t g e n e r a t i o nt o b e s u c c e s s f u l a r e a s s i g n e d , m a n y p a t h s a r e o f t e n l e f tu n s p e c i f i e d . J u s t i f i c a t i o n f i l ls in t h e u n s p e c i f i e d v a l u e s s o t h a t ap a r a m e t e r i z e d t e s t c a n a c t u a ll y b e g e n e r a t e d .

Paper 15.3211


6/7

T h e t e s t c a s e s y n t h e s i z e r b r i n g s t o g e t h e r p a r a m e t e r i z e d t e s t sp r o d u c e d b y t h e f u n c t i o n a l a n a l y z e r a n d t e s t p a tt e r n s t h a t h a v eb e e n d e v e l o p e d t h r o u g h p r i m i t iv e - l e v el f a u l t m o d e l s . I ts u b s t i t u t e s t h e f o r m a l p a r a m e t e r s b y a c t u a l t e s t d a t a a n de x p e c t e d r e s u l t s o b t a i n e d f r o m t h e t e s t p a t t e r n d a t a b a s e . T h eu s e r c a n s p e c i f y t h e c h a r a c t e r i s t i c s a n d c o s t c o n s t r a i n t s h e w a n t st o i m p o s e . O n l y p a t t e r n s m e e t i n g t h e r e q u i r e m e n t s w i l l b ec h o s e n f r o m t h e d a t a b a s e .T e s t P r o g r a m S y n t h e s i z e r - I f t h e s y s t e m b e i n g t e s t e d i s ac o m p u t e r , i t s i n st r u c ti o n s c a n u s u a l l y b e g r o u p e d b y t h e i ro p e r a n d f e t c h i n g m o d e s a n d m e m b e r s o f t h e s a m e s e t c a nu s u a l l y b e t e s te d w i t h s i m i l a r s e q u e n c e o f in s t r u c t i o n s . T e s tp r o g r a m t e m p l a t e s c a n b e d e v e l o p e d f o r e a c h g r o u p . F o r e a c ha c t u a l t e s t c a s e g i v e n , t h e s y n t h e s i z e r s e l e c t s t h e a p p l i c a b l et e m p l a t e s a n d t r i e s t o f i l l i n t h e n e c e s s a r y " b l a n k s " w h e n e v e rp o s s i b l e , p r o d u c i n g r e a d y - t o - r u n p r o g r a m s e g m e n t s a n da u t o m a t i n g t h e t e s t p r o g r a m m i n g p r o c e s s.

Z S D e si gu C o m i d e r a t l o mS e p a r a t i o n o f I s s u e s - a m a j o r g o a l o f t h e r e s e a r c h i s t ou n t a n g l e t h e i s s u e s i n v o l v e d i n f u n c t i o n a l t e s t i n g , d i v i d i n g t h e mi n t o s m a l l e r p r o b l e m s t h a t c a n b e a tt a c k e d o n e a t a ti m e . T h i se n a b l e s b e t t e r f o c u s i n g o n t h e i n d i v i d u a l is s u e s , m a k e s t h ef u n c t i o n a l t e s t i n g p r o b l e m a s a w h o l e m o r e m a n a g e a b l e , a n dp e r m i t s t h e d e v e l o p m e n t o f m o r e e f f i c ie n t s o l u t i o n s . T h i s isd o n e t h r o u g h o u t t h e m e t h o d o l o g y w h e n e v e r p o s s ib l e . T h es o l u t io n s a d o p t e d i n c l u d e:

1 . M u l t i - l e v e l f a u l t m o d e l s - f a u l t m o d e l s a r e d i v i d e d i n t og r a p h a n d p r i m i t i v e le v e ls .2 . M u l t i - s t a g e t e s t g e n e r a t i o n - t e s t s a r e g e n e r a t e d i n t w os t a g es . P a r a m e t e r i z e d t e s ts c a n b e r e p e a t e d l y u s e d b y t h et e s t c a s e s y n t h e s i z e r t o g e n e r a t e t e s t s f o r d i f f e r e n t

r e q u i r e m e n t s .M o d u l a r i t y a n d F l e x i b i l i t y - t h e m e t h o d o l o g y i s d e s i g n e d tof a c i li t a t e e v o l u t i o n a r y c h a n g e s a n d i n c r e m e n t a l i m p r o v e m e n t i ne v e r y o n e o f i t s b u i l d i n g b l o c k s . T h e f o l l o w i n g s o l u t i o n s w e r ea d o p t e d :* T e s t g e n e r a t i o n i s p e r f o r m e d i n s t a g e s e v e n w i t h i n t h e

f u n c t i o n a l a n a l y z e r , a h e u r i s t i c c a n b e " u n p l u g g e d " a n dr e p l a c e d b y a n e w o n e w i t h e a s e . W h e n e v e r p o s s i b l e , i n f o r m a t i o n i s g r o u p e d i n t o d a t a b a s e st h a t c a n b e e a s il y m o d i f i e d b y t h e u s e r . T h e d a t a b a s e si n c l u d e a l l t h e f a u l t m o d e l s , t h e t e s t p a t t e r n d a t a b a s e , t e s tp r o g r a m t e m p l a t e s , a n d d e f i n it i o n s o f t h e p r im i t i v e s u s e d i ng r a p h d e s c r i p t i o n s .

U s e r C o n l r o l - t h e u s e r i s g i v e n m a x i m u m c o n t r o l o v e r t h ew h o l e t e s t g e n e r a t i o n p r o c e s s b e c a u s e e a c h a p p l i c a t i o n h as i t so w n f a u l t d i s t r i b u t i o n a n d c o s t c o n s i d e r a t i o n s a n d i t i s b e s t t ol e t t h e u s e r s p e c i f y t h e p e r t i n e n t i n f o r m a t i o n r a t h e r t h a n f o r c i n gu n r e a l i s t i c a s s u m p t i o n s u p o n t h e u s e r . A u s e r c a n e x e r t c o n t r o lo v e r t h e t e s t g e n e r a t i o n p r o c e s s b y :

1 . S e l e c t i n g t h e l e v e l o f d e t a i l a t w h i c h t h e d i g i t a l s y s t e m o rp a r t s o f t h e s y s t e m i s s p e c i f i e d .2 . D e f i n i n g n e w g r a p h p r i m i t i v e s .3 . S e l e c t i n g o r d e f i n i n g f a u l t m o d e l s .4 . I m p o s i n g c o s t a n d o t h e r r e s t r i c t i o n s d u r i n g t e s t

g e n e r a t i o n .

3 . Expe r im e ntA f a u l t s i m u l a t i o n e x p e r i m e n t u t i l i z in g I S P S 4 ], a r e g i s t e r -t r a n s f e r l e v e l h a r d w a r e d e s c r i pt i o n la n g u a g e , w a s c o n d u c t e d t oe v a l u a t e th e e f f e c ti v e n e ss o f o u r m e t h o d o l o g y . T e s t s w e r eg e n e r a t e d f o r t h e D E C P D P - 8 m i n i c o m p u t e r a n d c o m p a r e da g a i n s t t e s t p r o g r a m s s u p p l i ed b y t h e c o m p u t e r ' s m a n u f a c t u r e r .A n I S P S d e s c r i p t i o n o f t h e P D P - 8 w a s s i m u l a t e d w i t h s i n g l es t u c k - a t f a u l t s i n a l l t h e d a t a p a t h s r e p r e s e n t e d i n t h ed e s c r i p t i o n , f o r a t o t a l o f 1 ,4 3 8 f a u l t s . D e s i g n f a u l t s w e r e n o tc h o s e n b e c a u s e o f t h e i r m u c h l a r g e r f a u l t s p a c e a n d i ti m p o s s i b l e t o s e l e c t a s u b s e t w i t h o u t u s i n g c o n s i d e r a b l es u b j e c ti v e j u d g m e n t .O n t h e o t h e r h a n d , s i n g l e s t u c k - a t f a u l ts h a v e l o n g b e e na c c e p t e d a s t h e s t a r t i n g p o i n t o f m a n y t e s ti n g s t r a t eg i e s a t b o t h

ISPS Description

I F a n l tGenerati~outines [ IS P S C o m p i l e rI I

d e p e n d e n t f a u l t s D a t a b a s e

I S I S S i m u l a t o r [IExperimental

R e s u l t s

F i g u r e 7 . D e s ig n o f E x p e r i m e n t sh i g h a n d l o w l e v e l s . I t i s t h e c l o s e s t t h i n g t o a q u a m i t a t i v ey a r d s t i c k b y w h i c h t e s ts g e n e r a t e d b y v a r i o u s m e t h o d o l o g i e s a r em e a s u r e d . I t is g e n e r a l l y a c c e p t e d t h a t a t e s t t h a t d e t e c t s ah i g h e r p e r c e n t a g e o f s i n g l e s t u c k - a t f a u l t s t h a n a n o t h e r t e s t i sp r o b a b l y t h e b e t t e r t e s t , e v e n t h o u g h m a n y o f t h e f a u l t s t h a t

a c t u a l l y o c c u r in p r a c t i c e a r e n o t s i n g l e s t u c k - a t f a u l t s . S i n g l es t u c k - a t f a u l t s w e r e t h e r e f o r e s e l e c t e d f o r o u r e x p e r i m e n t a s t h em o s t o b j e c ti v e a n d p r a c t i c a l m e a s u r e a v a i l ab l e . T h e d e s i g n o ft h e e x p e r i m e n t i s s h o w n i n F i g u r e 7.A n S T G g r a p h o f t h e P D P -8 w a s d e v e l o p e d b a s e d o n t h ec o m p u t e r ' s h a n d b o o k a n d h a s 1 58 n o d e s a n d 2 35 p a t h s . T h eg r a p h w a s r u n t h r o u g h o u r s y s t e m to p r o d u c e p a r a m e t e r i z e dt e s t s u s i n g t h e s in g l e - p a t h f a u l t m o d e l . T h e h e u r i s t i c ss u c c e e d e d i n g e n e r a t i n g t e s t s f o r 9 7 % o f t h e p a t h s s e l e c t e d b yt h e m o d e l . T h e t e s t p a t t e r n s u s e d i n t e s t c a s e s y n t h e s i s w e r es i m p l e : e a c h b i t o f e v e r y p a r a m e t e r w i t h i n a p a r a m e t e r i z e d t e s tn e e d t o b e t u r n e d o n a n d o f f a t l e as t o n c e. T h e t e s t c a s e s w e r eg i v e n t o a t e s t p r o g r a m m e r w h o s e m i - m e c h a n i c a ll y t r a n s l a t e dt h e m i n t o P D P - 8 a s s e m b l y c o d e ( a t e st p r o g r a m s y n t h e s iz e r w a sn o t w r i t t e n f o r t h e P D P - 8 ) . T h e t e s t p r o g r a m m i n g e f f o r t w a ss t r a i g h t f o r w a r d a n d n o i n - d e p t h u n d e r s t a n d i n g o f t h e s y s t e m

Paper 15.32 12


7/7

b e i n g t e s t e d w a s r e q u i r e d a t a l l . T h e s a m e t r a n s l a t i o n c o u l dh a v e b e e n p e r f o r m e d a u t o m a t i c a l ly b y s i m p l e te s t p r o g r a ms y n t h e s i s t e c h n i q u e s .T h e t e s t p r o g r a m s s u p p l i e d b y D E C e x e r d s e t h e m a c h i n e a t af u n c t i o n a l l e v e l a n d h a l t s w h e n e v e r a n e r r o r i s d i s c o v e r e d ,p i n p o i n t i n g t h e in s t r u c t i o n a t f a u l t . O u r f u n c t i o n a l t e s t sa c t u a l l y p r o v i d e b e t t e r d i a g n o s t i c r e s o l u t i o n s i n c e e a c h t e s t i sd e s i g n e d t o d e t e c t f a u l t s i n a f u n c t i o n a l p r i m i t i v e .R e s u l t s o f t h e ex p e r i m e n t a r e su m m a r i z e d i n T a b l e I . O u rt e s t s o u t p e r f o r m e d t h e m a n u f a c t u r e r ' s p r o g r a m s b y as u b s t a n t i a l m a r g i n - w e d e t e c t e d 9 8 . 5 % o f t h e s t u c k - a t f a u l t sc o m p a r e d t o 9 5 . 5 % . M o s t s u r p r i s i n g l y , t e s t s p r o d u c e d b y o u rm e t h o d o l o g y a c h i e v e t h e h i g h e r f a u l t d e t e c t i o n r a t e w i t h f a rf e w e r i n s t r u c t i o n s .

T e s t P r o g r a m I n s t r u c t i o n s E x e c u t e d F a u l t s D e t e c t e d

M a n u f a c t u r e r ' s > 1 0 , 0 00 9 5 . 5 %

O u r s 7 3 1 9 8 . 5 %

T a b l e 1 . R e s u l t s o f t h e E x p e r i m e n tF i r s t v e r s i o n o f t h e t e s t g e n e r a t i o n s y s t e m h a d o v e r 5 , 0 0 0 l i n e so f L I S P c o d e a n d r e q u i r e d 6 0 0K b y t e s to r u n . O n l y t h e g r a p hd e s c r i p t i o n l a n g u a g e a n d t h e f u n c t i o n a l a n a l y z e r w e r e f u l l yi m p l e m e n t e d i n t h e f i r s t v e r s io n . T e s t g e n e r a t i o n f o r t h e P D P - 8t o o k o n l y 2 1 m i n u t e s o f C P U l l m e r u n n i n g w i t h i n t e r p r e t e dL I S P c o d e o n a D E C - 2 0 6 0 . C o m p i l a t io n o f th e L I S P co d e a l o n ec a n s p e e d t h i n g s u p a b o u t f i v e t i m e s .

R e f e r e n c e s

1.

2 .

3.

4 .

5 .

S . B . A k e r s , " F u n c t i o n a l T e s t i n g w i t h B i n a r y D e c i s i o nD i a g r a m s " , i n P r e c . 8 t h I n t ' l C o n f e r e n c e o n F a u l t - T o l e r a n tC o m p u t i n g ( F T C S - 8 ) , p p 7 5 - 8 2 , I E E E C o m p u t e r S o c i e t y ,June 1978 .M . A . B r e u e r a n d A . D . F r i e d m a n , " F u n c t io n a l L e v e lP r i m i t i v es i n T e s t G e n e r a t i o n " , I E E E T r a n s a c t i o n s o nC o m p u t e r s C - 2 9 ( 3 ) , p p 2 2 3 - 2 3 5 , M a r c h 1 9 8 0 .S . M . T h a t t e a n d J .A . A b r a h a m , " T e s t G e n e r a t i o n f o rM i c r o p r o c e s s o r s ", I E E E T r a n s a c t i o n s o n C o m p u t e r s C -29(6) , June 1980 .M . R . B a r b a c d , G . E . B a r n e s , R . O . C a t t e l l , a n dD . P . S i e w i o r e k , " S y m b o l i c M a n i p u l a t i o n o f C o m p u t e rD e s c r i p t i o n s : T h e I S P S C o m p u t e r D e s c r i p t i o n L a n g u a g e " ,C S D e p t . T R C M U - C S - 7 9 - 13 7 , C a r n e g i e - M e l l o n U . ,A u g u s t 1 9 7 9 .K w o k - W o o n L a i , " F u n c t i o n a l T e s t i n g o f D i g i t a l S y s t e m s " ,P h D t h e s i s , C o m p u t e r S c i e n c e D e p t . T R C M U - C S - 8 1 - 1 4 8 ,C a r n e g i e - M e l l o n U n i v e r s i t y , D e c . 1 9 8 1 .

4. Condml~aW e h a v e t a k e n a p r a c ti c a l a p p r o a c h t o w a r d s t h e p r o b l e m o ff u n c t i o n a l t e s t i n g a t t h e s y s t e m l e v e l . W e l a i d d o w n t h eg r o u n d w o r k f o r a s y s t e m a t i c a s s a u lt o n t h e p r o b l e m a n d a l s op r o v i d e d a f r a m e w o r k f o r d i v i d i n g t h e o v e r a l l p r o b l e m i n t os m a l l e r , m o r e m a n a g e a b l e p r o b l e m s . A n a m b i t io u s fu n c t i o n a lt e s t i n g m e t h o d o l o g y w h i c h g e n e r a t e s t e s t s d i r e c t l y f r o m t h ef u n c t i o n a l s p e c i f i c a t i o n o f a d i g i t a l sy s t e m h a s b e e n d e s i g n e d ,i m p l e m e n t e d , a n d e v a l u a t e d . A u t o m a t i c g e n e r a t io n o f d e s i g nv a l i d a t i o n t e s t s is n o w c l o s e r t o r e a l i ty . T e s t c a s e s t h a t w o u l dh a v e t a k e n t e s t p r o g r a m m e r s m a n - m o n t h s a n d h a r d - e a r n e de x p e r ie n c e t o d e v e l o p c a n n o w b e g e n e r a t e d a u t o m a t i c a l ly in am a t t e r o f m i n u t e s .A l t h o u g h t h e s c o p e o f t h e e v a l u a t i o n e x p e r i m e n t i s l i m i t e d , i t sr e s u l t s a r e v e r y p r o m i s i n g . T h e q u a l i t y a n d t h e ef f i c ie n c y o ft h e t e s t s g e n e r a t e d f u r t h e r u n d e r s c o r e t h e p r o m i s e o f f u n c t i o n a lt e s t i n g . O n t h e o t h e r h a n d , t h e r e a r e s t i ll m a n y t h i n g s t h a t w ec a n n o t b e g i n t o u n d e r s t a n d u n t i l m o r e p r a c t i c a l e x p e r i e n c e i sg a i n e d . L a r g e s c al e e x p e r i m e n t s a r e c u r r e n t l y b e i n g p l a n n e d .I m p r o v e m e n t o f t h e s y s t e m w i l l c o n t i n u e a s w c o n t i n u e t ol e a r n . I

1 . The in te re s te d re a de r w i l l f ind a m ~ e de ta i l e d d i sc us~rm c ~ ourre su l t s in m y the s i s [5 ] .

Paper 15.3213

functional testing of digital systems

Documents