automatic vector generation using constraints and biasing

JOURNAL OF ELECTRONIC TESTING: Theory and Applications 16, 107–120 (2000)c© 2000 Kluwer Academic Publishers. Manufactured in The Netherlands.

Automatic Vector Generation Using Constraints and Biasing

JUN YUAN, KURT SHULTZ AND CARL PIXLEYMotorola Inc., 5918 W. Courtyard Drive, Suite 200, Austin, TX 78730, USA

jun [email protected]

kurt [email protected]

carl [email protected]

HILLEL MILLER1 Shenkar Str. E Herzelia, Industrial Zone Herzelia, Herzelia, 46120 Israel

[email protected]

ADNAN AZIZENS 536, Dept. of Electrical and Computer Engr., University of Texas at Austin, Austin, TX 78712, USA

[email protected]

Received December 15, 1998; Revised July 12, 1999

Editor: M.S. Abadir

Abstract. Constraining and biasing are frequently used techniques to enhance the quality of randomized vectorgeneration. In this paper, we present a novel method that combines constraints and input biasing in automaticbit-vector generation for block-level functional verification of digital designs, which is implemented in a tool calledSimGen. Vector generation in SimGen is confined to a legal input space that is defined by constraints symbolicallyrepresented in Binary Decision Diagrams (BDDs). A constraint involving state variables in the design defines astate-dependent legal input space. Input biasing can also depend on the state of the design. The effect of constraintsand input biasing are combined to form what we called the constrained probabilities of input vectors. An algorithmis developed to efficiently generate input vectors on-the-fly during simulation. The vector generation is a one-passprocess, i.e., no backtracking or retry is needed. Also, we describe methods of minimizing the constraint BDDs inan effort to reduce the simulation-time overhead of SimGen. Furthermore we discuss the application of SimGen toa set of commercial design blocks.

Keywords: vector generation, constraint, biasing, probability, BDD, partitioning

1. Introduction

As the semiconductor technology advances at an aston-ishing pace, verification is becoming a major bottle-neck in the product development cycle. While thereare a wide range of verification methodologies andtools [1], simulation is still the prevalent form of

functional verification of commercial designs [2–6].The significant aspects of simulation are the follow-ing (1) model under simulation, usually Verilog [7] orVHDL, (2) the simulator, which can be one of any num-ber of commercial or internal simulators, (3) a methodfor generating input stimulus, which is the subject ofthe present work, (4) a method for determining if the

108 Yuan et al.

simulation model responds correctly to the input stim-ulus, this can take the form of a predicted state resultingfrom the simulation or whether properties are violatedduring simulation, and (5) a method of measuring sim-ulation coverage, such as toggle coverage, Finite-StateMachine (FSM) coverage, code coverage and so forth.

Methods of generating input stimulus are furthersubdivided into two methods. The first, directed sim-ulation is a highly manual process in which the veri-fication engineer defines scenarios to specifically testwhether the design under verification (DUV) correctlyresponds to certain sequences of inputs prescribed bythe programmer. The primary weaknesses of directedsimulation are that devising verification scenarios isdifficult and time-consuming, its effectiveness dependsupon the ability of the verification engineer to imaginea wide-variety of possible stimuli and responses, anddirected simulation vector generation will only coverexactly the scenarios that the designer has devised.However, directed simulation is clearly desirable forgenerating well-understood event sequences with well-understood results. Some amount of directed simula-tion is almost always required.

Directed random simulation is a method in whichcertain sequences of inputs, such as instructions, areselected in a weighted, pseudo-random fashion, pos-sibly following constraints. Purely random simulationvector generation is not very useful, because most de-signs interact with their environments, therefore inputsfrom the environment are constrained, often by thestate of the design. In addition, values of the designinputs are usually not evenly distributed, i.e., they arebiased. For example, the assertion of “reset” shouldhappen infrequently compared with other input events.Input biasing is used to guide simulation into “uncov-ered areas,” i.e., into scenarios not already simulated.Therefore, input constraining and biasing are usuallyused together with random methods to generate goodquality test vectors.

The method of the present work is one in whichweighted inputs are selected only according to con-straints, with no direction other than weighting.Directed and directed-random simulation require aprogramming language such as C or C++ to spec-ify sequences of events. Purely constraint-based inputgeneration requires no direction other than weight-ing. However, constraint-based simulation does dependupon carefully specified constraints.

Directed simulation is probably the most com-mon type of simulation practiced for low-complexity

designs. It is a very time-consuming and error-proneactivity. For large commercial microprocessor de-sign, directed random vector generation is usuallychosen for full-chip simulation and unit-level simu-lation. We argue that our constraint-based method ismost suitable for module and block-level simulation.These constraints can be used in an assume/guaranteestyle of hierarchical reasoning. Constraints also math-ematically document the interfaces of the design. Infact, constraints play a pivotal role in our verifica-tion environment that integrates formal and informalmethods [8]. Model checking procedures based on con-straints were reported in [9].

In this paper, we present a novel method that com-bines constraints and input biasing in automatic simula-tion vector generation in functional verification, whichis implemented in a tool called SimGen. Vector gener-ation in SimGen is confined to a legal input space that isdefined by constraints. A constraint is any Boolean ex-pression which involves any signals of the design underverification. A constraint may involve state-dependentsignals since valid input usually depends upon thestate of the DUV. The distribution of an input vectorunder the effect of constraints and input biasing is de-termined by itsconstrained probability. During simu-lation, SimGen samples the current state in the design,and generates random inputs according to the con-strained probabilities on-the-fly. In any state of the de-sign, if there exists an input satisfying the constraints,SimGen will generate it directly, without need for back-tracking. When no input exists, SimGen flags an errorand halts.

The constraints and biases in SimGen are rep-resented symbolically in a graph based data struc-ture, called a Reduced Ordered Binary DecisionDiagram [10] (henceforth, BDD). Although BDD wasshown to be an efficient data structure in hardwaredesign verification [11], iterative application of BDDoperations, such as fix-point computation in modelchecking [12], can still cause BDD size explosion inthe verification of large designs. Fortunately, SimGenconstructs BDDs from constraints only once. SimGenalso employs a constraint partitioning method to mini-mize the size of BDD conjunctions. Our experiencewith SimGen showed that input constraints for designswith over 1300 latches can be represented with reason-ably small BDDs, including constraints that involve upto 670 variables.

The rest of this paper is organized as follows.Section 2 is a review of related work. In Section 3

Automatic Vector Generation 109

we describe how environment constraints and inputbiasing are represented and provide a mechanism forsimultaneously considering environment constraintsand input probabilities. Section 4 contains details of thealgorithm and vector generation procedures in SimGen.We describe methods used to reduce the BDD sizein Section 5. Experiments and a case study are dis-cussed in Section 6. We summarize and highlight fu-ture research topics related to SimGen in Section 7.

2. Previous Work

To our best knowledge, using BDDs to combine con-straints and biases in bit-vector generation is original.Another feature of our work is that the integration ofSimGen with the simulation verification flow is seam-less in the sense that both constraints and biases areexpressed in Verilog, that is, no proprietary descriptionlanguage is required.

Our work was inspired by the effective use of con-straints and biases in instruction level test generation inmicroprocessor verification. In [13], Freeman et al. re-ported a static-biased random test generation tool calledRIS that allows the user to bias the vector generationwithin a restricted set of choices—all of which sat-isfy the constraints. The biasing is static in that it isindependent of the state of design. Aharon et al. im-plemented a dynamically-biased test generator for atool called RTPG [2, 14], which decides the next inputbased on the current state of the design. A drawback forthe vector generator is the effort required to produce thefunctional model. AVPGEN, a tool introduced in [15,16], used various constraint solving techniques tailoredfor specific instructions. A language, called SIGL, wasused to defined symbolic test templates. A problem isthat one may need to perform “local retries” in case theconstraint-based test generation hits a “dead end”.

BDDs have found many applications in designand verification problems. The Algebraic DecisionDiagram (ADD) [17], which is an extension to theBDD, was used in [18] to represent the matrix of a statetransition graph and compute the steady-state prob-abilities for Markov chains. Although vector genera-tion was not the concern of that paper, the experimentsshowed that symbolic methods can handle very largesystems that direct equation solving methods cannot.

In [19], binary graphs were used to representBoolean functions, so that the probability distributionof the output of a function can be computed recursivelyfrom the probabilities of the inputs. This technique

was used to probabilistically decide the equivalence ofBoolean functions. A similar approach was adopted in[20] to compute exact fault detection probabilities forsome given signals in a design.

Both [19, 20] are related to our work from the point ofview that probabilities are computed recursively using adecision diagram. But this is where the similarity ends.Specifically, we do not deal with the probability of theoutput of a function or a design signal. The problemwe are facing is to generate input vectors which satisfya set of constraints, and are probabilistically influencedby the input distribution, or biasing. The constraintsand biasing we consider can both be state-dependent.

3. Constraints and Biasing

3.1. Environment Constraints

Typically inputs to a design are not totally free; theyare subject to certain constraints, on which the cor-rect functionality of the design is based. In SimGen,environment constraints specify thelegal input space,which can be state-dependent if the constraints involvestate information. Constraints not involving inputs aretreated as invariants and monitored during simulation.We will not distinguish these types of constraints here-after.

In SimGen, constraints are expressed as Boolean for-mulas. Consider the following example: suppose thatin an address state machine of a bus controller, the“transaction start” input should be asserted only if thedesign is in the “address idle” state. This constraint canbe captured by the following statement:

$constraint(trans start→ (addr state

== ADDR IDLE));

More examples of constraints will be given inSection 5.1 and 6.2.

The Boolean formulas of constraints are implicitlyrepresented by BDDs defined over input and state vari-ables. Efficient algorithms for building BDDs fromBoolean formulas, and forming the conjunction ofBDDs can be found in [21]. In the sequel, thecon-straint BDD refers to the conjunction of the BDDs ofall constraints, unless otherwise stated.

To facilitate our discussion later in the paper, we de-scribe briefly how a constraint is represented by a BDD.For a general introduction to BDDs, we refer the readerto [10]. A constraint BDD has two leaf nodes, namely

110 Yuan et al.

ONEandZERO. Each non-leaf node is either associatedwith an input variable (thus called an input node), or astate variable (thus called a state node). Given a truthassignment to the input and state variables, whether aconstraint is satisfied can be decided by evaluating thecorresponding BDD recursively, starting from the rootnode, and branching to itsthennode if its associatedvariable is assigned to 1, otherwise, to itselsenode.The path of above evaluation always terminates at oneof the two leaf nodes,ONEor ZERO, which means theconstraint is satisfied or violated, respectively. On anypath, a variable can be visited at most once (possiblynever), and the order in which the variables are visitedconforms to a global ordering.

For any given state of the design, the set of pathsto ONE that agree with the state variable assignmentrepresents thelegal input spaceunder that state. If theset is empty, we define the design to be in anillegalstate.

SimGen handles a rich class of constraints becauseof the constraints’ dependency on state information.For example, by adding auxiliary variables to remem-ber past states, SimGen can constrain the sequentialbehavior of the inputs; this is discussed in Section 6.2.

3.2. Input Biasing

It is often the case that in order to exercise the designin “interesting” scenarios, one needs to bias the inputs.To do so, we first define theinput probability.

Definition 1. The input probabilityPi of an inputui

is a real number in [0, 1], denoting the probability ofui taking the value 1.

Input probabilities with the value 0 or 1 are convertedto constraints. For example,Pi = 0 is replaced by$constraint(ui == 0).

Biasing on an input can simply be a constant inputprobability, or a set of input probabilities expressed bya function of the state that evaluates to a real in (0, 1)exclusive. The following gives a flavor of biasing inSimGen.

$setprob1(ui , state== UP ? 0.2 : state

== DOWN ? 0.8 : 0.5);

This statement assignsPi to 0.2, 0.8 and 0.5 re-spectively in theUP, DOWNand other states. We useVerilog expression for input biasing. The “?” is the

conditional assignment operator in Verilog which isfollowed by the “then” clause, the “else” clause fol-lows the “:”.

We stress that satisfying both the constraints and theinput probabilities is not always possible. Consider thecase where an input is assigned a high input probability(of being 1), but the design is constrained such that novectors in the legal input space assign the input to 1.In such situations, satisfying the input constraints takespriority.

3.3. Constrained Probabilities

To handle the combination of constraints and inputprobabilities in a unified way, we define thecon-strained probabilityof an input vector. We first intro-duce the notation “weight of an input valueαi of ui ” asp(ui = αi ), and “weight of an input vectorEα” asπ(Eα),which is given by

∏αi∈Eα p(ui = αi ). By Definition 1,

p(ui = 1) = Pi and p(ui = 0) = 1− Pi .The constrained probability of an input vector is then

defined as follows.

Definition 2. Let u0, . . . ,un−1 be the design in-puts, andUs be the legal input space under states.The constrained probability of an input vectorEα =〈α0, . . . , αn−1〉, given the states, is 0 if Eα /∈ Us; other-wise, the constrained probability is

π(Eα)∑Eβ∈Us

π( Eβ)

Conceptually, the constrained probability of an inputvector is the weight of that vector divided by the sumof the weights of all vectors that satisfy the constraint.

With our definition, an input vector is legal if andonly if its constrained probability is not zero. Also,the constrained probabilities are closely correlated toinput probabilities. Although intuitively clear, it stillrequires a proof which we will give in Section 4.3.

3.4. An Example of Constrained Probability

First, we consider the case in which there are no con-straints on the inputs. Take as an example a design withfour inputs,cmd[3], cmd[2], cmd[1] and cmd[0], andtheir input probabilities (of being 1) 1/2, 1/3, 1/4, and1/5 respectively. This results in the following inputvector probabilities: (Middle vectors are removed forbrevity as their computation is trivial.)


1/2 1/3 1/4 1/5cmd[3] cmd[2] cmd[1] cmd[0] P(vector)

0 0 0 0 1/2 · 2/3 · 3/4 · 4/5= 24/120

0 0 0 1 1/2 · 2/3 · 3/4 · 1/5= 6/120

0 0 1 0 1/2 · 2/3 · 1/4 · 4/5= 8/120

.

.

....

1 1 0 1 1/2 · 1/3 · 3/4 · 1/5= 3/120

1 1 1 0 1/2 · 1/3 · 1/4 · 4/5= 4/120

1 1 1 1 1/2 · 1/3 · 1/4 · 1/5= 1/120∑ = 120/120= 1

Note that the sum of all the values in the right columnis 1.

When we add constraints on the inputs, some vectorswill no longer be legal. Consider the above input setand probabilities with the following constraint:

$constraint(!reset→ (cmd[3:0]== 4’b1000‖cmd[3:0]== 4’b0100‖cmd[3:0]== 4’b0010‖cmd[3:0]== 4’b0001));

In a state in which “reset” is 0, the constraint restrictsthe valid input vectors to the four shown, enforcing a“one-hot” encoding. The valid input vectors and theirprobabilities (actually,weights, since the sum is lessthan 1 now) are as follows.

1/2 1/3 1/4 1/5 Unnormalizedcmd[3] cmd[2] cmd[1] cmd[0] P(vector)

0 0 0 1 1/2 · 2/3 · 3/4 · 1/5= 6/120

0 0 1 0 1/2 · 2/3 · 1/4 · 4/5= 8/120

0 1 0 0 1/2 · 1/3 · 3/4 · 4/5= 12/120

1 0 0 0 1/2 · 2/3 · 3/4 · 4/5= 24/120∑ = 50/120 6= 1

The constrained probability is computed by normali-zing the weights with regard to the set of all vectorssatisfying the constraints. The results are 3/25, 4/25,6/25, and 12/25, for the vectors in Row 1 throughRow 4, respectively.

Instead of enumerating all possible input vectors andstates as demonstrated in above tableau methods, theBDD-based symbolic method described in the next sec-tion computes the constrained probabilities implicitly.This method is shown to be very efficient in handling

large and complex constraints in commercial designs,as can be seen from the experimental results given inSection 6.

4. Vector Generation

Since constrained probabilities capture both the con-straints and input biasing, our vector generation goalnow becomes generating vectors according to their con-strained probabilities. In this section, we describe whatwe refer to as the “probability-tree”(p-tree) algorithmthat consists of a probability computing and a vectorgenerating procedure. We also explain why this algo-rithm achieves our goal.

4.1. The Branching Probabilities

By analyzing the problem, we found that generatingvectors from a constraint BDD is analogous to evalu-ating the BDD for an input vector and a state, exceptthat in the former we need to “choose” instead of look-ing up the input values when deciding what branchesto take. In the following, we describe how we makebiased choices according to “branching probabilities”.

First we introduce the notation “weight of a node”.We define the weight of nodeONEas 1, and the weightof ZEROas 0. The weight of a non-leaf nodeσ j corre-sponding to variablevi , givent j —the weight of itsthennode, andej —the weight of itselsenode, is recursivelydefined as the following:

w j =

p(vi = 1) · t j if vi is an input variable+ p(vi = 0) · ej

t j if vi is an state variableandvi = 1

ej if vi is an state variableandvi = 0

(1)

Conceptually, the weight of a node is the sum ofweights of partial vectors correspondent to the set ofpaths from the node toONE. By induction on the lengthof the paths, the weight of the root node is the sum ofweights of all legal vectors. A proof of this, without thecomplication of state variables, can be found in [19].Since the role of state nodes is to select a subset of inputnodes for weight computation, as shown in Eq. (1), ourintroduction of state nodes preserves the validity of thatproof for every given state.

112 Yuan et al.

Thebranching probabilityof an input nodeσ j is thefraction oft j or ej tow j , weighted by the weight of itsinput value, as given in the following formulas:

σ j .thenbranch= p(vi = 1) · t j

w j(2)

σ j .elsebranch= p(vi = 0) · ej

w j(3)

Note that the sum ofσ j .thenbranch andσ j .elsebranch is equal to 1.

The procedure,Weight, of computing weights andbranching probabilities is given in Fig. 1. We use thefollowing notations:node.var is the variable associatedwith a BDDnode; node.then andnode.else are the twochild nodes (branches) ofnode, taken whennode.var isassigned to 1 and 0, respectively;prob(var,st) returnsthe input probability ofvar under statest. In the im-plementation,prob is the input biasing consisting of aVerilog expression that can be evaluated by the simu-lator dynamically.

Note that procedureWeight computes branchingprobabilities only for input nodes. Observing that thebranching probability is always 0 (resp. 1) ifZERO

Weight(node,st){if (node== ONE) return 1;if (node== ZERO) return 0;if (nodeis visited)return node.weight;set visited(node);

if (nodeis an input node){t =Weight(node.then,st);e=Weight(node.else,st);}else if(nodeis a state node){

if (prob(node.var,st) == 1)node.weight=Weight(node.then,st);

elsenode.weight=Weight(node.else,st);

return node.weight;}

node.weight= prob(node.var,st) * t +(1 - prob(node.var,st)) * e;

node.thenbranch= prob(node.var,st) * t / node.weight;node.elsebranch= 1 - node.thenbranch;

return node.weight;}

Fig. 1. Computing branching probabilities.

is thethen(resp. else) node, we derive the followinglemma:

Lemma 1. From any input node, the probability oftaking a branch to the ZERO node is always0.

ProcedureWeightperforms a one-pass computationthrough a subset of nodes of the constraint BDD, andannotates the visited input nodes with branching prob-abilities. Its time complexity isO(n) wheren is thenumber of nodes in the constraint BDD. When presentstate nodes have received their sampled values, the non-selected branches implicitly point to ZERO. Then, theBDD becomes effectively the cofactor [10] of the con-straint BDD with regard to the current state. The dif-ference is that the implicit “cofactoring” inWeightis“in-place” in the sense that it never creates new BDDnodes, thus avoiding potential runtime BDD explosioncaused by normal cofactoring.

4.2. The Biased Vector Generation Procedure

The biased vector generation performs a top-downtraversal in the constraint BDD as the following: ata state node, take thethen (resp. else) branch if thecorresponding state variable is assigned to 1 (resp. 0)in the current state; at an input node, take a branch ac-cording to itsbranching probabilities, and set the valueof the corresponding input variable accordingly. Thisis implemented in the procedureWalkshown in Fig. 2.

Walk(node,st){if (node== ZERO) error(“illegal state”);if (node== ONE) return ;

if (nodeis a state node){if (node.var is assigned to 1 inst)

Walk(node.then,st);elseWalk(node.else,st);

} else{let r= random(0,1);if (r < node.thenbranch){

assignnode.var to 1;Walk(node.then,st);

} else{assignnode.var to 0;Walk(node.else,st);

}}}

Fig. 2. Generating test vectors.


Because a variable can be visited at most once onany path in a BDD, above procedure is guaranteed toterminate withinmsteps wherem is the sum of numbersof input and state variables in the constraint BDD. Atend of the traversal, we must be in one of the followingsituations:

1. We are at theZEROnode. From Lemma 1, the pre-vious node can only be a state node, which indicateswe are at an illegal state. We signal an error and exitthe simulation.

2. We are at theONEnode. We generated a legal inputvector.

For the inputs that were not visited by above proce-dure, the values are randomly assigned according totheir input probabilities. The default, 0.5, is used foran input if its input probability is not given by theuser.

ProcedureWeight is not always needed before weperformWalk. In particular, if all state variables occurbefore the inputs in the constraint BDD (which will bediscussed in Section 5) and we use only static biasing,then it is sufficient to executeWeightonly once at thebeginning of the simulation.

Figure 3 provides a high level view of SimGen. TheConstraint Compilerreads a Verilog model annotatedwith constraints and input biases, extracts and compilesthe constraints into BDDs which will be used by theVector Generator. During simulation, the Verilog-XLsimulator evaluates input biasing under current state ateach clock cycle (recall in Section 3.2 input biasing isrepresented by Verilog expressions). TheVector Gen-erator then samples the current state and generates aninput vector based on the constraint BDDs.

Fig. 3. SimGen flow chart.

4.3. Correctness and Properties

Recall that our goal was to implicitly represent con-straints and input probabilities, and generate vectorsaccording to their constrained probabilities. The fol-lowing theorems demonstrate that SimGen, by apply-ing thep-treealgorithm described in Sections 4.1 and4.2 achieves this goal.

Theorem 1. At a legal state, the p-tree algorithm willgenerate a legal input vector.

Proof: At a legal state, a branch taken from a statenode cannot reachZERO. This together with Lemma 1imply that the last node visited byWalkmust beONE,meaning the path represents a legal input for that state.

2

In other words, if there exist any legal inputs, thealgorithm guarantees to find one in one pass.

Theorem 2. The probability of generating an inputvector using the p-tree algorithm is equal to the con-strained probability of the same vector.

Proof: From Lemma 1,p-tree never generates aninput vector that is illegal. This agrees with the con-strained probability of an illegal input vector, which is0 by Definition 2.

We now consider legal vectors. Let〈α0, . . . , αn−1〉be a legal vector. Without loss of generality, assumethat input variablesu0, . . . ,um were visited by the pro-cedureWalkas the nodesσ0, . . . , σm, in that order; theother inputs were not visited. The last node,σm, mustbranch toONE from Lemma 1. Then the probabilityof this vector, as generated byp-tree, is given by

p(u0 = α0) · w1

w0· p(u1 = α1) · w2

w1· · ·

p(um = αm) · 1wm

·n∏

j=m+1

p(u j = α j )

which simplifies to [∏n

j=0 p(u j = α j )]/w0.Note thatσ0 is the root node, thereforew0 is the

sum of weights of all vectors. Thus the probabilityof generatingEα in the p-treealgorithm is exactly theconstrained probability ofEα as given in Definition 2.

2

114 Yuan et al.

Since the generated vectors match their constrainedprobabilities precisely, we derive the following impor-tant properties of thep-treealgorithm.

Lemma 2. The probability of generating an inputvector using the p-tree algorithm is independent of theinput and state variable ordering.

This is directly derived from the definition of con-strained probabilities, which has nothing to do withBDD variable ordering. This property is important,since variable re-ordering [21] is used in SimGen forBDD minimization.

Lemma 3. Using the p-tree algorithm, the probabil-ity of generating an input vectorEα in which ui equals1(or 0) monotonically increases as Pi increases(ordecreases).

Proof: We prove the case whenui = 1, the othercase is analogous. There are two cases. (1) the vari-able ui is not in the BDD: Branching probabilitiesin the BDD are independent ofPi . From the algo-rithm, the probability of the vector is proportionalto Pi , when ui = 1. (2) ui is in the BDD: Theprobability of Eα is

∏nk=0 p(uk = αk)/w0 from the

proof of Theorem 2. By Lemma 2,ui can be consid-ered to be at the root node. Then the probability be-comes

∏nk=0 p(uk = αk)/[ Pi · t0+ (1− Pi ) · e0]. Let

q be the product of p(u j =α j ) for all j 6= i ,consider ui = 1, rewrite above formula asq · Pi /

[ Pi · t0+ (1− Pi ) · e0], again asq/[t0+ ( 1Pi− 1) · e0].

Note t0 ande0 are independent ofPi sinceui is at theroot, also 0≤ p ≤ 1, therefore the above formulamonotonically increases inPi . 2

4.4. An Example of the Probability-Tree Algorithm

Using the prior example of the four inputs constrainedto be “one-hot”, we can show how the constraint BDDcan be marked with weights and how these weights canbe used by the vector generation engine to produce onlyconstrained input vectors and do so with predictable,desired probabilities as specified by the user.

The constraint BDD is shown in Fig. 4. Each nodeis labeled with its weight as assigned by thep-treealgorithm. Solid arcs representthenbranches. Dashedarcs representelsebranches. The variable ordering isshown to the right of the BDD.

As described in Section 4.1, the state nodes in a con-straint BDD are only used for in-place “cofactoring” in

Fig. 4. Constraint BDD with nodes weighted byp-treealgorithm.

thep-treealgorithm. The actual computation is donein the resulting sub-BDD. If in the current stateresetis asserted, then the sub-BDD contains only theONEnode, meaning that all inputs are possible, and that theconstrained probability of a vectorEα is simply the prod-uct

∏αi∈Eα p(ui = αi ) (cf. Section 3.3); the case we are

going to consider below is whenresetis 0, in which theresulting sub-BDD is the one rooted at nodea.

To illustrate the calculation, consider nodef . Ap-plying thep-treealgorithm yields the following calcu-lation:

weight( f ) = prob( f.var,reset:= 0) ∗ weight(ZERO)

+ (1− prob( f.var,reset:= 0))

∗weight(ONE)

= 1/5 ∗ 0+ 4/5 ∗ 1

= 4/5

Similarly,weight(g) = 1/5. Then the weight of nodee is calculated as:

weight(e) = prob(e.var,reset:= 0)) ∗ weight( f )

+ (1− prob(e.var,reset:= 0))

∗weight(g)

= 1/4 ∗ 4/5+ 3/4 ∗ 1/5= 4/5

= 7/20


We can now use the weightings shown to calculatethe probability that a given vector will be chosen dur-ing vector generation in SimGen. It should agree withthe previously calculated constrained probabilities inSection 3.4. Consider the vector〈1, 0, 0, 0〉. The prob-ability that this vector will occur is the the product ofthe probabilities for taking each branch on the vector’spath through the BDD.

Starting from nodea, using formulas 2 and 3, theproduct is:

1/2 · 2/55/12

· (1− 1/3) · 3/52/5

· (1− 1/4) · 4/53/5

· (1− 1/5) · 14/5

= 12

25

Each fraction represents the probability of taking theappropriate branch at each node. In this case, we takethe then branch first, followed by three else branches,hence generating the vector〈1, 0, 0, 0〉. Note that theresult, 12/25, is equal to the constrained probabilitywe calculated for vector〈1, 0, 0, 0〉 in Section 3.4.

5. Minimization of the Constraint BDD

We describe in this section how we minimize the sizeof constraint BDD in an effort to improve SimGen’sruntime efficiency. Note that all of these operationsare done prior to simulation.

5.1. Variable Ordering

The number of nodes in the BDD representation of alogic function depends on the function and the vari-able ordering. In many cases, the variable ordering canhave a dramatic effect on BDD size. For example, theconstraint

$constraint((addrstate!= 2’b11) ?

(input== PREV input) : 1);

says that ifaddr stateis not equal to binary value 11,theninputmust hold its previous value, which is storedin PREV input. Let’s sayinput is a vector of widthn.If we forced the variables ininput to occur before theones inPREV input in the ordering, the comparatorcorresponding to “==” would result in a BDD of 2n

nodes.

A technique called dynamic reordering [21] is oftenvery efficient at minimizing BDD size. The BDD forthe above mentioned constraint, with the ordering re-striction lifted, can reduce to just2nnodes with the helpof variable reordering. SimGen uses various reorder-ing techniques, such as sifting [22], to help reduce thesize of the constraint BDD. Variable reordering turnedout to be critical for building BDDs for such constraintsas mentioned above.

5.2. Constraint Partitioning

When there is a large number of constraints, formingthe conjunction BDD could be very expensive because(1) the computation could blow up because of large in-termediate BDDs, (2) the resulting BDD could be verylarge thus slow down the vector generation. To dealwith this problem, SimGen partitions the constraintBDDs into sets with disjoint input variable support us-ing the following procedure:

1. for each input variableui , create a partition;2. for each constraint BDDcj , if cj depends onui , put

cj in ui ’s partition;3. merge all partitions that share a common constraint

BDD until each constraint appears in at most onepartition;

4. put each of the constraint BDDs that are not in anypartitions yet in a new partition by itself. Observethat these constraints should depend only on statevariables;

5. form the conjunct BDD for each partition.

Thep-treealgorithm can then be applied to each par-tition separately. The resulting partial inputs are con-catenated to form the final input vector. The partitionsthat depend only on state variables don’t contribute tothe final vector; application ofp-tree to those parti-tions can have one of two possible results: the currentstate is legal, or illegal, in which case the simulation ex-its. The soundness ofp-treealgorithm under constraintpartitioning is guaranteed by the following theorem:

Theorem 3. For a given state, the probability of gen-erating an input vector by concatenating the resultsof applying p-tree to BDDs of constraint partitions isequal to the constrained probability of the same vectorgenerated by applying p-tree to the monolithic con-straint BDD.

116 Yuan et al.

Proof: In the proof, we shall refer to a “constraint”as the cofactor of the original constraint w.r.t. thegiven state. LetC be the conjunction of all con-straints andC1, . . . ,Cm be the partitions, i.e.,C =∧m

i=1 Ci . Let Eα1, . . . , Eαm be the corresponding par-tial vectors generated from the partitions, andEα =Eα1Eα2 · · · Eαm be the concatenated final input vector.Let w(Ci ) denote

∑Eβ∈Ci

π( Eβ) (cf. Definition 2), re-call π( Eβ)= ∏βi∈ Eβ p(ui =βi ). We shall prove that∏m

i=1(π(Eαi )/w(Ci ))≡π(Eα)/w(C). From the defi-nition of π , π(Eαi ) · π(Eα j )≡π(Eαi Eα j ), (i 6= j ),thus

∏mi=1π(Eαi )≡π(Eα). What’s left to be proved is∏m

i=1w(Ci )=w(C). Since for all i, j , i 6= j , Ci

and Cj have disjoint supports, therefore (1)β ∈Ci ∧ γ ∈ Cj ⇔ βγ ∈ Ci

∧Cj , (2) w(Ci ) ·

w(Cj )=∑Eβ∈Ci

π( Eβ) ·∑ Eγ∈Cjπ( Eγ )= ∑ Eβ∈Ci

[π( Eβ) ·∑Eγ∈Cj

π( Eγ )] = ∑Eβ∈Ci

∑Eγ∈Cj

[π( Eβ) · π( Eγ )] =∑Eβ∈Ci

∑Eγ∈Cj

π( Eβ Eγ ). From (1), (2) can be simpli-

fied to∑Eβ Eγ∈Ci∧Cj

π( Eβ Eγ )=w(Ci ∧ Cj ); inductively,∏mi=1w(Ci )=w(C1 ∧ · · · ∧ Cm)=w(C). 2

It is possible that the conjunction of all the con-straints is empty while none of the constraint BDDsof the partitions are empty. Fortunately, this will be-come apparent immediately after the vector genera-tion starts—theWalkprocedure will generate no vectorand exit since there is nolegal state, as described inSection 4.2.

6. Results

SimGen was used in the verification of a suite of sevenfunctional blocks in aPowerPCTM-based DSP appli-cation. We report results of experiments on buildingconstraint BDDs for these blocks in Section 6.1. InSection 6.2 and 6.3, we report a case study and SimGencoverage on one of the blocks.

The underlying BDD package is CUDD [22], de-veloped in the University of Colorado. All experi-ments were conducted on a 233 MHz UltraSPARC-60machine with 512 MB main memory.

6.1. Building the Constraint BDDs

Table 1 reports the complexity of our benchmark de-signs. The BDD variables reported include inputs andlatches. Columns 2 through 4 give the total numberof BDD variables, the number of constraints, and the

Table 1. Design and constraint statistics.

Example |Vars| |Constraints| |Cons Vars|

block-1 76 13 26

block-2 178 10 59

block-3 1437 11 153

block-4 446 33 175

block-5 407 34 297

block-6 185 107 156

block-7 1396 171 677

Table 2. Building the constraint BDD with-out partitioning.

Example Time(sec) Peak Final

block-1 0.0 6312 54

block-2 5.0 5110 119

block-3 26.0 6132 774

block-4 885.5 303534 110858

block-5 727.7 181243 82405

block-6 8.3 19418 4094

block-7 365.0 218708 98658

number of variables used in the constraints, respec-tively. Note thatblock-7has 1396 latches and 677 ofthem are used in its constraints.

The results of building constraint BDDs withoutpartitioning are shown in Table 2. Dynamic variablereordering was enabled in all experiments. Without par-titioning, thepeakBDD size can be as high as 300,000nodes (block-4), while block-4, block-5, andblock-7each have close to 100,000 nodes in thefinal constraintBDDs, as shown in Table 2.

Table 3 shows the effectiveness of using partitioning.Column 5 lists the total number of constraints, and the

Table 3. Building the constraint BDDs with partitioning.

Example Time(sec) Peak Final Partition

block-1 0.0 1022 43 13/5

block-2 4.0 5110 103 10/7

block-3 20.3 6132 609 11/9

block-4 38.0 13286 1595 33/10

block-5 33.4 22484 1962 34/9

block-6 2.0 10220 2722 107/16

block-7 53.0 33726 12535 171/18


number of partitions formed. Partitioning gives modestimprovement to BDD size forblock-1, block-2, block-3andblock-6, but reduces both time and space complex-ity for block-4, block-5andblock-7dramatically. Thecomplexity of the designs (close to 1400 variables) andconstraints (over 600 variables), together with the sizeof the constraint BDDs (less than 13,000 nodes, mostcases under 3,000 nodes) demonstrate that our tech-nique is feasible for block-level and possibly unit-leveldesigns.

6.2. A Case Study

SimGen was used to construct verification environ-ments for several commercial designs blocks. In thefollowing we present its application to aPowerPCTM

slave block—namely theblock-5 in the previoussubsection.

The first task is to develop the environment con-straints according to a specification for the block (in-cluding its interface) written in English. Because ofthe sequential nature of the specification, we neededto introduce auxiliary variables to remember previousinput or state value. With those auxiliary variables,constraints can also depend upon the past state of thedesign.

Conflicts between constraints occur often becauseof their complexity and the number of constraints in-volved. In such cases, methods such as prioritizationare used to resolve the conflicts. A prioritized constraintlooks like the following:

$constraint(st != 2’b11 ? a== PREV a :(!b & !c) ? u== PREV u :(!b & c & !(e == 0 ‖ e== 4)) ?

((f < 7) & (f > 0)) :(t == 0 ‖ t == 4 ‖ t == 7) ?

!(z == 6 ‖ z== 5) :1 );

Note thatPREV a andPREVu are auxiliary vari-ables holding the values ofaanduon the previous clockcycle. Because of the comparator “a== PREVa”, aninterleaved ordering of bits ofa andPREVa should beused.

The constraints for this block were written in abouttwo person-days. The end result was a concise speci-fication of the environment in a 200-line Verilog file,including 34 constraints, the auxiliary variables andabstract state machines. The benefit of constraining

can not be overemphasized. Unconstrained randomsimulations generally produce false negative results.We also noticed in unconstrained simulations that theX value was constantly generated on tri-state buses,indicating bus contentions, which made the simulationsmeaningless.

Developing input biasing was straightforward inmost cases. For instance, we wanted to limit the fre-quency that the “error” input (which signals an errorcondition in the environment) is asserted when test-ing the essential functionality of a design. This wasexpressed as

$setprob1(error, 0.2);

There were also cases where we needed to considerthe more involved dynamic biasing. For example, evenafter we give static biasing to over a dozen critical inputsignals, three major state machines stayed mostly idlethrough trial simulations. After studying the designfor about an hour (it should take the original designermuch less time), we were able to find a set of dynamicinput biases that stimulated many more events in activestates, such asreadandwrite. The following biasinginstructs SimGen to give much higher input probabilityto ui in stateIDLE, which triggers a transition out ofIDLE:

$setprob1(ui ,addr state== IDLE ? 0.9, 0.5);

We wrote a few dynamic biasing commands for eachof the state machines. Note that we did not use con-straints to express the condition, because “leaving” theidle state is not a mandatory action, it just serves ourtest purpose. Also, using input biasing instead of con-straints can avoid potential constraint contradictions,because the constraints take priority over input proba-bilities.

Of course, in general, state transitions are controlledby both state and inputs, and justifying an enable con-dition can be a very hard problem. However, with thehelp of automatic tools such as SimGen, this problemcan be lessened. Table 4 shows the effect of biasing on

Table 4. Effectiveness of biasing.

Biasing |R/W requests| t (sec)

None 14 6.8

Static 811 6.2

Dynamic 1918 6.3

118 Yuan et al.

Table 5. Overhead of SimGen.

Setting Total time(sec) SimGen overhead

Random 44.9 12.0%

SimGen 48.2 21.3%

SimGen with dump 63.6 16.0%

SimGen with monitors 635.6 1.7%

the number of active states (of three major state ma-chines) visited during simulations of 1000 cycles each.The dynamically biased simulation increased the cov-erage 130 times over that of the unbiased simulation.

Simulations usually run with monitoring processes,or dump the results to log files for post-processing.Table 5 summarizes the run time overhead of SimGenon block-5, with partitioned constraint BDDs and dy-namic biasing. All simulations ran for 10000 cycleseach. Rows 2 to 5 respectively represent the simulationswith pure random generation, stand alone SimGen,SimGen with Verilog dump, and with property moni-toring. It can be seen that the overhead of SimGen isfairly low.

6.3. Coverage

To measure the quality of vectors generated by SimGen,we conducted two simulations. First we ran 29 directedtests targetingblock-5, written by several design engi-neers over months. Because these tests were written inassembly code, the simulation had to be run at the chiplevel. This simulation took about 24 hours, the totalnumber of cycles is 150,000. In the second simula-tion, we used SimGen to driveblock-5, with minimumamount of biasing. For example, we disabled exter-nal error and scan testing, and enabled data input. Weran the simulation 100 times, each with 10,000 cycles.The total amount of time taken was 4.5 hours. In bothsimulations, values of signals inblock-5were saved infiles for coverage measurement.

Table 6 gives the coverage of state transitions onthree major state machines ofblock-5and the macromachine composed of those state machines. In Table 7,we measured the “condition coverage” on six multi-cycle properties about the block. These propertieswere written in a proprietary Verilog-like language,with nested if-then-else construct. The condition cov-erage is the percentage of the if/else conditions satisfiedduring simulation.

Out of a total of 10 cases (4 for state machines and6 for properties), the full-chip and SimGen simulation

Table 6. State transition coverage ofblock-5.

State machine Full chip SimGen

fsm1 100% 100%

fsm2 87% 87%

fsm3 100% 88%

Composite 26% 18%

Table 7. Property condition coverageof block-5.

Property Full chip SimGen

property1 100% 100%

property2 100% 100%

property3 100% 84%

property4 100% 100%

property5 100% 100%

property6 100% 100%

coverage tied on 7 cases, 6 of which were 100%. Thefull-chip simulation outperformed SimGen in the re-maining 3 cases, but not by much. Note the cover-age on the composite machine was very low. Besidesthe fact that the transitions of the composite machinewere harder to activate, we conjecture that our aggres-sive estimation on possible transitions also contributesto this low coverage.

This experiment showed that the quality of SimGengenerated vectors is very close to that of the hand-crafted test cases. Considering that writing constraintsand biases is easier than developing directed tests, andcan be done at the block level instead of after the inte-gration of the full chip, the advantage of using SimGenfor functional block verification is obvious.

In the verification of these blocks, SimGen, togetherwith a multi-cycle property description and monitoringtool, discovered 30 design bugs. These were found notonly by assertion failures in the monitors, but also bythe design entering states where the set of legal inputswas empty.

7. Conclusion and Future Work

We have described an automated method of generat-ing input vectors in functional verification. Constraintsare used to confine the generated vectors to a legal in-put space. Input biasing is used to direct simulationgeneration to desired scenarios. Both constraints and


input biasing can depend dynamically on the state ofthe design. Constraints are represented symbolicallyin BDDs. An algorithm was developed to efficientlygenerate input vectors during simulation without back-tracking or retry. The algorithm is implemented in atool called SimGen. Methods of minimizing the con-straint BDDs were presented. We also discussed theapplication of SimGen to commercial designs.

We plan to conduct experiment on coverage directedvector generation with SimGen. For example, after acertain amount of simulation, state and/or transitioncoverage for critical control logics is measured. Theunexercised states/transitions and their enable condi-tions can then be described as constraints to SimGenso that the generated inputs that follow will enable thedesired state transitions. Representing the enable con-ditions as constraints efficiently is a key problem toovercome. We are also considering methods of maxi-mizing the benefit of constraint partitioning, i.e., BDDsof each partition are built and saved separately, so thatBDD variables existing at any time are the ones pertain-ing to the current partition only, instead of all partitions.

References

1. C. Pixley, N.R. Strader, W.C. Bruce, J. Park, M. Kaufmann,K. Shultz, M. Burns, J. Kumar, J. Yuan, and J. Nguyen, “Com-mercial Design Verification: Methodology and Tools,”Proc.Intl. Test Conf., 1997.

2. A. Aharon, A. Bar-David, B. Dorfman, E. Gofman,M. Leibowitz, and V. Schwartzburd, “RTPG-A Dynamic BiasedPseudo-Random Test Program Generator for Processor Verifi-cation,” IBM Technical Report 88.290, July 1990.

3. J. Monaco, D. Holloway, and R. Raina, “Functional VerificationMethodology for the PowerPC 604 Microprocessor,”Proc. ofthe Design Automation Conf., 1996.

4. F. Casaubieilh, A. McIsaac, M. Benjamin, M. Bartley, F.Pogodalla, F. Rocheteau, M. Belhadj, J. Eggleton, G. Mas, F.Barrett, and C. Berthet, “Functional Verification Methodologyof Chameleon Processor,”Proc. of the Design Automation Conf.,1996, pp. 421–426.

5. S. Taylor, M. Quinn, D. Brown, N. Dohm, S. Hildebrandt, J.Huggins, and C. Farney, “Functional Verification of a Multiple-issue, Out-of-Order, Superscalar Alpha Processor,”Proc. of theDesign Automation Conf., 1998, pp. 638–643.

6. A. Evans, A. Silburt, G. Vrckovnik, T. Brown, M. Dufresne,G. Hall, T. Ho, and Y. Liu, “Functional Verification of LargeASICs.,”Proc. of the Design Automation Conf., 1998, pp. 650–655.

7. CADENCE. Verilog-XL User Reference.8. C. Pixley, “Integrating Model Checking Into the Semiconductor

Design Flow,”Computer Design’s Electronic Systems Journal,pp. 67–74, March 1999.

9. M. Kaufmann, A. Martin, and C. Pixley, “Design Constraintsin Symbolic Model Checking,”Proc. of the Computer Aided

Verification Conf., 1998.10. R. Bryant, “Graph-based Algorithms for Boolean Function

Manipulation,” IEEE Transactions on Computers, Vol. C-35,pp. 677–691, Aug. 1986.

11. J.R. Burch, E.M. Clarke, K.L. McMillan, and D.L. Dill,“Symbolic Model Checking: 1020 States and Beyond,”Infor-mation and Computation, Vol. 98, No. 2, pp. 142–170, 1992.

12. K.L. McMillan, Symbolic Model Checking, Kluwer AcademicPublishers, 1993.

13. J. Freeman, R. Duerden, C. Taylor, and M. Miller, “The 68060Microprocessor Functional Design and Verification Methodol-ogy,” On-Chip Systems Design Conference, 1995, pp. 10.1–10.14.

14. A. Aharon, A. Bar-David, B. Dorfman, E. Gofman,M. Leibowitz, and V. Schwartzburd, “Verification of the IBMRISC System/6000 by a Dynamic Biased Pseudo-random TestProgram Generator,”IBM Systems Journal, Vol. 30, No. 4, pp.527–538, July 1991.

15. A.K. Chandra and V.S. Iyengar, “Constraint Solving for TestCase Generation—A Technique for High Level Design Verifica-tion,” Proc. Intl. Conf. on Computer Design, 1992, pp. 245–248.

16. A. Chandra, V. Iyengar, D. Jameson, R. Jawalekar, I. Nair,B. Rosen, M. Mullen, J. Yoon, R. Armoni, D. Geist, and Y.Wolfsthal, “AVPGEN—A Test Case Generator for ArchitectureVerification,” IEEE Transactions on Very Large Scale Integra-tion (VLSI) Systems, Vol. 3, No. 2, pp. 188–200, June 1995.

17. R.I. Bahar, E.A. Frohm, C.M. Gaona, G.D. Hachtel, E. Macii,A. Pardo, and F. Somenzi, “Algebraic Decision Diagrams andtheir Applications,”Proc. Intl. Conf. on Computer-Aided Design,1993, pp. 188–192.

18. G.D. Hachtel, E. Machii, A. Pardo, and F. Somenzi, “SymbolicAlgorithms to Calculate Steady-State Probabilities of a FinitState Machine,”The European Design and Test Conference,1994, pp. 214–218.

19. M. Blum, A.K. Chandra, and M.N. Wegman, “Equivalence ofFree Boolean Graphs Can Be Decided Probabilistically in Poly-nomial Time,” Information Processing Letters, Vol. 10, No. 2,pp. 80–82, 1980.

20. R. Krieger, B. Becker, and R. Sinkovic, “A BDD-based Algo-rithm for Computation of Exact Fault Detection Probabilities,”International Symposium on Fault-Tolerant Computing, 1993,pp. 186–195.

21. K. Brace, R. Rudell, and R. Bryant, “Efficient Implementationof a BDD Package,”Proc. of the Design Automation Conf., June1990, pp. 40–45.

22. F. Somenzi, I. Bahar, H. Cho, E. Frohm, C. Gaona, C. Hua,J.Y. Jang, S.W. Jeong, B. Kumthekar, E. Macii, B. Manne, I.H.Moon, C. Musfeldt, S. Panda, A. Pardo, B. Plessier, K. Ravi, H.Shin, A. Shuler, and J. Sivesind, “CUDD: CU Decision DiagramPackage,” ftp://vlsi.colorado.edu/pub/

Jun Yuan received his B.S. from Tsinghua University, China in1989 and M.S. in Computer and Electrical Engineering from theUniversity of Texas at Austin in 1995, where he is current a part-time Ph.D. student. He works on formal and semi-formal verificationtechnologies in Motorola’s Design Verification group.

Kurt Shultz has been a software developer with Motorola’s DesignVerification group since 1995. He is currently studying Mathematicsat the University of Texas at Austin.

120 Yuan et al.

Carl Pixley received B.S., M.S. and Ph.D. degrees in Mathemat-ics from the University of Omaha (1966), Rutgers (1968), and theState University of New York at Binghamton (1971). He workedon the faculties of the University of Texas (Austin) and South-west Texas State University and worked at Burroughs, MCC, Mit-subishi Electric, and Motorola Inc. Dr. Pixley has published over40 papers and five patents, and received awards for his scientificand engineering inventions such as, the Pixley-Roy topology, theEaton-Pixley-Edwards-Miller theorem, and the BDD-based ap-proach to model checking, sequential equivalence, resetability andconstraint-based verification. He has served on the program com-mittees of IEEE Intl. Conf. of Computer Design, IEEE/ACM Intl.Conf. of Computer-Aided Design and Computer Aided Verification.

Hillel Miller received his M.Sc. in computer science from theTechnion—Israel’s institute of technology in 1992. Currently he isworking at Motorola Semiconductors as a verification CAD leader.He deals in the research and development of informal and formalverification techniqes.

Adnan Aziz received his Ph.D. from UC Berkeley in 1996; his under-graduate degree is from the Indian Institute of Technology at Kanpur.He joined the Department of Electrical and Computer Engineering atUT Austin as an assistant professor in 1996, where he currently con-ducts research on algorithms for automatic analysis and optimizationof digital integrated circuits.

automatic vector generation using constraints and biasing

Documents