W H I T E P A P E R

Considerations for Effective Implementation of Processor

Driven Testbenches

Jim KenneySystem-Level Engineering Division

Mentor Graphics Corporation

w w w. m e n t o r. c o m

Why Use a Processor-Driven Testbench?Leveraging the embedded core as a tool to inject bus cycles which read and write to IP registers is an attractiveadjunct to HDL-based testbenches. As an orthogonal approach to functional verification, processor-driven testscan expose bus interface and bus timing errors that HDL testbenches may miss. These tests are easy to write,highly portable, work well at all levels of abstraction, and can be run on live silicon.

As a supplement to manually developed tests, verification teams can adopt significant chunks of code from thefirmware team. Boot code, hardware diagnostics, and the RTOS hardware adaptation layer or board supportpackage are highly relevant to functional verification of the hardware design.

Highly PortableEmbedded code is the one true portable testbench. As long as the address map remains consistent throughout thehardware design process, a test written in C or assembly and compiled to run on the processor can be used at theblock, subsystem, chip, and system levels as well as run on the physical prototype. No other testbench languageor testbench tool can make this claim.

The magic to the portability of a processor-driven test is its independence of design detail. The test simply readsor writes data to a specified register or memory location. As long as this location remains fixed in the linearaddress space of the processor (address map), the test has no dependency on the actual transport mechanism.Figure 1 depicts this concept. A single test can accommodate to the following HW variations without changing asingle line of source:

2 Processor Driven Testbenches

Logic Simulator

Full FunctionalProcessor Model

SoftwareMemory Memory

Subsytem

IPBlock

IPBlock

IPBlock

Block under Test

Processor Driven Test

Figure 1: The same block-level test can be used throughout the verification process fromsimulation to live target. The test is immune to the inclusion of other blocks and changes tothe memory sub-system. Its primary dependency is a fixed location in the memory map.

Bus implementation: Bus cycles can be pin-level with detailed timing or high-level transactions. The processormodel dictates the level of detail inserted into each bus cycle.

Hardware description: Hardware can be modeled at the transaction-level, register-transfer level, gate level orimplemented in silicon. As long as the hardware description can deliver a read or write to the correct register inthe design, the test cares not about the level of detail simulated to accomplish the exchange.

Memory: The memory sub-system can be implemented in great detail (i.e. Flash and DDR2) or abstracted to asimple array. At the block level, the memory sub-system is typically absent. Co-simulation tools permit the userto declare memory space for program storage, stack, and memory references. At the sub-system level, detailedmemory models can be introduced without changing the block-level tests. Code must be added to configure theMMU, but once configured the block tests can be run unmodified. Mentor Graphics; Seamless® HW/SWintegration environment permits the user to easily declare and load an abstract memory array, then dynamicallydirect memory references to that array or the memories modeled in the logic simulator.

A test written in C or assembly and compiled to target can be run at vastly different speeds depending on howthe hardware platform has been implemented. Speeds range from real-time on a live target to ~ one clock/secondfor gate-level signoff simulation.

Processor Models Bridge Multiple Abstraction Levels

Much of the portability of processor-driven tests is owed to processor models of various abstraction levels andperformance. These models are an interface between the test and the target hardware. On the test side, all modelspresent a consistent view, that being the ability to run C or assembly compiled into executable form. This permits thesame test to be run on a range of processor models as well as the live target.

It's the internal detail and the interface presented to the hardware that separates these processor models. Thebreakdown is as follows:

Transaction-Level: Fast with a bus-transaction interface. Speeds range from the single millions to tens of millionsinstructions per second. Cache and pipelines are not modeled. In essence these are instruction-set simulators with atransaction-level interface to hardware. Strength is speed; downside is the need for a transaction-level hardwaredescription, which is typically not available without investing additional effort.

Cycle Accurate: Reasonable speed with a pin-level bus interface. Cycle accurate implies the cache and pipelines aremodeled. Clock for clock, these models should produce bus cycles identical to the silicon with pipeline pre-fetch andcache fill burst reads accurately modeled. Cycles may not occur with exact clock-edge timing, but activity betweenclock edges should be accurate. Speeds range from 5K to 500K instructions/second. Strength is the ability to drop intoan RTL description with no design modifications and cycle accuracy. Downside is that performance is several ordersof magnitude slower than real-time.

Signoff Accurate: Highly accurate but very slow. These models are typically derived directly from the RTL used tosynthesize the silicon, so they should be an exact match to the physical core. Strength is accuracy and downside issimulation speed of ~ one clock/second.

Live Target: The silicon itself. Accurate by definition and real-time execution speed. The main disadvantages are theneed for a physical prototype and lack of hardware debug visibility. Typically used late in the design cycle.

Processor Driven Testbenches 3

It's worth adding a note here about bus-functional models. While they insulate the user from bus complexities, theirtests have poor portability and can't be used with a live target. It's rare to find bus-functional models of differentabstraction with a common programmatic interface, so tests for BFMs are usually non-portable.

Leaps Live Targets in a Single BoundThe ability of processor-driven tests to span virtual simulation and physical prototype is a significant capability.Benefits include:

Power-up Tests are Fully Verified

Bringing up first silicon is always a tense process as there are many unknowns and everyone seems to bewatching. Pre-validating the tests to be run on first silicon eliminates a major unknown. Since processor testsspan the virtual and physical domains with no modification, the design bring up processes can be simulated anddebugged prior to tapeout. This eliminates the design and test errors that typically plague prototype bring-up andraises the confidence level that first silicon will boot with relative ease.

Test Coverage Metrics

The coverage level of live target tests are typically a guess at best. However, because the exact tests can be runin simulation and on silicon, virtual test metrics like code and assertion coverage can be used to grade thecompleteness of live target tests. Accurate coverage metrics bring a level of confidence that the physicalprototype had been sufficiently validated.

Isolate Prototype Failures with Simulation

Physical prototypes deliver blazing speed and great accuracy, but debug of logic errors is difficult due to limitedhardware visibility. Owed to their high degree of portability, a processor-driven test that is failing on the livetarget can be easily run in simulation where debug visibility and control are excellent. Errors that remainunresolved after two weeks of debug effort in the lab have been diagnosed and corrected within an hour of beingre-run in simulation.

Many Abstraction Levels - One DebuggerSince processor-driven tests can be applied over a wide range of verification disciplines, care should be taken toavoid a plethora of execution and debug environments. On the hardware side, advanced simulators can span thecontinuum from transaction-level SystemC to gate-level signoff. They present a common set of windows fordebugging these different hardware descriptions.

To debug the C or assembly code of the processor-driven test itself, a source-level software debugger is needed.Unlike a logic simulator GUI, source debuggers tend to be tightly associated with the processor model ratherthan a specific tool. For example, the source-level debugger for a transaction-level model may be distinct fromthat of the cycle-accurate model for the same embedded core. Since these models are written in C, they typicallyincorporate a debugger API to service debugger requests for processor state, register info and to accept stop, run,and step commands from the user.

A source-level debugger can also be used with a live target via a hardware probe connected to the JTAG port.Ideally, the same debugger would be used for processor-driven tests across all levels of abstraction. In practice,this is a bit difficult to achieve. Processor suppliers and EDA vendors tend to build the simulation models whileembedded-tools vendors support live-target debug. As mentioned earlier, the debugger tends to be model

4 Processor Driven Testbenches

dependant; if the models come from different sources there's a good chance the debuggers will vary. If a singlevendor offers both EDA and embedded software development tools, they're more likely to provide a singlesoftware debugger covering both the virtual and physical domains.

In contrast, sign-off models are typically compiled from the core's golden RTL, and as such, lack the internalmechanisms and API to support a source-level software debugger. Resolving test failures at the signoff leveltypically involves cross referencing static listings of the source, assembly, and symbol table with the dynamicvalues displayed in the wave window -- a tedious task at best and a far cry from the productivity realized with afull featured source-level debugger. Tool vendors are now beginning to address this issue and signoff modeldebuggers are available for a limited set of embedded cores.

While a single hardware debug environment exists for transaction to gate-level, it's more difficult to find aunified debug solution for the processor-driven test itself -- an odd occurrence given the source of the testremains constant yet the debug environment does not.

As processor-driven tests become an integral component of functional verification, it's reasonable to expect thelogic simulator GUI will be extended to encompass source-level debug. Figure 2 is one example of such asolution. The source view on the left contains the code running on the processor while the view at the lowerright is the HDL source-debug window. Both share an identical look and command set, making it painless for ahardware verification engineer to adapt to working with embedded code as a testbench.

Processor Driven Testbenches 5

Figure 2: Questa's integrated HW & SW debug. The processor-driven test source view on the leftshares a common look and feel with the HDL source view at lower right.

Corporate HeadquartersMentor Graphics Corporation8005 SW Boeckman RoadWilsonville, OR 97070-7777Phone: 503.685.7000Fax: 503.685.1204

Sales and Product InformationPhone: 800.547.3000

Silicon ValleyMentor Graphics Corporation1001 Ridder Park DriveSan Jose, California 95131 USAPhone: 408.436.1500Fax: 408.436.1501

North American Support CenterPhone: 800.547.4303

Europe Mentor GraphicsDeutschland GmbHArnulfstrasse 20180634 MunichGermanyPhone: +49.89.57096.0Fax: +49.89.57096.400

Pacific Rim Mentor Graphics (Taiwan)Room 1001, 10FInternational Trade BuildingNo. 333, Section 1, Keelung RoadTaipei, Taiwan, ROCPhone: 886.2.87252000Fax: 886.2.27576027

Japan Mentor Graphics Japan Co., Ltd.Gotenyama Hills7-35, Kita-Shinagawa 4-chomeShinagawa-Ku, Tokyo 140 JapanPhone: 81.3.5488.3033Fax: 81.3.5488.3004

For more information visit: www.mentor.comCopyright © 2007 Mentor Graphics Corporation. All Rights Reserved.This document contains information that is proprietary to Mentor Graphics Corporation and may be duplicated in whole or in part by the original recipient for internal business purposes only, provided that this entire notice appears in all copies. In accepting this document, the recipient agrees to make every reasonable effort to prevent the unauthorized use of this information. Seamless and Mentor Graphics areregistered trademarks of Mentor Graphics Corporation. All other trademarks mentioned in this document are trademarks of their respective owners.

MGC 06/07 TECH

ConclusionsProcessor-driven tests apply universally across all phases of functional verification. They are highly portable andexecute in both simulation and live-target environments. Mentor Graphics offers full-functional processormodels of various abstractions, which are the key to test portability. The work of firmware developers can beused to dramatically boost test coverage with minimal effort.

Questa encourages the use of processor driven tests with its integrated source-level debugger and support formulti-abstraction simulation and processor models. As embedded design complexity advances, processor-driventests will become critical to functional verification coverage. Mentor Graphics' continued investment in HW/SWsimulation technology insures that Questa is ready when you decide to make the move to processor-driventestbenches.