![Page 1: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/1.jpg)
Managing Highly Configurable
Design and Verification
Jeremy Ridgeway
Broadcom, Ltd.
![Page 2: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/2.jpg)
Agenda
• Root of the problem
• Automation required
– DUT
– Verification Planning
– Test bench
– Reporting Structures
• Conclusions
![Page 3: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/3.jpg)
Hardware Configurations
• One RTL set to target multiple customers
• Configuration due to modify a common capability
• Configuration due to add/remove a specific capability
DUT Customer A
Customer B
Customer C Cap X
Cap Y: 1,2
Customer A
Customer B
Customer C
Customer D Automation Required!
![Page 4: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/4.jpg)
Common Architectural Theme
• Super-set thing
• Customer-specific thing Most important
DUT
Cap Y: 1,2 Customer B
![Page 5: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/5.jpg)
Agenda
• Root of the problem
• Automation required
– Single CONFIG structure
– DUT
– Verification Planning
– Test bench
– Reporting Structures
• Conclusions
![Page 6: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/6.jpg)
Configuration Matrix and Object
• Must match a configuration matrix agreed upon with the customer
– We chose Perl5 (could be any OOP language)
Configuration Values Customer Selection
CAP Y 1, 2 ?
CAP X Y / N ?
…
Cust_A
CAP_Y => [ 1, 2 ]
CAP_x => 1
…
![Page 7: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/7.jpg)
Config: Center of Verification Universe
DUT
CONFIG
The CONFIG object drives the DUT transformation
![Page 8: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/8.jpg)
Config: Center of Verification Universe
• The configuration matrix drives:
– DUT Architecture & Implementation
– CONFIG Structure & Value Selection
DUT
CONFIG
Configuration Values Customer Selection
CAP Y 1, 2 ?
CAP X Y / N ?
![Page 9: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/9.jpg)
Config: Center of Verification Universe
DUT
CONFIG TB
![Page 10: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/10.jpg)
Config: Center of Verification Universe
DUT
TB
FCOV
TPLAN
REGR
REGR Report
FCOV Report
Customer Verification
Status
RAL
CONFIG
![Page 11: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/11.jpg)
CONFIG Object Representation
CONFIG CUST_B CUST_A CUST_C
![Page 12: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/12.jpg)
Agenda
• Root of the problem
• Automation required
– Single CONFIG structure
– DUT
– Verification Planning
– Test bench
– Reporting Structures
• Conclusions
![Page 13: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/13.jpg)
DUT Guidelines
• NO RTL should be simulated that is NOT delivered to customer
• Minimize the ability for the customer to make mistakes
NO Superset Simulation / Verification
Remove `ifdefs whenever possible
DUT
CONFIG
![Page 14: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/14.jpg)
DUT Transformation
DUT
Cap X
Cap Y: 1,2
`ifdef CAP_Y_1
codeY1
`elsif CAP_Y_2
codeY2
`else // Y1&Y2
codeY12
`endif
`ifdef CAP_X
module_x x()
`else
// no x
`endif
CUST_B
Customer B
codeY12
// no x
SV Preprocessor
`ifdefs in this style are fairly
easy to preprocess in perl
Some compilers can do this too
![Page 15: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/15.jpg)
Agenda
• Root of the problem
• Automation required
– Single CONFIG structure
– DUT
– Verification Planning
– Test bench
– Reporting Structures
• Conclusions
![Page 16: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/16.jpg)
Testplan: A Live Document
• ALL testing scenarios uniquely identified: the superset case
Scenario ID Feature Subfeature Generate Expect
CAP_X.DATA.1 CAP X Data Good …stimulus… …verification…
ERR.CAP_X.1 CAP X Data Bad data … …
CAP_Y.MODE_1.1 CAP Y Mode 1 Select mode 1 … …
CAP_Y.MODE_1.2 CAP Y Mode 1 Set FW mode 1 … …
CAP_Y.MODE_2.1 CAP Y Mode 2 Select mode 2 … …
ERR.CAP_Y.1 CAP Y Bad Bad select … …
![Page 17: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/17.jpg)
Testplan: A Live Document
• Customer-specific scenarios are indicated in the testplan
Scenario ID CUST_A CUST_B CUST_C
CAP_X.DATA.1 Y Y
ERR.CAP_X.1 Y Y
CAP_Y.MODE_1.1 Y Y Y
CAP_Y.MODE_1.2 Y Y Y
CAP_Y.MODE_2.1 Y Y
ERR.CAP_Y.1 Y Y Y
DUT
Cap X
Cap Y: 1,2
![Page 18: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/18.jpg)
Testplan: A Live Document
• Transform the testplan with customer CONFIG object
• Result indicates components required for customer-specific testbench
Scenario ID CUST_A CUST_B CUST_C
CAP_X.DATA.1 Y Y
ERR.CAP_X.1 Y Y
CAP_Y.MODE_1.1 Y Y Y
CAP_Y.MODE_1.2 Y Y Y
CAP_Y.MODE_2.1 Y Y
ERR.CAP_Y.1 Y Y Y
CUST_B
Scenario ID CUST_B
CAP_Y.MODE_1.1 Y
CAP_Y.MODE_1.2 Y
CAP_Y.MODE_2.1 Y
ERR.CAP_Y.1 Y
![Page 19: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/19.jpg)
Agenda
• Root of the problem
• Automation required
– Single CONFIG structure
– DUT
– Verification Planning
– Test bench
– Reporting Structures
• Conclusions
![Page 20: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/20.jpg)
Testing Scenarios
• Directed tests
• Directed-random tests
• Constrained-random test bench
![Page 21: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/21.jpg)
Directed Testing
• Scenario ID not tested when tests not run
• Scenario ID verified when tests run & pass
• Scenario ID fails when tests run & fail
• No functional coverage required
• Regression report sufficient for verification status
• No scenarios outside testplan expected No Unknown
Scenario Coverage
Scenario ID Test Regr
CAP_Y.MODE_1.1 Test1 Y
CAP_Y.MODE_1.2 Test2 Y
CAP_Y.MODE_2.1 Test3
ERR.CAP_Y.1 Test4 not run
![Page 22: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/22.jpg)
Directed-Random Testing
• Scenario ID not tested when tests not run
• Scenario ID verified when tests run & pass
and functional coverage hit
• Scenario ID fails when tests run & fail
• Functional coverage required
• Regression report + functional coverage sufficient for verification status
Some Unknown
Scenario Coverage
Scenario ID Test Regr
CAP_Y.MODE_1.1 Test1 Y
CAP_Y.MODE_1.2 Test1 Y
CAP_Y.MODE_2.1 Test1
ERR.CAP_Y.1 Test2
![Page 23: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/23.jpg)
Constrained-Random Testbench
• Scenario ID not tested when
functional coverage not hit
• Scenario ID verified when
functional coverage hit & tests pass
• Scenario ID fails when
functional coverage hit & tests fail
• Functional coverage required and is focus
• Regression report + functional coverage required for verification status
Scenario ID Coverage Test Regr
CAP_Y.MODE_1.1 group: a Base Y
CAP_Y.MODE_1.2 group: b Base Y
CAP_Y.MODE_2.1 group: c Base
ERR.CAP_Y.1 group: d EnErrs
Unknown
Scenarios Likely
![Page 24: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/24.jpg)
Functional Coverage Model
• Developed independent of testbench
• Metrics per scenario indicated
in the Testplan
• Hierarchical spreadsheet format
employing configuration values
Scenario ID Coverage Test Regr
CAP_Y.MODE_1.1 group: a Base Y
CAP_Y.MODE_1.2 group: b Base Y
CAP_Y.MODE_2.1 group: c Base
ERR.CAP_Y.1 group: d EnErrs
![Page 25: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/25.jpg)
Functional Coverage XLSX
• Transform functional coverage model with CONFIG
DUT
Cap X
Cap Y: 1,2
Variable Range Description
C_CAP_X 0,1 Capability X support (>0)
C_CAP_Y 0,1,2 Capability Y support (>0) and range
M_CAP_Y 0,1,2 Capability runtime select
FW_SEL 0,1 Capability Y selected by firmware
SEL_INVALID 0,1 Capability Y selection failed
Scenario ID Subfeature
CAP_X.DATA.1 Good
ERR.CAP_X.1 Bad data
CAP_Y.MODE_1.1 Select mode 1
CAP_Y.MODE_1.2 Set FW mode 1
CAP_Y.MODE_2.1 Select mode 2
ERR.CAP_Y.1 Bad select
Testplan Superset
Coverpoint Superset
![Page 26: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/26.jpg)
covergroup a;
coverpoint M_CAP_Y { bins B0 = { 1 };}
coverpoint FW_SELECT { bins B0 = { 0 };}
coverpoint SEL_INVALID { bins B0 = { 0 };}
cross M_CAP_Y, FW_SELECT, SEL_INALID {
sel_mode_1: binsof(M_CAP_Y.B0) &&
binsof(FW_SELECT.B0) &&
binsof(SEL_INVALID.B0); }
endgroup
Covergroup for CAP_Y.MODE_1.1
Name a
Points M_CAP_Y FW_SEL SEL_INVALID C_CAP_Y
sel_mode_1 1 0 0 1
Scenario ID Subfeature Coverage
CAP_Y.MODE_1.1 Sel mode 1 group: a
CAP_Y.MODE_1.2 FW mode 1 group: b
CAP_Y.MODE_2.1 Sel mode 2 group: c
ERR.CAP_Y.1 Bad sel group: d
sel_mode_1 is valid for CUST_B
![Page 27: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/27.jpg)
Configuration Filtering: Valid
• C_CAP_Y is defined in CUST_B CONFIG object as: CAP_Y: 1,2
Variable Range Description
C_CAP_Y 0,1,2 Capability Y support (>0) and range
CUST_B
Variable Range
C_CAP_Y 1,2
Name a
Points M_CAP_Y FW_SEL SEL_INVALID C_CAP_Y
sel_mode_1 1 0 0 1
Pass through filter
Preprocess
Cross Valid!
![Page 28: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/28.jpg)
covergroup a;
coverpoint M_CAP_Y { bins B0 = { 1 };}
coverpoint FW_SELECT { bins B0 = { 0 };}
coverpoint SEL_INVALID { bins B0 = { 0 };}
cross M_CAP_Y, FW_SELECT, SEL_INALID {
sel_mode_1: binsof(M_CAP_Y.B0) &&
binsof(FW_SELECT.B0) &&
binsof(SEL_INVALID.B0); }
endgroup
Covergroup for CAP_Y.MODE_1.1
Name a
Points M_CAP_Y FW_SEL SEL_INVALID C_CAP_Y
sel_mode_1 1 0 0 1
Scenario ID Subfeature Coverage
CAP_Y.MODE_1.1 Sel mode 1 group: a
CAP_Y.MODE_1.2 FW mode 1 group: b
CAP_Y.MODE_2.1 Sel mode 2 group: c
ERR.CAP_Y.1 Bad sel group: d
![Page 29: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/29.jpg)
covergroup a;
coverpoint M_CAP_Y { bins B0 = { 1 };}
coverpoint FW_SELECT { bins B0 = { 0 };}
coverpoint SEL_INVALID { bins B0 = { 0 };}
cross M_CAP_Y, FW_SELECT, SEL_INALID {
sel_mode_1: binsof(M_CAP_Y.B0) &&
binsof(FW_SELECT.B0) &&
binsof(SEL_INVALID.B0); }
endgroup
Covergroup for CAP_Y.MODE_1.1
Name a
Points M_CAP_Y FW_SEL SEL_INVALID C_CAP_Y
sel_mode_1 1 0 0 1
Scenario ID Subfeature Coverage
CAP_Y.MODE_1.1 Sel mode 1 group: a
CAP_Y.MODE_1.2 FW mode 1 group: b
CAP_Y.MODE_2.1 Sel mode 2 group: c
ERR.CAP_Y.1 Bad sel group: d Cross sel_mode_1 actually
covers the scenario
![Page 30: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/30.jpg)
Configuration Filtering: Invalid
• C_CAP_X is not part of CUST_B design, CUST_B CONFIG: CAP_X: 0
Variable Range Description
C_CAP_X 0,1 Capability X support (>0)
CUST_B
Variable Range
C_CAP_0 0
Name CapXData
Points M_CAP_X … C_CAP_X
good_data 1 … 1
Preprocess
Cross Invalid!
Covergroup not applicable
![Page 31: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/31.jpg)
Covergroup for CAP_Y
Name a
Points M_CAP_Y FW_SEL SEL_INVALID C_CAP_Y
sel_mode_1 1 0 0 1
fw_mode_1 1 1 0 1
sel_mode_2 2 0 0 1
sel_err * 1 1
Empty cells in a row indicate no dependency
Wildcard indicated all values are valid
sel_err0: FW_SEL==1 && SEL_INVALID==1
sel_err1: FW_SEL==0 && SEL_INVALID==1
![Page 32: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/32.jpg)
Covergroup for CAP_Y
Name a
Points M_CAP_Y FW_SEL SEL_INVALID C_CAP_Y
sel_mode_1 1 0 0 1
fw_mode_1 1 1 0 1
sel_mode_2 2 0 0 1
sel_err * 1 1
Scenario ID Subfeature Coverage Test Regr
CAP_Y.MODE_1.1 Sel mode 1 cross: tb::a.sel_mode_1 Base Y
CAP_Y.MODE_1.2 FW mode 1 cross: tb::a.fw_mode_1 Base Y
CAP_Y.MODE_2.1 Sel mode 2 cross: tb::a.sel_mode_2 Base Y
ERR.CAP_Y.1 Bad sel cross: tb::a.sel_err EnErrs Y
Covered when both sel_err crosses are hit
![Page 33: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/33.jpg)
Agenda
• Root of the problem
• Automation required
– Single CONFIG structure
– DUT
– Verification Planning
– Test bench
– Reporting Structures
• Conclusions
![Page 34: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/34.jpg)
Regression Report
• In this regression, only Base test is run
– Base test may be run 1000s of times
• Assume Base test passes 965 / 1000 times:
Scenario ID Subfeature Coverage Test Regr
CAP_Y.MODE_1.1 Sel mode 1 cross: tb::a.sel_mode_1 Base Y
CAP_Y.MODE_1.2 FW mode 1 cross: tb::a.fw_mode_1 Base Y
CAP_Y.MODE_2.1 Sel mode 2 cross: tb::a.sel_mode_2 Base Y
ERR.CAP_Y.1 Bad sel cross: tb::a.sel_err EnErrs
Regression score = (pass_rate * covered_scenarios) / total_scenarios
Regression score = (96.50% * 3) / 4 = 72.375% Regression score = (96.50% * 3) / 4 = 72.375%
![Page 35: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/35.jpg)
Regression History
• Testplan is a live document:
– Number of scenarios should (only) increase
– Jumps will likely force the score down until regr scenarios catch up
t
Total Scenarios
Regr Scenarios
Regr Score
Assuming 100% passing all time
![Page 36: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/36.jpg)
Testplan Report
• Each scenario indicates pass/fail with simulation & coverage
Scenario ID/Metric Instances Rate Result
CAP_Y_MODE_1.1 fail fcov_met
1 Base 1000 96.50% fail
1 cross: tb::a.sel_mode_1 100% fcov_met
Scenario ID Subfeature Coverage Test Regr
CAP_Y.MODE_1.1 Sel mode 1 cross: tb::a.sel_mode_1 Base Y
CAP_Y.MODE_1.2 FW mode 1 cross: tb::a.fw_mode_1 Base Y
CAP_Y.MODE_2.1 Sel mode 2 cross: tb::a.sel_mode_2 Base Y
ERR.CAP_Y.1 Bad sel cross: tb::a.sel_err EnErrs
![Page 37: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/37.jpg)
Testplan Report
• Testplan score combines functional coverage with regression score
• Assume 96.50% pass rate and CAP_Y.MODE_1.2 not covered
Scenario ID Subfeature Coverage Test Regr
CAP_Y.MODE_1.1 Sel mode 1 cross: tb::a.sel_mode_1 Base Y
CAP_Y.MODE_1.2 FW mode 1 cross: tb::a.fw_mode_1 Base Y
CAP_Y.MODE_2.1 Sel mode 2 cross: tb::a.sel_mode_2 Base Y
ERR.CAP_Y.1 Bad sel cross: tb::a.sel_err EnErrs
Testplan score = Regression score * functional_coverage_score
Regression score = (pass_rate * covered_scenarios) / total_scenarios)
Testplan score = ((96.50% * 3) / 4) * (2 / 3) = 48.25% Testplan score = ((96.50% * 3) / 4) * (2 / 3) = 48.25%
![Page 38: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/38.jpg)
Testplan Score
• Only includes functional coverage metrics indicated in the testplan
– Functional coverage collected outside of testplan is ignored
– Can include VIP or other external functional coverage
• Only includes run tests indicated in the testplan
– Simulated tests run but not listed in the testplan is ignored
• Is this the best way to calculate testplan score?
– I don’t really know – let’s discuss
![Page 39: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/39.jpg)
Conclusions
• Holistic approach requires buy-in from whole team and management
• Our method can accurately pinpoint scenarios:
– Verified
– Failing
– Missing
• Our method can manage multiple active programs simultaneously
• We have employed this approach on two major projects
– >5 simultaneous customer programs
![Page 40: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/40.jpg)
Agenda
• Root of the problem
• Automation required
– Single CONFIG structure
– DUT
– Verification Planning
– Test bench
– Reporting Structures
• Conclusions
![Page 41: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/41.jpg)
THANK YOU!
Questions Please
![Page 42: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/42.jpg)
Extra Slides
![Page 43: Managing Highly Configurable Design and Verification...Hardware Configurations •One RTL set to target multiple customers •Configuration due to modify a common capability •Configuration](https://reader034.vdocuments.mx/reader034/viewer/2022043020/5f3c2de93f27ae30b0378c06/html5/thumbnails/43.jpg)
Testbench Structure
• Common components / objects
• Extend to customer-specific
• Standard directory Structure
– Common
– Extended to customer-specific
• Class Inheritance Tree
– Common
– Extended to customer-specific
• Target the general case – common should be most heavily weighted
common
tb
tests
top_env
seq_lib
customer_a
tb
tests
top_env
seq_lib