concept testing plans cognitive walkthrough is a non human subject evaluation method. wizard of oz...

Concept Testing Plans Cognitive Walkthrough is a non human subject evaluation method. Wizard of Oz requires a task as in think aloud. The main difference

is that Wizard of Oz does not require a working (functional) prototype. You the “wizard” supplant the functionality.

Coding scheme for think aloud should develop codes to quantify the information you want to know about the prototype.

Timing people directly on cognitive tasks such as interpretation is problematic. Better to examine how they interpret data. No one likes to be timed. If they know they’re being timed, they may also throw the test.

Proof read! A sure fire way to get low marks on your final submission is to leave lots of typographical, grammatical, punctuation, formatting, and other manuscript errors. If I cannot get past your spelling & grammar, I won’t understand your content!

Final Submission (1) The design brief – A statement of what you

designed The technical memo – A complete technical

description of the artefact at the level of detail required to realise the artefact

The design – Models, diagrams, simulations, … An evaluation of the design – A rigorous, truthful

and credible analysis that the artefact satisfies the design goals as stated in the design brief

Five page essay / critique on ambient intelligence & situated awareness

Final Submission (2) The technical memo consists of

Your design brief (e.g., project description, user needs, functional requirements, technical specifications)

Your concept evaluation plan Results and analysis of concept evaluation

Your technical specifications may consist of Parts list (e.g., RFID tag, RFID reader) Database design (data definition and data manipulation

statements, ER diagram) XML (XML, XSL/T, XSD) Information flows diagram Product models Any technical details not included or obviated in one of

your prototypes

Final Submission (3) Your video / animation Your poster

A1 Design process and “advertisement” for your

project Submit everything to DROPBOX

Managing New Product

Development

Andy Dong

New Product Development System

Driven byStrategy and

Planning

Quality

Leader

Sponsor Eng.

Ops

Mktg. Matls.

ProductTeam

Executed by a Team

2 3 4 51

Product DevelopmentProcess

Managed by Management

Functions and Resources

Timeline

Supported by Information Technology

Driven by Strategy and Planning

Strategy and Planning Process and Tools Identify opportunities (client/user needs)

Market research, including qualitative (e.g., implicit needs) Brainstorming

Evaluate and prioritize projects Project screening (strategic fit, technology feasibility, market

fit, cost, organizational capability) Product platform and family planning, product architecture

(modular vs. integral) Product portfolio management (extent of product change vs.

extent of process change) Allocate resources and plan timing Launch product development process with clear mission

statement

Strategy Screen for Fit with:Target market segmentsBrand prioritiesCompetitive positionProduct platform strategyCustomer lifecycle management

Product Portfolio Planning Integrates the Criteria from the Other Approaches in a Set of Product Screens

Candidate ProjectsAdvanced development BreakthroughExtensions or derivativesProduct improvements

Technical Feasibility Screen:Technology strategy fit? Technology exist?Technology work?Manufacturing process?Supplier capabilities?

Market Acceptance Screen for fit with:Target customer setBrand imageChannel priorities

Economic Return:Risk assessmentCapital requirementsEngineering resource requirementsProduct

PortfolioPlan

Portfolio Planning Must Also Comprehend and Balance Workload and Resource Availability

Engineering Manufacturing Sales/Marketing Capital

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

Need clear visibility toresource constraints

Phase 1Definition

Phase 2Development

Phase 3Maturity

Phase 4 Qualification

Phase 5Ramp-up

The Output Is a Product Portfolio Map That Shows Where the Company Intends to Invest Its R&D Money

Platform projects

Derivativeprojects

New CoreProduct

Next GenerationProduct

Addition toProduct Family

Derivativesand En-hancements

Product Change More Less

New Core Process

Next Generation Process

SingleDepartmentUpgrade

Incremental Upgrade

Pro

cess

Ch

ang

e

Wheelwright and Clark

More

Less

Breakthroughprojects

Steps to Developing An Aggregate Project Plan Define project types as breakthrough, platform, derivative, R&D or

partnered projects Identify existing projects and classify by project type Estimate the average time and resources needed for each project

type based on past experience Identify existing resource capacity Determine the desired mix of projects Estimate the number of projects that existing resources can

support Decide which specific projects to pursue Work to improve development capabilities

A Mission Statement Is One Output of the Planning Process Brief, one sentence, product description

focuses on key customer benefit of the product.

Key business goals, e.g., for time, cost, quality

Target market(s) Assumptions and constraints Stakeholders

NPD Are Organized by a “Stage-Gate” Process All describe fundamentally the same steps All entail review points between stages at

which management examines progress and may stop or redirect the project

Generally are the starting point for formalizing NPD processes

Phase 0:IdeaValidation

Phase 1:ConceptualDesign

Phase 2:Specification and design

Phase 3:Prototypeproductionand testing

Phase 4:Manufacturingramp-up

Gate 0:Launchtheproject

Gate 1:Approveprojectimplementation

Gate 2:Releasedesign

Gate 3:Beginvolumemanufacturing

Define Products Design Products and Processes

Typical Stage-Gate Model

Functions of Stage-Gate Model Achieving discipline and consistency Identifying and reducing risk Allocating resources Achieving “loose” control Empowering the team Resolving design problems Smoothly shifting from development to volume

manufacturing

Group Portfolio Team (GPT)

Roadmap Working Group

Product Team 1

Product Team 2

Definition Maturity Qualification

Continuously develops and updates product and technology roadmaps

1 2 3 4 5 6 7 8 9 10 11 12Q1 Q2 Q3 Q4

Definition Development Maturity

BusinessPortfolioTeam (BPT)

Development Ramp-up

Time-Based View of Product Development Roles

Mil

es

ton

e-B

as

ed

Ca

len

da

r-D

riv

en

Management reviews include both milestone-based phase reviews and periodic roadmap reviews

Gates Are Critical Review Points To make “go/kill” decisions To make prioritization decisions To serve as a quality control check in the

process To chart the path forward

Phase Reviews Address the Right Questions at the Right Time

Definition Development Maturity Qualification Ramp-upGo/No-go

Go/No-go

Go/No-go

Go/No-go

• Validate requirements

• Specify the product and high-level design

• Fully plan the project

• Design and develop the product

• Develop the production process

• Develop the tests required to validate the product

• Mature and verify the product

• Implement the production process

• Prepare for market introduction

• Fully qualify the product

• Qualify the production process

• Begin market introduction

• Ramp-up production

• Achieve stable operations

• Capture project learningsP

has

e O

bje

ctiv

es

Phase Review: 1 5432

Are the requirementscorrect, and do wehave an integratedplan to deliver theproduct?

Is the design proven,and are we readyto start limitedproduction?

Does the productmeet requirements, and is it ready for customer qualification?

Do qualification results andmarket introductionpreparationsdemonstrate launchreadiness?

Have operationalstability and quality/reliability objectivesbeen achieved?

Key

Qu

est

ion

s

Executed by a Team

Cross-Functional Product Development Teams Are the Cornerstone of Successful NPD Organizations.

QualityAssurance

Team Leader

Production

SupplyChainEngineering

Marketing

Sponsor

CoreTeam

Own/drive the NPD project as a team

Ensure that the product is delivered on time, within budget, with agreed-upon features

Must be kept lean

Explicit Attention Is Given to Team Development

Timeline

Final delivery and celebration

Team Kick-off

Staff rating

Peer review Peer review

Team check-in

Celebrations Celebrations

Goals

Roles

Procedures

Relationships

Team Development

Compensation Is Tied to Team Performance

Rewarded for results, not merely phase or program completion

Rated as a team at end of each phase Conduct peer reviews against team

membership metrics Extended team bonus is voted by the

members of the core team.

Product Development

Process

Product Development Process:Generic

PlanningConcept

Development

System-Level

Design

DetailedDesign

TestingAnd

Refinement

ProductionRamp-up

2 3 4 51

Overlapped activities

Continuous interaction with users through rapid prototype iterations

Dynamic organization structures Development support tools

Uncertain Environments Require More Flexible Product Development Processes

Concept DevelopmentImplementation Implementation

Concept Development

Technical Possibilities

Customer Preferences

Prototyping Cycles

Product Development Process: Detailed View

IdentifyCustomer

needsEstablishTarget

SpecificationsGenerateProduct

ConceptsSelect

ProductConcepts

TestProduct

Concepts

SetFinal

Specifications

PlanDownstreamDevelopment


IdentifyCustomer



ConceptsSelect

ProductConcepts

TestProduct

Concepts

SetFinal

Specifications


Empathic designObservationInterviewsLead user researchCharacter creationScenario development

•Gather raw data from customers•Interpret the raw data as customer needs•Organize the needs into a hierarchy•Establish the relative importance of needs


IdentifyCustomer



ConceptsSelect

ProductConcepts

TestProduct

Concepts

SetFinal

Specifications


•Prepare list of metrics reflecting user needs•Collect competitive benchmarking information•Set ideal and marginally acceptable target values for each metric

•Must haves •Linear satisfiers•Delighters•Neutral


IdentifyCustomer



ConceptsSelect

ProductConcepts

TestProduct

Concepts

SetFinal

Specifications


•Clarify the problem•Search externally•Search internally•Explore systematically

Loosening up exercises Ball toss Alphabet listBrainstorming Affinity diagram Skits/improvisation“Tech” Box

A Tool: Theory of InventiveProblem Solving (TRIZ)


IdentifyCustomer



ConceptsSelect

ProductConcepts

TestProduct

Concepts

SetFinal

Specifications


•Prepare selection matrix•Rate the concepts•Rank the concepts•Combine and improve the concepts•Select one or more concepts

Benefit measurement modelsMarket research modelsEconomic modelsPortfolio management models

Tools: House of Quality or QFDPugh Concept Selection Method


IdentifyCustomer



ConceptsSelect

ProductConcepts

TestProduct

Concepts

SetFinal

Specifications


•Define purpose of concept test•Choose a survey population•Choose a survey format•Communicate the concept•Measure customer response•Interpret the results•Reflect on the process

Taguchi Design of ExperimentsCase: Team New Zealand Prototyping (small, full scale) Simulation Experience

Prototyping

Types of prototypes Physical (e.g., small-scale mock-up) vs. analytical

(e.g., equation, finite element model) Comprehensive vs. focused (e.g., looks-like,

functions-like) Prototype uses

Learning Communication Integration Milestones

Planning for Prototypes Define the purpose of the prototype Establish the level of approximation of the

prototype Outline an experimental plan Create a schedule for procurement,

construction and test

The Stages of Product Development IT Evolution

Intra-EnterpriseProduct Development

Engineering DataManagement

Extra-EnterpriseDesign/Supply Chain

Integration

CollaborativeProduct Commerce

Engineering IT Infrastructure

Enterprise Product Data Management

Design ChainIntegration

eDesign

50%

35%

10%

5%

PercentOf companiesAt each level

Accenture, Inc.

IT Infrastructure

Specialized

General

Specialty Tools (simulation accelerators, prototyping equipment, test

equipment, etc.)

Design and Modeling (CASE, MCAD, ECAD, etc.)

Engineering Productivity and Communication

(project management, PDM, virtual collaboration, etc.)

Engineering Business Applications: (finance/accounting, portfolio management, contract

management, HRM, etc.)

Foundation Infrastructure (network and computing infrastructure, integration,

messaging, security, etc.)

The key to this stage is controlling your IT infrastructure Consistent

data management

Release processes

Accenture, Inc.

New Product Development System

Driven byStrategy and

Planning

Quality

Leader

Sponsor Eng.

Ops

Mktg. Matls.

ProductTeam

Executed by a Team

2 3 4 51

Product DevelopmentProcess


Functions and Resources

Timeline


Designing Should be fun but is also hard work. The best designers consistently come up

with ideas AND realise those ideas. “Genius is one percent inspiration, ninety-

nine percent perspiration.” Design is a process. Successful companies

coach and monitor the process (design) to stimulate innovation without squashing creativity to ensure consistent, repeatable and (economically) successful outcomes.

concept testing plans cognitive walkthrough is a non human subject evaluation method. wizard of oz...

Documents

product team

planning slide

design brief

product architecture

new product development

planning process

design models

awareness slide