Windows Logo Program System Fundamentals Testing

Paul ReedProgram Manager IIMicrosoft Corporation

Objectives

Define the major aspects of System Fundamentals testingDescribe the Windows Logo Program testing policy List the new System Fundamentals tests added for Windows 7Understand the purpose of the System Power and Performance Test Suite

Agenda

Introduction to System FundamentalsTesting policyWhat’s newSystem Power and Performance Test SuiteCall to actionResources

Windows 7 Logo for Systems

Systems with this logo ship with Windows 7 and may be upgraded to any edition of Windows 7All systems contain logo-qualified hardware components and have passed logo testsSystems with Starter Edition preinstalled carry the Starter logo

Systems That Qualify for a Logo

Systems are composed of logo-qualified componentsUsing such systems enable you to:

Increase system reliability and stabilityImprove system compatibility with applications and peripheral hardwareLower support costs

Microsoft-signed device drivers are available on Windows Update

Compete for bids with governments and enterprises

System Form Factors for Windows 7

Desktop PCA uniprocessor or multiprocessor system that requires AC power for continuous operation; a desktop system does not include attached peripherals, such as mouse devices, monitors, keyboards, or printers

All-in-One PC A uniprocessor or multiprocessor system that has a permanently attached integrated display; this is a fixed form factor desktop system that typically includes integrated components based out of mobile hardware designs

Mobile PCA uniprocessor or multiprocessor system that can be physically carried, has integrated display and input devices, and has a battery that can provide continuous operation when not plugged in

Ultra Mobile PC A mobile system that has a 10.2-inch or smaller screen size

Requirements

For compatibility, reliability, and performance

Logo Requirement Bucket Number Example

x64 Related 16 x64-capable systems contain devices with drivers that support the x64 architecture

Core System and Drivers 13 Systems support S3 and S5 states

Device and Driver 28 System with HD Audio codecs follows applicable Plug and Play guidelines

Firmware 10 Systems with Trusted Platform Modules use TPM v1.2 or later

System Configuration 7 Windows 7 systems include necessary devices

Viewing Requirements

Requirements on LogoPoint

Viewing Requirements

Prepared Requirements Report

Key Functionality

Signature testingSigned Driver CheckEmbedded Signature Check

System fundamentalsSleep Stress with IOEnable/Disable with IOCommon Scenario System Partition ValidationUnreported Memory and IOS3 Resume VerificationSMBIOS

Key Functionality (cont.)

Integration testing of core functionalityAudio

Examples: Fidelity Test, Wave Test, KS Position Test

VideoExamples: Glitchfree WMV HD 720p Video Playback, CRASH – Display Stress, Graphics HDMI System Test

USBExamples: USB Self Powered Hub, Host Controller Compliance

Before You Start…

Get the latest filtersSave you time and hair when you have failuresCan do this anytime, even after testing has started

Update the latest QFEsCan’t do this in the middle of testingShould be filters for the latest issues anyway, but this may improve your experience or fix tests that can’t be filtered

Make sure all devices and drivers are logo-qualified

Before you Start… (cont.)

Make sure you have your latest shipping BIOSKnow what your solution is for the fidelity testing

Test, import, testing house

Preparing for Testing

Minimum three-system setup requiredController (can be used as Studio as well)System under test (SUT)Audio Precision (AP) host machine

Needs to be in the same machine pool as SUTUsed for communication from AP machine (separate testing machine) and controller or system; must have PCI slotInvokes user interface for importing and exporting audio test logs

Audio Precision 2700 Series Audio Analyzer (optional)

Preparing for Testing (cont.)

Dynamic device detection (Jack Presence Detect)

For Windows 7 to see all components of your audio solution, populate all jacks

Two USB 2.0 devicesTV-Tuner test requires supported signal feedDrive partitions – test what you ship

Recommended Testing Order

Signed driver check

Must use signed driversDon’t know PCI Vendor ID of MB, guess, allow to fail, test will provide this information

SMBIOS – Makes sure BIOS is compliant HAL

Logo S3 resume

May need to update BIOS or driversPCI Compliance

System common scenario, sleep stress, enable/disable

All long tests, run overnight

Video suite Audio suite Everything else

Testing Policy

Windows 7 testing should occur on Windows 7 Home Premium Edition or laterBest to test on Windows 7 UltimateTesting is not supported on Windows 7 Starter Edition

Systems – Previously Logo'd Motherboard

Time saver for system buildersMust use a motherboard and BIOS that have qualified for the logo

Information is available on Windows Hardware Compatibility List (HCL)

Use to qualify for Windows 7 logo Motherboard must be listed on HCLMotherboard cannot be altered

New Client Tests

Two new tests addedDisplay Devices – Verify Digital ConnectorSystem Partition Validation

Updated testsWindows Touch Logo Test

New categoryGPU Family Update

Program Objective

For Software(ISVs)

For Hardware and Devices

(IHVs)

For Systems(OEMs)

Hardware Logo

System Logo

Improving Overall

Experience

Improving Basic Compatibility

Portion of Market Portion of Market Portion of Market

FundamentalsQuality

Guidelinesand

Test Suite

FundamentalsQuality

Guidelinesand

Test Suite

Fundamentals Quality Guidelines

and

Test Suite

Software Logo

Windows Quality Programs

System Power and Performance Test Suite

Microsoft runs the test suite across a broad set of systems in multiple labs

Redmond, Taiwan, and Japan

Battery Life

Performance

Reliability

Battery Life

Resource utilization at idleCPUTime in lowest exposed C-state and P-state

SettingsWireless adapter settingTimeout settings are Energy Star 4.0 – compliant

USB selective suspend issuesNo integrated non-root USB hubs

Performance

On/off transitionsBoot timeShutdown timeStandby and resume timeHibernate and resume time

Disk performanceDisk write cache enabledDisk is not in programmed input/output (PIO) mode

Graphics hardware performanceWinSAT video memory bandwidth test

Reliability

Leverages Windows Logo Kit (WLK) testsSuspend and resume failuresDriver verifier

Best Testing Practices – Testing

Perform tests in a controlled way (always critical)Always make sure you have the latest driver, application, operating system, and BIOS updates Complete the installation and setup of all applications so that no pop-ups occurRun an anti-virus (AV) definition update; force an AV scan of the hard disk driveComplete installation of any original equipment manufacturer (OEM) applets

Best Testing Practices – Development

Use the Windows Performance Toolkit to measure system performance during all on/off transitions Perform testing in a controlled way and make comparisons against a valid baseline

Obtain a baseline measurement on a system with as few system extensions as possibleAdd devices, applications, and services one at a time and test for unacceptable regressions in on/off transition times

Use the Windows Driver Foundation (WDF) framework for writing driversApply the latest firmware updates to all system components

Best Practices – Startup

Ensure all BOOT_START drivers are embedded-signed Avoids the performance cost of checking catalog files

Ensure all WDM drivers return STATUS_PENDING from initialization IRPs

Increase parallelization during PnP initialization

Avoid using load order groups to express service dependenciesEnsure that all services report as running as soon as possible during startup

Avoids blocking the Service Control Manager (SCM)

Ensure that all non-essential services are converted to Demand or Trigger start

Frees up system resources during startup

Reduce display adapter initialization times

Best Practices – Sleep and Hibernate

Ensure that all drivers implement fast resume and optimize CPU resource usage on resume

Achieves better resume performance

Avoid delays in processing the suspend notification (WM_POWERBROADCAST message)Verify that services do not opt in to receive suspend notifications unless absolutely requiredEnsure that all services respond quickly to resume events and minimize post-resume CPU, disk, and network usageOptimize resume from hibernate performance by enhancing BIOS INT13 performance

Best Practices – Shutdown

Ensure that all services do not opt in to receive pre-shutdown and shutdown notifications unless absolutely required

SERVICE_CONTROL_PRESHUTDOWN and SERVICE_CONTROL_SHUTDOWN control codes

Ensure that all services that have opted to receive shutdown notifications respond quickly to the SCMEnsure that all applications respond quickly to shutdown notifications

WM_QUERYENDSESSION and WM_ENDSESSION messages

Reduce delays in the shutdown path of services and applications by minimizing CPU, disk, and network activity in response to shutdown notifications

Best Practices – Energy Efficiency

Focus on idle energy efficiency and resource utilization before expanding to other scenarios

Idle on pre-install imageMedia playbackOffice productivity

Pay special attention to driver versions and settings

Some drivers that ship with Windows do not enable low power modes by defaultEngage with independent hardware vendors (IHV) to ensure that low power modes are enabled

Best Practices – Energy Efficiency

Run Windows 7 diagnostics prior to conducting rundown testing

“Powercfg /energy”

Leverage Windows 7 capabilities to enable Trigger Start Services for pre-install image

Summary

System Fundamentals validates key functionalityReviewed testing policyNew tests validate new logo program requirementsSystem Power and Performance Test Suite can be used to find other areas to enhance system performance

Call to Action

Run the new systems testRun the System Power and Performance Test SuiteUse the results to make improvementsImplement best practicesFill out the evaluation form for this presentation

Resources

Windows Logo Program https://www.microsoft.com/whdc/winlogo/default.mspx

Windows Logo Kit https://www.microsoft.com/whdc/winlogo/WLK/default.mspx

WLK 1.5 Download https://connect.microsoft.com/site/sitehome.aspx?SiteID=668

Windows Logo Program Webinars http://www.microsoft.com/whdc/winlogo/logocast.mspx

Microsoft Support http://support.microsoft.com/oas/default.aspx?&prid=11880

Resources (cont.)

Windows Performance ToolsXboot’mgr.exe

Automates on/off trace collection

Xperf.exeGenerates on/off transition summary reportsProvides raw text dumps for detailed, low-level analysis

Xperfview.exeProvides graphical visualizations of performance datahttp://www.microsoft.com/whdc/system/sysperf/perftools.mspx

On/Off Transition Performance Analysis of Windows Vistahttp://www.microsoft.com/whdc/system/sysperf/On-Off_Transition.mspx

Windows On/Off Transitions Solutions Guidehttp://www.microsoft.com/whdc/system/pnppwr/powermgmt/OnOffTrans.mspx

Resources (cont.)

Windows Application Quality Cookbookhttp://code.msdn.microsoft.com/Windows7AppQuality/Release/ProjectReleases.aspx?ReleaseId=1734

Energy Efficiency Portalhttp://www.microsoft.com/whdc/system/pnppwr/mobilepwr.mspx

Mobile Battery Solutions Guidehttp://www.microsoft.com/whdc/system/pnppwr/mobile_bat.mspx#

Developing Efficient Background Processes for Windowshttp://www.microsoft.com/whdc/system/pnppwr/powermgmt/BackgroundProcs.mspx

Webcast “Improving Energy Efficiency with Windows 7 Power Management”http://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=1032414345&EventCategory=4&culture=en-US&CountryCode=US

© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after

the date of this presentation.MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.