case notes part 1
TRANSCRIPT
WRITTEN AND VIDEO CASES
Cases provide an alternative to a project emphasis and offer a hands-on supplement to the Discussion Questions, Close-Ups, Bringing Research to Life, Snapshots, and Classic and Contemporary Readings. We enjoy case use and find that it enlivens evening and weekend courses as well as other course structures that have longer session duration. Also, many instructors use cases to augment or replace projects. Some instructors also find cases an excellent evaluation exercise, using them to replace multiple choice tests.
The ninth edition of Business Research Methods contains both written and video cases. Four new video cases, prepared by the text authors, appear on the video cassette. Eleven new written cases, as well as several smaller written cases from the work of other Irwin/McGraw-Hill authors: Bryant and Smith (Practical Data Analysis: Case Studies in Business Statistics); Dillon, Madden, and Firtle (Marketing Research in a Marketing Environment, third edition, Irwin, 1994); and, one case from Siegel (Practical Business Statistics, third edition, Irwin, 1997).
Video cases and cases with data sets are indicated with special icons in the text case section at the end of each chapter. The majority of the cases have data sets that add further integration to the learning objectives by connecting statistical tools to concept understanding.
Video Cases by Chapter Use
Video Titles 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Covering Kids with Health Care
X X X X X
Cummins Engines X X X
Data Development Corporation
X X X X
Endries Fasteners X X X
Envirosell, Inc.
Goodyear's Aquatred X X X X
John Deere and Company
X
KNSD San Diego X X X X X X
Lexus SC 430 X X X
Outboard Marine Corporation
X X X X
Pebble Beach Co. X X X X X X X
Starbucks, Bank One, and Visa Launch Starbucks Card Duetto Visa
X X X X
USTA: Come Out Swinging
X X X X X X X X X
Volkswagen's Beetle X X X X
1
Written Cases by Chapter Use
Written Titles 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
A GEM of a Study X X X X X
Agri Comp X X X
AIDS Rates for Females X
BBQ Products Crosses Over the Lines of Varied Tastes
X X X
Calling up attendance X X X X X
Campbell-Ewald Pumps Awareness into the American Heart Association
X X
Campbell-Ewald: R-E-S-P-E-C-T Spells Loyalty
X X X X X
Can Research Rescue the Red Cross
X X X
Can the Study Be Saved? X X
Covering Kids with Health Care
X X X
Donatos: Finding the New Pizza
X X X X X X X X
Healthy Lifestyles X
HeroBuilders.com X X X X X
HiTech Engineering X
Inquiring minds want to know--NOW!
X X X X X X X X X X X
Mastering Teacher Leadership
X X X X X X X X X X X X
Matching Wits with Jason on Sampling Theory
X
McDonald’s Tests Catfish Sandwich
X X X X
Medical Laboratories X
NCRCC: Teeing up a New Strategic Direction
X X X X X X X X X X X X X X
NetConversions Influences Kelley Blue Book
X X X X
Open Doors: Extending Hospitality to Travelers with Disabilities
X X X
Overdue Bills X X
Performance Evaluations X X X
Ramada Demonstrates its Personal Best
X X X X X X X X
Retailers Unhappy with Displays from Manufacturer
X X X X
Rubbergate X
State Farm: Dangerous Intersections
X X X X X X
Written Titles (cont) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Sturgel Division X X X
2
The Brazing Operation X X
The Catalyst for Women in Financial Services
X X X X X X X X
T-Shirt Designs X X X
USTA: Come Out Swinging
X X X X X X
Violence on TV X X
Waste Paper X X
Xerox Abuses X X
Yahoo!: Consumer Direct Marries Purchase Metrics to Banner Ads
X X X
X
Teaching Tips
The videos accompanying this text were selected for discussion of the concepts in Business Research Methods, 9th edition. Each video synopsis that follows highlights how that video might be used, as well as describing the video and identifying the principals speaking on the videos.
The videos offer examples from both consumer goods and services and industrial goods companies. With video lengths ranging from 10-17 minutes, each video is designed to be discussed within a one-class format.
If you have not previously used video cases in teaching Business Research Methods, the following ideas suggest ways you might use the accompanying video cases.
For an INDIVIDUAL EXERCISE OR ASSIGNMENT: Show the video during or outside of class, possibly through a campus cable network or at assigned
times during several days prior to the case discussion. Assign a series of background and analytical questions for which the student must prepare answers.
Questions may include: What are the background facts relating to the case?
How does the management-research question hierarchy apply to the video case? How are one or more distinct text concepts relevant to the case? How do the various concepts interrelate to each other within the case? Which concepts might be applied differently if the video-case firm were a different type of
organization or in a different stage of development or facing a different management problem?
During the subsequent class, select an individual to answer each question, with one or more additional students chosen to contribute further detail or analysis to the initial student's answer. Move progressively from case fact or definition questions to analytical questions that tie case
facts to the text concepts. Show the video during class.
Assign each student the responsibility for focusing on how one or more distinct text concepts are relevant to the case.
Assign each student a set of analytical questions that form the foundation of the subsequent discussion. You can assign more than one student to each concept.
Allow students approximately 10 minutes following the video to draft answers to the question set. Call on individual students to present their answers to the assigned questions, with additional
students called to develop a full understanding of an appropriate answer to each question.
3
For a TEAM EXERCISE: Divide the class into teams: Show the video outside of class, possibly through a campus cable network or at assigned times during
several days prior to the case discussion. Assign each team the responsibility for discussing a series of analytical questions.
You can assign each team background questions, as well as analytical questions You can assign each team the same or different questions.
Have each team select a spokesperson to present that team's analysis. Show the video during class.
Assign each team the responsibility for discussing a series of questions. Questions may include:
What are the background facts relating to the case? How does the management-research question hierarchy apply to the video case?
How are one or more distinct text concepts relevant to the case? How do the various concepts interrelate to each other within the case? Which concepts might be applied differently if the video-case firm were a different type of
organization or in a different stage of development or facing a different management problem?
You can assign each team the same or different questions. Have each team select a spokesperson to resent that team's analysis.
For a COMBINATION exercise:
Assign a series of background and analytical questions for which the student must prepare answers. Questions may include:
What are the background facts relating to the case? How does the management-research question hierarchy apply to the video case?
How are one or more distinct text concepts relevant to the case? How do the various concepts interrelate to each other within the case? Which concepts might be applied differently if the video-case firm were a different type of
organization or in a different stage of development or facing a different management problem? Show the video outside of class, possibly through a campus cable network or at assigned times during
several days prior to the case discussion. During the subsequent class session:
Divide the students into teams. Assign each team the responsibility of developing a full analysis of the questions.
Have each team select a particular individual to share the team's analysis for each question. Every student in the team should serve as team spokesperson for at least one question.
Conduct a class discussion of the questions subsequent to this team discussion Start the discussion of each question with a different team, with a spokesperson from each of
the other teams contributing further detail or an alternative analysis to the initial team's answer. Move progressively from case fact or definition questions to analytical questions that tie
case facts to the text concepts and each other.
4
Video Cases
Covering Kids with Health Care Duration: 16 minutes
How/When Use: To discuss the Management-Research Question Hierarchy To discuss multi-stage research design To discuss exploration as a critical research step To discuss sampling design
o Sampling frameo Screening of participants
To discuss how research findings were incorporated into marketing materials
Company Background
Robert Wood Johnson Foundation: This health care philanthropic organization, among is broader mission, seeks to improve the health and health care of all Americans by assuring that all Americans have access to quality health care at reasonable cost. It supports training, education, research (excluding biomedical research), and projects that demonstrate the effective delivery of health care services. Rather than paying for individual care, it concentrates on improving the effectiveness of health care systems and the conditions that promote better health.
Wirthlin Worldwide is a full-service research organization. Communication strategy development and consulting is one of its core competencies. It serves as a strategic partner to corporations and their agencies on the full process of advertising/public relations development and evaluation.
GMMB is a full-service advertising agency with special expertise in working with non-profit organizations.
Company URLs:Robert Wood Johnson Foundation: www.rwjf.orgWirthlin Worldwide: www.wirthlin.comGMMB: www.gmmb.comAdvertising Research Foundation (ARF): www.arfsite.org
Video Content and Discussion:
This video describes the research done (focus groups, survey, ad testing) to increase enrollment in the federal government’s SCHIP program. Managed at the state level, the State Children’s Health Insurance Program provides basic health coverage for the children of the nation’s working poor. Research by Wirthlin Worldwide needed to discover why families who were eligible for the assistance weren’t enrolling. An early, unexpected finding (that working families thought they weren’t eligible because they were working and earning money) changed the direction of the campaign. The findings were used by GMMB, Inc. to develop a major advertising and public relations initiative to increase enrollment which was quite successful. The initial wave of the campaign resulted in the enrollment of more than one million eligible families. New research was planned and might now be available to prove the continuing success of the multi-year campaign. The research and subsequent marketing campaign were sponsored by the Robert Wood Johnson Foundation.
Video Presenters: Stuart Schear, Senior Communications
Officer, Robert Wood Johnson Foundation David Richardson, President Wirthlin
Worldwide David Smith, Partner, GMMB Jean Statler, Senior VP, Wirthlin Worldwide Annie Burns, Senior Partner, GMMB Maury Giles, Senior Research Executive,
Wirthlin Worldwide
Video Content:Issues Video Content
What is the SCHIP Program and when was the law State Children’s Health Insurance Program
5
Issues Video Contentpassed that enabled the program? (SCHIP) was created by Title XXI of the
Federal Balanced Budget Act of 1997. It is jointly financed by the Federal and State governments and is administered by the States. Within broad Federal guidelines, each State determines the design of its program, eligibility groups, benefit packages, payment levels for coverage, and administrative and operating procedures. SCHIP provides a capped amount of funds to States on a matching basis for Federal fiscal years (FY) 1998 through 2007. Federal payments under title XXI to States are based on State expenditures under approved plans effective on or after October 1, 1997.
What is Robert Wood Johnson Foundation? Why was Robert Wood Johnson Foundation willing to sponsor the research?
RWJF is a health care philanthropic organization that seeks to improve the health and health care of all Americans by assuring that all Americans have access to quality health care at reasonable cost.
What is the management dilemma driving the research?
Many families eligible for SCHIP have not enrolled. At the time of this research, more than 11 million children were without health insurance.
What is the research question driving the research design?
How do we convince working families without health insurance that SCHIP is for them and that they should sign up for this coverage.
RWJF used an independent consultant to manage the RFP process. How was the proposal process conducted? What did the independent contractor add to the process?
RWJF supervises many grants related to health-care initiatives. An independent consultant gave this particular program a coordinator who was undistracted by other projects.
What type of research firm is Wirthlin Worldwide? Why were they chosen for the research?
Full service research firm. They had a previous relationship with GMMB, the agency hired to develop the communication program.
In what ways was GMMB, the advertising agency chosen to develop the marketing campaign, involved in the research?
They were an equal partner in planning the design, influencing the questionnaire used for the survey, and the copy testing.
While names and other contact information of eligible families were known to government officials, it couldn’t be accessed to develop a sample frame due to privacy laws. How did Wirthlin develop the sampling design?
Wirthlin used screening procedures to identify whether the household they were speaking with was eligible for SCHIP.
What were some of the original hypotheses about why families weren’t enrolling for SCHIP benefits?
That people weren’t enrolling because of the ‘welfare’ sigma associated with government health care initiatives.
That people weren’t enrolling because healthcare for their children wasn’t a priority
Poverty, welfare, charity…these are believed to be exceedingly sensitive issues to the working poor. How did Wirthlin desensitize these issues?
They used a laddering interview process that reached for the emotional drivers beneath parenting decision making
Why were focus groups used and what did they reveal?
They were used to understand the attitudes of the working poor about preventative health care, and to determine if their attitudes differed from those of other families without health care insurance.
Why was a phone survey chosen? It was the quickest way to reach the largest
6
Issues Video Contentnumbers of a national population of working poor.
Wirthlin needed to screen large numbers of families to determine which were likely eligible for SCHIP and therefore a member of the relevant population; phone screens were expedient.
What key findings were critical in the development of the subsequent marketing materials?
That 86% felt being a good parent meant providing healthcare insurance for their children. (disproved early hypothesis)
That families weren’t enrolling because they thought that such programs were not for them, but for families where the head of the household didn’t work or made far less money; they needed to know that an income of 35,000 or less qualified them for the coverage in most states.
Major barrier to enrollment was low awareness of the program.
What are the criteria on which the ARF David Ogilvy Award for Excellence in Advertising Research?
Each year, the Advertising Research Foundation honors research that has been shown to make an important contribution to creating, identifying and improving great advertising. The award is named after the legendary adman, David Ogilvy, because of his passion for the role of research in crafting great advertising. Research case studies submitted for consideration must demonstrate measurable success and describe research that shaped the communication strategy, the actual advertising, the evaluation and strengthening of the advertising, or guided the media exposure.
Key Research FactsAfter the advertising,
42% of eligible households in the test group were aware of the program 55% of SCHIP-eligible families knew about any options. 22% of SCHIP-eligible families knew about SCHIP. 70% to 600% increase in SCHIP hotline calls across all states. In test market: 6 of 10 saw the ads; 25% of those called the hotline 95% of parents calling learned about the program from advertising, not PR efforts. Ads and PR efforts reached 78,850,000 families. Ad with the broadest appeal was “hard choices” (pink hair)—shown in the video.
Other Web sites of interest:ARF/Ogilvy Award: http://www.arfsite.org/awards/ogilvy_intro.htmlCovering Kids: www.coveringkids.org
Additional information of Interest:In 2004, according to the Robert Wood Johnson Foundation website, “Nearly 44 million Americans, over 8 million of them children, go without health insurance. This is the single greatest barrier to obtaining timely, appropriate health care services.”
Key Graphics in Case:
7
Cummins Engines Duration: 14 minutes
How/When Use: To discuss how statistical quality control is used to enhance quality. To discuss data from a statistical quality control program can be used in other functional areas of business. To discuss longitudinal studies (Customer Council), and when and how they are used. To discuss how a communication web site can also be used for compiling information. To discuss how special events (Cummins Signature 600 Tour) can be used for competitive intelligence
gathering.
Company Background:
8
Cummins Engines is a Fortune 500 company founded in 1919 with its headquarters in Columbus, Indiana. It makes advanced, fuel-efficient diesel power systems and engine related components, and specializes in customized diesel engine production, shipping more than 1000 engines per day to customers and dealers on every continent and purchasing engines for use in every conceivable situation and climate. Cummins has a long history of innovation, from pole performance at the Indianapolis 500 to the first natural gas fueled engine to pass California's tough emissions regulations. Cummins operates four strategic business units: power generation, automotive, industrial, and filtration. With more than 25,000 people around the world and 5400 authorized Cummins distributors, Cummins has the world covered. The Signature 600 engine is the newest and most advanced diesel engine on the market. It is so powerful and smooth in operation that it captured the competitor's attention during the kick-off promotional event. Company URL: www.cummins.com Video Content and Discussion:
The Signature 600 engine is the newest and most advanced diesel engine on the market. It is so powerful and smooth in operation that it captured the competitor's attention during the kick-off promotional event. Parts of this video are extracted from a corporate video introduction to Cummins' quality control programs. The video describes how customer and supplier relationships, and the information shared within these relationships led to the Signature 600 engine.
Video presenters:
none
Video Content:
Issues Video contentWhat type of data is generated by Cummins statistical quality control program? And what does the collection of this information permit Cummins to do?
Cummins electronic and advanced statistical control programs allow every employee to track every engine during its manufacture.
These programs also: time the delivery of parts and
components to assembly, as needed, permit each plant to
operate on a zero defects precision standard,
operate without inventory, and manage a robotic manufacturing
process (such as engine painting). permit monitoring of critical 'clean
environment' measures of temperature and dust--which impact engine performance.
How is longitudinal study data different than cross-sectional study data?
Cummins formed a Customer Council as part of its alliance program to produce more customer-focused products. The council would generate trackable attitudes over time that could be matched to manufacturing innovations (Ideas are
the power behind Cummins),
9
Issues Video content product alterations (keeping Cummins
on the leading edge of advanced, fuel-efficient engines) , or
changes in employee training or motivations programs.
How might employee input influence the creation of a new powerful engine like the Signature 600?
Seasoned, skilled line employees were used during the design of the Signature 600 engine.
How could managers of the various strategic business units use tracking of web connections by customers and suppliers?
It's extensive web site offers customers and dealers: wiring diagrams, sensor locations, a glossary of technical terms, shop-talk tips for making engine
repairs, and a mechanism for locating the closest
supplier within Cummins' extensive customer service network
How can both Cummins and its competitors (like Caterpillar) use special promotional events, like the Signature 600 tour, to collect information?
Prospective Cummins customers had an opportunity to test-drive the engine during the tour. Interviewing drivers would allow Cummins to verify performance and spot potential problems within a wider group of users.
Cummins representatives ask questions and collected comments made by potential customers. These could be used in the preparation
of second generation promotional materials for dealers.
Cummins could use competitor comments and questions to identify potential advantages or disadvantages that the Signature 600 might face when it is available to the diesel purchasing market.
Competitors studied the engine and the customers' reactions to the test-drive and performance discussions. This could be used in their formation of a counter-attack strategy.
Data Development Corporation Duration: 11 minutes
How/When Use: To provide an overview of some of the issues facing the research industry. To discuss the pros and cons of various sampling techniques. To discuss various research design issues.
10
Company Background:Founded in 1960, this New York headquartered research firm is one of the United State's largest research firms with a reputation for quality custom, qualitative, and quantitative research. With five offices and a staff that averages more than 20 years of experience, DDC has completed more than 17,000 domestic and international studies. A leader in in-home and office personal interviewing, DDC WATS centers have 170 CATI (Computer-Assisted Telephone Interviewing) equipped stations. They offer a network of CAPI (Computer-Assisted Personal Interviewing) in more than 180 mall locations, Interactive Software (STORE) simulations of store shelving, buildings, etc. to develop and evaluate logos, signage, packaging, etc. DDC's Internet Survey Group offers web-based studies.
Company URL: www.datadc.com
Video Content & Discussion: Officers and project directors for Data Development Corp. describe themselves as a 'generalist' research firm. They specialize in marketing research, but they do all types of research using a variety of research methods. President Joe Goldstein emphasizes that research is 'mostly art and partly science'. The video touches on numerous issues as the DDC officers describe what types of research they do, and how and why they do these types of research.
Video Presenters: Several DDC officers and project directors contribute their ideas in the video:
David Rausch, Sr. V.P.
Chip Lister, Sr. V.P.
Morris Cohen, Sr. V.P Shoshana Shapiro, Sr. V.P
Joe Goldstein, President and Managing Director
Video ContentIssue Video Content
Why research is used by businesses. Business owner/manager loses objectivity the more involved they become with a project
Due to the high cost of new product decision, managers need to predict with accuracy what is likely to happen before it happens.
What makes good research? Information where error is low Information where the data is logical and
makes sense.Impact of technology on research methodology.
Ability to use hand held computers and computer software to replace hand sorting of 'cards' used to measure complex attitude measurements.
Problems plaguing the research industry.
Declining willingness to participate in any type of research
Competition for possible respondents time from telemarketers selling products and services
Research that attempts to evaluate very minor differences or proposed changes, that results in consumer confusion
Appropriate sampling methods. Measuring attitudes demands a longer time frame (15-20 minutes) and often leads to
11
Issue Video Contentpurposive sampling rather than probability sampling (e.g. mall intercept interviews)
Random-digit dialing is an appropriate method for phone interviewing to get a more scientific sample.
Research Design issues. Face to face interviews are best to measure attitudes, e.g. personal interviews in the home and mall intercepts are mentioned.
Phone interviews are appropriate when you don't need the respondent to respond to visual stimuli.
Type of research gaining favor. Customer satisfaction (determining whether your customer is happy with your offering and how happy they are).
Endries Fasteners Duration: 6 minutes
How/When Use: To discuss difference between problem-focused and opportunity-focused research To discuss observation studies for generating information. To discuss process mapping as a research technique To discuss Electronic Data Interchange (EDI) as a research and communications technique To discuss data mining, the use of internally-generated secondary data in business decision making To discuss partnerships or alliances with customers or suppliers in data collection and information
management.
Company Background:Endries Fasteners, now Endries International is a wholesaler of parts to OEMs (original equipment manufactures) and repair service companies. Located in Brillion, Wisconsin, this company employs 200 people.
Company URL: www.endries.com
Video Content and Discussion: The president of Endries Fasteners and Supply, Inc. discusses the outcome of data collection, both internal and from its customers, which resulted in the development of new, profitable customer services that provide significant cost savings for its customers.
Video presenters:
Bob Endries, President
Video Content:
Issues Video contentWhat are 'soft costs' for a business or organization?
Soft costs include activities not related to the product, such as buying, storing, handling, quality inspection, and invoicing.
12
Issues Video content Several levels within a channel may
duplicate a soft cost-activity (e.g. quality inspection) that generates excessive and unnecessarily high soft costs.
Eliminating soft cost-activities or their duplication can yield significant savings.
Trends contributing to the discovery of opportunity
Increasing use of just-in-time manufacturing
Increasing concentration of purchasing among a one or small group of partner-suppliers.
Research on soft cost-activities that were duplicated or unnecessarily high.
Endries Fasteners 'followed' each part it supplied to its OEM or repair service customers.
What soft-cost activities were discovered as possible candidates for cost-saving programs?
Quality inspection of parts (quality audits) Invoicing
How EDI became part of the communication process
Endries now uses EDI with some of its customers.
What value added functions created savings for its customers
Reduced manufacturer inventories Manage procurement of parts Eliminate multiple invoicing, in some cases
reducing invoices to one Speeding communication between Endries
and its customers.
Envirosell Duration: 10 Minutes
How/When Use: To discuss observation studies. To discuss mechanical vs. human observation. To discuss alternative arenas for mechanical observation research (other than the marketing and operations
issues described here). To discuss alternative mechanical devices used for observation. To discuss ways of presenting data to clients. To discuss privacy issues involved with filming unsuspecting participants. To discuss the role technology has in observation studies (both generation of information and analysis of
that information).
Company Background:Founded in 1979 as Environmental Analysis & Planning Consultants, the firm changed its name in 1989 to Envirosell, Inc. Envirosell specializes in behavioral research, specifically in the retail environment. Envirosell's specialty is examining consumer-shopping behavior, and it has done this for Fortune 500 companies including banks, stores, restaurant chains, as well as consumer product companies. Envirosell has offices in New York, Milan (Italy), Sydney (Australia), and Sao Paulo (Brazil), with some thirty percent of total company revenue generated offshore. Paco Underhill, its founder, sees himself as a 'retail anthropologist'. Envirosell has sixteen full-time and thirty part-time staff members in its New York City headquarters. It films between 50,000 and
13
70,000 shoppers annually. While Envirosell is a for-profit-institution, Envirosellers regard themselves as consumer advocates and thrill to see the results of their work in stores, showrooms, restaurants, banks, urban streets, shopping malls and airports around the world. According to their web site, "Life is about casting shadows and we are proud of ours."
Company URL: www.envirosell.com
Video Content and Discussion: The managing director, research director, and senior analyst share information from several observational studies done in banks, as well as music, general merchandise, and other retail environments. Envirosell strives to understand what people buy and how to get them to buy more. Managing Director Underhill describes the process as an exercise in Zen: analysts watch video or time-lapsed still images repeatedly looking for clues that often emerge only in the eighth or tenth viewing.
Video Presenters: Several DDC officers and project directors contribute their ideas in the video:
Paco Underhill, Managing Director
Anne Marie Luthro, Senior Analyst
Barbara Weisfelt, Research Director
Video Content: The video touches on numerous issues as the Envirosell contributors describe the types of research they do or have done how these studies were conducted and reported to the client, and what they want to see as the result of their involvement.
Issue Video contentIn what scenarios is an observation study superior to a communication study for studying behavior?
Study participants remember and perceive their behavior differently than their actual behavior. This is not a result of intentional lying, but rather an inability to perceive reality.
The environments in which observation studies are conducted?
Video describes domestic and international studies and their results in banks, music, grocery, and general merchandise retail environments.
How Envirosell's studies are conducted?
Survey, time-lapsed fill cameras, video cameras.
What an Envirosell study might impact.
Overall floor plans for retailers can be affected.
Traffic patterns can be changed by moving fixtures
Aisle width can be adjusted Display locations are modified Merchandise can be move to a different
height on a shelving fixture Signage can change…in location and
contentHow observation information is presented to a client.
Envirosell uses time-involvement maps, where different colors on a map indicate the duration of time spent in any one location and arrows indicate direction of
14
Issue Video contentmovement.
Other Sources of Information:
Numerous articles have been written about research in the retail environment and Envirosell. These articles can be found in an archive on the Envirosell web site or via one of the numerous Internet search engines.
One article of particular interest for students might be:Labich, Kenneth. "Attention Shoppers: This Man is Watching You" Fortune, July 19, 1999.
For retail managers interested in learning more about this type of research, Paco Underhill has a book on the market, Why We Buy: The Science of Shopping. Simon & Schuster, 1999.
Goodyear's Aquatred Duration: 14 minutes
How/When Use: To discuss the coordination of a series of research projects involved in any large-scale management project. To discuss the strengths of various types of research designs to answer the various management and research
questions. To discuss the differences in sample design in researching the two different relevant populations of interest:
customers and Aquatred dealers. To discuss different ways to gain competitive intelligence. To discuss the management-research question hierarchy.
Company Background:Goodyear Tire and Rubber Co. was established in 1898 in Akron, Ohio. From its start manufacturing bicycle and carriage tires, Goodyear has become on of the world's largest corporations. Goodyear operates in the US and 30 other countries, with more than 28,000 investors and 105,000 associates. With more than $13 Billion in 1999 sales, their mission for the new millennium is to be the best tire and rubber company in the world and the uncontested leader in innovation. In 1993, the Aquatred tire, winner of more than a dozen awards, including Japan's prestigious "Good Product Design Award," reached two million units in the United States. This revolutionary tire pumps away over two gallons of water per second as you drive at highway speeds. And a new tread rubber compound provides road-hugging traction and extends the treadlife. By 1994, the highly popular Aquatred tire line had expanded with the introduction of a new aquachannel tire, the Wrangler Aquatred for light trucks and multi-purpose vehicles. Driven by Aquatred's success, all-time record sales for 1994 were $12.3 billion, with record income of $567 million. By 1997, with sales of $13.2 billion and earnings of $611 million, Goodyear completed its 97-year history once more in record territory. Fiscal 2000 sales were 1$14.4 billion with earnings of $40 million; 2001 sales were $14.2 billion, with a loss of $206.4 million. According to the 2001 Annual Report, “The Goodyear Aquatred tire, the most successful new product in the history of the tire industry, is a prime example of [a market driven strategy]. We created the wet-traction tire category and have led it for a decade.”
Company URL: www.goodyear.com Video Content and Discussion: This video profiles the genesis of the Goodyear Aquatred tire.
Video presenters:
15
Marco Molinari VP Sales & Marketing
John MontgomeryDirector of Marketing Communications
Video Content:
Issues Video contentWhat problem or opportunity drove the research for Goodyear?
Goodyear regularly tracked its customer satisfaction.
The 1992 study revealed a troubling development: a commodity mentality among tire buyer, most of who thought 'a tire is a tire'.
What research revealed the wet-traction market segment within the tire aftermarket?
Goodyear does on-going consumer tracking studies as part of its monitoring of the market. Goodyear also did focus groups to understand tire buyers needs.
Goodyear was in the midst of a consumer study to understand the criteria tire buyers use to choose tires when it discovered the commodity mentality dilemma. Its results showed the following criteria: tread life (10) and wet traction (6.2) were the most powerful criteria, followed by handling (1.8), snow traction (1.7) and dry traction (.7).
Given the Goodyear consumer study findings on tire buying criteria, what measurement strategy was employed?
Good year was using a rating scale, or a composite. A ten represented the highest value possible and the number on the bar chart was an average of the rating of the sample.
What was competition doing? Goodyear's competitive analysis indicated they were executing a high-mileage strategy (going after the treadlife segment).
2 new tires with more than 80,000 miles treadlife were about to be introduced.
What strategy did Goodyear pursue? Goodyear decided to pursue a 'wet traction to drive a safety response' strategy.
Goodyear wanted to 'position Goodyear as the undisputed leader in web traction, and hence the perceived leader in safety."
What research was done to hone in on this strategy?
Focus groups 1-on-1 personal depth interviews nationwide customer surveys
What decisions were made based on the research?
Deep aqua-channel design Name: Aquatred Decision to market to the tire aftermarket
rather than to the automobile company as an original equipment tire
Decision to price the tire at a 10% premium Decision not to discount the tire at its
16
Issues Video contentintroduction.
What additional information was collected via customer research?
Price elasticity Purchase intent Media habits
What information was known about the dealer given the promotional program undertaken to make them part of the Aquatred team?
The way they used point of sale (POS) displays and literature.
That persuasive selling was used with customers and that experiencing the tire on wet pavement was critical to such persuasion: (ride and drive events held in 80 U.S. cities)
What was the dealer structure? 600 franchised dealers 1047 company owned stores 4400 independent dealers
Where did Goodyear establish dealer contact?
At the annual industry trade show in Las Vegas.
What promotion decisions were made? Goodyear used print and broadcast directed publicity (earned coverage in numerous newspapers and magazines as well as Today and Good Morning America)
Goodyear used TV rather than magazine advertising, with the rest on cable or syndicated shows
Launch message: safety benefits of wet traction (achieve awareness, stimulate interest)
Follow-up message: Aquatred won numerous awards (precede from liking to preference)
2nd Follow-up approach: show product demonstration (from conviction to purchase)
Other Discussion Questions:
Issues Video contentApply the management -Research question hierarchy to the Aquatred scenario.
Management dilemma: why does the tire buyer see all tires as essentially comparable?
Management question: what can we do to distinguish Goodyear tires as superior? Will the tire buyer pay a premium for safety?
Research Question: What is the tire buyer looking for in a tire and why?
Investigative questions: what are the wants, fears, needs, driving habits, and purchase behaviors of the tire user.
17
Issues Video contentSampling Sample frame would have been readily
available for Goodyear's dealer network; earning participation would have been the challenge.
Given that Goodyear tires are original equipment on many new cars, an alliance with auto manufacturers could have generated an appropriate sample frame for the consumer study to gage the aftermarket.
John Deere and Company Duration: 12 minutes
How/When Use: To discuss environmental scanning as a critical business information component. To discuss plausible sources of information as part of an environmental scanning. To discuss management research hierarchy. To discuss plausible exploratory research to shed light on problems or opportunities identified during
environmental scanning. To discuss sampling strategy.
Company Background:John Deere has a rich 160-year history of serving the agricultural, construction, forestry and lawn care markets. It's emphasis on helping its customers achieve better productivity, has made it possible to successfully operate in more than 150 countries, in 50 currencies and every time zone of the world. During the 1970s, 1980s, and 1990s the company faced numerous challenges to its core businesses, yet its attention to environmental scanning and staying close to its customers permitted it to prosper when competitors have abandoned whole segments of the business for industrial equipment or gone out of business altogether.
Company URL: www.deere.com
Video Content and Discussion: The company's need for information is dependent on its volatile environment. The video traces the environmental influences on the company from the early 1970s through the late 1990s. The goal for John Deere was to double its sales from the mid-1990s by 2000.
Video presenters:
Mark Rostvold, Senior Vice President, C& CE Division
Video Content:
Issues Video contentThe environmental arenas that John Deere felt it necessary to monitor
Political-legal Competitive
18
Issues Video content Cultural-social Demographic
Trends contributing to strategic planning turmoil at John Deere
President Jimmy Carter impose an embargo on sale of grain products to Soviet Union or its allies (1979) Loss of sales to US farmers at a time
when they were highly leveraged Dramatic increase in farm bankruptcies
Government introduction of Payment-in-Kind subsidies, which paid farmers to lower their production by holding back acreage from cultivation.
Continued shift away from agriculture as an occupation.
Aging of the baby-boomers and their growing interest in gardening as a hobby.
Recession in the agriculture and construction industries in the early 1990s.
Aging equipment by late 1990s Rising farm incomes by late 1990s Low interest rates by late 1990s Increased acreage in cultivation brought
about by the 1996 Farm Act New environmental regulations in late
1990s dealing with Increasing soil erosion Engine exhaust emissions Water pollution by agricultural run-off
Increasing interest in and practice of 'precision farming' in the 1990s (the use of global positioning satellite technology to enhance grain yields by matching acreage productivity to seed and fertilizer practices).
Actions John Deere took in response to its environment
Expanded into financial services (John Deere Credit)
Engaged in extensive R & D to improve quality
Strengthened its dealer network Entered the lawn care industry. Became the supplier of equipment to the
Professional Golf Association tournament golf courses (to enhance its image in this market segment).
Teamed up with NASCAR, introducing the John Deere racing team, for more exposure.
Open its John Deere Healthcare program to non-employees.
19
KNSD San Diego Duration: 15 minutes
How/When Use: To discuss research design issues
Telephone survey Random digit dialing
Sample size
Sample Frame Survey length Extracting sensitive information Longitudinal studies Preliminary Data
Analysis Reporting Results
To discuss investigative questions and different response strategies feasible for measurement questions To discuss how syndicated research can be combined with custom research for better understanding of
respondents' motivations. To discuss how research information can enhance a 'product' for a company doing business with other
businesses. To discuss university-based research programs.
Company Background:NBC 7/39 got its start as an UHF station in 1965. It affiliated with NBC in 1977 and changed its call letters to KNSD in 1988. According to its web site, “In 1990, KNSD was awarded the Emmy for Outstanding News Station in San Diego. The station never lost this title, and has been awarded more Emmys this decade than all other stations in San Diego combined. On November 20, 1996, after a successful year as San Diego's most watched television station, KNSD was purchased by NBC. On January 1, 1997, it became known as NBC 7/39. NBC 7/39 is now owned and operated by the television network considered by many the world-leader in news, entertainment and sports programming.” In 2002, KNSD sees the following as its mission: “Our efforts will be focused on understanding and serving our community, benefiting the people of San Diego, attracting the largest possible audience, fulfilling the needs of our advertisers and increasing the profitability of our station.”
VALS, a research service of SRI (SRI Consulting Business Intelligence), is "one of the first major consumer segmentation systems based on lifestyle characteristics. Consumer products and services companies throughout the United States have used VALS to improve product development, product positioning, advertising effectiveness and corporate image." In 1989, SRI introduced VALS2, "a new segmentation system that incorporates recent lifestyle trends." Recently introduced, GeoVALS powers the VALS segmentation system with geocoded demographics.
Company URL: www.nbcsandiego.com www.sric-bi.com
Video Content and Discussion: The video describes the "KNow San Diego" research project, undertaken so that advertisers purchasing time on this NBC-owned station would have a better understanding of which programs attracted which potential target audiences. It describes the use of the VALs profiling developed based on a scaling system developed by SRI (Stanford Research Institute).
Video presenters:
Two KNSD anchors
Video Content:
20
Issues Video contentWhy was the KNSD project started? To learn more about the viewers of KNSC
programs, especially viewers of its award-winning news programs, so that advertising could be sold more effectively.
How was the study done? Telephone interviews were conducted with 1000 adults, 500 men and 500 women. Random Digit Dialing was used to select the household. The study is repeated annually to track changes in attitudes and behavior.
At what confidence level was the study conducted?
95%. (NOTE: The video explains in layman's terms what the confidence level means.)
What type of information was collected from respondents?
Participants were asked numerous questions about their TV viewing, entertainment, shopping, and other behaviors, as well as about their attitudes, likes and dislikes. Dozens of investigative questions are verbalized in the video. (NOTE: The actual measurement questions are on a 4-point scale of agreement)
How was the information used? VALS segmentation profiles were used to help advertisers select programs on which to advertise.
Dr. Pepper advertising is used as a case. An award-winning TV campaign which promoted peer acceptance (Dr. Pepper made a pepper out of me) didn't increase sales, sales actually decreased. After applying VALS, a new campaign repositioned the soft drink for non-conformists (Hold out for the unusual). Lost sales were recaptured and new customers were attracted.
Other information: http:// www.sric-bi.com/VALS/presurvey.shtml
Students can learn more about VALS and take a sample survey. The results are processed quickly and the student is assigned to one of the eight VALS segments. With the results they receive links to learn more about the segmentation system and their particular classification, as well as behavior and purchase patterns typical for that behavior.
Lexus SC 430 Duration: 8 minutes
How/When Use: To provide an example of the use of syndicated research To discuss the integration of qualitative and quantitative techniques.
21
To reveal a complex research design with many phases. To discuss how intercept personal interviews might be done.
Company Backgrounds:Team One Advertising: Team One Advertising is a full-service agency, including comprehensive relationship
marketing, event marketing/promotions, and interactive services. Headquartered in El Segundo, Calif., and a division of Saatchi & Saatchi, the agency has regional offices in New York, Chicago and Atlanta.
Lexus: Lexus, the luxury vehicle division of Toyota Motor Sales, U.S.A., Inc., markets and sells luxury cars and sport utility vehicles through 187 dealers in the United States. It is one of the fastest growing luxury nameplates in the industry.
Company URL: www.teamoneadv.com; www.lexus.com
Video Content and Discussion: This video case follows the research used to develop the newest Lexus, the SC 430, its hardtop convertible. From auto show interviews to Qual-Quant clinics and positioning analysis, students will learn about how Team One Advertising, Lexus’s U.S. agency, used research to position this latest entry into the crowded sport coupe category.
This research won Team One and Lexus the 2002 David Ogilvy Research Award, Durables Category.
Video Presenters::
Unspecified Executive at the auto show.
Video ContentIssues Video Content
What role did the Detroit Auto Show play in the research
It was the first observation study of the reaction of show attendees to the prototype SC430’s design.
Site of intercept personal interviews, getting reactions to the car
What motivates people to buy cars and how did Toyota discover this
Toyota’s Consolidated Dynamic Study CDS) revealed that two primary factors influence purchase: Rational (right) reasons and Emotional (real) reasons. Research revealed that Lexus did well on the rational reasons but not the emotional ones. Lexus SC 430 was specifically developed to be the emotional flagship for the Lexus brand repositioning.
What do the motivators affect? Toyota’s CDS revealed four key dynamics: brand dominance, model leadership, user needs and wants, and segment imagery.
What cars were primary competitors for the luxury coupe market
Corvette (leader), Jaguar XKE, Mercedes and Porsche.
How did Lexus use syndicated research?
Allison-Fischer International does several syndicated studies for the automobile industry.
Automotive Intensions & Purchases Study (quarterly, tracks demand, links brand equity to shopping and purchase
22
Issues Video Contentbehavior).
Automotive Shopping Study (records internet and dealer shopping activity and how retail process interacts with demand and sales performance; reveals the behavioral basis for automotive market segments
Advanced Automotive Features Study reveals purchase interest of auto features among new vehicle intenders; tracks interest in existing options; evaluates new features & technologies
RL Polk measures trends in the luxury and luxury coupe segment
How did Toyota use qualitative research?
They invited 154 people to Dallas to participate in several qualitative exercises, including
focus groups (done by Grieco Research Group)…to describe the typical driver in terms of financial security, affluence, professional success, and status consciousness
image sorts (done by Thompson Consulting)…to capture references that people had difficulty putting into words but that would influence the positioning related to the use of DESIRE.
How was the advertising that was ultimately created tested before it was aired?
Diagnostic Research tested the ads via “clutter reel” methodology. Lexus ad was buried in the middle position of other product ads and the audience dial tested their interest in the ad. Also, interviews were done with participants.
Car was seen to be sexy and seductive, alluring, sophisticated, and distinctively styled.
What did post-advertising tracking studies reveal?
That the car was distinctive looking, sporty, luxurious, had excellent acceleration and handling.
Did the web play a role in the research?
Yes, they tracked visitors to the Toyota and Lexus web sites over several months.
Visits increased 56% over the month immediately before the advertising.
More than 300,000 visitors to the SC 430 web site in several months after the ad campaign.
Was Sales tracking included in the research?
Yes, sales met or exceeded the monthly goal in each of the seven months following the campaign.
23
Additional Information:At time of press, the Lexus commercial campaign that resulted from this research was unavailable due to a pending law suit. Please watch the Business Research Methods web site; when this campaign becomes available we will make it available there.
Outboard Marine Corporation Duration: 12 minutes
How/When Use: To discuss data mining, secondary data. To discuss the management-research question hierarchy To discuss research design and sampling design. To discuss how the environment of research affects its design (industrial vs. consumer goods).
Company Background:The three Johnson brothers began building this world class outdoor recreational products company with the development of a two-cycle inboard marine engine in 1903. Meanwhile Ole Evinrude of Wisconsin was inventing the first successful vertical-crankshaft outboard motor. These two technology innovators would eventually merge to become Outboard Marine and Manufacturing Company in 1936, then shorten its name to Outboard Marine Corporation in 1956.
"Outboard Marine Corporation (Waukegan, IL) is a leading manufacturer and marketer of internationally-known boat brands, including Chris-Craft, Four Winns, Seaswirl, Javelin, Stratos, Lowe, Hydra-Sports, and Princecraft; marine accessories and marine engines, under the brand names of Johnson and Evinrude; and FICHT Ram Injection – the world’s premier low-emission two-stroke outboard engine technology."1 The company's history is rich with awards for its innovations. According to Soundings Trade Only, "The need to serve two different masters - government as well as consumers - has driven engine makers in recent years to develop the most innovative outboard technology the industry has seen in a generation." 1 For the nine months ended Sept. 30, 1999 OMC’s net sales increased 4.2 percent, to $850.2 million.1
The recreational products division of Bombardier International purchased the Evinrude and Johnson assets of OMC on March 12, 2001. Bombardier Inc., a diversified manufacturing and service company headquartered in Canada, is a “world leading manufacturer of business jets, regional aircraft, rail transportation equipment and motorized recreational products. It is also a provider of financial services and asset management. In 2002, the Corporation employed 56,000 people in 12 countries in North America, Europe and Asia, and more than 90% of its revenues are generated outside Canada.”2
1 www.omc-online.com Accessed June 2000.2 “Bombardier Finalizes Acquisition of OMC’s Engine Assets.” Press Release 3/14/2001
Company URL: www.bombardier.com
Video Content and Discussion: This video discusses the evolution of a product and the importance of keeping competition off balance by continually "attacking" your own best product and developing replacements that they competition cannot match or cannot as cost-effectively produce. The example used is the standard for bass fisherman, the 150hp Evinrude Intruder 150. No specific research is mentioned.
Video presenters:
24
Bob Shaughnessy, VP Sale and Marketing
Video Content:
Issues Video contentWhat type of data must OMC collect in order to execute its "attack your own best product" strategy? Which of that data can be internally data mined? Which must come from secondary sources? What must be newly collected?
OMC needs information about its customers: what they have been buying (data
mined) what competitive they consider when
comparing engines (data mined or secondary source exploration, or primary data collection)
The features and attributes they seek in their pleasure crafts and engines. (customer wishes and expectations need primary data--video indicates these are always changing)
OMC needs information about leading-edge technology affecting speed, performance, and other attributes (both secondary data searches and data mining their own product development logs)
OMC needs information about competitors' current products and competitive intelligence about product development (combination of secondary data searching and primary data via reverse engineering of competitors products)
Information about what competitors are advertising (secondary data searching).
Build the management-research question hierarchy. management dilemma 150 hp bass-fishing motor is nearing the
end of its life cycle; it is losing market share
management question 1. What should replace the current 150hp?2. What can be done to extend the life of the
current 150 hp motor?3. What can be done to reduce costs and
increase gross margins associated with producing the current 150 hp?
4. What celebrities would be appropriate to endorse the new 150 hp motor?
research question (1)
research question (2)
research question (3)
What features and benefits will attract today's bass fisherman?
What is attractive about the current features of the 150 hp that will attract water skiers, offshore fishers?
You could hypothesize about processes, materials, and supplier relationships here.
25
Issues Video content research question (4) Would Ken Cook or some other winning
boatman be appropriate for the new 150hp
Measurement and Investigative questions
Student could be asked to offer suggestions based on one of the above research questions.
Design an appropriate design for determining customer preferences and expectations.
Video features trade shows; this environment would allow OMC to cost-effectively reach boatman and/or dealers (two plausible relevant populations for research).
Information needed is motivational so communication is necessary at some level
How does the environment and industry affect research design?
This is a technology driven industry, so product life is relatively short, research design must make choices so that information is collected and shared quickly.
Dealers are mentioned as important. Research may be done more quickly and at a more detailed level if sample respondent is a dealer rather than a boat buyer.
Product is expensive, but boatmen have repeatedly been willing to pay for state-of-the-art. These characteristics may indicate that OMC could communicate with its relevant population in a high-tech environment: computer administered or Web surveying should be explored.
Design must not reveal OMC's plans to its competitors as product life is so short; this might lead to an in-house managed research process vs. outsourcing to a research firm.
Additional information: www.soundingstradeonly.com
Pebble Beach Co. Duration: 11 minutes
How/When Use: To discuss types of research needed within an organization's strategic planning. To discuss types of research needed to benchmark the achievement of the organization's vision. To discuss various issues involved with research design. To discuss sampling issues, including special problems sampling the very rich. To discuss the role that employee research could play at Pebble Beach Resorts in its achievement of its seven
core values.
26
Company Background:Samuel Morse, who acquired the extensive real estate in the Monterey Peninsula once held by Pacific Improvement Co. which he managed, founded Pebble Beach. Pebble Beach Company, a 5300 acre complex in Monterey (CA), offers three lodging options (Casa Palmero opened in September 1999, Inn at Spanish Bay opened in 1989 and the Lodge at Pebble Beach opened in 1919), four golf courses, plus a new 5-hole 'golf links', 8 restaurants, and an ocean-side Beach & Tennis Club. The tennis club’s newly remodeled state-of-the-art tennis facility offers ten hard surface and two clay courts and an extensive pro shop. The professional staff is available to arrange golf or tennis lessons and clinics for players of any caliber. The Spa at Pebble Beach, opened December 1999, is a "luxurious sanctuary designed to help guests relax, restore and rejuvenate." Pebble Beach has repeatedly won awards as American's best travel resort and is the host to the AT&T Pro-Amateur championship, the 1999 Amateur championship, and the U.S. Open in 2000. In January of 1999 The Inn at Spanish Bay was granted the coveted Mobil Five-Star award from the 1999 Mobil Travel Guide. Pebble Beach achieves its quality status by focusing on seven core values. Pebble Beach Company, which employs 1600 people, targets the upscale golf enthusiast.
Company URL: www.pebblebeach.com
Video Content and Discussion: Pebble Beach achieves its world-class standing by focusing on seven core values. The company is also
land-locked so it must develop ever-creative ways to make the facilities it has more intensively profit-generating. While research methodology is described only superficially, the video can be used to discuss the wide range of research that a corporation might need to maintain an award-winning reputation.
Video Presenters:
Richard Fowler, Banquet Asst. Mgr. Inn at Spanish Bay Several unnamed Pebble Beach employees contribute to this video.
Issue Video ContentWhat are the core values at Pebble Beach?
Service: "customers deserve our best effort" Teamwork Constant Improvement Fulfilled Employees Being a Good Neighbor Caring for the Environment Building Financial Value
What business practices are employed to execute its core values?
Service & Teamwork Mows golf courses at dawn to avoid
interrupting play Teamwork and Fulfilled Employees:
surveys employees Holds town meetings with
employees Extends golf privileges to
employees Recognizes employees with
service awards Being a good neighbor
Support employees volunteer
27
Issue Video Contentefforts at 64 charities
Donates land for community purposes
Finances the Pebble Beach Foundation
Build Financial Value Encourages local patronage of
its restaurants 2nd largest employer,
contributing $80 million through employee wages and bringing in $450 million to community through tourist expenditures.
Developed the Pebble Beach brand into a variety of merchandise offerings
Starbucks, Bank One, and Visa Launch the Duration: 10 minutesStarbucks Card Duetto Visa
How/When Use: To discuss multi-stage research designs To discuss the integration of qualitative and quantitative research To discuss the use of Web surveys. To discuss focus groups as an exploratory tool to refine a subsequent quantitative study To have the students generate measurement questions from the extensive list of investigative questions
presented.The Starbucks story is also featured in a written case on the CD that provides additional information, and is featured in a snapshot in Chapter 17.
Company Backgrounds:Starbucks: Starbucks Coffee Company is the leading retailer, roaster and brand of specialty coffee in the world,
with more than 8,700 retail locations in North America, Latin America, Europe, the Middle East and the Pacific Rim. In addition to its retail operations, the Company produces and sells bottled Frappuccino® coffee drinks, Starbucks DoubleShot™ coffee drink, and a line of superpremium ice creams through its joint venture partnerships.
Bank One: Bank One is part of the new JPMorgan Chase created on July 1, 2004 upon completion of the holding company merger between JPMorgan Chase & Co. and Bank One Corporation. The mergers of the broker/dealer, credit card companies, and lead banks will be completed over the next nine months. The Bank One brand continues to be used in the marketplace. It is the number one issuer of Visa cards in the world. JPMorgan Chase has assets of approximately $1.1 trillion and operations in more than 50 countries.
Visa: Visa is the world's leading payment brand and largest payment system, enabling banks to provide their consumer and business customers with a wide variety of payment alternatives. Cardholders in more than 150 countries carry more than 1 billion Visa-branded cards, accepted at millions of locations worldwide. Within the United States, nearly 14,000 financial institutions issue 396 million Visa cards, accounting for more than $1 trillion in annual transaction volume.
Company URLs: www.starbucks.com; www.bankone.com; www.visa.com
28
Video Content: In the very mature financial services industry, it is rare for a new financial product to garner much attention, let alone be named one of BusinessWeek’s outstanding products of the year. But what started as a way for Starbucks to add value to its existing Starbucks Card program developed into a financial product that many other institutions are interested in exploring. This case reveals the research that was done to develop this new payment option for Starbucks customers.
Video Presenters::
None
Video ContentIssues Video Content
What research led Starbucks to choose Visa?
Starbucks contacted several of financial institutions and credit card companies to determine their interest.
(Not in the video: Before the selection was made, they visited the Visa card operations of Bank One to determine if their operations were as customer focused as Starbuck’s own operations)
What investigative questions drove the research once the partners were chosen?
Would customers be confused by the dual function of the card
Would they feel the Duetto card was valuable?
(For the loyalty aspect of the card) What Monthly rewards would have the greatest appeal?
Would instant rewards from the card prompt customers to use the Duetto Visa rather than another credit card that also offered rewards (airlines, hotels, etc.)
What is the best way to deliver the rewards? What is the difference between a
stored-value card and the Duetto card?
A stored-value card…like a debit card…while the Duetto had both the stored-value feature and the credit card feature (2 functions on the same card)
How did Starbucks use qualitative research?
They used four focus groups comprised of current Starbucks customers; two groups of those who had used the stored-value Starbucks card and two groups of those who had not used the stored-value card.
The groups were used to determine how to explain the dual functionality in the quantitative study, to determine the appeal of different monthly awards being considered and how to deliver them; to determine if past negative credit-card experience would carry over to the Starbucks Duetto card; and to determine if the paring of the card with the
29
Issues Video ContentStarbucks foundation would be an important feature of the card.
How many quantitative studies were done before the Duetto was introduced
Two: a product functionality study and a product optimization study; both were done online.
Why do you think these quantitative studies were done online?
(not in the video: speed, easy availability of database of customer e-mails; in-store intercepts were considered but Starbucks and Bank One wanted to know the differences in geographic patterns)
How were quantitative studies used Product Functionality Study, among current users/non users of the Starbucks stored value card, was used to determine if the dual function was understood the impact on the brand the likelihood of future purchases if
person owned the Duetto Visa barriers to using the Duetto Visa perceptions of the different card attributes
Product Optimization Study, among current users/non users of the Starbucks stored value card, was used to Forecast sign-ups for the Duetto Visa Determine which Duetto card attributes
were most important Determine impact of the different card
features Identify ‘surprise and delight’ benefits Develop a profile of the likely Duetto Visa
applicant. Who was the launch announcement
used for research purposes? Announcement contained information about
how interested customers could get early e-mail notification of the card
Tens of thousands of customers requested early notification through the Starbucks web site.
What research followed the launch of the Duetto Visa
Major Brand Tracking study was done online to determine awareness of the card, understand how the card’s benefits were understood, determine intension to apply as well as those who had already applied, and why or why not.
Quarterly brand tracking studies evaluate the perception of the brand, number of cards issued, percent of card holders who use the card as their primary card, and the dollar value of purchases made with the card.
Bank One also tracks Ratio of approved accounts to applications
30
Issues Video Content Market cost of account acquisition Number of accounts with actual purchases Trends in monthly activity Patterns of spending on an account over
time. A Brand Loyalty Study measures who is
using the card, using one or both functions, and using the Duetto Visa to activate the auto reload feature of the stored-value function of the card.
Was the product introduction successful?
Yes, all partners said the card was meeting their expectations.
Special Content to Mention
Starbucks Card DuettoTM was selected by BusinessWeek as one of the outstanding products of 2003; it was the only financial product to receive that recognition.
The launch marketing program based on the research that is described in this video won a 2003 Silver SABRE award. (Silver SABREs are awarded for the best programs in specific industry sectors. Programs can involve work in any practice area, including marketing communications, public affairs, crisis or issues management, investor relations, or employee communications, for an organization in the relevant industry.)
The Starbucks Card DuettoTM Visa was named CardTrak’s Top Card of 2003.
31
U.S.T.A.: Come Out Swinging Duration: 11 minutes
How/When Use: To discuss telephone surveys To discuss two-stage surveys To discuss post-advertising testing To discuss ethnographic interviewing To discuss sample size and sampling issues
32
To discuss the management-research question hierarchyThis case also has a written counterpart that comes complete with survey instruments.
Company Background:The Taylor Research and Consulting Group: A privately-held company established in 1987 by Scott Taylor, it
provides qualitative and quantitative market and opinion research and consulting services to businesses in a variety of industries.
United States Tennis Association: Established in 1881, the USTA is the national governing body for the sport of tennis and the recognized leader in promoting and developing the growth of tennis on every level in the United States - from local communities to the crown jewel of the professional game, the US Open.
Vigilante: An urban advertising and marketing agency established in 1997, Vigilante crafts marketing communication programs relevant to the consumers of urban culture using both traditional and non-traditional channels including advertising, sales promotion, events, street and entertainment marketing, media planning, strategic planning and research.
Company URLs: www.usta.com; www.vigilantenyc.com; www.thetaylorgroup.com
Video Content and Discussion: The United States Tennis Association funded one of the most aggressive surveys ever undertaken about a
single sport in order to revitalize tennis in the minds of consumers. The survey results were supplemented with qualitative research by Vigilante, a specialist in urban communication campaigns. What resulted was a full-scale marketing initiative involving the establishment of Tennis Welcome Centers and the Come Out Swinging advertising, merchandising, and public relations campaigns. This case reveals the research and how the marketing initiative developed from it.
Video presenters:
none
Video Content:
Issues Video contentWhat is the basic research design? Two rounds of phone surveys were followed
by 30-40 street ethnography IDIs, and post advertising behavior tracking, which included Visits to tenniswelcomecenter.com Inquiries at tennis facilities Lesson sign-ups at tennis facilities Sales of rackets, balls, and apparel.
Why did the study include 25,000+ households
USTA needed a benchmark study that provided data in each of its 17 sections and that detailed information based on ethnicity, age,
What was the underlying management dilemma driving the research?
Tennis participation, while flat, suffered from the leaky-bucket syndrome: 5 million young players started each year, but 5 million young adult players lapsed each year.
Tennis is viewed as an elitist sport of
33
Issues Video contentwealthy Caucasian country-clubbers
What was the research question? How can we get people to see tennis as the well-rounded sport it is, one offering fitness, socialization and competition that appeals to all ethnic groups?
What were the major investigative questions driving the 5-minute random-dialed phone survey?
Who plays tennis…by demographic segment?
Why do people play? Why don’t they play? Who plays/doesn’t play tennis in the
household (among people over 6 years of age)?
What types of measurement questions were used in the shorter phone survey?
Pretested questions used on earlier surveys Open-ended questions to get people’s
perceptionsWhat were the various cells defined for the longer 10-15 minute phone survey among 2032 participants?
Three groups were defined: Current players Former players Never played
What types of measurement questions were used in the longer phone survey?
Pretested questions for comparison to earlier surveys
Closed questions…for greater ease in analysis
What were some of the findings? Minority participation was growing 5% of current players are African-
American; yet 10% of new players are 16% of current players are Hispanic-
American; yet 22% of new players are Unaware of Tennis facilities in their area Tennis is not perceived as a good fit
“Tennis is not for me” Culturally, ethnically, and financially
people didn’t think tennis fit themVigilante was charged with coming up with an ad campaign that could make tennis appear “cool, relevant, cutting edge” and accessible to former players. What research did they use to guide campaign development?
Their Street Spies methodology used street ethnography (30-40 IDIs) Video-taped interviews done as
intercepts Interviewers matched participants in
age, ethnicity and lifestyleDid the findings confirm or refute the quantitative study?
Confirmed “I can’t afford it.” “It’s too expensive.” “I’d rather play basketball or soccer or
video games.” “Don’t have a clue about how to start.”
Ethnography revealed the edginess that tennis offered to satisfied players that was the genesis of the “Come Out Swinging” tagline and “campaign rallying cry.”
Ethnography revealed the characteristics
34
Issues Video contentused for celebrity selection criteria: “no holds barred” attitude, physically fit.
What research went into selection of Vigilante as the ad agency?
Identified three agencies that had a good track record with sport related positioning ad campaigns.
Vigilante was one identified and they had a reputation for ethnically-based campaigns.
How was the quantitative research used to guide the campaign?
USTA provided Vigilante with all the results of the extensive telephone survey.
Survey revealed the four primary target audiences for the campaign: seniors, Caucasian men & women, ethnic players, youth players
Survey revealed the key communication messages for the campaign (fitness, socialization, competition)
Why wasn’t ad testing done before the print advertising campaign was used.
(Not in the video) Budget was too small to do both the quantitative study the section leaders wanted and extensive pre-campaign research.
Volkswagen's Beetle Duration: 16 minutes
How/When Use: To discuss how the 'problems' driving business change over time and how failure to recognize the change can
blind-side a business To discuss plausible exploratory research Volkswagen could have done to prevent disenfranchising its
customer group in the mid-70s. To discuss how qualitative research is conducted To discuss how qualitative research supports manufacturing and marketing decisions. To discuss the advantages and disadvantages of trade shows and other special events as research venues.
Company Background:Out of the rubble of World War II-torn Europe, the Beetle was originally introduced in the United States in 1949. Ferdinand Porsche designed the original Volkswagen Beetle while Ivan Hurst masterminded production. The Beetle became a symbol of the 1960's rebelliousness, but lost the love of a generation when it stressed engineering over style and low-cost operation, what the baby-boomers considered crucial in the 1970s. By 1974, the Beetle had lost ground to its aggressive Japanese rivals for the value segment of the US automobile market. And by 1979 you could no longer buy a Bug in the States. But the Beetle still had a franchise in the US, as one spokesperson comments: "Where ever it is introduced, the new Beetle garners the same reaction: people smile." In 1998 when the Beetle was reintroduced in the United States, it surpassed all sales estimates. The second year it doubled its sales. Historically, the Beetle is the world's best selling car, having sold in more countries than any other automobile, more than 21 million in its lifetime. 2002 model year was the fifth for the NewBeetle. In December 2002 Volkswagen introduced the first 180 horsepower version of the 1.8 T called the New Beetle Turbo S.
Company URL: www.vw.com
35
Video Content and Discussion: The video profiles the history of the original Beetle in the U.S. market from its introduction in 1949 to its demise in 1979, then follows the initial two years of the NEW Beetle's rebirth 1998-99. It is ideal for discussing qualitative research, as the advertising profiled is dependent on an in-depth understanding of the needs of today's small-car segment. Research is alluded to, but not described.
Video presenters:
Jack Dolby, TV Anchorman & Narrator
Maria Leonhauser, spokesperson for VW
Unspecified Man-on-the-street
Video Content:
Issues Video contentWhat management dilemmas should have led Volkswagen to research its U.S. market?
Emphasis on engineering, not style, when market was interested in more 'flash' or pizzazz.
Engineering was state-of-the-art but exterior styling stayed relatively constant; market couldn't value what it didn't know.
Oil embargo of the early 1970s forced gasoline prices up and generated long lines. By necessity, cars with exceptional gas mileage were favored over superior performance vehicles.
Japanese entered the market with more stylish and more cost efficient cars.
Is the reveal of the design of a new (or reborn) car the first research that Volkswagen was likely to have done?What type of research would be done at such an event?Is the attendee at such a show likely to be members of the relevant sample population?
The 'concept 1', what a 1990s Beetle would look, like earned enthusiastic reviews at auto shows around the world.
Beetle arrived with due pomp and circumstance, under spotlights, with the world's journalists snapping and filming the arrival.
Many concept cars are revealed at the major world auto shows.
What management dilemma/opportunity led to the reintroduction of the Beetle in the 1990s?
Japanese cars' prices had increased beyond the value-car segment's ability to buy.
European styling was once again gaining favor around the world.
What qualitative research could have revealed the major themes used in the advertising that re-introduced the Beetle?
One enthusiastic buyer relates the nostalgia that the car evokes, but also claims that everyone can see themselves in the Beetle because of its great engineering and great 'curves'.
Ads stress the power of the engineering--because
people thought the original Beetle was under-powered (Less flower. More
36
Issues Video contentpower).
Color (What color do you dream in?) Technology (Reverse engineered from
UFOs.) Attitude of Nostalgia (Engine's in
front, but the heart's in the same place.)
Other information from VW Press:
April 04, 2001VOLKSWAGEN IS AWARDED TWO BEST CAR PICKS FROM MONEY MAGAZINE
“…the Volkswagen New Beetle…earned a five-star safety rating, making it the only car in its class to achieve these outstanding results. With high safety standards, a unique style and excellent driving capabilities, it is not a surprise that the New Beetle was awarded Money’s Best Pick in the small car category.
After making its debut in 1998, the New Beetle has proven itself as much more than a sequel to its legendary namesake. It has won several distinguished automotive awards, including “North American Car of the Year,” as selected by the continent’s top automobile writers and Motor Trend’s Import Car of the year. The New Beetle has established itself as a totally modern creation, both functional and fun to drive.
Money Magazine describes the New Beetle is described as, “… pure passion on wheels … it offers more than just looks: The New Beetle is a terrific small car.”
Tuesday, November 02, 1999VOLKSWAGEN SALES UP 65.4 PERCENT-BEST OCTOBER IN 25 YEARS -BEST JETTA AND
PASSAT OCTOBER EVER-NEW BEETLE UP 21 PERCENT
Volkswagen’s popular New Beetle again posted healthy sales growth. In October, Volkswagen sold 7,390 New Beetles. This is a 21 percent increase over last year’s sales of 6,109. So far in 1999, New Beetles sales are 69,621, an increase of 65.2 percent over the same period in 1998 when sales were 42,146.
Wednesday, January 06, 1999AUTOMOBILE MAGAZINE ANNOUNCES ITS 1999 AUTOMOBILE OF THE YEAR, THE
VOLKSWAGEN NEW BEETLE, AT THE NORTH AMERICAN INTERNATIONAL AUTO SHOWAutomobile Magazine’s Editor and Publication Director David E. Davis, Jr. stated: "It gives us great pleasure to recognize the New Beetle for its character, its quality, its value, its impact on the global automotive industry, and the sheer joy it brings to anyone who drives it, sits in it, or sees it on the road. It is the most important automobile to debut in the 1999 model year and our kind of car!"
Tuesday, January 05, 1999VOLKSWAGEN RECORDS BEST SALES SINCE 1981-BEST DECEMBER IN 17 YEARS
“In 1998, Volkswagen sold 219,679 new cars in the U.S., up 59.3 percent over 1997. It was the best Volkswagen total in the U.S. since 1981 when the German automaker sold 278,513.”
Tuesday, October 12, 1999
37
THE 2000 VOLKSWAGEN BEETLESince its introduction, the New Beetle has garnered numerous distinguished automotive awards, including “North American Car of the Year,” as selected by the continent’s top automobile writers, Automobile Magazine’s Automobile of the Year, Motor Trend Magazine’s 1999 Import Car of the Year, J.D. Power and Associates’ Most Appealing Small Car, European Car Magazine’s Grand Prix 1998 winner, Consumers Digest’s Best Buy, MotorWeek’s Drivers’ Choice Award for Best of the Year, Time Magazine’s The Best of 1998 Design, Business Week’s Best New Products, African American on Wheels Urban Car of the Year, Popular Science’s Best of What’s New for 1998, and many others.
June 8, 1998VW'S NEW LOVE BUG STIRS BUYER PASSION: SUPPLIES SHORT, CONSUMERS PAY PREMIUMS, AND LOCAL CAR DEALERS ARE RIDING HIGH, Crain's Chicago Business,
Volkswagen of America Inc (Auburn Hills, MI) has shipped just 602 new Beetles to authorized dealers in Chicago since rolling out its reincarnation of the 1960s classic in late March. Customers pay up to $10,000 more than the model's $15,200 list price for the hottest color: yellow. Area VW showrooms are drawing double the usual number of customers -- and seeing total sales soar 40% to 50%. Meanwhile, the German automaker is overhauling other product lines, positioning its dealers to regain market share from their Japanese competitors for the first time in a generation. Customers' ardor hasn't been dampened by the German automaker's May recall of 10,100 Beetles to correct a wiring problem that could cause engine fires. Volkswagen is now poised to recapture some of the customer base it lost to Japanese brands after it pulled the plug on the original Bug in 1979. VW has redesigned the Passat and this fall will unveil an overhauled Golf compact that is already garnering high praise in Europe. Since rolling out the Beetle nationally in April, total U.S. sales for Volkswagen are up 42% over comparable 1997 totals.
38
Written Cases
Case: A GEM of a Study
Abstract: The Global Entrepreneurship Monitor Entrepreneurial Assessment, a joint project of The Kauffman Center for Entrepreneurial Leadership at Babson College and The London Business School, has undertaken a long-term, large-scale project to prove the causal links between a government's economic policies and initiatives, the resulting entrepreneurial activity and subsequent economic growth. This case describes multiple-stage research, including thousands of interviews in several countries by established research firms.
This case deals with the concept of causal studies vs. descriptive studies and what needs to be included in the research design of each study type. It also deals with what constitutes control in research design; in this context it is best used with chapters 4-9. This is also a great case to use to discuss constructs vs. concepts, as Exhibit C-GEM-1.1 clearly has struggled with defining numerous constructs and multiple-measurement variables and, therefore, could be used to further explore text Exhibit 2-3. The case also mentions the use of standardized data; in this context you could use the case to explore how "standardized" data from different countries really is and where the student seeks such country-specific data.
1. Using the GEM Conceptual Model, students should first identify that the dependent variable the study purports to measure is the construct of economic growth through GDP and jobs. But they might also note that the construct of business dynamics is also a dependent variable of interest, as it contains new firm/job creation, firm/job growth, firm/job dissolution, and firm/job shrinkage.
The independent variables in the study are numerous and contained within the GEM Conceptual Model as the constructs of general national framework conditions, entrepreneurial opportunities, entrepreneurial framework conditions, and entrepreneurial capacity. Each of these constructs contains numerous other concepts and constructs. Here is an excellent opportunity to discuss the nature of constructs and the importance of breaking down such complex entities in terms of more concrete and truly measurable elements as is done in Exhibit 2-3.
2. Many of the variables leading to the dependent variable could be seen as extraneous, intervening or moderating, as easily as they can be identified as independent. Extraneous variables are described in Chapter 2 as almost infinite in number and treated "as independent or moderating variables" and "assumed or excluded from the study." The GEM authors took great care in tracking as many of these variables as possible, and chose at the outset to use most as independent variables having some, hopefully measurable, influence of the chosen dependent variable(s). Many of the attitudinal variables in the model are treated as moderating variables--a class of "independent variables believed to have a significant contributory or contingent effect on the IV-DV relationship."
The business start-up rate was found to have a high correlation with people's perceived opportunities (0.88) and with the GEM Opportunity Perception Index (0.79)--which factors in perception of positive opportunity in a person's country to start a business, the capacity (skills and motivation) to pursue the opportunities, the level of respect for entrepreneurial efforts by others in the society, and the level of resentment against those who do well in an entrepreneurial venture).
You could use this question to discuss factors not mentioned in the GEM Conceptual Model that might contribute to business/job formation and ultimately to higher GDP. One factor that usually comes up in such a discussion is creativity of ideas. This can generate a lively discussion about how the study could have measured for 'creativity'. Students might also mention demographic characteristics of the country, such as median age. Using Bill Gates as a model, you can expect many students to believe in the notion that entrepreneurial
39
ventures are generally started by young adults. They might point out after you show the graphs below that Japan has a median age of almost 40 years compared to 34.3 for the U.S. Also, the percentage of the population under the age of 15 in the US is 22%, compared with Japan at only 15%. Additionally, students may raise the issue of economic stability. Given the unprecedented economic growth experienced in the U.S. in the latter part of the 20th century, they may want to use economic stability as a moderating variable.
SOURCE: GEM 1999 UK Executive Report
Such variables need to be assumed, discounted, or controlled in order for causation to be proven with some degree of certainty. A great effort has been expended in this study to achieve high levels of both validity and reliability. This question is designed to exhibit a lively discussion in the context of control as it is described in Chapter 13
3. Chapter 15 and sampling concerns are the focus of this question. The study has extracted information from two different samples in the participant countries: a survey of 1000 adults and an in-depth personal interview, plus a follow-on survey, with 40 key informants.
A key informant is an expert with substantial experience in each of the nine entrepreneurial framework conditions. Key informants were asked to identify the "single most important critical issue facing the entrepreneurial sector in their country." It was assumed that these experts would focus on factors with the highest correlations with business start-up rates. National teams of interviewers were created and asked to
40
develop a list of such key informants in their country. A discussion could focus on the issue of probability vs. nonprobability samples, and about the way that such individuals were chosen.
This question is also suitable for discussing how a national probability study could be conducted in each country, and the special considerations that would have to take place to conduct such a study in 10 countries with numerous languages, as well as cultural and social mores. The national study of 1000 adults was randomly selected in each country, but no sample frame is discussed in any of the methodology sections in the reports. A well-established research firm, with international offices in numerous countries, drew each sample and conducted each study. Using statistical profiles on age and education for each country, the research firm built a representative sample using random digit dialing procedures in 9 of the 10 countries (personal interviews were used in Japan).
4. This question addresses research design issues. The multi-stage study first identified critical issues affecting entrepreneurial activity. These issues were drawn from face-to-face, personal interviews with as few as 4 experts per country. Detailed interview records were compiled on each country's interviews. Following his or her interview, each key informant also completed a detailed, 12-page questionnaire. Multiple-item indices were developed from these interviews then used to developed the 10-item yes/no question survey given to the sample of 1000 adult in each country. Survey information was then combined with statistical data collected, via government and not-for-profit organizations, in each country to develop comparative indices for each country.
You might ask students to discuss the value of pre-selecting experts to refine the focus of each measurement question, the purpose and uses of multi-stage studies, and the types of bias or error built into the study by the multi-stage process.
5. This final question asks whether this study qualifies as a causal study (vs. a descriptive study). Depending on when you use the case in your course, some students may be encouraged to use the symbols of experimentation in Chapter 11 to describe this study. Their result will likely not reflect any of the standard models. Students should be further encouraged to address issues of reliability and validity, and asked how the study stacks up in this regard. Some students may conclude that the act of calculating correlation statistics makes this a causal study. Others will suggest that building the GEM Conceptual Model is a stage of the research design preliminary to the actual causal study. They may indicate that the model needs to be tested over time (in not only the countries included in the preliminary study but in other countries not included in these early-stage tests) by manipulating one or more variables comprising one of their indices and measuring the net effects. This last stance is likely the most viable, as GEM-study designers plan subsequent measures over time on each of their multiple-item indices, and simultaneous tracking changes in the factors which comprise their model.
Case: AgriComp
Abstract: AgriComp, a supplier of computer systems for farmers, has surveyed it dealers on whether to change its procedure for settling warranty claim disputes. Currently local dealers handle warranty services for customers via local repair followed by a reimbursement claim to Agri Comp. Denied claims follow an internal company appeal process. Dealers have been complaining about the fairness of the appeal process and in a recent survey were asked to respond to an alternative process, an impartial mediator. The student is asked to review survey results and determine whether the costly external mediator process would be worth implementing to keep the dealers happy.
41
This case offers a chance for students to deal with the data before it's crosstabulated. Nothing very fancy is required, but the students will need to recognize that a crosstabulation is in order (or at the very least that some separate tabulations are needed). The dealer preferences are different for those who have used the existing appeals process than for those who haven't. The more they've used the process, the less they perceive a need for change. If the data are tabulated in the aggregate, this trend is not apparent.
1. Jody wonders just how important the process is to the dealers? Was there widespread discontent or had he just heard from a few malcontents at the dealers' meeting?
You can start with a question like "How do the dealers feel?" and follow it up with "Do all of the dealers feel that way?" This will normally bring out a cross tabulation or something like the series of MINITAB dotplots given below. Then you should turn the discussion to how best to summarize or display the conclusions. Either some sort of crosstabulation (using appropriate percentages instead of counts) or plots like those below will work. The main point is to be sure students don't simply declare a "significant" lack of independence and let it go at that. They should have to say something about what kind of dependence they find, not just assert the absence of independence.
The more the dealers have used the existing appeals process, the less they agree with the statement that it should be replaced, so it appears that Jody was hearing from some malcontents. The cross-tabulation of responses by number of uses is given on the next page. The corresponding chi-squared is 82.16 on 12 degrees of freedom, so something is clearly going on. Students may offer a variety of summaries of just what is going on, and you should prompt them for such summaries if all they offer is chi-squared. The general trend is illustrated by such diagrams as the dotplots given on the page following the crosstabulation.
42
Crosstabulation (count) of REP by USE
USE REP 0 1 2 3 +------+------+------+------+ 1 | 12 | 12 | 4 | 6 |34 +------+------+------+------+ 2 | 6 | 27 | 18 | 12 |63 +------+------+------+------+ 3 | 8 | 18 | 16 | 18 |60 +------+------+------+------+ 4 | 4 | 9 | 8 | 52 |73 +------+------+------+------+ 5 | 1 | 12 | 12 | 37 |62 +------+------+------+------+ 31 78 58 125 292
USE (percent)REP 0 1 2 3 +------+------+------+------+ 1 | 38.7 | 15.3 | 6.9 | 4.8 |34 11.6% +------+------+------+------+ 2 | 19.4 | 34.6 | 31.0 | 9.6 |63 21.6% +------+------+------+------+ 3 | 25.8 | 23.1 | 27.6 | 14.4 |60 20.5% +------+------+------+------+ 4 | 12.9 | 11.5 | 13.8 | 41.6 |73 25.0% +------+------+------+------+ 5 | 3.2 | 15.5 | 20.7 | 29.6 |62 21.2% +------+------+------+------+ 31 78 58 125 292 100.0 100.0 100.0 100.0
43
Dotplot of REP by USE
USE 0
: : : : : : : : : : : : : : . 1 2 3 4 5
USE 1 . : : : : : : : : : : : : : : : : : . : : : : : : : : : : : : : : : : : : : : : 1 2 3 4 5
USE 2 : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 2 3 4 5
USE 3 (Each dot represents 3 points) : : : . : : : : : : : : : : : : : : : : : : 1 2 3 4 5
44
Case: AIDS Rates for Females
Abstract: In the early 1980s, AIDS was not generally considered a big issue for women. The student is asked to examine 1991 data from the U.S. Department of Health and Human Services to present summary date that will reveal if this was the case a decade later, and to highlight salient features or any important trends.
This case is richer than it may look. Neither formal technique nor business knowledge is required. Com-puting a few percentages and plotting or tabulating the results will handle just about everything here, but to answer the underlying question, the student must think hard about the underlying data gathering process and the quality of the data. It requires no computer access (though students familiar with spreadsheets often choose to use them for the graphics packages that come with them). It's a good case to assign early in the semester. It forces the students to formulate the question, decide what kind of numbers would help answer it and summarize those numbers. The student who has a good general education has as much to offer here as the technically advanced one, perhaps more. It deals with a topic of current interest, is moderately sensitive (to some), and helps to make the point that statistics is supposed to be a real exercise that helps us understand real things, not just a mathematics exercise.
The class discussion should focus on the following. What trends do you find in the new cases of AIDS for females in the US? What quantities in (or computed from) the data measure those trends? Do the data suggest these trends will continue? What assumptions are you making in your answer?
Students often approach a course in research methods with the idea that there must be one right answer to every study. We would like them to discard this idea in favor of the idea that while some answers may be better than others, generally the different approaches illuminate different aspects of the situation. It's easy to discourage those who aren't used to offering their views if you get too specific in what you say, so we prefer to begin discussion of this case in a general way, with some question like "What's going on with these data?"
A follow-up question like "Can you explain what features of the data led you to conclude that?" seems to work fairly well. Something like "What would be a good way to summarize what you've just said so as to communicate it to others?" may lead to a discussion about the strengths and weaknesses of various tables or charts.
At some fairly early point, you should ask for other approaches, too, both as to the quantities to be com-puted and the form of their presentation. If a good climate has been established, there will be at least two general kinds of analysis and four or five different forms of presentation. These provide motivation for a summary of the alternatives, discussions of ratios versus absolute numbers, tables versus graphs, different forms of graph, and so on.
At its best, this case points out to the mathematically oriented that they had better pay attention to the problem and to the data, not just to the equations, and points out to the mathophobes that it may not be as bad as they feared--simple techniques and some clear thinking go a long way.
Likely ResultsMost students will provide some sort of table or plot showing that the absolute number of new AIDS cases
in females has risen fairly sharply over the years. Most conclude that whether AIDS was a problem for women in the past or not, it certainly seems to be becoming one now.
Many of the students will also point out that the mix of sources of transmission of AIDS has also changed, with heterosexual transmission rising to over 30% of the new cases by 1990. Intravenous drug and heterosexual transmission now account for about three-quarters of the new cases in females.
The sharp drop in new cases in 1991 is the combined result of the reporting delays to CDC and the fact that the 1991 figures are only for six months. The delays affect the 1990 results, too. Students usually notice the missing six months in 1991, but have varying levels of success in recognizing the problem with the reporting delays (though it seems as though there was enough information in the case to alert them to this possibility). There are usually some who conclude, though, that "awareness has increased and the trend is leveling off," or something like that (which later data would not confirm). Students might be encouraged to check in their libraries for later figures for female AIDS cases to see how the trends for 1990 and 1991 look after more complete data are available (see
45
below). Alternatively, you could suggest using the information given in the case (50% to 60% reported within 3 months, etc.) to derive approximate figures, and discuss their strengths and weaknesses. If you want more updated statistics, check our web site or try http://www.cdc.gov/hiv/graphics/women.htm (active June, 2000).
Some students may have difficulty in discussing the topic, particularly the parts about sexuality. This may come out as hemming and hawing or reluctance to participate. The instructor will want to be sure he or she has established a climate of mutual respect to help with this. It's an opportunity to point out that real problems involving statistics don't come as antiseptic little modules at the end of the chapter; they come complete with strong feelings. Part of the art of statistics is to identify the part of those issues that can be dealt with statistically.
Case: BBQ Products Cross Over the Lines of Varied Tastes
Abstract: This case asks students to assess measurement and scaling issues in the context of the introduction of a frozen, microwaveable BBQ product line into the southeast by Rich Products, Buffalo, NY. The new line is being introduced with commercials depicting Ruby, a fictitious waitress at Pork-O-Rama who prefers the taste of the new frozen line.
This case deals with measurement and scaling issues. The student must select a method to measure attitudes toward the product category and the specific brand. In addition, the student must decide on a measurement method to measure preference among a number of brands in the product category. Student may attack issues such as validity and reliability of the measures.
1. What measurement and scaling issues should be considered when developing a study to measure consumers’ attitudes toward barbecue in general and specifically Rich Products Barbeque?
Chapter 13 presents difference methods for measuring attitudes. The student should select a method and defend why they chose that method. Their defense should rest on the issues of validity and reliability. Some additional issues that may be addressed are:
A. How will the managers use the attitude measures? If they simply want to determine the overall attitude toward Rich’s barbecues then different approaches are available. However, if they are interested in developing advertising copy, employing segmentation, or altering the product then the measurement device must collect information on the salient attributes. In essence, the student must decide whether they want to collect formative or reflective indicators of the attitude.
B. A fundamental issue for the collection of attitudes for Rich’s product is consumer awareness of the brand. The brand is new, consequently, will people have an attitude toward it? Will the company need to develop concept story boards, allow consumers to try the product, etc.?
C. Another issue is the target market. It would appear that some consumers will have well developed attitudes toward barbecue but others will not. The question is: Can the same instrument or measurement device be used for both types of consumers?
2. Assume Rich’s wanted to test people’s preferences for their barbecue versus the other leading brands (of which there are five). What would you recommend to measure these preferences?
A. The first choice the student will have to make is whether to use comparative or noncomparative methods. Comparative scaling results in data that must be interpreted in relative terms and has ordinal data properties. An attractive feature of comparative scaling is that relatively small differences among objects being compared can be detected. It must be remembered that the respondent is instructed to directly compare objects; consequently, differences are forced to surface. Comparative scales are, in general, easily understood by respondents but can become
46
time consuming as the number of objects to rate increases – leading to respondent fatigue.B. With noncomparative scaling the respondent is not instructed to compare the object being rated
against wither another object or some specified standard. Hence, small differences among objects may not surface.
C. Once students have discussed these issues they should choose a particular method and defend their choice. The advanced student may want to discuss other potential analysis for the data such as multidimensional scaling approaches.
Case: Calling Up Attendance
Abstract: A study by Prince Marketing for TCS Management Group, which markets TeleCenter System software, measures customer satisfaction with the current edition of the software and aims to predict attendance at a two-day educational event, Users Forum.
This case has numerous tie-ins to various chapters. Used with Chapter 3, you can build the management-research question hierarchy. Used with Chapter 4, your students can build the research process model up through data collection. Used with Chapter 15, you can discuss sample frames, and screening for qualified respondents, as well as various methods for drawing a sample from a sample frame (customer list). Used with Chapter 6 you can discuss the types of data being collected as well as reliability and validity issues. Used with Chapter 13 you could discuss why a rating scale is appropriate to evaluate customer service and likely attendance. Used with Chapter 10 you can discuss the appropriate communication methods. Used with Chapter 12, you could discuss preliminary analysis planning. Used with Chapter 16, you could discuss what to do with the 16% of the sample that felt ill-equipped to answer the issue questions and what preliminary analysis would be appropriate to answer the management questions.
6. This question gets the student in the habit of formulating the management-research question hierarchy (Chapter 3) as the beginning step of a research project. The management dilemma facing TCS is how to prepare for the scheduled Users Forum when it doesn't know what specific presentations to build into the program, nor how many people to expect at the Opryland Hotel event.
Management questions: What topics should be addressed in the presentations or materials distributed at the Users Forum? How should solutions be delivered, given the attendance expected (written materials? whole session devoted to the issue?)?
Research questions: What issues cause the most concern among the current users of the TeleCenter System software? How many current users see attendance at Users Forum as a viable means to address troublesome issues, such as ease of use, technical support access and responsiveness?
Investigative questions: What is the current users' evaluation of customer service, especially on ease of use, software-generated reports, technical support service, and effectiveness of software for its intended purpose? Who, if anyone, from a current user's company will attend the Users Forum? Do new users of the software have different concerns than more established users? How many representatives from a single company may come to the Users Forum? What type of respondent (user or manager) is most likely to attend the Forum? Do the different types of respondents have different concerns with TeleCenter System software?
Measurement questions: We don't have the actual instrument, but we know that Prince Marketing collected information relative to investigative questions using a 7-point rating scale, where "7" was the most positive on the scale. We also know they were able to classify the respondent as a software user or call center manager.
47
7. The obvious choice, as indicated by Exhibit 10-2, is the telephone interview. Because TCS needs the information is a relatively short amount of time, one must exclude personal interviewing as a collection method. Given the technical skills of the respondents, self-administered via computer was a possibility, if TCS had e-mail addresses or Prince felt the response timeframe would be met. Prince Marketing chose a phone interview because of the ability to control the process and meet their three-week promise, but also because of the limited number of data variable they were collecting. They could choose any of the three methods (human, CATI, or computer-delivered), but human-administered definitely gives Prince and TCS the most control. CATI, however, gives the telephone interviewer the ability to enter the responses in process, permitting Prince to tally the results more quickly possibly making them available to TCS in real time, so they could prepare for presentations without waiting for the formal estimate of attendance.
8. The request for a preliminary analysis plan relates well to Exhibits 11-1 and 11-2 and would be fairly simple for this study. TCS would want to frequencies on the attitudinal-scale variables. And we would want to cross-tabulate such variables by whether the respondent definitely would be, might be, or definitely would not be in attendance at the Users Forum. Frequencies (or cross-tabulated frequencies) would tell us the most pressing issues, assuming Prince has done a good job of anticipating issues in the design of the data collection instrument or was able to transform the data during data preparation into homogenous groups of responses on issues. We also want to be able to predict attendance, so we would want frequencies on the attendance variable, as well as their interest in more information about attending the Users Forum. Hopefully we can cross-tabulate this information against classification variables such as industry of the firm and years of software use that would be readily available from the customer database the TCS provided to Prince Marketing.
9. In data analysis, the new users who felt ill-equipped to evaluate the software ease of use question (16% of the sample), would likely not be included (counted as missing cases) in the analysis of this issue. However, their likely attendance could still be important to the prediction of attendees overall. If some of these new users did identify software installation, technical support, or use concerns, you could cross-tabulate a status variable (new user vs. established user) against the various issues variables. Such data exploration actions may very well lead to Users Forum sessions exclusively designed for new users.
10. You can use this question to tie the trends cited below and Exhibit 1-1 to the communication data collection approach. You could also tie this question back to Chapter 5 and the subject's right to privacy. The issue of incorporating marketing promotion within the context of customer satisfaction research is commonly done, although it can be a two-edged sword. From a marketer's perspective, you have a potential attendee on the phone and you could efficiently use his or her time by telling them about the Users Forum. From a professional researcher's perspective, such a combination use of the survey is totally inappropriate and contributes to the growing refusal rate among potential respondents. If research is accompanied by a sales presentation, potential respondents have the right to be angry when they agreed to participate only in a survey. Such a change in orientation and purpose is a violation of the subject's right to privacy.
Trends in Research Profession What Managers Should Watch ForThe Positive TrendsBudding recognition of the importance of the researcher participating in strategic planning
Strategic planning initiatives that have an appropriate research component.
Emerging return to the strong, internal research group or a long term partnership with an external research supplier
Better methodologies performed by knowledgeable professionals generating significant value to management decision making.
48
Trends in Research Profession What Managers Should Watch ForThe Negative Trends Increasing pressure for rapid measurement and feedback. Methodologies that raise questions about
representativeness of samples. Projects fielded without thoroughly understanding the
management dilemma, which results in information of little value to solve the dilemma.
Increasing pressure on research specialists to interpret research results and provide strategic recommendations.
Research providers who may be technically competent with expertise in quantitative skills, but who are not trained or have limited training in management.
Increasing demand for information privacy, reducing respondent willingness to cooperate.
Research providers who are knowledgeable about extracting valuable knowledge from internal databases (data mining).
General consulting firms moving toward functional specialization and increasingly doing their own research.
Consultants who lack the technical skill in research methodologies creating a lack of transition from management dilemma to research protocol.
Consultants sharing information among specialized clients, generating a loss of intellectual capital that fosters distinctive competencies—the basis of competitive advantage.
Continued perception of research as an expense rather than an investment in reducing uncertainty.
Research budgets being cut during economic downturns.
Increasingly wide range of competence among those offering services in the research industry.
Insufficient quality of credentials among research professionals being used or considered.
Assignment of projects to researchers with insufficient technical background to the quality research.
Widening cultural mindset gap between business strategists and research specialists.
Breakdown in communication between researcher and manager who will use the research results, resulting in information, not knowledge.
Consulting and industry’s expectation that new hires have received scientific research training in college.
Assuring that curricular demands placed on the new hires at their collegiate institutions have included research methodologies.
Decrease or decay in intellectual transfer between academia and research profession.
Assuring that firms hired as research specialists keep abreast of the newest methodologies through continuing education efforts.
These trends are drawn from various presentations at the annual American Marketing Association’s Marketing Research conference since 1997.
Case: Campbell-Ewald Pumps Awareness into the American Heart Association
Abstract: You wouldn’t think that an organization that does as much good as the AmericanHeart Association would have low awareness, but at the start of the described research program its unaided awareness level was just 16 percent. For a company reliant on contributions, low awareness is a major problem. This case profiles the research behind the American Heart Association’s first-ever paid advertising campaign. www.campbell-ewald.com; www.americanheart.org
1. After watching the ads, take the Learn and Live quiz. Take a position on the tracking of individuals taking the quiz as a measure of direct response of the ad’s effectiveness; defend your position. (http://www.americanheart.org)
You might find it useful to use this discussion question as a class exercise. You can divide your class into pro/con teams and have each come up with the arguments for that position.
Arguments for using involvement with the online quiz as a means of ad tracking might include:
49
That getting audience members to the Web site was clearly an important action objective of the ad. Engaging visitors by involving them in the taking of a quiz is more likely to reveal their understanding
of the underlying message of the ad campaign. Arguments for not using involvement with the online quiz as a means of ad tracking might include:
That many individuals that see/hear and understand the message may not be encouraged to visit the Web site…so the communication objective may be met while the action objective may not.
Individuals that see/hear the ad and understand the message may not have access to a computer. Since computer access was not a variable in choosing the media for the campaign to exclude this portion of the relevant population when tracking ad results creates error.
2. What are the advantages and disadvantages of using secondary data in this particular case? The primary reason Campbell-Ewald (C-E) used secondary data during their preparation of the AHA’s pitch for the advertising account was to gain a better understanding of the organization and of heart disease, as well as understand what motivated giving to a non-profit organization. Using secondary data shortened the time they would spend on research. Also, they quickly discovered the scope of the potential problem facing AHA. While they wouldn’t have wanted to craft a creative strategy based solely on secondary data, effective use of secondary data collected during exploration is actually expected in business research. It was the interview of John Paling (primary data) about how people perceived risk, however, that gave them the insight to develop the approach that won C-E the AHA business.
3. Discuss the role of Diagnostic Research in the American Heart Association’s selection of the personal stories creative approach.
Diagnostic Research was hired to pretest the various creative approaches that evolved from the focus group discussions, the interview with John Paling, and the secondary data search. DR showed the animatic prepared by C-E to a sample of 1000 adults in a mall setting. The AHA animatic was positioned within a pod of animatics for other products and services, similar to the way the audience would see the real AHA commercial. The subsequent interviews measured brand recall and the main idea that the audience obtained from the ad.
The DR methodology indicated that all three creative approaches tested would generate appropriate brand recall and message capture. That meant the C-E might need additional research to correctly select the best creative approach or they would need to use their vast experience to recommend the better approach for AHA to take at this particular time in AHA history. This is good time to mention that even great research doesn’t make the decision for the manager. Rather, research provides the undecided manager with evidence to sway his judgment in one direction or another, or it substantiates a direction in which the decision-maker might be leaning.
4. Explain why both qualitative and quantitative research techniques were used to develop the personal stories campaign?
The use of more than one research technique in building an advertising campaign is fairly common in business research. The quantitative data derived from previous AHA surveys and from the DR animatic testing allowed C-E to quantify the risk of each campaign. It was the qualitative research, first the interview with John Paling that gave C-E its understanding of personal risk, as well as the focus groups that put a face to real, heart-disease stories that gave C-E their strategic approach to the problem of raising AHA awareness and that, ultimately, provided the winning campaign creative approach.
You might want to discuss whether the mall setting was conducive to either the viewing of the animatic or whether patrons of malls represent an appropriate group from which to draw a sample. If DR has facilities in the mall, as many such firms do, then the group from which it draws its sample might suffer from over-testing and thus inject error into the process.
50
Depending on how much you wish to discuss advertising, other discussion opportunities include: Discuss how students would test which of the personal stories ads was most effective. Discuss whether the appropriate sample should have been weighted more toward females than males. Discuss what research could be used to determine whether an ad campaign would be more effective in raising
awareness and understanding of the AHA, compared with other promotional methods.
Case: Campbell-Ewald: R-E-S-P-E-C-T Spells Loyalty
Abstract: The case describes a detailed study conducted by Campbell-Ewald, who sought to substantiate the importance of “respect” when dealing with customers in different business related sectors. This agency, steadfast in its success through customer satisfaction principles, establishes through its research that the construct of respect is held in high regard by customers and therefore plays a significant role, when determining, among other things, the longevity of client relationships.
1. How would you operationally define the construct of respect? Take the perspectiveof each of the three client sectors: insurance, air travel, and retail.
This question is fitting for discussions concerning concepts and constructs. It prepares students to identify the similarities and differences involved with the use of both scientific terms (Chapter 2) and how dictionary meanings differ from those used in testing. Both terms are inherently similar yet the construct has specificity necessary for measurement and replication of a study. Constructs are not easily observable, created for the express purpose of testing, and generally more complex. The construct of respect though similar in various theories, assumes variations when operationally defined in the research literature and when viewed from the perspective of these three client sectors. The term respect as used in conversation has more individualized experiences attached to it and is idiosyncratic to individual speakers. Thus it is difficult to find agreement among students in this definition during a discussion. You will find that through brainstorming, students may indicate that the definitions overlap and they may point out difficulties while trying to “operationally” define them. When such constructs are used in pure research, measurement scales are essential to further specify meanings. The important thing for students to grasp is that through operational definitions, a person should be able to measure or conceptualize the construct, given the specific purpose of the research. Advise students that operational definitions often serve the same purpose for researchers, as common jargon does for narrow language groups. Here are some student definitions of respect after reading the case:
Insurance: tailor products for specific customer needs; maintain the individual’s right to privacy when dealing with client information; show concern for unfortunate mishaps; provide contractually correct compensation in a timely fashion, refraining from unethical and dishonest conduct
Air Travel: strive to meet expectations in check-in, boarding, onboard services, debarking, and luggage handling; providing appropriate reimbursements or compensation to customers when necessary; show concern for safety and on-time schedules.
Retail: maintain a polite and cordial demeanor when dealing with the public; listen to customers’ specific needs and match your product accordingly; remain professional when faced with unpleasant circumstances, regardless of who is at fault.
2. Map the overall design of the research described here.
The nature of this question gets students into the habit of identifying the different types of designs and the steps involved when conducting research. It allows students to become more familiar with the various methodologies utilized and is an opportune time to explore data collection techniques and the constraints researchers sometimes encounter when utilizing such techniques. As such this question is quite applicable to Chapter 6, which discusses design strategies as well as 11, which explores data collection methods.
51
The first step involved Campbell-Ewald’s exploratory attempts to discover the reasons for the disconnect between respect bonds and its customers. Campbell-Ewald discovered firms were accustomed to using Customer Relationship Management (CRM) software, as a mechanism for tracking satisfaction behavior among customers. However, research revealed that investment in technology and software to model likely behavior was not achieving its desired goal. Among the reasons for Campbell-Ewald interest was the apparent connection between customer satisfaction and customer loyalty and purchase behavior. This stage also tapped into feedback from Gartner Group and Accenture, relationship experts such as Dr. Phil and Steven Covey; and information garnered from existing literature on the dynamics of people relationships.
o This is an opportunity to ask students what other exploratory avenues that Campbell-Ewald might have pursued.
o This is also an opportunity to discuss the different between pure and applied research. The Campbell-Ewald clients in the three sectors did not have specific relationship management issues they were researching when they signed on as part of the respect research initiative.
The second stage of the research --data collection-- revealed the use of two levels of primary data. Through partnership with Synovate, Campbell-Ewald first utilized Synovate’s omnibus Telenation (nationally representative telephone survey) to ascertain whether the public viewed respect as an issue when they dealt with companies. This data was used to justify Campbell-Ewald’s belief that respect was a salient issue to customers. Here you can discuss the implications involved with using an omnibus study and how the process of asking a few questions to a national probability study can prove helpful to a researcher. Campbell-Ewald’s decision to explore this data may have yielded significant findings, which further validated their conclusion that “respect truly matters.” You can ask students their view of how credible the use of an omnibus study data is and why.
To establish the varying perspectives of respect concerning the different sectors (insurance, airline and retail) Synovate utilized a sample of 12 focus groups, four each in the above named sectors. This stage explored the scope of the research question and seeded the subsequent 4-page questionnaire. To better allow for a representative sample, the focus groups included men and women from both Chicago and Detroit. This is an apt time to discuss the advantages and disadvantages of using focus groups. Considering the sample used—four each in three sectors—raises questions about accuracy and representative results from two market areas. Feedback from the focus groups was used to design measurement questions for the mail survey sent to Campbell-Ewald’s clients’ customer lists from each sector. The focus groups were used fairly traditionally in this case to see a more detailed quantitative study.
The final data collection phase of the research was a national probability mail survey conducted by Synovate. One possible discussion question is: Why did Campbell-Ewald choose NOT to use the Synovate panel to increase the response rate to the survey. [Campbell-Ewald feared it wouldn’t be representative of the three sectors.] Another might relate to the non-response error inherent in a mail study and how it is handled. With only a small response rate (less than 9%) could Campbell-Ewald have concluded that respect matters to only a small portion of the relevant population and the remainder simply didn’t care? Generally this is a good time to discuss the strengths and weaknesses of mail surveys.
A. What types of studies were involved in Campbell-Ewald’s respect initiative?
There were three study types involved in Campbell-Ewald’ respect initiative. You can use this time to ask students how they would relate the case to other types of studies. You can also discuss the characteristics of these
52
studies and the purposes each serve in research. Given the outlines of each study, you can ask the student to indicate its strengths and weaknesses.
Exploratory study. Campbell-Ewald used secondary data searches to define the construct (including studies published by Gartner Group and Accenture, books by authors such as Dr. Phil and Steven Covey, and information from other existing literature), and focus groups to help refine attitudinal statements to be used in later quantitative research. The twelve focus groups study completed the exploratory phase of the research and segued into the survey.
Descriptive Study. Campbell-Ewald utilized data from Synovate’s Telenation telephone survey, which served as a preliminary step in the large quantitative study which followed.
Explanatory study. Here Campbell-Ewald’s research team attempted to isolate the components of respect and examine its relationship to satisfaction and loyalty. They were interested in discovering how customers viewed the issue of respect when dealing with companies. They aimed to ascertain what caused customers to remain loyal to businesses and in so doing “identify the tenets of strong personal relationships. This effort aided in the creation of its five “People Principles.” With its extensive clientele, which included customers in different sectors (insurance, travel, retail and government) Campbell-Ewald sought to discover, based on their five principles, how respect was viewed in each sector. To build on this, Campbell-Ewald capitalized on Synovate’s experience in conducting large scale mail surveys, which was a significant source in the data collection process. Its sophisticated analytical techniques, namely its “proprietary brand propensity model” called “The Momentum Engine,” aided in providing more detailed information in analyzing relationships with sales growth. Given its varying client list, therefore its sample size, Campbell-Ewald desired a more comprehensive feedback.
B. What are the strengths and weaknesses of the various methodologies?
This question is applicable to Parts 2 and 3 of the text where the methods of primary data collection are presented.
Mail Surveys (Self Administered Surveys): The use of this methodology, though often cost effective in research, can be problematic when it comes to response rates. Mail surveys are typically inexpensive. However, low cost is no guarantee that responses will be substantial. Five thousand surveys were distributed, and the amount returned (5% – 9 %) opens the discussion to nonresponse bias. Response rates for mail surveys are typically low, but steps can be taken to adjust this. This is an apt time to discuss the pros and cons of using different survey methods and the challenges researchers sometimes encounter. Was the survey too lengthy at four pages? Did that contribute to the low response rate? Were participants just not interested in the whole issue of respect from the companies they dealt with? The size of the dataset for each sector (200-500 cases) obscures the fact that while the data are sufficient for statistical analysis, representativeness is in question? You can ask the student:
What steps could Synovate take to increase response rates? How can a researcher minimize nonresponse error? How effective would incentives be given sample size and cost?
Campbell-Ewald’s decision to disguise their clients, by sending surveys under Synovate’s letterhead could have increased or decreased response rates. By disguising the survey’s sponsor participants may be less inclined to complete the survey. Many might have been unfamiliar with who Synovate, further reducing their likelihood of response. Some participation in any survey is dependent on participants recognizing the value they bring and on the rapport between participant and researcher, participants may be more likely to participate in the survey when they are familiar with the organization or feel some sense of loyalty to it. At
53
the same time, disguising the name may have proven helpful, as this would give respondents the opportunity to give an honest opinion of how they felt Campbell-Ewald’s clients treated them, or how they viewed the issue of respect.
Focus Groups: The use of focus groups, as a way of developing measurement questions that would clarify the core motivational drivers in categories being studied, proved useful as a way of defining the parameters of respect. However, having clients actively involved in this process of creating the measures and the subsequent development of the survey can, potentially, create some bias. As such, measurement questions could be created that only defines how specific “clients” may conceptualize respect and not necessarily be applicable to all member firms in a sector. Since a communication agency rarely represents more than one firm in a category, having the measurement questions apply more to one firm (e.g., Continental) in a category rather than all other firms in the sector is really not problematic. The use of mainly “top customers” as the sample frame, begs the question as to how representative that sample may be for pure research.
Telephone Omnibus Survey: An omnibus survey has a very short turn-around while it limits the number of questions contributed by any one sponsor. Therefore, the omnibus can sometimes represent a bundle of questions that have little direct connection. This leap from topic to topic can sometimes confuse a participant and as a result the sponsor may get less than the clarity hoped for. For an undefined construct like respect, the omnibus survey should have been a comfortable respite from more product/store/issue specific questions. The lower cost and shorter turn-around would have prevented the larger study from being slowed.
C. How do the methodologies complement each other?
The use of the focus groups for ideation and creating measurement questions is a useful way of getting first hand knowledge as to the type of issues that customers were most concerned about in each sector when it came to respect. Using feedback from the focus groups, Synovate was able to create and appropriately tailor the measurement questions that participants could relate to. Campbell-Ewald’s decision to actively engage its clients in this process, further enabled them to seed the survey questions. Additionally, the incorporation of a 5-point scale complements the attitudinal nature of the investigation.
3. Analyze the use of the 5-point scale for measurement of respect dimensions. What other statements would you add to the sample provided in Exhibit C-E-1?
This type scale as a measurement tool is a simple and useful way of recording responses. With the attitude statements connected to a five-point “strongly agree to strongly disagree” scale, comparisons between sectors is more efficient for data analysis of attitudes. Given the length of the survey, this is an easy way to allow participants to indicate their opinion, while providing them with a middle ground, in the event they neither agree nor disagree.
54
Though the scale channels a response within a specific context, participants responding to complex construct such as respect may either add to measurement error by responding to statements with which they have no experience or desire to make additional written clarification. Using some open-ended questions could have been advantageous. A useful comparison is the evaluation surveys that students complete at the close of each semester, where they rate their professors and the course. To generate a discussion you can ask students how likely they are to complete open-ended questions on such surveys, and why? You can also ask students to suggest statements they would include in the Campbell-Ewald study as opened-ended questions.
Campbell-Ewald and its partners used correlation, multiple regression, and other multivariate analyses in an effort to identify customers with the greatest potential to drive sales growth. Nonmetric scales would have presented severe data analysis problems since the lack the power to extract the information required by the research question.
Exhibit C-E-1 Sample of Attitudinal Statements
Researchers asked participants their degree of agreement with several respect validation statements. A sample of these statements, developed from focus group discussions, appears below.
Strongly Agree Neither Disagree StronglyAgree Agree nor Disagree
General Respect Initiatives, across categories
Is interested in listening to what is important to me as a customer Places their own interests above those of the customer Notifies me in advance of sales and special promotions Rewards me for repeat business Honor commitments/promises they’ve made to me Specific Respect Initiatives, with category
Accepts returns without a hassle (retail) Flights take off and land on schedule (airline travel) Handles claims in a timely manner (insurance)
4. Discuss the pros and cons of the sampling plan for the mail survey.
Campbell-Ewald received names of customers from their clients. These clients provided names mostly from their top customers. Synovate distributed five thousand surveys to each sector: insurance, air travel and retail. However, the case did not indicate as to how customers were chosen as part of the sample to determine if this was done in a randomized manner. It also did not indicate if criteria were set to choose customers, such as length of time with a particular client or company, their gender, or age groups.
Considering that clients provided names of their mostly from their top customers, establishing whether or not this is a representative sample of their customer population, is pertinent information. Subsequently, with the target population being mostly “top customers” this potentially limits the ability to make inferences about the general population and inevitably raise questions as to the accuracy of the findings. Given the nature of the research, and the fact that five thousand surveys were sent out to each sector with a small return, it is safe to assume that not every customer relevant sampling element is represented. This is a good time to discuss with students, the pros and cons of using probability versus nonprobability sampling. Students may argue that this was the most cost effective method to take, while others may argue that the sample produces much bias and therefore questions the accuracy of the results.
55
Case: Can Research Rescue the Red Cross?
Abstract: The American Red Cross, known for its worldwide efforts in providing disaster relief and assistance to the sick and needy, comes under intense scrutiny and attack for the misappropriation of post September 11 donations. The case describes the concerns expressed by board members of the Red Cross with regard to how such problems could be prevented in future fundraising efforts. It also outlines how this organization sought to address its existing dilemma of changing the public’s perception of the way in which funds are managed and whether public donations were being used appropriately.
This case is quite applicable to Chapters 1 through 3, where students are introduced to the research process and the different types of studies that the Red Cross could have undertaken to avoid their existing dilemma. The case also ties in to Chapter 3 and prepares students for the creation of the management-research question hierarchy (Exhibit 3-2, Management-Research Question Hierarchy). Additionally, it allows for discussions on the decisions made by researchers when designing samples (Chapter 15). By describing and discussing the dilemma faced by the Red Cross, students will be able to use Exhibit 3-3, Formulating the Research Question, to identify management dilemmas and in so doing formulate appropriate research questions that the Red Cross could consider for discovering better ways of funds management. Used with Chapter 12 and 13, students can discuss the types of measurement scales. In addition, the case ties into the data collection methods described in Chapters 10 & 11).
1. If you had been McLaughlin or Decker, what research would you want done?
You can begin discussions of this question by having students suggest how McLaughlin or Decker might define the dilemma. Some may suggest that the dilemma faced by the Red Cross lies in how to better manage and appropriately make use of public donations. Others may see the dilemma as how to overcome the negative perception that donors may have of the Red Cross’s fund raising activities and regaining public trust. Others might see the dilemma as one of education of the public and how best to do such activities. Students may also suggest that the Red Cross needs to find ways to reassure donors that their donations will be distributed appropriately. Both McLaughlin and Decker could benefit from exploring how current funds are pooled and distributed, how informed donors are of the ways in which the Red Cross manages and distributes donations. Management could also explore if its decision to use a portion of the Liberty Fund as reserve for future disaster, was consistent with its mission and previous fundraising strategies.
2. Create the Management-Research Question hierarchy for the research you think might help the Red Cross make decisions related to public relations efforts and future advertising soliciting donations.
Exhibit 3-2 and 3-5 are useful as a guide for this discussion: We suggest using the management dilemma facing the Red Cross as the public’s perception of its mismanagement of donations. The Red Cross must change how it is perceived by donors who may question whether their donations are being used for the causes for which the Red Cross was perceived to be soliciting funds. The Red Cross must able to provide services based on the contribution it receives from public donors. If it were to continue to be seen as engaging in activities inconsistent with its mission, the Red Cross could face a significant decrease in its funding sources. This may also place a negative stigma on the nonprofit charitable sector and its future fundraising activities. Therefore, the Red Cross must regain and maintain its credibility in the public’s eyes, so as to be able to ensure the continued receipt of donations.
56
Some of the options students might generate are:
Management Questions Research Questions Investigative QuestionsAll management questions are not equally addressed by research. However considering the gravity of the existing dilemma, how should the Red Cross make donors clearly aware of their policies with regard to how contributions are expended?
How should the Red Cross change the way it advertises is policies with respect to donation allocation?
What is donor understanding of the existing policies relating to how donations are raised and spent?
Should the Red Cross consider revising its policies relating to how donors’ contributions are pooled and distributed?
Should the Red Cross use separate fund-raising activities for each disaster or should it solicit funds only for the general fund that may be allocated to any disaster where needed?
What practice (specific fund or general fund) is more likely to generate more largesse among donors?
What level of trust do donors have that the Red Cross will spend its donations wisely?
What are the donation patterns: primarily to specific pleas for help or have donors simply donated to further the general mission of the organization?
How can the Red Cross better manage the funds it receives from fundraising activities, specifically funds raised in excess of the amount needed for a specific disaster?
Should the Red Cross utilize a predetermined donation goal, redirecting funds to the general fund after that goal for donations is received, and communicate this goal to the potential donors?
How is a dollar estimate of a donation need level determined?
What is the potential donor’s understanding of this estimation process?
Will a dollar donation goal for a particular disaster relief fund discourage donations once that goal is reached?
How would potential donors want to learn about progress toward goal achievement?
3. If you created a RFP, what would it contain?
Chapter 4 introduces the RFP, which is supplemented by the sample on your text DVD, and provides additional insight as to the contents and assessment of a Request for Proposal. The proposal would contain:
Definition of the problem facing the Red Cross (the management dilemma) Identifying the limitations involved Providing a description of the policies relating to Red Cross’s fundraising Guidelines
4. What considerations should influence sampling decisions in any research the Red Cross would do on this
57
issue?
Given that the American Red Cross is a national organization and their reputation was at stake, the selection of an appropriate probability sampling technique is indicated.
The larger question is whether sample units should be stratified in some way. Students should be encouraged to brainstorm all the possible subgroups within the donor population. Many donors may have made contributions to the Red Cross via its website, where they were given the option of contributing to a particular cause. To allow for accuracy and effectively address the issue of the existing policies relating to how donations are raised and spent, a sample could be drawn from persons who have been previous donors to a particular fund. Persons who visit the website for general information and not for donation purposes could also be targeted as possible sampling units.
Students should also be encouraged to identify the sample frames available to the Red Cross due to its past fundraising practices. Make sure that students don’t just focus on web site contributors as a sample frame when making sampling decisions, as the Red Cross’s population of donors extends beyond website contributors.
Once students have focused on a type of probability sample, they should also consider sampling procedures for drawing that sample. For example, if they have decided that a stratified sample is appropriate, and that the larger ‘former non-profit donor population’ is the basis for sample unit selection, how might such donors be screened to determine whether they have donated previously to the Red Cross general or disaster-specific funds.
5. If a survey is used, what scales would be most appropriate?
You can use this question as a group activity where students can formulate or use the class-generated investigative questions to argue for specific types of measurement questions. One debate to expect is the use of ranking versus rating scales, or for specific types of ranking or rating scales, for example Likert versus paired comparison scale. In Chapter 13, the section on Selecting a Measurement Scale along with Exhibits 13-2 and 13-9 provides the context for decision making.
Students should also be challenged to ask for the appropriate type of survey. Whether the students choose phone, intercept, or web survey should influence the types of scales used.
Also, given the large size of a national probability sample of donors, simplicity and practicality should be considerations in selecting appropriate scales. Students will likely offer that numerical and Likert scales meet such criteria. And both also allow for the production of ordinal and interval data, suitable for extensive analysis. With respect to the likelihood of donors making donations to a particular cause or a general fund, or having them indicate their preference to where their funds should be spent, the Constant Sum or Forced Ranking scales may also be offered and would be appropriate.
Case: Can this Study be Saved?
58
Abstract: This case depicts a discussion between individuals who are trying to assess the quality of research based on the sampling procedure and the results based on the sampling process. It asks the student to evaluate the sampling process and make recommendations to use the collected data or resample. Data table included.
1. Do you agree that drawing a second sample was a good idea?
It’s not entirely clear that this was a good idea and it may well have been unnecessary. There are several issues here:
A. While it’s usually better to have more information, it is not always worth the extra cost.
B. The second sample may not have been necessary because the standard error of the first
sample initial returns ($57.78=849.26/ 216 ) is already less than the $100 to within which they want to know average spending.
C. Those who chose to respond initially may not be entirely representative. Rather than choose a new sample and receive another biased group of responses, the follow-up of non-respondents is preferable.
D. However, there is nothing fundamentally wrong with choosing a second sample for the purpose of obtaining more information. It is permissible to draw two random samples and then combine them, forming a single random sample with a larger size, provided the decision to draw the second sample was not based on estimates obtained from the first sample.
2. Were the follow-up mailings were a good idea? Explain.
Yes, these probably were a good idea. Since the 400 in the initial mailing of the first sample were chosen to represent the entire membership, the opinions of the initial non-respondents are important. It can happen that those who respond initially are different from the others: perhaps they have more time, or are more actively involved in the issues. Since the purpose is to survey the membership (and not just the most active members) these extra efforts should help ensure representation.
3. Which of the results are useful? Are these data sufficient to solve the management problem or is further study needed?
A. There are some useful results here, but even after follow-up efforts, only about 70% have responded. We still lack information from about 30% of these people. The follow-up averages are much lower than the others. Evidently there is a relationship between planned spending and sending in the questionnaire. Those with high spending were more likely to send it in. Differences from one sample to the other are important. Considering the differences between initial mailing and follow-up responses, these 30% could well be below any of the reported averages. We don’t know. In the real world, there often is non-response that cannot be eliminated as a problem.
B. The pilot study should probably not be combined with the other responses. The pilot study was not a random sample to begin with and has served its purpose in testing the questionnaire. The pilot study averages are very much higher than any of the others. As these were not randomly drawn, we have no reason to consider them representative of the membership in general.
59
C. It might be argued that all other responses (first and second sample, initial mailings and follow-up responses) are useful and should be combined. The resulting average ($3,374) has a standard error of $64, which is less than the $100 required. This then represents the planned spending of the approximately 70% of members who are likely to answer these questionnaires.
D. Ideas for further study could include different design methods or additional follow-up of non-respondents.
Case: Donatos: Finding the new Pizza
Abstract: The case describes a multi-stage study conducted by Donatos, an independent, premium pizza restaurant chain, aimed at tracking interest and response rates to a newly introduced NO DOUGH pizza concept. The research conducted by Donatos is an attempt to test and subsequently meet the needs of its low-carbohydrate diet market. In so doing, it also aims to measure customer satisfaction with the new pizza concept.
1. Map the research design used by Donato’s for new product development.
This question provides an appropriate introduction for students to the various descriptors of research design discussed in Chapter 6, Exhibit 6-2 Descriptors of Research Design. Here the framework and procedures for research activity is outlined. Students will be able to discuss the exploratory steps taken by Donatos, such as their monthly WASSUP Meetings, e-mail comments from customers and monitoring of eating trends from different sources. The case also allows students to distinguish between different data collection methods, such as the self-administered intercept survey and call-back phone survey employed by Donatos. Secondary data was also used during the exploratory stages from sources that documented and monitored eating trends. Therefore, students can discuss the pros and cons of using this type of data. The experimental design also utilized will allow for discussions relating to the taste tests conducted among employees and how Donato’s decision to exclude its “special ingredients from the website may have affected the yielded responses.
The case indicates that Donatos has monitored different sources, such as the syndicated data, NPD Eating Trends, which provides indications of changes in eating habits. They have also garnered feedback from comments customers send via e-mail from their website, in addition to the WASSUP meetings held on a monthly basis, where employees were asked to provide feedback on existing social and cultural trends. These steps then establish the exploratory stages of their research. Through this stage, Donatos was able to discover that there was an existing interest in low-carbohydrate eating plans.
Stage 2 of the study, the research-based product development phase, involved development of the product prototype. Here data was collected from employee taste testing; in restaurant tests, where participants would complete self-administered intercept surveys; call-back phone surveys for customers who were serviced through delivery and concept screen activities where participants were shown photographs of food products and then questioned. You can use this time to discuss: Why a phone survey and self-administered intercept survey was used. The pros and cons of using the self-administered intercept and phone survey (Use Exhibit 10-5
Comparison of Communication Approaches as a guide). How reliable would the feedback be from employee taste testing activities?
Stage 3 involved choosing a marketable name for the new pizza concept, hereDonatos tested three different names using a weekend omnibus phone survey. The case indicates that
60
their typical new-product development research would routinely take 12-14 months to complete. Given the time constraints that Donatos faced (the possible short-term nature of the low-carbohydrate trend), you can ask students to discuss the implications of the findings yielded from the phone survey completed over a weekend period.
Stage 4, the final stages of the design, addressed the tracking of response rates where Donatos employed ongoing telephone tracking studies, in restaurant comment cards and e-mails from customers through its website. You can ask students to discuss the usefulness of this stage of the research. Some may indicate that for the purposes of future research, Donatos can benefit from the findings in terms of strategies that were or were not employed, or other data collection techniques (personal interviews or mail surveys) and sampling methods.
2. Evaluate the WASSUP meetings as an exploratory methodology to help define the research question.
The WASSUP meetings undertaken by Donatos, served as an exploratory step to further identifying the dilemma faced, that being how to address increasing interest in low-carbohydrate diets. These meetings not only supplemented reviews done of other sources, namely the feedback via e-mails and monitoring of eating trends, but it provided additional insight into the dynamics of other social and cultural trends. The WASSUP meeting is essential to Donatos, as it is able to gather useful information about the market from employees, and create a synthesis of this information to best determine how its decision will affect not only the general public, but also persons of different cultures. This feedback from the meeting will also be helpful when tailoring research questions, as through exploring knowledge from different cultures (likes and dislikes) Donatos can then establish a central focus and design specific research questions. You can have students suggest possible research questions that would be applicable to the study, which Donatos could take to solve the existing problem or address concerns about the low-carbohydrate diet market. Chapter 3 (Exhibit 3-4 Formulating the Research Question for MindWriter) can be used as a guide. Donatos may suggest the following examples as research questions:
Should we introduce a new pizza concept to satisfy the low-carbohydrate diet market? Should the crust-free pizza concept be modified? Should the soy crisp recipe be used or should we introduce another protein rich alternative?
3. Evaluate the test market Donato’s used. What were its pros and cons?
Evaluation of the test market relates well to Chapters 15, when discussing the intricacies of sampling and Chapter 11 on Experiments and Test Markets. This question also becomes useful when preparing students for data collection in research (Part 4) and discussing ethical implications involved when dealing with participants (Chapter 5). The test market (customer reactions to the product in two stores in one market) used by Donatos raises important reliability questions.
First, one city is rarely used in a test market as researchers have found distinct eating preferences by geography.
Second, while the test was conducted over several days in the two stores, unlike the introduction of many new food products no promotion was done to attract customers to the restaurant for the purpose of purchasing the NO DOUGH pizza. This might be indicative of a sample of customers who did not match the profile of the potential customer:, those individuals following a low-carbohydrate diet. As the test market continued over time, a screening question about whether a customer came to Donatos specifically to purchase the NO DOUGH pizza could have been used to distinguish these desired sample units from those who were likely not the primary target market segment for the low-carbohydrate pizza.
61
Third, the ideal situation for a true experiment would be test products prepared under controlled conditions. Most test markets, however, use actual market conditions with limits the availability of control. The case indicates that 16-17 year old employees were preparing the NO DOUGH pizzas that were used during the test market; this would be typical in most Donatos’ restaurants. A discussion of the type of experiment being conducted is appropriate here.
There are some advantages as well as disadvantages that this test market brings to the study. As a class activity, you can ask students to suggest possible pros and cons of the test market and instrument used, these may include the following:
In restaurant tests-
Pros:
Reduced research cost by virtue of having its own current customers as testers The ability to capitalize on a greater speed of data collection Requires less supervision and manpower as test was conducted on site Eliminates the need for higher-cost product-testing facilities.
Cons:
Walk-in customers may not be representative of the target market for low-carbohydrate pizza as pizza was not at the time considered a low-carbohydrate option.
Call-back phone survey-
Pros:
Allows for feedback from a wide range of customers Delivery customers may provide useful responses, given that the call must be initiated by them Ability to reach customers who otherwise would be inaccessible Customers can remain anonymous if they choose to express negative views about the taste of the
pizza.Cons:
Donatos cannot guarantee that customers will be willing to take the time to participate in a call-back phone survey, thus non-response error could be very large.
The study did not indicate some form of incentive for customers, upon making the call. Failure to do this may not encourage customers to want to participate.
Customer-initialed calls create a self-selection sample which might not be indicative of the desired sample unit.
Given that interviewers are needed to facilitate the survey, Donatos may incur additional costs with this type of instrument
4. What measurement scales would you have used on the survey used as part of the in-restaurant product tests?
You can begin by discussing the different types of rating and ranking scales that may be appropriate for the in-restaurant tests, using Exhibit 13-2, Sample Rating Scales, as an initial guide. You can have students debate the usefulness of rating scales, such as the Likert or Numerical scales, where the likes and dislikes of a product can be easily rated. Some may argue that utilizing ranking or paired-comparison scales would be most effective, as Donatos could provide other types of pizzas or different low-carbohydrate ingredients and ask customers to make comparisons and choose their preferences. To begin the discussion you can have students brainstorm the
62
different types of pizzas Donatos would sell and have them compare the types of ingredients/toppings that would most appeal to low-carbohydrate eaters. You can also have them rate and/or rank their suggestions. This can serve as a preface to discussions relating to ranking and rating scales.
For the in-restaurant tests, a numerical scale or Likert scale would be the most appropriate measurement scales. Both would allow participants the opportunity to indicate their “liking” or “preference” attitude toward the product. The Likert scale would allow Donatos to compare a customer’s preference to others, while the numerical scale provides the flexibility of using both ordinal and interval data for the purposes of analysis. “Intention to purchase” is another dimension that can be easily measured by the Likert or numerical scale. Another important advantage of the use of these scales is the simplicity involved in administering them.
Case: Healthy Lifestyles
Abstract: The case provides state by state data from the Centers for Disease Control and Prevention's (CDC in Atlanta) Annual Behavioral Risk Factor Surveillance Survey. The student is asked to present a summary of the data.
You might start the discussion by asking students how their state (the one in which they are currently residing or the one in which their college is located) fares in the healthy lifestyle race. Unless you live in one of the six unrecorded states, your students will be interested in finding out where your state falls relative to others. The question helps students to personalize the data set, which makes them more interested in studying the numbers.
There will probably be at least one surprise in the numbers, no matter where you live. If your state is not listed, choose Michigan or Utah. Utah falls below the averages in all categories except seat belt use, and Michigan exceeds the averages except in seat belt use.
1. Report any interesting (i.e., unexpected, humorous, or odd) differences between the states.There are lots of unexpected results, too. North and South Dakotans, for instance, don't seem to like to wear
seat belts. Wisconsin has the highest DWI rate (but, of course, we don't know whether this is more drinking and driving or a greater willingness to admit to this behavior--maybe people from Wisconsin are simply more honest than those in other states!) Also, higher percentages of sedentary respondents in a state tend to relate to lower percentages of bingers.
2. Devise a weighted index of all seven lifestyle variables. The weighted index is to serve as an overall or composite measure of health lifestyles. Apply your weight to the states of Minnesota, Florida, and California as an example of what your weighted index shows.
This question will probably create a spirited debate. There is no right or wrong answer, but can be a marvelous lead-in to weighted averages (or, in this case, weighted proportions).
3. Discuss any noteworthy limitations of the survey or the data set.Students will have to face missing data and will be frustrated: the data set clearly indicates missing data for
Alaska, Arkansas, Kansas, and Wyoming, but fails to add the states of Nevada and New Jersey to the "missing information" list. The moral of the study--just because you've got a data set doesn't mean it's all there.
Your students will likely have more questions than answers by the time they finish digging through the data set. For instance, there is no real clue in the case about why six of the states were excluded, nor if those states were somehow atypical. The WEI variable is not well defined.
63
The means and standard deviations are:
SMK WEI SED ACT ALC DWI SEAMEAN 23.356 22.489 57.867 29.91 14.289 2.711 28.80STDEV 2.781 2.409 6.497 7.06 4.450 1.218 12.76
Most of the dotplots tend to be fairly symmetric and bell shaped in appearance, except for ALC, which looks to be somewhat bimodal. Most of the interesting outcomes are clear from direct state-by-state comparison of items in the raw data.
Several interesting correlations arise. SMK and WEI are positively correlated (0.435). Several of the cor -relations will be very difficult to understand; for example, ACT is the percentage of people indicating no leisure time activity. Your students will struggle with the correlation of -0.663 between ACT and ALC. Before leaving the classroom discussion, be sure that you mention the limitations of looking at correlations among aggregate quantities.
Case: HeroBuilders.com
Abstract: This case describes a study conducted by the president of HeroBuilders.com, whose entrepreneurial zeal led to the creation of an e-commerce toy company. Owing to the increased demands for the creation of “hero dolls,” research was conducted to ascertain the viability of marketing action figure dolls to the then competitive commercial environment.
This case relates well to Chapter 1, where students will be introduced to the role of research in business, and how the application of different types of studies aids in furthering effective research strategies. It also exposes students to the mechanisms involved in exploratory studies and the benefits associated with its use (Chapters 3, 7, & 8).
1. Which of the four types of studies are presented in this case?
You can begin by discussing the usefulness of Vicale’s exploratory findings and ask students to suggest other types of exploration they would pursue before launching such a business; and, how he benefited from completing this stage of his research. You can ask students to create a list of the information needs they would prepare before starting an e-commerce business to sell dolls made in the image of modern day heroes. The most appropriate study, as presented in this case, is an exploratory one. Here the researcher attempts to gain additional insight as to the viability of marketing an action figure. This type of study gives the researcher the opportunity to explore the commercial market of action figure dolls to determine the pros and cons of such a venture. It also presents other prospects where Vicale choose to conduct qualitative research, where he consulted with a lawyer to determine the legal implications involved with marketing dolls in the image of living heroes or political figures. Uncertain of the potential of starting an action figure business, Vicale also explored retail stores that marketed action figure dolls, to further determine the possibility of products gaining distribution. Here he benefited from knowing that an extensive action figure market did exist, and identifying the companies involved in the production of action figure dolls.
If you use this case during discussions of Chapters 3 and 8, you can also use this question as a preface to discussions of other data collection methods.
2. Evaluate the research that HeroBuilders.com conducted prior to launching its hero and villain action figures using the criteria in Exhibit 1-8.
64
The research conducted by HeroBuilders.com prior to launching its hero and villain action figures presents some limitations based on the criteria listed in the exhibit.
Purpose clearly defined:The purpose of the research was clearly defined. Here Vicale aimed to explore the potential of the action figure business. However he failed to indicate the scope of the research with regard to the creation of villain action figures. The case indicates that his initial survey of the market primarily involved only hero action figures.
Research process detailed:The case indicates that Vicale explored the e-commerce business to sell dolls by reviewing the Internet sites of other companies involved in the marketing of action figure dolls; he also looked at the retail environment, where he further discovered the extent of the action figure market. In addition, he sought legal advice to determine the implications of creating prototypes of political figures. There was no indication of an interview guide or a questionnaire used to guide data collection, nor was there evidence of an observation checklist used in his discovery trips to retail toy stores. This is a good time to discuss the inconsistencies that might be introduced in research studies—even exploratory ones—if the researcher doesn’t have a mechanism for collecting comparable data. You might ask students to develop a list of questions that Vicale might have used when, for example, he visited the action figure sections of toy or general merchandise stores.
Research design thoroughly planned: The research design’s plan also lacked important detail. It was not clear if considerable thought was given to how the research would be executed. Vicale conducted the research himself, which raises questions as to the objectivity of his results and whether his personal bias did not play a role in its reliability. The assessment of the retail environment (toy stores) that he explored did not make clear how he chose the types of stores visited, what process was used, or if location was a factor in his consideration.
High ethical standards applied:Given that Vicale conducted his research independently provided him much flexibility in executing this study, it would therefore be difficult to determine if high standards of ethics were applied.
The case did not indicate whether Vicale encountered limitations during his exploratory research. Students should be asked to detail what limitations Vicale might have encountered. They should have limited difficulty in listing such limitations as a limited sample of stores that this busy executive might have visited, or the failure to check out different types of stores that also might be used for distribution—other than toy stores, or his obvious potential bias toward the idea of producing such dolls.
HeroBuilders.com proceeded with the e-commerce business of making action figure dolls, which indicates that the mode of analysis (not revealed) applied to the data received may have been favorable. However, given that the research process was highly subjective as it was carried out by the owner of the company, coupled with the limitations evident in the research process and design, one may question the conclusions drawn from this study.
Some students know instinctively that not all research employs every criteria listed and many studies possess flaws although not crippling to the result. Moreover, some researchers may be limited by cost and time factors that consequently affect the usefulness of their research.
3. What other issues other than those Vicale chose to evaluate, would you have included in your research plan for HeroBuilders.com?
The decision made by Vicale, to capitalize on the action figure business with the incorporation of his modern day hero concept, was apparently given much thought. However, an across-the-board study of the costs involved in
65
such a venture could have been beneficial, considering the competitive nature of the existing market. In addition to conducting research on the marketability of dolls in the image of heroes, it may also prove helpful to explore how the general market would respond to dolls patterned after villains such as Osama Bin Laden and Saddam Hussein. Despite the numerous requests for villain dolls, this introduction could be damaging to the sale of dolls depicted in a “hero like image.” Considering the delicate nature of 911 issues, patriotic Americans may find it distasteful that villain images such as those introduced by HeroBuilders.com are marketed. Despite the apparent profit motive involved, this market should be further explored prior to the introduction of such dolls, to evaluate long term implications.
As outlined, a majority of the responses that Vicale received regarding the creation of the product came from “friends and acquaintances.” As such, it may have been worthwhile to perform a broader survey to establish other marketing factors (e.g., how much persons would be willing to pay for a such a doll, their primary motivation for purchasing such a figure and for whom they would be purchasing the doll, or how they would like to learn about the doll), but also how the general market of doll buyers felt about villain dolls. Vicale could have also taken his research of the retail market a step further by exploring the cost/profit factors with individual stores and web sites that marketed action figures and toys.
Case: HiTech Engineering
Abstract: The student is asked to evaluate the effectiveness of a variety of promotional approaches used by this designer and manufacturer of industrial products. Based on the data, the student is asked to identify the most effective method currently employed or a combination of approaches.
1. What is the most effective type of advertising for HEI to undertake? Do you have any recommendations on the appropriate mix of the five different types of advertising?
The answer to the promotional method evaluation depends on how students interpret the word effective. If the goal is to find the media with the highest average number of responses (leads), then postcards are the way to go. If the goal is to minimize costs per response generated, then news releases, postcards, and literature all appear to be viable.
A nice lead-in to the discussion of this case is to ask: "What do you think management had in mind when they used the word effective, and how did your interpretation factor into your analysis?" Let the students discuss the issue of responses versus cost per response.
At some point, lead the discussion toward viability of ANOVA with unequal variances. Ask at some point whether there are any problems with the data. (Postcards have three zero values, which is unusual given the apparent effectiveness of postcards. One might suspect that these were actually missing observations replaced with zeroes in the coding of the data set.) Ask how these problems did, or should have, influenced the chosen statistical methodology.
It is crucial to mention that these data may not answer the real question that should be asked--how many dollars in sales, not just leads, are being derived from HiTech's advertising expenditures? While personal selling is a key promotional method for most industrial products, and the success of various sales representatives differ, the degree to which "leads" generated by a particular advertising approach is not addressed by the data.
Results are suspect for several reasons. Variances are unequal between groups and some of the data might be wrong. These issues can generate lively classroom discussions. A very important point about this case, which only a few of your students will identify, is that the data set doesn't really answer the critical question. The bottom line to management is whether spending translates into sales, not just requests for more information. This issue cannot be addressed with the given data set. The data set is also amenable to ANCOVA methods.
66
Typical Results and Approaches
ANOVA rejects equal means for both RESPONSE and $/LEAD (a generated variable defined to be SPEND/RESPONSE). Average RESPONSE is greatest for postcards, and average $/LEAD is lowest for postcards, news releases, and literature, as illustrated by the printouts that follow.
Analysis of Variance Printout for RESPONSE
ANALYSIS OF VARIANCE ON RESPONSE SOURCE DF SS MS F p MEDIA 4 1885716 471429 35.47 0.000 ERROR 80 1063246 13291 TOTAL 84 2948962
INDIVIDUAL 95 PCT CI'S FOR MEAN BASED ON POOLED STDEV LEVEL N MEAN STDEV -+---------+---------+---------+----- MAGS 17 74.4 34.7 (---*---) POST 17 445.9 239.3 (---*--) EDIT 17 46.0 28.7 (---*---) NEWS 17 94.6 43.7 (--*---) LIT 17 88.8 72.5 (---*---) -+---------+---------+---------+----- POOLED STDEV = 115.3 0 150 300 450
Case: Inquiring Minds Want to Know—Now!
Abstract: This case describes a multi-stage, communication study undertaken by the research department of Penton Media, a publisher of business trade magazines, to determine the long-term viability of a reader and advertiser service, the reader service card, a post-card sized device used by readers to request additional information from a particular advertiser.
The discussion questions guide the student through the research process. As such, the case may be used throughout the course—especially through Chapter 14—simply by assigning different discussion questions as exercises during different points in the course.
1. We suggest using Exhibit 3-2 for this discussion; it is helpful if you can project the graphic on the screen in front of the class. A. The management dilemma is a declining number of reader service cards returned which is causing a
smaller number of inquiries and thus a smaller number of sales leads for Penton advertisers. Students won't have too much difficulty in identifying the management dilemma, but you might use this opportunity to discuss that research can be proactive—actually preceding the development of a problem. If Penton saw this as a value enhancing service for advertisers, shouldn't they have been tracking the change in rate of response card return at the very least? Penton also should have been tracking advertisers' continued interest in the reader service card program--after all, the reader service card program has a cost associated with each issue (card printing, return postage, and card handling and forwarding) that needs to be offset by perceived value on the part of the advertiser. Students are usually quick to point out that advertisers don't always track the success of their advertising (not as true an
67
observation for business-to-business advertising as it is for consumer advertising) because not all advertising asks for a specific action. But in the case of advertising designed to generate leads, Penton clearly had the ability to track reader service card activity (a research activity in its own right).
B. Management question: What should be done to ensure that advertisers and readers alike are getting the information they need and want?
C. Research Question: Should the reader service card program be maintained, discontinued or modified? Rather than three alternatives, Penton may have only looked at maintaining or discontinuing the reader service card. Nothing in the questionnaire indicates that they were exploring other unspecified options.
D. Investigative questions: What means do advertisers offer to obtain information about advertisers' products and services? What influences a trade magazine reader in their choice of response method? What types of information are most frequently sought? What is the time frame in which information is needed? Do purchasing agents have different needs than supervisors? How many advertisers offer web access to information? How many readers use company web sites for information? Are readers changing their methods of response in the last 5 years? 5-10 years? Do shifts in response methods parallel the decline in reader service card returns? Does a reader's gender, age, or job experience affect their choice of information retrieval method? To what degree do advertisers value the reader service card program?
E. Measurement questions: see instrument in text.
2. Using Exhibit 5-1 is appropriate for this exercise. Since Penton is conducting this research internally, most of the sponsor's rights are covered. Subject's rights are much bigger issues here. Avoiding Subject Deception: Subjects are all subscribers to Penton's trade magazines, and the names are
drawn from the magazine's subscriber lists (the sample frame). The cover letter states that the study is being conducted to help "companies better understand and respond to your request for information." While it doesn't clearly state the reader service card is being considered for elimination, modification or replacement, any reader who has been reading the magazine for any length of time will know that the reader service card is one response option likely to be evaluated. This small deception is unlikely to offend or distort data from a reader responding to the survey.
Subject's right of informed consent (implied): The cover letter invites readers to participate, so they have the right to choose.
Students may want to discuss the right of privacy, believing that the magazine has used information extracted from reader service cards and subscriber records in ways that the reader never intended.
Subject's right to confidentiality. Penton offered an inducement--a drawing for a hand-held color TV--to participate, but that is not likely to alter the information that respondents are willing to share. But to obtain that entry right, the respondent must provide their contact information ("To ensure a correct entry in the random drawing for the held-held color TV, please make and necessary changes to your mailing label."), which affects their right of confidentiality.
3. The sampling plan called for using the subscriber database (1.7 million) as the sample frame. The case clearly specifies that they used a stratified disproportionate random sample, sending out 4000 surveys to obtain the 710 completed surveys of which 676 were considered useable (came from purchasing decision-makers). You can also discuss the sampling plan for the stage 2 study of ad content. And students might want to discuss why they didn't draw a sample of advertisers to participate in a parallel study.
In terms of sample size, Penton mailed 4000 magazine subscribers and received 676 usable out of 710 completed survey. Penton chose a stratified sample in order to check response patterns in different subsets (42 in all) of the business-to-business market. You can use this opportunity to discuss several sampling issues: The effect of such stratification on sample size. The effects of self-selection within a mail survey on the quality of the data (and non-response error). The importance of data preparation and why certain returned surveys may not be used. In Penton's case, the
discarded instruments were not completed by purchase decision-makers, the only qualified respondent from the viewpoint of the advertisers.
68
The effect of error as it is introduced by the quality of the sample frame.
4. Penton had started their project with exploratory research that revealed that advertisers perceived they were getting fewer viable sales leads with the advertising in 1998 than they were in earlier years. Penton could have been conducting monitoring research with their advertisers or might have been receiving increasing complaints or comments in normal dealings with them.
Stage 2 involved an observation study of 1330 past advertising placed in Penton magazines in two years, 1992 (648) and 1997 (690). This did show a change in advertiser behavior. What was missing at this stage was an indication of change in respondent inquiry behavior. This would lead us to believe that as Penton did not data mine for this information, that it was not tracking the ads that generated response card inquiries. Having the response cards in their possession it would have been quite easy to do a tally on a periodic basis for each issue's reader service cards.
Stage 3 involved a mail survey of subscribers, which involved an inducement to participate--a hand-held, color television giveaway. The mail survey was pretested in two ways: by phone, and then by mail. This is a perfect time to discuss: Various types of pretesting using Exhibits 14-4 and 14-9. Why the mail survey was tested twice Why a mail survey was preliminarily tested by a means other than that chosen for the survey itself. When such inducements might cause error, when they might inject ethical issues, and what typical
inducements are used: money, merchandise, and coupons for discounts are common.
Stage 4 involved 40 personal interviews "to gain a deeper understanding of their behavior and attitudes." This is the perfect time to discuss the limitations of a survey for collecting the nuances of attitudes, and the strengths of the personal interview communication methodology.
5. The easiest way to conduct this exercise is for the students, armed with the issues list from their text, to critique the data collection instrument as an out-of-class assignment. You could have them critique the whole instrument, or you might divide the class up into teams, with several teams dealing with a question-level critique and others dealing with the instrument as a whole. The following checklist might be used for such a critique.
Question Level CritiqueShould this question be asked?
Issue 1: Purposeful vs. Interesting Is the question of proper scope and coverage?
Issue 2: Incomplete or unfocused Issue 3: Multiple Questions Issue 4: Precision
Can the respondent answer adequately?Issue 5: Time for thought Issue 6: participation at expense of accuracy Issue 7: presumed knowledge Issue 8: Recall and memory decay Issue 9: Balance (general vs. specific)
Issue 10: Objectivity Will the respondents answer willingly
Issue 11: Sensitive information Issue 12: Shared vocabulary Issue 13: Unsupported Assumptions
69
Issue 14: Frame of Reference Issue 15: Biased Wording Issue 16: Personalization Issue 17: Adequate Answers Issue 18: Objective of the Study Issue 19: Thoroughness of Prior Thought Issue 20: Communication Skill Issue 21: Respondent Motivation
Instrument Level CritiqueIntroduction & screening Instructions Order/question sequencing Transitions between sections Conclusion & disposition of instrument
On an instrument level, students should compare their investigative questions developed in discussion question 1 to identify whether the boundaries of information request methodologies have been adequately covered. It appears the researcher was very thorough in this regard. Although the layout has been modified to fit the page format of the text, students should look at the layout of the instrument in terms of structure. The layout of the scaled response strategy is clear, and questions clearly delineate (bold text) the time frame of the intention or actual behavior.
Students should be asked to address the instrument's scope: Does it include all necessary questions? Are questions included that seem extraneous? Student's might observe that the instrument scope goes farther than might be technically necessary by asking the desired information response that advertiser's could provide. This question (3) fulfills the "what's in it for me?" query that most respondents ask prior to participating in a survey. As a result, it might be better placed earlier in the instrument. Others might ask why the respondent is not asked directly how they would feel about eliminating the reader service card.
This is the opportune time to ask whether the communication method (mail) would have required a different order or instructions if the survey had been done by phone. While early placement of the screening question (1) would serve a purpose in a telephone survey, in this instrument it serves as classification data and could better be placed in the end. Also, an interviewer instruction sheet, for repeating response scales, offering skip directions, probing for reasons for chosen method of response would be necessary if the survey was done by phone.
Concerning instructions and transitions, if a respondent were to check the 4th column in question 4 for every option, it doesn't tell them to discontinue. Skip directions also could have been offered in question 6a, to allow the respondent to skip question 6b. Students should be asked about the abrupt change between the core target questions and the classification questions. The purpose of a better transition between the target questions and the classification questions is to indicate why or how the personal information will be used, in order to gain full participation in these questions used for measuring association. This instrument loses that opportunity by its abrupt transition.
Finally the students should discuss the end of the survey. Both a conclusion and disposition instructions should follow the last question, telling the respondent, again, to return the completed survey in the postage-paid envelope and thanking them for their participation. It wouldn't hurt to repeat the request to correct the address label to insure an adequate entry for the television drawing. The information that is captured with the respondent's identify will provide other crucial association variables.
70
At the question level, ask your students if they feel the chosen response strategies (multiple choice-single response, checklist, free-response) are appropriate, and why or why not. Additionally, ask your students if any operational definitions are missing. Some students, for example, may not know what 'fax-on-demand' is. You might ask them to come up with operational definitions for those terms they feel the respondent might not know or which might cause confusion. Students should be able to determine if the correct concept and construct has been measured in each question (increase/decrease in activity for question 4, use/expected use of Internet in question 7, etc.).
6. Have your students code the survey for analysis, identifying the number of variables, the numerical codes for each likely response, and the variable labels. This is also a great opportunity for discussing the coding of free response questions as questions 2, 4b, 5a, 5b, 7b, 10, and 11 use the free response strategy.
Q # V Data Q #V Data Q # V Data Q #V Data1 7 Nominal 4b 15 Ordinal 6b 1 Nominal 9 1 Nominal2 45 Nominal 5a 1 Nomina
l7a 8 Nominal 10 1 Ratio
3 16 Ordinal 5b Nominal
7b 8 Nominal 11 1 Ratio
4a 15 Ordinal 6a 1 Nominal
8 1 Nominal
7. This question asks the student to build a preliminary analysis plan then fulfill it. It also allows you to discuss whether it is the researcher's role to provide a recommendation or merely to report the findings. You can also use this question to compare textual presentation (as is used by the case) with tabular and graphical formats. Finally, ask this question to determine what should be done with those original 42 subsets of interest, and what might have been used as the factor of incidence for this study. You might want to ask students to prepare both tabular and graphic depictions of the data and compare the results. Without a direct question about eliminating the reader service card any recommendation is problematic given the data presented. However, one statistic will stand out as revealing to the student: between 69-71% responded by mail during the last year when they didn't have an immediate need. This might explain why advertisers don't perceive reader service card respondents as good sales leads--these inquires may be primarily from potential purchasers whose need is not clearly defined or whose purchase is too far into the future to connect with that early reader service card inquiry. Is this enough to make a "continue the reader service card' recommendation? See below.
8. Every survey has limitations based on scope and methodology. This is a perfect time to discuss sources of error in this survey. The limitation of most concern is the non-response error. That same 71% who had used a reader service card in the past year looks very different when you look at the total original mailing. The 494 respondents translates to only 12.3% of the original sample. 710 of the original 4000 respondent, 17.75%. One could hypothesize that those who did not respond feel that they have sufficient ways to reach advertisers about their products or services, therefore they are not interested in preserving one method versus another. Could those who responded have some ulterior motive for wanting to keep the reader service card? You should also raise the concern about weighting the responses based on the disproportionate stratified sample that was drawn. Does this create or solve a limitation?
9. Ask your students if a decision could be made based on the information provided. This is an opportunity to discuss the risk associated with decision making in the absence of perfect information and the value of a decision. Penton obviously thought there was some risk to this decision or they wouldn't have undertaken such a comprehensive study. Some students might suggest an experiment at this point. Create a split run of
71
an upcoming issue, one group gets ads tied to a reader service card while a second group gets ads not tied to a reader service card. Be sure to bring out additional cost (two different ads created for each advertiser, shorter production runs, more time before making a decision on the issue at hand) of this experiment during the discussion. Other students might suggest tracking the returned reader service card use during the next several months (another descriptive study). Others could argue that the appropriate sample for the original study was incorrect, that it should have been advertisers; if advertisers thought the reader service card was no longer of value, then, and only then, should the device be eliminated.
Case: Mastering Teacher Leadership
Abstract: A multi-stage, communication study of teachers by Wittenberg University's Department of Education to determine the viability of starting a Master of Education program for Ohio-certified teachers working within school districts serving a five-county area.
The discussion questions guide the student through the research process. As such, the case may be used throughout the course—especially through case 15—simply by assigning different discussion questions as exercises during different points in the course. With the comprehensive data set available on the CD, this case can also be used for the data analysis chapters.
Not all research projects are well designed, and this one has some very obvious flaws that will be revealed during the discussion of the seven discussion questions. This is a good case to use to discuss whether questionable data helps reduce the risk of poor decision making. You can also tie this case back to the concerns in Chapter 1 about research being done by those untrained in research. Those in charge of this project were all highly educated, and they had been schooled in their doctoral programs to do research for their dissertations. But they were clearly not trained in business research methods as the research design, sampling and survey instrument demonstrate.
5. We suggest using Exhibit 3-2 for this discussion, by projecting the graphic on the screen in front of the class. Management dilemma: This is research based on an opportunity rather than a problem, so the symptom
which starts this research is the passage of a law by the State of Ohio that requires teachers to obtain a master of education degree prior to their second licensure renewal between years 5 and 7.
Management question: What should be Wittenberg's role in meeting teacher certification requirements? The survey straddles the issue of professional development coursework vs. master of education degree coursework.
Research question: Should Wittenberg offer a Master of Arts degree in Education? Investigative questions: Here are some questions your students should generate. How many teachers will
need to obtain a Master of Arts degree to become re-certified within the next five years? What do teachers seek in professional development programs in general and what would they seek in a masters program? How many teachers will pursue a Masters degree? How likely would these teachers be to attend Wittenberg? How many teachers are currently enrolled in masters programs? Why did they choose the programs they did? How many teachers are in the market area, especially Clark County? When would teachers be likely to take courses? How far will they drive to take courses? What are the price, structure, and content of competitive schools' Master of Arts in education programs? How many teachers might leave the profession (or Ohio) rather than comply with the new standards?
Measurement questions: see survey.
6. This question is designed to reveal the purpose and methodology for the exploratory data stage of most research projects. This question can be used with material from Chapter 7, especially Exhibits 7-1 and 7-2. You might first ask students to detail which information from their list of investigative questions (Q 1 above) might be extracted
72
from a secondary source. Then you might ask: What information drawn from a secondary source is profiled within the case? How and where/from whom might that information have been collected? You also might ask students to apply the five criteria for evaluation (purpose, scope, authority, audience and format) to each element of secondary data and its source. And you could ask students to construct query statements to do an electronic search for likely secondary data during this exploratory phase of research. Finally you can ask students to summarize the exploratory findings.
Information WittCPD collected and profiled in the case came from the following sources: Websites and catalogs provided competitor information on number, structure, format, and content of
Master of Arts programs in education. The school boards representing the various school districts in the market area provided turnover and hiring
information. The Ohio Board of Regents and the Ohio Department of Education provided documents detailing the new
teacher certification standards. Student teacher evaluation forms collected during the 1990s provided teacher attitudes about quality of
Wittenberg's undergraduate teacher preparation.
7. WittCPD obtained a mailing list of all 1600 teachers in the county--likely from the various school boards. First, students should evaluate the sample frame. There is no mention in the case that this sample frame grouped the teachers by age, years of teaching experience, area of expertise, or any other classification variable that was pertinent to the study based on the new teacher certification standards.
You may use the tabular Exhibit 15-1 for this discussion. Each teacher received a survey, so in essence WittCPD allowed each teacher to self-select themselves into the sample rather than designing a sampling plan. This is a good place to discuss census vs. sample. Students should then be asked to define the type of sample WittCPD obtained. Some will argue that the results are a simple random probability sample because every teacher had an equal chance of being included in the sample. Others will argue that WittCPD started out taking a census but didn't follow through and what is left is no better than a non-probability convenience sample at worst, or a purposive judgment sample at best. If we look at one measure of sample quality, accuracy, the very large non-response error makes us question this sample's accuracy. Some students may argue that WittCPD used "current teachers" as a surrogate for teachers who will really be affected by the new standards. Teachers who will be affected will be first certified in or after 2002. Most of those plan-to-be teachers would have not yet enrolled in an undergraduate program.
8. This question uses the chapters on measurement, scaling, and instrument structure. A. The cover letter clearly states the two-fold purpose of the study, but it could have been written from a different
perspective--the teachers. If so, the letter could have alluded to the new state requirements and Wittenberg's assessing their role in helping area teachers comply. Several of the students will challenge the use of classification data at the beginning of the instrument, especially if one purpose of the early questions is to build interest and motivation to respond to the target questions. If WittCPD had used question 3 to screen respondents that would have no interest in a masters because they already possessed one, it would make sense to have classification questions early. But, as the absence of branching or skip directions indicates, this was not done.
Students should assess the length of the instrument. The actual layout of the instrument took 3 pages, which on receipt might have appeared excessive. Ask your students to "pretest" the length of the instrument and determine whether the cover letter was sufficient to encourage participation in an instrument of that length. No transition appears between classification and target questions, but the survey does have a conclusion and disposition directions. The ranking questions do possess sufficient directions.
73
B. With reference to response strategy appropriateness, numerical ranking, multiple choice-single response, and checklist strategies are all appropriate for the data they wanted. But clearer instructions (check one only) could have been provided for question 13. Changing constructs within a question are troublesome (e.g. question 6 asks about enroll, but responses offer the construct apply).
Clearly WittCPD is concerned about the market's perception of price as being too expensive. Students might raise the issue of whether discounting cost as an issue (questions 5, 6) the designer further elevated the issue.
In ranking questions, three is an acceptable level. It might have helped if the teacher had been asked to put all 7 elements on a ranking question--without including the other to muddy the understanding.
Given the space, question 9 would be more informative as a rating scale that a checklist. We don't know how troublesome each checked item might be.
9. Data coding can be part of this exercise, at least to the point of identifying the number of variables related to each question in the instrument. Of course, savvy students could go to the data set and count the number of variables (76, ignoring CASE).
Exhibit C-WittCPD-1 provides the actual code sheet for the survey. Without this your students must guess at the variables in the dataset.
The preliminary analysis plan should start with descriptive statistics: frequencies with cross-tabluations against the variables in questions 1-3, as most of the data is nominal or ordinal. There might be enough information from the various school boards to determine if the sample is representative or if the frequencies of some subgroups need to be weighted more than others in looking at the total data set. The student's preliminary plan should detail how missing data will be handled with internal coding. Obviously some subsets of the sample (those without the Masters degree, those with few years of service) would be of more interest than others.
10. Students will see the real-life application when analyzing a real data set, even one with some questionable data. This is good preparation for analysis of their own data, and will point out the problems with preliminary analysis plans. This is a good exercise for interpreting hypothesis statistics, too, and to demonstrate that a researcher can crank out any statistic they desire but that some are not relevant to the data type provided.
11. Because they started with a geographic variable (address), and because respondents have been asked to provide their names and addresses if they want more information, mapping data using Geographic Information System (GIS) software is possible. Because various service divisions of the City of Springfield have done several GIS projects with the University, GIS codes for every address in the county are readily available. So a plot of those who have no degree yet show a high level of interest by giving their name and address would be possible. Using the GIS block codes for those people and referencing it with the location of teachers with similar patterns might give WittCPD information that could be used for promoting the new program, if they decide to move forward. You might assign students to check for current status of the project at:
http://www5.wittenberg.edu/academics/educ/masters.html.
Exhibit C-WittCPD-1: Variable and Value Codes for WittMasters.xls
1. Counting this year, how many years have you taught? (V=Q1)=1 0-5 years =2 6-10 years =3 11-15 years =4 16-19 years =5 20 or more years
2. Professional Responsibility and Subject Field. Check all that apply.
74
Grade Level (V=Q2GRADE) Subject Area (internal code 1=checked, 2=no checked)
Preschool to Grade three =1 Art (V=Q2_1SUB) Music (V=Q2_6UB) Grade four to Grade eight =2 Business/Economics
(V=Q2_2SUB) PE/Health (V=Q2_7UB)
Grade nine to 12 =3 English (V=Q2_3UB) Social Studies(V=Q2_8SUB)
Special Education =4 Languages (V=Q2_4UB) Science (V=Q2_9SUB)
Administration =5 Mathematics (V=Q2_5UB) Other ____________________
Other: =6__________________________________________
3. Highest level of education obtained. Please select one from the list below.(V=Q3, internal code: 1=checked, 2=not checked)
Less than a B.A./B.S. B.A./B.S. B.A./B.S. plus graduate work Currently in M.A./M.S. program M.A./M.S. Ph.D. or currently enrolled in Ph.D. program. M.A./M.S. plus additional graduate work
If you are currently enrolled in a graduate program, which college or university are you attending?______________________________________________________________________________
4. Which of the following qualities are most important to you in a graduate program? (Please rank the top three qualities with "1" being of most importance, and"2" being of next most importance, etc.) (Value codes internal code: 1=checked, 2=not checked)
____ Reputation (V=Q4_1) ____ Quality of Instruction (V=Q4_4) ____ Class Size (V=Q4_6)____ Schedule Flexibility (V=Q4_2)____ Cost (V=Q4_3)
____ Closeness to home (V=Q4_5)____ Other (V=Q4_8)
____ Individual Attention (V=Q4_7)
5. If costs were kept competitive, how likely would you be to apply to a master's degree program in education at Wittenberg? ____ Other (V=Q5)
=1 definitely would apply =2 might apply =3 would not apply
6. If costs were kept competitive, how likely would you be to enroll in graduate courses at Wittenberg to enhance skills without pursuing a master's degree? (V=Q6)
=1 definitely would apply =2 might apply =3 would not apply
7. Please indicate the three most important reasons for your interest in graduate education at Wittenberg, with "1" being your most important reason, "2" your next most important reason, etc. (Value code is actual rank)
__Professional requirements (V=Q7_1) ____ Increased employability (V=Q7_6)__Professional advancement (V=Q7_2) ____ Additional money (V=Q7_7)__Personal satisfaction (V=Q7_3) ____ Keep certification (V=Q7_8)__ Future requirement (V=Q7_4) ____ Upgrade certification (V=Q7_9)__ Career change (V=Q7_5) ____ Improving skills (V=Q7_10)
75
Other: (V=Q7_11)________________________________________________________________
8. Please check the two most important reasons for your lack of interest in graduate education at Wittenberg? (Value code is actual rank)
____ Cost (V=Q8_1) ____ Live too far away (V=Q8_5)____ Family Responsibilities (V=Q8_2) ____ Too near retirement (V=Q8_6)____ Time to complete the degree (V=Q8_3) ____ Lack of information (V=Q8_7)____ Professional Commitments (V=Q8_4) ____ Already have a master's degree (V=Q8_8)
____ Enrolled in master's program (V=Q8_9)Other: (V=Q8_10)_______________________________________________________________
9. Please indicate from the list below which of the following might be anticipated as an obstacle to your enrolling in a master’s level or graduate classes at Wittenberg? (Value code: 1=checked, 2=not checked)
____ Child/Elder Care (V=Q9_1) ____ Travel (V=Q9_4)____ Financial Need (V=Q9_2) ____ Employment Schedule (V=Q9_5)____ Family Commitments (V=Q9_3)
Other: (V=Q9_1)_______________________________________________________________
10. What professional development areas most interest you? Please rank the top three professional development areas that interest you, with "1" being your are of strongest interest, "2" being your area of next strongest interest, etc. (Value code: 1=checked, 2=not checked)
__ Enhancing subject matter knowledge (V=Q10_1) __ Teaching Arts (V=Q10_8)__ Using Technology in the classroom (V=Q10_2) __ Teaching Social Studies (V=Q10_9)__ Child development (V=Q10_3) __ Teaching English/Language Arts (V=Q10_10)__ Teaching reading/writing (V=Q10_4) __ Teaching Math (V=Q10_11)__ Specific learning disabilities (V=Q10_5) __ Teaching Science (V=Q10_12)__ Teacher Leadership Development (V=Q10_6) __ Urban Social Backgrounds (V=Q10_13)__ Developing social skills in students (V=Q10_7) __ Moral and Character Development (V=Q10_14)
Other: (V=Q10_15)_______________________________________________________________
11. How far would you have to drive to attend TC? (V=Q11)
=1 under 10 minutes =2 10-20 minutes =3 21-30 minutes=4 31-45 minutes =5 46-60 minutes =6 more than 60 minutes
12. When during the year would you be able to take graduate courses? (Check all that apply.)
Fall (Aug. - Dec.) (V=Q12_1) Spring (Jan. - April) (V=Q12_2) Summer (May - July) (V=Q12_3)
13. Which day and time scheduling option below most appeals to you? (Value codes: 1=checked, 2=not checked)
(V=Q13_1) Fall through Spring: Day (8:00 a.m. - 4:00 p.m.)(V=Q13_2) Fall through Spring: Late Afternoon (4:00 p.m. - 6:00 p.m.)(V=Q13_3) Fall through Spring: Evening (6:00 p.m. - 10:00 p.m.)(V=Q13_4) Fall through Spring: Saturday(V=Q13_5) Summer Day (8:00 a.m. - 4:00 p.m.)(V=Q13_6) Summer Evening: (6:00 p.m. - 10:00 p.m.)
76
If you would like to receive more information about graduate programming in education at TC, please put your name and mailing address below.Name: ______________________________________________________________________________Address: ______________________________________________________________________________Thank you for your time and assistance. Please return the survey in the postage-paid envelope by March 20.
Case: Matching Wits with Jason on Sampling Theory
Abstract: This case takes the format of a debate between researcher Jason Henry and the new market development VP of a bank, Jasmine "Jazz" Rogers. They are discussing the appropriate sample size for a study required by the bank of small business applying for a loan to expand its operations. The student is asked to anticipate sampling decisions and explain the rationale for each sampling decision.
This case is a perfect opportunity for students to delve into sampling theory, including 1) when the various formula are used, 2) the concept of incidence and how it affects sample size, 3) the role of pilot samples to determine incidence measures (or what to use when no such measures exist), 4) the effect of small population sizes when drawing probability samples, and 5) how data collection procedures, like callbacks, affect sample size.
You might assign this case as an in-class exercise, dividing the class into teams with one set of teams playing Jason's role, with the second set of teams playing the role of Jazz. Each team could be asked to write the dialog of their character at the appropriate place in the story or list the points that their character must make to win the debate.
Below each challenge we've included the dialog that might actually follow each debate challenge, as well as what you might expect the students to argue.
1. Well, we've set the stage for you. Now see if you can anticipate how Jason will convince Jazz to accept a small sample rather than a larger sized sample she obviously has in mind. Assume Jason wants to win this battle of wits. What would he argue?
The students' natural inclination will be to bring up the idea of sample size being calculated using arbitrary decisions about desired confidence level and precision (acceptable confidence interval around the sample estimate).
Given the nature of the study being discussed, a portion of the students' dialog should focus on a likely incidence factor. Since the Chevron manager, Audrey, wants to expand station hours and stay open from 5 a.m. to midnight, a logical factor of incidence will likely be work-related (e.g., % of residents who go to and from work before 6 a.m. or after 9 p.m., assumed current hours) or lifestyle-related (e.g., % of residents that currently purchase car-care services at times other than 6-9, or % of residents who drive for pleasure reasons before 6 a.m. or after 9 p.m.).
After the discussion and sharing of ideas, share the following story continuation with your students:
Jason: “Then let me try. Take the first question you want. ‘Are you currently retired from work?’ This is what we call a proportions question. That is, a certain proportion of people will say they are retired.”
Jazz: “Yes,” said the banker. “Suppose 20 percent of the existing customer base says they are retired. I would take that as a very bad omen for a station that opens early and closes late. Why would a retired person want to drop his car off at 6 am?”
77
Jason: “Exactly . . . well, say 20 percent of the people sampled say they are retired. That’s a proportion of .20, right? This means that, of the full 1,000-odd customers who comprise the ‘population,’ the actual proportion falls in a confidence interval centering around .20.”
2. Assume Jason interrupts Jazz before she specifies her error interval. What will Jason come back with if he is mentally calculating sample size?
The student's natural inclination will be to estimate the interval based on the fact that Jazz chose the most commonly used confidence level (95%). Jason is trying to convince Jazz that a small sample size is appropriate for Audrey, the Chevron station owner. So he'll have to make a larger confidence interval acceptable to Jazz. A standard acceptable confidence interval (+ 3%) would require too large a sample size.
After the discussion and sharing of ideas, share the following story continuation with your students:
Jason: “So you will settle for a 95 percent confidence interval. You remember your statistics, I can see. Well, we need to compute the standard error of the proportion in order to compute the width of the confidence interval. To compute the standard error, we will use the formula. . . .”
3. Is Jazz correct? Or has Jason won this point? What will Jason tell Jazz to do to improve her estimate?
The students should be encouraged to recreate Jazz's formula. To do so, they need to first realize that Jazz is estimating the confidence interval using her hypothetical incidence factor of 20% retired people in a hypothetical sample. To do this, she needs the z value for 95% confidence (1.96). And Jason's proposed sample size of 100. Even though the z value is 1.96, Jazz is using 2 as a rough estimate (+ 1.96 from the sample estimate includes 95% of all estimates in the distribution). This is an opportunity to use some simple algebra on the sample formula for proportions and recreate it solving for confidence interval.
After the discussion and sharing of ideas, share the following story continuation with your students:
Jason: “That’s very good, Jazz, and I’m going to let you keep that straw. But you failed to consider a significant factor. Would you care to improve your estimate?”
4. Assume Jason lets Jazz keep the straw even though she could have done her calculation differently. What might be considered Jazz's error(s)?
78
Jason believes Jazz can improve her estimate based on the fact that the population size is small, 1000. At least one student will remember (or you can remind them) that when a calculated sample size is greater than 5% of the population size that you can reduce the sample size without sacrificing the quality of the data.
At least one student will note that Jazz took liberties in calculating her estimate and suggest these were not appropriate shortcuts to take. You want to make sure that students knew she was using 2 as an estimate for 1.96.
After the discussion and sharing of ideas, share the following story continuation with your students:
Jazz: “Hah,” she said, “I should have used 99 instead of 100. ‘N – 1.’ Hah!” and she reached for another straw from the tumbler, but Jason snatched the straw from her hands and set it on his side of the desk. Now each of them had a straw.
Jason: “That’s too small a correction to matter. You lose.” Jazz: “OK,” she said with mock annoyance, “I should have multiplied by 1.96 instead of 2. That’s it,
isn’t it?” And she reached for another straw, but Jason clapped his hand over the tumbler. Jason: “Sorry,” he said, “that’s true, but again too small a correction to bother with. You should have
allowed for the fact that she is sampling 100 out of a population of 1,000. That shrinks the standard error of the mean by the square root of 900/999, which comes to. . . .”
Jazz: “Too small a correction to bother with,” shouted Jazz with glee. “Take your hands away from my straws!
5. Jason's being put through his paces here. What should his answer be this time? Look at the nature of the second question to determine the correct formula to use.
The second question, ‘What time do you leave your home in the morning to go to work?’ will generate a mean estimate rather than a proportion, so Jason needs to apply a different formula. This mean time is a second incidence factor.
If the interval estimate around the mean time is unacceptably large, then the sample size will need to be increased from the 100 proposed, even though the confidence interval around the proportion of retired customers (+.08) was acceptable to Jazz. This is a perfect time to indicate that when there is more than one critical measurement question that you always accept the largest sample size that is calculated to ensure the necessary confidence and precision in all estimates generated by all measurement questions.
After the discussion and sharing of ideas, share the following story continuation with your students:
Jason: “Well, we use a slightly different version of the formula this time. This time we divide the estimated standard deviation of the error by the square root of 100 to get the standard error of the mean.”
6. This is Jason's last chance to impress Jazz…and win. What must he know and where might he have gotten this information, for a quick answer to the standard error of the sample?
79
Jazz has challenged Jason that “one can't know the standard error of the sample until the sample has been taken.” But we know that prior knowledge will almost always reduce sample size. So Jason wants a proxy measure for this purpose. The value of prior knowledge is why large proposed studies almost always include pilot tests--not just to refine instruments but to collect critical information about incidence factors, reducing the sample size needed and thereby reducing the costs associated with large studies.
After the discussion and sharing of ideas, share the following story continuation with your students:
Jason: “Well, then you are wrong. Because I happen to know that the morning rush hour extends from about 6:30 to 9:30, a period of 180 minutes, and applying the rule of thumb that the standard deviation is roughly one-sixth of the range, I arrived at a rough value of 30 minutes for the standard deviation, which gives a standard error of the mean of 3 minutes, which is all the precision you could possibly want.” He reached for the straw, but this time she pulled it away.
Jazz: “Wait a second, Mr. Statistician. Where did you get that rule of thumb?” Jason: “I don’t know,” he admitted, “I picked it up during my graduate student days.” Jazz: “Hah,” she exulted. “Keep your straw-pickin’ fingers away from my straws!” Jason: “However,” he said, “on returning to my office I did have a twinge of doubt about giving this
advice to Audrey, so I called the planning department of the local bus company, which keeps enormously detailed statistical records as part of the requirement for their federal transportation subsidy, and they confirmed that the ridership of their number 45 bus through our town, and indeed the ridership of practically every other bus on the system, has a standard deviation of . . . 29 minutes.” He extracted the straw. Now he had two, to her one.
7. So what's Jason's final rebuttal?
Encourage your students to look at the data collection procedures. A mail study is being discussed. You might mention that initial response to such a survey is often well below the 50% that Jazz is estimating. Ask students what might be done within the context of a mail study to increase response. Students have often received mail studies themselves and will share the inducements that came with the survey: money, entry into a drawing for a vacation giveaway, coupons for theme park visits or free meals or merchandise.
Duplicate mailings will also likely be suggested, but you should remind your students that Audrey is trying to keep her costs down. She is, after all, borrowing money to erect improvements so she can keep her business open longer hours to increase sales and profits.
Callbacks are an obvious choice since 1) the sample size is small, 2) the firm conducting the research is known to the sample respondent and the respondent presumably values the relationship, and 3) the researcher has a daughter who will likely work for sub-standard wages.
After the discussion and sharing of ideas, share the following story continuation with your students: Jason: “Not if her daughter makes follow-up phone calls,” said Jason.
Full ScriptHere is the full Close-Up as it appeared in the 6th edition. The bold text is what has been omitted in the case that appears in the 8th edition.
“Do you know if the post office accepts checks?” asked Audrey, the business manager at the Chevron station.“Couldn’t say,” Jason replied.
80
“Because I am going to spend a fortune on stamps and I don’t want to carry a lot of cash down there.” She waved a thick folio of three-up mailing labels. “I’ve got to survey our customers. Our banker insists. You know that new loan officer? The MBA? The one named Jasmine, who calls herself ‘Jazz’?” We asked her for $50,000 so we can open at 5 A.M. and stay open until midnight—we have to put up a fence and some shrubbery, so as not to disturb the apartments in back, and install two new bays—and she told us to survey our customers to find out if many of them are retirees who go to bed early and wake up late and wouldn’t use our service. And find out when the others leave for work in the morning and come home at night.”
“So you have to do a survey.”“Yes. Three or four questions, maybe—nothing complicated. Like the two I mentioned and ‘How long have you been our
customer?’ ”“How many years have you been our customer,” Jason murmured reflexively.“Oh, yeah, right. More precise, isn’t it?” Audrey said. “But the point is, do you know what it costs to write to 1,000
customers?”He crunched the numbers in his head. Postage to send out the survey. Postage prepaid to have it mailed back. Paper and
printing. One envelope outbound and one inbound. Stuffing the outbound and opening and coding the inbound. “Minimum of $1.50 a survey,” he said, “if your kids do the stuffing and opening and encoding. Maybe more. You generate labels from your computer, so you don’t pay for labels, which saves a mint, maybe 25 cents a name.”
“Right. A $1.50 times 1,000. Plus the kids’ time, if I can pry them away from Little League, MTV, and personal calls.”“You don’t need to survey 1,000 customers you know.”“Whaaat?”“Pick 100 at random from your computer list. You’ll get a 10 percent margin of error and save $1,350, plus the kids’ time.”“That’s hard to believe.”“But it’s true,” he said. “Look, why don’t you take some of the money you save, and instead of just sampling 100 from your
existing customer base, also do a separate survey of every 20th person from the Chamber of Commerce directory. You’ll have one survey of customers and another of potential customers, and you’ll still end up saving over $1,000.”
She thought for a moment. “The banker will never believe this,” she objected.“The next time I make a deposit, I’ll explain it to her.”“You do that,” she said, “and we’ll wax your car every three months for the next two years.”Jason was excited by the idea of having his car washed and waxed at no charge. He knew he was trading consulting services
cheaply for automotive services, but it felt free, so the next day when he went to the bank he approached the new bank officer—the new market development V.P., actually—with carefully disguised excitement. This was going to be more fun than working for money.
Gosh, she was a tall woman. Jasmine Rogers. Thirty-five, he guessed, 6 feet 2 inches or even 6 feet 3. Hard to tell when she was sitting. She sat grandly behind a mahogany desk in a cubicle with three glass walls. But the back wall was covered—by sports memorabilia. A basketball shirt said, “Morgan State—African Tour, 1985,” and there was another shirt that said, “Harvard B-School Intramurals.” There were crossed field hockey sticks, too, and a boomerang, and a picture of her holding the boomerang with small folks he supposed were Australian nationals casting admiring glances at this tall American woman.
She came out to greet him and propelled him—almost lifted him—to a seat in her cubicle.“So, what’s up, Jason? Are we taking good care of you? Do you need another dozen computers? What?”“I wanted to talk to you about the survey you asked Audrey to make. She and Juan own the Chevron station. . . .”“Oh, sure. Audrey and Johnny. Good people . . . Say, if you are thinking of getting a BMW, I can give you a good rate on a
loan.”“Well, I’m not in that bracket, Jazz. . . . Look, you told Audrey to take a survey of her customers, and that’s what I wanted
to discuss.”“I didn’t intend her to have to hire a high-powered business consultant.”“Well, actually, this is more or less a favor, you see. . . .”She chuckled. He thought she had a nice disposition for a banker. “A favor? By any means did she offer to wax your car
about a million times for this favor? I had to turn down that particular offer, being her loan officer. Conflict of interest, you know. But if you want to help her, hey, I say that’s great.”
“Well, this is no big deal, Jazz. More or less a back-of-the-envelope computation, you see. Except I did it in my head, as there was no envelope handy.”
“In your head? Impressive. I was on the math team in high school. . . .”“Here’s the thing. Audrey was about to do a census of her customers. . . .”“A census? Surely not a survey of all her customers. I wanted just a sample. She must have misunderstood.”“Well, that’s the thing. . . . I mean, I believe that a sample of maybe 100 customers will do the trick.”“I took business research in B-school, Jason, and while I certainly don’t want to break Audrey’s back, a sample of 100
seems kind of . . . well, thin.”
81
“Maybe I can convince you.”“Maybe.” She steepled her fingers thoughtfully and smiled wickedly. “Are you a betting man, Jason?” She placed a tall
glass tumbler holding several dozen drinking straws between them. “I am willing to bet you cannot convince me.”“Then let me try. Take the first question you wanted. ‘Are you currently retired from work?’ This is what we call a
proportions question. That is, a certain proportion of people will say they are retired.”“Yes,” said the banker. “Suppose 20 percent of the existing customer base says they are retired. I would take that as a very
bad omen for a station that opens early and closes late. Why would a retired person want to drop his car off at 6 A.M.?”“Exactly . . . well, say 20 percent of the people sampled say they are retired. That’s a proportion of .20, right? This
means that, of the full 1,000-odd customers who comprise the ‘population,’ the actual proportion falls in a confidence interval centering around .20.”
“Let me jump ahead here, Jason, because lunch approaches rapidly. I am willing to settle for a 95 percent chance of correctly locating the population proportion within an interval. . . .”
“So you will settle for a 95 percent confidence interval. You remember your statistics, I can see. Well, we need to compute the standard error of the proportion in order to compute the width of the confidence interval. To compute the standard error, we will use the formula. . . .”
Jazz leaned forward and grasped Jason’s left hand with her right hand, to prevent him from writing formulas in the air, and held up the index finger of her left hand for attention. “One second, Jason. I am willing to bet you this,” she extracted one drinking straw from the glass, “this valuable drinking straw, that I can tell you the formula to use. We take .2 and multiply by .8, and divide by 100 (which is your proposed number of subjects), and take the square root, which is, um . . . ,” she rolled her eyes to the ceiling, “which is .04. And we multiply by 2. So the 95 percent confidence interval extends between .20 plus and minus 0.08, from .12 to .28.” She laughed merrily and moved the straw closer to her side of the desk. “Come on, Jason, play this game to win. I won’t take it out on Audrey.”
“That’s very good, Jazz, and I’m going to let you keep that straw. But you omitted to consider a significant factor. Would you care to improve your estimate?”
“Hah,” she said, “I should have used 99 instead of 100. ‘N – 1.’ Hah!” and she reached for another straw from the tumbler, but Jason snatched the straw from her hands and set it on his side of the desk. Now each of them had a straw.
“That’s too small a correction to matter. You lose.”“OK,” she said with mock annoyance, “I should have multiplied by 1.96 instead of 2. That’s it, isn’t it?” And she
reached for another straw, but Jason clapped his hand over the tumbler.“Sorry,” he said, “that’s true, but again too small a correction to bother with. You should have allowed for the fact
that she is sampling 100 out of a population of 1,000. That shrinks the standard error of the mean by the square root of 900/999, which comes to. . . .”
“Too small a correction to bother with,” shouted Jazz with glee. “Take your hands away from my straws! OK, I’ll give that to you, Jason, a sample of 100 will give me all the precision I want for that question.”
“But what about the next question, ‘What time do you leave your home in the morning to go to work?’?” Jazz asked. “Well, we use a slightly different version of the formula this time. This time we divide the estimated standard
deviation of the error by the square root of 100 to get the standard error of the mean.”She reached into the tumbler and withdrew another straw. “Wait just a second, Jason. I will bet you this straw that you don’t
know the standard error of the sample until after you have taken the sample.”“Well, then you are wrong. Because I happen to know that the morning rush hour extends from about 6:30 to 9:30, a
period of 180 minutes, and applying the rule of thumb that the standard deviation is roughly one-sixth of the range, I arrived at a rough value of 30 minutes for the standard deviation, which gives a standard error of the mean of 3 minutes, which is all the precision you could possibly want.” He reached for the straw, but this time she pulled it away.
“Wait a second, Mr. Statistician. Where did you get that rule of thumb?”“I don’t know,” he admitted, “I picked it up during my graduate student days.”“Hah,” she exulted. “Keep your straw-pickin’ fingers away from my straws!”“However,” he said, “on returning to my office I did have a twinge of doubt about giving this advice to Audrey, so I
called the planning department of the local bus company, which keeps enormously detailed statistical records as part of the requirement for their federal transportation subsidy, and they confirmed that the ridership of their number 45 bus through our town, and indeed the ridership of practically every other bus on the system, has a standard deviation of . . . 29 minutes.” He extracted the straw. Now he had two, to her one.
“OK, Jason, you win. You have convinced me. If she can retrieve 100 surveys, I’ll be satisfied. But I’ll bet she will have to send out 200 surveys to get 100 replies.”
“Not if her daughter makes follow-up phone calls,” said Jason.
82
Case: McDonald’s Tests Catfish Sandwich?
Abstract: This case describes the test marketing for McDonald's catfish sandwich in the Southeastern USA. It asks the student to assume they are the new-product development team and assess the research design described.
This case requires the student to assess the representativeness of the ten test cities in which the new catfish sandwich is being served in relation to three states in which this product will be rolled out. To assess the representativeness of the test cities in relation to the rollout plan, the student will have to collect secondary data on each of the test cities and the three states. There are a wide variety of sources that students can make use of. We have compiled some preliminary data on each of the test cities as shown in the accompanying table. This was abstracted from the 1992 Rand McNally Commercial Atlas and Marketing Guide.
City/State Population Per Capita Income
No. of H.H
Median H.H.
Income
Food Store Sales
Total Retail Sales
R.McNally City Rating
Bowling Green, KY
40,641 10,992 16,000 18,096 105,350 690,894 3-A
Memphis, TN 610,337 12,111 231,300 23,229 617,069 5,316,937 2AA
Chattanooga, TN 162,466 12,029 62,400 21,323 285,222 1,923,629 2-A
Jackson, TN 48,949 12,178 19,200 22,906 109,959 626,533 3-A
Huntsville, AL 159,789 16,588 64,000 32,070 265,205 1,769,326 3-AA
Jonesboro, AK 46,536 12,261 18,100 21,824 81,390 528,714 3-A
Columbus, MS 23,799 10,284 9,100 18,387 56,989 372,228 3-A
Tupelo, MS 30,685 12,298 11,800 24,526 66,172 492,178 3-A
Greenville, MS 45,226 8,291 15,200 16,669 70,902 362,498 3-A
Greenwood, MS 18,906 8,742 7,000 14,826 38,894 203,475 4-B
83
Case: Medical Laboratories
Abstract: A hospital faces a 20% price increase from its outside testing lab based on rising costs of testing and diminishing profits. The student is asked to analyze data from 16 different medical labs and make a recommendation about the financial health of each lab, supporting or refuting the basis of the proposed price increase.
General Comments
The trick to this data set involves the first task requested in the case--to build a model of OPASSTS. As it turns out, the multiple regression model is an accounting identity. The OPASSTS variable was created by the equation: OPASSTS = PPE + ASSETS - LIAB. The value for PPE of firm 1 in 1986 is actually 13.525, which was changed to 13.526 in the data set to permit calculation of the regression diagnostics. Your students will get an answer from the multiple regression, but should be tipped off by t ratios for the estimated slope coefficients of several hundred thousand!
The reason we created this case is that every semester when students are asked to design their own project and collect their own data, one or more students invariably comes up with a deterministic model. For instance, one group proposed studying total electricity consumption in Denver as a function of consumption from residential, commercial, and manufacturing sectors, a tautology. The confusion likely arises from the message that higher R2 values are better, yet we don't generally talk about what it means to have an R2 of 100%. This case will allow you to rediscover with your students the difference between stochastic and deterministic models and the fundamental nature of residuals and residual analysis to multiple regression model-building.
Discussion Questions
1. Is the external medical facility justified in claiming that labs around the country are experiencing hard times? How should you decide to answer this question?
2. What are the major financial determinants of OPASSTS?
Suggestions for Discussion
The interesting part of the class discussion involves the matter of the 100% R2. Some students will overlook the perfect R2 value and simply write up the report, bragging about what a fine model they have derived. Others will finally discover the trick to the case. The best way to lead the class discussion in our experience is to plead total ignorance to the issue of the accounting identity. Act as if you are completely surprised by the outcomes being reported by your students. Then let them take control of the discussion.
When the trick is finally uncovered, ask your students about the general nature of the error term in regression analysis. Ask them the purpose of the error term, where and how the error term comes into play in generating the regression statistics, and what it means to have an equation with zero residual values.
Typical Results and Approaches
The question of whether your medical laboratory should be passing on higher costs can be addressed by looking at the behavior over time and across firms of the OPASSTS variable. Although OPASSTS dropped from 15.75 to 14.73 on average over the 1986 to 1988 period, OPASSTS appears to be on its way back up thereafter, and reaches the level of 16.79 by 1990. However, when looking at the means for each firm over the five years, there is a substantial difference in OPASSTS among different firms. This view of the data suggests that your particular lab may be experiencing some difficulty, but not all labs are having the same problem. Further investiga tion into their claims seems to be warranted by these data.
84
On the model-building portion of this case, some students may actually discover the accounting identity inherent to the multiple regression model of OPASSTS on PPE, ASSETS, and LIAB and solve the puzzle by leaving out one or two of the predictors. Here's a printout that you can take to class that shows the essence of the case.
Regression of OPASSTS on PPE, ASSETS and LIAB OPASSTS = -0.000010 + 1.00 PPE + 1.00 ASSETS - 1.00 LIAB Predictor Coef Stdev t-ratio p Constant -0.00000953 0.00001488 -0.64 0.524 PPE 1.00000 0.00000 436979.66 0.000 ASSETS 1.00000 0.00000 872357.12 0.000 LIAB -1.00000 0.00000 -909799.63 0.000 s = 0.0001033 R-sq = 100.0% R-sq(adj) = 100.0% Analysis of Variance SOURCE DF SS MS F p Regression 3 42164 14055 1.318E+12 0.000 Error 76 0 0 Total 79 42164 Unusual Observations Obs. PPE OPASSTS Fit Stdev.Fit Residual St.Resid 1 13.5 67.9330 67.9338 0.0000 -0.0008 -8.69RX 2 15.1 67.2060 67.2058 0.0000 0.0002 1.85 X 46 35.9 90.9630 90.9629 0.0000 0.0001 0.96 X 49 20.4 26.5740 26.5740 0.0001 0.0000 0.10 X 54 33.2 43.2550 43.2550 0.0001 -0.0000 -0.43 X 55 33.3 38.2410 38.2411 0.0000 -0.0001 -1.07 X R denotes an obs. with a large st. resid. X denotes an obs. whose X value gives it large influence.
85