hometown competitiveness: conducting phase i assessment jennifer bott, phd, sphr associate professor...
TRANSCRIPT
![Page 1: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/1.jpg)
HomeTown Competitiveness:Conducting Phase I Assessment
Jennifer Bott, PhD, SPHRAssociate Professor of Management Miller College of Business Ball State University
![Page 2: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/2.jpg)
Agenda
•Background on HTC Assessment•Goals of Assessment•Types of Assessments
▫Focus Groups and interviews▫Designing surveys▫Archival data analysis
•Basic analyses and decision making•Resources
![Page 3: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/3.jpg)
![Page 4: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/4.jpg)
Background on Assessment
•Phase I Assessment phase typically lasts up to four months▫Community conducts local assessments of
development history, context, issues and assets
▫Create customized strategic opportunities•Leads into implementation of the
strategic development plan in Phase II
![Page 5: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/5.jpg)
Assessment Goals
•Determine perspectives of community stakeholders on chosen pillar within the community: ▫Entrepreneurship▫Charitable Assets▫Youth Engagement▫Leadership
![Page 6: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/6.jpg)
Assessment Outcome
•End product of the assessment:▫Collection of data that direct community
team toward path of action▫Summary of current thinking in community
on current status of selected pillar•Launching assessment with community
should engage them more in the process, outcome and implementation of the strategic plan
![Page 7: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/7.jpg)
Types of Assessments
•Traditional measures of assessment:▫Focus groups▫Interviews▫Surveys (telephonic, online, mail, intercept)
•Other opportunities:▫Re-analysis of previously-collected data
![Page 8: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/8.jpg)
Types of Data
•Two primary types of data emerge from assessments:▫Qualitative
Verbal responses to open or closed-ended questions
▫Quantitative Numeric responses to (usually) closed-ended
questions
![Page 9: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/9.jpg)
Choice of Assessment(s)• Assessment method should:–Provide most insight into questions at hand– Involve stakeholders–Adhere to following criteria:• Reliable• Valid• Cost-effective• Feasible
• Combinations of assessment methods may be most beneficial
![Page 10: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/10.jpg)
![Page 11: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/11.jpg)
Focus Groups
•Characteristics:▫7 – 10 members▫Share some relationship to topic of interest▫Represent divergent opinions on topic and
typically represent different stakeholders•Repeated several times with different
groups of people•Requires collection of qualitative data
![Page 12: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/12.jpg)
Topics for Focus Groups
•Strategic planning•Needs assessment•Current state analysis•Program design•Perspectives on a singular issue
![Page 13: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/13.jpg)
Uses of Focus Groups
•Focus groups are especially useful:▫Exploratory, preliminary study▫Gap in understanding or communication▫Uncover data related to complex behavior▫Discover interactive synergies between
participants▫Uncover additional information leading to a
large-scale study effort▫High value placed on open-ended data
collection
![Page 14: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/14.jpg)
When NOT to Use Focus Groups•Focus groups are not recommended for:
▫Emotionally-charged subject▫Researcher/HTC team not in control of the
environment or message▫Team expects statistical summaries from
data▫Is not economical to achieve data collection
needed▫Confidentiality can’t be guaranteed
![Page 15: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/15.jpg)
Outcomes of Focus Groups
•Categorized data from focus groups can be used:▫As stand-alone, qualitative information▫As starting point for development of a
quantitative measure
![Page 16: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/16.jpg)
Designing Successful Focus Groups• Create “safe” environment–May require use of a neutral facilitator
• Create a plan for focus group that details:–Purpose –Who to include • Participants• Number of sessions
– Incentives for participation (if necessary)–Location
![Page 17: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/17.jpg)
Focus Group Questions• Categories of questions:– Opening • Everyone answers quickly (10 - 20 seconds)
– Introductory • General topic of discussion, foster conversation
– Transition • Approach the focal questions
– Key • Drive the study – 2 to 5 questions that require
most attention– Ending • Bring closure and allow for reflection
![Page 18: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/18.jpg)
Focus Group Questions (cont’d)
• In most cases, focus group questions are open-ended– “What are our greatest weaknesses in
youth engagement?”• Closed-ended questions are not
appropriate for sparking discussion– “Would you participate in a training
success?”• Avoid asking why in response to a
participant’s answer
![Page 19: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/19.jpg)
Analyzing Focus Group Data
•Determining analysis strategy:▫Least time intensive = Memory based
Report is created immediately based on facilitator’s memory
▫Most time intensive = Transcript based Combines field notes with transcript to write
final report▫Most common = Note based
Relies on field notes to create focus group report
![Page 20: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/20.jpg)
Focus Group Report
•The focus group report summarizes:▫Participants▫Setting▫Key questions▫Themes that emerged from key question
responses▫Limitations▫Recommendations
![Page 21: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/21.jpg)
Conducting Interviews
•Interviews can also be used as assessment tools▫One-on-one or panel (2 or more) ▫Collect qualitative data▫Can be used as singular collection method
or in tandem with focus groups or surveys
![Page 22: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/22.jpg)
Uses of Interviews
•Appropriate when:▫Group perspectives are not valuable▫Topic is too sensitive to discuss in group
setting▫Want to collect limited information (e.g.,
only want to speak to a few individuals)▫Focus groups are not feasible
![Page 23: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/23.jpg)
![Page 24: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/24.jpg)
Uses of Surveys
•Surveys are appropriate when:▫Know the parameters of the content to be
assessed▫Have identified a sample that will provide
meaningful data▫Need quantitative figures (rather than
qualitative descriptions)•Can be coupled easily with qualitative
data collection
![Page 25: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/25.jpg)
Considerations with Survey Administration and Design•Decision points on survey design:
▫Sample Who will respond? How many?
▫Administration mode Telephone Intercept (in-person) Snail mail Internet
▫Necessary response rates
![Page 26: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/26.jpg)
Survey Questionnaire Development•Six basic steps:
▫Decide what information to collect▫Decide what type of questionnaire should
be used▫Develop first draft▫Revise questionnaire▫Pilot test▫Edit questionnaire and specify methods of
collection
![Page 27: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/27.jpg)
Question Writing Guidelines
•Characteristics of “good” items:▫Simple, direct, familiar language▫Clear and specific questions▫Do not use leading, loaded, or double-
barreled questions▫Use short statements (20 words or less)▫Edit for readability
![Page 28: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/28.jpg)
Designing the Scale
•Most common response options:▫Agreement▫Frequency▫Evaluation
•Number of response categories ▫3 to 5
•Neutral point – good or bad idea?
![Page 29: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/29.jpg)
Evaluate Wording
•Which is a better question?▫Age?▫What was your age on your last birthday?
•Is there a problem with this item?▫What do you like best about this
neighborhood? (We’re interested in anything, like houses, the people, the parks, or whatever.)
▫Do you favor or oppose gun control legislation?
![Page 30: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/30.jpg)
Item Stem Creation
•Agreement items – Declarative▫“I am uncomfortable around strangers.”
•Frequency items – Events, behaviors▫“You exercise strenuously enough to raise your
heart rate.”•Evaluation items – Persons, places, things
▫“How well your favorite sports team played last week.”
![Page 31: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/31.jpg)
Designing the Survey
•Some simple rules for the ordering and design of questions:▫Keep items together if they use the same
responses▫Don’t overfill each page with questions▫Avoid including too many different
response options•Pilot test with individuals similar to those
being sampled
![Page 32: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/32.jpg)
Importance of Instructions•Give direction for response for those
unfamiliar with format•Create a common frame of reference
▫“The following questions concern peoples’ opinions and beliefs about jobs and careers. These questions refer to jobs in general and not the job you presently have or a particular job you once had.” (Spector, 1992)
![Page 33: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/33.jpg)
![Page 34: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/34.jpg)
Using Archival Data
•In some cases, data has already been collected regarding your pillar of interest▫Ask groups that are associated with your
pillar (e.g., Chamber of Commerce, local schools, philanthropic groups)
▫Can be either quantitative (surveys) or qualitative (interviews, news coverage, etc.)
![Page 35: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/35.jpg)
Possible Analyses with Archival Data•Content analysis•Assessment of trends•Analysis of communications
![Page 36: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/36.jpg)
Limitations
•Archival data has many advantages and disadvantages▫Advantages:
Data is already collected (cheap, easy to use) Data may already be analyzed
▫Disadvantages: Can’t be tailored to fit needs Inability to determine the reliability of the
methods used to collect data
![Page 37: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/37.jpg)
![Page 38: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/38.jpg)
Conclusion of the Assessment
•Use the assessment data to identify “key findings” about the pillar assessment▫A key finding is a full sentence used to
summarize learning. Including a statistics strengthens this finding. “73% of high school students surveyed stated
that they would return to our community if opportunities exist.”
![Page 39: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/39.jpg)
Conclusion (cont’d)
•Key findings are used to drive the development of the problems to be solved at the end of Phase I.
![Page 40: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/40.jpg)
Tangible Outcomes of Assessment•A written report should be provided to
individuals who participated in the survey▫Details overall findings (at the group level
only)▫May provide information about uses of the
assessment Depends on confidentiality of the process
▫Can hopefully meet a need of the sample
![Page 41: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/41.jpg)
Moving to Decision Making
•Totality of evidence from both qualitative and quantitative assessment efforts should be examined▫Looking for evidence that supports teams’
beliefs or uncovers new, unexpected information
▫Evidence should create a logical next step in assessment or implementation
![Page 42: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/42.jpg)
![Page 43: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/43.jpg)
Resources to Aid in the Assessment Process•Center for Business and Economic
Research at Ball State University▫Partnered with Building Better
Communities•Professional consultants•Other HTC Communities
![Page 44: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/44.jpg)
Resources (cont’d)
•Books:▫Survey Research Methods (4th ed.), Floyd
Fowler (Sage Publications, $39.95)▫Focus Groups: A Practical Guide for
Applied Research, Krueger and Casey (Sage Publications, $45.32)
![Page 45: HomeTown Competitiveness: Conducting Phase I Assessment Jennifer Bott, PhD, SPHR Associate Professor of Management Miller College of Business Ball State](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649ddf5503460f94ad8eb2/html5/thumbnails/45.jpg)
Links to Nebraska Surveys
•http://www.zoomerang.com/Shared/SharedResultsSurveyResultsPage.aspx?ID=L22Z2S57GK54
•http://www.zoomerang.com/Shared/SharedResultsSurveyResultsPage.aspx?ID=L23KD72VDQVC
•http://www.zoomerang.com/Shared/SharedResultsSurveyResultsPage.aspx?ID=L23KDC4HWP6J