assessing sustainability literacy an instrument to measure knowledge of environmental, economic, and...
TRANSCRIPT
Assessing Sustainability LiteracyAn Instrument to Measure Knowledge of Environmental, Economic, and Social Sustainability
Adam Zwickle
Tomas Koontz
Kristina Slagle
Overview Need for a sustainability literacy
measure
Developing an assessment
Results
Where we go from here
Need for a sustainability literacy measure
STARS ER Credit 13: Sustainability Literacy Assessment
“This credit recognizes institutions that are assessing the sustainability literacy of their students. Such an assessment helps institutions evaluate the success of their sustainability education initiatives and develop insight into how these initiatives could be improved.”
Criteria“Institution conducts an assessment of the sustainability literacy of its students. The sustainability literacy assessment focuses on knowledge of sustainability topics, not values or beliefs.”
Need for a sustainability literacy measure Lacking in the scientific community
“Sustainability” has been defined and measured a myriad of ways Often focused merely on the environmental aspect
Immensely applicable Can test for correlations with various values,
attitudes, and behaviors Can serve as an explanatory variable
Developing an assessment Built upon the “triple bottom line”, the
“three legged stool”, the “3 p’s” Environmental (planet) Economic (prosperity) Social (people)
Basic knowledge from each knowledge domain
Developing an assessment Replicated questions used in the past
Coyle, 2005. “Environmental Literacy in America.”
Solicited topics and questions from experts
Held expert focus groups Pilot tested among professors, graduate,
and undergraduate students Narrowed down to 30 questions
Developing an assessment Administered 30 question assessment
Through an online survey Sent via email
Sent to over 10,000 currently enrolled undergraduates 19.3% response rate
Non-response follow up
Developing an assessment Demographics
OSU – average demographics This study - Slight trend towards more
liberal and environmentalist, but differences were non-significant
Non-respondants More likely to get a subset of the
sustainability knowledge questions wrong…(small difference, but significant)
No differences in pro-environmental behaviors or GPA
Developing an assessment Therefore, those choosing to take the
survey were more knowledgeable about sustainability issues to begin with
These findings represent an ‘upper bound’ of the student population’s knowledge
Developing an assessment Needed to reduce the length
Utilized Item Response Theory (IRT) To reduce down to 16 questions
And aside on IRT… Developed in the educational testing
field Used for standardized tests (GRE)
Focuses on the likelihood of answering something correctly… At a given level of the construct of
interest
And aside on IRT…
Information
Correct
Incorrect
50/50threshold
Standard deviations
Theta - The latent construct that is being measured
Probability of answering question correctly
And aside on IRT…
Information
Correct
Incorrect
50/50threshold
“Over the past 3 decades, what has happened to the difference between the wealth of the richest and poorest Americans?”
And aside on IRT…
Information
Incorrect
Correct
50/50threshold
Standard deviations
A question that was removed…
Probability of answering question correctly
And aside on IRT…
Very low information
Correct
Incorrect
50/50threshold
“Approximately how much of the oil used in the U.S. in 2010 was imported from other countries?”
Questions - Environmental
What is the most common cause of pollution of streams and rivers?
Ozone forms a protective layer in the earth's upper atmosphere. What does ozone protect us from?
What is the name of the primary federal agency that oversees environmental regulation?
What is the primary benefit of wetlands? Which of the following is an example of sustainable
forest management? In the U.S., what do we currently do with the
nuclear waste generated by nuclear power plants?
Questions - Social
Which of the following is the most commonly used definition of sustainable development?
The wealthiest 20% of people in the U.S. own approximately what percent of the nation’s privately held wealth?
Over the past 3 decades, what has happened to the difference between the wealth of the richest and poorest Americans?
Higher levels of education generally lead to... Which of the following populations has the
highest rate of growth?
Questions - Economic
Many economists argue that electricity prices in the U.S. are too low because…
Which of the following countries has now passed the U.S. as the biggest emitter of the greenhouse gas carbon?
Which of the following is a leading cause of depletion of fish stocks in the Atlantic Ocean?
Which of the following is the most commonly used definition of economic sustainability?
Which of the following is the primary reason that gasoline prices have risen over the last several decades in the U.S.?
Results Environmental
6 questions
Mean: 4.39/6SD: 1.48
73%
Social5 questions
Mean: 3.55/5SD: 1.2371%
Economic5 questions
Mean: 3.03/5SD: 1.2761%
Total16 questions
Mean: 11.08/16SD: 3.21
69%
Example Analysis
Mean1
4
7
10
13
16
9.5710.57 10.95 11.34
Freshman
Sophomore
Junior
Senior
ANOVAF(3,1330)=8.09, p<.001
Example Analysis Compare sub score by major
Economic score Economics: 3.82
Aerospace engineering: 4.00 Animal science: 3.15 Finance: 3.02 English: 2.80 Accounting: 2.63
Moving Forward Refine
Weaknesses Some answers change over time Give us your suggestions!
Publish
Apply (put path model here?)
Acknowledgements Funded by
The Ohio State Office of Sustainability http://sustainability.osu.edu/
OSU’s School of Environment
& Natural Resources http://senr.osu.edu/
The many faculty and graduate students who contributed their time and expertise
Questions? Environment and Social Sustainability
Lab www.ess.osu.edu Contains:
This presentation The knowledge questions with multiple choice
answers
Lab email [email protected]