collegiate assessment of academic proficiency: science ...€¦ · science, and critical thinking....

Collegiate Assessment of Academic Proficiency: Science Test Fall 2010

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, December 2010 of 33

Collegiate Assessment of Academic Proficiency:

Science Test

Fall 2010 Administration



Table of Contents Executive Summary ....................................................................................................................................... 3

Introduction .................................................................................................................................................. 4

Sample....................................................................................................................................................... 5

Results ........................................................................................................................................................... 8

CAAP Scores Overall .................................................................................................................................. 8

CAAP Scores by Sex, Major, GPA, and Motivation .................................................................................. 10

Additional Questions Scores Overall ....................................................................................................... 13

Additional Questions Scores by Sex, Major, GPA, and Motivation ......................................................... 14

Relationship between CAAP Scores and Additional Questions Scores ................................................... 16

Conclusion ................................................................................................................................................... 19

Appendix ..................................................................................................................................................... 20

Appendix A: Information about the CAAP Science Test ......................................................................... 20

Appendix B: Copy of the Additional Questions ....................................................................................... 22

Appendix C: Analysis of CAAP Scores by Sex, Major, GPA, and Motivation ........................................... 26

Appendix D: Analysis of Additional Questions Scores by Sex, Major, GPA, and Motivation .................. 28

Appendix E: CAAP Scores and Additional Questions Scores by Student Classification .......................... 30

Appendix F: CAAP Scores and Additional Questions Scores by Sex, Major, and Minor ......................... 31



Executive Summary The Collegiate Assessment of Academic Proficiency (CAAP) Science Test is designed to measure skills in scientific reasoning. In coordination with the various Texas Tech University (TTU) Colleges, the Office of Planning and Assessment administered the Science Test in select courses across campus at the beginning of the fall 2010 semester. Courses were chosen based on their enrollment of juniors and seniors to achieve a representative number from each College. ACT allows the administering institution to add nine additional questions to the chosen CAAP module. TTU chose to add the six Natural Sciences questions from the Online Senior Assessment (OSA) plus three newly developed Natural Sciences questions to allow for a comparison that could help validate the Natural Sciences questions from the OSA. A total of 495 Texas Tech University (TTU) students participated in the fall 2010 administration of the CAAP Science Test. The answers of 400 participants were sent to ACT for scoring. A final sample of 331 participants (161 juniors and 170 seniors) is included in this analysis. The final sample seems to be a fairly good representation of the population of all TTU juniors and seniors in terms of sex, ethnicity, and college. Looking at the CAAP scores overall it appears that the TTU juniors (mean = 61.4, SD = 4.1) score similarly on average to the national sample of juniors (mean = 61.4, SD = 4.7) and that the TTU seniors (mean = 61.3, SD = 4.1) score higher on average than the national sample of seniors (mean = 60.7, SD = 4.8) on the Science Test. There were statistically significant differences in CAAP scores by sex, major, motivation. Male participants score higher on average than female participants. Natural Science majors score higher on average than other majors. Participants reporting more motivation score higher on average than other participants. There was not a statistically significant difference in CAAP scores by GPA. For the additional questions scores there was a statistically significant difference in scores by major. Natural Science majors score higher on average than other majors. There was not a statistically significant difference in additional questions scores by sex or GPA. The interpretation of the additional questions scores needs to consider that these questions were administered at the end of participants’ testing. The patterns in additional questions scores are likely influenced by participants potentially rushing through this last part of the test. The analysis found a correlation between the CAAP scores and the additional questions scores of 0.34 (p value < 0.0001). This correlation suggests that there is a significant relationship between the additional questions scores and CAAP scores. This relationship may suggest that the CAAP and the additional questions are measuring similar domains of knowledge or that they are in some other way related. The results suggest that the nine additional questions might be a better predictor of CAAP scores than the first six questions alone (R2 of 0.1198 for the nine questions and 0.0883 for the six questions).



Introduction The Collegiate Assessment of Academic Proficiency (CAAP) “is the standardized, nationally normed assessment program from ACT that enables postsecondary institutions to assess, evaluate, and enhance student learning outcomes and general education program outcomes.”1 The CAAP offers six different modules: Reading, Writing Skills, Writing Essay, Mathematics, Science, and Critical Thinking. 2 As per decision of the University Assessment Committee in April 2010, the Office of Planning and Assessment administered the Science Test of the CAAP during fall 2010. The Science Test “is a 45-item, 40-minute test designed to measure students' skills in scientific reasoning. The contents of the Science Test are drawn from biological sciences (e.g., biology, botany, and zoology), chemistry, physics, and the physical sciences (e.g., geology, astronomy, and meteorology). The test emphasizes scientific reasoning skills rather than recall of scientific content or a high level of skill in mathematics or reading.” 3 Please find more information on how the test is constructed and scored in Appendix A. ACT allows the administering institution to add nine additional questions to the chosen module. The Core Curriculum Committee (CCC) of Texas Tech University (TTU) has been working on its own instrument to measure student abilities in various core areas, the Online Senior Assessment (OSA). The OSA was designed in 2008 and has been administered to graduating seniors from Texas Tech during the spring semester of 2008, 2009, and 2010. During the summer of 2010 the same assessment was administered to incoming transfer students at Texas Tech as the Transfer Student Assessment (TSA). The instrument has one section for each of the following areas: Humanities, Multicultural, Mathematics, Natural Sciences, Technology and Applied Science, and Social and Behavioral Sciences. The Natural Sciences section of the OSA is designed to measure abilities that are also measured by the CAAP Science Test. Since the Science Test is an established instrument it was decided to use the original six Natural Sciences questions from the OSA along with three newly developed Natural Sciences questions to start establishing validity for the OSA (see Appendix B for a copy of the additional questions). In coordination with the various Colleges, the Office of Planning and Assessment administered the Science Test in select courses across campus. Courses were chosen based on their enrollment by student classification and size. As juniors and seniors were the target group, courses with high junior and senior enrollment were selected to participate. In order to represent the Colleges equally larger courses were chosen from larger Colleges and smaller courses were chosen from smaller Colleges. Included in the sample were courses from the following Colleges: College of Agricultural Sciences and Natural Resources, College of Architecture, College of Arts and Sciences, College of Education, College of Human Sciences, College of Mass Communications, College of Visual and Performing Arts, Edward E. Whitacre Jr. College of Engineering, Honors College, and Jerry S. Rawls College of Business Education.

1 http://www.act.org/caap/, accessed 11/29/10

2 http://www.act.org/caap/test_modules.html, accessed 11/29/10

3 http://www.act.org/caap/test_science.html, accessed 11/29/10

http://www.act.org/caap/

http://www.act.org/caap/test_modules.html

http://www.act.org/caap/test_science.html



A total of 495 Texas Tech University (TTU) students participated in the fall 2010 administration of the Science Test. The Office of Planning and Assessment identified 17 irregularities (e.g., pattern scoring) that were exempt from scoring. In order to have a proportionate representation from each College, another 78 participants’ answer sheets were not selected for scoring. The remaining 400 answer sheets were scored by ACT.

Sample

ACT provided scores for the 400 answer sheets of students who participated in the fall 2010 administration of the CAAP. Since the students’ R numbers were used as the student ID on the Science Test, the Office of Planning and Assessment was able to obtain demographic information for the participants from Institutional Research and Information Management (IRIM). The demographic information obtained includes sex, ethnicity, student classification, and college. Since the target population for the assessment consisted of TTU juniors and seniors, the 69 participants of other student classifications (3 freshmen, 59 sophomores, 3 undergraduate second degree students, 2 graduate masters, 1 graduate doctoral, and 1 unreported) were excluded from this analysis for a final sample of 331 participants (161 juniors and 170 seniors). The following graphs compare the final junior and senior samples to the population of all TTU juniors or seniors by sex, ethnicity, and college.

The samples of both the juniors and seniors appear to be good representations of their respective populations in terms of sex.

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

Female Male

Sample and Population by Sex: Juniors

Sample

Population

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Female Male

Sample and Population by Sex: Seniors

Sample

Population



The samples of both juniors and seniors appear to be good representations of their respective populations in terms of ethnicity.

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

AI AS B HI M NR U WH

Sample and Population by Ethnicity: Juniors

Sample

Population

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

AI AS B HI M NR U WH

Sample and Population by Ethnicity: Seniors

Sample

Population



It appears that for the juniors the Rawls College of Business, the Whitacre College of Engineering, and the College of Visual and Performing Arts were somewhat overrepresented, and that the College of Arts and Sciences, the College of Education, and the College of Human Sciences were somewhat underrepresented. It appears that for the seniors the College of Arts and Sciences and the College of Visual and Performing Arts were overrepresented, and that the College of Education was underrepresented. The lack of participants from the College of Education is likely due to the number of students from other colleges that take courses in the College of Education for teaching certification. Students who are seeking teaching certifications are categorized according to their “home” college. For example, an undergraduate student seeking certification in Secondary Education might be a History major, and thus be counted under the College of Arts and Sciences.

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

AG AR AS BA ED EN HR HS MC UC UN VP

Sample and Population by College: Juniors

Sample

Population

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

AG AR AS BA ED EN HR HS MC UC UN VP

Sample and Population by College: Seniors

Sample

Population



Results

CAAP Scores Overall

The following table and graph give a summary of the juniors’ and seniors’ CAAP scores.

Summary of CAAP Scores

Juniors Seniors

n 161 170

Mean 61.4 61.3

SD 4.1 4.1

Min 52 51

Median 62 61.5

Max 70 71

Note that the TTU juniors (mean = 61.4, SD = 4.1) and seniors (mean = 61.3, SD = 4.1) performed similarly on the CAAP. The ACT website has a summary of the scores from 10,331 juniors and 8,237 seniors from 98 institutions that participated in the Science Test in fall 2010.4 On average the TTU juniors scored similarly to the national sample of juniors (mean = 61.4, SD = 4.7).5 On average the TTU seniors scored just above the national sample of seniors (mean = 60.7, SD = 4.8).6

4 http://act.org/caap/norms/, accessed 11/29/10

5 http://act.org/caap/norms/pdf/10Table12.pdf, accessed 11/29/10

6 http://act.org/caap/norms/pdf/10Table13.pdf, accessed 11/29/10

12

59 58

32

13

6065

30

2

0

10

20

30

40

50

60

70

45 50 55 60 65 70 75 80

Histogram of CAAP Scores

Juniors

Seniors

http://act.org/caap/norms/

http://act.org/caap/norms/pdf/10Table12.pdf




The following table shows the frequency of TTU juniors and seniors that received each score. The table also shows national percentiles for each of these scores for the juniors and seniors that were obtained from the ACT website.7 For each score, the national percentile shows the percentage of junior or senior participants from the national sample that scored at or below that score. For example, a score of 62 has a national percentile for juniors of 58 and a national percentile for seniors of 54. This means that 58% of the national sample of juniors scored at or below this score and that 54% of the national sample of seniors scored at or below this score.

National Percentiles for CAAP Scores

Score Juniors Natl. % Seniors Natl. %

71 0 99 2 99

70 4 98 2 99

69 1 96 2 96

68 7 93 4 93

67 5 89 5 89

66 15 85 17 85

65 7 79 7 78

64 10 72 11 71

63 17 66 15 64

62 23 58 20 54

61 1 48 12 45

60 15 41 16 39

59 14 34 13 32

58 15 28 15 26

57 3 22 8 21

56 12 17 8 16

55 5 13 3 12

54 5 8 6 8

53 1 5 1 5

52 1 3 1 3

51 0 1 2 1

Over half of the TTU juniors (89 participants, 55.2%) and over half of the TTU seniors (97 participants, 57.1%) scored above the 50th percentile nationally.

7 http://act.org/caap/norms/pdf/10Table12.pdf, http://act.org/caap/norms/pdf/10Table13.pdf, accessed

11/29/10





CAAP Scores by Sex, Major, GPA, and Motivation

The following table shows the summary statistics for CAAP scores by sex and major (see Appendix F for additional information).

CAAP Scores by Sex and Major

All Participants Natural Science Majors Other Majors

All Females Males All Females Males All Females Males

Mean 61.4 60.6 62.0 63.8 64.4 63.5 61.0 60.3 61.7

SD 4.1 3.8 4.2 3.9 3.3 4.2 4.0 3.6 4.2

Min 51.0 52.0 51.0 54.0 59.0 54.0 51.0 52.0 51.0

Median 62.0 60.0 62.0 64.0 64.0 64.5 61.0 60.0 62.0

Max 71.0 70.0 71.0 70.0 70.0 70.0 71.0 70.0 71.0

The following graphs show the average CAAP scores for TTU juniors and seniors by sex, major, GPA, and participants’ self report of how motivated they were in taking the assessment.

For both the juniors and seniors the male participants scored higher on average than the female participants. While not large, these differences by sex are statistically significant at the 0.05 level (see Appendix C for details). This suggests that on average male TTU juniors and seniors score better than female TTU juniors and seniors on the Science section of the CAAP.

60.662.1

60.661.9

50.0

52.0

54.0

56.0

58.0

60.0

62.0

64.0

Female Male

Average CAAP Scores by Sex

Juniors

Seniors



For both the juniors and the seniors the Natural Science majors scored higher on average than the other majors. These differences by major are statistically significant at the 0.05 level (see Appendix C for details). This suggests that on average TTU juniors and seniors with a Natural Science major score better than TTU juniors and seniors with another major on the Science section of the CAAP.

It appears that on average the seniors from the lowest GPA group scored lower than the other participants, while there does not appear to be much difference in average scores between the other GPA groups. The differences in scores by GPA are not statistically significant at the 0.05 level (see Appendix C for details). This suggests that on average participants’ GPA does not have an impact on participants’ scores for the Science section of the CAAP.

64.3

61.1

63.4

61.0

50.0

52.0

54.0

56.0

58.0

60.0

62.0

64.0

Natural Science Major

Other Major

Average CAAP Scores by Major

Juniors

Seniors

62.0 62.6 6260.3

62.1

54.0

61.660.4

62.0 61.5

50.0

52.0

54.0

56.0

58.0

60.0

62.0

64.0

below 2.00 2.01 - 2.50 2.51 - 3.00 3.01 - 3.50 3.51 and above

Average CAAP Scores by GPA

Juniors

Seniors



For both the juniors and seniors it appears that on average the participants that reported more motivation and effort on the assessment scored higher. These differences between the groups are statistically significant at the 0.05 level (see Appendix C for details). This suggests that on average TTU juniors and seniors that are more motivated to put forth effort on the assessment score higher.

62.661.7

58.9

57

62.761.1

59.7

56.0

50

52

54

56

58

60

62

64

tried my best gave moderate effort

gave little effort

gave no effort

Average CAAP Scores by Motivation

Juniors

Seniors



Additional Questions Scores Overall

The following table and graph give a summary of the sample’s scores on the additional questions.

Summary of Additional Questions Scores

Juniors Seniors

n 105 125

Mean 50.7% 53.0%

SD 15.4% 18.8%

Min 0.0% 0.0%

Median 55.6% 55.6%

Max 88.9% 88.9%

It appears that on average the seniors scored slightly higher on these additional questions than the juniors. This difference was not statistically significant at the 0.05 level (see Attachment D for details). Both the juniors and seniors answered, on average, between four and five of the nine additional questions correctly, with a low score of zero questions answered correctly and a high score of 8 questions answered correctly. When considering the scores for the additional questions it is important to note that these questions were answered at the end of the testing period (i.e., after the participants had already taken the 40-minute-long Science Test) and that participants may have been rushed in trying to complete them. As shown in the table, only 230 of the 331 participants actually answered the additional questions.

2 3

8 9

27

23 23

8

214

7

1820

28 28

16

3

0

5

10

15

20

25

30

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Histogram of Additional Question Scores

Juniors

Seniors



Additional Questions Scores by Sex, Major, GPA, and Motivation

The following table shows summary statistics for additional questions (AQ) scores by sex and major (see Appendix F for additional information).

AQ Scores by Sex and Major

All Participants Natural Science Majors Other Majors

All Females Males All Females Males All Females Males

Mean 51.9% 51.6% 52.2% 59.0% 65.7% 61.0% 50.2% 49.9% 50.4%

SD 18.6% 17.2% 19.7% 16.3% 10.5% 18.0% 18.6% 19.9% 17.1%

Min 0.0% 0.0% 0.0% 22.2% 44.4% 22.2% 0.0% 0.0% 0.0%

Median 55.6% 55.6% 55.6% 66.7% 66.7% 66.7% 55.6% 55.6% 50.0%

Max 88.9% 88.9% 88.9% 88.9% 88.9% 88.9% 88.9% 77.8% 88.9%

The following graphs show the average additional questions scores by sex, major, and GPA.

The female juniors scored higher on average than the male juniors and the female seniors scored lower on average than the female seniors. These differences are not statistically significant at the 0.05 level (see Appendix D for details). The lack of significant results for these differences may be due to a smaller number of participants finishing the additional questions and the greater variance in scores for the additional questions.

53.1%48.9%50.5%

55.1%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Female Male

Average AQ Scores by Sex

Juniors

Seniors



For both the juniors and the seniors the Natural Science majors scored higher on average than the other majors. These differences by major are statistically significant at the 0.05 level (see Appendix D for details). This suggests that on average TTU juniors and seniors with a Natural Science major score better than TTU juniors and seniors with another major on these additional questions.

It appears that on average the juniors from the lowest GPA group scored lower than the other participants, while there does not appear to be much difference in average scores between the other GPA groups. These differences are not statistically significant at the 0.05 level (see Appendix D for details). This suggests that on average a participants’ GPA does not have an impact on participants’ scores for these additional natural science questions.

61.4%

48.6%

60.6%

51.5%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

Natural Science Major

Other Major

Average AQ Scores by Major

Juniors

Seniors

58.6%

48.6% 47.1%53.0%

11.1%

49.5% 51.6%55.3% 55.9%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

below 2.00 2.01 - 2.50 2.51 - 3.00 3.01 - 3.50 3.51 and above

Average AQ Scores by GPA

Juniors

Seniors



Relationship between CAAP Scores and Additional Questions Scores

The following table shows how participants’ CAAP scores correlate with their overall additional questions (AQ) scores, with their partial additional questions scores (scores for only the first six additional questions that have been included in past administrations of the OSA), and with each separate additional question.

Correlations with CAAP Scores

AQ Score

AQ Partial AQ1 AQ2 AQ3 AQ4 AQ5 AQ6 AQ7 AQ8 AQ9

Correlation 0.34 0.30 0.06 0.12 0.19 0.19 0.19 0.17 0.08 0.21 0.12

p - value < 0.0001 < 0.0001 0.3195 0.0337 0.0010 0.0013 0.0015 0.0062 0.2103 0.0012 0.0678

n 230 230 291 289 283 276 272 269 248 242 237

The correlation between participants’ CAAP scores and participants’ additional questions scores (0.34) is statistically significant at the 0.05 level. This means that there is a relationship between participants’ CAAP scores and participants’ additional questions scores. This relationship may suggest that the CAAP and the additional questions are measuring similar domains of knowledge or it might suggest that participants’ performance is in some other way related (e.g., participants that do well on tests do well on both the CAAP and the additional questions). Looking at each additional question it appears that the amount of correct responses for most of the additional questions was significantly correlated with CAAP scores at the 0.05 level. The correlations between CAAP scores and correct responses for additional questions 1, 7, and 9 were not statistically significant at the 0.05 level. This might suggest that these questions are less related to the knowledge assessed by the CAAP science section.



The following graph shows how participants’ additional questions scores relate to their CAAP scores.

The following table shows a summary of the linear regression model using participants’ additional questions scores as a predictor for participants’ CAAP scores.

Regression Model of CAAP Scores by Additional Questions Scores

n F value p value R2

230 31.03 < 0.0001 0.1198

Variable Coefficient Stand Error p value

Intercept 57.44 0.77 < 0.0001

AQ Score 7.74 1.39 < 0.0001

The table confirms that participants’ additional questions scores are significantly related to participants’ CAAP scores. The R2 value of 0.1198 reported in the table means that approximately 12% of the variance in CAAP scores can be explained by the variance in additional questions scores. This again suggests that participants’ additional questions scores and CAAP scores are related.

40

45

50

55

60

65

70

75

0.000 0.100 0.200 0.300 0.400 0.500 0.600 0.700 0.800 0.900 1.000

CA

AP

Sco

re

Additional Questions Score

CAAP Scores by AQ Scores



The following table shows a summary of the linear regression model using participants’ scores from only the first six additional questions as a predictor for participants’ CAAP scores. The first six additional questions are the only questions that have been administered in past administrations of the OSA.

Regression Model of CAAP Scores by First Six Additional Questions Scores

n F value p value R2

230 22.08 < 0.0001 0.0883

Variable Coefficient Stand Error p value

Intercept 58.13 0.76 < 0.0001

AQ Score 5.22 1.11 < 0.0001

The table shows that participants’ scores for the first six additional questions are also significantly related to participants’ CAAP scores. The R2 value of 0.0883 reported in the table means that approximately 9% of the variance in CAAP scores can be explained by the variance in scores for the first six additional questions. This suggests that including all nine additional questions provides a better predictor of CAAP scores.



Conclusion Overall it appears that TTU juniors score similarly on average to other juniors nationally and that TTU seniors score slightly higher on average than other seniors nationally on the Science Test from the CAAP. Considering the CAAP scores by sex suggests that on average male juniors and seniors score higher than female juniors and seniors. Considering the CAAP scores by major suggests that on average juniors and seniors with a Natural Science major score higher than juniors and seniors with another major. Considering CAAP scores by motivation suggests that students reporting that they were more motivated in putting forth effort on the assessment score higher. GPA does not appear to have a significant influence on CAAP scores for TTU juniors and seniors. Considering scores for the additional questions by sex, the female juniors scored higher on average than the male juniors and the female seniors scored lower on average than the male seniors, but these differences were not statistically significant at the 0.05 level. This lack of significance might be due to a lot of variance in the additional questions scores. Considering scores for the additional questions by major suggests that on average juniors and seniors with a Natural Science major score higher than juniors and seniors with another major. GPA does not seem to have a significant influence on additional questions scores. The interpretation of additional questions scores needs to consider that these questions were administered at the end of participants’ testing. The patterns in additional questions scores are likely influenced by participants potentially rushing through this last part of the test. The correlation between additional questions scores and CAAP scores suggests that the additional questions scores are related to CAAP scores. This is also true for the partial score including only the first six additional questions that have been administered in past administrations of the OSA. This relationship may suggest that the CAAP and the additional questions are measuring similar domains of knowledge or that they are in some other way related (e.g., students that do well on tests do well on both the CAAP and the additional questions). Comparing the R2 values for the regression equation using scores for all nine questions and the regression equation using scores for only the first six questions suggests that the nine additional questions might be a better predictor of CAAP scores than the first six questions alone.



Appendix

Appendix A: Information about the CAAP Science Test8

8 Screenshots from http://www.act.org/caap/test_science.html (accessed on 11/29/10)

http://www.act.org/caap/test_science.html



Appendix B: Copy of the Additional Questions

Student ID: R______________________________

INSTRUCTIONS: Please turn to page 3 in your answer document and find block P, Additional

Questions. Each question below corresponds to a column in block P. Read the question and

then mark only one circle in each column that best describes your response. For example, if for

question A you think answer option 7 is correct, you would bubble in the 7 in column A, etc.

QUESTIONS Column A Which of the following is NOT a property that defines life?

0. Metabolism 1. Heredity 2. Homeostasis 3. Movement 4. Cellular organization

Column B The increased use of antibiotics in people and livestock is leading to:

0. increased mutation rates in bacteria 1. decreased mutation rates in bacteria 2. selection against antibiotic-resistant bacteria



3. selection for antibiotic-resistant bacteria 4. does not have effect on the evolution of bacteria

Column C The oldest oceanic crust on Earth formed approximately 160 million years ago. In view of the fact that Earth is 4.6 billion years old, why is oceanic crust so young?

0. Oceanic crust is continuously being destroyed by subduction. 1. Oceanic crust first began to form 160 million years ago; before that no oceans existed. 2. Older oceanic crust exists but has not been dated. 3. There is no difference between oceanic and continental crust.

For the following paragraphs select the terms that best complete the blanks in the sentences (Columns D - F). Sally the Scientist visited her father, who told her of a new product he is using to cure his baldness. Called “Bald Away,” it is an oil derived from a rare tropical tree and is very expensive. His doctor showed him a short film about the product where three men talked about the effectiveness of the oil and showed before and after pictures of their heads. Sally was worried that the so-called cure was really just an example of “Pseudoscience” because the only reason given to accept the conclusions was anecdotal evidence. Sally decided to study the effectiveness of the oil to help her dad decide whether to purchase the product or not. She wrote down, “When following the direction on the bottle, use of the oil leads to improved hair growth on bald men after one week.” This statement serves as her Column D.___________ for the study. To conduct the experiment, she recruited fifty bald men and the 25 in Group A were given “Bald Away” and Group B was given another oil that she knew did not affect hair growth.

0. non-control group 1. control group 2. conclusion 3. hypothesis

The fifty men served as her sample. Group B served as the Column E.___________ in the experiment.

0. non-control group 1. control group 2. conclusion 3. hypothesis



In order to make her study more scientifically valid, she randomly determined which group each man was put into and made sure that none of them knew if they were using Bald Away or the other oil. In addition, she had an assistant keep track of which group each man was in so that she did not know either. Because the men did not know what product they were using and neither did she, it is called a double blind study and she reduced the bias in the experiment. After a week, Sally met with each of the men and determined that 18 of them had increased hair growth. When her assistant told her which group each of the men was in, she found that 21 out of 25 using Bald Away had more hair and 2 out of 25 from the other group showed an increase. She concluded that her hypothesis was Column F.___________ and prepared a journal article explaining the experiment and its implications.

0. absolutely proved 1. supported 2. not supported 3. absolutely disproved

She sent it to the Journal of Bald Studies and the editor sent it to three baldness experts to evaluate the study, a process called peer review. They concluded that the study was properly done and the results can be published. Column G. You have collected data from a replicated experiment that has examined the impacts of increasing levels of atmospheric CO2 on the yearly growth of three tree species: red oak, loblolly pine, and western red cedar. Your results indicate that the different plant species respond differently to increasing amounts of CO2. In addition, you discover that red oaks under the highest levels of CO2 have the largest amounts of new shoot growth. How would you determine if the differences in mean shoot growth among the three tree species in response to varying levels of CO2 are accurate and not a random response?

0. Conduct a second set of experiments to see if similar results are obtained as in the first. 1. Conduct a statistical analysis of the data to determine if the mean shoot growth values

for each plant/CO2 level combination are significantly different from each other at an established probability.

2. Rank the values obtained from low to high and determine how close the values are to each other. This measure of similarity will establish if the outcome is random or not.

3. All of the above are acceptable approaches for determining treatment versus random effects.

Column H.



When you turn on your kitchen faucet some of the water that comes out of the tap has already been through several sets of kidneys before you have taken that first sip. Which of the following statements concerning watersheds, water use planning, and the water cycle should impact our water use and water management decisions?

0. In many cities and towns, grey water passing through wastewater treatment plants is directly routed back into rivers and streams after processing. This water is subsequently used by cities and towns downriver from the point of origin.

1. All of our water, whether it comes from reservoirs, a well, or even a bottle is affected by how people use the land through which water flows.

2. The water we all drink has been recycled through natural processes, human engineered systems, or both.

3. The goal of watershed management is to balance multiple, competing demands, using scientific studies in combination with human judgments to make decisions on how we use our water resources.

4. All of the above could contribute to developing a comprehensive water management plan.

Column I. Tomatoes are highly valued around the world; however fruit that is picked before fully ripening or allowed to over ripen can compromise the flavor and texture of the harvest before it reaches the consumer.

Which of the following approaches could be examined in a research context to produce a more appetizing tomato?

0. Identify and block late-stage tomato-ripening processes so that producers can selectively activate ripening at the time of sales.

1. Locate the producer closer to the consumer so that farm products can be more rapidly transported at the time of harvest.

2. Alert consumers when tomatoes are in season so that people can plan when to eat fresh tomatoes.

3. 0 and 2 are both correct responses 4. 0, 1 and 2 are all correct responses.



Appendix C: Analysis of CAAP Scores by Sex, Major, GPA, and Motivation

Analysis of CAAP Scores by Sex - Juniors n Mean St. Dev. t value p value

Female 68 60.6 3.84 2.29 0.0231

Male 93 62.1 4.17

Analysis of CAAP Scores by Sex - Seniors n Mean St. Dev. t value p value

Female 77 60.6 3.73 1.99 0.0481

Male 93 61.9 4.33

Analysis of CAAP Scores by Major - Juniors n Mean St. Dev. t value p value

Natural Science Major 17 64.3 4.3 3.13 0.0021

Other Major 144 61.1 3.9

Analysis of CAAP Scores by Major - Seniors n Mean St. Dev. t value p value

Natural Science Major 23 63.4 3.6 2.67 0.0084

Other Major 147 61.0 4.1



Analysis of CAAP Scores by GPA - Juniors n Mean St. Dev. F value p value

below 2.00 1 62.0 . 2.13 0.0795

2.01 - 2.50 22 62.6 4.58 2.51 - 3.00 41 62 3.80 3.01 - 3.50 56 60.3 3.91 3.51 and above 38 62.1 4.08

Analysis of CAAP Scores by GPA - Seniors n Mean St. Dev. F value p value

below 2.00 1 54.0 . 1.86 0.1206

2.01 - 2.50 12 61.6 3.63 2.51 - 3.00 49 60.4 4.71 3.01 - 3.50 58 62.0 3.69 3.51 and above 48 61.5 3.88

Analysis of CAAP Scores by Motivation - Juniors n Mean St. Dev. F value p value

tried my best 37 62.6 4.32 3.88 0.0049

gave moderate effort 99 61.7 3.85 gave little effort 16 58.9 2.92 gave no effort 2 57 1.41

Analysis of CAAP Scores by Motivation - Seniors n Mean St. Dev. F value p value

tried my best 56 62.7 4.08 4.61 0.0015

gave moderate effort 78 61.1 3.70 gave little effort 23 59.7 3.51 gave no effort 4 56.0 4.83



Appendix D: Analysis of Additional Questions Scores by Sex, Major, GPA, and

Motivation

Analysis of Additional Questions Scores by Sex - Juniors n Mean St. Dev. t value p value

Female 45 53.1% 15.0% 1.16 0.2498

Male 60 48.9% 20.6%

Analysis of Additional Questions Scores by Sex - Seniors n Mean St. Dev. t value p value

Female 57 50.5% 18.9% 1.36 0.1769

Male 68 55.1% 18.7%

Analysis of Additional Questions Scores by Major - Juniors n Mean St. Dev. t value p value

Natural Science Major 17 61.4% 18.5% 2.71 0.0079

Other Major 88 48.6% 17.8%

Analysis of Additional Questions Scores by Major - Seniors n Mean St. Dev. t value p value

Natural Science Major 20 60.6% 14.6% 1.99 0.0492

Other Major 105 51.5% 19.3%



Analysis of Additional Questions Scores by GPA - Juniors n Mean St. Dev. F value p value

below 2.00 0 . . 1.89 0.1357

2.01 - 2.50 18 58.6% 13.6% 2.51 - 3.00 27 48.6% 22.3% 3.01 - 3.50 38 47.1% 18.3% 3.51 and above 22 53.0% 15.3%

Analysis of Additional Questions Scores by GPA - Seniors n Mean St. Dev. F value p value

below 2.00 1 11.1% . 1.78 0.1370

2.01 - 2.50 11 49.5% 17.5% 2.51 - 3.00 39 51.6% 18.6% 3.01 - 3.50 37 55.3% 18.6% 3.51 and above 35 55.9% 18.4%

Analysis of Additional Questions Scores by Motivation - Juniors n Mean St. Dev. F value p value

tried my best 24 57.4% 16.9% 1.16 0.3348

gave moderate effort 68 48.9% 18.9% gave little effort 10 46.7% 18.0% gave no effort 1 44.4% .

Analysis of Additional Questions Scores by Motivation - Seniors n Mean St. Dev. F value p value

tried my best 42 54.5% 18.5% 2.55 0.0426

gave moderate effort 58 54.8% 18.9% gave little effort 16 52.8% 11.8% gave no effort 3 29.6% 23.1%



Appendix E: CAAP Scores and Additional Questions Scores by Student

Classification

Analysis of CAAP Scores by Student Classification n Mean St. Dev. t value p value

Junior 161 61.4 4.09 0.3 0.7643

Senior 170 61.3 4.10

Analysis of Additional Questions Scores by Student Classification n Mean St. Dev. t value p value

Junior 105 50.7% 18.4% 0.93 0.3545

Senior 125 53.0% 18.8%



Appendix F: CAAP Scores and Additional Questions Scores by Sex, Major, and

Minor

Natural Sciences Majors Male Female Total

BIOL 2 2 4

CHEM 2 0 2

GEOS 24 9 33

PHYS 0 1 1

Total 28 12 40

Natural Sciences Minors Male Female Total

ATMO 4 5 9

BIOL 0 2 2

CHEM 5 3 8

GEOL 1 0 1

GEOP 3 0 3

PHYS 1 0 1

Total 14 10 24



Median CAAP Scores by Sex and Major

Male Female Total

Natural Science Major 64.5 64 64

Other Major 62 60 61

All 62 60 62

Median CAAP Scores by Sex and Minor

Male Female Total

Natural Science Minor 63.5 63.5 63.5

Other Minor 62 60 62

All 62 60 62

Median CAAP Scores by Sex and Major or Minor

Male Female Total

Natural Science Major or Minor 64 64 64

Other Major and Minor 62 60 61

All 62 60 62

Mean CAAP Scores by Sex and Major

Male Female Total

Natural Science Major 63.5 64.4 63.8

Other Major 61.7 60.3 61.0

All 62.0 60.6 61.4

Mean CAAP Scores by Sex and Minor

Male Female Total

Natural Science Minor 63.1 63.7 63.3

Other Minor 61.9 60.4 61.2

All 62.0 60.6 61.4

Mean CAAP Scores by Sex and Major/Minor

Male Female Total

Natural Science Major or Minor 63.5 63.6 63.5

Other Major and Minor 61.6 60.2 61.0

All 62.0 60.6 61.4



Median Additional Questions Scores by Sex and Major

Male Female Total

Natural Science Major 66.7% 66.7% 66.7%

Other Major 50.0% 55.6% 55.6%

All 55.6% 55.6% 55.6%

Median Additional Questions Scores by Sex and Minor

Male Female Total

Natural Science Minor 55.6% 66.7% 66.7%

Other Minor 55.6% 55.6% 55.6%

All 55.6% 55.6% 55.6%

Median Additional Questions Scores by Sex and Major or Minor

Male Female Total

Natural Science Major or Minor 66.7% 66.7% 66.7%

Other Major and Minor 44.4% 55.6% 55.6%

All 55.6% 55.6% 55.6%

Mean Additional Questions Scores by Sex and Major

Male Female Total

Natural Science Major 59.0% 65.7% 61.0%

Other Major 50.4% 49.9% 50.2%

All 52.2% 51.6% 51.9%

Mean Additional Questions Scores by Sex and Minor

Male Female Total

Natural Science Minor 54.4% 64.8% 58.3%

Other Minor 52.0% 50.8% 51.5%

All 52.2% 51.6% 51.9%

Mean Additional Questions Scores by Sex and Major/Minor

Male Female Total

Natural Science Major or Minor 60.3% 64.8% 61.7%

Other Major and Minor 49.9% 49.9% 49.9%

All 52.2% 51.6% 51.9%

collegiate assessment of academic proficiency: science ...€¦ · science, and critical thinking....

Documents