research design and statistical analysis - 4th ed 2006

8/15/2019 Research Design and Statistical Analysis - 4th Ed 2006

1/318

i-8

Research Design and Statistical Analysis in Christian Ministry

© 4th ed. 2006 Dr. Rick Yount

Table of Contents

Unit I: Research Fundamentals1 ........................................................................................................................Scientific Knowing

Ways of Knowing 1-1Common Sense 1-1

Authority 1-2Intuition/Revelation 1-2Experience 1-3Deductive Reasoning 1-3Inductive Reasoning 1-3

Science as a Way of Knowing 1-4Objectivity 1-4Precision 1-4Verification 1-5Empiricism 1-5Goal: Theories 1-5

The Scientific Method 1-6Types of Research 1-6

Historical Research 1-7 Primary sources 1-7Secondary sources 1-7Criticism 1-7Examples 1-8

Descriptive Research 1-8An Example 1-8

Correlational Research 1-9An Example 1-9

Experimental Research 1-9An Example 1-10

Ex Post Facto Research 1-10An Example 1-10

Evaluation 1-10An Example 1-11

Research and Development 1-11An Example 1-11

Qualitative Research 1-11Faith and Science 1-12

Suspicion of Science By the Faithful 1-12Suspicion of Religion By the Scientific 1-13There Need Be No Conflict 1-13

Summary 1-14Vocabulary 1-15

Study Questions 1-15Sample Test Questions 1-15

2 ........................................................................................................................Proposal Organization

Front Matter 2-2Title Page 2-2Table of Contents 2-2List of Tables 2-2List of Illustrations 2-3

Introduction 2-3The Introductory Statement 2-3

Preface i-1Table of Contents i-5


2/318

i-9

Preliminaries


The Statement of the Problem 2-3Purpose of the Study 2-4Synthesis of Related Literature 2-4Significance of the Study 2-6The Hypothesis 2-6

Method 2-7Population 2-7 Sampling 2-8

Instrument 2-8Limitations 2-9

Assumptions 2-10Definitions 2-10Design 2-11Procedure for Collecting Data 2-12

Analysis 2-12Procedure for Analyzing Data 2-12Testing the Hypotheses 2-13Reporting the Data 2-13

Reference Material 2-13 Appendices 2-13

Bibliography, or Cited Sources 2-14Practical Suggestions 2-14

Personal Anxiety 2-14Professionalism in Writing 2-15

Clear Thinking 2-15Unified Flow 2-15Quality Library Research 2-15Efficient Design 2-15Accepted Format 2-15

Summary 2-16Vocabulary 2-16Study Questions 2-16Sample Test Questions 2-17

3 ........................................................................................................................Empirical Measurement 3-1

Variables and Constants 3-1Independent Variables 3-2Dependent Variables 3-2

Measurement Types 3-2Nominal Measurement 3-2Ordinal Measurement 3-2Interval Measurement 3-2Ratio Measurement 3-3Data Type Summary 3-3

Operationalization 3-3Definitions 3-4

An Example 3-4 Another Example 3-5Operationalization Questions 3-6

Summary 3-6Vocabulary 3-7 Study Questions 3-7 Sample Test Questions 3-7

4 ........................................................................................................................Getting On Target 4-1

The Problem Statement 4-1Characteristics of a Problem 4-1

Limit scope of your study 4-1Current theory and/or latest research 4-1


3/318

i-10



Meaningfulness 4-2Clearly written 4-2

Examples of Problem Statements 4-2Association Between Two Variables 4-2Association of several variables 4-2Difference Between Two Groups 4-3Differences Between More Than Two Groups 4-3

The Hypothesis Statement 4-4

The Research Hypothesis 4-4Association Between Two Variables 4-4Association of several variables 4-5Difference Between Two Groups 4-5Differences Between More Than Two Groups 4-6

The Directional Hypothesis 4-6The Non-directional Hypothesis 4-7 The Null Hypothesis 4-7

Revision Examples 4-8Example 1 4-8

Comments 4-8Suggested revision 4-8

Example 2 4-8Comments 4-8Suggested revision 4-9



Example 5 4-10Comments 4-10

Dissertation Examples 4-10Regression Analysis 4-10Correlation of Competency Rankings 4-11Factorial Analysis of Variance 4-11Chi-Square Analysis of Independence 4-11

5 ........................................................................................................................Introduction to Statistical Analysis 5-1

Statistics, Mathematics, and Measurement 5-1Descriptive Statistics 5-2Inferential statistics 5-2Statistics and Mathematics 5-2Statistics and Measurement 5-2

A Statistical Flow Chart 5-4Question One: Similarity or Difference? 5-4-1- Question Two: Data Types in Similarity Studies 5-4-2- Question Two: Data Types in Difference Studies 5-4

-3- Interval or Ratio Correlation 5-4-4- Ordinal Correlation 5-5-5- Nominal Correlation 5-5-6- Interval/Ratio Differences 5-6-7- Ordinal Differences 5-7

Summary 5-7Vocabulary 5-7 Study Questions 5-8Sample Test Questions 5-8

6 ........................................................................................................................Synthesis of Related Literature 6-1

A Definition 6-1Synthetic Narrative 6-1Recent Research 6-1


4/318


5/318

i-12




8 ........................................................................................................................Collecting Dependable Data 8-1

Validity 8-1Content Validity 8-2

Predictive Validity 8-2Concurrent Validity 8-2Construct Validity 8-3

Reliability 8-3Coefficient of Stability 8-4Coefficient of Internal Consistency 8-4Coefficient of Equivalence 8-5

Reliability and Validity 8-5 Answer 1: A Test Must be Reliable in Order to be Valid 8-5 Answer 2: A Test Can be Valid Even If It Isn’t Reliable 8-5

Objectivity 8-6Summary 8-7

Vocabulary 8-7 Study Questions 8-8Sample Test Questions 8-8

Unit II: Research Methods9 ........................................................................................................................Observation 9-1

The Problem of the Observation Method 9-1Obstacles to Objectivity in Observation 9-2

Personal Interest 9-2Early decision 9-2Personal characteristics 9-3

Practical Suggestions for Avoiding these Problems 9-3Definition 9-3Familiar Groups 9-3Unfamiliar Groups 9-3Observational Limits 9-3

Manual versus Mechanical Recording 9-3Interviewer Effect 9-3Debrief Immediately 9-4Participant Observation 9-4Undercover Observation? 9-4Observational Checklist 9-4

Summary 9-4Example 9-4Vocabulary 9-5


10 ......................................................................................................................Survey Research 10-1

The Questionnaire 10-1 Advantages 10-1

Remote subjects 10-1Researcher influence 10-1Cost 10-2Reliability 10-2Subjects’ convenience 10-2


6/318


7/318


8/318

i-15

Preliminaries


Formatting the Scale 12-4Write instructions 12-4Scoring the Likert scale 12-4

The Thurstone Scale 12-4Develop item pool 12-5Compute item weights 12-5Rank the items by weight 12-6Choose Equidistant Items 12-6Formatting the Scale 12-6

Administering the Scale 12-6Scoring 12-6

Q-Methodology 12-6Semantic Differential 12-7Delphi Technique 12-7Summary 12-8

Vocabulary 12-8Study Questions 12-8Sample Test Questions 12-8Sample Thurstone Scale 12-9Sample Thurstone Scale (with weights) 12-10

13 ......................................................................................................................

Experimental Designs 13-1What Is Experimental Research? 13-1Internal Invalidity 13-2

History 13-2 Maturation 13-2Testing 13-2Instrumentation 13-3Statistical regression 13-3Differential selection 13-3Experimental mortality 13-4Selection-Maturation Interaction of Subjects 13-4The John Henry Effect 13-4Treatment diffusion 13-4

External Invalidity 13-4Reactive effects of testing 13-5Treatment and Subject Interaction 13-5Testing and Subject Interaction 13-5

Multiple Treatment Effect 13-5Summary 13-5

Types of Designs 13-6True Experimental Designs 13-6

Pretest-Posttest Control Group 13-6Posttest Only Control Group 13-6Solomon Four-Group 13-7

Quasi-experimental Designs 13-7 Time Series 13-7Nonequivalent Control Group Design 13-8Counterbalanced Design 13-8

Pre-experimental Designs 13-9The One Shot Case Study 13-9One-Group Pretest/Posttest 13-9Static-Group comparison 13-10

Summary 13-10Vocabulary 13-10Study Questions 13-11Sample Test Questions 13-11


9/318


10/318

i-17

Preliminaries


Sample Statistics 16-9Estimated Parameters 16-9

Standard (z-) Scores 16-10Summary 16-12

Example 16-12Vocabulary 16-13Study Questions 16-14Sample Test Questions

16-1517 ......................................................................................................................The Normal Curve and Hypothesis Testing 17-1

The Normal Curve 17-1The Normal Curve Table 17-2The Normal Curve Table in Action 17-3

Level of Significance 17-6Criticial Values 17-6One- and Two-Tailed Tests 17-6

Sampling Distributions 17-7The Distinction Illustrated 17-8Using the z-Formula for Testing Group Means 17-9Computing Probabilities of Means 17-9

Summary 17-10Example 17-10Vocabulary 17-11Study Questions 17-11Sample Test Questions

17-13

18 ......................................................................................................................The Normal Curve: Error Rates and Power 18-1

Type I and Type II Error Rates 18-1Decision Table Probabilities 18-2Normal Curve Areas 18-2

Increasing Statistical Power 18-4Increase α 18-4Increase µ 1 - µ 2 18-4Decrease the Standard Error of the Mean 18-5

Decrease s 18-5Increase n 18-5Like Fishing for Minnows 18-6

Statistical Significance and Practical Importance 18-6Summary 18-6

Vocabulary 18-7 Study Questions 18-7 Sample Test Questions 18-7

Unit IV: Statistical Procedures

19 ......................................................................................................................One Sample Parametric Tests 19-1The One-Sample z-Test 19-1The One-Sample t-Test 19-2

The t-Distribution Table 19-2Computing t 19-3

Confidence Intervals 19-4 A z-Score Confidence Interval 19-4 A t-Score Confidence Interval 19-5

Summary 19-5


11/318


12/318

i-19

Preliminaries


Summary 22-8Vocabulary 22-8Study Questions 22-8Sample Test Question 22-9

23 ......................................................................................................................Chi-Square Procedures 23-1

The Chi Square formula 23-1The Goodness of Fit Test 23-2

Equal Expected Frequencies 23-2The Example of a Die 23-2Computing the Chi Square 23-2Testing the Chi Square Value 23-3Translating into English 23-3

Proportional Expected Frequencies 23-3The Example of Political Party Preference 23-3Computing the Chi Square Value 23-3Testing the Chi Square 23-4Translate into English 23-4Eyeball the Data 23-4

Chi-Square Test of Independence 23-4The Contingency Table 23-5

Expected Cell Frequencies 23-5Degrees of Freedom 23-6 Application to a Problem 23-6Party Preference Revisited 23-7 Strength of Association 23-8

Contingency Coefficient 23-8Cramer’s Phi 23-9

Cautions in Using Chi-Square 23-9Small expected frequencies 23-9

Assumption of Independence 23-10Inclusion of Non-Occurrences 23-10

Summary 23-11Example 23-11Vocabulary 23-12


Unit V: Advanced Statistical Procedures24 ......................................................................................................................Non-Parametric Statistics for Ordinal Differences 24-1

The Rationale of Testing Ordinal Differences 24-2Wilcoxin Rank-Sum Test (W s) 24-2

Computing the Wilcoxin W 24-3The Wilcoxin W Table 24-3

The Mann-Whitney U Test 24-3Computing the Mann-Whitney U 24-3The Mann-Whitney U Table 24-4

Wilcoxin Matched-Pairs Test (T) 24-4Computing the Wilcoxin T 24-4The Wilcoxin T Table 24-5

Kruskal-Wallis H Test 24-5Computing the Kruskal-Wallis H 24-5Using the Chi-Square Table with Kruskal-Wallis H 24-6

Summary 24-6Example 24-6Vocabulary 24-8Study Questions 24-8Sample Test Questions 24-8


13/318


14/318


15/318

1-1

Scientific KnowingChapter 1


Have you considered how you know what you know? As you sit in classes or talkwith friends, have you noticed that people differ in the way they know things? Look atsix students who are discussing the issue of "modern translations" of the Bible.

Student 1: "I use the King James Version because that's the translation I grew up using.Everybody in our church back home uses it."

Student 2: "I use the New King James because my pastor says it offers the best of beautyand modern scholarship."

Student 3: "I've prayed about what version to use. I like the Amplified Version because itis so clear in its language. It just feels right. "

Student 4: " I've tried five or six different translations for devotional reading and for

preparation for teaching in Sunday School. After evaluating each one, I've come backagain and again to the New International Version. It's the best translation for me. "Student 5: "The essense of Bible study is understanding the message, whatever transla-

tion we may use. Therefore, I use different translations depending on my studygoals."

Student 6: "I use the New King James because most of my congregation is familiar withit. In a recent survey, I found that 84% of our members use the KJV or NKJV ."

Each of these students reflect a different basis for knowing which translation to use.Which student most closely reflects your view? How did you come to know what youknow?

Ways of KnowingAs we begin our study of research design and statistical analysis, we need to

understand the characteristics of scientific knowing, and how this kind of knowingdiffers from other ways we learn about our world. We will first look at five non-scien-tific ways of knowing: common sense, authority, intuition/revelation, experience, anddeductive reasoning. Then we'll analyze the scientific method, which is based on induc-tive reasoning.

11111Scientific Knowing

Ways of KnowingScience as a Way of Knowing

The Scientific MethodTypes of Research

Common Sense

Authority

Intuition/Revelation

Experience

Deductive Reasoning

Inductive Reasoning

Unit I: Research Fundamentals


16/318

1- 2

Research Design and Statistical Analysis in Christian Ministry I: Research Fundamentals


Common Sense

Common sense refers to knowledge we take for granted. We learn by absorbing thecustoms and traditions that surround us —from family, church, community and nation.We assume this knowledge is correct because it is familiar to us. We seldom question,or even think to question, its correctness because it just is. Unless we move to another

region, or go to school and study the views of others, we have nothing to challenge ourway of thinking. It's just common sense!But common sense told us that “the earth is flat” until Columbus discovered other-

wise. Common sense told us that “dunce caps and caning are effective student motiva-tors” until educational research discovered the negative aspects of punishment.Common sense may well be wrong.

Authority

Authoritative knowledge is an uncritical acceptance of another’s knowledge . Whenwe are sick, we go to the doctor to find out what to do. When we need legal help, we goto a lawyer and follow his advice. Since we can not verify the knowledge on our own,

we must simply choose to accept or reject the expert's advice. It would be foolish toargue with a doctor's diagnosis, or a lawyer's perception of a case. This is the meaningof "uncritical acceptance" in the definition above. The only recourse to accepting theexpert's knowledge is to get a second opinion—from another expert.

As Christians, we believe that God’s Word is the authority for our life and work.The Living Word—the Lord Himself—within us confirms the Truth of the WrittenWord. The Written Word confirms our experiences with the Living Word. Scripture is avalid source of authoritative knowledge.

However, we spend a lot of time discussing Scriptural interpretations. Our discus-sions often deteriorate into conflicts about “my pastor’s” interpretations. We use ourown pastor’s interpretation as authoritative because of the influence he has had in ourown life. (We can substitute any authoritative person here, such as a father or mother,Sunday School teacher, or respected colleague.)

But is the authority is correct? Authoritative knowing does not question the sourceof knowledge. Yet differing authorities cannot be correct simultaneously. How do wetest the validity of an authority’s testimony?

Intuition/RevelationIntuitive knowledge refers to truths which the mind grasps immediately, without

need for proof or testing or experimentation. The properly trained mind “intuits” thetruth naturally. The field of geometry provides a good example of this kind of knowing.Let’s say I know that Line segment A is the same length as line segment B. I also knowthat Line segment B is the same length as line segment C. From these two truths, I

immediately recognize that Line segments A and C are equal. Or, in short hand, IF A=B and B=C, THEN A=C

I do not need to draw the three lines and measure them. My mind immediatelygrasps the truth of the statement.

Revelation is knowledge that God reveals about Himself. I do not need test thisknowledge, or subject it to experimentation. When Christ reveals Himself to us, weknow Him in a personal way. We did not achieve this knowledge by our own efforts,but merely received the revelation of the Lord. We cannot prove this knowledge toothers, but it is bedrock truth to those who've experienced it. Problems arise, however,


17/318

1-3



when we apply intuitive knowing to ministry programs. “Well, it's obvious that regu-lar attendance in Sunday School helps people grow in the Lord.” Is it? We work hardat promoting Sunday School attendance. Does it actually change the lives of theattenders? Is it enough for people to think it does, whether or not real change takesplace? Answers to these questions come from clear-headed analysis, not from intuition.

ExperienceExperiential knowledge comes from “trial and error learning.” We develop it when

we try something and analyze the consequences. You've probably heard comments likethese: “We've already tried that and it failed.” Or another: “We’ve found that holdingVacation Bible School during the third week of August, in the evening, is best for ourchurch.” The first is negative. The speaker is saying there's no need to try that ministryor program again, because it was already tried. The second is positive. This church hastried several approaches to offering Vacation Bible School and found the best time forthem. Their “truth” may not apply to any other church in the association, but it is truefor them. They’ve tried it and it worked. . .or it didn’t.

Much of the promotion of new church programs comes out of this framework. Wesay, “This program is being used in other churches with great success” (which meansour church can have the same experience if we use this program). How do we evaluateprogram effectiveness? What is success? How do we measure it?

Deductive Reasoning

Deductive reasoning moves thinking from stated general principles to specificelements. We develop general over-arching statements of intent and purpose. Then wededuce from these principles specific actions we should take. Determine “world view”first. Then make daily decisions which logically derive from this perspective.

When we take the Great Commission as our primary mandate, we have framed aworld view for ministry. That is, “Whatever we do, we will connect it to reaching outand baptizing (missions and evangelism), teaching (discipleship and ministry).” Now,how do we do it? We deduce specific programs, plans, and procedures for carrying outthe mandate. We eliminate programs that conflict with this mandate.

How do we arrive at this “world view?” Are our over-arching principles correct?Have we interpreted them correctly? Correct action rises or falls on the basis of twothings. First, correct action depends on the correctness of our world view. Secondly,correct action depends on our ability to translate that view into practical ministry steps.

Inductive Reasoning

Inductive reasoning moves thinking from specific elements to general principles.Inductive Bible study analyzes several passages and then synthesizes key concepts intothe central truth. Science is inductive in its study of a number of specifics and its use ofthese results to formulate a theory. The truths derived in this way are temporary andopen to adjustment when new elements are discovered. Knowledge gained in this wayis usually related to probabilities of happenings. We have a high degree of confidencethat combining “X” and “Y” will produce effect “Z.” Or, we learn that “B” and “C” areseldom found in combination with “D.”

I can demonstrate probability by using matches. Picture yourself at the kitchentable with 100 matches. You pick up the first one . What is the probability it will lightwhen you strike it? Well, you have two possibilities: either it will or it won’t . So theprobability is 50% (1 event out of 2 possibilities). You strike it and it lights. Pick up the


18/318

1- 4



second match. The probability is 0.50 that it will light: (1 event out of two possibilities:Yes or No.) But cumulatively, out of two matches (first and second), one lit. One out oftwo is 50%. So the probability of the second match lighting is 50%, because 1 of 2 havealready lit. You strike it and it lights.

Pick up the third match. Again, the third match taken alone has “p = 0.50” oflighting (read ‘probability equals point-five-oh’). However, taking all three matches

together , two of the three have lit and the probability is 2/3 (“p = 0.66”) that the thirdmatch will light. It does.Now, pick up the fourth match. The probability is 3/4 (p=0.75) that it will light,

taking all four matches together.What about the 100th match, given that the 99 previous matches have all lit ? The

probability is 0.50 for this particular match (yes, no) , but p = 0.99 taking all matchestogether. The probability is very high! Yet we cannot absolutely guarantee it will light.

This is the nature of inductive logic, and inductive logic is the basis of scientificknowledge. By definition, science does not deal with absolute Truth. Science seeksknowledge about processes in our world. Researchers gather information throughobservation. They then mold this information into theories. The scientific communitytests these theories under differing conditions to establish the degree to which they canbe generalized. The result is temporary, open-ended truth (I call it little-t truth to distin-guish it from absolute Truth). This kind of truth is open for inquiry, further testing, andprobable modification. While this kind of knowing can add nothing to our faith, it isvery helpful in solving ministry problems.

Science as a Way of Knowing

Scientific knowing is based on precise data gathered from the natural world we livein. It builds a knowledge base in a neutral, unbiased manner. It seeks to measure theworld precisely. It reports findings clearly so that others can duplicate the studies. Itforms its conclusions on empirical data. Let’s look at these ideals more closely.

ObjectivityHuman beings are complex. Personal experiences, values, backgrounds, and beliefs

make objective analysis difficult unless effort is made to remain neutral. Optimists tendto see the positive in situations. Pessimists see the negative. But scientists look forobjective reality — the world as it is — uncolored by personal opinion or feelings.

Scientific knowing attempts to eliminate personal bias in data collection and analy-sis. Honest researchers take a neutral position in their studies. That is, they do not try toprove their own beliefs. They are willing to accept empirical results contrary to theirown opinions or values.

Precision

Reliable scientific knowing requires precise measurement. Researchers carry outexperiments under controlled, narrow conditions. They carefully design instruments tobe as accurate as possible. They evaluate tests for reliability and validity. They use pilotprojects (trial runs of procedures) to identify sources of extraneous error in measure-ments. Why? Because inaccurate measurement and undefined conditions and unreli-able instruments and extraneous errors produce data that is worthless. Every score hastwo parts: the true measure of the subject, and an unknown amount of error. We canrepresent this as

Objectivity

Precision

Verification

Empiricism

Goal: Theories


19/318


20/318


21/318

1-7



tific method and the research proposal.

Types of ResearchUnder the umbrella of scientific research, there are several types of studies you can

do. These types differ in procedure — what they entail — and outcome — what they

accomplish. Here are four major and three minor types of research from which you maychoose.

Historical ResearchHistorical research analyzes the question “what was?” It studies documents and

relics in order to determine the relationship of historic events and trends to present-daypractice.

Primary sources

A source of information is primary when it is produced by the researcher. Reportswritten by researchers who conduct studies are “eye witness” accounts, and are pri-

mary sources of information on the results. Other examples of primary sources areautobiographies and textbooks written by authors who conduct their own research. Useprimary sources as the major source of information in the Related Literature sectionof your proposal. Primary sources take two forms: documents and relics.

Documents . Society creates documents to expressly record events. They areobjective and direct. Documents provide straightforward information. Average BibleStudy attendance listed on the Annual Church Letters on file in the state conventionoffice is more likely to be accurate than numbers given from memory by ministers ofeducation in local churches. However, information contained in documents may beincorrect. The documents may have been falsified, or word meanings in the documentsmay have changed.

Relics . Society creates relics simply by living. Relics are artifacts left by commu-nities and cultures in the past. People did not create these objects to record informationas is the case with documents. Therefore, information conveyed by relics requiresinterpretation. The historical researcher reconstructs the meaning of relics in the contextof their time and place.

Secondary sourcesA source of information is secondary when it is a second-hand account of research.

Secondary sources may take the form of summaries, news stories, encyclopedias, ortextbooks written by synthesizers of research reports. While secondary sources providethe bulk of materials used in term papers, you should use them only to provide a broadview of your chosen topic. As already stated, emphasize the use of primary sources inyour Synthesis of Related Literature.

CriticismThe term “criticism” has a decidedly negative connotation to most of us. A critical

person is one who finds fault, depreciates, or puts down someone or something. Theterm comes from the Greek krino, to judge . Webster defines “criticism” as the “art, skill,or profession of making discriminating judgments and evaluations, especially of literaryor other artistic works." 2 Criticism can therefore refer to praise as well as depreciation . A

HistoricalDescriptive

Correlational

Experimental

Ex Post Facto

Evaluation

Research/Dev

2"Criticism," The American Heritage Dictionary , 3rd ed., Version 3.0A, WordStar International, 1993.


22/318


23/318

1-9



Another application of descriptive research is whether two or more variables arerelated within a group. This latter type of study, while descriptive in nature, is oftenreferred to specifically as correlational research (see the next section).

An Example

The goal of descriptive research is to accurately and empirically describe differencesbetween one or more variables in selected groups. Dr. Dan Southerland studied differ-ences in ministerial roles and allocation of time between growing and plateaued ordeclining Southern Baptist churches in Florida. 6 Specified roles were pastor, worshipleader, organizer, administrator, preacher and teacher. 7 The only role which showedsignificant difference between growing and non-growing churches was the amount oftime spent serving as “organizer,” which included “vision casting, setting goals, leadingand supervising change, motivating others to work toward a vision, and buildinggroupness.” 8

Correlational Research

Correlational research is often presented as part of the descriptive family of meth-ods. This makes sense since correlational research describes association between vari-ables of interest in the study. It answers the question “what is” in terms of relationshipamong two or more variables. What is the relationship between learning style andgender? What is the relationship between counseling approach and client anxiety level?What is the relationship between social skill level and job satisfaction and effectivenessfor pastors? In each of these questions we have asked about an association between twoor more variables.

Correlational research also includes the topics of linear and multiple regressionwhich uses the strengths of associations to make predictions. Finally, correlationalanalysis includes advanced procedures like Factor Analysis, Canonical Analysis, Dis-criminant Analysis, and Path Analysis — all of which are beyond the scope of this

course.

An Example

The goal of correlational research is to establish whether relationships exist be-tween selected variables. Dr. Robert Welch studied selected factors relating to jobsatisfaction in staff organizations in large Southern Baptist Churches. 9 He found themost important intrinsic factors affecting job satisfaction were praise and recognition forwork, performing creative work and growth in skill. The most important extrinsicfactors were salary, job security, relationship with supervisor, and meeting familyneeds. 10 Findings were drawn from 579 Southern Baptist ministers in 153 churches. 11

Experimental ResearchExperimental research analyzes the question “what if?” Experimental studies use

carefully controlled procedures to manipulate one (independent) variable, such as

6Dan Southerland, “A Study of the Priorities in Ministerial Roles of Pastors in Growing FloridaBaptist Churches and Pastors in Plateaued or Declining Florida Baptist Churches,” (Ed.D. diss., Southwest-ern Baptist Theological Seminary, 1993)

7Ibid., 1 8Ibid., 29Robert Horton Welch, “A Study of Selected Factors Related to Job Satisfaction in the Staff Organiza-

tions of Large Southern Baptist Churches,” (Ed.D. diss., Southwestern Baptist Theological Seminary, 1990)10Ibid., 2 11Ibid., 61


24/318

1- 10



Teaching Approach, and measure its effect on other (dependent) variables, such asStudent Attitude and Achievement. Manipulation is the distinguishing element inexperimental research. Experimental researchers don’t simply observe what is. Theymanipulate variables and set conditions in order to design the framework for theirobservations. What would be the difference in test anxiety across three different typesof tests? Which of three language training programs is most effective in teaching foreign

languages to mission volunteers? What is the difference between Counseling ApproachI and Counseling Approach II in reducing marital conflict?In each of these questions we find a researcher introducing a treatment (type of

test, training program, counseling approach) and measuring an effect . ExperimentalResearch is the only type which can establish cause-and-effect relationshipsbetween independentand dependent variables. See Chapter 13 for examples of experimental designs.

An Example

The goal of experimental research is to establish cause-effect relationships betweenindependent and dependent variables. Dr. Daryl Eldridge analyzed the effect of know-ledge of course objectives on student achievement in and attitude toward the course. 12

He found knowledge of instructional objectives produced significantly higher scores onthe Unit I exam (mid-range cognitive outcomes) but not on the Unit III exam (knowl-edge outcomes). Knowledge of objectives did produce significantly higher scores on thepostcourse attitude inventory. 13

Ex Post Facto ResearchEx Post Facto (which translates into English as “after the fact”) research is similar to

experimental research in that it answers the question, “what if?” But in ex post factodesigns, nature — not the researcher — manipulates the independent variable. Instudying the effects of brain damage on the attitudes of children toward God, it wouldbe immoral and unethical to randomly select two groups of children, brain damage oneof them, and then test for differences!

But in an ex post facto approach the researcher defines two populations: normalchildren and brain-damaged children. Nature has applied the treatment of brain dam-age. The experiment is done “after the fact” of the brain damaged condition. Experi-mental studies involving juvenile delinquency, AIDS, cancer, criminal or immoralbehavior and the like all require an Ex Post Facto approach.

An Example

The goal of ex post facto research is to establish cause-and-effect relationshipsbetween independent and dependent variables “after the fact” of manipulation. Anexample of Ex Post Facto research would be “An Analysis of the Difference in SocialSkills and Interpersonal Relationships Between Congenitally Deaf and Hearing CollegeStudents.” Congenital deafness in this case is the treatment already applied by nature.

EvaluationEvaluation is the systematic appraisal of a program or product to determine if it is

accomplishing what it proposes to do. It is the application of the scientific method to the

12Daryl Roger Eldridge, “The Effect of Student Knowledge of Behavioral Objectives on Achievementand Attitude Toward the Course,” (Ed.D. diss., Southwestern Baptist Theological Seminary, 1985)

13Ibid., 2


25/318


26/318


27/318

1-13



His words reflect Jesus' teaching that He gives understanding to those who follow Him(Mt. 11:29; 16:24).

Blaise Pascal wrote in the 17th century, “The heart has reasons which are unknownto reason.... It is the heart which is aware of God and not reason. That is what faith is:God perceived intuitively by the heart, not by reason.” The truth of Christ comes byliving it out, by risking our lives on Him, by doing the Word. We grow in our know-

ledge of God through personal experience as we follow Him and work with Him. Webelieve in order to understand spiritual realities. This approach to knowing is privateand subjective . Such belief-knowing resents an anti-supernatural skepticism of open-minded inquiry. More than that, some scientists consider the scientific method to be their religion. Their “belief in evolution” may be a justification for their unbelief in God.Science is helpful in learning about our world, but it makes a poor religion. So thefaithful view science and its adherents with suspicion.

Sometimes, however, the suspicion of science by the religious has less to do withfaith than it does political power. In the Middle Ages, the accepted view of the universewas geocentric (“earth-center”). The moon, the planets, the sun (located between be-tween Venus and Mars) and the stars were believed to rotate about the earth in perfectcircles. This view had three foundations: science, philosophy and the Church.

Greek science ( Ptolemy) and Greek philosophy (Aristotle) supported a geocentricview of the universe. The logic was rock solid for centuries: Man is the pinnacle ofcreation. Therefore, the earth must be the center of the universe.

The Roman Catholic Church taught that the geocentric view was Scriptural, basedon Joshua 10:12-13.

“Joshua said to the LORD in the presence of Israel: 'O sun, stand still overGibeon, O moon, over the Valley of Aijalon.' So the sun stood still, and themoon stopped , till the nation avenged itself on its enemies, as it is written inthe Book of Jashar. The sun stopped in the middle of the sky and delayedgoing down about a full day.”

For the sun and moon to stand still, the Church fathers reasoned, they would haveto be circling the earth.

Then several scientists began their skeptical work of actually observing the move-ments of the planets and stars. Copernicus, a Polish astronomer, created a 15th centuryrevolution in astronomy when he published his heliocentric (“sun-center”) theory of thesolar system. He theorized, on the basis of his observations and calculations, that theearth and its sister planets revolved around the sun in perfect (Aristotelian) circles.Keplar later demonstrated that the solar system was indeed heliocentric, but that theplanets, including earth, orbited the sun in elliptical, not circular, paths. The RomanCatholic Church attacked their views because they displaced earth from its position ofprivilege, and opened the door to doubt in other areas. But Poland is a long way fromRome (it was especially so in the 15th century!), and so Copernicus and Keplar re-

mained outside the Church's reach.Galileo is the father of modern Physics and did his work in Italy in the 16th and17th centuries. He studied the work of Copernicus and Keplar, and built a telescope inorder to more closely observe the planets. In 1632, he published the book DialogueConcerning the Two Chief World Systems: Ptolemaic and Copernican, in which he supporteda heliocentric view of the solar system. He was immediately attacked by Church au-thorities who continued to espouse a geocentric world view. Professors at the Univer-sity of Florence refused to look through Galileo's telescope: they did not believe histheory, so they refused to observe. Very unscientific! Galileo, under threat of beingburned at the stake, recanted his findings. It was not until October 1992 that the Roman


28/318

1- 14



Catholic Church officially overturned the decision against Galileo's book and agreedthat he had indeed been right. Science questions, observes, and seeks to learn how theworld works. Sometimes this process collides with the vested interests of dogmaticreligious leaders.

Suspicion of Religion By the ScientificScience is meticulous in its focus on the rational structure of the universe. Scientists

look with suspicion at the simple faith of believers who glibly say “I don't know how,but I just know God did it.” Such a statement reflects mental laziness. How does theworld work? What can we learn of the processes?

There Need Be No ConflictMany of the European men and women who pioneered science were motivated by

the Reformation and their new found faith to discover all they could about God'screation. Stephen Hales, English founder of the science of plant physiology, wrote(1727),

“Since we are assured that the all-wise Creator has observed the most exactproportions of number, weight and measure in the make of all things, the mostlikely way therefore to get any insight into the nature of those parts of theCreation which come within our observation must in all reason be to number,weigh and measure.” 20

Hales’ commitment to scientific methodology in no way compromised his faith inthe “all-wise Creator.” Nor did his faith undermine his scientific precision.

Still, the skeptical neutrality of science often collides with the perspective offaith, acceptance and obedience. When I was in the sixth grade, our science class begana unit on the water cycle. I had always believed that “God sent the rain to water theflowers and trees,” because that's what mom told me ( authoritative knowing) when Iasked her why it rained.

Now, before my very eyes was a chart showing a mountain and a river, and anocean and a cloud. Carefully the teacher explained the diagram. “Water vapor evapo-rates from the ocean and forms a cloud. The wind blows the cloud to the mountain,where water condenses in the form of rain. The rain collects and forms a river whichflows back into the ocean. This is the water cycle.” I can vividly remember my confu-sion and fear — where was God in the water cycle? My dad helped when he got homethat night. “Well, the water cycle certainly explains the mechanical process of evapora-tion and condensation, but Who do you think designed the water cycle?” My confusionwas gone. My faith was strengthened — though less simplistic and naive than it hadbeen before (“If God sends rain to water the plants, why doesn't He send some to theareas of drought, where people are starving to death?”). And, I had learned somethingabout how the world works that I hadn't even thought about before. The faithful shouldnot use “faith” as a cop out for mental laziness.

And so, faith focuses on the supernatural and subjectively sees with the heart’s eyethat which is unseen by the natural eye. Scripture, the Objective Anchor of our subjec-tive experiences, is a record of personal experiences with God through the ages. Faithfocuses on the Creator.

Science focuses on the natural and objectively gathers data on repeatable phenom-

20“Stephen Hales,” The Columbia Dictionary of Quotations is licensed by Microsoft Bookshelf from Colum-bia University Press. Copyright © 1993 by Columbia University Press. All rights reserved.


29/318

1-15



ena, the machinery, so we may better understand how the world works. Science focuseson the creation.

There need be no conflict between giving your heart to the Lord and giving yourmind to the logical pursuit of natural truth.

Summary

In this chapter we looked at six ways of knowing. We discussed specificallyhow scientific knowing differs from the other five. We introduced you to thescientific method, as well as seven types of research. Finally, we made a briefcomparison of faith-knowing and science-knowing.

Vocabulary

authority knowledge based on expert testimonycommon sense cultural or familial knowledge, localcontrol of bias maintaining neutrality in gaining knowledgecorrelational research analyzing relationships among variablesdeductive reasoning from principle (general) to particulars (specifics)descriptive research analyzing specified variables in select populationsempiricism basing knowledge on observationsevaluation analyzing existing programs according to set criteriaex post facto research analyzing effects of independent variables “after the fact”experience knowledge gained by trial and error experimental research determining cause and effect relationships between treatment and outcomeexternal criticism determining the authenticity of a document or relichistorical research analyzing variables and trends from the pastinductive reasoning from particulars (specific) to principles (general)internal criticism determining the meaning of a document or relicintuition/revelation knowledge discovered from within

precision striving for accurate measurementprimary sources materials written by researchers themselves (e.g. journal articles)research and development creating new materials according to set criteriascientific method objective procedure for gaining knowledge about the worldsecondary sources materials written by analysts of research (e.g. books about)theory construction converting research data into usable principlesverification replicating (re-doing) studies under varying conditions to test findings

Study Questions

1. Define in your own words six ways we gain knowledge. Give an original example of each.

2. Define “science” as a way of knowing.

3. Compare and contrast “faith” and “science” as ways of knowing for the Christian.

4. Define in your own words five characteristics of the scientific method.

5. Define in your own words eight types of research.


30/318


31/318


32/318


33/318

2-1

Proposal OrganizationChapter 2


22222Proposal Organization

Front Matter The Introduction

The MethodThe Analysis

Reference Material

The research proposal is a concise, clearly organized plan of attack for analyzingformal research problems. The beginning point indeveloping a proposal — itself not a part of thefinal product — is the “felt difficulty.” Hope-fully, as you have read textbooks and journalarticles, as you have listened to lectures andparticipated in discussion, you have beenattracted to specific issues and concerns in yourfield. Perhaps there have been questions thatremain unanswered, problems which remainunsolved, or conflicts which remain unresolved.

These issues, your felt difficulties, hold the begin-ning point for your research proposal.

The first step toward an objective study of your felt diffi-culty is the choice of a topic. Consider a topic which has the potential to make a contri-bution to theory or practice in your chosen field. Afterall, a dissertation will consumelarge quantities of your time, your money, and your very self. Worthwhile topics canbe discovered by browsing the indexes of information databases such as the Educa-tional Resources Information Center (E.R.I.C.) or Psychological Abstracts (For detailedsuggestions, see Chapter 6, “Synthesis of Related Literature”). This search, whetherdone manually or by computer, can provide useful information for confirming orabandoning a research topic.

Once a topic has been determined, it must be translated, step by step, into a clearstatement of a solvable problem and a systematic procedure for collecting and analyz-ing data. We begin that translation process in this chapter by providing a structuralblueprint, as well as definitions of each proposal element, for the proposal you willeventually develop.

The following structural overview gives you a framework for organizing your ownproposal. Each element listed in the structural overview is defined. Study these elementuntil you can see the structure of the whole.


34/318

2-2



Front Matter

Title Page

The coversheet for the proposalcontains basic information for the reader.You will list on this page your schoolname, the proposal title, your majordepartment, your name and the date theproposal is submitted. The title of yourproposal should provide sufficient infor-mation to permit your readers to make anintelligent judgment about the topic andtype of study you’re proposing to do.Your doctoral dissertation will be cata-loged in Dissertation Abstracts upongraduation, so a clear title will attractmore readers to your work.

Table of ContentsThe Table of Contents lists the major

headings and subheadings and theirrespective page numbers within theproposal. Suggestion: organize yourproposal (and simplify the writing of theTable of Contents) using a three-ringbinder with dividers for each section andelement of the proposal. As you work oneach section, file your materials in properorder in the binder.

List of Tables

As you write your dissertation, youwill want to augment your written expla-nations with visual representations of thedata. One form of presentation is the“table,” which displays the data tabularform — rows and columns of figures —which enhances, clarifies, and reinforcesthe verbal narrative. The List of Tableslists each table by name and page number.

Let me suggest that you considercarefully the tables you will need to use todisplay your data and include a sample ofeach planned table in your proposal.Doing this shows that you have givenadequate consideration to the forms yourdata will take.

Title Page

Contents

Tables

Illistrations

Proposal Overview

Front Matter

Title PageTable of ContentsList of TablesList of Illustrations

INTRODUCTION

Introductory StatementStatement of the ProblemPurpose of the StudySynthesis of Related LiteratureSignificance of the StudyStatement of the Hypothesis

METHOD

PopulationSampling

InstrumentLimitationsAssumptionsDefinitionsDesignProcedure for Collecting Data

ANALYSIS

Procedure for Analyzing DataTesting the HypothesesReporting the Data

Reference MaterialsAppendicesBibliography


35/318

2-3



List of Illustrations

An illustration is a graph, chart, or picture that enhances visually the meaning ofwhat you write. The List of Illustrations lists each illustration by caption and pagenumber.

Introduction

The introduction section includes the introductory statement, the statement of theproblem, the purpose of the study, the synthesis of related literature, the significance ofthe study, and the hypothesis. The purpose of the introduction is to demonstrate thethoroughness of your preparation for doing the study. This section explains to others,like the Advanced Studies Committee for instance, why you want to do this study. Itfurther demonstrates how well you understand your specific field.

The Introductory StatementThe proposal begins with an introductory statement, usually several pages in

length, which leads like a funnel from a broad view of your topic to the specific State-ment of the Problem. It provides readers of the proposal your rationale, based onpublished sources, for doing the study.

For example, if I wanted to study priority research needs in religious education inSouthern Baptist churches, I might organize my introductory statement in nine para-graphs as follows:

Teaching in Jesus’ ministryTeaching in the early church

The Sunday School movement of the past centurySeminaries and Religious Education

Southwestern Baptist Theological SeminaryThe School of Religious Education

Doctoral degrees in the School of Religious EducationSources of problems for dissertation research

The need to establish research priorities in a given field

It is not necessary to begin with the Bible as I have done in my example. A study ofcognitive counseling theories might begin with Gestalt psychology in the 1920’s. Behav-ioral approaches to therapy might begin with B. F. Skinner in the 1950's. The point is tobegin with a broad view of the field you’re studying, and then narrow the focus to thepoint of the Problem Statement. Notice that my sample introductory statement outlinebegins with a broad overview of the field of “the teaching of Jesus” and ends with thespecific point of “research needs in religious education.”

Use objective language in writing the introductory statement. Document everystatement. Do not include the personal feelings, experiences, or opinions which in-spired your proposal. It simply isn't appropriate to say “I had a bad experience withXYZ one time and wonder what might happen if...”.

The Statement of the ProblemThe Problem Statement, usually no more than a single sentence, is the most impor-

tant part of the whole proposal. It identifies the variables you plan to study as well asthe type of study you intend to do. All other parts of the proposal grow out of the

Introductory

Statement

Problem

Purpose

Synthesis

Significance

Hypothesis


36/318

2-4



Problem Statement. Just as an instructional objective provides the framework for lessonplanning, so the Problem refects the very heart of the study. For example, look at thefollowing Problem Statements from the dissertations of Drs. Marcia McQuitty andNorma Hedin:

The problem of this study [will be] to determine the relationship between the dominant manage-ment style and selected variables of full-time ministers of preschool and childhood educa-tion in Southern Baptist churches in Texas. The selected variables [are] level of education,years of service on church staffs, task preference, gender, and age. 1

The problem of this study [will be] to determine the differences in measured self-concept ofchildren in selected Texas churches across three variables: school type (home school, Chris-tian school, and public school), grade (fourth, fifth, and sixth), and gender. 2

See Chapter Four for more information on writing a Problem Statement.

Purpose of the Study

The Purpose of the Study section expands the Problem statement and describes inmore detail the intention of the study. Use verbs like “to determine,” “to ascertain,” “toevaluate,” “to discover.” A listing of purposes for Dr. McQuitty's Problem Statementabove reads this way:

“The purposes of this study [will be] to determine:

1. the dominant management style of full-time preschool and children's ministers in SouthernBaptist churches in Texas

2. the relationship between the dominant management style and selected variables of level ofeducation, years of service on church staffs, task preference, gender, and age

3. areas of strengths and weaknesses in management style which could be addressed by addi-tional printed material, professional development seminars, and the addition or restructur-ing of seminary class content for preschool and children's ministers.” 3

Notice that the list of Purpose statements comes directly out of the Problem State-ment, and yet expands each component of it.

Synthesis of Related LiteraturePart of the proposal-writing process involves library research. Preliminary sources

such as literature indexes (“Dissertation Abstracts”) and key word thesauri (the“E.R.I.C. Thesaurus”) provide a doorway into millions of research articles. Use theseresources to locate recent journal reports and dissertations related to your subject.Analyze these sources and condense the information into a clearly organized narrative.The purpose of the literature search is to establish a solid foundation for your study aswell as prepare you to conduct the study. The Synthesis provides a backdrop for your

1Marcia G. McQuitty, “A Study of the Relationship Between Dominant Management Style andSelected Variables of Preschool and Children's Ministers in Texas Southern Baptist Churches,” (Ed.D. diss.,Southwestern Baptist Theological Seminary, 1992), 5. Tenses changed from dissertation past tense toproposal future.

2Norma Sanders Hedin, “A Study of the Self-Concept of Older Children in Selected Texas ChurchesWho Attend Home Schools as Compared to Older Children Who Attend Christian Schools and PublicSchools,” (Ed.D. diss., Southwestern Baptist Theological Seminary, 1990), 6. Tenses changed from disserta-tion past tense to proposal future.

3McQuitty, 5-6


37/318


38/318

2-6



What tangible contribution will it make? In short, it answers the so-what question.“You want to study something. You find what you expect. So what?!” The personalinterest of the student or his/her major professor is not sufficient rationale for approv-ing a proposal. The best rationale is a reference to one or more research studies statingthe need for what you propose to do.

Dr. Dean Paret wrote an effective statement of significance for his study on healthyfamily functioning: 5

This study [will be] significant in that:1. It provides empirical data for the relationship between family of origin in terms of autonomy

and intimacy roles that were adapted and the current family healthy functioning patterns.Empirical validation has been called for by Hoverstadt et al. 118 to support the theoreticalassumptions upon which family therapy techniques are based.

2. It provides empirical data for breaking the recurrent cycle perpetuating the adult childsyndrome. 119

3. It provides a basis for the development of specific parenting training for the ministry of thechurch.

4. It provides helpful information for the seminary to aide [sic] the students who are having adifficult time juggling married life and student life, by providing indicators of stress areasrelated to autonomy and intimacy. According to Dr. David McQuitty, Director of StudentAid, the seminary through his office sees an increase in problems encountered by studentsas their seminary journey increases, both in financial stress, and student stresses, that couldpossibly be related to issues brought forward from the family of origin. 120 It is thereforenecessary to provide empirical data to help in breaking down the dysfunctional patterns ofinteraction.

118Hoverstadt, et al., 287 and 296 119Fine and Jennings, 14120Conversation with Dr. McQuitty on August 18, 1990

Just before my Proposal Defense, I made one last trip to the North Texas Sciencelibrary. On that trip, I found a reference to a speech made two years earlier. Looking upthe speech, I found a gold mine! The writers had analyzed many of the procedures Iwas studying. Their conclusion was to call for a computer analysis of several of themost popular procedures. It was the focus of my study! I added this recommendationto my “significance” section. It provided a solid rationale for my study when I defendedit before my Proposal Committee.

The HypothesisThe Statement of the Problem describes the heart of your study in one or two

succinct sentences. The Statement of the (research) Hypothesis describes the expectedoutcome of your study. Base the thrust of your hypothesis on the synthesis of literature.Use the Problem Statement as the basis for the format of the hypothesis. Look at thisProblem-Hypothesis pair from the dissertation of Dr. Joan Havens :

“The problem of this study [is] to determine the difference in level of academic achievementacross four populations of Christian home schooled children in Texas: those whose parentspossessed (1) teacher certification, (2) a college degree, but no certification, (3) two or moreyears of college, or (4) a high school diploma or less.” 6

5Dean Kevin Paret, “A Study of the Perceived Family of Origin Health as It Relates to the CurrentNuclear Family in Selected Married Couples,” (Ed.D. diss., Southwestern Baptist Theological Seminary,1991), 36-37


39/318


40/318

2-8



The population of this study [will consist] of all hypotheses from Ed.D. and Ph.D. disserta-tions completed within the School of Religious Education at Southwestern Baptist TheologicalSeminary which met four criteria:

1. The hypothesis was included within a dissertation completed between May 1978 andMay 1996.

2. The hypothesis tested differences between groups as opposed to relationhips betweenvariables.3. The hypothesis was tested statistically by means of t-Test for Difference Between

Means, One-way ANOVA, Two Factor ANOVA, or Three factor ANOVA.4. Statistical significance was determined solely upon meeting a singular criteria, that

being a single statistical test. 12

See Chapter Seven for more information.

Sampling

The Sampling section describes how you will draw one or more samples from thepopulation or populations defined above. It also explains how many subjects youintend to study in these samples. Here are examples of sampling statements based onthe populations we defined above.

A twenty-five percent random sample [will be] obtained from the mailing list of the NationalAssociation of Social Workers in the State of Texas. The sample [is] estimated to consist of144 subjects. 13

A simple random sample of hypotheses [will be] conducted to produce two equal groups of fiftyhypotheses: hypotheses proven statistically significant (Group X) and hypotheses notproven significant (Group Y). . . .14

See Chapter Seven for more information.

Instrument

The Instrument section describes the tools you plan to use in measuring subjects.“Instruments” includes tests, scales, questionnaires and interview guides, observationchecklists, and the like). If you choose an existing instrument appropriate for yourstudy, then describe its development, use, reliability and validity. If you cannot find asuitable instrument, you will need to develop your own. Provide a step by step expla-nation of the procedure you will use to develop, evaluate, and validate the instrument.

Here is a portion of Dr. Hedin 's “instrument” section:

The instrument selected for this study [is] the Piers-Harris Children's Self-Concept Scale

(The Way I Feel About Myself), developed by Ellen V. Piers and Dale B. Harris in 1969. . .An-swers are keyed to high self-concept; thus, a higher total score [indicates] a positive concept ofself. . .Reliability coefficients ranging from .88 to .93, based on Kuder-Richardson and Spearman-Brown formulas, were reported for various samples 29 . . . Content validity was built into thescale by using children's statements about themselves as the universe to be measured as self-concept. By writing items pertaining to that universe of statements, the authors defined self-concept for their scale 31 . . . An attempt was made to establish construct validity during theinitial standardization study. The PHCSCS scale was administered to eighty-eight adolescentinstitutionalized retarded females. As predicted by Piers and Harris, these girls scored signifi-

12Clark, 30-31 13Da Silva, 7 14Clark, 31


41/318

2-9



cantly lower than normals of the same chronological or mental age. This was interpreted asmeaning that the PHCSCS did measure self-concept and discriminated between high and lowself-concept. 32

Dr. Wes Black developed his own instrument:

No standardized instrument was found to be applicable to this study. It is therefore neces-sary to devise such an instrument . . .thirteen experts received the questionnaire for theirevaluation. The learning objectives from the “Youth Discipleship Taxonomy” were arranged inrandom order under each of the five areas of Church Training task assignment. . .The expertswere asked to select ten items most appropriate for inclusion in a questionnaire on learningobjectives for youth discipleship training from each of the five task areas and rank order theirchoices from one (highest) to ten (lowest) in each area.

Responses from the experts were checked for completeness and correctness. The rankingswere reversed scored (a ranking of one received ten points; ranking of two received nine points;and so forth) and scores totalled for each item on the taxonomy. Ten items in each of the fiveareas resulted in clear choices of the experts to be included in the instrument for this study.

Table 1 [will provide] a summary of the experts. Appendix B lists the experts. The results ofthe content validity study [will be located]. . . in appendix C. 15

See Chapters Nine, Ten and Eleven for more information on developing instru-ments..

LimitationsThe Limitations section describes external restrictions that reduce your ability to

generalize of your findings. An external restriction is one that is beyond your control.Let's say you plan to randomly assign students in a local high school to one of threeexperimental teaching groups. When you check with the principal, he allows you to dothe experiment, but only if you use the regular classes of students — he does not wantyou disrupting classes through random assignment. Since random assignment is an

important part of experimental design, this is a limitation to your study and must bestated in this section.

Limitations differ from delimitations. Delimitations are restrictions you set on yourstudy. The fact that you decide to study single adults ages 20-50 is a delimitation of yourstudy, not a limitation. Choosing to study only 6 of the 16 scales of the 16PF Test is adelimitation, because you make that decision on your own. Limitations are externalrestrictions and belong in this section. Delimitations are personal restrictions andbelong in the “Procedures for Collecting Data” section of the proposal -- there is no“Delimitations” section.

One of Dr. Matt Crain 's limitations was:Due to the lack of a central organizational headquarters, no directory of Churches of Christ

exists whereby a true random sample of all congregations may be obtained. 16

Here's one from the dissertation of Dr. Charles Bass :

15Wesley Black, “A Comparison of Responses to Learning Objectives for Youth Discipleship Trainingfrom Minister of Youth in Southern Baptist Churches and Students Enrolled in Youth Education Coursesat Southwestern Baptist Theological Seminary,” (Ed.D. diss., Southwestern Baptist Theological Seminary,1985), 30-31

16Matthew Kent Crain, “Transfer of Training and Self-Directed Learning in Adult Sunday SchoolClasses in Six Churches of Christ,” (Ed.D. diss., Southwestern Baptist Theological Seminary, 1987), 8


42/318

2-10



This study [will be] subject to the limitations recognized in collecting data by mail, such asdifficulty in assessing respondent motivation, inability to control the number of responses,and bias of sample if a 100 percent response is not secured. 17

Assumptions

Every study is built on assumptions. The purpose of this section is to insure thatthe researcher has considered his assumptions in doing the study. In doing a mailedquestionnaire, the researcher must assume that the subjects will complete the question-naire honestly. In testing which of two counseling approaches is best, one assumes thatthe approaches are appropriate for the subjects involved.

Provide a rationale for the assumptions you state. It is not enough to copy assump-tions out of previous dissertations. Explain the why of your assumptions.

Here are several assumptions made by Dr. Darlene Perez :

1. All [112 Puerto Rican Southern and American Baptist] churches will have a youthSunday School enrollment.

2. The pastors and youth leaders will cooperate with the study and will insurecompletion of the questionnaires.

3. Since [all] 112 Southern Baptist and American Baptist churches were used in thestudy, it is assumed that the findings are important in that they represent thegeneral opinion of Baptist youth groups in Puerto Rico. . . . 18

Here are several assumptions made by Dr. Gail Linam :

2. The in-depth training provided to researchers who administrated and/or scored theIowa Tests of Basic Skills, the cloze reading comprehension test, and the retelling comprehension analysis insured consistency in test administration andobjectivity in scoring.

3. The Iowa Tests of Basic Skills, as a norm-based test, provided an accurate assessmentof the reading level of boys and girls in Arlington, Texas, and thus offers ameaningful base of reference for religious educators around the nation who seekto make application of the study's findings to their particular group of boys andgirls. 19

Definitions

If you are using words in your study that are operationally defined -- that is,defined by how they are measured -- or have an unusual or restricted meaning in yourstudy, you must define them for the reader. You do not need to define obvious orcommonly used terms. For example, Dr. Kaywin LaNoue studied differences in “spiri-tual maturity” in high school seniors across two variables: active versus non-active inSunday School, and Christian school versus public school. But what did she mean by“active” in Sunday School? What is “spiritual maturity” and how did she measure it?Here are her definitions for these two terms:

17Charles S. Bass, “A Study to Determine the Difference in Professional Competencies of Ministers ofEducation as Ranked by Southern Baptist Pastors and Ministers of Education,” (Ph.D. diss., SouthwesternBaptist Theological Seminary, 1998), 45

18Darlene J. Perez, “A Correlational Study of Baptist Youth Groups in Puerto Rico and Youth Curricu-lum Variables,” (Ed.D. diss., Southwestern Baptist Theological Seminary, 1991), 12

19 Gail Linam, “A Study of the Reading Comprehension of Older Children Using Selected BibleTranslations,” (Ed.D. diss., Southwestern Baptist Theological Seminary, 1993), 85


43/318

2-11



Active.Active means those students attending their Sunday School at least three Sundaysa month. 2

Spiritual maturity.Peter gives the steps in a Christian's growth toward maturity when he lists the at-tributes of the Christian life in the order by which they should be sought. He does this in 2

Peter 1:5-8. . . . In this study, spiritual maturity [is] the extent to which the students haveassimilated (internalized) the virtues of goodness, knowledge, self-control, perseverance,godliness, brotherly kindness, and love. 21

Dr. LaNoue used an adaptation of the Spiritual Maturity Test, developed andpublished by Dr. James Mahoney, to convert the virtues listed above into a test score. 22

Sometimes special terms are used to communicate complex concepts quickly. Theseterms need to be defined. For example, the term "k, J combination" makes no sense untilit is clearly defined:

k,J combination. -- This term refers to two major variables in this study: the number of groups inan experiment, k, and the sample size category, J. There [will be] four levels of k represent-ing three, four, five, and six groups. There [will be] seven levels of J. J(1) through J(5) [willrepresent] equal n sample sizes of 5, 10, 15, 20, and 25 respectively. J(6) [will represent] anunequal set of n j's in the ratio of 1:2:3:4:5:6 with n 1= 10. That is, when k=3, the sample n's[will be] 10, 20, and 30. J(7) [will represent] a set of n j's in the ratio of 4:1:1:1:1:1 with n 1=80.That is, when k=3, the sample n's [will be] 80, 20, and 20. This provides twenty-eight combi-nations of k,J. 23

See Chapter Three for more information on operationalizing variables.

Design

The Design section describes the research type of your study. It is here you declareyour research to be correlational, or historical, or experimental. See the overview of

Research Types in Chapter One for a description of eight major design types. Describekey factors that make your study of the stated type. If you are using an experimentaldesign, explain which you are using and why. Dr. Brad Waggoner explained his designthis way: 24

The method of research [which will be] employed in this study [is] “Research and Develop-ment”. . . This type of research [is] accomplished in two phases. The first phase [will in-volve] the development of the product. The second phase [will consist] of evaluating the useor effects of the product. xx Although the exact number of specific stages of Research andDevelopment vary from author to author, the following five steps [will be] applied: xy

1. The identification of a need, interest, or problem2. The gathering of information and resources concerning the problem or need

3. The preliminary product or process [is] developed4. The product or process [is] field-tested

20Kaywin Baldwin LaNoue, “A Comparative Study of the Spiritual Maturity Levels of the ChristianSchool Senior and the Public School Senior in Texas Southern Baptist Churches With a Christian School,”(Ed.D. diss., Southwestern Baptist Theological Seminary, 1987), 25

21Ibid., 26 22Ibid., 93-9723William R. Yount, “A Monte Carlo Analysis of Experimentwise and Comparisonwise Type I Error

Rate of Six Specified Multiple Comparison Procedures When Applied to Small k's and Equal and UnequalSample Sizes,” (Ph.D. diss., University of North Texas, 1985), 8

24 Waggoner, 7-8


44/318

2-12



5. The product or process [is] refined based on the information obtained from the field-testing.

xxC. M. Charles, Introduction to Educational Research (New York: Longman, 1988), 13 xyIbid., 13-14

Dr. Martha Bergen described the design of her study this way: 25

The design of this study [is] descriptive in nature. [A] questionnaire [will be] designed todetermine the attitudes of Southwestern Seminary's full-time faculty toward computers forseminary education. Further, certain variables [will be] examined to determine their possiblepredictions of these attitudes.

See Chapter Thirteen for more information on experimental designs.

Procedure for Collecting Data

The Procedure for Collecting Data section explains step by step how you plan toprepare instruments and gather data. Anticipate problems you may encounter andmake contingency plans as needed.

Avoid fuzzy over-generalized statements such as, “Prepare and mail out surveyforms.” This phrase requires many specific actions: development, evaluation, roughdraft, pilot testing, revision, final draft, printing, packaging, and mailing. Consultrelated dissertations and primary sources to discover the best procedures to use whencollecting the particular type of data you need.

At the end of this section, you should picture yourself with data sheets filled withnumbers linked to each subject and every variable in the study. If the METHODSsection is properly planned and executed, the result will be valid and reliable data readyfor analysis.

See Chapters Nine, Ten, Eleven, and Twelve for more information on collectingdata.

Analysis

The third and final major section of the proposal is the analysis section. TheANALYSIS section describes how you plan to process the numbers on the data sheets.This section moves step by step through the application of selected statistical proce-dures, the testing of hypotheses, and the reporting of the data in a systematic, coherentway.

Procedure for Analyzing DataThe Procedure for Analyzing Data explains step by step how you plan to statisti-

cally analyze your data. What statistical procedure(s) will you use? Procedures mustagree with the stated Problem and Hypothesis.I was impressed by the importance of this section during my very first semester as

the faculty member responsible for research and statistics. A doctoral student came intomy office with a box full of inventory sheets. He had spent nearly $1,000 on printingand postage. He sat down, looked painfully at the box and asked,“Now, what do I do

25Martha S. Bergen, “A Study of the Relationship Between Attitudes Concerning Computer-EnhancedLearning and Selected Individual and Institutional Variables of Full-Time Faculty Members at Southwest-ern Baptist Theological Seminary,” (Ed.D. diss., Southwestern Baptist Theological Seminary, 1989), 52

Analyzing Data

Testing Hypothesis

Reporting Data


45/318


46/318

2-14



4 - Cornell Inventory for Student Appraisal of Teaching and Courses [7]5 - Letter to Research Associates for Validation of Cognitive Tests [2]6 - Test Item Analysis - Unit 1 Exam [2]7 - Letter to Research Associates for Validation of Precourse Attitude Inventory [2]8 - Report Form For Student Test Scores [1]9 - Session Goals and Indicators [4]

10 - Unit 1 Exam, Final Form [8]11 - Unit 3 Exam, Final Form [5]12 - Cognitive PreTest, Final Form [4]13 - Postcourse Student Inventory [8]14 - Precourse Student Inventory [3]15 - Tentative Class Schedule [4]16 - Course Syllabus, Fall Semester [3]17 - Course Syllabus, Spring Semester [5]18 - Quizzes Over SBC Curriculum [6]19 - Letter to Cornell University [1]20 - Selected Comments From the Postcourse Inventory and Student Evaluations [3]21 - Raw Scores For All of the Instruments [4]22 - A Comparison of Scores Across Semesters for the Various Instruments [2]23 - Statistical Analysis for Each of the Instruments Across Semesters [5]

You provide a clear, categorized filing system for supportive information by pack-aging materials in appendices. Small parcels of this information can be drawn fromthese appendices for explanation and illustration in the body of the dissertation. Such adesign permits you to provide complete information, through references to the appen-dices, without bogging down the flow of thought in the dissertation itself. In the pro-posal development stage, think ahead concerning what appendices you will need andinclude an empty copy of each as an appendix to the proposal. This demonstrates to theCommittee forethought and critical thinking.

Bibliography, or Cited Sources

The bibliography lists all primary and secondary references footnoted in the bodyof your proposal. List books first, then published articles and periodicals, then disserta-tions, then unpublished sources, interviews and, finally, other. Format bibliographicalreferences according to the current style manual.

Practical Suggestions

Here are some practical suggestions to help you write a solid proposal.

Personal AnxietyThis assignment is complex. Some students experience a frightening sense of

anxiety as they consider the daunting task of writing a research proposal. A researchproposal taxes the thinking skills of the best students. You are confronted with learningnew definitions (knowledge), understanding new concepts (comprehension), discover-ing conceptual links among numerous articles (analysis), writing an integrative narra-tive (synthesis), choosing the correct design and statistical procedures (evaluation) andputting all of this together in a single-focused, comprehensive document. Your educa-tional experiences in high school and college may have emphasized rote memory,

Personal Anxiety

Professionalism


47/318

2-15



recall, and simple concepts rather than clear thinking. Therefore, writing an originalresearch proposal is “a strange new thing” for some. Many paths to choose. Manydecisions to make. What topic will I choose? What kind of research will I select? Wheredo I begin? For some, too many “neat ideas” compete for attention. For others, “neatideas” are nowhere to be found. Don't panic. Take each section, each step of the pro-cess, one at a time.

Professionalism in WritingA research proposal should be written in a clear, professional manner or it will not

be understood. Here are some suggestions.

Clear Thinking

Your proposal should show clear thinking. Write and revise. Squeeze out fuzzyphrases, word magic 28 and awkward grammar. Write simply and clearly. Use profes-sional jargon only when simple English can’t convey the thought.

Unified FlowThere should be a unified flow through the proposal. Take care not to ramble or

lose focus in the details. March step by step in a single direction from the first page tothe last.

Quality Library Research

The proposal should demonstrate extensive yet focused library research. Use primary sources less than five years old to establish current trends. Use secondarysources less than ten years old to establish the scope of your study. Use sources olderthan ten years only to establish historical trends.

Efficient DesignYour proposal should demonstrate your understanding of research design and

statistical analysis, and how they work together. The proposal should present a narra-tive that is all-of-one-piece rather than a disjointed collection of pieces. Problem, Hy- pothesis, and Statistic should form its backbone.

Accepted FormatFinally, write in the accepted professional format of your school. Content is more

important than format, but a professional format is required.

SummaryThis chapter lays out the complete skeletal organization, with examples from actual

dissertations, for the proposal you are developing. Study each component individually,as well as its relationship to the whole. Refer to this chapter and to the EvaluationGuidelines in Chapter 27 throughout the writing process to insure that you are oncourse. You will add to your understanding of each of these components as the semes-

28I use the term “word magic” to refer to high-sounding, emotive words that have little substantivemeaning. The majestic purpose of the American school is to instill in the hearts and minds of our youth therequisite essentials which will allow them to take their rightful place in society and fulfill their destiny. Huh? Wehear word magic in sermons and classrooms as well. It “gets the amens” but communicates little.


48/318

2-16



ter progresses. Use this overview to anchor “the big picture” in your mind.

Vocabulary Analysis describes step-by-step the analysis of collected data Appendix an addendum to a proposal which contains supporting examples

Assumptions stated presuppositions upon which a proposed study is basedBibliography a list of references used in d