literature reviews: evolution of a … the literature review methodology and discusses some of the...

33Australian Journal of Advanced Nursing 2000 Volume 18 Number 2

ABSTRACT

Literature reviews summarise all past research effortsand because of this are increasingly being used as away of addressing the rapid accumulation of healthcare information Because reviews rather thanprimary research are now being used as the basis formany health care decisions it is important that theyare conducted with the same rigour of the primaryresearch To ensure this rigour the past two decadeshave seen a progressive evolution in reviewmethodology to the point where reviews are nowconsidered research in their own right Despite thisdevelopment the standard of many published reviewsremains poor This paper summarises the developmentof the literature review methodology and discussessome of the implications for nursing

INTRODUCTION

L iterature reviews play an important role in theadvancement of a discipline because theyaccumulate past endeavours summarise major

issues and are an important way to disseminate theinformation generated by a large number of individualstudies There are many different types of literaturereviews Some present a simple discussion of a topicwhile others provide a comprehensive and systematicsummary of past research Regardless of their scopereviews differ from other research endeavours in that theyare based on a body of completed works rather than newresearch

As the aim of the review is to present the lsquostate of thesciencersquo on a specific health care topic it follows that therigour that is required of the original primary researchmust also be applied to the review of this researchliterature Without this rigour summarised information isat risk of the same errors that threaten the credibility ofprimary research Unfortunately the commonly usedapproaches for reviewing the research literature have oftenlacked scientific rigour and as a result have not been heldin high regard Literature reviewers have also had to copewith an ever-increasing volume of health care researchThe quality of this primary research varies considerablyand results are often contradictory This situation meansthat a literature review can be used to promote the opinionsof the reviewer in that only research that supports aparticular view point is included rather than representingan accurate and balanced summary of all the research onthe topic

To address this problem there has been a progressiveevolution in review methods This evolution started in the1960s and 1970s and is still occurring today Thisdevelopment has seen the status of reviews change to thepoint were some reviews are now seen as research in theirown right (Mulrow and Cook 1997) This papersummarises this evolution of the methods for reviewingthe health care literature and discusses some of theimplications for nursing

David Evans RN DipN BN MNS is Coordinator of Reviews atThe Joanna Briggs Institute For Evidence Based Nursing andMidwifery Adelaide South Australia

Inge Kowanko BSc(Hons) PhD is a Senior Research Fellowat Flinders University Adelaide South Australia

Accepted for publication March 2000

LITERATURE REVIEWS EVOLUTION OF A RESEARCH METHODOLOGY

SCHOLARLY PAPER

Key Words literature review systematic review meta-analysis

Australian Journal of Advanced Nursing 2000 Volume 18 Number 2

A concern for standards of reviews

Due to the accumulative nature of scientific knowledgetrustworthy accounts of past research are necessary(Cooper 1984) In the 1960s Price described the role of theliterature review as replacing those papers lost from sightby the research front (Price 1965) Primary researchinformation is digested sifted classified and synthesised(Manten 1973) Because reviews deliberately aim atcumulating knowledge in a specific area they play a majorrole in the progress of a discipline in that they bringtogether previous work identifying past achievements andpossible future directions (Feldman 1971) Without thesereviews of the literature it is likely that a large amount ofthe research would be lost to the profession as the researchfront continues to move to new ground or rediscovers pastareas of interest on which to start fresh investigations

Literature reviews have also been likened to primaryresearch and as such seek to make accurate generalisationsabout phenomena from limited information (Jackson 1980)Cooper and Rosenthal (1980) suggested that literaturereviews also have great information-gatekeeping potentialbecause knowledge is communicated to undergraduatesand the lay public through reviews rather than throughthe primary research reports However a criticism of thereviews of the 1970s was that they made no attempt atrigorous definitions or standardisation of the techniquesused and therefore the findings could be biased byfactors that would be unforgivable in primary research(Glass et al 1981)

Prior to the 1980s there were no explicit methods forconducting reviews and so reviews varied widely in bothquality and scope which made assessment of their qualitydifficult (Jackson 1980) Light and Smith (1971) used afour category typology to characterise the approach takenby reviewers of this period by which they

listed any factor that had shown an effect on adependant variable

excluded all studies except those that supported aparticular point of view

averaged statistics across relevant studies in one formor another or

counted studies with similar results

Even in the late 1980s many scholarly publications hadno specific or formal definition of a literature review(Cooper 1988) As the volume of research continued togrow the traditional literature review which cited a smallnumber of studies no longer did justice to the literatureand prompted the term lsquothe misinformation explosionrsquo(Glass 1976) The volume of literature has continued togrow dramatically with the annual publication of 20000 to30000 biomedical journals and 17000 biomedical texts(Booth 1996) This makes presentation of an accuratesummary of the research a daunting challenge for not only

must the relevant studies be found in this mass ofpublications but issues such as the variable quality ofstudies and contradictory result must be addressed Boothdescribed the current dilemma facing health care workersas lsquoDrowning in information thirsting for evidencersquo(Booth 1996 p 25)

Types of reviews

One early attempt to describe the potential foci ofliterature reviews identified three different componentsand suggested that a comprehensive review will likelyaddress two or more of these areas (Cooper 1984) Thesecomponents were

integrative reviews - summarise past primary researchdraw overall conclusions highlight unresolved issuesand provide direction for future research

theoretical reviews - present theories to explainphenomena compare them in terms of breadth internalconsistency and the nature of their predictions and

methodological reviews - examine and critique theresearch methods and operational definitions that havebeen applied to a problem and address rigour and therisk of bias

In the late 1980s Cooper (1988) developed a taxonomywhich provided a framework for reviews that hadpreviously been lacking This taxonomy addressed sixcharacteristics of reviews

Focus primary research outcomes research methodstheories or the application of findings of research

Goals integration criticism or the identification of centralissues

Perspective neutral where the reviewer attempts topresent all arguments espousal were the perspective is aparticular argument or issue paying little attention to otherviews

Coverage may range from exhaustive to only arepresentative cover of the available research

Organisation how the literature is presented in thereview for example historically in chronological orderconceptually with literature relating to the same ideaspresented together or methodologically in which literatureis grouped according to the methods employed in theprimary research

Audience the differing audiences to whom the reviewer isdirecting the work

These developments provided a structure for reviewsand acknowledged the differing methods used byreviewers With this taxonomy came the recognition thatlike primary research the scope of literature reviews alsodiffered from a simple discussion to a comprehensivecoverage of the literature using methods with a rigoursimilar to that of the research it summarised

SCHOLARLY PAPER

34


Meta-analysis

The traditional approach to summarising the resultsfrom more than one study was by narrative discussionsuch as highlighting studies with similar or contradictoryfindings These narrative reviews of research have existedfor as long as there has been scientific literature (Petitti1994) However this approach has certain limitationsFirstly while a narrative discussion is possible with asmall number of primary research studies it is inadequatefor large numbers of studies addressing the same topicSecondly the lack of rigour of the approach makes anaccurate and credible summary difficult For example ifindividual primary research studies report contradictoryresults it is difficult to determine which results to includein the review Glass has suggested that the traditionalapproach used when inconsistent findings wereencountered was to eliminate all but a few studies thenadvance these as the truth (Glass 1976) He describednarrative summaries as lsquoairy speculation unbefitting anempirical sciencersquo (Glass 1976) Finally attempting tosummarise the results of many studies in a short narrativediscussion inevitably means that large amounts ofinformation are lost due to the difficulty of dealing withthis magnitude of information in a simple discussion Toovercome these difficulties statistical methods weredeveloped that were able to deal with large amounts ofnumerical data

Statistics have been used to combine the results frommultiple studies since the 1930s (Petitti 1994) Meta-analysis has been described as a quantitative approach forsystematically combining the results of previous researchin order to reach conclusions about the body of research(Petitti 1994) Glass suggested that the analysis of researchdata occurs at three different levels (Glass 1976) The firstlevel primary analysis is the original analysis of theresults of a single research study The next levelsecondary analysis is the re-analysis of the results of asingle research study to better answer the researchquestion or to answer new questions The third level ismeta-analysis which is a method for the integration andstatistical analysis of data from a number of separatestudies Meta-analysis provides a logical framework to aresearch review in that similar measures from comparablestudies are listed systematically and when possible themeasures of the effect of an intervention are statisticallycombined (Dickersin 1992) Meta-analysis has now beenused extensively in reviews as a way of dealing objectivelywith the large amounts of data generated by a group ofsimilar individual studies More recently in response tothe variability of results when there is more than onereview utilising meta-analysis for a topic the term lsquometameta-analysisrsquo has emerged (Katerndahl and Lawler1999) This term has been used to represent the combinedanalysis of two or more meta-analyses

Systematic reviews

While meta-analysis brought greater rigour to thesummary of research data many other aspects of literaturereviews were still conducted in a haphazard manner Theterm lsquoSystematic Reviewrsquo came into use in the early 1990sto emphasise the importance of an extensive systematicprocess of identifying appraising and summarising allresearch on the review topic These systematic reviewshave been described as scientific tools which are used tosummarise and communicate the results and implicationsof otherwise unmanageable quantities of research (NHSCentre for Reviews and Dissemination 1996) In contrastto the traditional narrative literature review systematicreviews use explicit rigorous methods to identify criticallyappraise and synthesise relevant studies (Mulrow 1987)When appropriate meta-analytic techniques are used tocombine statistical data from individual studies To datethe major research design summarised by systematicreviews is the randomised-controlled trial (RCT)

Like primary research systematic reviews have distinctstages which cover the review question the search forrelevant studies critical appraisal data extraction andanalysis (Mulrow and Oxman 1997 NHS Centre forReviews and Dissemination 1996)

Review question each review has one or more specificquestions which will likely encompass four componentsthe population of interest the intervention a comparisonor control and an outcome of interest (Counsell 1997NHS Centre for Reviews and Dissemination 1996)Inclusion criteria are developed from these whichoperationalises the review question putting it intopractical terms that enables relevant studies to be selectedfor inclusion in the review

Search the aim of the search is to locate as muchresearch on the topic as possible while minimising thenumber of relevant studies missed A comprehensivesystematic review includes both published andunpublished research from around the world The searchstrategy is documented in sufficient detail to allow othersto replicate the search and to critique its quality (Counsell1997 Jadad and McQuay 1993)

Critical appraisal this involves making judgementsabout the validity of certain aspects of primary researchstudies such as determining whether the methods used instudies can be trusted to provide a genuine accurateaccount of the phenomena being studied (Avis 1994) Theappraisal focuses primarily on bias as a result of selectionof participants differences in treatment of participantsdifferences due to losses of subjects from a study anddifferences in measuring the outcomes of interest (Mulrowand Oxman 1997)

Data extraction the data used by systematic reviewsare results produced by individual studies This data is

SCHOLARLY PAPER

35


extracted from the primary research reports using aspecially developed form to minimise the risk of errorwhile results are being transcribed

Data analysis the objective of the systematic review isto summarise the results from different studies and whenappropriate this will be achieved using meta-analysis It isonly appropriate to use meta-analysis when the studiesinvolved the same type of populations used the sameinterventions and measured the same outcomes

Each of these stages is documented beforecommencement in a systematic review protocol whichis the equivalent of a research proposal This protocolminimises the risk of bias as a result of subjectivedecisions made during the conduct of the review It alsoallows the review to be critically evaluated or replicated

Hierarchy of evidence

To address the fact that primary research has manypossible study designs with different risks of bias andcredibility of findings hierarchies of evidence have beendeveloped These scales rank the evidence according to itsrisk of bias and error The most commonly used scales arethose ranking research into the effectiveness ofinterventions and while there are a variety of these scalesone example is listed below (NHMRC 1995)

Level I Evidence obtained from a systematic review of allrelevant randomised controlled trials

Level II Evidence obtained from at least one properlydesigned randomised controlled trial

Level III 1 Evidence obtained from well designed controlledtrials without randomisation

Level III 2 Evidence obtained from well designed cohort orcase control analytic studies preferably from morethan one centre or research group

Level III 3 Evidence obtained from multiple time series with orwithout the intervention or dramatic results inuncontrolled experiments

Level IV Opinion of respected authorities based on clinicalexperience descriptive studies or reports of expertcommittees

Other scales have been developed and used for studiesaddressing other issues such as diagnosis prognosis andclinical practice guidelines (Meltzer et al 1998) Howevera limitation of the hierarchies of evidence for theeffectiveness of interventions and treatments is thatqualitative research is ranked at the level of expert opinionthe lowest level in the hierarchy This ranking ignores therigour of qualitative research which may be comparable toother research designs The major difference is thatqualitative studies seek answers to different types ofquestions This ranking also fails to recognise theimportant contribution qualitative studies can make

particularly by ensuring that the perspective of therecipient of the treatment is part of any evaluation of itseffectiveness and appropriateness

Assessing the quality of reviews

Based on the growing importance of literature reviewsthe increasing number that are published each year andthe fact that not all reviews are of equal quality criteriahave been developed to assess their scientific quality Avariety of assessment tools are available one goodexample developed by Oxman and Guyatt (1991) consistsof 10 questions

1 Were the search methods reported

2 Was the search comprehensive

3 Were the inclusion criteria reported

4 Was selection bias avoided

5 Were the validity criteria reported

6 Was validity assessed appropriately

7 Were the methods used to combine studies reported

8 Were the findings combined appropriately

9 Were the conclusions supported by the reported data

10 What was the overall scientific quality of the overview

Because clear and precise reporting of the methodsemployed is important for literature reviews questionsone three five and seven address reporting Without thisreporting of methods judgement of scientific quality ofreviews would be impossible Other questions address thescope of the search the process of selecting studies theircritical appraisal and whether findings were combinedappropriately Question nine lsquowere the conclusionssupported by the reported datarsquo highlights that likeprimary research the findings of a review must accuratelyreflect the studies summarised Finally the review musthave the rigour and the scientific quality that characterisesall reputable research endeavours and this is achievedthrough adherence to accepted review methods

Structured reviews

Another approach to summarising the literature is theconceptually broader structured review This type ofreview has a much broader scope than that of a systematicreview and may cover more than one intervention or evenan entire disease or health care topic It has been suggestedby Stevens and Milne (1998) that structured reviews areneeded because end users such as planners of health careoften require a reliable perspective on a topic which goeswider than that provided by a systematic review Thesereviews are often needed within a short time scale toaddress immediate problems precluding the lengthyprocess of the systematic review The need for structuredreviews also results from the rapid rate of diffusion of newhealth care technologies (Stevens and Milne 1998)

SCHOLARLY PAPER

36


Although many of these reviews follow the strict rulesof the systematic review they often have to be based onlimited literature and an element of modelling to provideuseful conclusions (Stevens and Milne 1998) Structuredreviews are appropriate for some nursing topics wherethere is limited research on which to baserecommendations Additionally many issues ofimportance to nursing are broader in nature than the singleintervention medication or surgical procedure commonlysummarised in systematic reviews However basingrecommendations on limited evidence brings with it agreater risk of bias and error Despite the potentialusefulness of structured reviews there is very littlepublished information about them and minimal attempts toensure the appropriateness of current methods

IMPLICATIONS F OR NURSI NG

Literature reviews are not static scholarly activities butrather an evolving research method As the volume ofresearch literature increases reliance on these summarieswill also increase This reliance will see literature reviewsforming the basis of many health care decisions rather thanthe original primary research It is therefore important thatthey are conducted with similar rigour to the studies theysummarise

Despite the importance of rigour in review methods thequality of reviews have been questioned An examinationof reviews in the nursing literature in 1987 found that themajority fell short of the standards used by the primaryresearchers and included problems such as failing to statehow the research reports were identified and chosen andinadequate reporting of the methods used by the reviewerto appraise and integrate studies (Ganong 1987) A similarlack of scientific quality was identified in a study ofmedical review articles (Mulrow 1987) This lack ofquality and rigour in review articles is a problem yet to beaddressed by nurses as a more recent appraisal of 33review articles published in one nursing journal found onlytwo papers specified the methods used in the reviewprocess (Redfern 1997)

Another area of concern for nursing is that themethodological development for literature reviews hasfocused on empirical research primarily the randomisedcontrolled trial How qualitative research can best beidentified in the literature critically appraised and thensummarised has yet to be determined For without theinclusion in literature reviews of qualitative nursingstudies this research will be lost to the profession as theresearch front moves on

Hierarchies of evidence have been developed and usedto help determine which study design best answers clinicalquestions The lsquobestrsquo in this context are study designs atleast risk of bias and error In order to evaluate theeffectiveness of a treatment or intervention a well

conducted randomised controlled trial is cited as thedesign that will provide the best evidence Similarly fordetermining risk factors for a specific condition cohortand case control studies are seen as the best design(Counsell 1997 Fowkes and Fulton 1991) Thesehierarchies commonly rank qualitative research at thesame level as the opinion of respected authorities andexpert committees (NHMRC 1995) However welldesigned qualitative research like other research is basedon clearly identified populations rigorous collectionanalysis and interpretation of data and carefullydocumented methods Qualitative research providesimportant information on issues such as theappropriateness of an intervention the behaviour andattitudes of clients and health care professionals whyclients fail to follow treatment protocols and the clientrsquosexperience of illness or treatment As these hierarchies ofevidence are mainly used for medical research nursesmust evaluate their appropriateness for research andliterature reviews addressing nursing issues

CONCLUSION

There has been a progressive evolution of reviewmethodology and this has reached a point where reviewsthemselves are now considered research activities Despitethese dramatic developments review methods are stillevolving to meet the needs of health professionals Todate nursing has had only limited participation in thisprocess instead borrowing methods developed primarilyfor medicine or the social and behavioural sciences

Some issues of importance have still to be addressed bynursing and these relate to the quality of reviews and tothe methods used If these reviews are to move nursingknowledge from the researcher to the practitioner theymust be useful to the profession This will meandeveloping review methods that allow the broad nature ofsome of nursing questions to be appropriatelysummarised It will also mean incorporating researchinformation into reviews that are meaningful to nursesWhile randomised controlled trials will certainly answersome nursing questions there are nursing researchquestions best approached by other means To not includequalitative studies in literature reviews will have importantconsequences for nursing Rigorous methods to identifyappraise integrate and summarise qualitative research areyet to be developed

While there will always be a place for the briefdiscussion paper reviews that aim to change clinicalpractice must be of the same standard as the primaryresearch they summarise This high standard must beapplied during the conduct of the review and also in thereporting of the methods used

SCHOLARLY PAPER

37


REF ERENCESAvis M 1994 Reading research critically An introduction to appraisal Designsand objectives Journal of Clinical Nursing 3227-34

Booth A 1996 In search of the evidence Informing effective practice Journalof Clinical Effectiveness 1(1)25-29

Cooper H 1984 The integrative research review A systematic approach SagePublications Beverly Hills

Cooper HM and Rosenthal R 1980 Statistical versus traditional proceduresfor summarising research findings Psychological Bulletin 87(3)442-449

Cooper HM 1988 Organizing knowledge synthesis A taxonomy of literaturereviews Knowledge in Society Spring104-126

Counsell C 1997 Formulating questions and locating primary studies forinclusion in systematic reviews Annals of Internal Medicine 127380-387

Dickersin K and Berlin JA 1992 Meta-analysis State of the scienceEpidemiologic Review 14154-176

Feldman KA 1971 Using the work of others Some observations on reviewingand integrating Sociology of Education 486-102

Fowkes FGR and Fulton PM 1991 Critical appraisal of published researchIntroductory guidelines British Medical Journal 3021136-40

Ganong L H 1987 Integrative reviews of nursing research Research inNursing and Health 101-11

Glass GV 1976 Primary secondary and meta-analysis of researchEducational Researcher 5(10)3-8

Glass GV McGaw B and Smith ML 1981 Meta-analysis in social researchSage Publications Beverly Hills

Jackson GB 1980 Methods for integrative reviews Review of EducationalResearch 50438-460

Jadad AR and McQuay HJ 1993 Searching the literature Be systematic inyour searches British Medical Journal 30766

Katerndahl DA and Lawler WR 1999 Variability in meta-analytic resultsconcerning the value of cholesterol reduction in coronary heart disease A metameta-analysis American Journal of Epidemiology 149429-41

Light RJ and Smith PV 1971 Accumulating evidence Procedures forresolving contradictions among different research studies Harvard EducationalReview 41429-471

Manten AA 1973 Scientific literature reviews Scholarly Publishing 575-89

Meltzer S Leiter L Daneman D Gerstein HC Lau D Ludwig SYaleJ Zinman B and Lillie D 1998 1998 clinical practice guidelines for themanagement of diabetes in Canada Canadian Medical Association Journal159(8 Suppl)S1-S29

Mulrow CD 1987 The medical review article State of the science Annalsof Internal Medicine 106485-8

Mulrow CD and Cook DJ 1997 Systematic reviews Critical links in thegreat chain of evidence Annals of Internal Medicine 126389-391

Mulrow CD and Oxman AD 1996 Cochrane collaboration handbook[updated 1 March 1997] In The Cochrane Library [database on disk andCDROM] The Cochrane Collaboration Oxford Update Software Updatedquarterly

NHMRC 1995 Guidelines for the development and implementation of clinicalguidelines 1st ed Canberra Australian Government Publishing Service

NHS Centre for Reviews and Dissemination 1996 Undertaking SystematicReviews of Research on Effectiveness CRD Guidelines for Those CarryingOut or Commissioning Reviews York University of York

Oxman AD and Guyatt GY 1991 Validation of an index of the quality ofreview articles Journal of Clinical Epidemiology 44(11)1271-8

Petitti DB 1994 Meta-analysis decision analysis and cost-effectivenessanalysis New York Oxford University Press

Price D 1965 Networks of scientific papers Science14956-64

Redfern S 1997 Review articles in the Journal of Clinical Nursing Journalof Clinical Nursing 6251-252

Stevens A and Milne R 1998 A knowledge-based health service Howdo the new initiatives work Journal of the Royal Society of Medicine91(Suppl 35)26-31

SCHOLARLY PAPER

38


A concern for standards of reviews

Due to the accumulative nature of scientific knowledgetrustworthy accounts of past research are necessary(Cooper 1984) In the 1960s Price described the role of theliterature review as replacing those papers lost from sightby the research front (Price 1965) Primary researchinformation is digested sifted classified and synthesised(Manten 1973) Because reviews deliberately aim atcumulating knowledge in a specific area they play a majorrole in the progress of a discipline in that they bringtogether previous work identifying past achievements andpossible future directions (Feldman 1971) Without thesereviews of the literature it is likely that a large amount ofthe research would be lost to the profession as the researchfront continues to move to new ground or rediscovers pastareas of interest on which to start fresh investigations

Literature reviews have also been likened to primaryresearch and as such seek to make accurate generalisationsabout phenomena from limited information (Jackson 1980)Cooper and Rosenthal (1980) suggested that literaturereviews also have great information-gatekeeping potentialbecause knowledge is communicated to undergraduatesand the lay public through reviews rather than throughthe primary research reports However a criticism of thereviews of the 1970s was that they made no attempt atrigorous definitions or standardisation of the techniquesused and therefore the findings could be biased byfactors that would be unforgivable in primary research(Glass et al 1981)

Prior to the 1980s there were no explicit methods forconducting reviews and so reviews varied widely in bothquality and scope which made assessment of their qualitydifficult (Jackson 1980) Light and Smith (1971) used afour category typology to characterise the approach takenby reviewers of this period by which they

listed any factor that had shown an effect on adependant variable

excluded all studies except those that supported aparticular point of view

averaged statistics across relevant studies in one formor another or

counted studies with similar results

Even in the late 1980s many scholarly publications hadno specific or formal definition of a literature review(Cooper 1988) As the volume of research continued togrow the traditional literature review which cited a smallnumber of studies no longer did justice to the literatureand prompted the term lsquothe misinformation explosionrsquo(Glass 1976) The volume of literature has continued togrow dramatically with the annual publication of 20000 to30000 biomedical journals and 17000 biomedical texts(Booth 1996) This makes presentation of an accuratesummary of the research a daunting challenge for not only

must the relevant studies be found in this mass ofpublications but issues such as the variable quality ofstudies and contradictory result must be addressed Boothdescribed the current dilemma facing health care workersas lsquoDrowning in information thirsting for evidencersquo(Booth 1996 p 25)

Types of reviews

One early attempt to describe the potential foci ofliterature reviews identified three different componentsand suggested that a comprehensive review will likelyaddress two or more of these areas (Cooper 1984) Thesecomponents were

integrative reviews - summarise past primary researchdraw overall conclusions highlight unresolved issuesand provide direction for future research

theoretical reviews - present theories to explainphenomena compare them in terms of breadth internalconsistency and the nature of their predictions and

methodological reviews - examine and critique theresearch methods and operational definitions that havebeen applied to a problem and address rigour and therisk of bias

In the late 1980s Cooper (1988) developed a taxonomywhich provided a framework for reviews that hadpreviously been lacking This taxonomy addressed sixcharacteristics of reviews

Focus primary research outcomes research methodstheories or the application of findings of research

Goals integration criticism or the identification of centralissues

Perspective neutral where the reviewer attempts topresent all arguments espousal were the perspective is aparticular argument or issue paying little attention to otherviews

Coverage may range from exhaustive to only arepresentative cover of the available research

Organisation how the literature is presented in thereview for example historically in chronological orderconceptually with literature relating to the same ideaspresented together or methodologically in which literatureis grouped according to the methods employed in theprimary research

Audience the differing audiences to whom the reviewer isdirecting the work

These developments provided a structure for reviewsand acknowledged the differing methods used byreviewers With this taxonomy came the recognition thatlike primary research the scope of literature reviews alsodiffered from a simple discussion to a comprehensivecoverage of the literature using methods with a rigoursimilar to that of the research it summarised

SCHOLARLY PAPER

34


Meta-analysis



Systematic reviews







SCHOLARLY PAPER

35




























Structured reviews


SCHOLARLY PAPER

36









CONCLUSION




SCHOLARLY PAPER

37






























SCHOLARLY PAPER

38


Meta-analysis



Systematic reviews







SCHOLARLY PAPER

35




























Structured reviews


SCHOLARLY PAPER

36









CONCLUSION




SCHOLARLY PAPER

37






























SCHOLARLY PAPER

38




























Structured reviews


SCHOLARLY PAPER

36









CONCLUSION




SCHOLARLY PAPER

37






























SCHOLARLY PAPER

38









CONCLUSION




SCHOLARLY PAPER

37






























SCHOLARLY PAPER

38






























SCHOLARLY PAPER

38

literature reviews: evolution of a … the literature review methodology and discusses some of the...

Documents