economics working papers

A Survey of JEL Codes:

What Do They Mean and Are They Used Consistently?

Lea-Rachel Kosnik1

2016

1 University of Missouri-St. Louis, Department of Economics Email: [email protected]

Working Papers Economics

UMSL Department of Economics Working Paper #1011

Department of Economics 408 SSB

University of Missouri – St. Louis 1 University Blvd

St. Louis, MO 63121 https://www.umsl.edu/econ/

https://www.umsl.edu/econ/

A Survey of JEL Codes:

What Do They Mean and Are They Used Consistently?

Lea-Rachel Kosnik Department of Economics

University of Missouri-St. Louis St. Louis, MO 63121-4499

Tel: 1-314-516-5564 Fax: 1-314-516-5352

[email protected]

2016 - Draft

Abstract

The use and prevalence of JEL code categorization is wide in the field of economics, but what do JEL code classifications actually tell us? And are they used with consistency by academics in the field? Utilizing a dataset of articles published in the American Economic Review from 1990-2008, we investigate whether there is heterogeneity in JEL codes assignments between authors and editors. We find that there is. A secondary goal of this paper is to survey overall thematic trends in JEL code usage over the past four and a half decades. One result is that JEL category M: Business Economics, in particular, appears to be thematically and spatially distinct from much of the rest of the published literature in the top general interest journals in the field.

JEL Code: A1; B0 Keywords: text analysis, JEL code, economics research, economics literature, thematic analysis Acknowledgements: Many thanks to Robert Whaples, David Laband, Dan Hamermesh, Stefano DellaVigna, Bob Tollison, Allen Bellas, two anonymous reviewers and seminar participants at the University of Missouri-St. Louis, and the Southern and Midwestern regional economics meetings.

2

Introduction

The use and prevalence of JEL code categorization is wide in the field of economics, but

what do JEL code classifications actually tell us? And are they used with consistency by

academics in the field? Cherrier (2015) has pointed out in her thorough analysis of the history of

JEL code construction that there were often fierce debates within the profession as to what the

purpose of the JEL code system was, and how it should be both constructed and subsequently

utilized. Do such disagreements continue to have an effect on how JEL codes are assigned

today? The first goal of this paper is to analyze a set of papers with both editor-assigned and

author-assigned JEL codes and analyze them for significant differences. Understanding JEL

code usage is important for many reasons; they are now the standard classification system used

by most researchers in the field, JEL codes are prevalent across national and international

economics journals and numerous classification databases such as EBSCO and EconLit, and they

have been used as input variables in research studies that seek to determine subject focus of

academic research (Card and DellaVigna, 2013; Kelly and Bruestle, 2011; Whaples, 1991). This

paper tests the standard assumption that JEL codes are used with consistency in classifying

papers in the field.

A second goal of this paper is to survey the primary 16 JEL subject categories currently

in use, and analyze them for top thematic trends.1 What have been the big issues studied in

labor, for example, or natural resource economics, over the past four and a half decades, and how

have these top foci changed over time? Through a textual analysis of JEL code usage, and an

accompanying spatial network analysis of key term frequencies, this paper explores thematic

1 There are officially twenty current JEL subject categories, but the first two - “A: General Economics & Teaching” and “B: History of Economic Thought, Methodology, & Heterodox Approaches” - and the last two – “Y: Miscellaneous” and “Z: Other Special Topics” – are omitted from this analysis as they are used rather infrequently, making it difficult to run empirical analyses with so few observations.

3

trends, including which research subjects tend to be investigated together, and which are

spatially far apart. Spatial network analysis can highlight well-investigated, nodal areas of

economics research, as well as outliers in the field where perhaps more research attention is

needed, or where new trends in thought on the edges of the horizon are being developed.

Ultimately, it is important to understand how economists categorize their own research

literature, as much can depend on it. If researchers, editors, and authors are using JEL codes

disparately, papers may not be indexed correctly and prevalent misinformation could lead to

inefficiencies in research access; papers not being read in related thematic categories that should

be, and other papers appearing prominently in areas for which they are only tangential.2 Cherrier

(2015) points out that a vague JEL code system can also be confusing to those outside the field –

for example to employers, government agencies, or journalists – when they are trying to navigate

research output and trends in economics. A thorough JEL code analysis will also highlight,

perhaps unrealized top thematic categories studied, and networked areas of research focus and

attention. It will give an indication of how the field has been spending its intellectual capital

over the past four and a half decades.

Literature Review

The JEL classification system was developed over one hundred years ago as a method of

classifying scholarly literature in the field of economics.3 It is now the standard classification

system used by most researchers in the field, and JEL codes are prevalent across national and

2 Working paper websites such as SSRN (Social Science Research Network), for example, have search functions based on JEL codes. 3 This description is taken directly from the JEL classification system webpage: https://www.aeaweb.org/econlit/jelCodes.php.

4

international economics journals and numerous classification databases such as EBSCO and

EconLit.

JEL codes are used by employers to identify researchers and their work, they are used by

journalists to find articles relevant to understanding contemporary policy topics, they are used by

online portals to categorize work, and they are often used by academics in the field when trying

to categorize and understand the kind of research that gets published in top academic journals

(Rath and Wohlrabe, 2015; Grijalva and Nowell, 2014; Card and DellaVigna, 2013; Kelly and

Bruestle, 2011; Kim et al., 2006; Durden and Ellis, 1993). While the usefulness of the JEL code

classification system is without controversy, analyses of the JEL code classification system itself

are rare. Cherrier (2015) has put together a historically insightful look at the behind-the-scenes

creation of the JEL code system, including some of the politics and egos that went into its

various iterations, but that work is a qualitative historical narrative.4 This research is a more

quantitative investigation that surveys JEL code usage over time, and whether or not there has

been agreement between authors and editors in the utilization of JEL codes assigned to the exact

same papers.

The main contributions of this paper are three-fold. First, it investigates whether JEL

codes have been used consistently in the field, as represented by JEL code assignations by

authors and editors to the very same articles. Second, this research adds to the discussion of

economics research trends more broadly by analyzing JEL subject categories themselves and

what they have stood for in the top general interest journals in the field since 1969; this is a new

angle to the research trends literature. Finally, the analysis includes spatial network and textual

analysis, unique methodological tools relatively new to the field, though increasingly popular in

4 If the reader wishes to understand the narrative history of changes to the JEL code, they should refer to that work.

5

their application (Kosnik, 2015a; Kosnik, 2015b; Baker et al., 2014; Gentzkow and Shapiro,

2010; Tetlock, 2007; Antweiler and Frank, 2004).

Theory

When an editor (or editorial assistant) assigns a JEL code to a paper, what is her

objective? Is she trying to maximize the amount of informational content conveyed by the JEL

code classification, and so will use it broadly and assign it liberally? Alternatively, is her goal to

accurately reflect a tight understanding of a certain subject category and not allow it to be diluted

with only tangentially related research, so that when people search on it, they know what they are

getting? Conflicting interpretations of the use of JEL code assignments indeed went into the

creation of the original JEL code classification system (Cherrier, 2015), as well as affected its

subsequent iterations. Debates were had amongst top researchers in the field as to whether the

JEL category codes should be broadly interpreted, or succinctly refined. Some of these debates

have never been satisfactorily settled.5

Motivations for JEL code usage may also differ by assignee. When an author (as

opposed to an editor) assigns a JEL code to her own paper, what is her objective? To identify the

paper to its most likely readers, or to broaden its appeal as well as its readership by assigning

codes in a more tangential manner? Would the latter lead to more cites and a greater impact on

the author’s professional reputation?

It is hard to decide a priori which motivation should dominate either editors, or authors.

Both face an objective function where they are likely to desire maximization of readership of the

article under assignment, but subject to reputation constraints from assigning far-flung JEL codes

5 This may be why the JEL classification system appears to be headed into yet another iteration – see the minutes of the meeting of the Executive Committee, January 2, 2014 at: https://www.aeaweb.org/AboutAEA/meeting_minutes.php

6

that waste a reader’s time. Which effect dominates? In this paper, we compare JEL code

assignments by authors and editors, on the very same papers, and test whether there is

heterogeneity in the number and type of JEL code assignments between the two groups. Our

null hypothesis, therefore, is that authors and editors assign JEL codes to the same papers in the

same manner, as opposed to the alternative where they assign them differently:

: , ,

: , ,

where i represents a given academic paper, and j=1, …, n and k=1, …, m represent paper-specific

JEL codes assigned by the editors and authors respectively, j ≥ 1, k ≥ 1.

Data

Many articles, when they are first submitted to a journal for publication consideration,

contain JEL codes assigned (or suggested) by the author. Later, if those articles are accepted and

published, editorial staff assign the official JEL codes which end up in the EBSCO database.

From 1990-2008 the American Economic Review (AER) published articles with the usual editor-

assigned JEL codes, but also with the original author assigned JEL codes remaining visible on

the first page of the publication. This availability of dual JEL code assignments – for the same

papers - allows us to test whether, and how, editor assigned codes differ from author assigned

codes, at least for that two decade time span and in the journal AER.

In this paper we also investigate thematic trends of the top JEL codes currently in use.

For this analysis we extend our dataset to a longer time span, 1969-2014, and beyond just the

7

AER. For this part of the research we examine JEL code usage in five top general interest

journals in the field, including: American Economic Review (AER), Econometrica (E), Journal

of Political Economy (JPE), Quarterly Journal of Economics (QJE), and Review of Economic

Studies (RES).6

All article abstracts published in these five journals, for the years 1969-2014, are in the

database. The corpus includes abstracts from all research-oriented articles that have been

published in English,7 including full-length monographs, full-length book reviews, and

comments and replies (which do occasionally include an abstract). Entries not included in the

dataset include editor’s notes, conference announcements and programs, auditor’s reports,

indexes, and other similar non-research focused entries. As well, entries with no JEL codes

listed whatsoever were not included (there were few of these, and generally they were

aberrations in the EBSCO database). Special symposium articles are included.8 Given these

criteria the corpus includes 15,514 articles, some descriptive information for which can be found

in Table 1.

The starting year of 1969 was chosen for a specific reason. The JEL classification code

system has undergone two significant revisions since its initial implementation at the turn of the

twentieth century.9 The first major revision was in 1968, the second major revision in 1990. In

order to avoid construction of two different mapping systems to try and harmonize three different

JEL code classification schemes, the dataset begins in 1969, thus avoiding any papers that

utilized the initial iteration of the JEL code classification scheme. We employ a single mapping

6 This list was chosen after considering a number of different rankings, including Engemann and Wall (2009), Kalaitzidakis et al. (2001), and a variety of online listings. In addition, these journals are the most common ones used in published research that investigates trends in the discipline of economics (Kosnik, 2015; Kosnik, 2014b; Hamermesh, 2013; Card and DellaVigna, 2013; Laband et al., 2002; Laband and Tollison, 2000). 7 Some of these journals, especially in earlier years, included the occasional article in French or German. 8 It is worth noting, however, that the American Economic Review’s annual Papers and Proceedings issue is not included. 9 And as Cherrier (2015) points out, a few less significant revisions as well.

8

strategy, therefore, to bring the pre-1990 (but post-1968) JEL codes into alignment with the post-

1990 JEL codes. This mapping strategy relies on that used in Card and DellaVigna (2013),

editing it only when a code or category was found to be unrepresented in that scheme.10

Appendix A provides the pre-1990 to post-1990 JEL code mapping strategy. As Cherrier (2015)

notes, the 1968 revision was about rationalizing multiple classifications that were originally

pushed by professionals outside of the discipline who wanted a way to identify categories of

expertise for governmental war efforts.11 The 1990 revisions were prompted by economists’

frustration with the later lack of space, as new approaches in economics developed.

For all of the article abstracts in the dataset we have editor assigned JEL codes, as listed

in the EBSCO Information Services database – these are the JEL code assignations you would

see if you looked these articles up in EconLit, for example. All of the articles in our dataset have

at least one JEL code, 37% have two JEL codes, 19% three, 7% four, 3% five, a little more than

1% have six, and a little less than 1% have as many as seven JEL codes assigned. Seven appears

to be the limit for editor assigned JEL codes.

Methodology

For the comparison of author and editor assigned JEL codes, standard statistical analysis

was utilized. For the thematic and spatial network analysis, textual analysis12 was employed.

Textual analysis is the accumulation of large amounts of textual data, the cleaning and parsing of

10 Note that the Card and DellaVigna (2013) mapping scheme is constructed from information provided in the Journal of Economic Literature (1991), which describes how the pre-1990 JEL codes correspond to the post-1990 codes. 11 A perusal of the pre-1968 codes is fascinating for the level of minute, and what seems today extremely superfluous, detail. 12 Textual analysis as a methodological tool has taken off in the last decade in many social science disciplines (most notably political science and psychology), and it has begun to be utilized in the economics literature as well (Kosnik 2015, 2014a, 2014b; Baker et al., 2014; Gentzkow and Shapiro, 2010; Tetlock, 2007; Antweiler and Frank, 2004).

9

the text with unique algorithms, and then the turning of the text into a database where the words

themselves are statistically analyzed for trends and correlative patterns.

The unstructured text of the abstract from each research article was organized within a

vector-space model (VSM). In the VSM each element of the vector indicates the occurrence of a

word within the document. A collection of documents results in a collection of vectors, and

there were 15,514 in this study. Once the raw text from each abstract was input into a relational

database, a number of algorithms were performed to clean the data. A typical lemmatization

process was then applied in order to reduce the words to their root form, taking note to preserve

technical economic terms such as “externality” and “regression.” The text also underwent a

standard exclusion process in order to remove words with little semantic value such as pronouns

and conjunctions. Finally, in order to make the thematic analysis (discussed below) stable,

approximately 10% of the least frequent words in each of the JEL categories studied were

excluded.

The method used to extract thematic topics from the documents (first segmented by JEL

category type, C-R) was factor analysis (Rummel, 1970). All words with a factor loading higher

than 0.40 were retrieved as part of an extracted topic.13 The number of topics returned per

analysis was set to ten, and generally ten were returned, but in some instances the algorithm

returned fewer than that. The thematic results presented below also include eigenvalues, which

indicate the strength, or degree of confidence, in the thematic topics chosen - higher eigenvalues

imply greater confidence that the thematic topic described indeed represents a theme in the

corpus. Finally, % cases gives the percentage of articles within each JEL category that is

counted as including a particular theme – a higher % cases implies that the theme is widely

13 Note that topic modeling using factor analysis (as opposed to hierarchical cluster analysis, for example) allows words to be associated with more than one factor. This is often more realistic of the way in which, particularly polysemous words, are used.

10

represented across the JEL category corpus, while a smaller % cases implies that the theme

(which may be strong, due to a high eigenvalue) is at the same time discussed in a relatively few

number of articles overall.

Results - Editor vs. Author Assigned JEL Codes:

In this first section of results we examine whether there is significant heterogeneity

between editor and author assigned JEL codes, as assigned to the exact same papers. Our dataset

focuses on AER articles from 1990-2008, of which there are 1,756. However, while editor

assigned JEL codes are provided for every article in the dataset, including reviews and

comments, author assigned JEL codes are available only for full-length research articles. Our

comparative dataset, therefore, is reduced to 970 articles. Of these, editor assigned JEL codes

were different than author assigned JEL codes 43% of the time – a significant difference.

The fourth and fifth columns of Table 2 show the breakdown of these 970 articles by

editor (E) and author (A) assigned JEL category.14 In total there are 2489 editor assigned JEL

codes for these papers, and 2649 author assigned codes. On average, editors assign 2.57 JEL

codes per paper, while authors assign 2.73 codes. A one-tailed t-test finds this difference

statistically significant at the 1% level, though it is a numerically small difference.15 Authors are

in general more liberal in their use of JEL code assignment than editors.16

At the same time, many of these extra author assigned JEL codes appear to differ only by

subcategory (for example H00 and H01), and not by broad category (H versus I). When

14 One article can be assigned to more than one JEL code, so the fourth and fifth columns in Table 2 will not sum to 970. 15 The mean for editor assigned JEL codes, µe, is 2.565979. The mean for author assigned JEL codes, µa, is 2.730928. = 1.293069. = 1.421415. The t-statistic is 3.118, and the p-value is 0.0009. 16 Note that editor assigned JEL codes appear to be capped at seven, while authors can assign an unlimited number of codes to a single article.

11

subcategories are combined so that each article is represented by its broad categories only, there

are 1,764 editor assigned codes and 1,582 author assigned codes. The difference is now reversed

in favor of editors, as it appears that editors are more liberal in their tendency to assign an article

across multiple disciplines. Overall, papers have different JEL code assignments by broad

category 52% of the time. On average, editors assign 1.83 broad JEL codes per paper, while

authors assign 1.64 broad JEL codes. A one-tailed t-test finds this difference also statistically

significant at the 1% level, though again the actual numerical difference is small. 17, 18 The black

(for “Editor) and gray (for “Author”) frequencies in Figure 1 illustrate this comparison.

Figure 1 tells us a few things. First, there are not any enormous height differences

between the “Editor” and “Author” frequencies at any of the category markers, implying roughly

similar amounts of category code assignments between authors and editors. However, it is worth

noting the “In Common” frequencies, in green, which corresponds to the sixth column in Table 2

(turned into percentages). This shows the total number of articles in each category that received

the same JEL code assignment by both editors and authors. This is everywhere less than the

code assignments by editors and authors alone. JEL category “P,” for example, has 30 articles

assigned to it by both editors and authors, but they aren’t the same 30 articles (!); only 20 are in

common.

Second, it appears from Figure 1 that authors are more eager to assign their papers to

what they likely perceive as the general categories of “C: Mathematical & Quantitative

Methods” and “D: Microeconomics.” Editors, on the other hand, are more discerning when it

comes to categories “C” and “D”. At the same time, however, editors are more liberal in their

17 The mean for editor assigned broad JEL codes, µe, is 1.834021. The mean for broad author assigned JEL codes, µa, is 1.64433. = 0.612657. = 0.750818. The t-statistic is 4.203, and the p-value is 0.0000. 18 While both broad category and total category usage differ by approximately a fifth of a code, note that this difference is more significant for broad category assignments, as less of them are assigned in the first place. In other words, the difference is about 6% for all categories, but an 11% usage difference for the broad category codes.

12

use of nearly all the other categories. In sum, editors seem to be making more of an effort to

have articles cross discipline boundaries, while authors don’t cross-list, as much as they fine tune

JEL code assignments within a broad category (through their use of numerous subcategory

assignments).

This seems to imply, regarding the theoretical motivations described earlier, that editors

are more influenced by the motivation to have a JEL code apply as broadly as possible, perhaps

in an effort to bring in readers beyond just the most obvious classification categories. Authors,

however, are more influenced by the motivation to firmly self-identify their papers into well-

defined, specific subject categories, perhaps in order to position themselves to close colleagues

in the field. The ultimate actions of authors and editors when assigning JEL code classifications

do differ, and in a statistically significant (if numerically small) way.

This result holds for the universe of articles investigated, but are there any differences by

subject category? For example, do authors and editors assign codes more similarly in “Q:

Agricultural and Natural Resource Economics,” as opposed to “D: Microeconomics”? The final

column in Table 2 investigates this question, by providing the percentage of articles assigned in

common by both authors and editors. All subject categories have differences, but the percent in

common ranges from a low of 57.5% in “C: Mathematical & Quantitative Methods,” to a

maximum of 84.3% in common in “F: International Economics.” The results in this column

highlight again the fact that “C: Mathematical & Quantitative Methods,” in particular, appears to

be a catch-all category for authors who like to give their papers at least one “quantitative”

designation, while editors are more discerning as to what constitutes a truly quantitative paper

category designation. This sort of difference/confusion in category interpretation is exactly what

13

was behind many of the conflicts in the JEL code classification creation story, as described by

Cherrier (2015).19

What about over time? The number of observations per year is relatively small (on

average, 51 articles per year), however Table 3 does show the number of these articles in each

year that have different editor and author assigned JEL codes, and what that percentage is of the

overall count of articles. The large (on average 43%) discrepancy between author and editor

assigned JEL codes has stayed relatively consistent over the time span under study, except for

the last two years of the dataset, 2007-2008. This appears to be when AER began a concerted

effort to align author and editor assigned JEL codes, which came to complete fruition in 2009. 20

An interesting final question to ask, is whether these somewhat different JEL code

assignments between editors and authors imply any thematic differences as well. Are specific

topics or policy applications filed differently by authors and editors across the subject

categories? This would be particularly important for employers, government agencies,

journalists, or others outside the field who may search economics research by JEL code, seeking

specific topical information. We will return to this question after we introduce thematic trends in

the JEL code categories more broadly in the next section.

Results - Overall Thematic Analyses:

Table 2 provides the 16 JEL categories studied, including (in the third column) the

number of articles represented from all five journals studied, from 1969-2014, and thus the

observations included in the thematic/spatial network analysis. The total number of articles adds

19 Indeed, besides the broad versus tailored debate about how detailed to create JEL categories, there were debates about whether to create additional categories that distinguished theory, methodologies, and applied work. It may be that authors assume methodology is divided up into category “C,” and that is why they use it so much, as opposed to editors who see it is as but another category of overall research. 20 2009 marks the first year that author and editor assigned JEL codes are always and for every paper identical.

14

to more than 15,514 because articles listed with more than one JEL code are represented more

than once. Some categories, for example “K: Law and Economics,” and “M: Business

Administration & Business Economics; Marketing; Accounting” had relatively few articles,

while others, like “D: Microeconomics” and “E: Macroeconomics” had many; the categories

with more articles were often able to return a greater number of themes than the categories with

fewer articles and a smaller word base. It is worth reiterating that the thematic analyses

uncovered here represent themes from these JEL categories as published just in the top general

interest journals studied, and not across the entire economics literature. Categories with prolific

field journals, for example, may certainly have had other or additional topics represented over

this time period; what is presented here are the main topics discussed in the top general interest

journals in economics.

Tables 4-19 display the thematic results for each of the 16 JEL categories studied. This is

an analysis of all the research article abstracts that include that JEL category,21 for the entire

length of the study (from 1969-2014). The first column in each table, Theme, describes the

themes for each research category22, the second column, Keywords, lists the keywords that the

algorithm identified as composing those themes, and the last two columns present the

eigenvalues and the percentage of cases that include that theme. A few observations are

immediately apparent.

21 Most of the research articles (83%) have more than one JEL code, and so are categorized in more than one JEL corpus; at the same time, if an article has the same JEL code twice (for example H00 and H01), it is utilized just once in the given JEL code (“H”) corpus. 22 The exact label (e.g. “Game Theory”) was assigned by the author after a perusal of both the keywords utilized and the corresponding articles assigned to that theme.

15

First, there are a number of themes that cross JEL categories and appear repeatedly

throughout the corpus.23 “Labor & Employment” is the most prevalent theme, appearing in

seven of the eighteen categories. “Voting & Elections,” “Gender Issues,” “Risk Aversion,”

“Auctions,” “Estimation Techniques,” and “Game Theory” are also relatively prevalent. This

illustrates that there are some topics which dominate the research interests of economists, across

disciplines.

A second observation is that, while there are some themes that are common across many

categories, at the same time, there are a few JEL categories which are extremely distinctive and

share very few, if any, top themes with any of the other categories. There are three of these

distinctive categories and they are “I: Health, Education, and Welfare,” “M: Business

Administration & Business Economics, Marketing, Accounting,” and “Q: Agricultural & Natural

Resource Economics, Environmental & Ecological Economics.” The top themes in these

categories are often applied and include things like “Donor Exchanges,” “Newborns,”

“Advertising,” “Entrepreneurship,” “Sulfur Emissions,” and “Forestry Resources.”

Overall, the top themes in each category accord with what one would expect for each JEL

code, including macroeconomic categories (i.e. “E” “F” “G”) containing monetary policy as a

top theme, and things like “Public Goods” being a top theme in “H: Public Economics,” and

“Racial Demographics” being a top theme in “J: Labor and Demographic Economics.” The

results appear to confirm that categorization of research articles by JEL category code conform

to expectations and are meaningful. This is reassuring, especially given the contentious, and at

times confusing, tug-of-war that went into the creation of the JEL code classification system

(Cherrier, 2015).

23 Note that these common themes are often supported by somewhat different keywords in different JEL category analyses. This implies that the particular foci of research questions studied across JEL categories may have differed, while the broader category of, say, “Game Theory” more generally applied.

16

Returning to the dataset of just AER articles from 1990-2008, we investigate the top

themes as described by the author-assigned articles versus the editor-assigned articles. Specific

results are available from the author upon request, but on average only about half of the top

thematic categories for each JEL category were shared between editor assigned and author

assigned papers. This is not actually surprising. As the “In Common” frequencies in Figure 1

indicates, quite a number of articles were not similarly assigned by editors and by authors,

therefore, it is not all that surprising that a textual analysis of their top themes differs as well.

What this implies for outsiders exploring academic research, however, is that authors and editors

may view papers rather differently and that they should explore broadly and widely to discover

thematic topics that may be very specific.

Thematic Analyses Over Time:

Next, for the JEL categories that contain enough research articles for stable decennial

analysis, we explore how top themes may have changed over time. We divide 1969-2014 into

four distinct time periods: I: 1969-1979, II: 1980-1989, III: 1990-1999, and IV: 2000-2014, and

run the same thematic algorithm described in the methodology section above, but for each

period. This analysis yielded several interesting results.

For category “C: Mathematical & Quantitative Methods,” one discovery is that “Input-

Output Models” were a top theme in period I, but at no other time. In addition, applied themes

were nowhere to be found in this category except for in the very last period, IV, where “Gender

Issues” suddenly showed up as a top research theme.

For category “D: Microeconomics,” the main interesting result was that the top themes

changed substantially in nearly every period I through IV. Microeconomic papers can have very

17

applied contexts, and this shows, with topics like “Stocks” “Taxes” “College & Students” and

“Traffic” showing up as top themes in the early years, and completely different topics, including

“Gender Issues” “The Firm” “Contracts” and “Auctions,” showing up in periods III and IV. The

JEL category “D: Microeconomics” appears to have a lot going on within it!

For category “E: Macroeconomics,” the topics were relatively similar across periods.

“Risk” appeared as a top theme across the decades, however, when digging deeper and

investigating what types of research papers composed this topic, the type of risk studied did seem

to change. In period I “Risk” was mostly about portfolio risk, while in time periods II-IV the

theme of “Risk” morphed more into risk aversion and utility effects. “Borrowing,” including

private sector, life-cycle, and government borrowing, appears to have been an extremely strong

theme in period III, but not in any of the other time periods. Finally, it may be noteworthy that

only in period IV do we get a top theme labeled “Disasters” which includes such keywords as

rare, disaster, risk-free, premium, equity, and Barro.

For most of the other categories that were able to be broken down by time period,24 a

main result across the JEL category codes appears to be that the top themes became more and

more applied as time went on. Particularly in period IV we start to see themes that are less

theoretical or estimation oriented, such as “mathematical techniques” and “models of utility,”

and more about particular contexts including, “Health Care” “Cars” “IPOs” “Oil” and

“Immigration.”

24 Specific results per JEL category available from the author upon request.

18

Spatial Network Analysis:

Finally, we can investigate with spatial network analysis the relationships between

different JEL categories and themes, to try and elucidate and investigate areas of economics

research that do, or do not, seem to occur (or at least, be categorized) together.

To begin, Figure 2 presents a network analysis of the sixteen JEL codes over the entire

timespan of the dataset, 1969-2014. The graph was created with the open source platform

Gephi,25 and the layout derives from a Force Atlas algorithm (Jacomy et al., 2014). The nodes

are the 16 JEL category codes analyzed throughout this paper, the edges are created by a count of

the number of times any two JEL codes appear together in a paper in the dataset (as assigned by

editors), and a modularity process was created to distinguish two communities: relatively

strongly related categories (green, and with thicker edges) and relatively weaker connections

(red, and with thinner edges). Approximately 2/3 of the connections are categorized as strong,

1/3 as weak. Figure 2 gives you a sense of the relationships of the JEL codes between each

other. Categories “C,” “D,” and “E” are some of the strongest and most central, while many of

the alphabetically later categories (i.e. “M” through “R”) are weakly related and do not appear to

be centrally categorized areas of research. Similar network analyses for the time periods I

through IV reveal remarkably similar graphs.

Table 20 further elucidates the network analysis by providing information on the

percentages of the 15,514 articles, as assigned by editors, that have JEL codes listed in more than

one category. Again one can see the centrality and prevalence of categories C, D, and E to the

network, and the relative isolation of the later categories, including K, M, N, P, Q, and R.26

25 Gephi can be downloaded at: http://gephi.github.io/ 26 Information on overlaps at a finer level of detail (i.e. 2-character, C0, and 3-character, C00 overlaps) can be made available by the author upon request.

19

In an effort to reveal spatial relationships between themes, and not just JEL categories,

we also performed a network analysis of the 91 themes described in Tables 4-19. Figure 3

presents that spatial relationship for the time period 1969-2014.27 Due to the fact that there are

91 nodes, and subsequently, 4,186 undirected edges, the graph is too dense to label everything

with clarity, so instead just the thematic “outliers” are labeled in an effort to illustrate the less

connected themes.28 One thing of interest to note is that many of the outlier topics are listed as

top themes in JEL category “M: Business Administration & Business Economics,” which in

Figure 2 is also an outlier as a JEL category. Business Economics as a category appears to be

somewhat set apart from the rest of the research discussion in the wider field of economics, even

more so than some of the other outlier fields from Figure 2 (i.e. JEL categories N-R).

Similar network analyses for the time periods I through IV reveal graphs with many of

the same outliers.29 The exact shapes of the network analyses change somewhat in each decade,

but a majority of the nodes portrayed as outliers (including “Firm Takeovers,” “Retail Sales,”

“Entrepreneurship,” and “Bait and Switch and Seller Disclosures”) remain the same. In other

words, the relationships between the categories C-R has remained relatively consistent over time.

Conclusions

A main result from this research is that there is indeed a statistically significant disparity

in use of JEL code assignments between editors and authors, for the same papers. This

27 This graph was also created with Gephi and utilizes a Force Atlas algorithm. 28 The reasons for the “outlier” statuses are not clear. It could be that these topics are simply tangential to much of the rest of the research discussion in the field, or, it could be that these topics are up and coming and will become more integrated in the future. There are many possible reasons these themes are located to the edges of the network analysis, a further investigation into such reasons would be a useful area for future research. 29 This is unsurprising as the 91 themes analyzed are the same in every time period I through IV. If instead different network analyses were performed, limited to the top themes from each particular decade only, then the relevant graphs and outliers would likely be different. When analyzing all 91 themes over time, however, the change in relative emphasis over the decades appears small.

20

surprising result is tempered by the fact that while the statistical significance is strong, the actual

size effect is small, with often just one or two different JEL codes per paper. Specifically, these

quantitative results uncover the surprising fact that authors tend to apply more total JEL codes to

their papers (though they are distinguished often by differing subcategories and not by broad

category), while editors assign less total JEL codes per paper, but more codes to a given paper

that cross discipline boundaries. Perhaps editors (and their staff) are making an effort to market

the articles they publish across a wider audience? Debates as to whether JEL codes should be

broadly interpreted or narrowly defined, as well as whether new methodological categories of

JEL codes should be created appears to be ongoing. Future research into the motivations for this

result would be worthwhile. It would also be helpful to understand this result before any further

iterations to the JEL code classification scheme are considered in the future.

The second result from this research survey is a more comforting one; that JEL category

codes do appear to represent papers that study topics and themes one would expect to be

assigned to those codes. Natural resource economics (“Q”), for example, includes papers

analyzing sulfur emissions and forestry resources, and labor economics (“J”) includes papers

analyzing labor, employment, education, and racial demographics. Had this been different, that

would have been surprising indeed.

A third result from this research is that over the long time span from 1969-2014, across

all JEL category codes, a common trend has been the move to more applied topics and papers

and away from primarily theoretical papers. As the top themes suggest, the discipline of

economics is moving towards a more applied, public policy focused direction.

Finally, spatial network analysis has given us a glimpse into which thematic topics appear

to be relative outliers in the broader research discussion in economics, and which are more

21

integral. While many of the later JEL categories (i.e. “M” through “R”) are spatially further

away, “M: Business Administration & Business Economics” wins for having the most top

themes the furthest away from other topics studied in the field. It is as if business economics

really is housed in a college separate from the rest of the economics school.

22

Journal 1969 1970s 1980s 1990s 2000s 2010s Totals American Economic Review 122 1,168 1,193 866 988 626 4,963 Econometrica 63 967 839 544 605 319 3,337 Journal of Political Economy 77 975 675 521 425 154 2,827 Quarterly Journal of Economics 50 534 564 448 413 211 2,220 Review of Economic Studies 45 518 524 394 430 256 2,167 Totals 357 4,162 3,795 2,773 2,861 1,566 15,514

Table 1 - Article Counts, per Journal and Over Time

23

JEL Subject

Code Subject

Total Number

of Articles

E

AER Articles

A

AER Articles

E&A AER Articles in Common

E&A AER Articles in Common

Percentage*

C Mathematical & Quantitative Methods 3902 86 133 63 57.5 D Microeconomics 4766 359 362 280 77.7 E Macroeconomics & Monetary Economics 3723 216 193 153 74.8 F International Economics 1614 136 125 110 84.3 G Financial Economics 1394 133 107 98 81.7 H Public Economics 1475 84 82 55 66.3 I Health, Education, and Welfare 638 62 44 38 71.7 J Labor and Demographic Economics 2648 164 124 104 72.2 K Law and Economics 240 26 35 22 72.1 L Industrial Organization 1757 163 117 89 63.6 M Business Administration & Business Economics; Marketing; Accounting 263 36 21 17 59.6 N Economic History 318 45 36 26 64.2 O Economic Development, Technological Change, & Growth 1056 158 110 92 68.7 P Economic Systems 362 30 30 20 66.7

Q Agricultural & Natural Resource Economics; Environmental & Ecological Economics 576

31

19

18

72

R Regional, Real Estate, and Transportation Economics 486 28 24 19 73.1

Table 2 – JEL Subjects

* Percentage calculated by taking the average of “E AER Articles” and “A AER Articles” and dividing it into “E&A AER Articles in Common”

24

0%

5%

10%

15%

20%

25%

C D E F G H I J K L M N O P Q R

Editor Assigned

Author Assigned

In Common

Figure 1 – Editor vs. Author Assigned JEL Codes

25

Year

# Articles with Different JEL Code

Assignments

% Articles with Different JEL Code

Assignments 2008 8 13.56 2007 1 1.96 2006 17 30.91 2005 28 50.00 2004 31 53.45 2003 27 50.94 2002 28 58.33 2001 23 43.40 2000 20 43.48 1999 28 63.64 1998 27 57.45 1997 21 48.84 1996 26 50.98 1995 19 35.19 1994 31 57.41 1993 21 39.62 1992 13 24.07 1991 27 50.94 1990 14 36.84

Table 3 – Author and Editor Distinctions Over Time

26

Theme Keywords Eigenvalue % Cases

1 Estimation Techniques identification; estimation; nonparametric 11.28 11.05

2 Price Theory demand; labor; wage; supply; price 4.88 21.60

3 Game Theory game; player; payoff; strategy; repeat; equilibrium; nash; equilibria; play; action 3.64 27.81

4 Hypothesis Testing null; test; hypothesis; testing; wald; statistics; ratio; statistic; multiplier 3.3 15.76

5 Equilibrium economy; exchange; allocation; competitive; equilibrium 2.77 24.50 6 Auctions seller; buyer; bid; offer; auction 2.71 6.56 7 OLS square; least; variable; estimator; equation; regression; coefficient 2.65 25.55

8 Expected Utility & Risk Aversion risk; utility; aversion; expect 2.57 14.45

9 Time Series cointegration; root; series; time; unit 2.44 16.20 10 Monte Carlo carlo; monte; sample 2.38 8.18

Theme Keywords Eigenvalue % Cases 1 Bequests bequest; child 6.16 2.06 2 Auctions auction; bid; bidder; seller; buyer 2.94 6.74 3 Voting & Elections voter; election; vote; electoral; candidate; political 2.73 9.27 4 Game Theory game; player; payoff 2.57 9.23 5 Risk Aversion risk; aversion; risky 2.47 8.02

6 Labor & Employment worker; wage; labor; job; employer; unemployment; employment 2.39 9.27

7 Moral Hazard & Adverse Selection hazard; moral; adverse; contract; selection 2.27 9.44

8 Household Consumption data; estimate; household; consumption; test; expenditure 2.2 28.07


1 Monetary Policy rate; monetary; nominal; inflation; interest; real; money 6.79 38.25 2 Competition & Firms firm; competition; competitive; monopoly 2.8 19.29

3 Labor & Employment worker; job; unemployment; wage; labor; employment; employer 2.67 17.35

4 Taxation income; consumption; tax; wealth; propensity 2.62 18.43 5 Hypothesis Testing estimate; hypothesis; test 2.42 15.85

6 International Economy country; foreign; world 2.35 10.42

7 Production Functions production; input; substitution; factor; function 2.34 23.72 8 Business Cycle business; cycle; fluctuation; procyclical 2.2 10.23 9 Risk Aversion risk; aversion 2.12 4.51

10 Federal Reserve bank; reserve; federal 2.06 4.43

Table 5 – Thematic Analysis JEL Category “D”

Table 4 – Thematic Analysis JEL Category “C”

Table 6 – Thematic Analysis JEL Category “E”

27


1 Exports exporter; destination; export; firm 13.09 15.68 2 Growth Factors growth; technical; capital; steady; function 4.02 26.21 3 Monetary Policy monetary; exchange; balance; rate 3.91 32.03

4 Factor Endowments & Trade factor; endowment; comparative; theorem; heckscher-ohlin 3.31 15.86

5 Tariffs tariff; import 3.23 14.62 6 Sovereign Debt debt; default; sovereign; lending; borrowing 2.95 5.27 7 Exchange Rates parity; exchange; spot 2.89 15.49

8 International Investment

multinational; foreign; subsidiary; fdi; investment; direct; multinationals 2.77 21.31

9 International Trade Agreements multilateral; negotiation; agreement; wto; organization; member 2.69 5.33

Theme Keywords Eigenvalue % Cases 1 Lending borrower; loan; lender; lending; credit; bank 12.53 15.42 2 Risk Aversion utility; aversion 3.8 7.89 3 Firm Takeovers takeover; shareholder; tender 3.37 3.44

4 Estimation Techniques asymptotic; procedure; estimation; estimator 3.25 4.81

5 Trade trading; trader; price; rational 2.99 31.35 6 Health Insurance health; insurance; insurer; employee; coverage; plan 2.87 8.11

7 Labor & Employment employment 2.81 1.22

8 Taxation retain; tax; taxation; corporate; personal 2.69 9.25 9 Monetary Policy nominal; monetary; interst; rate; real 2.62 27.98

10 Investment Returns premium; return 2.6 21.09


1 Public Goods public; goods 13.08 22.17

2 Education school; student; score; achievement; parent; district; teacher; peer; causal; college 4.16 6.51

3 Retirement security; social; annuity; generation; bequest; retirement 3.72 14.24 4 Auctions auction; bidder; bid; procurement; highway; contract 3.45 3.32 5 Income Taxation mirrlees; optimal; utility 3.33 15.46

6 Rebates & Transitory Income rebate; permanent; transitory 3.08 1.83

7 Domestic & Foreign Market Instruments domestic; import; foreign 2.91 2.98

8 Corporate Taxation capital; corporate; tax; rate 2.79 42.37

9 Labor & Employment labor; supply, hour 2.73 12.20

Table 8 – Thematic Analysis JEL Category “G”

Table 7 – Thematic Analysis JEL Category “F”

Table 9 – Thematic Analysis JEL Category “H”

28


1 Health Insurance insurance; insurer 33.86 10.82

2 Estimation Techniques regressor; liml; estimator; coefficient; square; statistics 6.05 3.45

3 Education score; student; school; achievement 5.77 27.43 4 Donor Exchanges donor; transplant; kidney; donation; exchange; priority 5.48 1.57 5 China china; sex; female; girl; ratio 5.35 5.64 6 Newborns newborns; gram; threshold; discontinuity; side; diagnostic 4.94 4.23

Theme Keywords Eigenvalue % Cases 1 Racial Demographics black; urban; white 7.77 5.66 2 Education school; student; college; score; attend 3.1 7.78 3 Gender Issues woman; female; marry; man; marriage 2.96 11.52

4 Labor & Employment worker; job; firm; search; labor 2.75 47.81

5 Human Capital human; capital; investment; accumulation 2.62 14.39 6 Retirement security; retirement; pension; life; social 2.56 12.39

7 Estimation Techniques estimate; estimation; estimator 2.49 22.09

8 Unions bargaining; union; collective; strike; dispute 2.42 7.36 9 Children & Family child; parent; family; fertility; mother 2.31 12.42

10 Unemployment unemployment; duration; spell 2.24 11.52


1 Law Enforcement violation; origin 33.53 4.17

2 Estimation Techniques

lasso; iv; post-lasso; estimator; validity; modification; conditional; sample; selection; deal; widely; perform; endogenous; rely; imperfect 9.79 14.58

3 Incarceration release; prison; prisoner; offender; inmate; recidivism; month; certainty; parole 7.34 8.75

4 Voting & Elections intolerant; citizen; majority; voter; emerge; society; potentially; mistake; end; seek; auditing; punish; improve; enforce; vote 6.63 17.5

5 Energy Regulation diesel; fuel; substitute; elasticity; tax; fall; respect; program; fraction; regulatory; similar; federal 6.14 14.17

6 Racism race; racial; defendant; jury; trial; white; percentage; black 6.05 8.75

7 Game Theory communication; player; signal; shock; cut; fact; receive; cooperation; end; imperfect; repeat; long; subject 5.84 16.67

8 Bankruptcy scale; bankruptcy; file; avoid; entry; damage; asset; hazard 5.69 7.50 9 Internet internet; sex; access; consumption; victim; finding 5.41 10.00

10 Mergers & The Firm surplus; antitrust; consumer; merger; authority; commitment; competition 5.37 9.17

Table 11 – Thematic Analysis JEL Category “J”

Table 12 – Thematic Analysis JEL Category “K”

Table 10 – Thematic Analysis JEL Category “I”

29


1 Retail Sales retail; retailer 8.83 3.02 2 Transportation congestion; traffic; road; highway; airport; airline 2.89 2.68 3 Contracts contract 2.78 8.99 4 Firm Integration downstream; upstream; vertical; integration 2.73 3.81

5 Environmental Regulation

regulation; environmental; regulate; regulatory; air; pollution; regulator 2.53 7.34

6 Trade trade; export; domestic; foreign; country; international; exporter 2.48 12.69 7 Voting & Elections medium; vote; news; television; newspaper; political 2.44 3.93 8 Auctions bid; auction; bidder; lease 2.42 3.13

9 Labor & Employment labor 2.34 6.66

Theme Keywords Eigenvalue % Cases 1 Workplace personnel; workplace; friend; social; externality; works; piece 38.69 6.84

2 Advertising

contest; participant; win; attract; attention; nonpecuniary; restrict; existence; random; averse; advertisement; lack; enter; participation; frequently 7.27 15.97

3 Structural Decision-Making

Hierarchy; decentralize; decision-making; organizational; division; coordinate; dominate; authority; strategically; centralize; coordination; maker; dissent; credibility 7.08 10.27

4 Suppliers seller 6.71 4.94 5 Seniority & Status status; prior; award; unite 6.63 3.80

6 Bait & Switch & Seller Disclosures disclosure; sender; receiver; switch; bait 6.29 2.28

7 Taxation taxable; responsiveness; exercise; elasticity; rich; salary; short-run; respect; tax; timing; zero 6.24 10.65

8 Entrepreneurship entrepreneur 6.02 3.04


1 Gender Issues young; gi; bill; korean; veteran; college; man; female; attainment; sharp 41.76 10.69

2 GDP & Fertility

unprecedented; expectancy; figure; acquire; widespread; virtually; elite; gdp; fertility; illustrate; sustain; vast; traditional; simultaneously; majority 8.35 11.64

3 Colonialism import; interwar; stagnation; nation; asia; material; metropolitan; foreign; responsible; fuel; effort; japanese; farm; assume; colonial 7.83 12.89

4 Progress & Growth smith; remarkable; attention; james; progress; cite; considerable; famous 7.19 6.92

5 Voting & Elections voter; electoral; politician; vote; election; party; implement; radio; competition; decide; turnout; convergence; focus 6.99 11.32

6 Underdeveloped Countries

life; brazil; conjecture; ldc; analytical; correspond; product; persistent; sample 6.96 7.23

7 Monopoly Power monopoly; conduct; company 6.60 4.40

Table 13 – Thematic Analysis JEL Category “L”

Table 14 – Thematic Analysis JEL Category “M”

Table 15 – Thematic Analysis JEL Category “N”

30


1 Decentralized Planning Procedures

hurwicz; lange; malinvaud; arrow; iterative; scale; decentralize; mixture; command; category; procedure; partially; applicable; resemble; clear 10.15 13.54

2 Rationing Goods anticipate; inventory; priority; error; operating; thing; shortage; practice; occur; meet; soviet-type; norm; ration; highly; post 9.3 11.05

3 National Trade

non-economic; throughout; constrain; implicit; variety; strategic; export; constitute; national; opportunity; specific; import; distinguish; entail; turn 8.93 10.22

4 Optimal Growth & Dynamics

stationary; stochastic; gale; generalization; infinite; optimality; optimal 8.57 16.02

5 Gender Issues miss; girl; birth; widely; africa; gender; female; woman; age; proportion; comparison; composition; india; comparable; unite 8.19 7.73

6 Industrialization

divergence; exception; late; structural; proportion; closely; justification; comparable; difficult; industrialization; emphasize; agriculture; respect; union; recently 7.93 12.98

7 Input-Output Models requirement; sectoral; terminal; excess; input-output; full; revision; capacity; series; create 7.44 9.39

8 Equity & Consumption consume; save; equity; retain; corporate 7.35 3.87


1 Natural Resource Exhaustion dasgupta; certainty; exhaustion; balance; begin; date; utilization 7.2 5.90

2 Sulfur Emissions sulfur; allowance; dioxide; saving; abatement; emission; trading; amendment; air 6.26 10.42

3 Forestry Resources bidder; auction; timber; bid; forest 5.71 3.82 4 Production Functions cobb-doublas; substitution 5.55 3.13

5 Air Quality tsps; nonattainment; housing; air; robust; specification; particulate; county; status 5.35 9.20

6 Agriculture landlord; tenant; tenancy; rent; contract; sharecropping 5.19 7.64


1 Education randomize; school; score; deviation; experiment; child 18.86 9.09

2 Progress & Growth harrod; hick; sato; neutral; neutrality; progress; david; technical; function 4.7 14.39

3 Lending default; borrower; lender; repayment; loan 4.23 2.65 4 Innovation imitation; schumpeter; innovative; perfect; innovation; monopoly 4.12 14.96 5 Voting & Elections electoral; election; political; politician; voter 3.49 5.02

6 Labor & Employment skill-biased; skill; skilled; wage; worker; unskilled; premium 3.36 13.73

7 Gender Issues marriage; woman; man; female; sex 3.34 4.17 8 Insurance hazard; moral; insurance 3.18 2.46

Table 18 – Thematic Analysis JEL Category “Q”

Table 17 – Thematic Analysis JEL Category “P”

Table 16 – Thematic Analysis JEL Category “O”

31


1 Germany east; west; german; reunification; germany 38.52 2.26

2 Estimation Techniques

lasso; post-lasso; iv; conditional; expectation; procedure; validity; domain; sum; modification; normal; estimator; error; inference; perform 8.35 7.61

3 Game Theory subgame; hotelling; game; simultaneously; solution; nonlinear; charge; pure 7.46 6.58

4 Transportation transit; mode; trip; fare; auto; peak; automobile; travel; passenger; congestion; bay 6.73 6.17

5 Education school; nonwhite; peer; student; racial; socioeconomic; score; district 6.57 5.97

6 Gender Issues youth; male; man; poverty; young; reservation; participation; adult 6.29 5.35 7 Traffic queue; delay; capacity; dominant; traffic; maximum 6.15 5.14

Table 19 – Thematic Analysis JEL Category “R”

32

Figure 2 – Network Analysis of JEL Codes, 1969-2014

33

C D E F G H I J K L M N O P Q R C 12.9 D 5.78 9.62 E 3.51 4.09 7.3 F 0.40 0.93 2.13 4.47 G 1.02 2.33 1.81 0.54 2.34 H 0.71 3.42 1.90 0.61 0.54 1.82 I 0.31 1.02 0.18 0.06 0.24 0.63 0.65 J 1.63 2.86 2.69 0.61 0.37 1.26 1.78 5.72 K 0.09 0.45 0.02 0.05 0.13 0.12 0.12 0.32 0.37 L 1.01 3.77 2.29 0.88 1.04 0.52 0.26 1.13 0.30 1.73 M 0.10 0.70 0.14 0.05 0.26 0.02 0.01 0.44 0.02 0.59 0.14 N 0.03 0.38 0.53 0.24 0.25 0.12 0.15 0.46 0.05 0.17 0.01 0.17 O 0.30 1.65 1.41 1.10 0.58 0.31 0.59 1.39 0.17 1.31 0.10 0.29 0.73 P 0.30 0.43 0.54 0.17 0.09 0.20 0.05 0.20 0.03 0.17 0.01 0.10 0.28 0.55 Q 0.21 0.81 0.41 0.30 0.17 0.24 0.09 0.25 0.04 0.57 0.02 0.18 0.52 0.11 1.07 R 0.19 0.62 0.38 0.12 0.15 0.44 0.18 0.69 0.09 0.43 0.05 0.13 0.25 0.05 0.14 0.65

Table 20 – Network Analysis of JEL Categories (%)

34

Figure 3 – Network Analysis of Top Themes, 1969-2014

Law Enforcement

Retail Sales

Firm Integration

Entrepreneurship

Bait & Switch & Seller Disclosures SuppliersForestry

Resources

Firm Takeovers

Transportation

Insurance

Racism

Racial Demographics

Rebates & Transitory Income

Bequests

Germany

Children & Family

China

Newborns

Environmental Regulation

35

Appendix A: JEL Code Matching Strategy

Subject post-1990 JEL subject code

pre-1990 JEL subject code

General Economics & Teaching A 011, 012, 110, 115 History of Economic Thought, Methodology, & Heterodox Approaches B 031, 036, 316-318, 329, 360

Mathematical & Quantitative Methods C

021, 026, 210-215, 220, 222, 229, 260-262, 2110, 2112-2120, 2130, 2132-2135, 2140, 2150, 2200, 2220, 2270, 2290

Microeconomics D

020, 022, 024, 025, 114, 200, 224, 225, 227, 228, 240, 242-244, 250-252, 511-513, 522, 600, 921, 1140, 2240, 2280, 5110, 5120, 5130, 5131, 5220, 9210-9213

Macroeconomics & Monetary Economics E

023, 112, 120-124, 130-134, 221, 223, 226, 230-235, 239, 311, 1120, 1210, 1211, 1213-1217, 1221, 1223, 1224, 1228, 1230, 1243, 1244, 1310, 1312, 1313, 1320, 1322-1324, 1330-1332, 1340, 1342, 2210, 2212, 2213, 2230, 2260, 3110, 3112, 3116

International Economics F

111, 400, 411, 420-423, 431-433, 441-443, 1110, 1112, 4000, 4110, 4112-4114, 4200, 4210, 4220, 4230, 4232, 4233, 4310, 4312-4314, 4320, 4330, 4410-4412, 4420, 4430

Financial Economics G 310, 312-315, 521, 3120, 3130-3132, 3140, 3150-3153, 5200, 5210

Public Economics H

320-325, 641, 915, 3200, 3210, 3212, 3216, 3220, 3221, 3226, 3228, 3230, 3240-3243, 3250, 6410, 9150

Health, Education, and Welfare I 911, 913, 914, 9100, 9110, 9130, 9140, 9300

Labor and Demographic Economics J

811-813, 820-826, 831-833, 841, 850, 851, 912, 917, 918, 8110, 8120, 8130-8135, 8210, 8220-8226, 8230, 8240-8243, 8250, 8260, 8300, 8310, 8320-8322, 8330-8332, 8410, 8510, 9120, 9170, 9180

Law and Economics K 916, 9160

Industrial Organization L

514, 610-616, 619, 620, 631-636, 5140, 6110, 6120, 6130, 6140, 6150, 6160, 6190, 6300, 6310, 6312-6318, 6320, 6322, 6323, 6333, 6340, 6352-6358, 6360

Business Administration & Business Economics; Marketing; Accounting M 531, 541, 5310, 5410

Economic History N 041-048, 410-412, 420, 430, 440, 450-452, 463, 470, 473

Economic Development, Technological Change, & Growth O 621, 718, 6210-6212, 7180

Economic Systems P 027, 050-053, 113, 270-272, 500, 510, 520, 530, 1113, 1114, 1130, 1132, 1136

Agricultural & Natural Resource Economics; Environmental & Ecological Economics Q

710-717, 721-723, 7100, 7110, 7120, 7130, 7140, 7150, 7151, 7160, 7170-7172, 7210, 7211, 7220, 7230

Regional, Real Estate, and Transportation Economics R 731, 931-933, 941, 2250, 7310, 9310, 9320, 9330, 9410-9413

36

References

1. Antweiler, Werner and Murray Z. Frank. 2004. “Is All That Talk Just Noise? The Information Content of Internet Stock Message Boards.” The Journal of Finance 59(3):1259-1294.

2. Baker, Scott R., Nicholas Bloom, Brandice Canes-Wrone, Steven J. Davis, and Jonathan

Rodden. 2014. “Why Has US Policy Uncertainty Risen Since 1960?” American Economic Review: Papers & Proceedings 104(5):56-60.

3. Card, David and Stefano DellaVigna. 2013. “Nine Facts About Top Journals in

Economics.” Journal of Economic Literature 51(1):144-161.

4. Cherrier, Beatrice. 2015. “Classifying Economics: A History of the JEL Codes.” Working Paper.

5. Cropper, Maureen L. 2000. “Has Economic Research Answered the Needs of

Environmental Policy?” Journal of Environmental Economics and Management 39:328-350.

6. Durden, Garey C., and Larry V. Ellis. 1993. “A Method for Identifying the Most Influential Articles in an Academic Discipline.” Atlantic Economic Journal 21(4):1-10.

7. Engemann, Kristie M., and Howard J. Wall. 2009. “A Journal Ranking for the Ambitious Economist.” Federal Reserve Bank of St. Louis Review 91(3):127-139.

8. Gentzkow, Matthew and Jesse M. Shapiro. 2010. “What Drives Media Slant? Evidence

from U.S. Daily Newspapers.” Econometrica 78(1):35-71. 9. Goyal, Sanjeev, Marco J. van der Leij, and Jose L. Moraga-Gonzalez. 2006. “Economics:

An Emerging Small World.” Journal of Political Economy 114(2):403-412.

10. Grijalva, Therese, and Clifford Nowell. 2014. “What Interests Environmental and Resource Economists? A Comparison of Research Output in Agricultural Economics versus Environmental Economics.” Agricultural and Resource Economics Review 43(2):209-226.

11. Hamermesh, Daniel S. 2014. “Age, Cohort, and Co-Authorship.” Working Paper.

12. Hamermesh, Daniel S. 2013. “Six Decades of Top Economics Publishing: Who and How?”

Journal of Economic Literature 51(1):162-172.

13. Jacomy, Mathieu, Tommaso Venturini, Sebastien Heymann, and Mathieu Bastian. 2014. “ForceAtlas2, a Continuous Graph Layout Algorithm for Handy Network Visualization Designed for the Gephi Software.” PLoS ONE 9(6): e98679.

37

14. Kalaitzidakis, Pantelis, Theofanis P. Mamuneas, and Thanasis Stengos. 2001. “Rankings of Academic Journals and Institutions in Economics.” Working Paper.

15. Kelly, Michael A. and Stephen Bruestle. 2011. “Trend of Subjects Published in Economics

Journals 1969-2007.” Economic Inquiry 49(3):658-673.

16. Kim, E. Han, Adair Morse, and Luigi Zingales. 2006. “What Has Mattered to Economics Since 1970.” The Journal of Economic Perspectives 20(4):189-202.

17. Kolstad, Charles. 2010. Environmental Economics, 2nd Edition. Oxford University Press.

18. Kosnik, L-R. 2015a. “In Tandem or Out of Sync? Academic Economics Research and Public Policy Measures.” Contemporary Economic Policy In Press.

19. Kosnik, L-R. 2015b. “What Have Economists Been Doing for the Last 50 Years? A Text

Analysis of Published Academic Research from 1960-2010.” Economics 9:1-38.

20. Kosnik, L-R. 2014. “Determinants of Contract Completeness: An Environmental Regulatory Application.” International Review of Law and Economics 37:198-208.

21. Laband, David N., and Robert D. Tollison. 2000. “Intellectual Collaboration.” Journal of

Political Economy 108(3):632-662.

22. Laband, David N., Robert D. Tollison, and Gokhan Karahan. 2002. “Quality Control in Economics.” Kyklos 55:315-334.

23. Perman, Roger, Yue Ma, Michael Common, Daivd Maddison, and James Mcgilvray. 2012. Natural Resource and Environmental Economics, 4th Edition. Prentice Hall.

24. Rath, Katharina, and Klaus Wohlrabe. 2015. “Recent Trends in Co-Authorship in

Economics: Evidence from RePEc.” CESifo Working Paper No. 5492.

25. Rosell, Carlos, and Ajay Agrawal. 2009. “Have University Knowledge Flows Narrowed? Evidence from Patent Data.” Research Policy 38:1-13.

26. Rummel, R.J. Applied Factor Analysis. Evanston, ILL: Northwestern University Press.

27. Tetlock, Paul C. 2007. “Giving Content to Investor Sentiment: The Role of Media in the

Stock Market.” Journal of Finance 62(3):1139-1168.

28. Tietenberg, Tom, and Lynne Lewis. 2012. Environmental & Natural Resource Economics, 9th Edition. Prentice Hall.

29. Weterings, Anet, and Rokerik Ponds. 2009. “Do Regional and Non-Regional Knowledge

Flows Differ? An Empirical Study on Clustered Firms in the Dutch Life Sciences and Computing Services Industry.” Industry and Innovation 16(1):11-31.

38

30. Whaples, Robert. 1991. “A Quantitative History of the Journal of Economic History and the

Cliometric Revolution.” The Journal of Economic History 51(2):289-301.

economics working papers

Documents