maryann feldman, ph.d. - the longer term effects of federal ......& daniel smith...
TRANSCRIPT
The longer term effects of federal subsidies on firmsurvival: evidence from the advanced technologyprogram
Daniel Smith1,2 • Maryann Feldman2 • Gary Anderson3
� Springer Science+Business Media, LLC 2017
Abstract The goal of this paper is to conduct a survival analysis to determine the causal
impact of federal R&D subsidies on firms’ long-term survival. The data are small firms
which applied to the Advanced Technology Program (ATP) in 1998 and 2000. The ATP’s
focus was on ensuring that early stage, high-risk research was eventually commercialized
successfully and resulted in broad economic benefits for society overall. This paper
therefore explores whether the knowledge and benefits the ATP initially provided to a firm
allowed it to more successfully transition future research projects from development and
testing to commercialization. This paper utilizes a variant of the Heckman (Econometrica
47(1):153–161, 1979) research design to control for inherent pre-award differences
between awarded and non-awarded firms. By using administrative data on reviewer scores,
this analysis shows that the impact of ATP on small firm survival is robust to sample
selection. This paper’s findings suggest that recei ving an ATP award can have a significant
and positive causal effect on firm survival.
Keywords Federal � R&D � Subsidies � Firm � Survival � Innovation
JEL Classification H2 � O3
& Daniel [email protected]
Maryann [email protected]
Gary [email protected]
1 North Carolina Department of Commerce, Raleigh, NC, USA
2 University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
3 National Science Foundation, Alexandria, VA, USA
123
J Technol TransfDOI 10.1007/s10961-017-9633-5
1 Introduction
The basic justification for government subsidies of R&D performed by the private sector is
that there are positive externalities associated with knowledge creation. Consequently, the
private sector will underinvest in R&D (Klette et al. 2000). R&D subsidies potentially
increase firms’ longer term chances of survival by both directly encouraging firms to
innovate and by encouraging firms to collaborate more with others, thereby increasing their
ability to quickly respond to new technological developments in their industries (David
et al. 2000; Cefis and Marsili 2005; Saxenian 1994). R&D subsidies to private firms (as
opposed to universities, non-profit organizations, or government agencies) are important
because economic growth requires a well-balanced mixture of both R&D and business
development (Leyden and Link 2013). In addition, public R&D subsidies can serve as a
signal to private financiers that a given research project is worth investing in, especially
when the company is relatively new (Takalmo and Tanayama 2010; Colombo et al. 2010).
Despite these theoretical benefits, federal subsidization of privately performed R&D
remains a controversial issue. In recent years, federal programs such as the Advanced
Technology Program (ATP) and the Technology Innovation program each administered by
the National Institute of Standards and Technology (NIST) have been eliminated by
Congress (NIST 1998, 2007; NIST n.d.). However, other federal programs which subsidize
the performance of R&D by private entities (such as the SBIR program, the STTR pro-
gram, and the ARPA-E program) remain in existence (National Institutes of Health 2016;
U.S. Department of Energy n.d.; U.S. Department of Energy 2016). The varied fates of
these federal programs demonstrates that the role and impact of federal support for pri-
vately performed R&D is remains unsettled, at least in the United States.
While some of these programs have had aspects of their performance analyzed, there
has been relatively little empirical research on the impact that these programs have had on
long term firm survival, and the research that has been done has focused primarily on the
short term impact of federal R&D support. For instance, Czarnitziki et al.’s (2007) analysis
of the economic effects of federal and provincial Canadian R&D subsidies considers a
3-year impact horizon. Likewise, Einio (2014) examines the effects of R&D subsidies on
firms’ growth up to 3 years after a firm received a subsidy. This is in spite of the fact that
Mansfield (1995) finds that it generally takes 7 years for a firm to move from basic
research to commercialization. In addition, empirical research on this subject has been
hampered due to qualitative differences between firms selected to receive a particular
subsidy and firms that were not, resulting in selection bias (Wallsten 2000; Murnane and
Willett 2011).
This paper examines the longer term impact of firm participation in the ATP. The ATP
was intended to promote high risk research with large potential societal and economic
benefits (National Research Council 2001). The ATP was also specifically intended to
increase the competitiveness of U.S. firms (National Research Council 2001). The ATP did
this by funding particular high-risk high-reward research projects. ATP program managers
remained actively engaged in project management to ensure that firm’s research remained
true to project goals and program criteria. Since a firm’s survival is intertwined with its
innovative output and competitiveness, the ATP both directly and indirectly impacts the
outcomes examined here (Chesbrough 2003).
This paper studies the survival through 2014 of small firms that applied to the ATP in
the 1998 and 2000. This is a much longer time horizon than most studies which examine
the effects of public R&D subsidies received by private corporations (Czarnitziki et al.
2007; Einio 2014). Importantly, this analysis includes all small firm applicants regardless
D. Smith et al.
123
of whether the applicant was actually funded by ATP. Additionally, this paper exploits
ATP proposal reviewer scores to correct for potential selection bias using a variant of the
research method pioneered by Heckman (1979). Reviewer scores provide a rare and
valuable opportunity to control for inherent pre-existing differences between firms that
received an ATP award and those that did not. Failure to control for such differences is a
common issue in the existing literature focused on evaluating the effects of government
subsidies (Wallsten 2000).
This paper is organized as follows. Section 2 gives an overview of the existing literature
on this subject and the ways this paper is expected to contribute to it. Section 3 provides
both an overview of the ATP program and specific details relating to the 1998 and 2000
ATP competitions. Section 4 discusses some of the broad considerations relating to the
research design of this paper, while Sect. 5 describes the specific econometric methods
employed. Section 6 discusses the empirical data and variables being used. Finally, Sect. 7
details the empirical results and Sect. 8 concludes.
2 Literature review—R&D subsidies and firm outcomes
While any subsidy which is explicitly designed to promote firm growth or employment
tends to be focused on the short term, the primary firm level outcome examined by this
paper will be longer term firm survival (Feldman 1994). Theoretically, to the extent that
R&D subsidies increase firms’ innovative output and absorptive capacity, such subsidies
should also have a positive impact on firm survival. Unfortunately, there has been very
little empirical research directly examining these impacts (Bercovitz and Feldman 2007;
Deeds and Decarolis 1999; Chesbrough 2003). Because firm survival is generally closely
related to shorter term impacts of R&D subsidies such as innovative outputs, increased
interfirm collaboration, and firm growth (Musso and Schiavo 2008; Cefis and Marsili
2005), this literature is also discussed in this section.
2.1 Effects of federal R&D subsidies on firms’ innovative output
There are a multitude of theoretical reasons why federal R&D subsidies would be expected
to increase firms’ chances of survival. To begin with, if federal subsidies do boost the
innovative output of firms, then they would also increase the overall lifespan of those firms.
Both Cefis and Marsili’s (2005) survival analysis of Dutch firms and Esteve-Perez and
Manez-Castillejo’s (2008) survival analysis of Spanish manufacturing firms find that
indicators of innovative output increase a firm’s chances of survival. The existing empirical
work done on whether R&D subsidies boost the actual innovative output of a firm gen-
erally suggests that they do. There has been a great deal of empirical research performed on
this issue at various levels. Table 1 provides some examples of this research. This work has
shown that R&D subsidies can positively impact firm patenting and the introduction of
new commercial products.
2.2 Effects of federal R&D subsidies on firms’ ability to collaborateeffectively
However, the benefits a firm gains by receiving a federal subsidy extend beyond increasing
its innovative output. By increasing the amount of internal R&D a firm conducts, subsidies
The longer term effects of federal subsidies on firm survival…
123
would also cause firms to increase their absorptive capacities (David et al. 2000; Bercovitz
and Feldman 2007; Kumar et al. 1999; Aldieri and Cincera 2009). Since absorptive
capacity refers to the ability of firms to make profitable use of external knowledge, this
would make firms more likely to collaborate with other organizations in the future by
raising the potential returns of such collaborations (Bercovitz and Feldman 2007; Dyer and
Singh 1998). In addition, subsidies which explicitly encourage interorganizational col-
laboration (as the ATP does) would be likely to further encourage firms to collaborate with
different organizations by initially assisting firms in establishing social norms which would
make future collaborations less costly (National Research Council 2001; Dyer 1997;
Schrank and Whitford 2011).
Increased interorganizational collaboration would, in turn, be expected to allow com-
panies to more rapidly become aware of and adapt to new technologies, thus improving
their chances of survival and growth (Saxenian 1994). Saxenian (1994) argues that Silicon
Valley outperformed the IT cluster along Route 128 in Massachusetts primarily because
heightened formal and informal collaboration between the firms there allowed the firms
there to better adjust to changes in information technology. Even large, well-run firms
which have invested heavily in internal R&D can be hurt or even fail altogether because of
their inability to adjust quickly enough to new technology (Chesbrough 2003; Christensen
1997). Private firms appear to be becoming more aware of this, as the propensity of firms to
collaborate with organizations such as universities has been increasing over the past few
decades (Hall et al. 2003; Cunningham and Link 2015; Link and Scott 2005; Audretsch
et al. 2012; Link 2015, Loof and Brostrom 2008; Caloghirou et al. 2001; Hanel and St
Pierre 2006; Sachwald 2008). Thus, R&D subsidies would be expected to encourage firms
to collaborate with other organizations, which would improve their chances of survival.
However, there is some debate in the literature about whether federal or regional
governments are better able to promote interorganizational collaboration. Knoben and
Table 1 Empirical literature on R&D subsidies and innovative output
Author(s) Year Geographicarea(s) studied
Unit ofanalysis
Main Findings
Johnstoneet al.
2010 International Country R&D subsidies have significant and positive effects onpatent counts for more costly technologies
Azoulayet al.
2014 United States Researcharea
NIH funding increases the number of patents directlysupported by the NIH, the number of patents citingpatents directly supported by the NIH, and theoverall number of patents in the same research areaas the patents funded by the NIH (Azoulay et al.2014)
Czarnitzikiet al.
2007 Finland;Germany
Firm R&D subsidization had a positive effect on thepatenting activity of Finnish firms but not WestGerman firms
NishimuraandOkamura
2010 Japan Firm R&D subsidization in general had no significant effectbut R&D subsidization in which a firm collaboratedwith a university had a positive and significant effecton patenting activity
Czarnitzikiet al.
2011 Canada Firm R&D subsidization does in fact boost both the numberof new commercial products a firm sells and thenovelty of a firm’s innovations
D. Smith et al.
123
Oerlemans (2006) conduct a literature review of the research on the relationship between
various types of proximity (including geographical proximity) and interorganizational
collaboration, and find that geographic proximity does seem to play a definite role in
determining the success of an interorganizational partnership.
The importance of geographic proximity to successful partnerships suggests that
regional governments might be better suited to enact policies aimed at promoting
interorganizational collaboration than a federal agency. Oughton et al. (2002) argue that
regional governments will be more effective than federal governments at promoting col-
laboration since they are better able to understand and change the institutional environment
of a region. This is supported by the fact that state governments within the United States
are focusing increasingly on developing innovation policies that reflect the specific com-
petitive advantages of the region they are in (Hall and Link 2015). However, it is also true
that regional governments generally do not have access to the same level of resources as
federal governments (Morgan 1997).
The fact that the ATP is a purely federal program without any regional component
makes it ideal for evaluating whether a federal agency can effectively spur interorgani-
zational collaboration and firm survival (National Research Council 2001). This paper thus
provides an indirect contribution to the debate over whether federal governments should be
involved in such efforts. Although it does not directly examine the impact of a 1998 ATP
award on a firm’s number of annual interorganizational collaborations (since the quality of
the collaborations and characteristics of the collaborating partners are generally more
important than the raw number of collaborations a firm engages in), a positive finding in
the survival analysis would provide circumstantial evidence that federal governments
could effectively stimulate beneficial interorganizational collaborations (Stuart 2000;
Baum et al. 2000; Sampson 2007).
2.3 Effects of federal R&D subsidies on firms’ growth
There is very little direct empirical examination of the effect of R&D subsidies on firm
survival. However, R&D subsidies can boost firm productivity both by creating new
innovations that a firm can benefit from and making the firm better able to take advantage
of existing knowledge generated by actors other than the firm of interest (Griffith et al.
2004). Higher firm productivity is, in turn, usually associated with higher firm growth and
thus firm survival (Wagner 2002; Musso and Schiavo 2008). Thus, empirical analyses of
the effects R&D subsidies have on firm growth are germane to the discussion of their
effects on firm survival.
There has been a fair amount of empirical work examining the relationship between
R&D subsidies and firm growth drawing data from a variety of different countries. While
most studies find evidence of a positive and significant relationship between R&D sub-
sidies and firm growth, Wallsten (2000) finds that, once selection bias has been controlled
for, the R&D subsidies they examine have no significant effect on firms’ growth. This
appears to contradict the findings of Bozeman et al. (2008), who find that a lack of early
stage capital is identified by nanotechnology firms in the state of North Carolina in the
United States of America as being one of the primary obstacles to their growth and
competitiveness, as well as Hardin and Link’s (2015) research suggesting that organiza-
tions which attract funding from multiple sources generally create more jobs. However,
Wallsten’s (2000) findings are even more clearly contradicted by those of Einio (2014),
who uses an instrumental variables approach to explicitly control for selection bias and
nevertheless finds that R&D subsidies do have a positive and significant effect on firm
The longer term effects of federal subsidies on firm survival…
123
growth. However, Einio (2014) draws data from Finnish firms while Wallsten (2000)
examines firms in the United States in their analysis. Thus, one of the contributions of this
paper will be to find out whether or not Einio’s (2014) findings also hold true in the United
States. Additionally, Einio (2014) exploits the fact that the Finnish government allocates
R&D subsidies based partially on the population density of a given region to run their
instrumental variables (IV) regression, while Wallsten (2000) bases his IV regression on
the differing budgets that government agencies are required to set aside for SBIR funding.
Also, while Einio’s (2014) study includes all firms which applied for an R&D subsidy,
Wallsten’s (2000) study only uses firms which applied for an SBIR grant with the National
Aeronautics and Space Administration or the Department of Defense as its primary control
group. Table 2 summarizes some of the empirical literature on this issue. Research
demonstrates that under certain conditions R&D subsidies can have a positive impact on
employment, growth, and sales.
2.4 Effects of federal R&D subsidies on firms’ survival prospects
While there has been a fair amount of research relating to factors that could encourage firm
survival, there have been relatively few survival analyses of firms (Cefis and Marsili 2005).
The studies that do exist provide some theoretical reasons for why R&D subsidies might be
Table 2 Empirical literature on R&D subsidies and firm growth
Author(s) Year Geographicarea(s) examined instudy
Main Findings
Lerner 2000 United States There is a positive correlation between receiving an R&Dsubsidy and higher employment at a firm
Wallsten 2000 United States Once selection bias is controlled for there is no significantrelationship between R&D subsidization and firmemployment
Hussinger 2008 Germany Public R&D subsidies have (statistically) the same positiveand significant effect on firm growth as private R&Dspending
Hall andMaffioli
2008 Argentina, Brazil,Chile, and Panama
Public R&D subsidies have positive effects on firm growth
Czarnitzikiet al.
2011 Canada R&D subsidization boosts the sales and generalperformance of manufacturing firms
BecchettiandTrovato
2002 Italy Subsidies of any sort will tend to boost the growth of smalland medium sized Italian firms
HytinnenandToivanen
2005 Finland Government R&D subsidies disproportionately boost thegrowth of firms in industries that tend to be moredependent on some form of external finance (Hytinnenand Toivanen 2005)
Audretschet al.
2002 United States Completing Phase II of the SBIR program does have apositive and significant effect on a firm’s sales of thetechnology it developed through the SBIR program
Einio 2014 Finland After controlling for selection bias, R&D subsidies are stillfound to have positive and significant effects on firms’sales and employment
D. Smith et al.
123
expected to increase recipient firms’ chances of survival. For instance, Cefis and Marsili
(2005) find that firms’ innovative outputs do have a positive and significant effect on their
survival chances, while Esteve-Perez and Manez-Castillejo (2008) find that firms which
perform their own R&D are likely to survive longer. Table 3 summarizes many of the
studies on firm survival.
Significantly, none of the survival analyses which have been conducted thus far have
directly examined the role governments can play in encouraging firm survival, despite calls
for the governments of developed nations to implement such policies in the literature
(Esteve-Perez and Manez-Castillejo 2008; Holmes et al. 2010). This work therefore fills an
important gap in the literature by evaluating what role federal policy can play in increasing
firms’ survival chances.
3 Description of the ATP and the 1998 and 2000 ATP competitions
The ATP was intended to promote high risk research with large potential social and
economic benefits (National Research Council 2001). The ATP was also specifically
intended to increase the competitiveness of U.S. firms (National Research Council 2001).
The ATP did this by funding particular high-risk high-reward research projects. The
program was created as a response to concerns within the U.S. of a loss of global com-
petitiveness on the part of U.S. firms in the face of increasing competitive pressures from
Asian businesses (National Research Council 2001). The overarching goal of the ATP was
to fund high risk research with large potential social and economic benefits (National
Research Council 2001). Additionally, the ATP was intended to foster greater collabora-
tion between firms (National Research Council 2001). Both individual U.S. owned for-
profit businesses and joint ventures between at least two for-profit businesses are eligible to
apply for the ATP (National Institute of Standards and Technology 2004).
It is important to note that ATP grants were not general purpose R&D subsidies but
were instead intended to fund specific projects that met selective funding criteria. To
ensure that the program targeted high-risk high-reward research projects, the program
established funding criteria related to both scientific and technological merit as well as the
Table 3 Empirical literature on R&D subsidies and firm survival
Author(s) Year Geographicarea(s) examined instudy
Main Findings
Klepper 2002 United States Firms which enter industries earlier and thus have moreexperience are more likely to survive
Cefis and Marsili 2005 The Netherlands A firm’s innovative outputs do increase that firm’schances of survival
Musso andSchiavo
2008 France Financial constraints have a negative and significantimpact on a firm’s chances of survival
Esteve-Perez andManez-Castillejo
2008 Spain Firms which conduct their own R&D are likely tosurvive longer than firms which outsource their R&Dactivities
Holmes et al. 2010 United Kingdom Small and medium enterprises are most likely to fail inthe first 8 or 9 years after their establishment
The longer term effects of federal subsidies on firm survival…
123
potential for broad-based economic benefits (National Institute of Standards and Tech-
nology 1998). Table 4 presents the three detailed criteria in each of these two broad
categories (Feldman and Kelley 2006; National Institute of Standards and Technology
2004). Throughout the review and award process, each criterion received equal weight.
Notably, there are significant differences between these selection criteria and typical
business investment criteria. In contrast to standard business investment criteria, projects
that had a high degree of technological innovation as well as high technical risk, given a
credible technical plan to overcome this risk, were viewed favorably by the ATP (National
Institute of Standards and Technology 1998). Similarly, the criteria favored projects that
had significant economic benefits beyond those captured by the proposing firm itself
(National Institute of Standards and Technology 1998). Finally, the criteria rewarded
particular projects that did not have access to capital from external or internal sources
(National Institute of Standards and Technology 1998). Together, these evaluation criteria
are aimed at ensuring that the ATP invested in innovative, high risk research that the
private sector would not have funded on its own yet had the potential to generate large
social and economic benefits. Table 4 provides a summary of the assessment criteria for
ATP projects.
As the first part of the ATP review process, all proposals were reviewed by one or more
independent experts (National Institute of Standards and Technology 1998). These
reviewers assigned equally weighted scores ranging from 0 to 10 for each of the criteria in
Table 4 (Feldman and Kelley 2006; National Institute of Standards and Technology 2004).
Reviewers were instructed to give high scores to proposals that demonstrated a high degree
of technical innovation, had a high degree of technical risk alongside a credible plan to
overcome that risk, and a quality R&D plan (National Institute of Standards and Tech-
nology 1998). Proposals with significant benefits beyond those to the proposing firm, a
demonstrated lack of internal and external funding and a demonstration of a pathway to
market were scored highly (National Institute of Standards and Technology 1998). NIST
Table 4 Assessment criteria for the ATP
Criteria Description
Scientific and technological merit
1. Innovation inTechnology
Projects must display a high degree of innovation. Innovation may be relatedto the objectives of your research or the approach to achieving theseobjectives
2. High Technical Risk andFeasibility
Projects must be high risk—the mission of ATP is to overcome high-risktechnical barriers—and projects must be credible with respect to technicalapproach
3. Quality of R&D Plan Projects must have a detailed technical plan over the life of the project. Initialtime periods must have more details. Decision points and alternativesshould be discussed in the context of R&D strategy
Potential for broad-based economic benefits
4. National EconomicBenefit
Projects must demonstrate significant economic benefits to the Nation
5. Need for ATP funding Project must need public funds. Applicants must demonstrate that full privatefunding is not available that the project does not meet company criteria orprofile for internal funds
6. Pathway to EconomicBenefit
Proposal must demonstrate how technology, if successful, will be brought tomarket or enter use
D. Smith et al.
123
assembled expert panels that considered these reviews alongside other materials in order to
make recommendations to the ATP Selecting Official who ultimately determined which
projects would be funded (Feldman and Kelley 2006; NIST 2004).
Beyond the criteria and review process, NIST assigned technical personnel to actively
manage and monitor each funded ATP project over the duration of the grant (National
Institute of Standards and Technology 1998). These personnel had significant technical and
research experience in areas directly related to the particular funded project. ATP project
managers remained actively engaged in project management to ensure that a firms’
research remained true to project goals and program criteria (National Institute of Stan-
dards and Technology 1998).
Both the 1998 and 2000 ATP competitions consisted of a general competition for all
technologies and several smaller competitions for specific technology areas (Balutis and
Lambis 2001). There were 502 total project proposals sent in for the 1998 competition
which involved a total of 822 organizations (Feldman and Kelley 2003). 741 of these
organizations were for-profit firms (Feldman and Kelley 2003). For the 2000 ATP com-
petition, a total of 417 proposals involving 555 different organizations were submitted
(National Institute of Standards and Technology 2005). 536 of these organizations were
for-profit firms.
A total of 79 ATP awards were given out in the 1998 competition, while only 58 awards
were given out in the 2000 competition (National Institute of Standards and Technology
n.d.; National Institute of Standards and Technology 2005). The funded projects from the
1998 competition involved 171 organizations, while the funded projects from the 2000
competition only involved 85 organizations (National Institute of Standards and Tech-
nology 2005). Out of these organizations 151 were for-profit firms in the 1998 competition
while 74 were for-profit firms in the 2000 competition (National Institute of Standards and
Technology 2005). The dollar value of the average ATP award given out in the 1998
competition was $3,028,714, while the dollar value of the average received 2000 ATP
award was $2,532,493 (National Institute of Standards and Technology n.d.; National
Institute of Standards and Technology 2005). This information is summarized below in
Table 5 Information Concerning the 1998 and 2000 ATP Competitions
Measure Data Source
Number of TotalApplications
919 Feldman and Kelley (2003), National Institute of Standards andTechnology (2005)
Number of TotalApplicants
1377 Feldman and Kelley (2003), National Institute of Standards andTechnology (2005)
Number of Total For-Profit Applicants
1277 Feldman and Kelley (2003), National Institute of Standards andTechnology Administrative Records
Number of FundedProjects
137 National Institute of Standards and Technology n.d., NationalInstitute of Standards and Technology (2005)
Number of Awardees 256 National Institute of Standards and Technology administrativerecords, National Institute of Standards and Technology (2005)
Number of For-ProfitAwardees
225 National Institute of Standards and Technology AdministrativeRecords
Average Amount ofAward in 1998
$3,028,714.37 National Institute of Standards and Technology AdministrativeRecords
Average Amount ofAward in 2000
$2,532,493 National Institute of Standards and Technology AdministrativeRecords
The longer term effects of federal subsidies on firm survival…
123
Table 5. An appendix containing a case study to demonstrate the nature of ATP-funded
research is omitted for the sake of brevity but available upon request.
4 Research design considerations
This section discusses the broad empirical strategy used to isolate the causal effect of
receiving an ATP award on the performance of awardee firms. The treatment group
consists of firms which applied for and received an ATP award, while the control group
consists of firms which applied for but did not receive an ATP award in any year. For this
reason, the basic research design employed by this paper is a posttest-only design that
includes a control group (Shadish et al. 2002). Because the assignment of ATP awards is
non-random, this should be considered a quasi-experimental research design (Murnane and
Willett 2011, p. 31).
The primary methodological issue with this type of research design is that there can be
significant differences in the treatment and control groups prior to treatment (Shadish et al.
2002). To the extent that these differences are correlated with the outcome measure of
interest, estimates of treatment impacts that ignore these differences will be biased
(Shadish et al. 2002). This is likely to be problematic when attempting to evaluate the
effects of government R&D subsidies on firms. Wallsten (2000) points out that government
agencies are more likely to award subsidies to firms that they believe will perform better.
This means that if pre-existing differences between awardee firms and firms that did not
receive awards are not taken into account then the benefits firms receive from subsidies are
likely to be overestimated (Wallsten 2000). This problem is known as selection bias
(Morgan and Winship 2007).
The fact that the control firms examined in this paper are not drawn at random from the
general population of U.S. firms but are drawn from the population of firms that applied for
a 1998 or 2000 ATP award means that they can be considered an internal control group
(Shadish et al. 2002, p. 122). The differences between the control and the treatment firms
prior to treatment are therefore likely smaller than those between the treatment group and a
control group of firms selected from the general population (Shadish et al. 2002, p. 122).
However, the use of an internal control group is not by itself enough to guarantee that
there are no pre-existing differences between treatment and control firms, especially if
better performing firms were selected to receive ATP awards (Shadish et al. 2002; Wallsten
2000). To fully control for pre-treatment differences between treatment and control firms,
it is necessary to utilize a variant of the research design pioneered by Heckman (1979) in
which the predicted probability of a given observation receiving treatment is included in
the regression on the outcome variable of interest. The analysis takes into account Puhani’s
(2000) finding that for this approach to be unbiased it is necessary to include at least one
variable in the regression on the likelihood of receiving treatment which is excluded from
the final regression.
This paper employs the average reviewer scores for one of the ATP criteria related to
the potential for broad-based economic benefits as such a variable. The analysis uses the
average reviewer score for the Need for ATP criteria. Reviewers rated projects highly
when the project had a demonstrated need for public funds (National Institute of Standards
and Technology 1998). Highly scored projects demonstrated that external public and
private funding was not available and that the project did not meet the company criteria or
profile for internal funding (National Institute of Standards and Technology 1998).
D. Smith et al.
123
Because it is one of the three sub-criteria used to determine a firm’s overall business score,
this variable should be significantly correlated with whether or not a firm receives an ATP
award (National Institute of Standards and Technology 1998). However, because this
project-level funding criteria does not directly relate to overall firm performance or
innovativeness, it is likely uncorrelated with overall firm survival. This variable is there-
fore suitable to control for selection bias (Puhani 2000).
5 Empirical methodology
Since this paper employs a modified version of Heckman’s (1979) method for eliminating
selection bias, it is necessary to run a first stage Probit regression featuring an indicator
variable for whether a firm received an ATP award as the dependent variable. The inde-
pendent variables in this regression will be the sub-score for how well a firm demonstrated
that it would only carry out its research project if it received ATP funding and all control
variables employed in the final analysis. This regression will then be used to predict the
probability of each individual firm receiving an ATP award, which will be included in the
final analysis as a control variable. This first stage equation is given below in Eq. 1. A
represents an indicator variable for whether a firm received an ATP award, S represents the
reviewer sub-score for the criteria used only in the first stage, and Z* represents a vector of
the control variables used in a specific analysis.
Equation 1. First Stage Probit Regression Equation of Indicator for Firms’ ATPAwardee Status
PðAi ¼ 1jSi; Z�i Þ ¼ 2pð Þ�1=2
e�
aþb1Siþb�Z�ið Þ2
2
� �ð1Þ
For the survival analysis, a Cox regression is used in accordance with the work of Cefis
and Marsili (2005). The assumption of proportional hazards made by the Cox regression is
tested by calculating the Schoenfeld residuals and determining whether they take on a
nonzero slope when graphed over survival time using the Stata command ‘‘stphtest’’
(Cleves et al. 2004). Equation 2 gives the hazard rate for a firm j in year t as defined by the
Cox regression. P* represents the predicted probabilities for whether a firm received an
ATP award from the stage 1 regression, A represents an indicator variable for whether a
firm received an ATP award, and Z represents a vector of the control variables.
Equation 2. Hazard Rate for Firm J in Year T as Defined by the Cox Regression
h tj P�j þ Z�
J
� �� �¼ h0 tð Þexp b1P
�i þ b2Ai þ b�Z�
i
� �: ð2Þ
6 Data and variables
The primary data set used in these analyses is a cross-sectional set consisting of 302
unique, firm-level observations relating to small for-profit firms that participated in either
the 1998 ATP competition, the 2000 ATP competition, or both. All of these firms employ
fewer than 500 individuals in total, a standard the U.S. Small Business Administration
The longer term effects of federal subsidies on firm survival…
123
often uses to determine whether or not a given firm qualifies as a small business (U.S.
Small Business Administration 2014). This analysis focuses on small firms because they
often lack the resources to conduct major R&D efforts, and are thus generally less likely to
survive than large, multinational firms (U.S. Small Business Administration n.d.). This data
was drawn primarily from administrative records maintained by the National Institute of
Standards and Technology, but it was supplemented with data from some other sources
which are described below.
The dependent variable in all analyses will be the number of full years after a firm
participated in an ATP competition that it survived up until 2014. The competition in the
year 2000 is treated as the starting point for firms that participated in both competitions.
For the purpose of this analysis, only firms that have gone bankrupt are considered to have
died. Firms which merge with or are acquired by other organizations are not considered to
have expired unless the merged or acquiring entity subsequently goes bankrupt. To ensure
that all observations were non-zero (and thus able to be included in the analysis) a value of
one was added to each observation. Data on this variable is drawn from reputable publi-
cally available online sources such as news articles from well-known publications, industry
news websites, and the databases for different state-level government organizations
responsible for keeping track of business registration details. Because firm level ATP
administrative data, particularly data on firms which applied for but did not receive ATP
awards, is highly sensitive and proprietary, it cannot be publicly released.
The primary independent variable of interest used in all analyses will be an indicator
variable for whether or not a firm received an ATP award at any point in its history. Using
binary variables to represent participation in a federal program is common in the literature
(Feldman and Kelley 2006; Blanes and Busom 2004). This data is drawn from adminis-
trative records maintained by the National Institute of Standards and Technology.
As discussed above, to correct for selection bias this paper will utilize the average
reviewer score for the Need for ATP (NATP) criteria. This score relates to how well a firm
is able to convince the proposal evaluators at the National Institute of Standards and
Technology funding for the proposed project is not available from either external or
internal sources and that it will only carry out its proposed research project if it receives
ATP funding (National Institute of Standards and Technology 1998). As with the overall
business and technical scores, the NATP score can range from 0 to 10 with 10 being the
best score (Feldman and Kelley 2006). Each proposal is reviewed by at least one inde-
pendent expert. Where there are multiple reviews for a single proposed project, the analysis
uses the average NATP. Because the analysis is at the company rather than project level,
NATP reviewer scores were also averaged across proposals for those companies with
multiple proposed projects. This data was all derived from the administrative records of the
National Institute of Standards and Technology.
It is important to control for the positive effects that other, non-ATP government
funding could have on firms. Unfortunately, there is currently no reliable source detailing
all of the subsidies made by the U.S. government to firms (Welsh 2014). For this reason, an
indicator variable for whether a given firm had ever received a Small Business Innovation
Research (SBIR) or Small Business Technology Transfer (STTR) award before its
application to the ATP program was created as a proxy for how likely a firm was to receive
government funding from sources other than the ATP program. Taken together, the SBIR
and STTR programs are one of the largest providers of early stage funding to innovative
small businesses in the United States (National Institute of Health 2015). This data was
drawn from the Small Business Administration Technet database.
D. Smith et al.
123
The remaining control variables used in the analyses are mainly based on those used in
the firm level survival analysis employed by Cefis and Marsili (2005). The control variable
for firm size in the year the firm participated in a competition or the year immediately prior
to it is an ordinal variable ranging from 1 to 13, corresponding to employee numbers
ranging from 0 to 499. Link and Rees (1990) have shown that firm size can have a
significant impact on the advantages a firm derives from some of its innovative activities.
A set of three indicator variables corresponding to whether the project proposal a firm
submitted related to biotechnology, electronics, or information technology is included to
control for industry effects. A dollar measure of a firm’s R&D budget in 1997 (for firms
which participated in the 1998 ATP competition) or a firm’s R&D budget in the year 2000
(for firms which participated in the 2000 ATP competition) is included to control for the
overall level of innovative inputs that a firm has. This data is all drawn from two separate
sources: a survey the National Institute of Standards and Technology administered to firms
which had participated in the 2000 ATP competition and the survey Feldman and Kelley
(2006) administered to firms which participated in the 1998 ATP competition.
To control for the overall quality of a firm’s innovative capacity, this analysis will use
the average of the overall technical scores assigned to a firm (generated using the same
process as that used to create the variable relating to a firm’s NATP score). Because the
technical score is an ex ante assessment of a given firm’s project, it is not a perfect measure
of the quality of a firm’s innovative output. Nevertheless, its focus on the level of inno-
vation, feasibility (albeit with high technical risk viewed positively), and quality of a firm’s
R&D plan makes it a reasonable proxy of the overall quality of a firm’s innovation. This
data was drawn from the administrative records of the National Institute of Standards and
Technology.
Finally, as a robustness check an additional regression will be run which will include the
averaged overall business score associated with a firm (generated using the same process as
that used for the NATP and technical scores) instead of the predicted probabilities of a firm
being selected to receive an ATP award as a control variable. It is important to note that the
overall business score refers to the composite of the three sub-scores discussed in Sec-
tion III. Data on this variable is again drawn from the administrative records at the National
Institute of Standards and Technology. Table 6 provides the summary statistics and
sources of the variables discussed above for the applicants considered in this analysis.
7 Results
We examine the impact of ATP on firm survival using a number of alternative econometric
specifications. Results are presented in Table 7. Column (1) presents the results of a Cox
regression which in addition to the standard control variables suggested by the literature
review included a number of variables based on ATP administrative data. Treatment is
measured using an indicator for whether an applicant received an ATP award. Addition-
ally, we control for the overall technical and economic merit of the proposals using average
reviewer scores This regression therefore did not take any explicit steps to control for
selection bias and the estimate of the impact of receiving an ATP award may be biased.
The results indicate that receiving an ATP award did have a positive and significant effect
on a firm’s survival chances at the 5% level. The number of employees at a firm was also
found to have a positive and significant effect on a firm’s chances of surviving at the 5%
level, while a firm’s R&D budget had a positive and significant effect at the 10% level.
The longer term effects of federal subsidies on firm survival…
123
Table 6 Summary statistics
Variable Mean Median SD Skewness Kurtosis Source
Number of fullyearssurvived
13.381 15.000 4.216 - 1.593 4.369 Various sources;see
AveragedNATPbusiness score
3.927 3.500 2.016 0.641 3.064 NationalInstitute ofStandards andTechnology
Indicator forwhether firmreceived anATP award
0.440 0.000 0.497 0.240 1.058 NationalInstitute ofStandards andTechnology
Indicator forwhether afirm receivedSBIR orSTTRfunding
0.387 0.000 0.488 0.462 1.214 U.S. CensusBureauthrough theNationalInstitute ofStandards andTechnology
Firm R&Dbudget
2,438,129.00 600,000.00 7,424,061.00 8.214 81.879 NationalInstitute ofStandards andTechnology;Feldman andKelley (2006)
Number ofemployees
4.424 4.000 2.457 1.237 4.485 NationalInstitute ofStandards andTechnology;Feldman andKelley (2006)
Indicator for anATP projectfocused onbiotechnology
0.245 0.000 0.431 1.186 2.406 NationalInstitute ofStandards andTechnology;Feldman andKelley (2006)
Indicator for anATP projectfocused onelectronics
0.308 0.000 0.462 0.832 1.692 NationalInstitute ofStandards andTechnology;Feldman andKelley (2006)
Indicator for anATP projectfocused oninformationtechnology
0.139 0.000 0.347 2.086 5.352 NationalInstitute ofStandards andTechnology;Feldman andKelley (2006)
Indicator forparticipationin the 1998ATPcompetition
0.354 0.000 0.479 0.609 1.371 NationalInstitute ofStandards andTechnology
D. Smith et al.
123
These findings are in line with those of Cefis and Marsili (2005), who find that firm size
and innovation both boosted a firm’s survival prospects. Of course, to definitively show a
causal link between receiving an ATP award and firm survival it is necessary to explicitly
control for selection bias. The Breslow method for tied failures was employed in this
regression. The test of the Schoenfeld residuals for this regression returned a Chi squared
value of 8.110 and a corresponding p value of 0.703, indicating that the proportional
hazards assumption held and a Cox regression was appropriate.
The results from the Probit regression to predict the likelihoods of firms receiving ATP
awards are in column (2). These results show that the averaged NATP score had a positive
and significant effect, with an estimated coefficient of 0.289 and a corresponding p value
smaller than 0.001. Despite this, however, the correlation between the averaged NATP
score and the number of full years a firm survived was only 0.224. This indicates that the
NATP score is a variable which satisfies the criteria put forward by Puhani (2000). The size
of a firm, averaged technical score associated with a firm, and indicator variable for
whether a firm participated in the 2000 ATP competition were also found to all have
positive and significant coefficients.
Column (3) contains the main result of interest. The results of the Cox regression with
the predicted probabilities derived from the Probit regression as a control variable indicate
that receiving an ATP award does have a positive and significant effect on a firm’s survival
chances. The number of employees at a firm was also found to be positive and significant at
the 5% level, while a firm’s R&D budget was positive and significant at the 10% level.
This is consistent with the findings of Cefis and Marsili (2005). The Schoenfeld residuals
test for this regression yielded a Chi squared value of 7.610 and a corresponding p value of
0.748. This regression used the Efron method to account for tied failures and robust
standard errors.
As a robustness check, column (4) displays the results of a second Cox regression which
was identical to that used in the primary analysis except that it used the Breslow method to
account for tied failures instead of the Efron method. The results of this regression did not
differ from those of the primary regression in any significant way. The Schoenfeld
residuals test for this regression produced a Chi squared value of 7.230 and a corre-
sponding p value of 0.781.
Table 6 continued
Variable Mean Median SD Skewness Kurtosis Source
Indicator forparticipationin the 2000ATPcompetition
0.735 1.000 0.442 - 1.066 2.135 NationalInstitute ofStandards andTechnology
Averagedoveralltechnicalscore
4.282 3.833 2.191 0.623 2.647 NationalInstitute ofStandards andTechnology
Averagedoverallbusiness score
3.805 3.424 2.059 0.568 2.869 NationalInstitute ofStandards andTechnology
The longer term effects of federal subsidies on firm survival…
123
8 Conclusion
This paper has demonstrated that receiving ATP funding has had a positive and significant
causal impact on the lifespan of small firms which participated in either the 1998 or the
2000 ATP competition and received ATP funding. By employing a variant of the research
design first pioneered by Heckman (1979) and exploiting the NATP scores assigned during
the selection process, this paper has addressed the common econometric issue of selection
bias in subsidy evaluation (Puhani 2000; Wallsten 2000; Murnane and Willett 2011). In
doing so, this paper fills a significant hole in the literature relating to firm survival analysis
Table 7 Impact of ATP award on firm survival
(1) (2) (3) (4)Dependent variable
# Yearssurvived
Indicator: receivedATP award
# Years survived # Years survived
Indicator: ATP awardrecipient
0.534**(0.160)
0.501**(0.141)
0.505**(0.141)
Indicator: Prior SBIR or STTRrecipient
0.750(0.189)
0.079(0.175)
0.762(0.191)
0.760(0.187)
Firm R&D Budget 1.000*(1.53e - 08)
- 3.42e - 10(1.59e - 08)
1.000*(1.57e - 08)
1.000*(1.54e - 08)
Number of employees 0.858**(0.054)
0.121**(0.036)
0.860**(0.062)
0.862**(0.061)
Indicator: biotechnologyproject
1.312(0.379)
0.340(0.217)
1.297(0.373)
1.298(0.366)
Indicator: electronics project 0.912(0.266)
0.236(0.207)
0.892(0.256)
0.898(0.254)
Indicator: informationtechnology project
0.549(0.226)
0.210(0.260)
0.538(0.217)
0.542(0.216)
Indicator: 1998 competitionparticipant
1.321(0.719)
- 0.460(0.339)
1.195(0.623)
1.192(0.616)
Indicator: 2000 competitionparticipant
0.793(0.429)
1.303**(0.374)
0.855(0.511)
0.853(0.504)
Average technical score 0.881(0.077)
0.351**(0.077)
0.864(0.113)
0.865(0.111)
Average NATP score 0.289**(0.073)
Average business score 0.912(0.083)
Predicted probability ofreceiving an ATP award
0.795(0.811)
0.802(0.807)
N = 302 N = 302 N = 302 N = 302
CoxRegression
Probit Regression Two-Stage CoxRegression
Two Stage CoxRegression
Standard error is given in parentheses below estimated coefficient
*Indicates a coefficient is significant at the 10% level
**Indicates a coefficient is significant at the 5% level
D. Smith et al.
123
by directly examining the role that government policies can play in boosting firms’ life
spans. By focusing on a 14–16 year timespan, this paper also delivers a much needed
examination of the longer term benefits of R&D subsidies. This paper should be seen as
supporting the idea that Einio’s (2014) findings hold for the United States as well. Of
course, it is important to note that Einio’s (2014) paper and this paper examined related but
different firm-level outcomes. For policy-makers, this paper implies that national gov-
ernments can play a beneficial role in stimulating innovation in industry if they structure
R&D subsidy programs appropriately.
This paper also contributes to the ongoing debate over public sector entrepreneurship,
which is an important factor in technology transfer (Dublin Institute of Technology 2017).
Public sector entrepreneurship has been defined as ‘‘the promulgation of innovative public
policy initiatives that generate greater economic prosperity by transforming a status-quo
economic environment into one that is more conducive to economic units engaging in
creative activities in the face of uncertainty’’ (Link 2016, p. 355). Given its focus on
stimulating the innovative projects of various firms in a way that would not occur without
government intervention, the ATP clearly falls into this category (National Institute of
Standards and Technology 1998). This paper shows that, in the United States, programs
such as the ATP might be most useful to nascent entrepreneurs, who typically have fewer
resources than well-established entrepreneurs (Leyden 2016). In addition, by emphasizing
interorganizational collaboration, the ATP could help nascent entrepreneurs form the
strong ties to other organizations that they need (Leyden 2016). The paper can be seen as
supporting the argument by Richardson et al. (2016) that the ATP provides capital to
entrepreneurial firms which they might not otherwise receive due to the uncertain nature of
entrepreneurship. Thus, this paper provides significant new insights on the question of how
public sector entrepreneurship can help to facilitate technology transfer from the laboratory
to the market.
It is important to acknowledge the limitations of this research. These findings may not
be applicable to firms located outside of the United States, or to firms with more than 500
employees. Additionally, there is some amount of ambiguity regarding what constitutes a
firm’s ‘‘death.’’ While this analysis did not treat a firm’s acquisition as the equivalent of its
bankruptcy, it is at least possible that in some cases a larger organization might not receive
any overall economic benefit from acquiring a smaller firm but would nevertheless con-
tinue to exist. Unfortunately, it is impossible to determine which particular acquisitions
ultimately benefitted the acquiring entity in some way and which did not without an
unfeasibly large data collection effort. However, the fact that most profit-maximizing
businesses would be unwilling to acquire another business without an extensive exami-
nation of what that business had to offer them suggests that this should not be too great of
an issue. Also, this analysis does not directly address the issue of whether federal R&D
funding crowds out private R&D funding (David et al. 2000). In theory, if a firm that
received ATP funding had its own funding crowded out, then it could use its private R&D
funding on another project and thereby boost its survival chances still further. While the
inclusion of the NATP score in the review process was intended to prevent crowding out
(National Institute of Standards and Technology 1998), it would require a causal analysis
of awardees’ innovative output to determine if this was the case.
Also, the conclusions of this paper may not apply to R&D subsidy programs which
differ significantly from the ATP in terms of structure or selection criteria. The ATP was
fairly unique in terms of both the standards it used to evaluate applications and the fact that
it continued to actively monitor and manage (in collaboration with recipient firms) funded
ATP projects (National Institute of Standards and Technology 1998). Thus, this analysis
The longer term effects of federal subsidies on firm survival…
123
does not indicate that any U.S. federal R&D subsidy would be expected to boost firm
survival. Instead, it suggests that the programs with selection criteria and project man-
agement practices similar to the ATP may have a positive and significant causal effect on
the survival chances of recipient firms.
There are two main directions for future research suggested by this paper. The first is to
compare the effects of the ATP to the effects of other R&D subsidy programs with similar
goals but different structures and awardee selection processes. This would be useful in
determining how best to organize R&D subsidies going forward. One such feature of the
ATP which deserves further examination is that, beginning in 1994, businesses were
allowed to play a role in establishing several sub-competitions within the ATP focused on
promoting research in technology areas they felt would yield broad and significant eco-
nomic benefits (Balutis and Lambis 2001). Businesses would submit white papers calling
for research in specific technology areas, and then staff at the National Institute for
Standards and Technology would evaluate these proposals and work with the businesses to
finalize the technology fields included in the competition (Balutis and Lambis 2001). It will
be interesting to see whether this collaboration of government and industry (coupled with
the rigorous project review process of the ATP) is able to overcome the potential problem
of government being unable to determine which R&D projects it would be socially optimal
to invest in (National Research Council 2001; Stiglitz and Wallsten 1999). Comparing the
results of ATP subsidies given out after 1994 to the results of subsidies with similar goals
but with no input from industry would shed light on this question.
Another feature of the ATP which deserves further examination is the specific scores it
uses to evaluate applications. One of these is the NATP score. This score was included in
the review process to prevent federal funding from crowding out private R&D funding for
R&D projects which would have occurred regardless of whether the recipient firm received
a subsidy (David et al. 2000). However, the NATP score only constituted one-third of one
of the two scores used to determine whether an ATP applicant received funding (National
Institute of Standards and Technology 1998). Comparing the causal effects of ATP funding
to the causal effects of both subsidy programs which placed more emphasis on preventing
crowding out and those which placed less emphasis on that issue could provide valuable
insight into how major or minor an issue crowding out is in practice.
Similarly, the fact that one of the sub-scores used to determine the technical score
positively relates explicitly to how high the technical risk of a given project was might
provide a possible research opportunity (National Institute of Standards and Technology
1998). Because businesses typically shy away from R&D projects involving a high level of
technical risk, there is a strong theoretical justification for arguing that businesses which
received an ATP award would not have engaged in the ATP projects they proposed without
ATP funding. It would be interesting to check the empirical validity of this claim by
comparing the performance of ATP recipients with recipients of a similar subsidy which
did not place an emphasis on high technical risk.
Finally, the fact that the ATP is a purely federal program provides an interesting avenue
for further research. As discussed above, there is some disagreement in the literature as to
whether federal or regional agencies would be more effective at promoting the interor-
ganizational partnerships necessary for encouraging innovation (Oughton et al. 2002).
While this paper has shown indirectly that federal agencies do seem to have the ability to
foster these partnerships through R&D subsidies, the question of whether regional agencies
could conduct this task more effectively remains open. Comparing the causal effects of the
ATP on firm outcomes to the causal effects of R&D subsidies administered by regional
agencies would help to answer this important question.
D. Smith et al.
123
The second direction of research suggested by this paper is a further examination of the
benefits produced by the ATP. The ATP was explicitly intended to promote broad eco-
nomic benefits (National Institute of Standards and Technology 1998). Firm survival time
is one metric of such benefits, but it is not the only one. Analyses focused on determining
other potential benefits of the ATP, such as its impact on firms’ patenting behavior or their
successful commercialization of research projects, would shed more light on the overall
desirability of federal R&D subsidies structured similarly to the ATP in the United States.
References
Aldieri, L., & Cincera, M. (2009). Geographic and technological R&D spillovers within the triad: Microevidence from U.S. patents. The Journal of Technology Transfer, 34(2), 196–211.
Audretsch, D., Leyden, D., & Link, A. (2012). Universities as research partners in publicly supportedentrepreneurial firms. Economics of Innovation and New Technology, 21(5–6), 529–545.
Audretsch, D., Link, A., & Scott, J. (2002). Public/private partnerships: Evaluating SBIR-supportedresearch. Research Policy, 31(1), 145–158.
Azoulay, P., Graff-Zivin, J., Li, D., & Sampat, B. (2014). Public R&D Investments and Private SectorPatenting: Evidence from NIH Funding Rules. Kellogg School of Management. Retrieved from http://economics-eng.huji.ac.il/Uploads/Contents/documents/Pierre_Azoulayy.pdf.
Balutis, A., & Lambis, B. (2001). The ATP Competition Structure. In C. Wessner (Ed.), The advancedtechnology program: Assessing outcomes (pp. 175–188). Washington, D.C.: National Academy Press.
Baum, J., Calabrese, T., & Silverman, B. (2000). Don’t go it alone: Alliance network composition andstartups’ performance in Canadian biotechnology. Strategic Management Journal, 21(3), 267–294.
Becchetti, L., & Trovato, G. (2002). The determinants of growth for small and medium sized firms: The roleof the availability of external finance. Small Business Economics, 19(4), 291–306.
Bercovitz, J., & Feldman, M. (2007). Fishing upstream: Firm innovation strategy and university researchalliances. Research Policy, 36(7), 930–948.
Blanes, J., & Busom, I. (2004). Who participates in R&D subsidy programs? The case of Spanish manu-facturing firms. Regional Policy, 33(10), 1459–1476.
Bozeman, B., Hardin, J., & Link, A. (2008). Barriers to the diffusion of nanotechnology. Economics ofInnovation and New Technology, 17(7–8), 749–761.
Caloghirou, Y., Tsakanikas, A., & Vonortas, N. (2001). University-industry cooperation in the context of theEuropean framework programs. The Journal of Technology Transfer, 26(1–2), 153–161.
Cefis, E., & Marsili, O. (2005). A matter of life and death: Innovation and firm survival. Industrial andCorporate Change, 14(6), 1167–1192.
Chesbrough, W. (2003). Open innovation: The new imperative for creating and profiting from technology.Watertown, MA: Harvard Business Press.
Christensen, C. (1997). The innovator’s dilemma: When new technologies cause great firms to fail. Boston,MA: Harvard Business Press.
Cleves, M., Gould, W., & Gutierrez, R. (2004). An introduction to survival analysis using stata (Reviseded.). College Station, TX: Stata Press.
Colombo, M., Mustar, P., & Wright, M. (2010). Dynamics of science-based entrepreneurship. The Journalof Technology Transfer, 35(1), 1–15.
Cunningham, J., & Link, A. (2015). Fostering university-industry collaborations in European Unioncountries. International Entrepreneurship and Management Journal, 11(4), 849–860.
Czarnitziki, D., Ebersberger, B., & Fieri, A. (2007). The relationship between R&D collaboration, subsidies,and R&D performance: Evidence from Germany and Finland. Journal of Applied Econometrics, 22(7),1347–1366.
Czarnitzki, D., Hanel, P., & Rosa, J. (2011). Evaluating the impact of R&D tax credits on innovation: Amicro-econometric study on Canadian firms. Research Policy, 40(2), 217–229.
David, P., Hall, B., & Toole, A. (2000). Is public R&D a complement or a substitute for private R&D? Areview of the econometric evidence. Research Policy, 29(4–5), 497–529.
Deeds, D., & Decarolis, M. (1999). The impact of stocks and flows of organizational knowledge on firmperformance: An empirical investigation of the biotechnology industry. Strategic Management Jour-nal, 20(10), 953–968.
The longer term effects of federal subsidies on firm survival…
123
Dublin Institute of Technology. (2017). Topics and themes: T2S2015. Retrieved from http://www.t2s2015.com/page/9/topics-and-themes/.
Dyer, J. (1997). Effective interfirm collaboration: How firms minimize transaction costs and maximizetransaction value. Strategic Management Journal, 18(7), 535–556.
Dyer, J., & Singh, H. (1998). The relational view: Cooperative strategy and sources of interorganizationalcompetitive advantage. Academy of Management Review, 23(4), 660–679.
Einio, E. (2014). R&D subsidies and company performance: Evidence from the geographic variation ingovernment funding based on the ERDF population-density rule. The Review of Economics andStatistics, 96(4), 710–728.
Esteve-Perez, S., & Manez-Castillejo, J. (2008). The resource-based theory of the firm and firm survival.Small Business Economics, 30(3), 231–249.
Feldman, M. (1994). The university and economic development: The case of Johns Hopkins University andBaltimore. Economic Development Quarterly, 8(1), 67–76.
Feldman, M., & Kelley, M. (2003). Leveraging research and development: Assessing the impact of the U.S.Advanced Technology Program. Small Business Economics, 20(2), 153–165.
Feldman, M., & Kelley, M. (2006). The Ex Ante assessment of knowledge spillovers: Government R&Dpolicy, economic incentives, and private firm behavior. Research Policy, 35(10), 1509–1521.
Griffith, R., Redding, S., & Van Reenen, J. (2004). Mapping the two faces of R&D: Productivity growth in apanel of OECD countries. The Review of Economics and Statistics, 86(4), 883–895.
Hall, M., & Link, A. (2015). Technology-based state growth policies: The case of North Carolina’s greenbusiness fund. Annals of Regional Science, 54(2), 437–449.
Hall, B., Link, A., & Scott, J. (2003). Universities as research partners. The Review of Economics andStatistics, 85(2), 485–491.
Hall, B., & Maffioli, A. (2008). Evaluating the impact of technology development funds in emergingeconomies: Evidence from Latin America. The European Journal of Development Research, 20(2),172–198.
Hanel, P., & St Pierre, M. (2006). Industry-university collaboration by Canadian manufacturing firms. TheJournal of Technology Transfer, 31(4), 485–499.
Heckman, J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161.Holmes, P., Hunt, A., & Stone, I. (2010). An analysis of new firm survival using a hazard function. Applied
Economics, 42(2), 185–195.Hussinger, K. (2008). R&D and subsidies at the firm level: An application of parametric and semiparametric
two-step selection models. Journal of Applied Econometrics, 23(6), 729–747.Hytinnen, A., & Toivanen, O. (2005). Do financial constraints hold back innovation and growth?: Evidence
on the role of public policy. Research Policy, 34(9), 1385–1403.Johnstone, N., Hascic, I., & Popp, D. (2010). Renewable energy policies and technological innovation:
Evidence based on patent counts. Environmental & Resource Economics, 45(1), 133–155.Klepper, S. (2002). Firm survival and the evolution of oligopoly. RAND Journal of Economics, 33(1),
37–61.Klette, T., Møen, J., & Griliches, Z. (2000). Do subsidies to commercial R&D reduce market failures?
Microeconometric Evaluation Studies. Research Policy, 29(4–5), 471–495.Knoben, J., & Oerlemans, L. (2006). Proximity and inter-organizational collaboration: A literature review.
International Journal of Management Reviews, 8(2), 71–89.Kumar, V., Kumar, U., & Persaud, A. (1999). Building technological capability through importing tech-
nology: The case of Indonesian manufacturing industry. The Journal of Technology Transfer, 24(1),81–96.
Lerner, J. (2000). The government as venture capitalist: The long-run impact of the SBIR program. TheJournal of Private Equity, 3(2), 55–78.
Leyden, D. (2016). Public sector entrepreneurship and the creation of a sustainable innovative economy.Small Business Economics, 46(4), 553–564.
Leyden, D., & Link, A. (2013). Knowledge spillovers, collective entrepreneurship, and economic growth:The role of universities. Small Business Economics, 41(4), 797–817.
Link, A. (2015). Capturing knowledge: Private gains and public gains from university research partnerships.Foundations and Trends in Entrepreneurship, 11(3), 139–206.
Link, A. (2016). Public sector entrepreneurship: Introduction to the special section. Economica e PoliticaIndustriale, 43(3), 355–356.
Link, A., & Rees, J. (1990). Firm size, university based research, and the returns to R&D. Small BusinessEconomics, 2(1), 25–31.
Link, A., & Scott, J. (2005). Universities as research partners in U.S. joint research ventures. ResearchPolicy, 34(3), 385–393.
D. Smith et al.
123
Loof, H., & Brostrom, A. (2008). Does knowledge diffusion between university and industry increaseinnovativeness? The Journal of Technology Transfer, 33(1), 73–90.
Mansfield, E. (1995). Academic research underlying industrial innovations: Sources, characteristics, andfinancing. The Review of Economics and Statistics, 77(1), 55–65.
Morgan, K. (1997). The learning region: Institutions, innovation, and regional renewal. Regional Studies,31(5), 479–490.
Morgan, S., & Winship, C. (2007). Counterfactuals and causal inference. New York, NY: CambridgeUniversity Press.
Murnane, R., & Willett, J. (2011). Methods matter: Improving causal inference in educational and socialscience research. New York, NY: Oxford University Press.
Musso, P., & Schiavo, S. (2008). The impact of financial constraints on firm survival and growth. Journal ofEvolutionary Economics, 18(2), 135–149.
National Institute of Health. (2015). What are SBIR and STTR programs?. Retrieved from https://sbir.nih.gov/.
National Institute of Standards and Technology. (1998). Advanced Technology Program proposal prepa-ration kit. Gaithersburg, MD: National Institute of Standards and Technology.
National Institute of Standards and Technology. (2004). ATP Proposal Preparation Kit. Retrieved fromhttp://www.atp.nist.gov/atp/kit-04/final-kit.pdf.
National Institute of Standards and Technology. (2005). Survey of ATP Applicants 2000: Introduction.Retrieved from http://www.atp.nist.gov/eao/applicants/intro.htm.
National Institute of Standards and Technology. (2007). America COMPETES Act Brings ImmediateChanges to NIST. Retrieved from https://www.nist.gov/news-events/news/2007/08/america-competes-act-brings-immediate-changes-nist.
National Institute of Standards and Technology. (n.d.). Technology Innovation Program. Retrieved fromhttps://www.nist.gov/technology-innovation-program.
National Institutes of Health. (2016). What is SBIR and STTR?. Retrieved from https://sbir.nih.gov/about/what-is-sbir-sttr.
National Research Council. (2001). The Advanced Technology Program: Assessing outcomes. Washington,D.C.: National Academy Press.
Nishimura, J., & Okamuro, H. (2010). R&D productivity and the organization of cluster policy: Anempirical evaluation of the industrial cluster project in Japan. The Journal of Technology Transfer,36(2), 117–144.
Oughton, C., Landabaso, M., & Morgan, K. (2002). The regional innovation paradox: Innovation policy andindustrial policy. The Journal of Technology Transfer, 27(1), 97–110.
Puhani, P. (2000). The Heckman correction for sample selection and its critique. Journal of EconomicSurveys, 14(1), 53–68.
Richardson, A., Audretsch, D., & Aldridge, T. (2016). Motivating entrepreneurship and innovative activity:Analyzing U.S. policies and programs. In D. Audretsch & A. Link (Eds.), Essays in Public SectorEntrepreneurship. Berlin: Springer.
Sachwald, F. (2008). Location choices within global innovation networks: The case of Europe. The Journalof Technology Transfer, 33(4), 364–378.
Sampson, R. (2007). R&D alliances and firm performance: The impact of technological diversity andalliance organization on innovation. Academy of Management Review, 50(2), 364–386.
Saxenian, A. (1994). Regional advantage: Culture and competition in Silicon Valley and Route 128.Cambridge, MA: Harvard University Press.
Schrank, A., & Whitford, J. (2011). The anatomy of network failure. Sociological Theory, 29(3), 151–177.Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi-experimental designs for gener-
alized causal inference. Belmont, CA: Wadsworth.Stiglitz, J., & Wallsten, S. (1999). Public-private technology partnerships: Promises and pitfalls. American
Behavioral Scientist, 43(1), 52–73.Stuart, T. (2000). Interorganizational alliances and the performance of firms: A study of growth and
innovation rates in a high-technology industry. Strategic Management Journal, 21(8), 791–811.Takalmo, T., & Tanayama, T. (2010). Adverse selection and financing of innovation: Is there a need for
R&D subsidies? The Journal of Technology Transfer, 35(1), 16–41.U.S. Department of Energy. (2016). 2016 ARPA-E Energy Innovation Summit. Retrieved from https://arpa-
e.energy.gov/?q=events/2016-arpa-e-energy-innovation-summit.U.S. National Institute of Standards and Technology. (n.d.). ATP fiscal year 1998 competition results.
Retrieved from http://www.atp.nist.gov/comps/index98.htm.U.S. Department of Energy. ta (n.d.). ARPA-E: General questions. Retrieved from https://arpa-e.energy.gov/
?q=faq/general-questions.
The longer term effects of federal subsidies on firm survival…
123
U.S. Small Business Administration. (n.d.). About SBIR. Retrieved from https://www.sbir.gov/about/about-sbir.
U.S. Small Business Administration. (2014). Summary of size standards by industry sector. Retrieved fromhttps://www.sba.gov/content/summary-size-standards-industry-sector.
Wagner, J. (2002). The causal effects of exports on firm size and labor productivity: First evidence from amatching approach. Economics Letters, 77(2), 287–292.
Wallsten, S. (2000). The effects of government-industry R&D programs on private R&D: The case of thesmall business innovation research program. RAND Journal of Economics, 31(1), 82–100.
Welsh, W. (2014). GAO pushes better reporting of federal grants data. Information Week.
D. Smith et al.
123