exploring the societal returns of research conducted within hospitals new frontiers in evaluation...
TRANSCRIPT
Exploring the Societal Returns of Research Conducted within
Hospitals
New Frontiers in Evaluation ConferenceVienna 2006, April 24th - 25th
Antonio García Romero and Luis Royuela MoralesAgencia Laín Entralgo.
Consejería de Sanidad y Consumo de la Comunidad de Madrid.
Presentation Outline
1.1. IntroductionIntroduction2.2. FrameworkFramework3.3. The studyThe study4.4. ResultsResults5.5. ConclusionsConclusions
1.1. IntroductionIntroduction• How much to spend on medical
research?It could be useful to measure the
societal impacts of medical research
• Difficult but NecessaryDifficult: we have a limited
understanding of how research induce societal effects
Necessary: In US: $70 billion / year saved in
medical care costs$90 billion / year sales of products
unrelated to health care resulted from NIH R&D
Silverstein S. 1995
2. Framework2. Framework
The evaluation of scientific research based mainly bibliometrics and expert judgement (peer review).
These methods are not very useful if the objective is to measure the returns from research in terms of wealth, employment or health.
Development of new indicators and methods are needed
2. Framework2. FrameworkStatement 1: Improved health is a consequence
of several factors and health care quality is only one of them
Health Care (hospitals)
EnvironmentalConditions
Genetics
Life style decisions HEALTH
2. Framework2. FrameworkStatement 2: Health care is a result of the
interaction of several factors such as research, training, health technologies and other organizational characteristics
Health Care Drugs / Health Technologies
Research
Training
2. Framework2. FrameworkStatement 3: The research conducted within
hospitals generates several societal outcomes, in addition to health care
Research within
hospitals
Drugs innovation
OtherHuman Capital
Health Tech.
Innovation
Clinical practice improvement
2. Framework2. FrameworkStatement 4: The relationship between medical
research and its societal outcomes is very complex
Lewison, G From Biomedical Research to Health Improvement, Scientometrics. 54(2): 179-92.
2. Framework2. Framework
A consequence of this complexity is the existence of a wide set of methodological approaches that can be applied to analyze this topic
• Macro approachesUseful to estimate aggregated
effects on employment, wealth or population’s health
• Meso & Micro approachesUseful to estimate direct / indirect
effects, policy analysis, comparative analysis
2. Framework2. Framework
A) The Region of Madrid• 6 mill. Habitants • 26.000 € GDP/per capitaB) The Public Health Care System
of Madrid (SMS)• 24 + 8 hospitals• + 200 Primary Care Centers• Around 10,000 physicians• 2006 Budget: 6,066 mill. €C) Research in SMS• 1,650 researchers (INE)• 13% of their time• 2,000 papers SCI / year
3. The Study3. The Study
• Objective: To analyze the effect that physicians’ research has on their clinical performance
• Two Assumptions:• Research within hospitals should
generate both scientific and societal returns
• research allows physicians to improve their skills and knowledge level.
• Methodology:• Individual data (micro approach)• Motivation, Satisfaction and Group
Structure as explanatory variables• Structural Equation Modeling
3. The Study3. The Study
Baseline model:Comprises two sets of hypotheses
Satisfaction (*)
Gender
SCI
Ph.D
Impact
Motivation
GroupStructure
Negative (-)(0: no Ph.D 1: Ph.D)
Negative (-)(0: Men 1: Women)
Positive (+)Positive (+)
Positive (+) Positive (+)
Positive (+)
Positive (+)
Positive (+)
3. The Study3. The Study
Data 279 physicians-scientists working in hospitals of SMS (Madrid)
Table 1. Population and sample composition.
Questionnaires Number of Respondents
(%)
Number of respondents (%) (physician-scientists)
Hospitals 2.038 410 (20,1) 279 (67,8)
Primary care
1.818 347 (19,1) 74 (21,3)
Total 3.856 757 (19,6) 352 (46,5)
3. The Study3. The StudyVariables:3 observed variables & 4 constructs
Table 2. Definition of constructs and variables used in the study
Construct/ Variable Definition
PhD 1 if respondent has the PhD degree; 0 otherwise
GENDER Gender of the respondent (0: men; 1: women)
SCI Papers SCI Journals (last 3 years)
IMP Construct for perceived impact of R&D on clinical practice
SAT_CAR, SAT_CLI and SAT_SCI
Constructs for job satisfaction
MOT Construct for motivation
GROUP Construct for group structure
We have estimated three models• Satisfaction with professional
career• Satisfaction with job climate• Satisfaction with scientific issues
4. Results4. Results
Table 4. Structural models standardized coefficients
Effect Model 1 SAT_CAR
Model 2SAT_CLI
Model 3SAT_SCI
Satisfaction → Impact research clinical performance
0.59(4.032)
0.64(4.190)
0.67(4.483)
Motivation → Impact research clinical performance
0.33(3.400)
0.30(3.330)
0.31(3.546)
SCI→ Impact research clinical performance 0.10(1.603)
0.14(2.147)
0.10(1.605)
Goodness of Fit Summary
χ 2 (N = 279)(1) 384.99 (p<0.001)
415.78 (p<0.001)
543.87 (p<0.001)
Goodness of fit Index (GFI)(2) 0.88 0.88 0.86
Comparative Fit Index (CFI)(3) 0.92 0.93 0.92
Incremental Fit Index (IFI)(4) 0.92 0.93 0.92
(1)Non-significant χ 2 indicates that the hypothesized model fits the data(2)Similar to R2 in regression analysis(3)CFI > 0,9 indicates a good fit(4)IFI > 0,9 indicates a good fitThese cut off values are recommended by HU et al (1995)
4. Results4. Results
SAT_CLI
Gender
SCI
Ph.D
Impact
Motivation
GroupStructure
-0.24
-0.17
0.21 0.30
0.640.20
0.14
5. Conclusions5. Conclusions
Motivation and Job Satisfaction have a significant and positive effect on the perceived impact of research activity on clinical performance
There is a significant relationship between Research Productivity and gender, academic degree, Motivation and Group structure
These results suggest the strategic value of the human factor within R&D management