Preliminary Results and Research Design and Methods

George LeikaufUniversity of Cincinnati

ORTHE TRUTH HURTS

WHAT THE EXECUTIVE SECRETARY WON’T TELL YOU

YOU CAN TRY IT YOUR OWN WAY BUT THE REVEIWERS PROBABLY WON’T LIKE IT

LIFE OR DEATH ON THE TENURE TRACK

Preliminary Results

The overall purpose of the research plan is to describe the what, when, why, and how of the proposal

Preliminary Results contributes to both the why and how

IT IS ALL ABOUT YOU, YOUR WORK, YOUR JUDGEMENT, andHOW YOU INTERPRET DATA

Preliminary Results

This section is not mandatory for new applicationsBUT IT IS VIRTUALLY IMPOSSIBLE TO OBTAIN

A FAVORABLE REVIEW WITHOUT STRONG PRELIMINARY RESULTS

In a competing renewal application, this section becomes a progress report describing studies performed during the last grant period

UNLESS YOU HAVE OUTSTANDING PROGRESS CONSIDER A NEW PROPOSAL

Preliminary Results

Recommended length:

6 - 8 pages

Preliminary Results

Content: In a new application, the preliminary results contributes to why of the proposal:

• Provide experimental support for the hypothesis and the research design

• Brief description of older published studies by the applicant that provide important background information relevant to the proposed project

Preliminary Results

Content: In a new application, the preliminary results contributes to how of the proposal:

Present unpublished studies by the applicants to establish:• the feasibility and importance of the

project• the applicants’ competence and experience

with the experimental techniques to be used in the project

Preliminary Results

Content: May also include:

Results of previous studies by the applicant not directly relevant to the proposed project that demonstrated the applicants’ competence and experience with the experimental techniques to be used in the project

CLEARLY IDENTIFIED AS SUCH.PUBLISH IN RIGOROUS JOURNALS

BETTER PLACED IN BACKGROUND?

Preliminary ResultsSuggestions

• Organization: Link directly with Specific Aims (i.e. Use headings “Preliminary Data Supporting Aim 1: To determine…”)

SUPPORT EACH AND EVERY AIM!• Include all Tables and Figures necessary

for the presentation of preliminary resultsTYPICALLY ORDINATE (Y AXIS) IS THE

INDEPENDENT VARIABLE.

Preliminary ResultsSuggestions

• Full-size glossy photographs of materials such as electron micrographs, gels, etc. may be included in the appendix, but only if a photocopy (reduced in size, as appropriate) is included in the body of the Research Plan

DON’T EXPECT APPENDICIES TO BE READ

Preliminary ResultsSuggestions• Figures and Figure legends must be legible.

• Observe limits on type size given in the application instructions, but beyond this rule, the critical factor is whether the data are legible and convincing to the reviewers

REVIEWERS ARE NOT IMPRESSED BY SMALL DIFFERENCES

Preliminary ResultsSuggestions

• All data must be analyzed with proper statistical methods

STATISTICAL VS. MEDICAL SIGNIFICANCE

• Don't show raw data or data you have not fully analyzed

Preliminary ResultsSuggestions

• Don’t dwell on results already published

PUBLISHED RESULTS CAN PROVE COMPENDENCE

BUT PUBLISHED RESULTS REDUCE INNOVATION

Preliminary ResultsSuggestions

• Present result objectively USE SUPERLATIVES SPARINGLY

“DRAMATIC” “ABSOLUTELY” “CRITICAL”“IMPRESSIVE”

• Don’t be overly impressed by your own findings

HAVE CONFIDENCE IN YOUR REVIEWER

Preliminary ResultsSuggestions

• Make sure you are ready

• Collect at least 1-2 years of preliminary data

• Consider publishing 2 manuscripts in the area to make your research questions more robust

Preliminary ResultsEditing

• Consider including mainly data that demonstrate expertise with the more difficult methods

• Combine figures into multiple panels

• Place Tables into text (present data parenthetically)

Preliminary ResultsEditing

• Eliminate any figure that will initiate a debate in the mind of the reviewer

• Don’t ignore different possible interpretations

• Be as cautious as is prudent

Preliminary ResultsEditing

• Word choice: Our findings “suggest”, “imply”, “support ”, or “indicate”

• Present your results (even if they are preliminary) in as professional a manner as possible, with clear and complete figure/table legends, calibrations, statistical analysis, etc.

Research Design and Methods

The overall purpose of the research plan is to describe the what, when, why, and how of the proposal

Research Design and Methods contributes to what, when, why, and how

APPROACH > SIGNIFICANCE > INNOVATION

Research Design and Methods

Recommended length:

20 pages

TOTAL LIMIT IS 25 PAGES SO, YOU CAN NOT GIVE EVERY LITTLE DETAIL IN METHODS SECTION

SOME REVIEWERS ARE TECHIQUE DRIVEN AND MAY MAKE MOUNTAINS OUT OF MOLE HILLS

Research Design and Methods

Content: Research Design and Methods contributes to what, when, why, and how:

• What? Experimental Model and Specific AimsREAGENTS IN HAND AND EXPERIENCE

• When? Sequence of Specific AimsINTERACTING BUT INDEPENDENT

• Why? Meaningfulness of results, and their relationship to hypothesis• HUMAN HEALTH SIGNIFICANCE

• How? Experimental Design and MethodsSTATISTICAL APPROACH

Research Design and Methods

Content:

• Description of any new methodology used and why it represents an improvement over the existing ones

• Detailed description of specific methods to be employed

• Projected sequence of Specific Aims and timetable

Research Design and Methods

Content:

• Experimental design (how the research will be carried out)

• Detailed discussion of the way in which the results will be collected, analyzed, and interpreted

Research Design and Methods

Content: Detailed discussion of the way in which the results will be collected, analyzed, and interpreted :

• Expected results, and alternative approaches that will be used if unexpected results are found

• Discussion of potential difficulties and limitations and how these will be overcome or mitigated

DON’T SAY THERE WON’T BE ANY

Research Design and Methods

Suggestions

• Organize the sections in this part of the application to correspond to the numbers of the Specific Aims

• Give sufficient detail

DON’T ASSUME THE REVIEWER WILL KNOW HOW YOU INTEND TO PROCEED

Research Design and Methods

Suggestions

• Avoid excessive experimental detail by referring to publications that describe the methods to be employed

MIXING LEVEL OF DETAIL CAN BE CONFUSING

• Pull all the details on Methods together in the end of the proposal

Research Design and Methods

Suggestions

• Publications cited should be by the applicants, when reasonable

• Citing publication by others establishes that you know what method to use, but citing your own (or collaborator) establishes that the applicant personnel are experienced with the necessary techniques

Research Design and Methods

Suggestions

• If relevant, explain why one approach or method will be used in preference to others. This establishes that the alternatives were not simply overlooked

• Give not only the "how" but the "why"

Research Design and Methods

Suggestions

• When employing a complex technology for the first time, take extra care to demonstrate familiarity with the experimental details and potential pitfalls

• What have you (and your collaborators) done to establish the feasibility of what you are proposing to do?

Research Design and Methods

Suggestions.

• Add a co-investigator or consultant experienced with the technology if necessary

• Document proposed collaborations and offers of materials or reagents of restricted availability with letters from the individual involved

Research Design and Methods

Suggestions.

• Use reagents in hand, proposing to generate a reagent (e.g., knockout mouse lines) is dangerous

Research Design and Methods

Suggestions.

• Be sure to explain how the results to be obtained will be used to test the hypothesis

• Discovery science (genomics/proteomics) is harder than “Hypothesis-driven” science to support: Complete an initial discovery phase should lead to hypothesis.

Research Design and Methods

Suggestions.

• Include Timeline to demonstrate the objectives are attainable within the stated time frame

• Don't bite off more than you can chew. A small, focused project is generally better received than a diffuse, multifaceted project

Summary

You have succeeded if you demonstrate to the reviewer that YOU:

• Can design logical, well-controlled experiments

• Present your results in a clear and thoughtful manner

Summary

You have succeeded if you demonstrate to the reviewer that YOU:

• That you demonstrate what you (and your collaborators) have done to establish the feasibility of any new methods you are proposing to do in the proposal

Summary

You have succeeded if you demonstrate to the reviewer that YOU:

• Are experienced (hopefully published) with most of the experimental techniques proposed in the application

Summary

You have succeeded if you demonstrate to the reviewer that YOU:

• Have you clearly described how will the research be accomplished?

• Describe Who? What? When? Where? Why?

Conclusions

The assessment of the Preliminary Results and the Research Design and Methods will largely determine whether or not the proposal is favorably recommended for funding.

APPROACH > SIGNIFICANCE > INNOVATION