nwx-os-ogc-rkvl moderator: tammy bartasavich february 19 ... · moderator: tammy bartasavich...

26
NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 1 NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19, 2020 1:30 pm CT Tammy Bartasavich: Welcome this is the webinar for …FY2020 Teen Pregnancy Prevention (TPP) Tier 2 Phase II: Rigorous Evaluation of Promising Interventions Opportunity Number: PA-TP2-20-001 Objectives of this webinar are to review the goals, expectations, application content, submission and instructions of the Tier 2 Phase 2 rigorous evaluation of promising interventions. My name is Tammy Bartasavich. I’m a Project Officer at OPA and we will also have two other presenters today who are (Roscoe Bruno). He is from the Office of Grants Management and also another project officer at OPA (Elizabeth Laferriere). Our call logistics are you will be in listen only mode for the entire webinar. We will be taking questions at the end but we will ask you to submit them through the chat box. Call materials posted to OPA Web site in about a week after this webinar today and if you have other questions beyond the ones you put in the chat box please check the FAQs on our OPA Web site and you’ll see that on the screen. And otherwise you can submit to me, [email protected]. This is our agenda. We’ll start with the OPA overview and now I’m going to turn it over to (Roscoe Bruno) from the Office of Grants Management.

Upload: others

Post on 04-Jul-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CTConfirmation # 9835441

Page 1

NWX-OS-OGC-RKVL

Moderator: Tammy Bartasavich February 19, 2020

1:30 pm CT

Tammy Bartasavich: Welcome this is the webinar for …FY2020 Teen Pregnancy Prevention (TPP)

Tier 2 Phase II: Rigorous Evaluation of Promising Interventions

Opportunity Number: PA-TP2-20-001 Objectives of this webinar are to review

the goals, expectations, application content, submission and instructions of the

Tier 2 Phase 2 rigorous evaluation of promising interventions. My name is

Tammy Bartasavich. I’m a Project Officer at OPA and we will also have two

other presenters today who are (Roscoe Bruno). He is from the Office of

Grants Management and also another project officer at OPA (Elizabeth

Laferriere).

Our call logistics are you will be in listen only mode for the entire webinar.

We will be taking questions at the end but we will ask you to submit them

through the chat box. Call materials posted to OPA Web site in about a week

after this webinar today and if you have other questions beyond the ones you

put in the chat box please check the FAQs on our OPA Web site and you’ll

see that on the screen. And otherwise you can submit to me,

[email protected].

This is our agenda. We’ll start with the OPA overview and now I’m going to

turn it over to (Roscoe Bruno) from the Office of Grants Management.

Page 2: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 2

(Roscoe Bruno): Thank you Tammy. Good afternoon everyone. I must mention that for the Tier

2 Phase 2 FOA and currently for all three TPP FY agency listed here we’ve

encountered a brief systematic issue. We’re working to resolve this issue as

soon as possible. We’re attempting to search or view these three FOAs within

grants.gov. The FOA number currently begins with the prefix AH and not PA

as shown on this slide.

You may register for and begin application completion but while registering,

we highly encourage you to subscribe to grants.gov communications for alerts

when creating your profile. Doing so will alert you about important message

updates or system changes. Once this issue has been resolved our office will

send out a notification indicating so. Please note before applying for a funding

opportunity you need to register an account with grants.gov.

As listed above OPA currently has three open FY 2020 front end opportunities

for the TPP program. Today’s webinar is focused on the FY 2020 TPP

(Unintelligible) Phase 2 rigorous evaluation of promising interventions FOA.

As listed here you can see the estimated amount of funds available for this

competition as $5 million, the anticipated awards are five to ten. Please note

that this is very important. The total of period of performance is projected to

not exceed three years. The funding for the range of awards is for $500,000 to

$1 million per annual budget period. The anticipated start date is July 15 for

this year. Each annual budget period is projected to be 12 months in Atlanta.

The award type is a cooperative agreement which means there will be

substantial federal involvement during the course of the project.

The primary purpose and role of the funding opportunity announcement is to

provide information and guidance related to proper application submission.

Page 3: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 3

It’s very important for you to read the entire funding announcement and

follow the FOA currently. All information provided in FOA takes presence

over any conflicting information and other documents or references. And now

I’ll turn it back to Tammy.

Tammy Bartasavich: Okay. The goal of the TPP Tier 2 Phase 2 FOA is to rigorously evaluate

promising interventions that could contribute to adolescent optimal health,

preventing pregnancy and reduce sexually transmitted infection, STI rate. The

applicant is expected to evaluate intervention that already has project merit,

positive and preliminary evidence, readiness and feasibility. This funding will

support rigorous evaluation designed to answer important research questions

about the impact and implementation of the intervention.

Projects may conduct either efficacy or effectiveness studies that may be able

to complete and may - must be able to complete the project within a three year

period interventions, particular interest to OPA particularly interested in

promising interventions and populations and settings with greatest needs.

FOA page 9, in order to ensure health equity and promote the optimal health

OPA is particularly interested in rigorous evaluation of promising

interventions and populations and settings with greatest need and those

demonstrated significant health disparity. This may include but is not limited

to interventions and juvenile justice or foster care, child welfare setting with

expectant and parenting teen, youth with disabilities, youth - for homeless

youth or for caregivers. Additionally OPA is interested in projects that impact

not only individuals but also the system within which individuals are situated.

So now we’re going to review the expectations of this particular FOA. And if

you look at this slide these are the eight expectations of this - of funding

announcement and we’re going to go into these, each one with a little bit more

specifics. So expectation 1, meaningfully contribute to optimal health,

Page 4: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 4

preventing pregnancy and reduce STIs, demonstrate project merit and

preliminary positive evidence, the applicants are expected to meaningfully

contribute and show how the project has potential to improve optimal health

preventing pregnancy and reduce STIs.

OPA is especially interested in targeting resources to populations

disproportionately impacted by teen pregnancy and STI rate, to reduce health

disparities and to improve equity and optimal health. Interventions supported

by this funding are expected to be innovative and already have compelling

positive preliminary evidence from their formative evaluation including

effects on the target outcome. The intervention should also have widespread

support from participants and stakeholders and be a good fit for the population

and the setting.

In order to make a meaningful contribution interventions evaluated under this

setting are expected to have the potential to scale and to be replicated by

others. Applicants should demonstrate in the application the potential to scale

of the intervention upon project completion. Expectation 2, demonstrate

intervention readiness and feasibility. The applicant is expected to describe

the procedures for intervention, implementation and the training necessary for

staff.

In addition to making a meaningful contribution OPA expects applicants to

have all the intervention materials finalized, ready, accessible and available

for implementation. The intervention and supporting material should be fully

developed and applicants or their partners should have implemented the

intervention multiple times prior to their funding. Materials must also already

be medically accurate and age appropriate and should be trauma-informed,

user centered.

Page 5: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 5

Changes to the intervention should’ve already been made based on the

participant feedback. Procedures and activities of the intervention are

expected to be clear including mode of delivery, frequency duration, location,

setting and monitoring mechanisms. The applicant is expected to describe the

procedures for intervention, implementation and the training necessary for

staff. Applicants are expected to ensure feasibility of the implementation

within their application but also throughout the project period. This includes

but is not limited to a reasonable timeline and memorandum of understanding

or other agreements with partners to begin at the start of the project period.

This information will also be covered in the conduct high quality

implementation and capacity management expectation. And I just wanted to

mention too that any copywriting curriculum materials that are proposed in

the project must have signed agreements with the developer and it should

include but not limited to permission the materials as proposed in the

application or alter them as needed for compliance with that OPA medical

accuracy review.

Expectation 2, you must also demonstrate readiness and feasibility of the

evaluation.

The evaluation is expected to be ready and feasible at the time of the award.

Here is the (unintelligible) as mentioned and it’s very important to note that

you must have clear plans that are counter factual, comparison or control

group. Any services, programming or materials proposed for the counter

factual group must also be ready for implementation and accessible to the

applicant at the time of the grant award. Methods and procedures for

participant recruitment can (unintelligible) and study group formation should

be reasonable and feasible for the proposed project and the three year

timeline. All project activities including data collection, data analysis and

Page 6: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 6

reporting are expected to be completed within the three years which includes

dedicating the final six months of your funding project to analysis and

reporting.

Applicants are expected to propose a feasible sample size for their project to

detect impact and account for attrition or other potential loss of data. The

evaluation design should include power calculation based on the proposed

sample size. Evaluation designs are expected to be sufficiently powered to

project impact on the primary research question. Applicants are also expected

to have the IRB approval or waiver by the start of the project period and be

ready to begin their evaluation immediately upon receipt of funding.

All MOUs or agreements between the applicant and their key implementation

partners such as implementation side, their independent evaluators and

program implementers should be in place for the evaluation to begin

immediately at the start of the project. Applicants are expected to include all

of these fund agreements in the appendix of the application.

Expectation 3 is to conduct high quality implementation.

It is expected that during the project, the awardee will conduct high-quality

implementation of the intervention, including but not limited to: maintaining

fidelity, quality assurance, high dosage, intervention saturation, appropriate

materials, staff coordination and management, qualified facilitators,

participant engagement and satisfaction, and stakeholder support.

The applicant should describe the implementation settings, including partners

and the status of agreements with partners. High-quality implementation

should also be realized through adequate training and codified procedures for

the intervention and, if applicable, the counterfactual.

It is required that throughout the project the implementation would be

Page 7: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 7

medically accurate, age appropriate and expect that it will be trauma

informed, user centered as we mentioned earlier for both the project and the

counter factual. Additionally awardee is expected to use implementation

findings to improve the implementation and delivery of the project. Given the

implementation of research it’s likely to occur in an environment that has been

historically under resourced and marginalized.

There is an expectation to appropriately understand that context and the big

picture in which the issues of teen pregnancy and STIs exist. Systems thinking

can help applicants demonstrate an understanding of contextual factors,

relationships between those factors and how things can change over time. The

applicant should also explain an understanding of the context, the big picture

and how their intervention or research is situated. Additionally the applicant

should explain underlying assumption and inhibiting or enabling factors

related to the project.

Awardees throughout the project are expected to adequately consider the

complexity of the implementation environment. Expectation 4 is rigorous

evaluation. Rigorous evaluation can include either randomized control trial or

a quasi-experimental design. With the counter factual condition that should be

the most robust possible design that is feasible for the intervention.

Randomized control trials or RTCs are experiments that randomly assign

participants into different unique groups, the treatment or intervention group

or the control group or the counter factual.

A QED form intervention and counter factual groups through methods other

than random assigned participants. Evaluations supported by the funding can

be either efficacy or effectiveness study. Efficacy studies evaluate

interventions under ideal conditions often with close involvement of the

intervention developer whereas effectiveness study evaluate an intervention

Page 8: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 8

during a routine or actual condition. Applicants are also expected to conduct

an implementation evaluation based on implementation and science practices

and designed to answer questions on how and why an intervention does work.

Now to be more specific applicants are expected to describe the type of design

selected and indicate if it’s an efficacy or an effectiveness trial. You should

also describe the full study design including implementation and impact

evaluation. Research questions are expected to affect both behavior and

implementation outcome and clearly in line with the intervention and theory

of change. Primary and secondary questions should be determined and

evaluation designs are expected to be sufficient and well powered to detect

impacts for the primary research questions.

OPA expects that all studies will have a counter factual as we mentioned and

there will be baseline equivalency between the intervention group and the

counter factual. Additionally it’s expected for the treatment and counter

factual groups to be different enough to allow for a meaningful test. For

RCTs, the unit of random assignment is expected to be clearly defined and in

line with the unit of analysis. For QEDs the applicant is expected to describe

the formation of the counter factual. Differences between the group should be

described along with strategies to minimize potential contamination, should

the - and that should be a potential concern.

And to continue on with the rigorous evaluation - the intervention expectation

the applicant is expected to have data collection instruments that have already

been used or tested or administrative data that can be accessed. The applicant

is expected to describe having plans to estimate the casual impact of the

intervention, describe the minimal detectible impact as needed for the impact

and explain how the MDE or minimal detectible effect was calculated.

Awardees will collect and analyze and report on the data provided providing

Page 9: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 9

updates to stakeholders regularly.

The evaluation is expected to be conducted through a credible and neutral

process which includes a minimum evaluation step who do not have a

perceived or actual conflict of interest and can remain independent.

Okay, moving on to expectation 5, collaborate with partners, participants and

stakeholders. Given the purpose of the funding to improve optimal health of

you, preventing pregnancy and reduce STIs, it is important to collaborate and

maintain these partnerships with those involved in the project, including

partners, participants and stakeholders.

OPA expects that applicants will demonstrate how they have previously

engaged with participants and stakeholders and maintained a collaborative

environment and structured their approach accordingly. OPA does not expect

one organization to carry out this entire project. Rather the applicant and their

team partners should have collected expertise necessary to successfully

accomplish the goal and objective. Collaborative partners should - are

expected between the evaluator and the intervention team and with OPA.

The awardee is responsible for ensuring that all partners meet the expectation

successfully and fulfill their role. Expectation 6 is to monitor and improve the

project. OPA expects awardees to monitor the overall project based on key

findings, make improvements to the project. This could include improvements

to the evaluation or implementation. For example the monitoring

improvement plan should include performance measures for the overall

project. Furthermore awardees will use information guarded on the evaluation

monitoring and performance measures to determine the next steps for the

intermittent mentioned and project.

Page 10: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 10

This could include positioning the intervention for widespread dissemination

and scale or it could include changing the design of the intervention based on

findings and preparing for another type of evaluation. All recipients are

expected to collect a common set of performance measures to assess project

implementation and use the measures for monitoring and improving. And in

Appendix C of the FOA there is a drop set of performance measures. They’re

pending right now for OMB approval but it will give you an idea.

Okay, section - expectation 7. Communicate project activities and disseminate

results. Communicating information is not only important for this project’s

success, sustainability and to stakeholders. The public internal stakeholders

and external stakeholders should be informed about the activities of the

project. OPA expects recipients carefully document the results of the

intervention using that best practices for reporting such as the consolidated

standards of reporting trials or the transparent reporting of evaluations with

non-randomized design.

Recipients are expected to present results at local, state and national

professional and academic conferences and meetings and to share findings in

respected journals or other publications or OPA reports. And the very last

expectation, last but not least manage the project and ensure capacity. The

applicant should propose a team, partnerships and structure to successfully

accomplish all expectations of the grant. OPA does expect the applicant to

provide an overview of the different organizations involved along with the

key team members.

Key team members should have resumes with credential expertise and

capacity that align with their position. The applicant in particular should

clearly articulate who will serve as the evaluation and intervention leads and

include resumes with relevant experience. The projects should be conducted

Page 11: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 11

through a credible and neutral process which includes at a minimum

evaluation staff who do not have perceived or actual conflict of interest and

can remain independent from the intervention. We can’t say that enough.

A visual depiction of the project structure including key partners and role

should be involved. The applicant should outline how they plan to manage the

project. For all partnerships the application should include the signed MOUs

as we mentioned that describe how partners will contribute to the project,

clearly outline the rules, responsibilities and expectations. Awardees should

assess the professional development needs of staff on a regular basis and use

the results to develop a plan for providing ongoing professional development

and support for staff.

Awardees may use up to the first three months to complete project planning

and finalize agreements. However this funding is designed to support projects

that are ready, feasible, already have project merit and demonstrate positive

preliminary result. Thus, we will say it again. It is expected for awardees to

begin their projects upon award. Now I’d like to turn it back over to (Roscoe)

to discuss the application content.

(Roscoe Bruno): Thanks again Tammy. Now let’s discuss the application content portion of the

application. Application elements, this is a list of the five forms and three

uploaded file sections for submission of the application, the application for

federal assistance consists of form SF-124, the budget information for non-

construction programs, form SF-124A, insurance for non-construction

program, form SF-424B, disclosure of (unintelligible) activities, the form SF-

LLL and then there’s also a form called the Project Abstract (unintelligible)

form which is a national form within your application package and it must be

complete for submission for the application.

Page 12: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 12

The guidance for completion is referenced within the FOA. Three portions

which are the uploaded portions of the application, the project narrative, we

ask that you submit all project narrative content as a single acceptable file

(unintelligible) the budget narrative portion of the application submitted as a

narrative content as a single acceptable file and then the appendage C’s

portion. Submit all appendices content as a single acceptable file in the

application sections of your grants.gov application.

Application format, when considering application format be sure to follow the

(unintelligible) project narrative format instructed in the FOA. The application

will be disqualified if it does not conform to the formatting requirements. You

must double space the project narrative pages. You must use 12 point font.

You use easy, readable type such as Times New Roman or Arial. You may

single space tables or use alternate fonts but you must ensure the tables are

easy to read.

For appendices and budget narrative you should use the same formatting

specified for the project narrative. Appendix documents such as resumes may

use alternate fonts, common to such documents. Now I’ll turn it over to

Elizabeth to discuss the project narrative portion of the application.

Elizabeth Laferriere: Thank you (Roscoe). The project narrative is the most important part of the

application since it will be used on a primary basis again which we will

determine whether the project you are proposing meets the minimum

requirement for an award. The project narrative should include a clear and

concise description of your project and should include content which

addresses our three primary components which are project merit, contribution

and evidence, technical approach and project management and capacity.

As detailed information for each of these components can be found in

Page 13: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 13

(unintelligible) on pages 24 to 30 I will go quickly through these slides. For

the component of project merit, contribution and evidence successful

applications would contain all of the following information, including

addressing a specific problem for your project and what context does your

project take place and what are the historical, structural and environmental

factors that would impact your project and what are your underlying

assumptions for your project in what you plan to do.

Please do briefly describe your intervention, your theory of change and

evaluation design. And indicate why your project is important and what it will

contribute to the greater field. Please make sure you describe why your

particular intervention meets the needs of the population you serve and use

data directly from that population to make that case. Be clear as to why this is

an innovative approach project or intervention and explain what the research

context your project will build upon and why your evaluation is necessary.

Share the preliminary evidence of effectiveness that you have for your

intervention, for their positive effectiveness on the key outcomes of interest.

Explain how many times this intervention has been implemented already as

well as in what circumstances and how it’s evolved and improved over time.

And finally please share how your intervention is demonstrated to be

impactful, to be scaled and replicated to others and what barriers you would

need to address in order to do so.

This second component of your project narrative is your technical approach.

You’ll see in the funding opportunity now (unintelligible) technical approach

contains seven sub-categories that you should address and those are A through

G within the technical approach. The first is intervention and theory of

change. So is your intervention and theory of change ready for an evaluation?

What is the status of your intervention materials, your training, et cetera? Are

Page 14: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 14

they available, accessible, finalized and ready for use? Are they trauma

informed, medically accurate, age appropriate and user centered?

Please make sure you include a visual depiction of your theory of change as

an appendix. Next is the counter factual. Please provide details on your

counter factual, what is it, what services and programming, et cetera will the

counter factual receives? Are you ready to go with the counter factual? Are

there materials for the counter factual accessible and available and finalized as

well as well as trauma informed, medically accurate, age appropriate and also

user centered and how will your project will enter contract between the

intervention and the counter factual group.

The next slide is for technical approach is implementation. So how will you

implement the intervention and why? Have you done it this way before and

has it worked? What benchmarks and monitoring tools do you have to ensure

successful implementation? Do you have final signed agreements or MOUs

with all implementation partners that show you are implementation ready?

And is it clear that you have established ready to go partnerships?

Next is the evaluation planning which will make up a significant portion of

your technical approach. Rather than read through all of the questions in the

funding opportunity announcement I do recommend you carefully examines

pages 27 and 28 and I’ll highlight what’s here on this slide. Your evaluation

plan should show your design and describe the study. Describe and justify

your robust design, indicate your research question and how they’ll assess

both implementation and evaluation, address and describe statistical power,

explain how the counter factual group will be designed and how you plan to

minimize contamination. Include outcome measures and justification for

them, explain your data collection tools and processes and show how

measurement tools and instruments align with the research question.

Page 15: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 15

Next up is readiness and feasibility of rigorous evaluation. Please provide a

status update on IRB approval as well as what data instruments will be used

and whether they’ve been tested and developed already. If you’re already

using administrative data or plan to, do you know how you’ll get it? Have you

already received permission to do so? Do you have all your agreements in

place to conduct your evaluation and can your evaluation plan be successfully

completed within three years? Share your timeline.

Next how will you monitor your project? Share how often monitoring

activities will take place and the purpose of them. Also share how you will use

data to improve your projects and the processes you will put in place to

monitor and ensure ongoing project improvement. The final component of

Part G of the technical approach is collaboration, communication and

dissemination.

Please tell us how you will work with partners as well as participants and

other key stakeholders to maintain a collaborative environment and ensure

equity through engagement. How will you communicate about your project

and how will you apply best practices to disseminate your results? And finally

the third component for your project narrative is project management and

capacity. Please provide an overview of the implementing and evaluating

organization along with the organizational structure of the project. How will

the project be managed? Who are the partners if any?

What field and expertise do they contribute and how will they monitored?

Please be sure to attach all finalized MOUs and agreements and indicate the

status of agreements that have not yet been finalized. Tell us who your key

members are. Does the lead evaluator have relevant experience such as

activities like receiving IRB approval and going through the IRB process,

Page 16: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 16

conducting rigorous evaluation and publishing result? Does the

implementation staff have experience with the target population and to the

proposed intervention? Provide a brief summary list and include training,

qualifications and experience with similar projects and target population. In

addition please provide resumes for leadership and leader members on your

team such as your lead evaluator and lead interventioner.

How will you ensure neutrality and independence between the evaluation and

intervention? Are there any perceived or actual conflicts of interest? Please

address. And now I will turn it back over to (Roscoe) from the Office of

Grants Management to discuss the budget narrative.

(Roscoe Bruno): Thanks (Lizzie). Budget narrative and forms, the project budget information

includes the SF-424A as previously mentioned. This budgetary document is

completed as one of the five forms within the application package. The budget

narrative consists of a detailed nine item budget justification. In preparing the

budget narrative it must be consistent with the requirements as referenced in

the FOA. The budget costs must be reflective of the proposed activities

included in the application.

Please note the form, budget narrative and detailed budget justification do not

count toward the page limit. Again they do not count against the page limit.

The various budget line items, descriptions and justification requirements are

explained on pages 30 through 40 of the FOA. Its adjusted table format has

been included within the FOA. As required and mentioned on page 40 of the

FOA you must ensure the budget narrative section of the application includes

a written plan for oversight of federal awarded funds.

The appendices, appendices is a single uploaded portion of the application. It

must include the file that’s shown here on this slide, the work plan, the project

Page 17: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 17

timeline, organizational charts, visual depiction of the project, (unintelligible),

memorandum of understanding and letters of commitment or other

agreements. Detailed information for each component can be found on pages

40 through 42 of FOA. Appendices should count towards the total page limit

of the application and submit your appendices section as a single electronic

file in the attachment section on grants.gov.

Now let’s move on to the application submission section of your application.

Once again the application is due by 6:00 p.m. Eastern Standard Time on

Friday, April 17. The submission time will be determined by the date and

timestamp provided by grants.gov when you complete your submission. We

strongly encourage you to submit your application a minimum of three to five

days prior to the application due date. Grants.gov may take up to 48 hours to

notify you of a successful submission. If you fail to submit the application by

the due date and time we will not review it and it will receive no further

consideration.

As shown on this slide and highlighted on page 20 of the FOA the following

is a list of eligible applicants for this announcement. No requirements exist to

provide cost sharing or matching in the proposed budget. You obtain an

application package electronically by searching on the grants.gov Web site

using the following link that’s shown on this slide. There are multiple options

to search but you may search using CFDA number on page 2 of the FOA,

93.297. Once again we highly encourage you to subscribe to the

communications and notification section under your profile after registering

and account within grants.gov. This will allow you to receive any updates to

the FOA or if systematic changes or updates are made you’ll receive that -

those notifications.

Application submission, our offices require that all applications are submitted

Page 18: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 18

electronically via grants.gov unless an exception has been granted. If you

submit an application using any other electronic communication it will not be

accepted for review. You may access the grants.gov Web site portal for all

funding opportunities and grant application packages made available through

grants.gov. An application will not be considered valid until all application

components are entered into grants.gov and received by offices according to

their line and view dates as specified in the date section onto page 2 of the

FOA. Contact grants.gov for any questions or concerns regarding the

electronic application process. Use this phone number and email list that’s on

this slide.

All applications must be submitted as three files only. File 1 must contain the

entire project narrative. File 2 must contain the entire budget narrative

including supporting documentation as described in the budget narrative

content section of the FOA. File 3 must contain all documents and the

appendices uploaded in the attachment section of your grants.gov application.

The exceptions are only the required standard forms which do not apply to the

submission requirements as stated in the disqualification criteria.

Any files uploaded or attached to the grants.gov application must be of the

following file formats, Microsoft Word, Excel or PowerPoint, Adobe PDF or

images formats as shown here on the slide. We strongly recommend that

electronic applications be uploaded as Adobe PDF. If you convert the PDF

prior to submission it may prevent an unintentional formatting that might

occur with submission of an editable document.

As previously mentioned make sure the application is complete and do not

blank on forms unless the information is not applicable. Ensure the individual

submitting the application has the legal authority so that it’s an authorized

official who may act on behalf of the organization. To help ensure a

Page 19: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 19

successful submission of the application please follow the step by step

instructions provided at grants.gov as shown on the link in this slide. These

specific instructions are kept up to date and provide additional links to

frequently asked questions and other troubleshooting application submission

information.

The system for awards management, grants.gov will reject submissions from

applicants who have non-existing or expired system for awards management

registration. If you are registered - if you are registering a new entity in

SAM.gov you must mail an entity administration notarization letter to the

federal service desk or retention to SAM.gov registration processing. The

letter with all required details must be mailed. Your registration will not be

activated until the letter has been submitted and reviewed.

The minimal timeframe, completion and initiative to complete an initial

standardization is approximately 30 minutes. The standardization must be

reviewed every year. The approximate time (unintelligible) for an application

registration becomes active is up to ten days. You should check for an active

registration in SAM well before the application deadline. If you successfully

receive an award you must maintain an active SAM registration with current

info at all times during the active award period.

If you have not completed - comply with these requirements we may

determine that you are not qualified to receive an award and we may use that

determination as the basis for making an award to another applicant. If you

successfully complete and receive an award please note all first tier sub-award

recipients must have a DUNN number at the time you as the recipient make a

sub-award. On restrictions the liability, applicability, reasonableness and

assisting of direct expense, indirect costs may be charged to the grant in

accordance with the department regulations and policies in effect at the time

Page 20: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 20

of the award.

The current requirements can be found online via 45-CFR part 75 which is the

uniform administration requirement, cost principles and audit requirements for

HHS awards. Indirect costs may be included for 45-CFR part 75-4.14.

Applicants should indicate which method already is used for this application.

The current salary limitation for 2020 is $197,300 effective January 5 of this

year.

This is so you can see - these are located - these are application review

information criteria that’s shown on the slide that basically are posted here so

you can see what the factors are for the application review criteria

information. And this information is located on page 50 of the FOA with

explicit detailed information.

The application disqualification criteria, you must submit electronically via

grants.gov by due date and time unless an exception has been granted two

business days prior to the deadline. If you successfully submit multiple

applications for the same project we will only review the last application

received by the deadline. The Office of Grants Management may deem your

application eligible. There could be a disqualifying criteria. Your project

narrative must be double spaced or the equivalent of 8 1/2, 11 page size with 1

inch margin on all sides and fonts not less than 12 point.

The project narrative must not exceed 50 pages in your federal including

indirect cost must not exceed the maximum amount indicated and the award

(unintelligible) $1 million. (Unintelligible) award requested to include indirect

cost is not below the minimum indicated in the award full amount of

$500,000. The application must meet the application responsiveness criteria.

The total application including project merit must not exceed 100 pages.

Page 21: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 21

Application review, the applicants that lack the required support of the

application or submit additional appendices files will not be disqualified from

competitive review. However it may impact your application scoring under

the evaluation criteria. Be sure to follow submission instructions carefully. All

eligible application reviews are scored by panel and independent reviewers

with technical expertise in the applicable fields according to criteria listed in

the announcement. Objective review committee process is formal and

confidential so staff members are available for questions and to ensure the

process is consistent and fair but do not participate in the discussion and

scoring.

Applications are also reviewed by the Office of Grants managed there for

administrative and business compliance and by the Office of Population and

Affairs staff for programmatic compliance.

Funding decisions, the director of Office of Population Affairs will make all

final award selections to be recommended to the grants management office of

solution analysis. In making a decision the following additional considerations

would be reviewed, representation of the various interventions, representation

of a range and populations disproportionately impacted by teen pregnancy,

(unintelligible) of research questions. Funding process, please note we’re not

obligated to make any federal awards as a result of this announcement. Only

the grants officer may bind the federal government to the expenditure of funds

and it is attritional signatory for all grant business.

If you receive communication negotiate an award request for additional or

clarifying information, this does not mean you will receive an award. It only

means that your application is still under consideration. All award decisions

including level of funding if an award is made or found and you may not

Page 22: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 22

appeal. Review for risk, we’ll evaluate each application in the fundable range

for risk posed by the applicant before issuing an award in accordance with 45-

CFR part 75.205.

We’ll use a risk-based approach and may consider any items such as the

following as stated in FOA the applicant’s financial stability, the quality of

management systems and ability to meet the management standards this year

the performance, the applicant’s record in managing federal awards including

timeliness of compliance, applicable reporting requirements, performance up

to determine conditions of previous federal awards, reporting and findings

from audits performed and the applicant’s ability to effectively implement

statutory, regulatory or other requirements imposed on non-federal entity.

Notice of Award will be issued to notify this list or applicant a list selection.

The award amount, project and budget periods, those include any conditions

on the award include requirements that must be met as condition of receiving

a grant fund. It will include standard terms, reporting requirements and

contact information for OGM and the program office. If we do not make an

award to you because we determine the organization does not meet either or

both of the minimum requirement standards as described in 45-CFR part 75

we must report that determination to FAPIIS which is the Federal Award

Performance and Integrity Information System.

These certain conditions apply. At a minimum the information is a system. If

you are a prior federal award recipient must demonstrate the satisfactory

record of executing programs and that can be under federal grants, cooperative

agreements or the cumulative award in integrity and business ethics. This

information will then be available for other organizations to review when

considering you for an award.

Page 23: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 23

So in the funding process the Office of Grants Management is the official

contact for the grantee. All official communication related to the grant is

between the grants management officer and the successful applicant. The

program office will notify all unsuccessful applicants. And now I’ll turn it

over to (Lizzie) to mention some application tips.

Elizabeth Laferriere: Please pay extremely careful attention to the application qualification for

Tier as (unintelligible) covered on slide 53 and also on page 51 of the funding

opportunity announcement. And make sure that your application is not

disqualified from being reviewed. Do not wait until the last minute to begin

your electronic submission and electronic submission is required. Do not wait

as well until the last minute to begin SAM registration or update your

registration in that system. And please use this opportunity to demonstrate

your own deep familiarity with the issues and do not simply regurgitate data

and content from the funding opportunity award in your - funding opportunity

announcement in your narrative.

Be clear, complete and concise in the project description flow and address

exactly what is requested in the funding announcement. Please don’t make the

reviewer search for the required information. Generally the user of the

application is to review the letter of the score. Clearly identify the section of

the application and indicate which component is being addressed there. Pay

careful attention to page limits.

Remember the project narrative must include all required information within

the designated limit. Do not appendices as a way to extend that page limit.

Please make sure that you have clear goals and ensure that your objectives are

as much or specific, measurable, achievable, realistic and time framed

activities presented in the work plan should relate directly to the proposed

goal and objective for your project. The program work plan, your evaluation

Page 24: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 24

plan and budget should all provide a complete picture of how the applicant

will address the service area needs as well as address the purpose and

expectations within the program announcement.

Applicants should ensure that staffing for the project is appropriate and

reasonable for the goals, objectives and activities that are proposed. Please be

sure to be complete in describing the expertise required, what staff will do,

who will do it and what percentage of time of staff will be assigned to the

project. Applicants can include position description and biographic sketches

for key staff in their appendices. Let me turn it back over to (Roscoe).

(Roscoe Bruno): Thanks (Lizzie). Let me briefly mention your budget should include the

required funds to carry out your proposed work plan, evaluation and

administrative responsibilities of the project. It should be reasonable and

related directly to the goals and objectives of the project and program. Do not

request multiple funds that are available as listed in the funding

announcement. Again this amount is inclusive of both direct and indirect

costs.

Operating budget should be complete and include the federal, non-federal

fund as applicable, projected program income from fees or third-party payers

and other contributing funds. And now I’ll turn it over to Tammy with some

last mentioned to close out this webinar.

Tammy Bartasavich: I just wanted to point out the sites for program questions. You can email

me Tammy Bartasavich at HHS dot gov and for any financial questions or

administrative questions you would email (Roscoe). And I know for myself

probably email is the best way to do it. I’m not sure about you (Roscoe) but

our phone numbers are there.

Page 25: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 25

(Roscoe Bruno): Email.

Tammy Bartasavich: Email is actually the preferred way for us. So the emails are there on this

slide. Also if you have any inquiries I think we have already mentioned this as

well, Grants.gov applicant support, www.grants.gov, email support at

grants.gov and that’s the telephone number for you. Now we are going to ask

you if you have any questions. Please type them into the chat box and we will

collect all of those questions and we will have the answers to them posted in

approximately a week., they will be on the OPA Web site. So go ahead and

start typing your questions in the chat box and while you’re doing that I can

read some FAQs and provide some answers to some of them we’ve already

received.

Does the funding announcement require a letter of intent? The answer is no.

The funding announcement does not require a letter of intent so that is not

something that you have to worry about. Will special consideration be given

to organizations that have previously received TPP funding? Answer to that is

no. Eligible applicants for this funding include any public or private entity

including faith-based, community based and Indian tribe or tribal

organizations. You can find more information on that on page 20 of the FOA.

Can an organization submit more than one application to the same FOA? If an

applicant submits more than one application with the same content, example

you submit your application then you make an update and submit the updated

application at a later time we will only review the last application received by

the deadline. If an individual or organization submits more than one

application for different projects, different content, different programs,

different communities all submitted applications will be reviewed as the

project described is unique.

Page 26: NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich February 19 ... · Moderator: Tammy Bartasavich 2-19-20/1:30 pm CT Confirmation # 9835441 Page 3 . It’s very important for you to read

NWX-OS-OGC-RKVL Moderator: Tammy Bartasavich

2-19-20/1:30 pm CT Confirmation # 9835441

Page 26

Okay. When do I need to begin my evaluation? The answer is awardees may

use up to the first three months to complete project planning and finalize

agreement. All applicants are expected to justify their intervention is ready for

rigorous evaluation and their rigorous evaluation would be ready to start

within the first three months of the project period and that is on page 9 of the

FOA. And I think I’ll do one more question and then we will collect your

questions as I said and you can look on the Web site to get those answers.

So what is optimal health? Optimal health is a dynamic balance of physical,

emotional, social, spiritual and intellectual health. Lifestyle change can be

facilitated through a combination of learning experiences that enhance

awareness, increase motivation and build skill and most important through the

creation of opportunities that open access to environments that make positive

health practices the easiest choice. And that definition is on page 73 of the

FOA as well.

I wanted to give you our Web site. It is www.hhs.gov\opa. And that is where

the answers will be for the questions that you submitted. Okay once again the

applications are due Friday, April 17 this year 6:00 pm Eastern Time. These

are our Web sites and contact information. So thank you all for coming and

good luck with your application. And that concludes our webinar for today.

Thanks everyone.

Coordinator: That concludes today’s conference. Participants you may disconnect at this

time. Speakers please allow a moment of silence for your post conference.

END