topic ii: good practice guidance on the basic steps of the ... · topic ii: "good...

Topic II: "Good practice" guidance on the basic steps

of the review process

Presented at the:

REFRESHING SEMINAR ON GOOD PRACTICE APPROACHES TO INVENTORY

ISSUES IDENTIFIED DURING THE REVIEW PROCESS

Kiyoto Tanabe, Sandro Federici, Simon Eggleston, Jongikhaya Witi

Introduction

This presentation is to consider good practices in each stage

of the UNFCCC inventory review process

Good practices in the preparatory phase (before the

review week)

Good practices in the review week

Good practices after the review week

Consideration is given also to:

What should the lead reviewers do at each stage?

What should each reviewer do at each stage?

• Remember “each reviewer” includes lead reviewers! !

Good practices in the preparatory phase (before the review week)


What should the lead reviewers do?

Facilitate communication among experts

Plan for the review week

Give consideration to how to help new (not experienced) reviewers

Provide basic guidance to experts by:

Reminding them of the materials available

Highlighting issues particularly important

• e.g., new conclusions from the latest LR meeting

Encouraging them to prioritize categories to be reviewed

These are additional to a “normal” technical preparation (study of

materials) that each reviewer should do!!


What should each reviewer do?

(What guidance should LRs give to them?)

Examine materials available (To do this efficiently,

prioritization of categories is needed.)

Status Report

S&A Part I & II

Previous review reports

Previous review transcript

NIR & CRF

KP-CRF

SEF & SIARs

These are useful to prioritize

categories to be carefully

reviewed.

For efficiency’s sake, priority

should be given to the

categories that are deemed

particularly important.




Identify all categories that are reported as: Not estimated (NE)

Not occurring (NO)

Not applicable (NA)

Included elsewhere (IE)

This can and must be done at first, well in advance of review week.

It does not require much time – can be done easily using Status Report

and CRF (summary tables for each sector).

It is often the case that these categories have potential problems

(underestimation of emissions).

Availability of IPCC methods for them should be also checked.

This is important to make judgments about potential problems.

Relevant case studies will be discussed in Topic III.




Prioritize categories and consider appropriate time allocation.

Gases/categories that could be prioritized are:

Key categories

Categories for which the previous ERT(s) made

recommendations

Recalculated categories

Categories for which S&A Part II identifies potential issues

Deviation from Revised 1996 IPCC Guidelines and GPG should

be assessed for these categories.

Relevant case studies will be discussed in Topic III.




These preliminary assessments will help identify “potential

problems” already before the review week.

Questions based on the findings from these preliminary

assessments should be sent to the Party prior to the review.

It is quite important to do these assessments and to send

questions to the Party well in advance of the review week.

It significantly benefits both the ERT and the Party.

• The ERT can make more efficient and deeper examination of

the issues.

• The Party can have more opportunities to provide better

clarifications.

Eventually, it will reduce difficulties in the ERT work on SP!!


Categories to be prioritized – those with previous

recommendations

The lead reviewers agreed at the 8th LR meeting:

High priority needs to be given to previous recommendations in

the early stages of an individual review, particularly during the

review experts’ preparation for the review and in the first days of

the review week.

Where previous recommendations relating to language of a

mandatory nature in the reporting guidelines have not been

addressed by Parties after several years, the ERTs should

consider whether this should lead to the identification of a

potential problem, which if not resolved may lead to a question

of implementation, for Parties being reviewed under the KP.

“Previous review recommendation template” should be used –

it will help you focus on key problems at later stages of review.


Categories to be prioritized – recalculated categories (1)

They deserve special attention because they have NOT been

reviewed by any previous ERTs!!

Need to check:

Impact/Result

• Increase or decrease in recent years/base year?

• TACCC improved? (e.g., uncertainties reduced?)

Reason/Justification

• To follow previous recommendations?

• As errors have been found in the previous estimates?

• As new data/information have become available?



All years of the commitment period that have been

recalculated should be carefully checked.

Remember - the figures recalculated will replace the earlier

reported values for the commitment period years and will be

entered in the accounting of Annex A sources for KP.

Particular attention should be given to recalculations on

categories that have been adjusted in the past.

Sometimes consideration may need to be given to changes

in methods or assumptions as compared to the KP initial

report.



The lead reviewers agreed at the 6th LR meeting that:

“… when changes in methodologies are identified …, the

ERTs should ascertain whether:

Such changes are carried out only to improve accuracy,

completeness and/or time-series consistency, and are well

justified and documented in accordance with the UNFCCC

reporting guidelines and the IPCC good practice guidance;

Recalculations, including the recalculation of base year

emissions, are performed with a view to having a consistent

time series. However, such recalculations will not affect the

assigned amount established for the Party concerned following

the initial report review under the Kyoto Protocol.



At first, explain the plan (timetable) for the review week. In particular,

emphasize by when potential problems need to be detected.

Ideally, most potential problems have been already detected through the

preliminary assessments before the review week.

Organize team meetings at least once a day.

Check the progress of each reviewer’s work

• As concretely as possible – making it clear what has been done

according to schedule and what has not.

• Suggest adjusting or fine-tuning the work schedule

Discuss issues

• Particularly those associated with “potential problems”

• Advise NOT to stick too much to minor issues that are not required

by IPCC Guidelines, GPG or UNFCCC Guidelines

Reviewers sometimes stick to issues of their personal interest that are

not important in the context of inventory review under UNFCCC/KP.



Encourage discussion among reviewers.

Between generalist and sector reviewers (particularly on

issues relating to NS)

Between different sector reviewers (to discuss cross-

cutting issues)

Between reviewers for the same sector (only in

centralized review)

Between experienced reviewers and new reviewers

Take the lead in preparation of Saturday Paper.




Review the inventories!! (Raising questions to the Parties)

Produce review transcripts and zero-order-draft of review reports.

Discuss with other reviewers in the team on cross-cutting issues.

Generalist and sector reviewers

Energy and IP (e.g., issues on non-energy use of fossil fuels,

particularly carbon mass balance associated with iron & steel

production)

Energy and Waste (e.g., issues on energy use of combustion of

wastes or recovered methane)

Agriculture and Waste (e.g., issues on sludge applied to cropland)

LULUCF and Agriculture (examples shown later)

Give careful consideration to all the potential problems for Saturday Paper.


Examples of cross-cutting issues that need to be jointly

addressed with other reviewers


TACCC TACCC (particularly, transparence & completeness

& consistency) of each sector needs to be

assessed by sector reviewers, and informed to

generalist.

Key category analysis There may be inconsistency in the description

about key categories within the NIR and/or in the

NIR and CRF . It should be checked jointly.

Reviewers should understand/asses why, if any,

there are differences between KCA by the Party

and KCA by the secretariat, in particular when both

are based on tier 1 approach.

Uncertainty analysis There may be inconsistency in the description

about key categories within the NIR. It should be

checked jointly.





QA/QC It is necessary to review whether QA/QC (in

particular Tier 1 QC) is implemented consistently

throughout the inventory. This needs to be checked

jointly.

It is desirable that the role of sector reviewers and

the generalist in the review of QA/QC be clarified

before the review start.

Reviewers should clearly understand what QA is,

what QC is and what verification is.

Any other issues

relevant to NS

See the case study


What are QA, QC and verifications…?

Quality control (QC) is a system of routine technical activities to

measure and control the quality of the inventory as it is being

developed. The QC system is designed to:

i. Provide routine and consistent checks to ensure data integrity, correctness

and completeness;

ii. Identify and address errors and omissions;

iii. Document and archive inventory material and record all QC activities.

Quality control activities include general methods such as accuracy

checks on data acquisition and calculations and the use of

approved standardized procedures for emission calculations,

measurements, estimating uncertainties, archiving information and

reporting. Higher tier QC activities also include technical reviews of

source categories, activity and emission factor data and methods.

[Paragraph 3(b) in Annex to Decision 19/CMP.1]



Quality assurance (QA) activities include a planned system of

review procedures conducted by personnel not directly involved

in the inventory compilation development process, to verify that

data quality objectives were met, ensure that the inventory

represents the best possible estimate of emissions and sinks

given the current state of scientific knowledge and data available,

and support the effectiveness of the QC programme.

Verification refers to the collection of activities and procedures

that can be followed during the planning and development, or

after completion of an inventory that can help to establish its

reliability for the intended applications of that inventory. E.g.,

Comparisons with estimates made by other bodies

Comparisons with emission and uptake measurements determined from

atmospheric concentrations or concentration gradients of these gases

[Paragraph 3(b) in Annex to Decision 19/CMP.1]

[Chapter 8 and Annex 2 of IPCC Good Practice Guidance]



Among others, the ERT should note that implementation of

Tier 1 QC is a mandatory obligation by the Party (under KP).

What is the Tier 1 QC according to the IPCC GPG…?

It is general inventory level QC procedures.

Consult Table 8.1 in Chapter 8 of IPCC GPG!!

[Paragraph 14(g) in Annex to Decision 19/CMP.1]




LULUCF and Agriculture sector reviewers

N2O emissions from

histosols in cropland

and managed grassland

I. Have these N2O emissions been properly reported under

the relevant category (cultivation of histosols) of the

agriculture sector?

Note that countries may have reported N2O emissions from

drained grassland under LULUCF instead of Agriculture

while it is good practice to report all N2O emissions due to

cultivation of soils (with intensive or extensive practices) in

table 4.Ds1 of agriculture sector.

A check to be done is that the sum of organic soils areas

reported under cropland (table 5.B) and managed

grassland (table 5.C) equals the area of cultivation of

histosols reported in table 4.Ds1.





N2O emissions

associated with land-use

conversion

II. Have N2O emissions from mineralization of organic matter in

organic soils under land use conversion, to cropland and

managed grassland, been reported under the relevant

category (cultivation of histosols) of the agriculture sector?

Note that tables 5(III) and 5(KP-II)3 contain rows to allow

reporting N2O emissions from mineralization of soil organic

matter in organic soils under land conversion; however

emissions should not be reported here..

Direct N2O emissions

from N fertilization

III. Have N2O emissions from fertilization of tree plantations not

included in forest land been reported under the relevant

category (Synthetic Fertilizers) of the agriculture sector?

Note that it could happen that in a country’s GHGI some tree

plantations are reported under forest land use and some

other under the cropland use . All emissions due to N

fertilization applied to tree plantation in cropland have to be

reported under the agriculture sector;.

Note that all emissions from manure deposited by animals

grazing in forest land have to be reported under the

agriculture sector (Animal Manure Applied to Soils).





N2O and CH4 emissions

from prescribed burning

of biomass occurring in

cropland and managed

grassland

IV. Have these N2O and CH4 emissions been reported in the

relevant categories (prescribed burning of savannas,; field

burning of crop residues) of the agriculture sector?

Note that tables 5(V) and 5(KP-II)5 contain rows to allow

reporting N2O and CH4 emissions from prescribed burning in

cropland and in managed grassland; however emissions

should not be reported here.

Use of organic fertilizer V. Is there any incoherency among data reported on organic

fertilization, if any, of managed forest land, cropland and

managed grassland and data on management of manure?

Consistency with

assigned amount

calculation

VI. Is there any inconsistency in emissions allocation between

the GHGI of the first report under the Kyoto Protocol and the

current GHGI?

Note that an emission reported under the agriculture sector

in the GHGI of the first report under the Kyoto Protocol

should be consistently reported under the agriculture sector

for the whole CP even if it should have been reported under

the LULUCF


Preparation of Saturday Paper (1)

All the potential problems that could lead to “adjustments” under

Article 5.2 of KP or “Question of Implementation” should be sent to

the Party on the last day of review week (= “Saturday Paper”)

Underestimation of emissions, Overestimation of removals

Lack of transparency which prevents ERT from being convinced of no

underestimation of emissions or no overestimation of removals

Failure of implementation of mandatory functions of NS

This is ERT’s collective responsibility. Team meeting should discuss:

What is the problem?

Is it really a serious problem? – The rationale is clear?

What recommendation should be given to the Party?

• To help the Party resolve the problem, and

• To eventually help the ERT itself make the review more successful and less

stressful. (The necessity of applying adjustments will be reduced.)




Recommendation should:

mirror the key points of the problem identified

(in other words, the ERT should point out specifically what are the key

points of the problem as clearly as possible)

be clear and specific with regard to what the ERT expects to receive

from the Party to convince them that the potential problem is resolved

Good practice in making advice/recommendation is to make

reference to specific reporting requirements and/or to specific

guidance given in the IPCC Guidelines and GPG.

For NS problems, make reference to specific paragraphs in Decision

19/CMP.1. --- Case studies were discussed in Topic I.

For missing estimates or underestimation of emissions, make

reference to specific pages/chapters and/or equations in IPCC

Guidelines or GPG. --- Case studies will be discussed in Topic III.



It is often stressful - Time available is short while judgment requires

careful consideration.

It can be more successful and less stressful by:

Early examination of materials before the review week

Sharing findings with ERT at an early stage during the review week

• In many cases, problems in NS can be found through discussion on specific

problems that are commonly found across sectors.

Good preparation for Saturday discussion

• LRs should keep ERT members aware of specifically what issues are going

to be discussed on Saturday. (The list of issues needs to be emphasized and

updated at the team meeting every day.)

• Each reviewer should request the Party to swiftly provide answers to the

questions. (In principle, all answers should be obtained by Friday evening.)

• Each reviewer should give his/her own consideration to each issue before

Saturday discussion.



Keep watching the progress of each reviewer’s work, and

encourage them to produce draft reports in a timely manner.

Read carefully zero-order drafts prepared by each reviewer, and

provide comments.

Check the revised drafts by each reviewer to see if and how

comments from LRs and the secretariat are treated and if

something crucial is missing.

Advise reviewers to follow the sequence in drafting paragraphs:

i. identification of the issue (with details, e.g. values, %, etc)

ii. assessment of the issue, why is this an issue (TCCCA)?

iii. recommendation/encouragement consistent with the assessment

of the issue.




Write the draft review report!!

Also, complete other review outputs such as review transcript.

Consider which is more appropriate - “recommend” or

“encourage” for each issue.

Ensure that each recommendation/encouragement provides

the Party with advice on a way forward.

Difference between “recommend” and “encourage” in the review report

“Recommend” – for issues related to mandatory requirements (“shall”

requirements in UNFCCC Annex I reporting guidelines and KP decisions) ;

“Encourage” – for issues related to non-mandatory requirements (“should”

requirements in UNFCCC Annex I reporting guidelines and KP decisions)

Case study:

Tier 1 methods are used for several key categories – is it a serious

problem in National System?

Case study

Case study: Tier 1 methods are used for several key categories – is it a serious problem in NS?

Stage Action

Before the

review week

Each sector reviewer looked at key categories and

previous recommendations and noticed: Tier 1 methods are used for some KCs in all sectors.

For all such KCs, the previous ERTs recommended the

Party collect necessary information/data to apply higher

tier methods.

However, those recommendations do not necessarily

contain concrete advice to obtain such information/data.

Each sector reviewer sent questions about this issue

to the Party.

Review week

Day 1 Team meeting

This issue was shared with ERT members.

The ERT felt that this might imply a serious problem.

LRs reminded the ERT that Tier 1 could be used for

KC in line with GPG depending on the Party’s national

circumstances.

LRs encouraged each sector reviewer to further

investigate into each of such KCs and to discuss with

generalists (in charge of NS review) about this issue.

Case study


Stage Action

Review week

Day 2

Each sector reviewer obtained answers from the Party.

However, most of the answers are not satisfactory. Simply saying “efforts are being made to collect data”

without any details.

Each sector reviewer further asked for information on: specifically what kind of efforts are being made

when those data will be made available

After short discussion with sector reviewers, Generalist

noticed that: The Party has its inventory improvement plan.

However, the NIR contains only a summary of the plan.

Generalist requested the Party to provide the inventory

improvement plan.

Review week

Day 3

Generalist obtained the inventory improvement plan

from the Party.

Case study


Stage Action

Review week

Day 4

Each sector reviewer obtained additional answers from the

Party. Most of them contained information on: specifically what kind of efforts are being made: e.g.,

• Measurement programmes to develop CS-EFs

• Consultation with data holders (e.g., private companies)

when those data will be made available (e.g., by 2012)

• According to their plan, all the relevant improvements

will be completed by 2013 (= before the end of 1st CP)

Generalist and each sector reviewer jointly looked into the

inventory improvement plan and the answers to each

sector reviewer, and found: For most of the KCs in question, the improvement plan and

the answers given by the Party were consistent.

For one KC, they were inconsistent.

For one KC, the information given by the Party was not

included in the improvement plan.

Case study


Stage Action

Review week

Day 4 Team meeting

The findings from the joint work by Generalist and each

sector reviewer were shared at the team meeting.

After discussion, the ERT came to provisional

conclusion: It is not likely that this issue is a serious problem (=

potential “Question of Implementation”), because the

Party almost successfully clarified: • Higher tiers cannot be used for the KCs in question at

present due to lack of data.

• Specific efforts are actually being made to collect necessary

data for each KC in question.

However, there are some problems such as: • Lack of transparency in the NIR

• Inconsistency between inventory improvement plan and the

answer

LRs noted this provisional conclusion, and suggested Generalist and sector reviewer further examine the

detected inconsistency.

The ERT discuss this issue again on Saturday to come up

with final conclusion.

Case study


Stage Action

Review week

Day 5

Generalist and sector reviewer communicated with the

Party about the detected inconsistency, and got

confirmation that: The answers to sector reviewer’s question were correct.

The inventory improvement plan was not fully updated.

Review week

Day 6 Team meeting

on Saturday

Paper

Generalist shared with the ERT the confirmed fact

obtained from the Party yesterday.

The ERT re-confirmed the conclusion provisionally

made on Day 4, and finally concluded that this issue

should not be included in the Saturday Paper.

The ERT also agreed : Generalist should raise in the review report the issues on:

• Need of updating the inventory improvement plan

• Need to improve transparency of NIR

Sector reviewers should mention in the review report on:

• Need to improve transparency of NIR

• Need to implementation of the planned

improvements. (ASAP in view of KP 1st CP)

Case study


Stage Action

After review

week

Generalist included in the draft review report the

encouragements for the Party:

To keep the inventory improvement plan updated

To elaborate more on the improvement plan in the

NIR (Particularly, to include explanation of how

KCs will be improved)

Each sector reviewer included in the draft review report

the encouragements for the Party:

To include explanation of on-going efforts for data

collection in the relevant chapter in the NIR

To implement the specific improvement plan and

apply higher tier methods for the KC in question.

LRs advised generalists and sector reviewers to use

“strongly recommend” rather than “encourage” for these

issues.

Generalists and sector reviewers agreed with LRs, and

changed the words to “strongly recommend”.

Summary

Summary

<Before the review week>

Early examination of materials available before the review

week, focused on prioritized categories, will help efficient

review during the review week.

Identification of all categories reported as NE/NO/NA/IE

Examination of deviation from IPCC methods

It is quite important to do these assessments and to send

questions to the Party well in advance of the review week.

Ideally, most potential problems should be already detected

through the preliminary assessments before the review week.

Summary

<In the review week>

Cooperation and communication among ERT members during the

review week is essential to successful review.

Sharing important issues with ERT members at an early stage is

important particularly to detect potential problems in National System.

Daily team meeting led by LRs is important.

To check the progress of each reviewer’s work

To keep the ERT work on the right track (to avoid deviation from major

problems)

To better prepare for Saturday discussion

Summary

<In the review week>

Preparation of Saturday Paper is ERT’s collective responsibility.

Team meeting should discuss:

What is the problem?

Is it really a serious problem? – The rationale is clear?


• Recommendation should be clear and specific with regard to what the

ERT expects to receive from the Party.

• Making reference to specific reporting requirements and/or to specific

guidance given in the IPCC Guidelines and GPG will be helpful.

<After the review week>

Produce the draft review reports and other materials in a timely

manner!

Make recommendation/encouragement clear and consistent with

the assessment of the issues.

topic ii: good practice guidance on the basic steps of the ... · topic ii: "good...

Documents