hybrid means testing pilot sarp: social assistance reform project janusz szyrmer irena topinska...

Hybrid Means Testing Pilot

SARP: Social Assistance Reform Project

Janusz SzyrmerIrena Topinska

Dmytro BoyarchukViacheslau Herasimovich

Kyiv, March 25, 2009

BITALLY

Excerpts from SARP’s Terms of Reference (1): Excerpts from SARP’s Terms of Reference (1): ProblemsProblems

1

During the past several years, while developing this project, the During the past several years, while developing this project, the MOLSP has conducted a number of analytical surveys and Pilot MOLSP has conducted a number of analytical surveys and Pilot Projects that have revealed many unsolved problems, which Projects that have revealed many unsolved problems, which include: include:

A lot of time and effort are required for processing applications, A lot of time and effort are required for processing applications, verifying documents and determining SA (social assistance) verifying documents and determining SA (social assistance) eligibilityeligibility

Frequent changes are made in the level of various social Frequent changes are made in the level of various social assistance benefits and the rules for eligibilityassistance benefits and the rules for eligibility

Different criteria are used to evaluate family income status in Different criteria are used to evaluate family income status in determining its eligibility for different types of SAdetermining its eligibility for different types of SA

Excerpts from SARP’s Terms of Reference (1): Excerpts from SARP’s Terms of Reference (1): Problems Problems ((cont.cont.))

Lack of public access to information about the current Lack of public access to information about the current system of social protection – either about the legislative system of social protection – either about the legislative basis of SA or the administrative procedures used to basis of SA or the administrative procedures used to process applications process applications

SA programs do not fully cover the poorest; available SA programs do not fully cover the poorest; available financial resources appear to be used in an unsatisfactory financial resources appear to be used in an unsatisfactory manner and programs do not meet the poverty reduction manner and programs do not meet the poverty reduction targettarget

2

Excerpts from SARP’s Terms of Reference (2): Excerpts from SARP’s Terms of Reference (2): PilotsPilots

Prior to taking decisions on changes in the mechanisms of provision Prior to taking decisions on changes in the mechanisms of provision of SA, the Government wants to study their social, economic, and of SA, the Government wants to study their social, economic, and fiscal impacts by implementing pilot projects in selected regions of fiscal impacts by implementing pilot projects in selected regions of Ukraine:Ukraine:

Pilot #1: Cashing out housing subsidies based on standard Pilot #1: Cashing out housing subsidies based on standard ratesrates

Pilot #2: Cashing out housing and utilities privilegesPilot #2: Cashing out housing and utilities privileges

Pilot #3; Introducing uniform households means testing criteria Pilot #3; Introducing uniform households means testing criteria for granting all types of SA, and improving the methods of for granting all types of SA, and improving the methods of calculation of family’s income calculation of family’s income

3

Excerpts from SARP’s Terms of Reference (2): Excerpts from SARP’s Terms of Reference (2): Pilots Pilots ((cont.cont.))

Pilot #4: Strengthening the institutional capacity and legal Pilot #4: Strengthening the institutional capacity and legal framework of social inspectors, and broadening their framework of social inspectors, and broadening their functions and competences functions and competences

Pilot #5: Ensuring interaction between pilot SA offices and Pilot #5: Ensuring interaction between pilot SA offices and employment centers in terms of record keeping, employment centers in terms of record keeping, employment and skills upgrade of the able-to-work employment and skills upgrade of the able-to-work members of socially vulnerable familiesmembers of socially vulnerable families

4

Excerpts from SARPS’s Terms of Reference (3): Excerpts from SARPS’s Terms of Reference (3):

TasksTasks

5

1.1. Develop recommendations concerning the mechanisms to be Develop recommendations concerning the mechanisms to be implanted within the framework of the Pilot Projects based on implanted within the framework of the Pilot Projects based on international experience international experience

2.2. Propose unified criteria for family status evaluation in determining its Propose unified criteria for family status evaluation in determining its eligibility for SAeligibility for SA

3.3. Suggest improvements for the calculation methods of family’s total Suggest improvements for the calculation methods of family’s total income for all types of SA income for all types of SA

4.4. Design a program for information and explanatory activities to be Design a program for information and explanatory activities to be conducted among population and local self-government institutions in conducted among population and local self-government institutions in pilots’ locationspilots’ locations

Excerpts from SARPS’s Terms of Reference (3): Excerpts from SARPS’s Terms of Reference (3): Tasks Tasks ((cont.cont.))

5.5. Prepare training materials on new piloted approaches, and Prepare training materials on new piloted approaches, and deliver training to the personnel of the labor and SA offices deliver training to the personnel of the labor and SA offices

6.6. Monitor and evaluate impacts of the pilot projects, and study Monitor and evaluate impacts of the pilot projects, and study the public opinionthe public opinion

7.7. Write policy papers outlining options to improve SA Write policy papers outlining options to improve SA regulationsregulations

6

Excerpts from SARP’s Terms of Reference: Excerpts from SARP’s Terms of Reference: OutputOutput

7

By the end of the project all Pilot Offices will:By the end of the project all Pilot Offices will:

Operate as model SA offices with streamlined processing of Operate as model SA offices with streamlined processing of applicationsapplications

Provide adequate SAProvide adequate SA

Provide an expanded range of services to assist the clients in Provide an expanded range of services to assist the clients in (re)entering the labor market, and achieving financial (re)entering the labor market, and achieving financial independenceindependence

Disseminate information about SA in order to increase awareness Disseminate information about SA in order to increase awareness of SA programsof SA programs

HMT: Tasks (1)HMT: Tasks (1)

HMT (Pilot #3): Introducing the HMT method (hybrid means testing) for HMT (Pilot #3): Introducing the HMT method (hybrid means testing) for granting all types of SA, and improving the methods of calculation of granting all types of SA, and improving the methods of calculation of family’s income family’s income

1.1. To use existing To use existing international best practiceinternational best practice experience in order to experience in order to improve targeting of SA benefits in Ukraineimprove targeting of SA benefits in Ukraine

2.2. To elaborate and test an To elaborate and test an HMT methodologyHMT methodology in terms of its in terms of its completeness, accuracy, efficiency/effectiveness, verifiability, and completeness, accuracy, efficiency/effectiveness, verifiability, and feasibilityfeasibility

8

HMT: Tasks (1) HMT: Tasks (1) ((cont.cont.))

3.3. To secure internal To secure internal consistency, clarity and transparencyconsistency, clarity and transparency of the of the means testing procedures; to clarify all definitions of terms used in means testing procedures; to clarify all definitions of terms used in the means testing procedure and minimize subjectivity of the means testing procedure and minimize subjectivity of declaration of applicants and judgments of civil servantsdeclaration of applicants and judgments of civil servants

4.4. To formulate a To formulate a proposal concerning all kinds of regulationsproposal concerning all kinds of regulations necessary for the pilot implementationnecessary for the pilot implementation

5.5. To take account of To take account of advice and feedbacks from social inspectorsadvice and feedbacks from social inspectors and other workers of SA offices as well as to arrange and other workers of SA offices as well as to arrange trainingtraining and and provide guidance and support to social inspectors in order to provide guidance and support to social inspectors in order to facilitate their work and improve its effectiveness; an important facilitate their work and improve its effectiveness; an important instrument for increasing this effectiveness will be the so-called instrument for increasing this effectiveness will be the so-called client profilingclient profiling

9

HMT: Tasks HMT: Tasks (2)(2)

10

6.6. To help achieve To help achieve uniformization/harmonizationuniformization/harmonization of all SA benefits (to of all SA benefits (to apply a uniform approach to household material status assessment and to apply a uniform approach to household material status assessment and to all kinds of SA grants)all kinds of SA grants)

7.7. To help To help assess impactsassess impacts of SA reforms (improvements in income of SA reforms (improvements in income verification, use of client profiling, harmonization of benefits, introduction of verification, use of client profiling, harmonization of benefits, introduction of different thresholds for benefit entitlements, and possibly other new different thresholds for benefit entitlements, and possibly other new targeting rules) on:targeting rules) on:

- Poverty indicatorsPoverty indicators- Income inequality indicatorsIncome inequality indicators- Budget expenditure on benefitsBudget expenditure on benefits- Quality of services provided to clients of SA officesQuality of services provided to clients of SA offices- Efficiency of social inspectorsEfficiency of social inspectors- Costs of maintaining SA officesCosts of maintaining SA offices

HMT: Tasks HMT: Tasks (2)(2) ((cont.cont.))

8.8. To set up a To set up a databasedatabase consisting of all information collected during consisting of all information collected during field surveys, and processed/analyzed by the staff of SA offices and field surveys, and processed/analyzed by the staff of SA offices and SARP’s expertsSARP’s experts

9.9. To arrange an To arrange an information campaigninformation campaign which should involve all which should involve all stakeholders about the pilot activities and its outcomes, as well as to stakeholders about the pilot activities and its outcomes, as well as to

initiate a initiate a public dialogpublic dialog on all issues related to the pilot on all issues related to the pilot

11

HMT: Background (1)HMT: Background (1)

12

Hybrid Means Testing has been chosen for piloting for several Hybrid Means Testing has been chosen for piloting for several reasons: reasons:

It has proved to work properly in various countries using means-It has proved to work properly in various countries using means-testing procedurestesting procedures

It has been found efficient by analytical works on Ukrainian It has been found efficient by analytical works on Ukrainian household data, and household data, and

It may also be useful for “client profiling” which helps identify It may also be useful for “client profiling” which helps identify suspicious cases for inspections suspicious cases for inspections

HMT: Background (1) HMT: Background (1) ((cont.cont.))

The pilot will be carried out in selected SA offices located in The pilot will be carried out in selected SA offices located in the following oblasts:the following oblasts:

1.1. LuganskaLuganska

2.2. MikolayevskaMikolayevska

3.3. CherkaskaCherkaska

4.4. ChernivetskaChernivetska

5.5. ChernigivskaChernigivska

13


Source: Finance Ministry of Ukraine

14

The HMT method will be applied to income estimation for the sake of The HMT method will be applied to income estimation for the sake of all benefits which are currently processed by the SA offices. all benefits which are currently processed by the SA offices.

Initially the pilot will last three months and will not have a direct Initially the pilot will last three months and will not have a direct impact on actual benefit payments. This will come eventually after impact on actual benefit payments. This will come eventually after evaluation and correction of the new approach. evaluation and correction of the new approach.

The pilot would be accompanied by a desk-study exploring additional The pilot would be accompanied by a desk-study exploring additional household surveys. household surveys.

Separate piloting will be undertaken regarding the work of social Separate piloting will be undertaken regarding the work of social inspectors and the cooperation between the SA offices and labor inspectors and the cooperation between the SA offices and labor offices. Efforts will be made to coordinate these three pilots in a way offices. Efforts will be made to coordinate these three pilots in a way that they will support each other and avoid unnecessary duplications. that they will support each other and avoid unnecessary duplications.


15

HMT implies that the total income consists of:HMT implies that the total income consists of:

i.i. easy-to-verify incomes, EVI, such as official wages, pensions easy-to-verify incomes, EVI, such as official wages, pensions and allowances, andand allowances, and

i.i. hard-to-verify incomes, HVI, derived for instance from self-hard-to-verify incomes, HVI, derived for instance from self-employment, land plot use or informal activitiesemployment, land plot use or informal activities

It assumes that means testing relies on the assessments of the total It assumes that means testing relies on the assessments of the total household income calculated as:household income calculated as:

- a sum of EVIs and HVIs, as reported by a claimant, anda sum of EVIs and HVIs, as reported by a claimant, and

- a sum of EVIs, as reported by the claimant, and HVI, as a sum of EVIs, as reported by the claimant, and HVI, as imputed from the HMT model on the basis of HVI indicatorsimputed from the HMT model on the basis of HVI indicators


16

Income imputationIncome imputation

Benefit claimants provide information on their main personal and household Benefit claimants provide information on their main personal and household characteristics which are found to be important determinants of incomes, in characteristics which are found to be important determinants of incomes, in particular:particular:

Demographic and social characteristics of a claimant and his/her Demographic and social characteristics of a claimant and his/her family family members (such as sex, age, education, family status, etc)members (such as sex, age, education, family status, etc)

Work related data (employment status, and type of work contract)Work related data (employment status, and type of work contract) Ownership of selected durables (car, color TV, and freezer)Ownership of selected durables (car, color TV, and freezer) Information on dwellings and utilities (surface, # of rooms, and Information on dwellings and utilities (surface, # of rooms, and

equipment)equipment) Location of dwellings, andLocation of dwellings, and Ownership of land plot and livestockOwnership of land plot and livestock


SARP experts estimate coefficients for all indicators based on SARP experts estimate coefficients for all indicators based on the the 2006 Ukraine Household Budget Survey Data2006 Ukraine Household Budget Survey Data (provided (provided by the State Statistics Committee). The coefficients are used to by the State Statistics Committee). The coefficients are used to convert the indicators into respective fractions of imputed convert the indicators into respective fractions of imputed income. income.

17


Example of income imputation by means of the HMT methodExample of income imputation by means of the HMT methodEstimation of total monthly income of a hypothetical familyEstimation of total monthly income of a hypothetical familyPrices as of January 2009 (adjusted for inflation)Prices as of January 2009 (adjusted for inflation)

18

Indicator Coefficient/person [hryvnia]

Unit Income [hryvnia]

BASE income 581.72/month 1 581.72 Entrepreneur 16.66 1 16.66 Construction 72.28 1 72.28 Spouse self-employed 59.41 1 59.41 Motorcycle 39.88 1 39.88 New color TV 16.54 1 16.54 Room size 2.12/sq. m. 20 42.40 Two+ earners 96.04 1 96.04 Children, 0-2 years -75.14/child 1 -75.14 Children, 3-6 years -86.35 2 -172.70 Number of elderly 43.37/elderly person 1 43.37 Electricity 0.11/UAH spent/month 15 1.65 Telephone 1.91/UAH spent/month 10 19.10 Residence in Kyiv 44.51 1 44.51 Total income/person SUM 785.72 Total family income 785.72 6 4714.32


19

Pilot design Pilot design

The pilot will be carried out at selected five regional SA offices The pilot will be carried out at selected five regional SA offices starting from April 2009starting from April 2009

Initially the field work will last three months and will be followed Initially the field work will last three months and will be followed by the evaluation stageby the evaluation stage

After adjustment and correction, a new round of a pilot might After adjustment and correction, a new round of a pilot might be implementedbe implemented


Pilot participantsPilot participants

Benefit claimants: will provide information on their incomes and Benefit claimants: will provide information on their incomes and main individual and family characteristics necessary for the income main individual and family characteristics necessary for the income imputation and for the total income assessment imputation and for the total income assessment

Workers of social offices: will collect and process informationWorkers of social offices: will collect and process information

Social inspectors: will verify material status applicants selected, in Social inspectors: will verify material status applicants selected, in part, with the use of ‘client profiling’ and partially randomlypart, with the use of ‘client profiling’ and partially randomly

20


21

Client profilingClient profiling

Total amount of incomes reported by a claimant is compared with the total Total amount of incomes reported by a claimant is compared with the total income assessed using HMT imputationincome assessed using HMT imputation

A large difference between the two numbers, imputed versus actual, signals A large difference between the two numbers, imputed versus actual, signals a need for checking a claimant by social inspector a need for checking a claimant by social inspector


HMT method usefulnessHMT method usefulness

Social inspectors will provide their estimates of actual Social inspectors will provide their estimates of actual incomes of randomly selected households, based on in-site incomes of randomly selected households, based on in-site inspectionsinspections

Comparison of the actual income estimates (by social Comparison of the actual income estimates (by social inspectors) with the estimates generated by means of the inspectors) with the estimates generated by means of the method currently used by the SA offices and those produced method currently used by the SA offices and those produced by the HMT method will enable an evaluation of the by the HMT method will enable an evaluation of the effectiveness/usefulness of the HMT methodeffectiveness/usefulness of the HMT method

22

Pilot implementation: Pilot implementation:

Phase 1. Preparatory stagePhase 1. Preparatory stage

23

Elaboration of main points of a regulatory document which will Elaboration of main points of a regulatory document which will implement the pilots implement the pilots

Elaboration of income indicators and coefficients Elaboration of income indicators and coefficients

Preparation of a questionnaire Preparation of a questionnaire

Elaboration of procedures, including computer toolkit, for data Elaboration of procedures, including computer toolkit, for data collection and data processingcollection and data processing

Preparation of training materials (guidelines, instructions, and Preparation of training materials (guidelines, instructions, and codes).codes).

Pilot implementation: Pilot implementation: Phase 1. Preparatory stage Phase 1. Preparatory stage ((cont.cont.))

Training of social workers and social inspectorsTraining of social workers and social inspectors

Development of the methodology for examination of pilot results\Development of the methodology for examination of pilot results\

Preparation of information campaignPreparation of information campaign

Preparation of the monitoring and evaluation indicators and Preparation of the monitoring and evaluation indicators and proceduresprocedures

24

Pilot implementation: Pilot implementation: Phase 2. Data collectionPhase 2. Data collection

25

Using a questionnaire for all information needed for HMTUsing a questionnaire for all information needed for HMT

All necessary income indicators as well as household incomes All necessary income indicators as well as household incomes split into an EVI part and HVI part should be provided by split into an EVI part and HVI part should be provided by claimantsclaimants

The pilot would cover the household per-capita monthly income, The pilot would cover the household per-capita monthly income, derived from the annual figuresderived from the annual figures

Pilot implementation: Pilot implementation: Phase 2. Data collection Phase 2. Data collection ((cont.cont.))

Income will be evaluated in the same way for all claimants, Income will be evaluated in the same way for all claimants, regardless of benefit type they apply forregardless of benefit type they apply for

An important part of questionnaire will be its documentation An important part of questionnaire will be its documentation which will provide all definitions and instructions needed for the which will provide all definitions and instructions needed for the filling up the formsfilling up the forms

26

Pilot implementation: Pilot implementation: Phase 2. Data collection Phase 2. Data collection ((cont.cont.))

27

There are several possibilities for arranging of filling in the There are several possibilities for arranging of filling in the questionnaire:questionnaire:

- filling in by the applicant in the officefilling in by the applicant in the officefilling in by the applicant in the office or at home with help from an filling in by the applicant in the office or at home with help from an

office staffoffice staff- filling in by the office staffer or social inspector during an filling in by the office staffer or social inspector during an

interview with the applicant in the officeinterview with the applicant in the officefilling in by the applicant through internet, andfilling in by the applicant through internet, and- a mixed method combining some of the abovea mixed method combining some of the above

During the pilot, these different ways could be tested for their During the pilot, these different ways could be tested for their feasibility, efficiency and accuracyfeasibility, efficiency and accuracy

Pilot implementation: Pilot implementation: Phase 3. Creation of databasePhase 3. Creation of database

Social workers will input all information from the questionnaires into a Social workers will input all information from the questionnaires into a PC (unless applicants themselves would be entering data to the PC (unless applicants themselves would be entering data to the computer). This process would go in part simultaneously with the data computer). This process would go in part simultaneously with the data collection phase.collection phase.

A simple PC toolkit, which provides coefficients and formulas for A simple PC toolkit, which provides coefficients and formulas for income imputation, will be used.income imputation, will be used.

For each claimant, three income assessments (monthly, per capita) For each claimant, three income assessments (monthly, per capita) should be derived and clearly stated: (i) total income reported, (ii) should be derived and clearly stated: (i) total income reported, (ii) reported EVI plus imputed HVI, and (iii) total imputed. All three reported EVI plus imputed HVI, and (iii) total imputed. All three assessments are needed for the implementation of hybrid means-assessments are needed for the implementation of hybrid means-testing procedures.testing procedures.

28

Pilot implementation: Pilot implementation:

Phase 4. Income checkingPhase 4. Income checking

The selection of applicants for inspection will be organized in a The selection of applicants for inspection will be organized in a systemic way. It will allow for further examination of the effectiveness systemic way. It will allow for further examination of the effectiveness of “client profiling” with the use of HMT methodology. of “client profiling” with the use of HMT methodology.

Two groups of inspected cases will be created. The first group will Two groups of inspected cases will be created. The first group will include only those claimants for whom the difference between imputed include only those claimants for whom the difference between imputed and reported incomes is large [e.g., a threshold may be set at 20%]. and reported incomes is large [e.g., a threshold may be set at 20%]. The second group will consist of randomly selected applicants. The second group will consist of randomly selected applicants.

A survey of social inspectors will be arranged in order to get feedbacks A survey of social inspectors will be arranged in order to get feedbacks related to this experiment and the usefulness of the HMT method in related to this experiment and the usefulness of the HMT method in the format designed for this experiment.the format designed for this experiment.

29

Pilot implementation: Pilot implementation: Phase 5. InvestigationPhase 5. Investigation

Investigation phase will consist of:Investigation phase will consist of:

Checking the income imputation proceduresChecking the income imputation procedures

Comparing outcomes of actual means-testing and simulated HMT Comparing outcomes of actual means-testing and simulated HMT procedures procedures

Examining HMT as a pure means-testing tool, andExamining HMT as a pure means-testing tool, and

Verifying income imputation approach as a ‘client profiling’ methodVerifying income imputation approach as a ‘client profiling’ method

Verifying the usefulness of the HMT method as compared with the Verifying the usefulness of the HMT method as compared with the current means testing methodcurrent means testing method

30

Criteria for assessment of performance Criteria for assessment of performance of the pilot (1)of the pilot (1)

CompletenessCompleteness: Does this method cover all important kinds of : Does this method cover all important kinds of incomes, both monetary and in-kind, and does it include all incomes, both monetary and in-kind, and does it include all kinds of important items which should be accounted for in the kinds of important items which should be accounted for in the estimation of the HTV kinds of incomes?estimation of the HTV kinds of incomes?

ConsistencyConsistency: Is it internally consistent? Does it involve any : Is it internally consistent? Does it involve any kind of duplication and/or repetition?kind of duplication and/or repetition?

Clarity and easinessClarity and easiness: Is the questionnaire and all procedures : Is the questionnaire and all procedures clear and easy to understand for the applicants, office stuffers clear and easy to understand for the applicants, office stuffers and the social inspectors?and the social inspectors?

31

Criteria for assessment of performance Criteria for assessment of performance of the pilot (1) of the pilot (1) ((cont.cont.))

• AccuracyAccuracy: Does this HMT method reflect a true income of : Does this HMT method reflect a true income of applicants and works properly as a means-testing tool? Should applicants and works properly as a means-testing tool? Should the list of income indicators be changed? This would involve the list of income indicators be changed? This would involve cross-checking of incomes reported and assessed with the use cross-checking of incomes reported and assessed with the use of HMT methodology, which could be approached by setting up a of HMT methodology, which could be approached by setting up a procedure of requesting social inspectors to make judgments on procedure of requesting social inspectors to make judgments on claimants’ incomes (“In your opinion the monthly income of a claimants’ incomes (“In your opinion the monthly income of a given family belongs to a specified income bracket, such as: given family belongs to a specified income bracket, such as: below 100 UAH, 100-200 UAH, 300-500 UAH, etc.). below 100 UAH, 100-200 UAH, 300-500 UAH, etc.).

32


HonestyHonesty: Does it enable detection of fraud and corruption? What are : Does it enable detection of fraud and corruption? What are the main risks? Does it provide right incentives for all persons the main risks? Does it provide right incentives for all persons involved? Are there any additional ways to reduce the number of involved? Are there any additional ways to reduce the number of fraudulent cases? fraudulent cases?

Efficiency/effectivenessEfficiency/effectiveness: Does it assist the social inspector in his/her : Does it assist the social inspector in his/her work? Is it helpful in client profiling? By applying this method is the work? Is it helpful in client profiling? By applying this method is the social inspector using his/her time efficiently?social inspector using his/her time efficiently?

VerifiabilityVerifiability: Is it possible to independently verify all kinds of : Is it possible to independently verify all kinds of information provided by the applicants and the inspectors?information provided by the applicants and the inspectors?

UniformizationUniformization: Does it treat all kinds of benefits in a uniform way? : Does it treat all kinds of benefits in a uniform way? Does it accomplish the harmonization of SA, which was one of Does it accomplish the harmonization of SA, which was one of important tasks for this pilot?important tasks for this pilot?

33


LegalityLegality: Do all procedures used in this method conform to the : Do all procedures used in this method conform to the existing law?existing law?

Data organization and processingData organization and processing: Is the data well organized : Is the data well organized and easily accessible for processing, updating, maintaining, etc.? and easily accessible for processing, updating, maintaining, etc.? Are the data safety and privacy secured?Are the data safety and privacy secured?

InformationInformation: Are all stakeholders of the SA well informed about : Are all stakeholders of the SA well informed about this pilot? Is the effective public dialogue arranged?this pilot? Is the effective public dialogue arranged?

FeasibilityFeasibility: Can this method be handled by social workers, : Can this method be handled by social workers, social inspectors and claimants? Is the split of income between social inspectors and claimants? Is the split of income between HVI and EVI workable? Does imputation procedure involve HVI and EVI workable? Does imputation procedure involve problems?problems?

34

Criteria for assessment of performance Criteria for assessment of performance of the pilot (3)of the pilot (3) ( (cont.cont.))

SensitivitySensitivity: Evaluating the impact of new thresholds or new : Evaluating the impact of new thresholds or new rules for screening out applicants (incl. also recipients of the rules for screening out applicants (incl. also recipients of the benefits which are not income tested). This issue is treated as benefits which are not income tested). This issue is treated as optional.optional.

35

Potential risks of pilot implementation (1)Potential risks of pilot implementation (1)

Accuracy of information provided by the applicant:Accuracy of information provided by the applicant:

Errors due to misinformation/misunderstanding of rulesErrors due to misinformation/misunderstanding of rules Filling in mistakes, incl. calculation mistakes, andFilling in mistakes, incl. calculation mistakes, and Cheating/fraudCheating/fraud

Adequacy of uniform income reporting rules (by how much they reflect Adequacy of uniform income reporting rules (by how much they reflect and enable one to assess the effective income stream of applicant)\and enable one to assess the effective income stream of applicant)\

Accounting for effective living costs per family member Accounting for effective living costs per family member (accomplished by application of regional (oblast-specific) (accomplished by application of regional (oblast-specific) coefficientscoefficients

Non-accounting for the effective costs incurred by applicant, andNon-accounting for the effective costs incurred by applicant, and Accurate accounting of price inflationAccurate accounting of price inflation

36

Potential risks of pilot implementation (1)Potential risks of pilot implementation (1) ((cont.cont.))

Adequacy of proxy and imputed income methodsAdequacy of proxy and imputed income methods

Stock-to-flow conversion methodsStock-to-flow conversion methods Accounting for quality (e.g., market value of a car), and Accounting for quality (e.g., market value of a car), and Price levels, price inflationPrice levels, price inflation

37

Potential risks of pilot implementation (2)Potential risks of pilot implementation (2)

Accuracy of information collected from the other-than-applicant Accuracy of information collected from the other-than-applicant sourcessources

Distortions due to incomplete informationDistortions due to incomplete information Factual data errorsFactual data errors Corruption Corruption Vested interests (rent seeking); program capture by elitesVested interests (rent seeking); program capture by elites Populist-type distortions (efforts to gain political support)Populist-type distortions (efforts to gain political support) Local values and beliefs (such as not-pro-poor Local values and beliefs (such as not-pro-poor

preferences), andpreferences), and Local policy strategic targeting (priorities of budget fund Local policy strategic targeting (priorities of budget fund

spending)spending)

38

Potential risks of pilot implementation (2)Potential risks of pilot implementation (2) ((cont.cont.))

Accuracy of assessments by social inspectorsAccuracy of assessments by social inspectors

Inaccurate skillsInaccurate skills Inaccurate information sourcesInaccurate information sources Subjectivity, andSubjectivity, and Corruption/fraudCorruption/fraud

Feasibility of the pilot projectFeasibility of the pilot project

Insufficient support for the pilot from local authoritiesInsufficient support for the pilot from local authorities Lack of interest from applicants (this pilot will not affect their Lack of interest from applicants (this pilot will not affect their

benefits), and benefits), and Work load of staff members of local SA offices; they may Work load of staff members of local SA offices; they may

not be able to allocate enough time and effort to pilot’s not be able to allocate enough time and effort to pilot’s activitiesactivities

39

Concluding remarksConcluding remarks

International experience should be used in pilots, but tailored International experience should be used in pilots, but tailored to the specificities of Ukraine. to the specificities of Ukraine.

All piloting experiments must reflect social policy reform All piloting experiments must reflect social policy reform options considered by the Government and provide assistance options considered by the Government and provide assistance in policy decision making.in policy decision making.

All methods, indicators and activities of SARP are being All methods, indicators and activities of SARP are being discussed with the Ministry of Labor and Social Policy, and will discussed with the Ministry of Labor and Social Policy, and will be implemented after a clearance from the Ministry is granted.be implemented after a clearance from the Ministry is granted.

They must be explicitly formulated in the regulatory documents They must be explicitly formulated in the regulatory documents issued by the Government of Ukraine. issued by the Government of Ukraine.

40

Concluding remarks Concluding remarks ((cont.cont.))

The activities of this pilot must be closely coordinated with the The activities of this pilot must be closely coordinated with the activities of other pilot projects undertaken by SARP.activities of other pilot projects undertaken by SARP.

There is a need for a very close cooperation between SARP There is a need for a very close cooperation between SARP project staff members, MLSP officials, local authorities, project staff members, MLSP officials, local authorities, employees of local SA offices, local NGOs, as well as local employees of local SA offices, local NGOs, as well as local communities and SA applicants. communities and SA applicants.

The guiding approach of the pilot is gradualism and the learning-The guiding approach of the pilot is gradualism and the learning-by-doing principle: the consecutive iterations of piloting should by-doing principle: the consecutive iterations of piloting should enable us to experiment with different alternatives and provide enable us to experiment with different alternatives and provide solid foundations for the SA reform in Ukraine.solid foundations for the SA reform in Ukraine.

41

Questions and commentsQuestions and comments

hybrid means testing pilot sarp: social assistance reform project janusz szyrmer irena topinska...

Documents

pilot sa offices

sarps terms of reference

pilot projects

pilot offices

sarpss terms of reference

model sa offices

competences pilot

sa regulations6 excerpts