does ‘work for the dole’ work?

8/12/2019 Does Work for the Dole work?

1/50

Does Work for the Dole work?

Jeff Borland* and Yi-Ping Tseng**

June 2003

ACKNOWLEDGEMENTS: We are very grateful for assistance from the LDS groupat FaCS, particularly Shaun Burnham and Gerry Carey, and to Tony Hedditch fromCentrelink for providing data on the pilot phase WfD projects. We are also gratefulfor the opportunity to present this research at the 2002 DEWR Evaluation Workshop,for data provided to us by DEWR, and for helpful discussions with Robert Lipp andLinda Richardson. The work has benefited from comments made at a FaCS seminar.

* Department of Economics and Melbourne Institute of Applied Economic and SocialResearch, University of Melbourne, Melbourne VIC 3010, Australia Email:

[email protected].** Melbourne Institute of Applied Economic and Social Research, University ofMelbourne, Melbourne VIC 3010, Australia Email: [email protected]
mailto:[email protected]:[email protected]


2/50

i

Executive Summary

Objective

This project examines the effect of the introduction of the Work for the Dole (WfD)program on exit from payments of Newstart allowance (NSA) recipients. The focusof the analysis will be the pilot phase when WfD operated between November 1997and June 1998 prior to the introduction of Mutual Obligation (MO).

What is Work for the Dole?

The objectives of the WfD scheme are to provide opportunities for unemployedpeople to gain work experience; build networks; improve their self-esteem,communication skills, and motivation; and contribute to projects that are of value tothe community. Under the pilot phase of WfD eligible unemployment payment

recipients were required to participate in specified projects of benefit to thecommunity. The target population for the scheme was recipients of NSA aged 18 to24 years on full rate of income support who had been in receipt of income support forat least six months. Participation in WfD was required for a maximum of six months;working for six hours per day for two days if aged 18 to 20 years, and working for sixhours per day for two and a half days if aged 21 to 24 years.

How are the effects of WfD measured?

Effects of WfD are examined for a variety of outcome measures related to receipt ofunemployment payments (i) The effect of WfD on the incidence of exit from payments by 6 months and 12months after WfD commencement;(ii) The effect of WfD on whether payment recipients are on payments at 9 monthsand 15 months after WfD commencement; and(iii) The effect of WfD on the number of fortnights on payments during the 6 monthsand 12 months after WfD commencement.

What can be learned from this study?

In analysing the effect of WfD on exit from payments this study is obviously focused

on just a subset of its objectives; and this must be taken into account in seeking toextrapolate from the findings of this study to an overall assessment of the program. Itis also the case that the study focuses on the very earliest stage of the program, so it isnecessary to be cautious in generalising the findings to WfD today; on the other hand,we argue that there are several reasons for thinking that there are indeed lessons in thestudy for WfD today.

Descriptive information

There are 888 payment spells during the period from November 1997 to June 1998where a NSA recipient is observed to participate in WfD. Commencement of

participation in WfD during the pilot phase does not appear to be highly concentratedat any single point in payment spell duration. Over 50 per cent of participants began


3/50

ii

participation between 13 and 26 fortnights, about 25 per cent between 27 and 52fortnights, and about 10 per cent at longer than 52 fortnights. There appears to bequite a high degree of geographic concentration of participation in WfD between ABSLabour Force regions. For example, about 10% of the population of individuals aged18 to 24 years with payment spells during the sample period are in regions where

there is zero participation in WfD; by contrast, 53.2% of participants in WfD are inregions where only 21.2% of those with payment spells reside.

Empirical methodology

The empirical approach used to estimate the effect of the WfD is a quasi-experimentalmatching method. Fundamentally, this involves comparing payment outcomes for atreatment group of NSA recipients who participate in WfD, and a matched controlgroup who never commence WfD. Hence the policy effect that is identified is theaverage effect of commencing participation in WfD for payment recipients aged 18 to24 years who commence participation during the pilot phase compared to matched

payment recipients who never commence participation in WfD.

The type of quasi-experimental matching method used in this study is an exactmatching method. Treatment group observations (participants in WfD) are matchedwith a control group observation(s) with the same: Payment spell duration; Quarterspell on WfD commences; Age; Gender; Country of birth; ATSI status; Marital status;Activity type in previous fortnight; Rate of unemployment in ABS Labour Forceregion; and Payment history over previous 12 months. With this approach eachtreatment observation can potentially be classified in one of 3,369,000 cells.

The effect of WfD Basic results

The main conclusion is that there appear to be quite large significant adverse effectsof participation in WfD. For example, for the group of matched WfD participants it isfound that there is a difference in exit from NSA payments between WfD participantsand non-participants equal to minus 12.1 percentage points; the difference in the

proportions of WfD participants and non-participants on NSA payments at 9 months(after start of spell on WfD) is 8.5 percentage points; and the difference in fortnightson NSA payments between WfD participants and non-participants (in the first 6months after start of spell on WfD) is 0.99 fortnights. Each of these differences isstatistically significant at the 1% level.

Outcome measure Difference in outcomes:

WfD participants minus

non-participants

% Off payments

By 3 months -12.1

By 6 months -10.5

% On payments

At 6 months +8.5

At 12 months +8.9

Time on payments

(fortnights)First 6 months +0.99


4/50

iii

First 12 months +2.20

More detailed analysis of exit from payments suggests that there is an adverse effectof WfD on exit from payments associated directly with the period of participation in

WfD, but that there is partial catch-up by WfD participants after the conclusion ofWfD.

How robust are the findings?

Several methods for assessing the robustness of the basic empirical approach areused. One method addresses the issue that some WfD participants are excluded fromthe basic analysis because no matching control observations exist. Results from thismethod (bounds analysis) suggest that the distribution of possible outcomes is heavilytowards adverse effects of WfD; although it is not possible to exclude the possibilityof a zero or small positive effect on receipt of payments from WfD participation. The

second method is to test for whether referral effects of WfD (whereby paymentrecipients exit payments on referral to WfD) might have biased estimates of the effectof WfD. The answer appears to be negative. The third type of analysis is to examinehow estimates of the WfD effect are changed by introducing extra variables to match

between WfD participants and control group payment recipients. The findings arefound to be not significantly altered by use of education attainment in this way.Hence, we make the general conclusion is that the results are highly robust.

The effect of WfD Results for sub-groups

Effects of WfD for disaggregated groups of payment recipients can be estimated, andare of potential interest. WfD effects are found to be slightly more adverse forfemales than males; and more adverse for payment recipients aged 21 to 24 years thanfor those aged 18 to 20 years. Participation in WfD has large negative effects onlabour market outcomes for payment recipients who have received payments in atleast three quarters in the previous 12 months compared or who did not receive

payments in the previous 12 months, but has no significant effect for paymentrecipients who received payments in one or two quarters in the previous 12 months.Hence there is not a consistent ordering of WfD effects by payment history. Betweengeographic regions, WfD participation has a significant adverse effect on labourmarket outcomes for participants in regions with above-median rates of

unemployment, but generally no significant effect in below-median rate ofunemployment regions. It is also found that WfD effects are more adverse for NSArecipients who had not participated in JSD in the previous 12 months, whose WfDclassification was compulsory (compared to voluntary), and who did not haveearnings from labour market activity in the fortnight prior to commencing in WfD.

Why might WfD have adverse effects on exit from unemployment payments?

What might explain negative effects of WFD participation on exit fromunemployment payments? There appear to be three main potential explanations: (i)Stigma effects; (ii) Effect on job search activity Participation in WfD may allow

participants to reduce their job search activity, and may adversely affect the type ofjob search activity undertaken; and (iii) Scale of intervention The WfD program


5/50

iv

represents a relatively minimalist intervention. Of these explanations, the potentialchilling effect of WfD on job search activity, seems to be most supported byinternational evidence. This explanation also seems to be supported by the time-series

pattern of WfD effects (increasingly adverse during the six-month phase ofparticipation in WfD, but then reversing to some extent after that time). However,

there are some aspects of our findings that are more difficult to reconcile with effectof WfD on job search activity being the main explanation for the negative estimates ofthe program impact.


6/50

1

1. Introduction

This project examines the effect of the introduction of the Work for the Dole (WfD)

program on exit from payments of Newstart allowance (NSA) recipients. The focus

of the analysis will be the pilot phase when WfD operated between November 1997

and June 1998 prior to the introduction of Mutual Obligation (MO).

The main motivation for the project is the importance that the WfD program has

assumed as part of activity test requirements for unemployment payment recipients.

For example, in 2002-03 the federal government provided funding for 55,000 WfD

places. WfD is the residual category of activity for payment recipients undertaking

a MO requirement; and payment recipients not required to participate in MO may still

be directly referred to WfD.

Despite the importance of the WfD program, there has thus far been little analysis of

its effects. There has been only two studies of the net impact of the WfD program,

and we argue that the findings in those studies are problematic due to methodological

issues (see literature review in section 3 for details). More generally, there appear to

be no other evaluations of the net impact of work experience programs in Australia.

This is perhaps surprising, given that the international literature raises significant

doubts about whether such programs will have positive effect on labour market

outcomes for participants.

In this study a quasi-experimental matching method is used to estimate the effects of

participation in WfD on durations of unemployment payment spells. Since during

the pilot phase of WfD - unemployment payment recipients could commence WfD

participation at different payment spell durations, it is necessary for the matching

method to accommodate this complexity. These issues are addressed in this study by

estimating a policy effect that is the weighted average outcome from commencing

participation in WfD in a given fortnight of a payment spell, compared to outcomes

for payment recipients who never commence WfD during their payment spells; and

then aggregating across all payment spell durations where some payment recipient

commences on WfD. The other critical aspect of methodology is to provide a


7/50

2

justification for the validity of a matching estimator as a method to examine the

effects of the WfD. It is argued that first, there is a sufficiently rich set of covariates

that can be used for matching; and second, details of the implementation of WfD

during the pilot phase specifically, the geographic distribution of WfD

participation due to limits on project funding suggest that there is a significant

source of randomness in assignment of unemployed persons between participating

and not participating in the JSD.

The WfD program is on-going, yet in this report we choose to focus on the initial

pilot phase of the program. There are three main reasons. First, after 1 July 1998

WfD became part of the Mutual Obligation (MO) program. Under MO,

unemployment payment recipients aged 18 to 24 with payment spells of at least six

months are required to participate in one of a range of possible activities (including

WfD). As payment recipients can choose the activity that best suits their interests,

under MO it is not likely that participants in WfD constitute a random sample of

payment recipients, and it will be more difficult to satisfy the assumption necessary

to implement matching quasi-experimental methods that there is a group of non-

WfD participants who, after controlling for differences in observable characteristics,

are identical to WfD participants. Second, on 1 July 1998 Youth Allowance (other)

(YA(o)) replaced NSA for payment recipients aged 18 to 20 years. The YA(o)

differed from the NSA in that eligibility and payment amount depend on a parental

means test; hence it is likely that the eligible population for WfD is different before

and after 1 July 1998. A third reason is that the relatively small number of WfD

participants in the pilot phase allows an exact method of quasi-experimental

matching to be applied.

Because our results are obtained for a pilot phase of the WfD program, it would of

course be necessary to exercise caution in using those results to make judgements

about outcomes from WfD in more recent time periods. On the other hand, it cannot

simply be assumed that the findings in this study are uninformative about WfD

outcomes today. First, it is shown that our results are quite similar to findings from

international studies on work experience programs. Second, although details of the

operation and administration of WfD have evolved since the pilot phase, the

essential objectives and nature of the program have not changed.


8/50

3

Section 2 describes the WfD program. Section 3 presents a review and evaluation of

some relevant literature. Section 4 describes the data source, and sample of payment

recipients, used in this study, and in section 5 descriptive information on the sample

of pilot phase WfD participants is presented. Section 6 provides a brief overview of

predictions about the effect of a program such as WfD on unemployment from

standard search theory models of the labour market. The empirical methodology is

described in section 7, and results are presented in section 8. An evaluation of the

results is undertaken in section 9.

2. The WfD program

The objectives of the WfD scheme were (and continue to be) to provide opportunities

for unemployed people to (Department of Family and Community Services, 2002a,

section 3.2.8.80):

i) Gain work experience;

ii) Build networks;

iii) Improve their self esteem, communication skills, and motivation; and

iv) Contribute to projects that are of value to the community.

Under the pilot phase of the scheme eligible unemployment payment recipients were

required to participate in specified projects of benefit to the community. The target

population for the scheme was recipients of NSA aged 18 to 24 years on full rate of

income support who had been in receipt of income support for at least six months.

Participation in WfD was required for a maximum of six months; working for six

hours per day for two days if aged 18 to 20 years, and working for six hours per day

for two and a half days if aged 21 to 24 years. Participants were paid a WfD

supplement of $20 per fortnight in recognition of working costs.

The obligation to participate in MOI derives from social security legislation on

eligibility conditions for receipt of unemployment benefit payments. The Social

Security Act 1991 requires that (unless exempted) unemployment payment recipients

must meet an activity test to be actively looking for work, or undertaking activities


9/50

4

to improve their employment prospects, and be willing to accept offers of suitable

employment (Section 601). Subject to meeting the activity test requirement, there is

no time limit on the duration for which unemployment payments can be claimed in

Australia.

As a program with multiple objectives, the assessment in this study of whether WfD

achieves a specific objective improving labour market outcomes for participants -

can only constitute a partial test of its performance. Nevertheless, it does seem valid

to undertake such an analysis of the effect of WfD. Work experience is stated as an

objective of the program, and it would seem difficult to argue that meaningful work

experience is being provided in the absence of evidence that the work experience has

improved participants labour market outcomes (over some time horizon) and thereby

reduced receipt of welfare payments. (Originally, the work experience objective was

expressed as being to develop work habits in young people DEWR, 1999.) As

well, there is more direct evidence of an expectation that the Work for the Dole

program would improve labour market outcomes. For example, on its introduction

the Prime Minister, John Howard, stated that the government will for the first time

introduce, as part of the armoury of the policies it is employing to attack the problem

of youth unemployment, a work for the dole scheme (Commonwealth of Australia,

House of Representatives, 10 February 1997; Hansard volume H, R211-212, p.466).

WfD services are managed by Community Work Coordinators, and delivered through

community or government organisations or agencies. During the pilot phase 174

projects commenced. Projects in the pilot phase involved environmental work such

as tree planting, construction projects such as building cycling and walking tracks,

maintenance of community facilities, and work in hospitals and the aged care sector

undertaking administrative tasks and roles such as meals on wheels delivery

(Department of Employment and Workplace Relations, 1999, pp.2, 33).

3. Literature review

Three studies of the effects of WfD have been undertaken. One study has specifically

examined outcomes from the pilot phase of WfD (Department of Employment and


10/50

5

Workplace Relations, 1999). That study was primarily concerned with a description

of outcomes for project participants, and analysis of qualitative data on their attitudes

to the program. The other studies have sought to examine the net impact of WfD on

labour market outcomes first, for a sample of payment recipients who completed

spells on WfD in late 1999 (Department of Employment and Workplace Relations,

2000); and second, for a sample of payment recipients who commenced payment

spells between January and June 1998 and commenced Mutual Obligation activities

between July and December 1998 (Richardson, 2003).

The pilot phase study involved analysis of employment outcomes for WfD

participants, case studies of WfD projects, and in-depth interviews with some

participants. The main findings were that the program received a high degree of

support from stakeholders such as participants and communities, that WfD was

meeting its objectives of developing work habits, and that over 50 per cent of WfD

participants had some spell of employment in the three months after exiting from the

program.

The DEWR (2000) analysis of the net impact of WfD involved a comparison of

labour market outcomes for 2100 WfD participants who left a WfD project in August

1999 with outcomes for 2100 matched payment recipients who had not participated in

or been referred to WfD in the previous six months. Matching between participants

and non-participants was on the basis of age, gender and duration of payments. The

main finding from the study is that over the first five months after exit from WfD the

average proportion of WfD participants off payments is 30 per cent whereas for non-

participants the proportion is about 17 per cent. In the first month after ending

participation in WfD the difference in the proportion off payments is about 20

percentage points; by the fifth month this has declined to 5 percentage points. It is

estimated that 80 to 90 per cent of those who move off payments from WfD are

shifting to work or education. The other main dimension of this study is analysis of

the effect of referral to WfD. It is found that there appears to be a significant referral

effect of WfD. For example, it is estimated that about one-third of payment recipients

referred to WfD during 1999 did not commence participation; and furthermore, rates

of exit from payments are similar for those who are referred to WfD but do not

commence, as for those who commence.


11/50

6

Significant criticisms of the methodology applied in the net impact study of WfD

have been raised (Productivity Commission, 2002, Appendix E, and OECD, 2001,

p.220). One problem derives from the assumption in the net impact methodology that

participation in WfD is time out of the labour force. But during the period of WfD

participation it is possible that some payment recipient will exit payments. Hence, the

DEWR methodology matches a group of program participants, some of whom have

already left payments, with a group of non-participants, all of whom are unemployed

on payments. Other things equal, this will tend to bias upward estimates of the net

impact of WfD. The second problem is that the group of WfD participants is matched

to a group of non-participants using duration of spell on unemployment payments

prior to WfD participation. Hence, WfD participants will on average have longer

payment spells than non-participants. Since the probability of finding employment is

likely to decline with duration of payment spell, therefore, the effect of longer average

spell duration is that WfD participants will have lower probabilities of exiting

payments than non-participants. In other words, this problem imparts a downward

bias to estimates of the net impact of WfD.

Analysis of the net impact of WfD by Richardson (2003) is part of a more general

study of the effect of participation in Mutual Obligation (MO) activities. The main

sample group is payment recipients aged 23-24 years who participate in an MO

activity between July and December 1998. A matched difference-in-difference

method applied is to compare outcomes for MO participants within each stream of

activity to a control group of matched payment recipients aged 25-26 years; and then

to difference with the difference in outcomes for matched samples of payment

recipients aged 23-24 years and 25-26 years in a pre-MO sample period. The main

finding with regard to WfD is that there is no significant effect of participation on

financial dependence on income support either in the 6 or 12 months after

participation in WfD.

A difficulty with the Richardson study is that the validity of the matching

methodology applied requires that conditional on the set of matching covariates

used assignment between participation in WfD and the control group should be

random (see for example, Gerfin and Lechner, 2002 for discussion of necessary


12/50

7

conditions for identification of program effects where participants in a program may

enter different streams of types of programs). This random assignment requirement

seems problematic in the case of assignment to WfD. For example, selection between

the MO activities of part-time work and WfD is likely to involve those payment

recipients who are already in part-time work choosing part-time work as their activity,

and individuals without labour market attachment being assigned to WfD. (In

general, we would expect MO activity selection to be such that payment recipients

with the worst labour market prospects to be assigned to WfD. This is consistent

with evidence presented in Richardsons study that shows the likelihood of

participating in WfD is positively related to time on payments in previous time

periods.)

The key question is whether the set of matching covariates used are sufficient to

control for all unobserved differences between WfD participants and the control

groups. Certainly, the problem of selection bias is mitigated by comparing outcomes

between WfD participants and a control group who do not participate in MO (rather

than for example, MO participants whose activity is part-time work), and by using

payment history as a matching variable. Nevertheless, in the absence of evidence of a

mechanism (or natural experiment) that suggests random assignment between MO

activities, and with evidence that WfD participants are particularly disadvantaged in

their labour market prospects, there must be concern about the validity of the

matching methodology. To the extent that the unobserved characteristics of WfD

participants are associated with worse labour market outcomes, it suggests that

Richardsons study will under-estimate the effect of WfD.

More generally, findings from studies of other types of work experience programs are

relevant background for the WfD. For Australia the main type of work experience

programs related to WfD have been public sector job creation schemes such as New

Work Opportunities in the mid-1990s. Studies of these programs have focused on

describing outcomes for participants. For example, Webster (1998, p.198) offers the

following review of these studies:

existing studies [of public sector job creation schemes] have placed little emphasis

on estimating whether the subsidised participant would have gained a job in the


13/50

8

absence of the program. Most evaluations have concentrated on the post-program

effects, which generally are assessed to be marginally positive or negligible.

Recent reviews of studies of work experience programs in Europe and the United

States are fairly dismal in their assessment of the likely effects of those programs.

From reviews of European policies Kluve and Schmidt (2002, p.439) conclude that

both direct job creation and employment subsidies in the public sector almost

always seem to fail; and Robinson (2000, p.24) suggests that there is no

convincing evidence that work programs improve the employment chances of

participants. Heckman et al. (1999, p.2053) summarise evidence on government

employment programs for the United States. The main findings are that: (i) On

average these programs appear to have at best a modest positive impact on adult

earnings but no impact on youths earnings; (ii) There is a high level of

heterogeneity in estimated programme impacts; and (iii) There is some evidence that

the largest program effect is for low-skill labour force participants.

4. Data source and sample

a. Data source

The database for this study is the Department of Family and Community Services

Longitudinal Administrative Data Set (LDS). The LDS is created from administrative

records of social security payment receipt in Australia. It includes information on the

date on which any social security payment was made; type and amount of payment;

assets, income, and demographic characteristics of payment recipients (for example,

date of birth, country of birth, and family characteristics) (Department of Family and

Community Services, 2002). Payments are made at fortnightly intervals, and hence

that is the periodicity of the database. In this study a special-purpose data set from the

LDS is used - a 20 per cent sample of unemployment payment recipients who had

payment spells commencing between 1 January 1997 and 30 June 1999.

The LDS has advantages and disadvantages for evaluating the impact of WfD.

Heckman et al. (1998) suggest that the quality of any quasi-experimental evaluation


14/50

9

study using a matching method is likely to be significantly affected by three key

features whether data for treatment and control groups is collected using the same

survey instrument; whether it is possible to control at a detailed level for local labour

market conditions; and whether it is possible to match treatment and control

observations using labour market history. 1 On each of these criteria the LDS

performs well. First, data on WfD participants (treatment group) and WfD non-

participants (control group) can be drawn from the same database. Second, data on

the region of residence is available in the LDS at a highly disaggregated (postcode)

level. Third, the LDS allows variables to be constructed that provide a detailed

representation of unemployment payment history.

The main disadvantage of the LDS is that it does not provide information on payment

recipients for time periods where they are not receiving social security payments.

This has the important implication that, for unemployment payment recipients

observed to exit payments, it is not possible to determine labour market status or

income. Therefore, analysis of effects of activity test arrangements must focus on

outcomes that are related to receipt of unemployment payments.

b. Sample

In this study the potential sample of payment recipients is any person receiving NSA

at some time between 1 October 1997 and 30 June 1998. WfD participation is

identified from the WfD Compulsory and WfD Voluntary activity type

variables in the LDS. A payment recipient is classified as being a participant in the

pilot phase of WfD if their activity type in some fortnight during the period between

1 October 1997 and 30 June 1998 is one of the WfD categories. 2

To measure payment spell duration, it is necessary to have a procedure for defining

the starting date for a payment spell. A new spell on NSA is defined to begin if a

payment recipient has been off any social security payment for at least four

consecutive fortnights where that payment spell duration is less than or equal to 23

fortnights; or off all payments for at least seven consecutive fortnights where that

payment spell duration is more than 23 fortnights. 3 Exit from a spell is defined to


15/50

10

occur where a payment recipient is off unemployment-related payments (NSA) for at

least three consecutive fortnights. A payment recipient is defined to be on payments

in any fortnight in which they lodge a claim form (SU19) regardless of payment

entitlement.

5. Descriptive information on WfD participation

Results from descriptive analysis of participation in WfD during the pilot phase are

presented in Tables 1 to 4, and in Figures 1 and 2. There are 888 payment spells

during the period from November 1997 to June 1998 where the payment recipient is

observed to participate in WfD. These are concentrated mainly amongst persons aged

19 to 22 years. The size of the potential control group (payment recipients aged 18 to

24 years who do not participate in WfD during the sample period) is very large

about 65,000. Commencement of participation in WfD does not appear to be highly

concentrated at any single point in payment spell duration. Over 50 per cent of

participants began participation between 13 and 26 fortnights, about 25 per cent

between 27 and 52 fortnights, and about 10 per cent at longer than 52 fortnights.

Commencement of participation in WfD during the period of this study appears to

have been concentrated between December 1997 and April 1998.

There appears to be quite a high degree of geographic concentration of participation

in WfD between ABS Labour Force regions. For example, about 10% of the

population of individuals aged 18 to 24 years with payment spells during the sample

period are in regions where there is zero participation in WfD; by contrast, 53.2% of

participants in WfD are in regions where only 21.2% of those with payment spells

reside.

6. Effects of WfD Search theory

Participation in WfD may potentially have two types of effects on exit from payments

(and time on payments) first, a referral effect that causes an increase in the rate of

exit from payments at the time at which participation in WfD would be required to

begin; and second, an effect due to participation in WfD. In this study the focus is on


16/50

11

estimating the effect of participation in WfD. The standard approach to predicting the

effect of participation in WfD on exit from unemployment would be to use a search

theoretic labour market model (Pissarides, 2000). An objective of WfD is to improve

work habits of the unemployed. In the search theoretic framework this can be

represented as an increase in a job-seekers skills that should increase the arrival rate

of job offers, and hence increase the rate of outflow from unemployment.

7. Effects of WfD Empirical method

a. Outcome measures

The study will examine effects of WfD on a variety of outcome measures related to

receipt of unemployment payments. The WfD requirement is for a maximum six

months period. Outcome measures have been chosen to attempt to capture short-run

effects at the end of participation in WfD, and longer-run post-participation effects.

One measure will be the effect of WfD on the incidence of exit from payments by 6

months and 12 months after WfD commencement. A second measure will be the

effect of WfD on whether payment recipients are on payments at 9 months and 15

months after WfD commencement. The first and second measures will diverge where

payment recipients exit payments, but then begin a new payment spell that is on-going

at the specified duration. The third measure applied is the effect of WfD on the

number of fortnights on payments during the 6 months and 12 months after WfD

commencement.

Outcome measures

Effect of WfD on:

Incidence of exit from unemployment payments at 3 months after JSD

commencement;

Incidence of exit from unemployment payments at 6 months after JSD

commencement;

Incidence of receipt of unemployment payments at 6 months after JSD

commencement;


17/50

12

Incidence of receipt of unemployment payments at 9 months after JSD

commencement;

Total time in receipt of unemployment payments during 6 months after JSD

commencement; andTotal time in receipt of unemployment payments during 6 months after JSD

commencement.

b. Introduction

The empirical approach used to estimate the effect of the WfD is a quasi-experimentalmatching method. Fundamentally, this involves comparing payment outcomes for a

treatment group of NSA recipients who participate in WfD, and a matched control

group. In this sub-section the exact definition of the treatment and control groups,

and the policy effect identified, are described.

Participation in WfD can begin for an individual payment recipient at many different

payment spell durations; and occurs for different payment recipients throughout the

sample period. This potentially complicates the classification of payment spells as

treatment or control observations. Our basic approach is to define: (a) Treatment

group NSA recipients who commence WfD participation during the sample period;

and (b) Potential control group - NSA recipients who never commence WfD. Control

group payment recipients would be required to comply with the regular activity test

that involves a requirement to undertake job search and to nominate two job search

contacts made each fortnight.

Using this approach estimates of the effect of WfD participation are the average effect

of commencing participation in WfD for payment recipients aged 18 to 24 years who

commence participation during the pilot phase compared to matched payment

recipients who do not commence participation in WfD. Therefore, the estimated

effect of WfD participation is the average effect of treatment on the treated.

c. Motivation


18/50

13

For the quasi-experimental matching method to be a valid estimator of the JSD

treatment effect, it is sufficient that (Rubin, 1979):

(a) Conditional Independence Assumption (CIA) - Conditional on a set of observable

variables (X), participation in treatment is unrelated to outcomes in the absence of

treatment; and

(b) Common support assumption - For each possible combination of observable

variables there is a non-zero probability of non-participation.

Part (a) effectively requires that matching between treatment and control group

observations should be conditional on all variables that affect both participation in the

WfD and outcomes in the absence of the WfD (Augurzky and Schmidt, 2001). Or,

alternatively, after conditioning on the set of X variables, assignment between the

treatment and control groups is random. Part (b) is necessary to ensure that, for any

treatment group observation, there will be a control group observation with the

combination of observable characteristics to which the treatment observation can be

matched.

Almost certainly the most important issue in undertaking a matching analysis is to

justify why for the particular study being undertaken the CIA is likely to hold. In

this study we take two approaches to making that justification. First, treatment and

control group observations can be matched using a relatively rich set of covariates.

Most significantly, it is possible to match on the basis of local labour market

conditions, and unemployment payment history of each payment recipient. These two

factors have been identified as of particular importance in evaluations of matching

estimators (for example, Card and Sullivan, 1988, Heckman et al., 1999, and Kluve et

al., 2001). Although the LDS does not allow matching on some potentially important

covariates such as education attainment, in the Australian context this is likely to be

compensated for by being able to control for unemployment payment history. Recent

studies for Australia, using other data sources, establish the importance of labour

force history in explaining labour market status. Studies by Le and Miller (2001) and

Knights et al. (2002) have shown that once labour market history is controlled for,

other standard covariates have very little explanatory power for whether a labour

force participant is unemployed or employed. In this study of course it is payment


19/50

14

history rather than labour market history that is included as a covariate; however,

recent work by Moffitt (2001) suggests that total time on welfare payments is strongly

(inversely) related to an individuals employment rate.

The second justification for validity of the CIA is to suggest a likely source of

randomness in assignment of unemployment payment recipients between participation

and non-participation in the JSD. Our argument is that during the initial phase of its

operation, there was exogenous assignment of JSD participation between geographic

regions that was uncorrelated with local labour market conditions. This pattern of

geographic assignment which is explained by rationing of projects due to the size of

the program budget - effectively constitutes a source of randomness in assignment of

NSA/YA(o) payment recipients between treatment and control groups.

Figure 2 shows the distribution by ABS Labour Force region of all NSA payment

spells that are on-going during some part of the sample period, and payment spells

that involve WfD participation. With random assignment all observations should lie

along the 45 degree line. It is evident that there is a only a low degree of

concentration along the 45 degree line, and a relatively large number of observations

that lie some distance from the line. More formally, we apply the dartboard test

statistic for geographic randomness devised by Ellison and Glaeser (1997). That test

statistic measures the deviation of actual geographic concentration from predicted

concentration under an assumption of random distribution. More formally, we apply

the dartboard test statistic for geographic randomness devised by Ellison and

Glaeser (1997). That test statistic measures the deviation of actual geographic

concentration from predicted concentration under an assumption of random

distribution. The calculation of predicted concentration under the assumption of

random distribution incorporates information on (in this case) the size distribution of

WfD projects. Table 5 reports findings from the test. For two alternative geographic

classifications into 35 or 67 regions the difference between the actual and

predicted random degree of concentration appears highly significant. 4

The existence of geographic non-randomness appears to be explained by limited

funding initially allocated to the scheme so that only a small subset of applications for


20/50

15

projects were funded (see for example, Ewin Hannan, Plan to expand work for the

dole, The Age, January 14, 1998, p.1).

Second, it can be demonstrated that the geographic distribution of JSD participation is

not correlated with local labour market conditions. Figure 3 shows the rate of

unemployment and incidence of WfD participation by ABS Labour Force Region.

Appendix Table 1 (Panel A) reports results of a regression of the rate of

unemployment on the proportion of payment recipients participating in WfD by ABS

LFR. It is evident that the hypothesis of a significant relation can be rejected at the 5

per cent level.

This evidence is consistent with exogenous assignment of WfD participation between

geographic regions that is uncorrelated with local labour market conditions. On the

basis of this evidence our empirical approach is therefore to use local labour market

conditions (rate of unemployment) as a matching variable, but to still believe that

there is geographic randomness in assignment to WfD participation that is not

controlled for in the set of matching covariates.

d. Implementation

The type of quasi-experimental matching method used in this study is an exact

matching method. Treatment group observations (participants in WfD) are matched

with a control group observation(s) with the same:

1. Payment spell duration (for example, a payment recipient who begins participation

on WfD in the jth fortnight is matched with payment recipients who have on-going

spells in the jth fortnight, whose activity type in that fortnight is job search, and who

never participate in WfD) (65);

2. Quarter spell on WfD commences (3);

3. Age (18-20 or 21-24 years) (2);

4. Gender (2);

5. Country of birth (Australian-born; ESB immigrant; NESB immigrant) (3);

6. ATSI status (2);

7. Marital status (Single, Partner - not on payments; Partner - on payments) (3);

8. Activity type in previous fortnight (6);


21/50

16

9. Rate of unemployment in ABS Labour Force region (4); and

10. Unemployment payment history over previous 12 months (5).

The five categories of payment history are never on unemployment payments;

frequent/recent on payment; frequent/not recent on payments; not frequent/recent on

payments; and not frequent/not recent on payments. Frequent (not frequent) is

defined as being on payments in at least one fortnight in 3-4 (1-2) quarters in the

previous 12 months. Recent (not recent) is defined as being on payments in at least

one fortnight in the quarter immediately prior to commencement of the new payment

spell (not on payments in quarter immediately prior to commencement of new

payment spell).

As well, payment recipients are only used as a matching control group observation if

they have zero earnings from labour market activity in the fortnight of payment spell

duration at which their treatment observation commences participation in WfD. This

restriction is imposed since payment recipients were exempt from WfD if they had

positive earnings from labour market activity. 5

With this approach each treatment observation can potentially be classified in one of

3,369,000 cells. For each cell in which there is a treatment group observation and at

least one control group observation, differences in average outcomes are calculated

for control and treatment observations. A weighted average of the cell differences

(taking the proportion of treatment group observations in each cell as weights) is then

taken as the overall average effect of WfD participation. Payment recipients are only

used as a matching control group observation if they have zero earnings from labour

market activity in the fortnight

This exact matching estimator can be expressed formally as (Smith, 2002):

i j

1k 1k 1i 1k 0j 0k k ki k {D =1} i k {D =1}

= [n / n ][ (Y /n ) - (Y /n ) ]

where: are respectively the number of treatment and control observations

in cell k, are respectively outcomes for treatment group observation i, and

1k 0k n and n

1i 0jY and Y


22/50

17

control group observation j, and D is an indicator variable for participation in

treatment.

e. Standard errors

Using the type of standard errors commonly generated in statistical packages to test

for differences between treatment and control group outcomes involves an assumption

that only normal sampling variation exists. However, estimation of propensity

scores and the process of matching between treatment and control observations are

both extra sources of variation that need to be taken into account (Smith, 2000, p.13).

Therefore, in this study boostrapped standard errors are reported. The bootstrap

procedure involves several stages first, a sample with replacement is drawn from the

set of treatment and control observations used in the basic model equal to the total

number of treatment and control observations used; second, the basic model is

implemented to obtain an estimate of the WfD effect; and third, stages one and two

are repeated 1,000 times. The output is a distribution of estimated WfD effects for

each outcome measure. Five per cent confidence intervals from the boostrapped

estimates of WfD effects are reported. (These are generated as the 2.5 and 97.5

percentiles of the distribution of estimated WfD effects for each outcome measure.)

8. Effects of WfD

a. Preliminary information on matching

Using the exact matching method it is possible to match 802 (of the 888) WfD

participants to control group observations. The WfD participants and control

observations are matched into 727 cells. The median number of WfD participants in

each cell is one, and the median number of control observations in each cell is 21.

(Further information on the distribution of treatment and control observations by cell

is in Appendix Table 2.) Table 6 presents information on average characteristics of

the full and matched treatment groups. Some differences are apparent primarily

with regard to country of birth, indigenous status, and marital status. Since not all

WfD participants can be matched to control group observations, therefore the


23/50

18

estimated effects of WfD reported below are for a subset of WfD participants. In this

circumstance, where there is heterogeneity in the impact of WfD between

participants, the estimated effect can no longer be interpreted as the average effect of

treatment on WfD participants. Instead, it represents the average effect for WfD

participants with the same characteristics as those participants who can be matched

with control group observations. It would be a complex exercise to characterise this

type of treatment effect. Rather, the approach that will be applied to deal with this

problem is to use information on outcomes for matched and unmatched treatment

group observations to estimate upper and lower bounds on estimated program effects

(Lechner, 2000).

b. Basic results

Table 7 reports findings from the exact matching analysis. The main conclusion is that

there appear to be quite large significant adverse effects of participation in WfD. For

example, for the group of matched WfD participants it is found that there is a

difference in exit from NSA payments between WfD participants and non-participants

at 6 months after JSD commencement is equal to minus 12.1 percentage points (28.4

per cent for WfD participants compared to 40.6 per cent for non-participants). And

the difference in fortnights on NSA payments between WfD participants and non-

participants (in the first 12 months after start of spell on WfD) is 2.2 fortnights (11.13

fortnights on average for WfD participants compared to 10.14 fortnights on average

for non-participants). From the confidence intervals reported each of these

differences is statistically significantly different from zero at the 5% level.

Figures 4a and 4b show the proportions of WfD participants and the matched control

group who exit NSA payments in each month after the commencement of WfD

participation. Over the first 6 months after commencement there is a steadily

widening gap between outcomes for WfD participants and the matched control group;

after that time there is some reversal over the next 12 months. The difference after 6

months is 12.1 percentage points, and after 18 months is 8.9 percentage points and

over the final few months the gap appears largely to have stabilized. It is notable that

the break-point in the evolution of the difference in outcomes for WfD participants

and the matched control group is at 6 months, which is the duration of participation in


24/50

19

the WfD program. It suggests that there is an adverse effect of WfD on exit from

payments associated directly with the period of participation in WfD, but that there is

partial catch-up by WfD participants after the conclusion of WfD.

b. Bounds analysis

In the basic model 802 out of 888 WfD participants can be matched with control

observations. It has been noted earlier that where there is heterogeneity in WfD

effects - this may bias the estimate of the aggregate WfD effect.

As one way to address this potential problem we estimate upper and lower bounds for

the aggregate WfD effects (Lechner, 2000):

TTUB = ( ) + (1- )( ) ; and

TTLB = ( ) + (1- )( )

where = proportion of WfD participants matched with control observations; =

estimated average treatment on treated WfD effect from basic model; and

TT

and

are respectively the maximum and minimum possible values of the WfD effect for

example, for the outcome exit from payments by 3 months after WfD

commencement the maximum possible WfD effect is +1, and minimum possible

effect is 1. This method calculates lower and upper bound WfD estimates by

assuming that the estimated WfD effect applies to WfD participants who can be

matched to control observations, and that the effect for the non-matched WfD

participants is (respectively) as adverse and as positive as is possible.

Results from application of bounds analysis are reported in Table 8. It shows that for

each outcome measure the distribution of possible outcomes is heavily towards

adverse effects of WfD; although from the upper bound estimates (apart from the

outcome measure of exit from NSA by 6 months after WfD commencement) it is not

possible to exclude the possibility of a zero or small positive effect on receipt of

payments from WfD participation.


25/50

20

An alternative approach to estimating the aggregate effect of WfD is to impute WfD

effects for WfD participants who it was not possible to match with control

observations. This is done in two stages first, an OLS regression model is estimated

with the difference in outcome between WfD participants and matched control

observations as the dependent variable, and the matching covariates (payment spell

fortnight commence WfD spell is aggregated to 13 categories) as explanatory

variables; and second, results from the model are used to predict WfD effects for WfD

participants not matched with control observations. (This approach assumes that

effects of characteristics of payment recipients on WfD effects are homogenous

across cells.) The aggregate WfD effect is then calculated as a weighted average of

the estimated WfD effects for participants who could be matched, and imputed WfD

effects for participants who could not be matched. Estimated WfD effects using the

regression imputation approach are reported in Table 8. The effects are slightly lower

than the basic model, but are still statistically significant at the 5% level.

c. Results for disaggregated groups

Effects of WfD for disaggregated groups of payment recipients can be estimated, and

are of potential interest. Table 9 reports estimated WfD effects for groups of NSA

recipients disaggregated on the basis of characteristics used as covariates in the

matching analysis. There are several findings. First, WfD effects appear to be

slightly more adverse for females than males. Second, WfD effects are more adverse

for payment recipients aged 21 to 24 years than for those aged 18 to 20 years. Third,

participation in WfD has large negative effects on labour market outcomes for

payment recipients who have received payments in at least three quarters in the

previous 12 months compared or who did not receive payments in the previous 12

months, but has no significant effect for payment recipients who received payments in

one or two quarters in the previous 12 months. Hence there is not a consistent

ordering of WfD effects by payment history. Fourth, between geographic regions,

WfD participation has a significant adverse effect on labour market outcomes for

participants in regions with above-median rates of unemployment, but generally no

significant effect in below-median rate of unemployment regions. Finally, the

adverse effect of WfD participation is significant for payment recipients who

commence participation in fortnights 14-52 of their payment spells, but for other


26/50

21

durations does not have a significant effect (although this is probably mainly relate to

small sample sizes in other payment spell categories).

Table 10 reports estimated WfD effects for groups of NSA recipients disaggregated

on the basis of characteristics not used as matching covariates. These estimates are

obtained by classifying treatment observations on the basis of the extra characteristic

(for example, participate/not participate in JSD) as well as the matching covariates,

and then calculating a weighted average of the estimated WfD effects across those

categories. (Since we are classifying using a characteristic not included in the set of

matching covariates, it is not possible to calculate standard errors.) The findings

suggest that WfD effects are more adverse for NSA recipients who had not

participated in JSD in the previous 12 months, whose WfD classification was

compulsory, and who did not have earnings from labour market activity in the

fortnight prior to commencing in WfD. The findings appear plausible in that it is

likely that WfD participants without a history of labour market activity and who are

made to compulsorily to participate in WfD would have worse employment prospects,

and hence be less likely to exit payments. Nevertheless, even for the classifications

whose WfD outcomes are relatively better (such as payment recipients with earnings

from labour market activity, and who participate voluntarily in WfD), there are still

quite substantial adverse estimated effects of WfD participation.

d. Robustness checks

In this sub-section three types of robustness checks on the findings on the effects of

WfD are presented. One type of check is motivated by a concern that a referral effect

of WfD may cause bias in estimates of the effect of participation in WfD. The second

type of check is motivated by wanting to control for potential differences between

WfD participants and control group observations that have not been taken into

account in the analysis above. The third check is to examine the effect of an

alternative definition of exit from payments.

d.i. Referral effects


27/50

22

From the group of unemployment payment recipients referred to WfD some may exit

payments prior to participation, and others will begin participation. Where the

number of drop-outs is large this has two potential implications for quasi-

experimental matching analysis:

(i) Individual payment recipients in the control group may exit payments due to the

threat of participation in WfD; and

(ii) Individual payment recipients in the treatment group a group of those referred to

WfD who have decided to participate rather than exit payments - may differ in their

unobserved characteristics from other payment recipients.

Of these two effects, the first would tend to cause a downward bias in the estimated

program effect since part of the referral effect of treatment is being manifested in a

higher rate of exit from payments for the control group; and it is probably most

reasonable to characterize the second as having an ambiguous impact on the estimated

program effect (on the one hand, participation in WfD may reveal higher motivation;

on the other hand, participation may reveal an absence of work opportunities).

In order to investigate the potential effect of a referral effect on our findings, we

undertake a difference-in-difference comparison of rates of outflows from payments

between labour force regions (for 18-24 year old payment recipients) with different

levels of participation in WfD and between the periods prior to the beginning of the

WfD pilot phase and during the pilot phase. Evidence of a significant referral

effect of participation in WfD would be a larger increase (or smaller decrease) in the

rate of outflow from unemployment payments in regions with higher rates of

participation in WfD. (Individual-level data on referrals to WfD are not available for

the pilot phase of WfD. This is why it is necessary to adopt an aggregate-level

approach to testing for referral effects.)

Figure 5 presents data on a difference-in-difference measure of the rate of outflow

from payments, and the rate of participation in WfD by unemployment payment

recipients aged 18 to 24 years, by ABS Labour Force region. It is evident that there

does not appear to be a significant relation between the series. This is confirmed by

regression analysis that finds no relation between the series at the 10% level of

significance (see Panel B in Appendix Table 1).


28/50

23

Another possibility is that estimates of the effect of WfD may be biased by exit from

payments due to cancellation of payments for WfD related breaches. Payment

recipients whose payments are cancelled could potentially appear as control group

observations who exit payments. This would increase the rate of exit from payments

of the control group; but since the explanation for their exit is failure to participate in

WfD, this should be considered as a source of downward bias in the estimated effect

of WfD. To investigate this possible source of bias we excluded control group

observations with WfD-related breaches. The results are reported in Table 8, and it is

apparent that the estimated effects are almost identical to the basic model.

d.ii. Extra matching variables

For the matching estimator to provide valid estimates of the effect of the WfD

program, it must be that any treatment and control group observations matched on the

basis of the observable characteristics used in the exact matching, are otherwise

identical. To investigate the robustness of the basic model to introduction of another

matching variable that is likely to be an important predictor of labour market

outcomes and possibly WfD participation - education attainment (four categories

Not completed high school; completed high school; trade qualification/diploma; and

degree and above) is introduced as an extra matching characteristic. (This is not

included as a matching variable in the basic model as missing data for some treatment

and control observations would cause a significant reduction in sample size.)

Findings from this exercise are reported in Table 8. The estimated adverse effects of

WfD are found to be very similar to the basic model. This result adds to the degree of

confidence that can be attached to our belief that the matching covariates in the basic

model control in a satisfactory manner for differences in labour market outcomes and

likelihood of WfD participation between WfD participants and the control group.

d.iii. Alternative exit definition

In this exercise, exit from payments is defined to occur only where a NSA/YA(o)

recipient exits from all income support payments. This represents a stricter definition

of exit since exit will not now be defined to occur where a NSA/YA(o) recipient


29/50

24

exits from the unemployment-related allowance but commences a spell on some other

income support payment (such as Disability Support Pension (DSP)). Results in

Table 8 show that using this alternative definition does reduce somewhat the

estimated adverse effect of WfD participation; however, there is still an adverse effect

on receipt of payments that is statistically significant. The findings suggest that WfD

participants who exit NSA are slightly less likely to exit to other payment types than

the matched control group.

9. How to explain the findings?

What might explain negative effects of WFD participation on exit from payments?

There appear to be three main potential explanations:

(i) Stigma effects Participation in WfD may act as a negative signal to potential

employers, and hence decrease the rate of job offers for WfD participants relative to

non-participants;

(ii) Effect on job search activity Participation in WfD may allow participants to

reduce their job search activity, and may adversely affect the type of job search

activity undertaken; and

(iii) Scale of intervention The WfD program represents a relatively minimalist

intervention. It is designed only to provide work experience not training and for

that reason its effect the human capital of payment recipients is likely to be relatively

small. Therefore it should not be expected to have a significant impact on outcomes

for payment recipients (Curtain, 2001, pp.16-18).

Of these explanations, the potential effect of WfD on job search activity, seems of

most interest. There is a growing international literature that suggests that a lock-in

or attachment factor may be an important dimension of understanding the effects of

programs for unemployed. For example, a recent evaluation of the Community Work

Program in New Zealand found that many participants viewed their work experience

placements as work and therefore did not engage in job search activity (de Boer,

2000, p.6; see also Carling et al., 1996; and Van den Berg and van der Klaauw, 2001)

Some evidence from this study does appear consistent with the existence of a lock-

in effect of WfD participation due to a chilling effect on job search activity.


30/50

25

Detailed time-series analysis (Figure 4b) showed that the WfD effect on exit from

unemployment payments becomes progressively more negative throughout the

duration of the WfD program (from one to six months after commencement of WfD

spell); but that after the six-month duration point there is a partial reversal of the

negative effect of WfD. This suggests that WfD participants may reduce job search

activity relative to non-participants during the period of WfD participation hence

they are less likely than non-participants to exit payments. After WfD participation

concludes, those WfD participants still on payments may increase job search activity

and hence their rate of exit from payments is more rapid in the post-WfD phase than

for non-participants. But there is not complete catch-up. The difference in the

proportions of WfD participants and the matched group of non-participants who have

exited payments after 6 months is minus 12.1 percentage points, and after 18 months

is still minus 8.9 percentage points. This suggests that there may be some permanent

scarring effect on WfD participants. Such an effect could arise due to behavioural

changes in payment recipients as a result of WfD participation, or to employers

stigmatizing WfD participants. 6

However, there are also some aspects of our findings that appear inconsistent with an

effect of WfD on job search. For example, if it is correct that WfD disrupts job

search, it would be expected to have the largest negative effect on exit from payments

for unemployment payment recipients whose job search is otherwise most likely to

result in exit to a job. Since the effect of job search on exit to employment is likely to

be negatively correlated the rate of unemployment in the local labour market,

therefore on this argument, the largest negative effect of WfD would be expected for

those who reside in regions with the lowest rates of unemployment. By contrast, in

the results in Table 9, the impact of WfD appears to be most adverse in ABS LFRs

with highest rates of unemployment. Of course this argument assumes that WfD

participation has the same chilling effect on job search of all unemployment

payment recipients. Where the chilling effect of WfD on job search is largest for

unemployment payment recipients with lowest job finding probabilities and this

might occur due to psychological factors then this could explain the pattern of WfD

effects. This is an issue that it seems would require further investigation using

alternative empirical approaches to resolve in a satisfactory manner.


31/50

26

Endnotes

1. It is suggested access to a geographically-matched comparison groupadministered the same questionnaire as program participants and access to detailed

information on recent labor force status histories and recent earnings are essential inconstructing comparison groups that have outcomes close to those of an experimentalcontrol group (Heckman et al., 1999, p.1021).

2. Advice from FaCS and DEWR is that both categories should be treated asidentifying compulsory participants.

3. Our definition of new payment spells is slightly stricter than the FaCS definition.The Social Security Act 1991 defines a notional continuous period of receipt ofincome support payments as one in which the maximum break from payments in thefirst 12 months of payment receipt is 6 weeks, and in which the maximum break in

subsequent months is 13 weeks; and where a break in payments begins prior to, butwithin 6 weeks of, 12 months duration, the 13-week test applies.

4. Formally, for each case the difference between actual and predicted randomconcentration is at least four times greater than the standard deviation of the mean ofconcentration under the null hypothesis of randomness. Actual geographic dispersion

is measured as where are respectively the share of WfD

participants in ABS Labour Force Region (LFR) i and the share of payment recipientsin LFR i. The benchmark geographic dispersion for random assignment is

where

2

i iiG = (s -x ) is and xi

2

iiE(G) = (1- (x ) )H

2

jjH = (z ) , and = the proportion of WfD

participants in the jth project. Data on the number of participants in the pilot phaseWfD projects was obtained from Centrelink. For the variance formula see Ellison andGlaeser (1997, p.907).

jz

5. Information provided by Robert Lipp of DEWR.

6. Some evidence of employer stigma effects is the unwillingness of unemployedpersons to request employers at jobs they would genuinely like to be offered to fill outEmployer Contact Certificates see Tann and Sawyers (2000, p.16).


32/50

27

References

Augurzky, B. and C. Schmidt (2001), The propensity score: A means to an end,Discussion Paper no.271, IZA.

Blundell, R. and M. Costa Dias (2000), Evaluation methods for non-experimentaldata, Fiscal Studies, 21, 427-468.

Borland, J. and Y. Tseng (2002), How do administrative arrangements affect exitfrom unemployment payments? The case of the Job Seeker Diary in Australia,mimeo.

Card, D. and D. Sullivan (1988), Measuring the effect of subsidized trainingprograms on movements in and out of employment, Econometrica, 56, 497-530.

Carling, K., P. Edin, A. Harkman and B. Holmlund (1996), Unemployment duration,unemployment benefits, and labor market programs in Sweden, Journal of PublicEconomics, 59, 313-334.

De Boer, M. (2000), The outcomes and impact of expanded CTF and communitywork, Centre for Operational Research and Evaluation, Department of Work andIncome, New Zealand.

Department of Employment, Workplace Relations and Small Business (1999),Evaluation of the Work for the Dole pilot program, Report 1/99, Evaluation andMonitoring Branch.

Department of Employment, Workplace Relations and Small Business (2000), Workfor the Dole: A net impact study, mimeo, Evaluation and Program PerformanceBranch.

Department of Family and Community Services (2002a), Guide to Social SecurityLaw, accessed at http://www.facs.gov.au/guide/ssguide.

Department of Family and Community Services (2002b), FaCS LongitudinalAdministrative Data Set (LDS) 1% Sample, mimeo.

Ellison, G. and E. Glaeser (1997), Geographic concentration in U.S. manufacturingindustries: A dartboard approach, Journal of Political Economy, 105, 889-927.

Finn, D. (2001), A new deal for unemployed Australians?, mimeo, University ofPortsmouth.

Gerfin, M. and M. Lechner (2002), A microeconomic evaluation of the active labourmarket policy in Switzerland, Economic Journal, 112, 854-893.

Heckman, J. and J. Hotz (1989), Choosing among alternative nonexperimental

methods for estimating the impact of social programs: The case of manpowertraining, Journal of the American Statistical Association, 84, 862-874.


33/50

28

Heckman, J., H. Ichimura, J. Smith and P. Todd (1998), Characterizing selection biasusing experimental data, Econometrica, 66, 1017-1098.

Heckman, J., R. Lalonde and J. Smith (1999), The economics and econometrics of

active labor market programs, pages 1865-2097 in O. Ashenfelter and D. Card (eds.)Handbook of Labor Economics Volume 3A (Amsterdam, Elsevier).

Kluve, J., H. Lehmann, and C. Schmidt (2001), Disentangling treatment effects ofPolish active labour market policies: Evidence from matched samples, mimeo, IZA.

Kluve, J. and C. Schmidt (2002), Can training and employment subsidies combatEuropean unemployment?, Economic Policy,

Knights, S., M. Harris and J. Loundes (2002) Dynamic relationships in the Australianlabour market: Heterogeneity and state dependence, Economic Record, 78, 284-298.

Le, A. and P. Miller (2001), Is a risk index approach to unemployment possible?,Economic Record, 77, 51-70.

Lechner, M. (2000), A note on the common support problem in applied evaluationproblems, mimeo, SIAW, University of St.Gallen.

Moffitt, R. (2001), Experience-based measures of heterogeneity in the welfarecaseload, forthcoming in C. Citro, R. Moffitt and S. Ver Ploeg (eds.) Data Collectionand Research Issues for Studies of Welfare Populations (Washington, NationalAcademy Press).

OECD (2001) Innovations in Labour Market Policies: The Australian Way (Paris).

Productivity Commission (2002), Independent Review of Job Network Draft Report(Canberra, Ausinfo).

Reference Group on Welfare Reform (2000), Participation Support for a MoreEquitable Society: Interim Report (Canberra).

Richardson, L. (2003), The Mutual Obligation Initiative and the Income Support

Dynamics of Youth Unemployment Benefit Recipients: An Empirical Analysis,unpublished Ph.D thesis, Australian National University.

Robinson, P. (2000), Active labour-market policies: A case of evidence-basedpolicy-making?, Oxford Review of Economic Policy, 16, 13-26.

Rubin, D. (1979), Using multivariate matched sampling and regression adjustment tocontrol bias in observational studies, Journal of the American Statistical Association,7, 34-58.

Smith, J. (2000), A critical survey of empirical methods for evaluating active labour

market policies, Swedish Journal of Economics, 136, 1-22.


34/50

29

Smith, J. (2002), Evaluation of labor market programs: part II, Lecture notes, ANU.

Tann, T. and F. Sawyers (2000), Survey of FaCS unemployed people: Attitudestowards the Activity Test, mimeo, Department of Family and Community Services.

Van den Berg, G. and B. van der Klaauw (2001), Counseling and monitoring ofunemployed workers: Theory and evidence from a controlled social experiment,Discussion paper no.374, IZA.


35/50

30

Table 1: Payment spells of NSA recipients aged 18 to 24 years with at least one

fortnight participation in WfD, November 1997 to June 1998

Number of observations Frequency

Age

18 58 6.5319 206 23.220 141 15.8821 149 16.7822 131 14.7523 107 12.0524 96 10.81Total 888 100Table 2: Distribution of starting date on WfD by duration of payment spell -

NSA recipients aged 18 to 24 years with at least one fortnight participation inWfD, November 1997 to June 1998

Number of observations Cumulative frequency

Duration - fortnights

1 4 0.452 1 0.563 7 1.354 8 2.255 5 2.816 7 3.6

7 4 4.058 8 4.959 7 5.7410 8 6.6411 14 8.2212 11 9.4613 22 11.9414 24 14.6415 35 18.5816 49 24.1

17 48 29.5118 45 34.5819 29 37.8520 42 42.5821 38 46.8622 38 51.1423 42 55.8724 41 60.4925 37 64.6626 25 67.4827-39 162 85.7240-52 58 92.2853+ 69 100


36/50

31

Table 3: Distribution of starting date on WfD by month - NSA recipients aged

18 to 24 years with at least one fortnight participation in WfD, November 1997 to

June 1998

Number of observations FrequencyMonth

1997 November 6 0.681997 - December 119 13.41998 January 164 18.471998 February 206 23.21998 March 164 18.471998 April 111 12.51998 May 62 6.981998 June 56 6.31


37/50

32

Table 4: Participation in WfD by ABS Labour Force region - NSA recipients

aged 18 to 24 years with at least one fortnight on payments, November 1997 to

June 1998

LabourForce

Region Par ticipate in WfD Percent A ll Payment Spel ls Percent

2 0 0 1253 1.78

4 0 0 624 0.89

5 22 2.48 917 1.30

6 14 1.58 916 1.30

7 45 5.07 1436 2.04

8 11 1.24 892 1.27

9 3 0.34 813 1.15

10 4 0.45 879 1.25

11 0 0 356 0.5112 1 0.11 224 0.32

13 0 0 340 0.48

14 24 2.7 1046 1.48

15 0 0 178 0.25

17 33 3.72 1049 1.49

18 2 0.23 496 0.70

19 31 3.49 2025 2.87

20 13 1.46 624 0.89

21 31 3.49 421 0.60

22 0 0 1173 1.66

23 61 6.87 2237 3.1724 25 2.82 1784 2.53

25 14 1.58 959 1.36

28 2 0.23 1237 1.75

29 9 1.01 2526 3.58

30 0 0 1209 1.72

31 0 0 1566 2.22

32 6 0.68 1347 1.91

33 17 1.91 917 1.30

34 11 1.24 1195 1.70

35 18 2.03 1461 2.07

37 4 0.45 1317 1.8738 31 3.49 830 1.18

39 22 2.48 834 1.18

40 18 2.03 985 1.40

41 2 0.23 972 1.38

42 63 7.09 869 1.23

44 15 1.69 1805 2.56

45 16 1.80 1939 2.75

46 7 0.79 1399 1.98

47 34 3.83 1585 2.25

48 0 0 70 0.10

49 9 1.01 1350 1.92

50 41 4.62 928 1.32


38/50

33

51 21 2.36 1414 2.01

52 8 0.90 779 1.11

53 13 1.46 1039 1.47

54 7 0.79 1110 1.57

55 11 1.24 2051 2.91

56 7 0.79 252 0.3658 13 1.46 1540 2.18

59 1 0.11 953 1.35

60 4 0.45 899 1.28

61 6 0.68 1228 1.74

62 23 2.59 768 1.09

63 5 0.56 523 0.74

66 2 0.23 452 0.64

67 2 0.23 730 1.04

68 9 1.01 1623 2.30

69 18 2.03 1033 1.47

70 19 2.14 1165 1.6571 4 0.45 796 1.13

72 14 1.58 907 1.29

75 24 2.70 1060 1.50

76 5 0.56 590 0.84

77 8 0.90 474 0.67

79 3 0.34 932 1.32

80 2 0.23 1185 1.68


39/50

34

Table 5: Dartboard test for geographic randomness in distribution of WfD

participants, November 1997 to June 1998

Index E(G) Index-E(G) SD(G)

(Actual) (Random) (Diff.)35 regions 0.01377 0.00701 0.00676 0.00177

67 regions 0.01483 0.00710 0.00773 0.00123


40/50

35

Table 6: Characteristics of WfD participants and all payment recipients - NSA

recipients aged 18 to 24 years with at least one fortnight on payments, November

1997 to June 1998

TreatmentMatched

Treatment

Control(Characteristic

at 1 Oct)

No. of Observations 888 802 65481

Mean(age) 21.41271 21.28998 21.38112

Female 307 277 27127

34.57 34.54 41.43

Male 581 525 38354

65.43 65.46 58.57

Age

18-20 405 375 2928445.61 46.76 44.72

21-24 483 427 36197

54.39 53.24 55.28

Country of Birth

Australia 802 741 56096

90.32 92.39 85.67

ESB Immigrant 28 17 3367

3.15 2.12 5.14

NESB Immigrant 58 44 6018

6.53 5.49 9.19

Payment HistoryNo UB history 498 466 39814

56.08 58.1 60.8

Not frequent / not recent 251 221 15863

28.27 27.56 24.23

Frequent / not recent 110 90 7054

12.39 11.22 10.77

Not frequent / recent 7 6 1220

0.79 0.75 1.86

Frequent / recent 22 19 1530

2.48 2.37 2.34

ATSI statusNon ATSI 784 790 57760

88.29 98.5 88.21

ATSI 104 12 7721

11.71 1.5 11.79

Marital Status

Single 798 750 60055

89.86 93.52 91.71

Married (partner on IS) 11 6 1679

1.24 0.75 2.56

Married (partner not on IS) 79 46 3747

8.9 5.74 5.72


41/50

36

Children

Have no child 848 772 63727

95.5 96.26 97.32

Have child 40 30 1754

4.5 3.74 2.68

State of ResidenceACT 2 2 1144

0.23 0.25 1.75

NSW 334 301 18936

37.61 37.53 28.92

VIC 203 188 16160

22.86 23.44 24.68

QLD 189 165 14615

21.28 20.57 22.32

SA 52 49 5492

5.86 6.11 8.39

WA 68 62 62777.66 7.73 9.59

TAS 37 32 1969

4.17 3.99 3.01

NT 3 3 888

0.34 0.37 1.36


42/50

37

Table 7: Effect of WfD on exit from payments and time on payments Exact

matching method - NSA recipients aged 18 to 24 years, November 1997 to June

1998

Treatment Control Difference 5%

confidence

interval

% Off

payments

By 3 months 28.4 40.5 -12.1 (-16.2 ~ -8.5)

By 6 months 50.0 60.5 -10.5 (-14.9 ~ -6.7)

% On

payments

At 6 months 67.7 59.2 8.5 (1.9 ~ 12.5)

At 12 months 58.7 49.8 8.9 (4.8 ~ 12.8)

Time on

payments

First 6 months 11.13 10.14 0.99 (0.37 ~ 1.25)

First 12 months 19.79 17.59 2.20 (1.60 ~ 2.87)

number of

observations

Observationsmatched

802

Total no. of

observations

888


43/50

Table 8: Effect of WfD on exit from payments and time on payments Exact matching method Robu

aged 18 to 24 years, November 1997 to June 1998

Off payment On

payments

Time on

payments(fortnights)

6 months 12 months 9 months 15 months 6 months 1

Basic model -12.1(-16.2 ~ -8.5)

-10.5(-14.9 ~ -6.7)

8.5(1.9 ~ 12.5)

8.9(4.8 ~ 12.8)

0.99(0.74 ~ 1.25)

Bounds analysis

Lower bound -20.7 -19.1 17.4 17.7 2.05

Upper bound -1.3 0.2 -2.0 -1.7 -0.27

Regression adjusted

average effects for all

treatment observations

-11.2(-15.7 ~ -7.0)

-10.0(-15.1 ~ -5.1)

7.9(3.4 ~ 12.1)

8.7(3.9 ~ 13.4)

0.91(0.60 ~ 1.21)

Basic model

Excluding individuals

with WFD breaches

-11.9(-15.8 ~ -8.2)

-10.1(-14.7 ~ -6.2)

8.5(4.6 ~ 12.5)

8.5(4.3 ~ 12.8)

0.99(0.73 ~ 1.27)

Basic model - Plus

education attainment

-11.7(-16.2 ~ -8.0)

-10.0(-15.6 ~ -5.9)

8.3(4.3 ~ 13.1)

6.9(2.9 ~ 12.4)

1.00(0.69 ~ 1.31)

Basic model Exit all

payment

-10.2(-14.4 ~ -6.7)

-9.0(-13.6 ~ -5.3)

6.3(2.8 ~ 10.3)

7.0(3.2 ~ 11.1)

0.86(0.62 ~ 1.14)


44/50

Table 9: Effect of WfD on exit from payments and time on payments Exact matching method - NSA r

November 1997 to June 1998

Off payment On payments Time on paymen6 months 12 months 9 months 15 months 6 months

Basic model -12.1(-16.2 ~ -8.5)

-10.5(-14.9 ~ -6.7)

8.5(1.9 ~ 12.5)

8.9(4.8 ~ 12.8)

0.99(0.74 ~ 1.25)

Disaggregated:

Gender

Female -14.5(-20.4 ~ -8.0)

-10.0(-17.2 ~ -3.2)

11.3(4.2 ~ 17.9)

8.1(0.9 ~ 15.4)

1.17(0.67 ~ 1.66)

Male -10.9

(-16.0 ~ -6.8)

-10.7

(-15.9 ~ -6.3)

7.1

(2.8 ~ 11.9)

9.3

(4.4 ~ 13.9)

0.89

(0.61 ~ 1.25)Age

Less than 21 years -10.6(-16.1 ~ -5.3)

-8.5(-15.4 ~ -3.1)

6.2(0.6 ~ 12.0)

5.5(-0.5 ~ 11.3)

0.85(0.49 ~ 1.24)

21 years andabove

-13.5(-18.6 ~ -8.8)

-12.2(-18.0 ~ -6.9)

10.6(5.6 ~ 15.9)

11.9(6.3 ~ 17.4)

1.10(0.74 ~ 1.47)

Payment history

No UB history -13.0(-17.7 ~ -8.4)

-12.2(-17.0 ~ -6.9)

9.0(4.0 ~ 13.8)

10.6(5.0 ~ 15.3)

0.99(0.64 ~ 1.34)

Not frequent -8.8(-16.6 ~ -1.7)

-4.7(-14.0 ~ 3.2)

2.6(-4.9 ~ 10.7)

4.3(-3.4 ~ 13.1)

0.90(0.39 ~ 1.40)

Frequent -15.7(-27.7 ~ -6.2)

-15.2(-28.7 ~ -3.9)

18.6(8.6 ~ 28.7)

11.0(0.4 ~ 23.6)

1.16(0.44 ~ 1.92)


45/50

Region UE

1stquartile -15.4(-24.3 ~ -7.1)

-8.4(-17.6 ~ -0.3)

4.6(-4.2 ~ 12.8)

5.7(-3.5 ~ 14.5)

0.96(0.27 ~ 1.63)

2ndquartile -8.3

(-17.7 ~ 1.7)

-1.4

(-11.2 ~ 9.7)

-2.1

(-13.9 ~ 7.3)

4.0

(-7.3 ~ 14.6)

0.89

(0.17 ~ 1.62)3rdquartile -7.9

(-15.0 ~ -0.6)-7.9

(-15.6 ~ -0.5)9.6

(2.4 ~ 16.2)8.1

(-0.4 ~ 15.7)0.82

(0.31 ~ 1.31)

4thquartile -15.0(-20.9 ~ -9.8)

-17.0(-24.2 ~ 10.6)

14.3(8.9 ~ 20.5)

13.1(6.6 ~ 20.0)

1.15(0.77 ~ 1.57)

Fortnight

commence WfD

1-13 -5.0(-16.3 ~ 5.1)

-8.9(-20.2 ~ 1.

does ‘work for the dole’ work?

Documents