social security policy branch – random sample surveys
TRANSCRIPT
Social Security Social Security
AssuranceAssurance
Tracking Process Error Tracking Process Error
using Sample Surveysusing Sample Surveys
ITSEW 2009 TallbergITSEW 2009 Tallberg
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Cost of Compliance Customer Attitude to Compliance Compliance Strategy Debt Recovery Strategy
Reparation sought through the courts
FaCSIA Fraud and Compliance Response
Those customers whose deliberate and organised non-compliance results in a significant attack on the integrity of the payment
Fraud Investigation/Prosecution Action/Criminal Conviction
Those who deliberately not comply
Targetted Compliance Activities e.g. Data Matching, Compliance Reviews
Those who do not take reasonable care to comply
Those who do not take reasonable care to comply
Those who do not take reasonable care to comply
Those who get it right
Small Debt Waiver: <$50
Adapted from Braithwaite Regulation Model
Those who try to comply but make honest mistakes/do not understand their requirements
Education Programmes e.g. Real Estate and Overseas Pensions publicity campaigns
Targetted Prevention e.g. Service Profiling
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
• Began in 98/99
• Continues as an ongoing project
• Methodology tested, improved and ratified by External Consulting
Actuaries
• Audited by the Australian National Audit Office (ANAO)
• Recommendations have been implemented
Random Sample Surveys Background
Social Security Policy Branch – Random Sample Surveys
Service level agreement
• FaHCSIA and Centrelink have an SLA in place to manage the RSS
• SLA is a set of agreed processes and procedures to be followed by both
parties
• Living Document – can be modified to meet changing political or
departmental requirements
Social Security Policy Branch – Random Sample Surveys
Sample Design
• Define Sample Population (Age Pension, Carer Payment)
• Make Exclusions where Necessary
• Define Strata – Population Groups defined by characteristics (eg Gender,
Rate, Marital Status)
• Consideration for Remote Customers (Clustering)
• Allocate Sample (operational requirements and strata)
Social Security Policy Branch – Random Sample Surveys
Exclusions and Substituons
• In an Ideal sample ALL customers have a chance of selection
• RSS Population is not a closed system
• Need to balance randomness of the sample against burden to the
customers
• Exclusion Rules ensure that:
• customers that have an outstanding or recently completed review;
• customers that are seriously ill or out of the country;
Are not selected
Social Security Policy Branch – Random Sample Surveys
Stratification
• The target population is divided into discrete, non-overlapping sub
populations
• This assists in ensuring a Representative Sample is selected
• Exploits the depth of administrative information held on populations
under review
• Sharpens the final estimates around variables associated with
compliance status
• Allows for unequal probability selection
Social Security Policy Branch – Random Sample Surveys
RSS SAMPLE SPECIFICATION 2008-09 - DISABILITY SUPPORT PENSION
STRATUM ID POP Mar08 SAMPLE SKIPBY TRI MESTER
D01 22,140 <50yrs 128 172 43D02 47,143 50+yrs 187 252 62D03 33,402 <50yrs 158 212 53D04 84,984 50+yrs 252 338 84D05 35,302 <35yrs 162 218 54D06 54,720 35-49yrs 202 271 67D07 90,896 50+yrs 260 349 87D08 54,656 <35yrs 202 271 67D09 74,531 35-49yrs 236 316 79D10 93,920 50+yrs 265 355 88D11 52,644 198 266 66D12 42,384 178 239 59D13 46,629 165 283 55D14 46,736 165 283 55
3000TOTAL 780,088 2757
Note: sample is allocated using Caroll formula (that is proportional to the square root of the stratum population)The allocation of sample between nonremote and remote areas is proportional to respective customer populations
Nonremote
Max rate
Partnered
Female
Male
Unpartnered
Female
Male
Reduced rate
STRATIFICATION
PartneredUnpartnered
RemotePartnered
Unpartnered
Social Security Policy Branch – Random Sample Surveys
Clustering - Background
• Centrelink’s most remote Areas (North & Central QLD,
North Australia, Western Australia) have smaller
customer populations over wider geographic areas
• Need to balance the requirement to sample in these
locations with the cost of travel and distance
• Clustering concentrates the selections to a smaller
location within the remote Area
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Selection
• Design Delivered to Centrelink
• Random Selection of Customers within Allocation Boundaries by
Centrelink
• FaHCSIA views selection
• Centrelink proceeds after approval
Social Security Policy Branch – Random Sample Surveys
Reviews
• Selected Customer attends an Interview in Centrelink Office (CSC or ASO)
• Full Re-Assessment of Entitlement conducted via Questionnaire (Income,
Assets, Residency, POI, Qualification, Accommodation)
• More Detailed information from Review filled out by Centrelink Officer after
review concluded and entered into RRRS
• Quality Assurance is performed by Centrelink on a sample of the results
Social Security Policy Branch – Random Sample Surveys
Review Results
Assessment of Current and Prior Entitlement can have the following results
• Cancellation of Payment
• Variation (Upward or Downward) of Current Fortnightly Rate
• Debt Raised for an overpayment in an earlier period
• Arrears Paid for an underpayment in an earlier period
These Results are not mutually exclusive. A Cancellation/Variation can
happen in conjunction with a Debt and/or Arrears.
Supplementary Components (Eg RA, PHA, TAL) are also reviewed.
Social Security Policy Branch – Random Sample Surveys
FaHCSIA Processing
• Centrelink Review Results Stored on ISIS/IRS/RRRS
• Data is Transmitted from Centrelink to FaHCSIA
• FaHCSIA tests the data integrity
• Validates a sample (10%) of the Reviews - (Centrelink QAs this sample)
• Tabulates and Analyses the Results
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Results
• Results used by FaHCSIA to:
• Provide an estimate FaHCSIA Programme Outlay Accuracy
• Identify Emerging Risks
• Assess effectiveness of Compliance Budget measures
• Evidence Base for Compliance Policy
• Latent Debt Measure – Debt KPI
• Improve predictors for Service Profiling
Social Security Policy Branch – Random Sample Surveys
Outlay Accuracy
• Modelled Estimate of the Mispayed Amount / Outlayed Amount subtracted
from 100%
• Does not discriminate between customer and Centrelink errors
• Weighted by $ Value – a $450 cancellation has a greater effect than a
$0.05 upward variation
• Only counts current episodes of mispayment ie. is not affected by debts for
prior periods of mispayment
• Sourced from Dataset 1
Social Security Policy Branch – Random Sample Surveys
Payment Correctness• Modelled Estimate of the Number of Customers with an Administrative Error that affected
Payment / Total Population subtracted from 100%
• Only counts customers where the error was made by Centrelink which caused a dollar impact
• Does include dollar impact from debts (ie prior periods of mispayment)
• Does not count multiple errors – if any 1 of up to 4 Errors meets the criteria the customer is
counted once
• Not weighted by dollar value - Counts customers not amounts – a $35,000 debt contributes
the same as a $0.05 variation
• Sourced from Dataset 4
Social Security Policy Branch – Random Sample Surveys
Latent Debt
• Measure of the level of undetected indebtedness in the benefit population
• Key Prevention Indicator for Debts is the Latent Measure of
Indebtedness to Outlays
• Make the distinction between “Current Year” Indebtedness and New
Debts
Social Security Policy Branch – Random Sample Surveys
Figure 1.1 Current Year Indebtedness to Outlay (Latent measure) by Benefit, 2003-2007
12 month moving aggregate, 1 Standard Error bar
0.00%
0.20%
0.40%
0.60%
0.80%
1.00%
1.20%
1.40%
Ind
ebte
dn
ess:
ou
tlay
(%
)
AGE DSP New debts AGE New debts DSP CAR New debts CAR
Social Security Policy Branch – Random Sample Surveys
Addressing emerging risktop 10 reasons for mispayment• Random Sample Results are further subdivided by reason for
mispayment
• These indicate the Risks to Outlays to be addressed by Compliance
and Prevention measures
• Mispayments in debt and variation can be for the same or different
reasons
• Risks are ranked by number (incidence) and value (magnitude)
• Charts observe the Higher risks to outlays
Social Security Policy Branch – Random Sample Surveys
Age Pension - Top 10 Risks as at T1 08/09
$0 $2,000 $4,000 $6,000 $8,000 $10,000 $12,000 $14,000
Investment income from banks/buildingsocieties etc
Assets - value of investments
Assets - property/real estate
Income from income stream
Employment income
Other
Self employed - income from businessprecludes payment
Income from foreign pension
Investment income from shares
Assets - business (including Farm)
Rea
son
for
Var
iati
on
FN Variation Amt ($)
12 Months to T1 08/09
12 Months to T3 07/08
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Trimester 1 Trimester 2 Trimester 3No of Reviews 1,666 1,667 1,664 4,997Fortnightly Amt 689,241.64 730,792.91 693,409.48 2,113,444.03Variation Amt 12,076.42 13,117.55 10,805.92 35,999.89Accuracy 98.25% 98.21% 98.44% 98.30%No Debts 623 626 537 1,786Debt Amt 309,569.68 333,658.37 325,313.97 968,542.02No of Reviews 499 498 499 1,496Fortnightly Amt 207,949.68 214,720.42 205,873.27 628,543.37Variation Amt 7,410.93 5,444.86 5,590.83 18,446.62Accuracy 96.44% 97.46% 97.28% 97.07%No Debts 133 134 133 400Debt Amt 82,086.46 104,244.20 127,261.13 313,591.79No of Reviews 84 83 83 250Fortnightly Amt 8,337.32 8,406.98 8,349.80 25,094.10Variation Amt 98.50 100.60 201.20 400.30Accuracy 98.82% 98.80% 97.59% 98.40%No Debts 0 0 1 1Debt Amt 0.00 0.00 359.12 359.12No of Reviews 919 919 920 2,758Fortnightly Amt 407,118.93 420,268.90 430,148.63 1,257,536.46Variation Amt 13,461.32 10,819.27 5,823.34 30,103.93Accuracy 96.69% 97.43% 98.65% 97.61%No Debts 190 171 148 509Debt Amt 283,469.87 197,850.06 87,941.00 569,260.93No of Reviews 0 791 716 1,507Fortnightly Amt 0.00 112,820.11 93,517.24 206,337.35Variation Amt 0.00 2,654.63 2,326.26 4,980.89Accuracy #DIV/0! 97.65% 97.51% 97.59%No Debts 0 17 8 25Debt Amt 0.00 13,484.20 2,902.52 16,386.72No of Reviews 0 498 0 498Fortnightly Amt 0.00 178,507.46 0.00 178,507.46Variation Amt 0.00 6,916.30 0.00 6,916.30Accuracy #DIV/0! 96.13% #DIV/0! 96.13%No Debts 0 161 0 161Debt Amt 0.00 63,009.87 0.00 63,009.87
Annual Total
Random Sample Surveys - Actual Outlay Accuracy
Carer Allowance
Disability Support Pension
Family Tax Benefit
Special Benefit
Benefit Reviewed Survey Period
Age Pension
Carer Payment
Social Security Policy Branch – Random Sample Surveys
Trimester 1 Trimester 2 Trimester 3
No of Reviews 1,666 1,667 1,664 4,997Fortnightly Amt 689,241.64 730,792.91 693,409.48 2,113,444.03Variation Amt 12,132.41 13,073.02 11,186.50 36,391.93Accuracy 98.24% 98.21% 98.39% 98.28%No Debts 616 619 533 1,768Debt Amt 309,131.03 321,667.94 325,258.36 956,057.33No of Reviews 499 498 499 1,496Fortnightly Amt 207,949.68 214,720.42 205,873.27 628,543.37Variation Amt 7,410.93 5,404.19 5,545.27 18,360.39Accuracy 96.44% 97.48% 97.31% 97.08%No Debts 133 130 131 394Debt Amt 82,347.65 98,154.77 126,986.99 307,489.41No of Reviews 84 83 83 250Fortnightly Amt 8,337.32 8,406.98 8,349.80 25,094.10Variation Amt 98.50 100.60 201.20 400.30Accuracy 98.82% 98.80% 97.59% 98.40%No Debts 0 0 1 1Debt Amt 0.00 0.00 359.12 359.12No of Reviews 919 919 920 2,758Fortnightly Amt 407,118.93 420,268.90 430,148.63 1,257,536.46Variation Amt 13,927.48 10,838.34 5,823.34 30,589.16Accuracy 96.58% 97.42% 98.65% 97.57%No Debts 188 169 146 503Debt Amt 283,193.39 195,016.42 87,890.70 566,100.51No of Reviews 0 791 716 1,507Fortnightly Amt 0.00 112,820.11 93,517.24 206,337.35Variation Amt 0.00 2,486.65 2,263.36 4,750.01Accuracy #DIV/0! 97.80% 97.58% 97.70%No Debts 0 17 9 26Debt Amt 0.00 13,100.00 2,987.32 16,087.32No of Reviews 0 498 0 498Fortnightly Amt 0.00 178,507.46 0.00 178,507.46Variation Amt 0.00 8,747.94 0.00 8,747.94Accuracy #DIV/0! 95.10% #DIV/0! 95.10%No Debts 0 152 0 152Debt Amt 0.00 60,948.71 0.00 60,948.71
Family Tax Benefit
Special Benefit
Age Pension
Carer Payment
Carer Allowance
Disability Support Pension
Random Sample Surveys - Restored Outlay AccuracyBenefit Reviewed Survey Period
Annual Total
Social Security Policy Branch – Random Sample Surveys
Trimester 1 Trimester 2 Trimester 3
Actual 98.25% 98.21% 98.44% 98.30%Restored 98.24% 98.21% 98.39% 98.28%Actual 96.44% 97.46% 97.28% 97.07%Restored 96.44% 97.48% 97.31% 97.08%Actual 98.82% 98.80% 97.59% 98.40%Restored 98.82% 98.80% 97.59% 98.40%Actual 96.69% 97.43% 98.65% 97.61%Restored 96.58% 97.42% 98.65% 97.57%Actual #DIV/0! 97.65% 97.51% 97.59%Restored #DIV/0! 97.80% 97.58% 97.70%Actual #DIV/0! 96.13% #DIV/0! 96.13%Restored #DIV/0! 95.10% #DIV/0! 95.10%
Family Tax BenefitSpecial Benefit
Random Sample Surveys - Restored Outlay AccuracyBenefit Reviewed Survey Period
Annual Total
Age Pension
Carer Payment
Carer Allowance
Disability Support Pension
Social Security Policy Branch – Random Sample Surveys
Downward Variation
No Review Conducted
Present Rate to
Continue
Upward Variation
Downward Variation 1 7 3 2 13Present Rate to Continue 2 13 15 1 31Upward Variation 1 3 2 3 9total 4 23 20 6 53Downward Variation 1 1 3 0 5Present Rate to Continue 0 6 28 0 34Upward Variation 0 1 0 0 1total 1 8 31 0 40Present Rate to Continue 0 1 2 0 3total 0 1 2 0 3Present Rate to Continue 3 12 18 1 34Upward Variation 0 1 1 0 2total 3 13 19 1 36Cancellation 0 2 0 0 2Present Rate to Continue 0 11 18 0 29Upward Variation 0 0 1 0 1total 0 13 19 0 32Cancellation 0 0 1 0 1Downward Variation 1 0 2 2 5Present Rate to Continue 4 0 14 2 20Upward Variation 0 0 2 3 5total 5 0 19 7 31
Special Benefit
Random Sample Surveys - Replaced Case Result ComparisonBenefit
ReivewedActual Result
Carer Payment
Carer AllowanceDisability Support PensionFamily Tax Benefit
Restored Result total
Age Pension
Social Security Policy Branch – Random Sample Surveys
1 2 3No of Reviews 24 15 14 53Fortnightly Amt 10,320.63 6,962.89 5,984.37 23,267.89Actual Var Amt 57.12 93.42 74.7 225.24Restored Var Amt 113.11 48.89 455.28 617.28Actual No. Debts 7 8 4 19Restored No. Debts 0 1 0 1Actual Debt Amt 438.65 12,011.77 55.61 12,506.03Restored Debt Amt 0 21.34 0 21.34No of Reviews 15 13 12 40Fortnightly Amt 6,372.74 5,642.59 4,544.09 16,559.42Actual Var Amt 0 41.27 45.56 86.83Restored Var Amt 0 0.6 0 0.6Actual No. Debts 1 5 2 8Restored No. Debts 1 1 0 2Actual Debt Amt 30.76 6,092.87 274.14 6,397.77Restored Debt Amt 291.95 3.44 0 295.39
Age Pension
Benefit Reivewed
Result Component
Carer Payment
Period Total
Social Security Policy Branch – Random Sample Surveys
No of Reviews 2 0 1 3Fortnightly Amt 197.00 0.00 100.60 297.60Actual Var Amt 0 0 0 0Restored Var Amt 0 0 0 0Actual No. Debts 0 0 0 0Restored No. Debts 0 0 0 0Actual Debt Amt 0 0.00 0 0.00Restored Debt Amt 0 0 0 0No of Reviews 16 10 10 36Fortnightly Amt 7,589.88 4,010.15 4,923.61 16,523.64Actual Var Amt 3.34 0 0 3.34Restored Var Amt 469.5 19.07 0 488.57Actual No. Debts 2 3 2 7Restored No. Debts 0 1 0 1Actual Debt Amt 276.48 2,834.16 50.3 3,160.94Restored Debt Amt 0 0.52 0 0.52
Carer Allowance
Disability Support Pension
Social Security Policy Branch – Random Sample Surveys
No of Reviews 0 19 13 32Fortnightly Amt 0.00 2,810.23 1,245.20 4,055.43Actual Var Amt 0 167.98 62.9 230.88Restored Var Amt 0 0 0 0Actual No. Debts 0 1 0 1Restored No. Debts 0 1 1 2Actual Debt Amt 0 485.75 0 485.75Restored Debt Amt 0 101.55 84.8 186.35No of Reviews 0 31 0 31Fortnightly Amt 0.00 11,740.69 0.00 11,740.69Actual Var Amt 0 602.43 0 602.43Restored Var Amt 0 2434.07 0 2434.07Actual No. Debts 0 10 0 10Restored No. Debts 0 1 0 1Actual Debt Amt 0 2,061.21 0 2,061.21Restored Debt Amt 0 0.05 0 0.05
Special Benefit
Family Tax Benefit
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
2002/03 2003/04 2004/05 2005/06Sample Selected 1,493 1,500 1,495 1,489No. Cust with Debts 195 183 319 365No. Debts 239 232 425 461No. Cust w Prior Reviews 491 353 785 765No. Reviews 801 671 1,859 1,880Cust with Debts and Reviews 87 74 213 229Cust with Debts and No Reviews 108 134 106 136Cust with Reviews and No Debts 404 279 572 536Cust with no Debt or Review 894 1,013 604 588No. Cust wit Overlap 52 35 90 107No. Debts Overlap 58 42 116 118
Age Pension Random Sample Cases 02/03 - 05/06Overlap Between RSS Debt and Prior Review History
Age Pension