Evaluating Minnesota’s Subsurface Sewage Treatment (SSTS) Professional Experience Program

Nicholas Haig

Introduction

Presentation Outline

•Minnesota’s SSTS Program▫Advisory Committee ▫Research Context

•Evaluation Plan▫Focus Groups▫Survey Development▫Survey Implementation▫Data Analysis▫Recommendations

•Discussion and Future Research

• http://septic.umn.edu/committees/mentoringworkgroup

SSTS Certification

•Training and Examination

•Experience Requirement

•Continuing Education

Designated Certified Individual

•Individual is authorized by the MPCA to independently complete SSTS work

•Authorized to supervise other non-certified skilled workers

•May become a certified mentor

MPCA Endorsement Categories

•Designer •Installer•Inspector •Maintainer •Service Provider •Advanced Designer•Advanced Inspector

Research Focus

SSTS Advisory Committee

•Mentoring Workgroup▫Bring a higher level of education and

competence to the industry’s professionals. Honest assessment of current training

effectiveness Identify and overcome the barriers to the

current mentoring program Identify potential improvements

Guiding Research Questions• What are the best practices of mentoring that contribute

to the development of high quality practitioners? • What variables affect the success of the experience

program?▫High vs. a low level of professional pride and

motivation to succeed▫Previous experience vs. no previous experience▫Completion of state required pre-certification training

prior to or after completion of required experience▫Regular interaction with local inspector(s)▫Mentoring quality

• To what extent has the experience program improved practitioners’ abilities to perform both essential and best operating practices?

Focus Groups

• Detroit Lakes▫8 participants

• Two Harbors▫6 participants

• Willmar▫9 participants

1

2

3

Focus Group ConclusionsPositive Attributes1. Value of experience,

learning from each other

2. Multiple endorsements or mentors provided for a broader experience

3. Experience should be cumulative as one attains additional endorsement categories

Challenges1. Lack of experience was

considered the most common cause of mistakes

2. Quality variations in local programs and inspector endorsement issues

3. Availability of mentoring opportunities

4. Good and bad mentors are not differentiated in current program

Good Mentoring vs. Bad Mentoring

•Present

•Builds confidence

•Teaches practical skills

•Not Present

•Does not correct poor behaviors

•Teaches incorrect practices

Survey Development

• KAP Questionnaire: Design and Review▫Knowledge ▫Attitudes▫Practices▫Certification▫Relationship with mentor

• Mailing List▫Data Privacy

• Planning for success▫N = 3276▫ .95 Confidence Level = 550 Responses▫1500 Randomly selected names

Survey Implementation

•Dillman’s “Total Design Method”- Enhanced▫N=3276▫1100 surveys sent▫57% response rate▫543 valid surveys▫.94 Confidence

Surveys Received

0

20

40

60

80

100

120

140

160

12/19/2009 - 3/27/2009

SSTS Mentoring Surveys Received

Knowledge Questions

•Which septic system best protects public health and the environment?▫A septic tank to an in ground system with

36” of vertical separation - 85.2% ▫3 other options

•In what way does soil texture MOST affect the design of a septic system?▫The size of the soil treatment area - 78.4%▫The type of system - 18.7%

Practice Questions

•How often do you require installation contractors to redo their work?▫Never or rarely - 60.1%▫Sometimes - 39.1%

•Has an inspector ever required you to improve or redo your work?▫Yes - 29.5%▫No - 69.7%

Mentor relationship

Additional Measures:Present? 2.04Build Confidence? 1.68Help to avoid mistakes? 1.71

How did you obtain your mentor?

• 30.7%• 21.8%• 30.4%• 1.1%• 14.3%

Response Rate

At Work

LGU

On Own

Pro Org

Other

Access to Mentors (attitude)

Should someone make sure that new SSTS Pros have ac-cess to opportuni-ties to complete

their required ex-perience?

YesNoI don't know

• 91% felt it was important or very important that new SSTS professionals should have guaranteed access to mentors

Percent of respondents that answered “very important” or “important” to:

Q10 Q11 Q120%

10%

20%

30%

40%

50%

60%

70%

80%

90%

New SSTS InspectorsAll SSTS Professionals

Q10: Hold multiple endorsement categories to be considered an expert

Q11: Install SSTS prior to designing SSTS

Q12: Design and install SSTS prior to inspecting SSTS

What did we find?

• In general:▫ SSTS Professionals place a high value on “on-the-job”

training, though they also value classroom training

▫ It is difficult to differentiate “required experience” from everyday learning

▫ The survey tool further confirmed universality of opinions identified in focus groups

▫ The KAP methodology is a viable evaluation tool when investigating natural resources policy effectiveness

What did we find?

•Specifically:▫ The current experience requirement was not

implemented equitably across the industry

▫ Clear guidance for mentors and apprentices does not exist

▫ Changes made for latest MN Rules 7083 were positive beginning steps in addressing concerns of the industry

▫ Changes made to one area of concern have potential to counteract or worsen another area of concern

Recommendations . . .

1. Develop written guidance for both mentors and apprentices

2. Increase aspiring inspectors exposure to the SSTS industry

Written Guidance for Experience Program

• Written guidance for both mentors and apprentices ▫ Define purpose and context of experience program

▫ Identify what is expected of an apprentice Define value of gaining experience Identify steps in acquiring a Mentor

▫ How and why to provide sound mentorship Checklist of critical activities where presence is

required Limiting liability as a business that provides

mentorship Other incentives

▫ Define “observation” Encourage observation and mentoring with a

variety of SSTS practitioners

Level the playing field . . .

• Proposed change to 7083.1050 subp 5 Item C:

An applicant for certification as a basic inspector must have co-completed with a mentor a minimum of 15 inspections of Type I, II, or III systems, as defined under parts 7080.2200 and 7080.2300 with a flow of 2,500 gallons per day or less, with a minimum of one aboveground system inspection, and a minimum of one belowground system inspection. An applicant must observe five soil evaluations, system designs and management plans being developed. An applicant must also observe five system installations, and five service or operational instances, with mentorship not required. No additional experience is required to qualify for the advanced inspector certification.

Discussion and future research

•Emphasize mentoring relationships beyond context of rule requirements

•Conduct quality control for contractors and local inspectors▫88.9% of respondents agree or strongly

agree that differences in local SSTS programs influence the practices of SSTS professionals

Tough and thor-ough local pro-grams result in

high quality SSTS and decreased pub-

lic health risksAgree or Strongly AgreeDisagree or Strongly Disagree

Believe there should be a uniform

SSTS program across Minnesota

Agree or Strongly AgreeDisagree or Strongly Disagree

Further Research

•Break into cohorts and search for correlations▫Tenure in industry▫Geographical differences

•Develop metrics for knowledge, attitude, and practice “scores” for correlation purposes▫Those that incorrectly answered knowledge

questions▫Were those that answered certain

questions differently more likely to follow practice “X”. . .

▫Links between stated competence and mentor behaviors