· web viewthe evaluation of the tribal vocational rehabilitation institute (tvri) includes a...

Evaluation of the Tribal Vocational Rehabilitation Institute 2017-2018 Academic year

The evaluation of the Tribal Vocational Rehabilitation Institute (TVRI) includes a summary of interviews with TVRI graduates during the 2017-18 academic year and their supervisors. The course evaluations which include student assessments of the quality of the courses and instructional method are also summarized. Tables summarize the degree to which student evaluations indicated that they believe the course instructional objectives were met. During this academic year, the TVRI began to offer technical assistance (TA) for the first group of graduates and their Tribal Vocational Rehabilitation (TVR) programs. Six graduates requested TA services which were addressed in phone and email communications. The TVRI has set of a forum for participants to ask questions through discussion and board postings. Since this is a new service, it has not been formally evaluated at this time.

Summary of Interviews with Graduates and their SupervisorsThe course of study and benefits of the program were praised in the interviews with graduates and their supervisors. The TVR directors affirmed the need for the TVRI as it is one of the few professional development resources focused on tribal situations. The graduates described theways they applied their learning, and contributions they made to the operations of their own programs. The director interviews verified these contributions. The major recommendations for program improvements included adding more information on the administrative tasks (grant and budget development) in TVR, and linking the program to further education at the Associate’s and Bachelor’s degree level.

Graduate InterviewsInterviews were conducted with six graduates who completed their certificates in winter, spring, or summer quarters of the 2017-18 academic year. The students work in Tribal Vocational Rehabilitation (TVR) programs in the Southeast, Midwest, Southwest and Northwest areas of the country. The interviews were conducted by phone during December 2018 and January 2019. Graduates were asked about their course offerings, the quality of the instruction, and how they shared and applied their knowledge and skills in their workplace.

Reflections on CoursesThe graduates were asked to comment on courses in general and any which were especially helpful. All the students indicated that the entire course of study for the certificate were helpful. They chose to highlight specific courses for different reasons. One noted the course on Access and Eligibility (303). The course was especially useful because it helped “break the mold” used in the graduate’s program, which relied on state eligibility criteria rather than tribal vocational criteria. Another noted the first course on the history of TVR (301). The course (304) covering the Individual Employment Plan (IEP) was important for a third graduate, while a couple of

Mekinak ConsultingApril 2019 1

graduates noted Job Search and Employment (306). One explained his reason for choosing this course, “I just started the position. As soon as I got the job, I got enrolled in the class within a day or two, so I had no prior knowledge of this at all. I really enjoyed learning about job development and strategies and how we can go about making jobs for people or finding the right fit.”

One graduate cited the course on Addiction (308) and offered the following explanation:

I was explaining to everybody I work with that I don't understand addiction because I've never had a problem. So, I wanted to tap in and see exactly what our consumers are going through, how the neural transmitters are working in the brain, how I can better redirect them into a better positive course of their life, so that's the one that I really enjoyed.

Another graduate chose the Communications course (302) as it also helped in working with clients with addiction issues.

I think one of the courses that I really, really liked was the 302, the Effective Communication with Consumers. That includes the piece on Carl Rogers. We unfortunately have a lot of substance abuse in our community, so his philosophy, his way of thinking is great to apply to that group of people.

None of the graduates said that any of the courses was unhelpful. The following observation was characteristic of responses.

I don't want to say any of them were unhelpful. I mean, because I got little glimmers from each one of them. You know, like a little gemstone from each one of them to take away. So, I don't really say any of them were unhelpful. I mean, it's good to learn about history. It's good to learn about communication. For someone that doesn't have any case management, it's definitely useful to learn about case management techniques. And the IEP, that's obviously the bread and butter of VR services, so learning about that is essential too.

Graduates liked the overall organization of the course of study and the pacing. They noted the instructors were very supportive when they had academic or personal issues to address. However, almost all the graduates commented on different instructional styles. The instructors who encouraged interaction and participation were praised, compared to a couple of instructors who relied on lecturing and going over material student had already read. One person described the give and take of an instructor who encouraged participation as follows:

[The instructor would ask] ‘where are we going go with this, let's explore the information, how does it apply’. You just learned so much from just the other students … ‘this is what I faced, and this is what I've gone through, and this is how I've dealt with this,’ or things in that manner.


An older graduate valued the opportunities to share her insights based on her years of experience in TVR and other social services as well as in the Armed Forces. Another graduate was pleased that the Institute that TVR directors were teaching some courses.

I think it's beneficial that they do ask some of the directors to be some of the instructors. I think that's a real big positive. They're in the field, they know what the situation. I really like the fact that they're using the directors.

Most of the instructors encouraged and fostered participation so students could learn from each other. However, a couple of instructors relied on a lecture format and did not allow for student interaction. Another concern was repetitive information shared across some of the courses, with one student noting that, “I had no prior knowledge of tribal VR, but what I did find were a couple classes gave repetitive information, which again is kind of a good learning experience, so it kind of like sticks it into your head, but I just thought it was kind of a waste of time doing the same stuff.”

All but one of the graduates attended TVRI classes held Seattle or Spokane or at CANAR conferences. They found these opportunities very helpful and enjoyed meeting fellow students and instructors. As one graduate explained: “We presented our presentations there for everybody, and it was good to put some names and faces together and meet people. It kind of breaks the ice, because everybody's like oh yeah, we're all doing this together, hey this is what we think about that, so you know. It was good.”

Recommendations for the Course of Study or Further CoursesAlthough the graduates praised the course of study and most of the instructional styles, they did have some recommendations for the program. A couple recommended more information on financial information, including budgeting and procurement. A couple of graduates recommended including more information on administration, especially grant writing, to better understand the TVR Director role.

One graduate noted that they would benefit from learning how to conduct training.

There should be some training on how we train others. When we do placement of our consumers, the workforce place doesn't know how to work with someone that has a particular disability. We've done our best, because we have a casino and they do take our consumers. But they want training on whatever that person's disability is, to come in and kind of talk to them. I think that would be hard to do because there's so many disabilities, but there could be some that come to the top or that are the top disabilities that we see, and how to train an employment site.

Related to training employers, one graduate was concerned with the “grey area” between regulations and interpretation.


It would've been nice to have some more information on supported employment. I mean the timelines and things like that, especially for trial work experience. I know trial work experience has some rigid timelines, but if you're using it to determine eligibility, it says right in the regulation -- sufficient time. So how do you define sufficient time? I feel like there was a lot of good general information to steer you in the right direction. …But obviously the regulations are even written grey, there's a huge grey area.

Also, a few noted the need to expand the offerings to include Associate’s and Bachelor’s degrees. One graduate explained her continued interest:

I am still interested in staying with the college, to go and see if I can get something close to something beyond the TVR to enhance my counselor skills. I'm hoping that something is in the works, because it would be so awesome if there was a degree for that would cover the TVR, or parallel to what you do in your job.

All those interviewed were very pleased with the support they received from the staff of the TVRI. There were many comments, such as the following, regarding their understanding.

The administration there is very helpful with following up with any concerns that we have, as well as letting us know what the next schedule is going to be. I'm pretty sure that they're receptive when we say, ‘Well, you know what? We just need this one more class to finish the seven.’ They've worked with us on that, as well as they do those coffee break things right now, which is really good for people to share.

Sharing with Colleagues and Application of KnowledgeAll the graduates reported sharing their knowledge with their work colleagues. Sharing could be formal at staff meetings, or informal dialogue with colleagues. One graduate was able to conduct training sessions for the new staff in the office: “our director is new, and our job coach job developer, and our program assistant are new, so I was able to kind of teach them what I've learned.”

Another noted the value of sharing materials and forms with her colleagues:

And then, materials that if I thought we had a client where that would be applicable to their situation, I would say, ‘Hey, I learned about this, wonder if this would work with them?’ But anytime one of the other programs shared any of their paperwork, how they do things, I would share that with my coworkers.

One of the graduates who valued learning more about addiction shared her insights with her colleagues. She cautioned them to not short-change clients who have drug or alcohol issues and are from families with a history of addiction. She encouraged colleagues to think outside the box in looking for services and resources.

The graduates shared several ways they applied their learning. One explained that her office had


two different record systems. Using her course information, in collaboration with her coworkers, she developed a system that worked well for her office and Tribe. She explained that when they work with the other tribes in her state, they will share their form and see how it can be enhanced once they see what others are doing.

I developed an [IEP form] and had it put into our system here so that we could pull it up and fill it out and copy it off, and everybody can use the same system. I have three counselors and I had a problem with them all doing different ways. So, I kind of standardized it to do all these things[that] we do with everyone. And then the ability to add in any extra things that they do, or different things that they do for that particular consumer. So, I was able to put that into a template and then everyone can just use it.

This graduate also worked with her colleagues to develop Power Point presentations based on information learned in courses. These will be helpful when marketing TVR services. She explained that they were planning a visit to a local women’s prison to meet with Native American inmates nearing release. She hopes that the information will help them take advantage of TVR resources.

Another graduate noted the ways she is working with counselors to rethink their approach to clients suffering from addiction. Rather than offering quick employment they are now focusing more on longer term development and including the client in planning.

I'm working with a counselor now so that it is not just placing them at the casino, and then they're going to slide back into their substance abuse again, and then you get that roller coaster effect that just keeps rolling and rolling. Let's really tap into what they really want to do with their life. And so, by us implementing that, we've been having success right now, because they're like, ‘yeah, you're right.’ And I said, ‘yeah, if you want to do your GED, let's focus on your GED while you are doing your substance abuse treatment. Let's master that.’ So, I kind of promote them and engage them into their own plan.

A second graduate also noted the influence on her work with clients with substance abuse issues.

I think maybe going back to that communication with the consumers ... like I said we have a lot of substance abuse [clients]. They face a lot of barriers, even the ones that are in recovery, it is a very small community and they are judged on their past. And I feel like, by taking the materials I learned in that class, I am able to be positive and encouraging them that the prior lifestyle they had does not define who they can be. They have the chance to turn it around.

For one graduate, she is working to change the approach used by a relatively new counselor.

Well, the difficulty, of course, is again butting heads with the non-tribal member, the counselor, because she comes from a correctional realm where she doesn't understand. I


keep trying to educate her, I give her the teaching of Native Americans and tell her to go in and watch what happened, show her videos, but she doesn't get that concept -- she still has the prison mentality. I would like to implement some staff training on receptiveness and to be on-target with our grant requirements. By doing the [TVRI] training I'm thinking, ‘Oh, I can do that now.’

Benefits of Courses to Graduates and their ClientsGraduates reported that they benefited from the courses, with some getting more recognition and responsibility in their programs. More than one graduate explained having more confidence.

I know it gives me more confidence. To be able to say that I did these courses and I've learned so much and just to have a certificate helps me to feel like 'Hey, I do know what I'm doing' where as before, I was just like, this is all new. …But it helps me to know that I have this backup.

One graduate noted that she is now the number two person in her office, “I think my director kind of realized I am the number two person now. We are a very small staff, but she comes to me a lot, so I think I've gained a lot of knowledge.”

However, opportunities for promotion are very limited, and none of the graduates had been promoted. Obstacles included lack upward mobility in small tribal offices or not having a degree. Examples of comments included the following:

I would love to get a promotion, but I don't know if that will happen. But, an increase in pay would be nice, but you know I really would like to see us go further with this TVR, and like I said to just actually get a degree program would be nice.

I asked if there's promotional opportunities and [the director] says ‘no, there's not. And of course, being Native American myself, I kind of think, ‘well I've been here for you and been doing this.’ I asked in a professional manner ‘I don't understand why you hired a non-Native counselor.’ [The director responded], ‘well, you don't have a master's degree’. And I [wonder] what does a degree have to do with it when I have a bachelor's in social behavior, plus I have over 36 years’ experience in the social service field and I know our Native people, because I am one. So, I feel like there's just no room for moving up like I want to do, because I have so much stuff to offer.

The confidence gained from completing program benefits graduate’s clients. One explained this as follows:

I think when you learn more exactly what vocational rehabilitation is all about, you're able to pass that information on with confidence to your consumers, and then they in turn develop that confidence as well -- to get out and find employment and come back and also use us as a resource even later on, after they've found employment. They come back


and look into other things that we might have to help enhance their ability to retain their positions. …So, I think the classes develop confidence in us as students, that is felt by our consumers, that we know what we're talking about, and then they have confidence in us as well as themselves.

Another graduate was not interested promotion; however, she noted that the courses influenced the expectations she has for her staff. She explained that they have expanded their outreach, “we are all doing information tables when we're at every event in our community, on our reservation, even on the outside of the reservation, and we're getting to be known more now because we are out there in the community. We have a lot of information to give out.”

The courses influenced another to become a stronger cheer leader, “By being a cheerleader for them to build them up, to let them know they are on the right road. She explained that once her clients:

Go out there and show that you're a productive member, people’s ideas of who and what you are will begin to change. And, you know, for a couple of them, it was tough at first when they got a job, but once they started working, they were able to see for themselves that, 'Hey, I'm doing this, and people are giving me a chance.'

Another explained that participation led to their office meeting 100% of their grant’s goals and objective for the first time in eight years. She noted, “I attribute it to the TVR Institute and learning all the stuff that we can do and can't do. And what's allowable and what's not.”

When asked if there were areas that were especially challenging for their clients, one noted the following:

Well, I think maybe one of the hard parts is the job search and employment. Again, the partnering with potential employers, we are in an area where we are very tourist driven, we don't have a lot of manufacturing jobs, anything like that. So, it is kind of a challenge to find employers that will listen to you even, when you explain about the tax credits and the help you give to employ people.

Another challenging area is finding resources in isolated, rural areas. For one graduate, the issue was transportation and assessment. Another graduate also noted assessment concerns. Examples of their comments include:

I think one thing that would make us more effective, would be more trainings around assessment, and when is the appropriate time to implement a specific assessment based on a case need. There's the need for learning assessment.

The main thing is to get those psychological assessments. I mean, you have to have a professional and we're in such a rural area that we really need them, because we did have a gentleman that just retired … and it was wonderful because he would give us his


recommendation whether the person can work or not. So that's the barrier's that I see.

Director InterviewsThe TVR Directors for four of the graduates were interviewed. One of the graduates is serving as the Director for her DVR program, and one Director did not respond to requests for an interview. The Directors were very pleased with the TVRI. One explained:

We don't have a consistent training tool for professional learning for any of our staff. And I've worked in TVR since 1995. Some things have come and gone, but this is the best I've seen, in terms of providing a consistent program. I like the availability for staff. Having the online classes and offering scholarships to make things even more accessible. I think it has been wonderful. And I like the format of the class ... having it that accessible, I think the course outlines looked good. I was enthusiastic about the program.

Expectations for TVRIEach of the directors described their expectations for their workers enrolled in the program. Their comments included the following:

We had an opening, and we had [name of worker] and the classes started his first week on the job. The timing was awesome for us. When the counselor position opened and we hired [the worker], it fit with our grow-your-own idea, working with tribal members to become staff. So our expectations were high, because I have hired staff for positions that they don't really know, and it is really difficult to provide the depth of knowledge and the level of training that I think people need to understand and to do their job well while they are on the job.

Well, my expectation for the courses was to have a better understanding of vocational rehabilitation in general, and when it came to the substance abuse and the other topics, to have that foundation so when you're working with our consumers and they have that, whether it's a relapse, or they don't have that drive to complete their plan for employment, just to have that hope and that faith. And also, to think outside of the box. That was my main goal for [the worker] in taking these courses, is just to open up her mind, and also to help us as a department, because there are some times where we get to doing things and we become robots, and then you're trying to treat every other patient the same. So, it's always good to get refreshed.

I thought it would give her really good knowledge as it pertains to the VR program just because there is really nothing that is VR specific in any kind of degree programs that are out there. And this was a good starter. The worker that took these courses is an older lady, and traditional college just would not have been what she wanted to do. So, this was the perfect program for her to enroll in.

I really wanted her to go back to school. [In] our work here within the Department of


Education, …education is very highly looked at, and when she took these classes I was thinking ‘Let's see if she'll move on to her bachelor's degree or something to further her education.’ I think the education is very important because it helps her with her professional development, not only in her career path, but also dealing with the consumers that we deal with every day.

Each of the directors noted that their expectations were met. When asked to rate how well on a scale of one to five with one being “not at all” and five being “extremely well,” all gave a rating of five. A couple of directors explained their ratings as follows.

I give it a five. I talked with [the graduate] and I talked with [name of staff person], who is the other counselor. The [graduate] worked a lot with [name of staff person] during the process, more directly than he worked with me because she's a counselor. She's been with us four years and she is a [name of college] graduate. In fact, it worked so well, [the graduate] is now enrolled in a master's program through [name of college] in vocation rehabilitation counseling.

I believe it's a five. She would come and tell me how the classes were going, and she seemed very positive about it. When you go back to school and you haven't been for a long time it sometimes can be frustrating, but she handled it very well. I was very proud of her for that, because she worked full time and then she was taking classes on top of that. The expectations are well met.

Graduate’s Contributions to the TVR ProgramThe directors allowed their workers to work on aspects of their courses while at work. Some viewed it as time devoted to professional development. Others explained ways the course work benefited their entire programs. They described the value of having their graduate share course information formally in staff meetings and informally with their coworkers. In one case, coworkers would help the TVRI student with topics for assignments. The director explained, “we tried to think of ways that [the assignment] would be helpful, that are still within his interest area, but would be helpful to something we needed here.” Another director noted how the graduate brought new ideas based on what she was learning into their monthly case reviews. A third director noted that for every course the graduate took, she would think of ways to connect the content with the program’s clients. She explained that the graduate, “helped us when it came to different consumer’s plan for employment. Another described a brochure the graduate developed that is being used for outreach activities.

The directors described many contributions to their programs. In one situation, the program operates thirteen “clean and sober living” apartments. The TVRI graduate assumed a lead role in working with the clients in the apartments. He developed a Power Point to explain TVR services and related orientation and outreach materials. The program does on-site assessments, and the graduate evaluated them and revised their assessment regimen.


He completed all the assessments and evaluated them ... read about them, completed the ones we were looking at himself, and then talked about his experience in taking the different assessments, and made suggestions for which ones we might adopt and put together to provide a mini-assessment. [He considered] how a person might feel taking it and the approach to take and timing and how to put things together and arrange a sequence. And then he did, and he still does, many of the assessments of people. I know he would not have been able to do this without taking the classes.

Another director described the support the graduate provided when the office revised its IEP form, “she provided the information that was given to her in the classroom and provided a lot of different examples that were shared with her, to help us come up with one that would fit our needs better.” A director explained the value of the graduate’s contribution to their understanding of job search and employment issues. She noted that, a lot of different programs have a job coach and we don't have that. So, we utilize the staff that we do have to work with each of our consumers and assist their different needs.’

Two directors noted that the graduates had expanded their thinking about ways to deal with client with substance abuse issues and how consider the culturally relevant aspect of their services.

Just the morale of the office and when we're talking about the consumers who do the revolving door, there's hope rather than, ‘Okay, we're not going to serve them. They're not able to benefit from our program.’ I think these courses have assisted not only [the graduate], but the whole department in thinking outside the box, having that hope for people, and being more creative when it comes to writing the IEPs.

Pretty much like I was talking about the cultural relevant services and how different ways people are doing things, because I think we were so concerned about how we were going to handle this. It was good to hear about other people's ways of dealing with those little bit of struggles in their departments. So that was really good to hear those things and sharing those ideas and what we could possibly do. Maybe it could work for us or help us develop into a stronger team.

Directors noted that they had not collected pre/post information on closure rates, and at least one program did not track closure rates by worker. However, one director believed the graduate’s work ethic, which had always been good, had improved: “she is going, going, going, going, and she wants to make sure she has everything that needs to be in the report to get her people in for successful closures.” Another believed her graduate was able to handle more complex cases, “it's helped her because now she's thinking through other ideas, maybe what other people have shared with her when she was in coursework. It has helps her quite a bit.”

Although graduates may have expanded their roles such as helping with grant development,


many of the TVR programs are small and there is little opportunity for promotion. A couple of directors were interested in upgrading salaries for their graduates; however, this requires action from the tribal government.

As of right now, with how our program is set up, we don't have different spots to move up. That's something that we're working on within our own tribal government. I'm pushing for [the graduate] to be in a different salary bracket, but, again, that goes to tribal government. I have to answer to the other six council members, too, and then also our Human Resources department so we can get this changed. But it's something I'm working on because it is deserved. [The graduate] does assist with the management of our financials at the office. She also has her own cases. She assists the counselors. She also does some of the courses that we have at the department.

None of the graduates had any CAP referrals or grievances so there was no impact on these program statistics. A few noted ways the clients had benefited from the graduate’s training. One noted that, “she shows a lot more understanding than she's ever shown prior to taking these courses. And, it has allowed her to be able to manage her case load better.” Another director whose graduate is from the community, and was known for other roles he played, has won the confidence of his clients as he can demonstrate his understanding of TVR.

Final CommentsThe directors were pleased with the online structure of the courses and felt their workers were able to manage the technology. One commented on the synchronistic element. She thought it was important to keep this format since her graduate really learned a lot from discussions with his fellow classmates. However, the director of the program in the Eastern time zone noted that some of the courses offered in late afternoon in the Pacific time zone would be after hours for her worker. Since the internet was spotty in their rural area, the graduate would have stay in the office to get her work done.

There were no recommendations on the course of study. However, one director would like students to learn more about the director’s role – especially what goes into developing the TVR grant.

We would all like to serve everybody but sometimes we can't and [understanding the grant development] might be a really an eye-opener for them. I don't know if lot of times people feel intimidated by that, maybe some directors might not like their staff knowing what they do, but I think it's important. It helps build the relationship with the staff.

The on-site opportunities at CANAR conferences was noted by another director, “[the graduate] got to meet people face-to-face, as well as some of the instructors. I thought that was very positive. This category of people that went through the classes together could be an ongoing support for each other in their own professional development.” Another director noted the value


of the TVRI’s marketing strategy.

The [TVRI staff] was pretty adamant in sending those emails and having that phone conversation with both myself and the [graduate]. And I think that helps because we do get busy with the day to day activities that we have to do, and then also our personal lives. So, I think by the [TVRI staff] being that much more informative, and then also trying to get more students from the VR programs enrolled in these courses really helped.

Directors were quite supportive of the TVRI and expressed the importance of maintaining the training.

I think it's a big step to have this and it must be sustained. It's hard to attract qualified workers to most of our projects. Reservations are in remote areas. Also, there is pressure to hire tribal members, which I support, but there aren't tribal members with backgrounds in vocational rehabilitation, so we really need to have the knowledge and a way to get that knowledge in a meaningful way to the new person, so they can be successful because they really want to do a good job. They want to know that they're doing things the right way and a good way. So, it has my vote.

Well, I think it's a wonderful program. It gives more in-depth knowledge on VR in the tribal world. I think it is wonderful for people, such as [the graduate], to get some kind of education and knowledge under their belt that they may not be able to receive in any other way. So, I absolutely think that they are to be commended for providing such a program to the employees of the VR world.

Summary of 2017-2018 Course SurveysDuring the fall of 2017 and the winter, spring and summer quarters of 2018, the Tribal Vocational Rehabilitation Institute (TVRI) taught the full seven courses required for a program certificate. The survey sought students views on satisfaction with and expectations for the course, the value of course to their work, ways in which they shared their learning at work, the clarity of various course elements, challenges and ways to improve the course. This report summarizes the finding of the final survey that was administered after each of the courses for the academic year 2017-2018. The first section includes tables that summarize the questions involving a scaled set of choices. The second section summarizes how well the courses met their specific learning objectives. The final section summarizes open-ended questions from the survey. Table 1 describes the number of students who completed each course and the response rate to the course evaluation survey. In general, half of the students in the course completed an evaluation, except for 305 in which only a quarter of the students took the survey.


Table 1. Number of Students Completing Courses and Survey Response Rate

ITVR Course Quarter # Completed Course

Survey Response Rate

301-History of TVR

Fall 8

53% Winter 11Spring 9Summer 8

302- CommunicationsWinter 14

59%Summer 8

303- Access/Eligibility Fall 23 40%Spring 24

304- IEP Development Fall 8 54%Winter 16

305- Case Record and ManagementFall 12

28%Winter 13Spring 15

306- Job Search and Employment Winter 2 47%Summer 15

307- Resource ManagementWinter 4

48%Spring 8Summer 11

Overall, the students rated the courses highly. Most were general statements; however, in the summary, the responses that described specific examples in response to the questions were listed.

Survey TablesThe following tables combine the quarter surveys for each of the courses. Each table has the number of students who answered the question and the percentage within each response choice.

Table 2 Satisfaction with the Course

Course #Extremely dissatisfie

d

Moderately dissatisfied

Slightly dissatisfie

d

Neither satisfied/

dissatisfied

Slightly satisfied

Moderately satisfied

Extremely satisfied

301 19 5% 5% 16% 74%302 13 15% 85%303 19 5% 5% 47% 43%304 13 8% 23% 69%305 11 9% 9% 82%306 8 12% 12% 12% 12% 52%307 11 18% 82%

The majority of the students were extremely satisfied with five out of the seven courses. For the


other courses, the majority were extremely to moderately satisfied. For course 306, Job Search and Employment, three out of the eight students who completed this item were slightly to extremely dissatisfied (Table 2).

A large majority believe that the courses improved their effectiveness as a TVR employee, although one student in 303 was critical of the course (Table 3).

Table 3 Improved Effectiveness as a TVR Employee

Course # Not well at all Not well Moderately

well Very well Extremely well

301 16 6% 56% 38%302 13 8% 69% 23%303 16 6% 6% 56% 32%304 12 8% 42% 50%305 3 67% 33%306 8 100%307 11 44% 56%

Students were asked how each of the courses met their expectations in terms of organization, materials, and the course instructor. Again, the ratings are high as noted in tables 4, 5, and 6. Two students were critical of the instructor, which may explain satisfaction with the course.

Table 4 Met Expectations for Course Organization

Course # Far short of expectations

Short of expectations

Equals expectations

Exceeds expectations

Far exceeds expectations

301 17 6% 18% 47% 29%302 13 % 38% 39% 23%303 17 12% 12% 41% 35%304 11 9% 36% 36% 19%305 10 20 80%306 8 43% 14% 43%307 11 33% 67%

Table 5 Met Expectations for Course Materials



Equals expectations



301 17 24% 53% 23%301 8 25% 50% 25%302 13 15% 62% 23%303 17 6% 23% 53% 18%304 15 9% 36 36% 19%




Equals expectations



305 10 10% 70% 20%306 8 % 43% 14% 43%307 11 22% 78%

Table 6 Met Expectations for Course Instructor



Equals expectations



301 17 6% 53% 41%302 12 15% 23% 62%303 17 12% 12% 41% 35%304 11 9% 18% 55% 18%305 10 10% 50% 40%306 8 14 14% 29% 43%307 4 22% 78%

Students were asked to rate the degree to which the documents shared in the course related to their work. In general, they responded that it related to their work always to most of the time.

Table 7 Content of Document Related to My Work

Course #Never Sometime

About half of the time

Most of the time Always

301 17 6% 6% 29% 59%302 13 23% 77%303 17 6% 23% 71%304 11 18% 9% 73%305 10 10% 90%306 8 14% 14% 14% 58%307 11 11% 89%

Similar ratings were given to the degree the documents related to the students’ educational interests, but a few students noted that this happened only sometimes in the 301 and 302 courses.


Table 8 Documents Related to My Educational Interests




301 17 6 12% 29% 53%302 13 31% 69%303 9 6% 47% 47%304 11 9 18% 73%305 10 20 80%306 9 14% 29% 57%307 11 33% 67%

Students found the documents easy to understand.Table 9 Ease of Understanding Documents




301 17 23% 77%302 13 15% 85%303 17 23% 77%304 11 18% 82%305 10 100%306 9 14% 14% 72%307 11 11% 89%

Students rated the clarity of aspects of the course on a five-point scale ranging from extremely clear, somewhat clear, neither clear or unclear, somewhat unclear, and extremely unclear. The courses were rated highly on all aspects, with only a few students neutral or finding them somewhat to extremely unclear. The following four tables summarize the ratings for clarity of course requirements, course materials, GoToTraining sessions, and discussion sessions.

Table 10 Course Requirements 301 302 303 304 305 306 307

Extremely to Somewhat Clear 100% 92% 94% 100% 100% 100% 100%Neither clear or unclear 8%Extremely to Somewhat Unclear 6%

Table 11 Course Materials 301 302 303 304 305 306 307

Extremely to Somewhat Clear 100% 100 94% 100% 100% 100% 100%Neither clear or unclear

Extremely to Somewhat Unclear 6% 8%


Table 12 GoToTraining Sessions

301 302 303 304 305 306 307

Extremely to Somewhat Clear 100% 100% 94% 91% 100% 100% 100%Neither clear or unclear 9%

Extremely to Somewhat Unclear 6%

Table 13 Discussion Sessions 301 302 303 304 305 306 307

Extremely to Somewhat Clear 94% 100% 94% 91% 100% 100% 100%Neither clear or unclear 6% 9%

Extremely to Somewhat Unclear 6%

Students could choose from a list of suggestions on ways to improve the courses. The most popular suggestions were more course case studies, more visual materials, and opportunities to get to know classmates (Table 14).

Table 14 Suggestions for Improving Courses

Improvement Suggestions 301 302 303 304 305 306 307 Totals

More training on how to use Canvas before the start of the course.

6 2 1 2 3 14

More training on how to use GoToTraining before the start of the course.

4 1 5

More visual material like videos or pictures. 5 3 5 5 2 1 3 24

Simplified materials presented during class time (e.g., a PowerPoint presentation).

5 1 2 3 4 1 1 17

More case studies or examples. 8 7 5 2 3 2 2 29Getting to know my classmates better. 4 2 5 3 3 2 1 20

Better organization of course materials. 1 2 3 3 5 14

The students could choose challenges they faced from a pre-determined list. As noted in Table 15, making sure they were on track with assignments was the most challenging. Communicating during Go-To-Training, posting on the Canvas discussion board, and using Canvas email were the next most challenging.


Table 15 Challenges

Challenges 301 302 303 304 305 306 307 TotalsMaking sure I was on track with all my assignments. 7 9 7 6 3 5 4 41

Communicating during GoToTraining sessions (chat box, phone, mic).

4 2 4 4 4 1 19

Accessing course materials. 1 1 2Understanding the instructor's lectures. 1 1 1 3

Understanding course materials. 1 1 2Posting on the discussion board in Canvas. 4 2 3 2 2 2 15

Using the Canvas email. 6 1 2 2 2 1 14Knowing how to get technical support. 3 1 4 1 9

Learning ObjectivesEach course had a set of specific learning objectives. A retrospective measure was used to assess the students sense of their own learning for each objective. They were asked to rate their level of knowledge of the objective before the course and then to rate their knowledge at the end of the course. The non-parametric Wilcoxon test was used to test the changes in the rankings before and after the course. The changes in knowledge were viewed as statistically significant if the Wilcoxon test was p≤.05 or less for a two tailed test. However, with the assumption of movement towards more knowledge, or a one-tailed test, many more of the objectives noted below would be statistically significant. Course Tables in which all of the objectives were met are located in the Appendix.

The objectives for the introductory course 301 were changed for the spring and summer courses. Prior to the changes in objectives, none of the Wilcoxon statistics met the p≤.05 test for significance (Table 16). However, students in the spring and summer courses found that their knowledge change was statistically significant (Table 21 in Appendix).

Table 16 Change in Knowledge for Course 301 Fall and Winter Quarters

Learning Objectives for 301 TVR Foundations History (Fall

& Winter)

Before Course After CourseWilcoxon 2-

tailed significanceMean

Std. Deviation Mean

Std. Deviation

Points of history for VR and TVR 2.0 1.23 3.8 .447 .059The relationship between the State VR agencies and the TVR program 2.8 1.30 4.0 .707 .059

Concepts required in a written agreement between State and Tribal VR

2.2 1.11 3.40 .548 .063


Learning Objectives for 301 TVR Foundations History (Fall

& Winter)



Std. Deviation Mean

Std. Deviation

Differences between a "qualified VR counselor" and an essential TVR counselor

1.8 1.30 4.0 .707 .059

Sole authorities of the TVR counselor 2.2 1.30 3.6 .548 .066Characteristics used to determine whether a VR applicant/client meets the definition of having a significant disability

2.8 .837 3.8 .447 .059

Appellate rights of VR consumers 2.4 1.14 4.0 .000 .066 Roles of consumers in the VR process 3.0 1.00 4.2 .447 .063Issues VR needs to address in preparing a client for employment 3.0 1.00 4.2 .447 .063

However, the changes in objectives for the 302 made for the summer quarter had an opposite impact. Students noted significant gains in their knowledge for the set of objectives for the winter course (Table 22 in the Appendix) but did not make statistically changes for seven for eleven of the objectives in the summer quarter (Table 17).

Table 17 Change in Knowledge for Course 302Summer Quarter

Learning Objectives for 302 TVR Foundations Communications (Summer)


tailed significanceMean Std. D Mean Std. D

Defining and describing the application and intake form and process 3.50 .577 4.50 .577 .046

Describing the key purposes and considerations for an intake interview 3.25 .500 4.25 .500 .046

Describing or characterizing counseling and its use within the TVR process 3.25 .500 4.25 .500 .046

Identifying the points in the TVR process when participants should be notified of their rights and responsibilities including the Client Assistance Program

3.00 .816 4.25 .500 .059

Defining the purpose and value of informed choice in the TVR process 3.25 .957 4.50 .577 .059

Describing the effectiveness and practice of confidentiality policies in a TVR agency 3.50 .577 3.75 .577 .046

Describing the options an applicant or participant has for pursuing a grievance or in selecting the administrative review process.

2.25 .957 3.75 .500 .059

Describing the key roles of the Tribal Governing Body as related to the TVR agency Key roles the Governing Body (e.g., Tribal Council) has related to the VR grant.

2.25 .957 3.75 .500 .059

Sharing the purpose of and at least one strategy for tribal community outreach 2.50 .577 4.25 .500 .059

Describing common ethical dilemmas in a TVR or tribal cultural setting and a strategy for working through ethical dilemmas

2.25 .577 4.00 .815 .066

Describing the process of adjusting to limitations related to a disability 2.25 .500 4.50 .577 .059


The 303 course had significant findings for all the objectives (Table 23 in the Appendix).

Two objectives failed to meet the test for statistical significance for 304 (Table 18).

Table 18 Change in Knowledge for Course 304

Learning Objectives for 304 TVR Foundations IPE Development



Std. Deviatio

n Mean

Std. Deviatio

nIdentifying the required elements of the IPE. 3.25 .866 4.14 .835 .039Identifying the definitions of appropriate VR employment outcomes. 3.25 1.05 4.00 .835 .041

Describing how to determine the need and purpose for up-front assistive services. 2.75 1.139 4.00 .739 .025

Describing the types of services that a VRC could not provide with agency funds before fully considering the availability of comparable services and benefits.

3.08 .900 4.25 1.138 .048

Identifying the six (6) "exempt” services that do not require full consideration for comparable services and benefits. 2.58 .996 4.33 .985 .010

Describing the procedure for providing traditional healing or other culturally-appropriate services. 3.00 .853 3.92 .900 .031

Describing why "terms and conditions" are required elements of the IPE. 3.25 1.06 4.33 1.155 .064

Describing the types of entities that require a list of responsibilities related to the IPE. 3.17 1.27 3.83 1.115 .098

Describing how to measure a client’s progress in the IPE towards employment. 3.00 1.21 4.00 1.128 .039

Describing the conditions and characteristics of post-employment services. 3.00 1.21 4.00 1.206 .046

The 305 course had significant findings for all the objectives (Table 24 in the Appendix).

Course 306 failed to get statistically significant changes for three of the nine objectives (Table 19).

Table 19 Change in Knowledge for Course 306 Job Search and Employment

Learning Objectives for 306 TVR Foundations Job Search & Emplymt



Std. Deviation Mean

Std. Deviation

Describing how rehabilitation, culture, and employment are linked in TVR. 3.29 .756 4.00 .816 .025

Describing the relationship between eligibility determination and employment options. 3.29 1.113 3.71 1.113 .180

Describing the purposes of a trial work plan for an individual with a significant disability.

3.00 1.000 3.86 .900 .034

Describing the expectations of an employer when using real work settings. 3.14 1.215 3.86 .900 .025


Learning Objectives for 306 TVR Foundations Job Search & Emplymt



Std. Deviation Mean

Std. Deviation

Describing the situation when it is necessary to discuss reasonable accommodations and assistive technology with an employer.

3.29 .951 4.14 .690 .034

Identifying when it is necessary to consider supporting a client interested in self-employment. 2.57 1.134 3.86 .900 .014

Describing the pros and cons of using CRPs for client vocational assessment. 3.57 .976 3.86 .900 .414

Describing when it is time to assist the client in preparing a job resume. 3.86 .690 4.43 .535 .102

Summarizing the requirements for providing post-employment services. 3.00 .816 3.86 .900 .014

For 307 Resource Management, two of the nine objectives failed the test for statistical significance (Table 20).

Table 20 Change in Knowledge for Course 307 Resource Management

Learning Objectives for 307 TVR Foundations Resource Mngmt

Before Course After CourseWilcoxon 2-tailed

sigMeanStd.

Deviation MeanStd.

DeviationDescribing methods used to establish a positive understanding of TVR in Tribal service communities. 3.11 .928 4.33 .816 .024

Identifying (in a funded TVR proposal) the described resources that require a management system. 2.33 .577 4.33 .816 .109

Describing the management system used by your TVR agency that tracks and reports grant expenditures, draw downs, match and indirect costs.

2.89 1.167 4.56 .816 .031

Describing a TVR agency management system essential to provide and maintain case records. 3.00 1.118 4.44 .572 .071

Identifying the contents of a TVR cuff account (refer in part to ED 524A&B). 2.33 1.118 4.33 .500 .010

Describing the gathering of TVR annual performance data required for RSA reporting. 2.67 1.118 4.44 .726 .041

Describing key elements in building and maintaining an effective team in a TVR agency. 2.67 1.118 4.56 .527 .027

Defining effective ways of building and maintaining positive organizational awareness in the TVR program and the greater community.

2.89 1.000 4.44 .726 .041

Identifying methods used to develop and maintain an effective relationship with the agency’s Governing Body. 2.33 1.000 4.44 .527 .017

AppendixThe Appendix has the tables in which all the course objectives were met. Also, student comments for open-ended questions for each of the courses are listed.


Courses in which All Objectives Had Positive Statistical Significance ChangesTable 21 Change in Knowledge for Course 301Spring and Summer Quarters

Learning Objectives for 301 TVR Foundations History (Spring & Summer)

Before Course After Course Wilcoxon 2-tailed

significanceMeanStd.

Deviation

Mean

Std. Deviatio

nPoints of history for VR and TVR 1.38 .518 3.75 .707 .011 The relationship between the State VR agencies and the TVR program 1.88 .835 4.00 .535 .011

Concepts required in a written agreement between State and Tribal VR 1.50 .756 3.75 .886 .011

Describing the sole authorities of TVR agencies 1.63 .744 3.50 .756 .010

Listing the rights and responsibilities of participants in the TVR process 2.75 1.035 4.38 .518 .016

Describing the importance of continuing education and career pathways in TVR 2.63 1.408 4.25 .707 .016

Describing barriers unrelated to impairments that TVR needs to address 2.75 1.282 4.13 .641 .026

Describing appellate rights of TVR participants 2.25 .886 3.88 .641 .017

Describing the importance of holistic TVR services 2.50 1.069 4.25 .707 .026

Describing how TVR agencies conduct outreach to potential applicants

2.25 .886 4.25 .707 .011

Describing prevalent disabilities in tribal communities2.50 .756 4.13 .641 .010

Table 22 Change in Knowledge for Course 302 Winter Quarter

Learning Objectives for 302 TVR Foundations Communications (Winter)



Defining why an application form for VR should be a comprehensive document. 3.44 .882 4.33 .500 .011

Key purposes for an intake interview as well as the kind of information that should not be included. 3.33 .707 4.33 .500 .014

Characteristics of motivational interviewing and its use within the VR process. 2.78 .667 4.00 .707 .009

Points in the VR process when a client should be notified of their rights and responsibilities. 3.67 .866 4.44 .527 .020

Purpose and value of "informed choice" in the VR process. 3.56 .726 4.56 .527 .014

Effectiveness and practice of confidentiality policies in a TVR agency. 3.78 .441 4.56 .527 .011

Options an applicant or participant has in selecting the administrative review process. 3.44 .726 4.33 .707 .011

Key roles the Governing Body (e.g., Tribal Council) has related to the VR grant. 3.22 .833 4.44 .726 .004


Learning Objectives for 302 TVR Foundations Communications (Winter)



Essential partnerships your TVR agency has within your community. 3.67 .707 4.44 .527 .005


Learning Objectives for 303 TVR Foundations Access/Eligibility



Std. Deviation

Mean

Std. Deviation

Identifying several sources for gathering information essential for determining eligibility.

3.12 .332 3.94 .429 .000

Describing methods to use to gather additional detail needed for eligibility. 3.18 .529 3.88 .485 .001

Describing how to verify where the person lives for the residency requirement. 3.47 .717 4.00 .500 .003

Describing three (3) types of disability impairments and the licensed medical/ psychological/ treatment provider needed to assess the reported impairments.

3.18 .728 3.94 .659 .001

Describing Describe the methods to use to determine if the impairments would provide substantial impediments to employment.

3.12 .697 3.88 .697 .000

Identifying the factors and issues to consider when determining if a client can benefit from VR services in terms of achieving an employment outcome

3.12 .600 3.82 .529 .001

Describing why a person who met all other eligibility criteria would not meet the criteria of requiring VR services. 2.88 .600 3.82 .636 .001

Listing the options a client has for getting help in drafting their IPE. 3.24 .752 3.88 .697 .001


Learning Objectives for 305 TVR Foundations Case Record & Mngmt*



Std. Deviation Mean

Std. Deviation

Listing the required contents of a case record. 3.10 .994 4.30 .675 .010Describing the follow-up on a case that is determined “incapable of benefiting."

2.60 1.075 4.00 1.054.010

Describing the follow-up on a case that closed due to severity of disability.

2.50 1.179 3.90 .994.004

Listing the criteria required to close a case where an employment outcome is successfully achieved.

3.30 1.160 4.20 .789.014

Describing why it is important to have a single case record format for a TVR agency.

3.00 1.054 4.30 .823.010

Describing your agency's strategy for ensuring that case records meet agency and legal standards, including confidentiality.

3.20 1.033 4.40 .699.016


Learning Objectives for 305 TVR Foundations Case Record & Mngmt*



Std. Deviation Mean

Std. Deviation

Describing the agency's methods for gathering and reporting the case record data to RSA.

3.30 1.160 4.40 ..699.009

Describing your agency's methods for ensuring client services are deliverable and follow the design of the case record when the assigned VRC is absent.

3.20 1.033 4.30 .823

.016

Describing how a counselor would prioritize caseload duties such as progress notes, paperwork and client contacts.

3.30 1.160 4.40 .843.009

Describing the legal requirements for retaining closed case records.

3.10 1.197 4.30 .823.010

Summary of Open-ended QuestionsThere were four open-ended questions on the survey and an opportunity for students to add “other” information that was not listed on the questions listing ways to improve the course or challenges they faced during the course. Many of the responses were general statements. Those that were more substantive or gave examples are shared below. Many students described being unaware or lacking knowledge. Some have goals to move beyond their current positions in to management and to get a four-year degree. If no comments are listed after the course title, none of the students chose to write a comment.

If you work for a Tribal VR program, how well has the information in this course influenced studies so far.

301 History of TVR The course has helped me to have better understanding of TVR. Has me thinking about

things to take into consideration when dealing with our consumers and has improved my knowledge on what I already knew. This course has influenced me to continue with my education.

Hearing others' practices for their TVR Programs. Helped to understand more about the qualifications. Also the background helps to

understand why we are doing what we are. Helped me get familiar with CRF codes and where to navigate and access them. I learned a lot from the instructor as well as from other students and gained new insight

on how to better serve my clients.

302 Communications

It made me aware of different variables that contribute to the way of communicating with the consumer with disabilities. There are a lot of differences in the way to counsel and


guide the individuals that come into the Blackfeet VR office. The resources that were given to communicate with will be very much used in my office setting.

It influenced me to try to understand each individual that applies for VR services as they are and to see where they are coming from --to be in their shoes to sort of speak. To have positive unconditional regard for each individual I see.

The TVR institute classes have helped me to gain knowledge about my work, it has helped me put information to use in my work, and it has helped to boost my confidence in the work that I already do. When I started in my job I jumped in with both feet. There is a lot of information thrown at you on many different levels. History of TVR, WIOA changes, program requirements, as well as tribal preferences for program nuances. These classes have helped me to connect the dots, integrating all aspects.

This course of Communication has taught me a great respect for our consumers and their families. I've learned to be “Silent” and “Listen” better, to look outside the box and give our consumer the understanding and compassion they need.

This course has helped me better understand the process for communicating to consumer about how the TVR program works. It has also helped me learn new ways to network with community resources.

303 Access and Eligibility

In the implementation of assessments and accessing information to make sound decisions in regard to participants.

The information in this course and learning from others in the discussion boards has given me new ideas on how to approach the TVR process.

I really enjoy Donna's teaching style. She makes it interesting and encourages the students to learn from each other instead of her just being a talking head.

Learned more about different types of assessments.

Makes me more conscientious when I go through a file, from beginning to the end. It has given me a glimpse into the importance of verifying information and making sure the eligibility has a solid foundation because it allows us to see through to the impediments.

I have a better understanding of the process. I am more confident in making decisions on my own. I am more familiar with documentation and ADA law.

Your courses have enabled me to have a better understanding of the CFR's and how to search for what I need. I also have a better grasp of the whole intake process and how to improve and stream line this service.


Among the many things I have learned more about historical trauma and the effects it can have on people whether it is realized or not. Most importantly I have learned that other TVR programs also experience the same situations with consumers that we do.

I have utilizing the CAPS/COPES assessments for years and complied information through the intake interview and their assessment that has been released to us after their evaluations to deem them eligible. This course helped me better understand and identify other methods that I will begin to implement during that eligibility stage. I also gained a better understanding of how to manage ineligible applicants and the process that it takes. I may not have deemed anyone ineligible yet, but this knowledge will be available for me in the future.

It has helped me realize there is so much more to learn about TRV and we can learn from each other and learn about resources to do our job well.

There was one critical comment

I learned some new things, but I felt like we were so rushed and there were technical difficulties and times we didn't even get to finish a module. that held us back from learning things that we needed to. Sometimes as counselors we don't have time to study after class and that time in class is so valuable to us to try to learn as much as possible. Some have children that they have to tend to and that can deter them from continuing to study after class as well. This course was the main one that I needed and was so vital to me as a counselor because I seem to have issues with this part of the whole VR process. I would hope that in this specific class that things could be improved.

304 IPE Development

Showed me the correct way to fill out an IPE It’s been great to have a review of the requirements as I did need a refresher. It also

helps to discuss the various types of services. I am more aware of the intentions behind the program and how it was intended to work. This course has given me the knowledge of comparable benefits, a very comprehensive

list of VR services, and a long list of do's and don'ts of client services. I feel that our program will be managed successfully, if the staff here fully implement the suggested changes.

I somewhat understood what needed to be in the IPE. I now understand that our consumers should have employment within 60 to 90 days of applying for services and on the program now more than a year with successful employment.

305 Case Record and Management

It has provided me with helpful information on how to improve the management of my case files and to write case notes in a way in which the information is easy to understand should others have the need to review my notes to assist the consumer.


Making me more aware of the use of my time as commodity for the program and the people we serve.

I think the information I learned in this class will help me to be a better counselor. I really enjoy learning not just from the instructor, but the open dialogue that is encouraged in class.

I have learned that managing a case load involves all areas of collaboration. Whether from providers, other agencies, the consumer, your own VR program personnel etc., working together toward that employment goal for the consumer to become successful. Also, that TVR counselors should be diligent in providing timely and accurate documentation from referral to closure of case.

It has made me realize the importance of a clean file so that if someone else has to take the file and see the case through, there is a smooth transition and an understandable format

306 Job Search and Employment

This course has opened my eyes to the importance of working directly with employers and the benefits that could have with our clients.

A new appreciation for job development and all that is required to be a successful job developer. As a VR counselor and taking the course 306, our program should have a full-time job developer.

Understanding the new terms used in TVR programs need to know. I feel we are making process of making our TVR programs strong in our field of work. I feel that we are ready to make a difference in our presentations to our Council people. I feel that we need to have a stronger stand in what we can accomplish in providing services to our disability people who want to work.

307 Resource Management

This course has helped me learn more about my position as a counselor trainee. I found the policies and procedures where one of the most useful things that I learned about this semester.

The course has helped me better understand the process of grant writing and the responsibility of TVR programs use of grant monies.

Influenced me to be more aware of our TVR grant goals and objectives

I realize the importance of the policies and the RSA's oversight.

Please describe how this course was relevant to your work or your educational goals:

301 History of TVR


Refresher on a lot of things that I have learned over the past nine years. I was able to be inspired and apply the new knowledge to my position and improve our process and orientation forms.

I am a VR counselor who is new to the field and this course made a difference on a daily basis.

This course helped me with understanding the laws and history of TVR. It also gave me a good refresher on my role as a TVR counselor and how to be an even better counselor. It helped me with ideas for working with clients as well as how to do different forms of outreach in my community. I plan on utilizing the resources I've gained in this course to help me in staying up to date with the laws. I also plan to utilize the knowledge I've gained to better serve my clients and to do more outreach in my community.

I’m currently a caseworker and applying for a management position. Every aspect will be applied to my job.

I am working as a program aide for a TVR program. In the future, if re-funded, my position will be written out and a new counselor position will open. I am hoping to apply for that position.

This coursework is extremely important to me because it is one of the only ways I am learning how to do my job. I do get help from coworkers when I need it but I hate asking all the time. I feel as if I am a burden. So, this classes have helped me in that way.

302 Communications

Addictions (alcohol and/or drugs) as a disability; guidance and counselling. This course enhanced my knowledge of communication with the consumer with disabilities. It gave me resources that I can reference when I need to. This course was awesome, especially the part of historical trauma and intergenerational trauma, these are part of my personal recovery. Since I experienced this in my own personal journey, I try to remember all Tribes/Native Americans have some history that has been carried from generation to the next generation that has to be worked through.

Grievances, communication on Informed Choice and working with difficult people. Based on the topics covered in the session, it was very educational for me as a new counselor working to help individuals with disabilities.

All aspects of this course are relevant to my TVR position. Communication is a must with serving people. You have to be able to talk about the program and what we can do and who we serve, but we must be able to listen to our participants to understand what they need and how best to assist.


Tribal Government and Tribal Administration? Although we have a collaboration with all tribal entities, we (or I) do not really know the what the depth of the relationship is with these two and/or what are the roles? I learned a lot from this course and the base of its meaning. Communication is an essential part of the relationship between consumer and counselor, as well as the overall program. How we first respond is an "ever lasting impression we leave."

This course was very relevant to my position as counselor assistant because I am the one responsible for most of the correspondence between the consumer and the program. I also help with the intake, so I am usually a consumers’ first impression. Learning better ways of communicating how our program works and describing consumer rights and responsibilities has been very helpful.

I am a new director of a TVR program so this is very relevant.


I am a Counselor Assistant and one of my main duties is helping with the eligibility determination. The information in this course has helped me gain a better understanding of the requirements of eligibility and what they mean.

Neutral. Difficult to really learn online.

It was very relevant to my work, it has changed my perspective of relationship with the client and allowing them to be fully aware of the criteria for eligibility, and also it gives me a better understanding of the particular barriers that could/should be addressed.

I like the way it brought out historical trauma and how it affected different tribes in different parts of the nation.

I am a case manager for TVR. I am currently working to complete my AAOIT (almost there). This TVR Institute program has made me aware of the opportunities at NWIC were I hope to complete my Bachelors program.

Very relevant, the eligibility stage of the process is important to determining their status within our program. The ineligible process can be helpful on how I handle those potential applicants.

The importance of assessment and the different skills and tests available to use for consumers. It helps in placing the consumer in a job most likely they will excel at and enjoy.

304 IPE Development


This course helped me to know more about the IPE development and what was required as I did not have a concrete idea of what was involved in the IPE process. I came in and flew solo, my knowledge is growing with each course.

I have been using this course as a reference manual for my IPE elements.

I didn't know how to do an IPE correctly. What is required, etc. An area that is a must to know to meet requirement to be I in compliance to Voc Rehab programs, dos and don'ts. I love having classmates’ input and critiques.

These courses have allowed me to gain more knowledge in VR even though I've been in VR for quite some time. I do not have the background that most individuals do have in VR but these classes are helping me to gain that background that I need. I am trying to continue my education, and this will help me a lot.

I was or currently a TVR consumer and started out as an OJT. My heart is to help people like me who's been in this position who has struggled with some sort of barrier. I still do to this day, but I would like to encourage my people and help them overcome the barriers that are workable and let them know there is a way out to life.


The course is extremely relevant to my work as a VRC and as I am working towards becoming a CRC it will help me with that goal.

It has helped me to prioritize my cases and to teach me the importance of case management and case notes. This has been very relevant.

All of the info in this class and previous courses were relevant specifically to TVR. That helped me to gain a better understanding in TVR related issues and made me more aware of what some of the TVR counselors in other parts of the country have encountered and how they have dealt with pop up issues.

My confidence level has grown stronger after each class. I trust my decision making more and more. With this course I feel I have learned the purpose of good case management by controlling my distractions and trying to keep my cases up to date and understandable with order and goal oriented.



I learned the importance of a trial work plan, how to determine the expectations of an employer, and summarizing the requirements for providing post-employment services. These three topics are bound to come up in my career sometime but not yet. When they do, I will have an idea of what to do.

It gave me a better understanding of how DRS works with all the different divisions.

As a TVR employee, the course is relevant in my role as Work Adjustment Skills Trainer. The knowledge that I have gained will certainly be used in the service of those individuals with disabilities.


This coursework is extremely important to me because it is one of the only ways I am learning how to do my job. I do get help from coworkers when I need it, but I hate asking all the time. I feel as if I am a burden. So, these classes have helped me in that way

This information was very relevant to my work as I am a VR counselor.

My goal is to be a Director of a TVR Program someday. This course taught me many things I have heard of but did not know exactly what it was.

The course has given me a better understanding of all the information that is required to submit for a grant and once a grant is received the needed reports from TVR programs to RSA that show why we initially applied for grant and how the goals have been accomplished.

This course assisted me in my overall view of tribal VR as to where the Blackfeet VR is within the AIVRS program. It taught me why I need to keep better data, which assists in reporting requirement, which makes us eligible for more funding. I learned about the grant process, policy and procedure and the laws and regulations of Tribal VR.

How do you plan on applying to what you've learned in this course to your job?

301 History of TVR

Refer back to what I've learned to make sure I follow the rules, regulations, etc. to help my consumers the best way I know how.

One of the things I liked/use now in our program is the Next Step Form given by the instructor. Also sharing information with one another was helpful.

Every little bit helps. After this course I have edited our forms in orientation/intake to better clarify each form on our policy, procedure, and process.

302 Communications


I will use this in the guidance and counselling process, so I will better understand where the consumer is coming from and where the consumer wants to go. I will be able to better serve the consumer with disabilities for a successful employment outcome.

I have been using this course as a reference for my IPE.

I plan to apply all that I can to do a better job in my role as the first person they see when they come in seeking VR services. I do the Intake and get needed documents and send file on to VR counselor.

I am working on updating forms for our program to simplify for use. I am always looking for better ways to interview and do intake with participants. To better help them share information without being intrusive or intimidating.

Be Silent and Listen. Have compassion and empathy for your consumer at the point of walking through the door, among the important but there is much more I've learned and I'm very grateful for these course and opportunity.

I plan to use the information I learned in this class to help me communicate better with consumers. I think the person-centered theory was very helpful.

Policy and procedure review

A lot of what is talked about in class is relevant to applying it to my job. I like sharing information with other staff members in my program.

To provide more culturally sensitive services


Assisting the consumer in identifying the adjustments they may be going through with a new disability and making good assessments

I am a case manager for TVR. All information in this course will help me to streamline the eligibility and assessment part of participant services.

I will apply the knowledge I have learned to better serve and utilize the information for my participants. Especially trial work.

I can better apply my knowledge to the process, and the printed materials are very helpful for me to train and prepare new employees.

I would share this information with coworkers that are NOT counselors but work with consumers on career goals and applications.

I will definitely look into other methods of assessments.

304 IPE Development This would be my reference book


Updating IPE and understanding more about approving services

To properly using information to put into correct form. I wasn't sure what was required to put into an IPE. I was only guessing and putting in what I thought needed to be put in.

Every bit of the information I was taught, I will use it. I believe this will help so many of our people if the ones in those positions truly help and not be petty or personal. It is very important that all staff of a TVR program be open minded and want to help ALL people not just family, not just friends but to all who are eligible.

I have structured an IPE plan but with the understanding that counselors can make adjustments necessary to fit every consumer.

I have developed an IPE form that encompasses what is required and when I get it approved with my program director it will be the document to use for the Blackfeet VR.


I plan on doing a better job when it comes to putting a file together.

I apply what I have learned throughout my training every single day I am at work. Case management, service delivery, case notes are things that I have improved by taking this course and other courses I have taken.

Being more aware of the time in the use of my time and coworkers time and making sure that they are used in the most effective and efficient way possible.

I like how the information builds from one class to the next. In the beginning we just had a foundation, now we have walls and it's starting to look like a recognizable structure.

I intend to, empathize more with consumers while encouraging them to believe in themselves, keep better timely accurate narratives, keep the case moving forward, reach out and work with other agencies better, and provide documentation that is accurate for any case reporting needed.


Print out some of the information and put it my clients’ orientation folders, go back to information when needed for clients.

I plan on using the lessons learned in this course to improve our relations with employers.

Spend more time in job development by reaching out to employers and assisting consumer with more hands on when job searching.



I plan on helping out more when it comes to the budgeting aspect. I would like to have a more important role when it comes to that. But then again, I am in no rush.

I plan on continuing to use all this information on a daily basis. I have become knowledgeable on issues I wasn't aware of such as the budget, cuff accounts and draw downs.

Paying attention to detail and asking to review our budget together.

I plan on looking at future changes within the Blackfeet VR, doing more outreach, looking at data more thoroughly, reviewing our grant goals and objectives, and updating policy and procedures.

Making sure the team has a better communication from me. Helping my coworkers feel like I am part of the team by doing my part to document effectively.

Please describe who you share info from the course with and how it is shared (ex. with co-workers at a staff meeting):

301 History of TVR

Well, it is mostly asking questions from my supervisor, like ‘how are we doing this or that, how do we carry this out,’ things like that. I also communicate with VR counselor about the same things I ask my supervisor.

Co-workers just through normal conversation, nothing formal.

With my subordinate staff in future staff meetings to implement these new ideas. I also share it with my administration to let them be aware of how the program operates. It's very important to share this with consumers and update the way we do our process

I am able to share the information I learned with my co-workers at staff meetings and staffing over cases being worked on by fellow VRCs.

302 Communications

I share with other coworkers during our monthly meetings any and all information that sparked my interest.

We have a staff meeting every Friday and sometimes a situation comes up and I can reference the training and share.

I have shared it with all of my coworkers. With my Director - I have developed a form that she reviews that assists her with the determination of eligibility which includes all pertinent information as well as my recommendation on next steps. With my Job-Developer - I have gotten him to be more cognizant of the importance of documentation and how it is key to good case management. His skills have really improved. With my


Rehab Tech- Program Assistant - I have taught her how to gather the pertinent documentation needed and she has learned how to complete the eligibility forms once our Director has determined eligibility. So overall, I have seen a lot of improvement with our program.


We talk about what we learn in class during staffing.

I share information from the course with my staff team. We meet to discuss the lessons and homework.

My supervisor has sat in on a few of the classes which allows her to see the type of information shared throughout the course. I also use information to help streamline intake process which will be shared with the overall program.

With coworkers when available and needed. With other tribal VR programs, not much but some. Mainly to inform them of the training. I also share the information with consumers and people looking for tribal services.

I will be utilizing some of the course material to be my guide when I am in the 90-Day period when I hire a new employee. Also, it will always be very helpful to review in staff meetings.

We usually share all information from webinars, classes, and outreach events in our staff meetings or in our reports that we share with our co-workers after returning from a conference or an outreach activity.

I share course information with both of my coworkers, our director and other counselor. It is all a learning process and always room for program improvement.

304 IPE Development

I shared with co workers and other individuals like veteran representatives.

Co-worker, giving ideas and saying what's required in a IPE. Also I have classmates and instructors that I can ask questions or ideas to. That's really a plus for me.

I am able to share with the director and with coworkers.

I share with the Director and the Blackfeet Manpower One Stop Center program administrators (BVR Advisory Board) in a meeting weekly.


305 Case Record and Management With co-workers, new hires, students who intern with us.

Co-workers - weekly telephone case staffing and monthly face to face case staffing/team meeting.

When sharing course information and what I have learned is done at staffing, talking with other VR personnel, at council meeting and generally at every opportunity I get public or private. I try and always encourage other OK Tribal VR programs to take some of the courses.

Co-workers/VR staff and since I work with transition high school studenst I'd share some information with I.E.P. team members.

I share the information with our office staff, director and another counselor. they are looking forward to taking some of the classes too.


There are several co-workers in my department who are taking the course. We collaborate with homework.

Coworkers, during daily conversations and staff meetings. Consumers during the initial interview or when we refer them to DRS.

Our program is in the process of working with the [name of the program] based at Northern Arizona University so I am able to share what I have learned with the team which is helping to better our program overall.


I share different things during a normal day at work. I usually will tell my coworker/lead counselor. I like to tell her what I learned during the week and see if I am on the right track. It is also convenient because she did an internship with the state VR program. So i like comparing TVR to the state VR.

I am able to share information about this course with my co-workers. I have been encouraging everyone in our program to take these classes.

Hopefully to review grant and budget as a team

Counselors within our TVR program and other TVR tribes in the state. Also sharing information about courses to other departments with in tribal services that are not VR.

I am an Administrator of the BVR which is under the umbrella of the Blackfeet Manpower Programs One Stop Center over 16 programs. Each program has an


Administrator and we meet at least once a month with the Director and Deputy Director (more if needed). I give a BVR update as so do the rest of the Administrators, this is where I share my information.

I have shared information with the director, office staff and counselors. It is done in person, face-to-face. It has been shared at staff meeting and one-on-one meetings.

What would improve your experience with the course?

301 History of TVR

Starting the class on time and ending the class at the time it was to be finished.

302 Communications

Did not have time for all material during class- some had to be on own.

Sharing questions & answers and Ideas with class mates. (which we did very well in this course)


304 IPE Development

More class discussions.

Would like to converse about the daily subject with all.

I want to take online class and go at my own pace, check in during course and finish on due date. I feel this is a Hybrid not an online class like other colleges I’ve attended.

Able to contact classmates

I felt that we kind of sped through it all and there wasn't enough time to ask questions sometimes.

305 Case Record and Management 1. System froze, lost some of my writing. 2. Improve communication when there is no

class. 3. Give other alternatives on how to submit coursework when having trouble uploading our work.



Which of the following did you find challenging? Please check all that apply: - Other (please explain): - Text

I really feel the course has all the instruction and documents required. I find myself, because I work, I tend to put assignments off due to my service to VTR participants


301 History of TVR

302 Communications

Most was due to prolonged absence. issues with attaching doc in canvas

Getting material in a timely manner to review for class


Being disconnected from learning experience. Being in chat rooms/discussion boards isn't real. Not being able to get online.

Office firewall prevents some site access

All was great. Unfortunately, I didn't spend the time I thought was needed on course. So much great information to take in.

I fell behind due to my short staff and large case load. That initially was not an issue for the course, it was my performance wasn't up to par. Thankfully I was able to catch up and have that ability to catch up that made all the difference.

Confused sometimes due to being on one module one day and then going straight to another one the next class time without completing the prior one.

304 IPE Development

I work in four locations, so I find it hard to keep up with the designated times. I have login from my phone, laptop, desktop, library, etc... but I don’t have internet at home so trying to follow the class on a time schedule gives me less time for my j

How to contact classmates, their information available in case we would like to contact them. Get to know them better.


It’s tough to work full time and to take classes and then to have to respond to discussion boards



Definitely time management, between work and class it was challenging. I enjoyed this class, the time frame was a little difficult for me because of access to the internet. I had to travel back to work after hours to use the wifi here because i don't have it at my house in the country.
