ipt 530-edubose evaluation methodology team project

Merchandise Unlimited UPS Truck Crew Process Evaluation

Final Report

Evaluation Team: Eboni DuBose, Perri Kennedy, and Chester Stevenson IPT 530 Evaluation Methodology

Boise State University Spring 2011

1

Table of Contents

Preliminaries ................................................................................................................................... 2

I. Executive Summary .................................................................................................................. 2

II. Preface ..................................................................................................................................... 4

III. Methodology .......................................................................................................................... 5

Foundations .................................................................................................................................... 6

1. Background and Context ......................................................................................................... 6

2. Descriptions and Definitions ................................................................................................... 7

3. Stakeholders .......................................................................................................................... 12

4. Resources .............................................................................................................................. 12

5. Dimensions of Merit and Importance Weighting ................................................................. 14

Sub-Evaluations ............................................................................................................................. 15

6. Process Evaluations ............................................................................................................... 15

7. Exportability .......................................................................................................................... 17

Conclusions ................................................................................................................................... 18

8. Overall Significance ............................................................................................................... 18

9. Recommendations ................................................................................................................ 21

Reporting ................................................................................................................................... 23

References .................................................................................................................................... 24

Appendix A: MU UPS Receiving Process Assessment Rubrics with Criteria Highlighting ............. 25

Appendix B: Summary of Ratings with Rationale ......................................................................... 28

Appendix C: Data Sources for Instrument Dimension .................................................................. 29

Appendix D: Data Sources for Workspace Dimension .................................................................. 31

Appendix E: Data Sources for Process Management Dimension ................................................. 34

Appendix F: Sub-dimension Weighting of Importance by Stakeholders ...................................... 38

Appendix G: Scoring Rubrics for MU UPS Receiving Process Evaluation ..................................... 39

2

Preliminaries

I. Executive Summary

Introduction Merchandise Unlimited (MU, pseudonym), a merchandise distribution company based in the Midwestern region of the United States, had not examined the processes and resources in place to receive inbound shipments for at least a decade. The Director of Operations and the Operations Manager requested a formative evaluation to both establish how well the inbound receiving process (hereinafter referred to as “evaluand.”)should function and discover how well it actually does function. In the spring of 2011, three graduate students (hereinafter referred to as “evaluation team”) from Boise State University’s Instructional & Performance Technology program were asked to conduct an evaluation of the inbound receiving process. The stakeholders asked for the evaluation team to look specifically at the 10-person truck crew (hereinafter referred to as “UPS Truck Crew”) which offloads and sorts inbound shipments of used and returned merchandise which were delivered via the United Parcel Service (UPS). The evaluation team created assessment rubrics based on industry best practices and the needs of MU, and then examined the practices and resources in place.

Methodology The evaluation team conducted a formative goal-based evaluation structured on Scriven’s (2007) Key Evaluation Checklist. It was designed to examine the current state of the UPS receiving process and how it could be improved. After discussions with the stakeholders, the evaluation team focused on three dimensions:

Instruments

Workspace

Process Management

Each dimension was weighted based on the stakeholders’ consensus determination of its importance (Important, Very Important, or Extremely Important). The evaluation team created an assessment rubric based on warehousing best practices, and then compared the current processes and resources in use by the UPS Truck Crew to the assessment rubric. Conclusions Using a 5-point scale (Poor, Mediocre, Acceptable, Very Good, and Excellent), the overall quality of the UPS receiving process is rated “Acceptable” for current organizational needs. However, the evaluation team concluded that the current set of processes and resources may not be sufficient to handle future needs. A major concern is that Process Management, weighted as Extremely Important by the stakeholders, was rated as “Mediocre” when

3

compared to industry best practices. The overall evaluand rating and dimensions with ratings and weighting are shown in Table 1.

Table 1. MU UPS Truck Crew Receiving Process Evaluation Results

Strengths Based on the evidence gathered, the evaluation team noted the following strengths of the UPS receiving process:

Equipment in use is well maintained and sufficient to maintain current operation levels.

Space is utilized as well as possible given its limitations.

The supervisor and members of the UPS Truck Crew are capable, experienced, and work together as a cohesiveness unit.

Opportunities for Improvement Based on the evidence gathered, the evaluation team made the following recommendations to the stakeholders:

Develop and document a standardized operating procedure to ensure that tacit knowledge is not lost and that new hires receive consistent on-the-job training.

Develop performance metrics to drive continual process improvements.

Develop a standardized package labeling system for use by retail and individual customers.

Examine all options for increasing floor space.

Evaluate existing hardware for possible replacement or upgrades to meet future needs.

Look into a possible redundancy in the initiation process, and consider the options for having initiation performed at the next stage of the receiving process.

Rating

Overall Evaluand

Dimension Weighting

Instruments

Important

Workspace

Very

Important

Process Management

Extremely Important

Poor Mediocre Acceptable Very Good Excellent

4

II. Preface Background Merchandise Unlimited, a merchandise distribution company based in the Midwestern region of the United States, had substantially upgraded and modernized much of its warehouse operations in recent years. However, the processes and resources in place to receive inbound shipments had not been updated or even examined for at least a decade. In January 2011, the Director of Operations and the Operations Manager of Merchandise Unlimited requested an evaluation of the processes in use by the receiving department, which handles all inbound shipments delivered to the company’s warehouse. Although other warehouse processes had been assessed and improved, the receiving functions had not been evaluated for many years. The stakeholders asked for the evaluation to look specifically at the receiving process conducted by the UPS Truck Crew, a 10-person workgroup which handles overstock and used merchandise shipped back, via UPS, to MU in individual non-palletized packages. The volume of shipments handled by this crew ranges from 1,500 to 3,500 packages each working day. The Director of Operations and the Operations Manager stated that they planned to adopt any recommendations which, in their judgment, will be both feasible and cost-effective to implement within the next two years. Evaluation Focus This evaluation examined the merit of the UPS receiving process. The Director of Operations and the Operations Manager wanted to assess the intrinsic quality of the UPS receiving process and see how it compared to industry best practices. The purpose of the evaluation was to develop the definition of merit for the receiving process, as the company had never established any benchmarks or ideals of performance. Evaluation Type This project was developed as a formative evaluation, as the stakeholders wanted to understand the current level of performance and what changes to the process or environment could improve that performance. Formative evaluation is defined as one “typically conducted during the development or improvement of a program… and it is conducted, often more than once, for the in-house staff of the program with the intent to improve” (Scriven, 1991, pp. 168-169). Audience This evaluation report was created for the Director of Operations, the Operations Manager, the Inbound Manager, the UPS Truck Crew Supervisor, and the Vice President of Human Resources.

5

III. Methodology

Evaluation Approach When the evaluation project was proposed, the stakeholders had a general goal of improving efficiency in the UPS receiving process; they defined efficiency as the speed, accuracy, and cost-effectiveness of the overall process. However, no performance targets or ideals for the receiving department had ever been defined nor was any system in place to measure speed, accuracy or cost-effectiveness. Without any existing metrics against which to compare current performance, it was impossible to determine efficiency based on the stakeholders’ definition. After discussing this with the stakeholders, the evaluation team agreed to instead develop an assessment rubric based on industry best practices and then evaluate how the UPS receiving process compared to those best practices. The evaluation framework was based on Scriven’s (2007) Key Evaluation Checklist (KEC), which outlined a systemic examination of both processes and outcomes. Due to the lack of existing metrics, this evaluation was conducted strictly as a process evaluation which would examine the procedures and resources currently in use by the UPS Truck Crew. The methodology used was a goal-based dimensional evaluation of the evaluand’s absolute merit, in that the evaluation team identified dimensions of merit for the evaluand and then examined how the procedures and resources utilized in each dimension compared to the best practices defined in the assessment rubric. To develop the assessment rubric, the evaluation team researched best practices for warehouse receiving departments to determine ideal processes and identify key issues to examine. The evaluation team also documented the current procedures and environment in place to look for strengths and weaknesses in actual practice. The evaluation team developed its assessment rubric based on industry best practices compiled by Supply Chain Visions and sponsored by the Warehousing Education and Research Council (WERC). The evaluation team then hosted a focus group with the Operations Manager, Inbound Manager, and the UPS Truck Crew Supervisor to discuss the key elements of warehousing best practices. Some proposed dimensions were eliminated as not relevant or not of sufficient importance. The remaining dimensions were then shaped into a set of assessment rubrics which the evaluation team used to identify the areas of greatest opportunity for improvement. The process assessment rubrics are included as Appendix A. Data Collection Methods The evaluation team used a combination of data collection methods including interviews, extant data, and observation. Specific methods used for each dimension of merit are identified in Appendix C (Instruments), Appendix D (Workspace), and Appendix E (Process Management).

6

Foundations

1. Background and Context

Merchandise Unlimited (MU) is a merchandise distribution company based in the Midwestern

region of the United States. MU has been in business in different incarnations for about a

century, and has existed in its current form for nearly four decades. Its primary objective is to

collect and distribute new and used products and ship them to its customers at the lowest cost

possible. MU’s single warehouse is in operation 24 hours a day, five days a week to receive,

sort, and distribute more than 20 million pieces of merchandise each year to both individual

consumers and the company’s 2,500+ retail clients. For many years, the primary concern of

warehouse management has been how fast products are shipped to the company’s customers,

so almost all efforts at efficiency and modernization have been focused on this area. The

current sorting and distribution areas of the warehouse make extensive use of modern

technology, including robotics and 3-D scanning, to ensure that products are shipped accurately

and rapidly.

Management has recently turned its focus on how fast products are brought into the building, a

process that has not been re-evaluated in over a decade. Out of all the products received in the

warehouse, approximately 75% are delivered by a single carrier, UPS. The operations

department would like to evaluate the existing UPS receiving process to improve inefficiencies

and reduce costs.

The stakeholders asked the evaluation team to conduct the first evaluation by focusing on the

10-person workgroup (the UPS Truck Crew) which handles offloading, sorting, and initiating

individual packages of overstock and used merchandise shipped to the warehouse via UPS; this

crew does not handle packages sent through other carriers, nor does it handle palletized loads.

This merchandise includes both retail store overstock and returns from individual customers.

The packages are classified by the UPS Truck Crew as large or small.

Large packages: Shipments from retail customers

Small packages: Shipments from individual customers, or internal mail

Constraints

Seasonal Fluctuations

Due to the nature of its business, MU experiences predictable and significant seasonal

fluctuations in both shipping and receiving merchandise through its warehouse. During normal

7

times, the UPS Truck Crew may offload and sort 1,000-1,500 packages daily, most of which are

picked up by the receiving departments that same day. During seasonal peak times, the crew

may process as many as 3,500 packages daily; since the receiving departments cannot handle

this inbound volume immediately, the UPS Truck Crew must store the packages for later pickup.

Warehouse Space

The UPS Truck Crew’s assigned space was sufficient a decade ago, when the average daily

volume was approximately 800 packages. However, the average daily volume of packages has

nearly doubled since that time. No additional floor space is available for the crew’s use. The

storage area’s shelves are permanent fixtures which cannot be reconfigured or removed.

2. Descriptions and Definitions

Evaluand Description

The receiving department is broken into four sections:

Truck crews, including the UPS Truck Crew, which unload the inbound shipments

Receiving, which receives large packages from the truck crews

Direct Receiving, which receives small packages from the truck crews

Internal Mail, which receives small packages from the truck crews

The UPS receiving process begins when the UPS truck driver unloads packages from his or her

trailer onto a mechanical roller, which rolls the packages into the building. As each package

comes down the roller, a member of the UPS Truck Crew removes it from the roller and

examines its label. Large packages which are part of a shipment of three or more boxes are

sorted directly onto pallets and then initiated. One- or two-box shipments of large packages are

put on the express belts, initiated, and then sorted onto pallets. All small packages are put on

the express belts, initiated, and then sorted into bins. Large package pallets are picked up by

Receiving, and small package bins are picked up by Direct Receiving or Internal Mail.

During the initiation process, a crew member enters a package’s shipping information into the

warehouse inventory software using a laptop computer. Most package labels include UPS

barcodes, which are scanned using a USB device that feeds a limited amount of package

information into the warehouse inventory software; the crew member must initiate the rest of

the information into the system. During times of normal inbound volume, the pallets and bins

are picked up promptly by the destination department.

8

When inbound volume is very high, the UPS Truck Crew moves the pallets into the storage area,

where they may remain for up to two weeks awaiting pickup. A visual representation of this

process is shown in Figure 1.

Figure 1. MU UPS Truck Crew Receiving Process.

Pallets and bins picked up by Receiving, Direct Receiving, and Internal Mail employees

Packages initiated into inventory system using laptop computer and handheld scanner

Packages removed from mechanical roller by UPS Truck Crew

Packages sorted by type and placed on wooden pallets (Receiving)

or onto express belts (Direct Receiving and Internal Mail)

Packages unloaded by UPS employee from trailer onto mechanical roller

Peak receiving volume:

Pallets and bins stored on shelving for

later pickup

Normal receiving volume:

Pallets (Receiving) and bins (Direct

Receiving and Internal Mail) prepared for

immediate pickup

9

Process Logic Model

The MU UPS Truck Crew Process Logic Model (see Figure 2) describes the factors and crew

activities involved in the UPS receiving process, what activities the evaluation team performed

in examining the process, and the intended results of the evaluation and subsequent changes

to the process.

Figure 2. MU UPS Truck Crew Receiving Process Logic Model

10

Definitions Table 2 defines terminology used in this report, including evaluation concepts and terms used by MU for elements involved in the receiving departments. Absolute Merit The quality or value of something in absolute

terms measured against a defined standard. Assessment Rubric A tool which provides an evaluative

description of what performance looks like

at each of two or more defined levels.

Dimension of Merit

Criteria of merit. These are aspects of an evaluand that define whether it is good or bad and whether it is valuable or not.

Dimensional Evaluation An analytical evaluation in which the quality of the evaluand is determined by looking at its performance on multiple dimensions (criteria) of merit.

Direct Receiving The division of the receiving department which opens small packages and sorts their contents for processing.

Downstream Impactees Impactees that are not directly affected by the evaluand.

Evaluand That which is being evaluated (e.g., program, policy, project, product, service, organization).

Express Belt A 10-ft long mechanical roller with a computer and scanner at the end, used to initiate individual packages.

Formative Evaluation An evaluation conducted for the purpose of improving a currently operational evaluand for the purpose of making improvements.

Goal-based Evaluation A type of evaluation based on the goals of the evaluand.

Inbound Shipments coming into the warehouse from outside vendors or customers.

Initiating MU terminology for the process of entering package data into the warehouse inventory software.

Internal Mail Packages received which are not return merchandise. These packages are given to the company’s internal mail facility, which delivers them to the appropriate recipients.

11

Table 2. Definitions of Evaluation and MU Terminology

Mechanical Roller A non-electrical device consisting of a series of rollers which is used to transport boxes from one area to another.

Merit Intrinsic value, quality or significance of the evaluand.

Overstock Excess merchandise in a store’s inventory, which may be returned to the vendor.

Package Refers to a box, carton, or other shipping container received. They are typically referred to at MU as small or large depending on their actual size.

Palletize To load packages onto wooden pallets for

transportation or storage.

Primary Consumers Actual or potential users of the evaluand.

Program Logic Model A diagram that illustrates the cause-and-effect relationships between resources and activities, and the outcomes and impacts of the program.

Truck Crew A crew of MU employees which sorts merchandise as it is unloaded from a truck into the warehouse.

UPS Receiving Process Refers to the operations involved in unloading, sorting, and initiating inbound packages.

UPS Truck Crew The 10-person truck crew which processes shipments of overstocks and used merchandise delivered by UPS to the warehouse. The UPS Truck Crew is the primary consumer of this evaluand.

Upstream Stakeholders Stakeholders who are involved in the development, implementation, or management of the evaluand.

Weight The determination of which evaluation criteria are the most important.

12

3. Stakeholders

This evaluation was concerned with three sets of stakeholders: upstream stakeholders, primary

consumers, and downstream impactees. The stakeholders are listed in Table 3.

Upstream Stakeholders Primary Consumers Downstream Impactees

Director of Operations

Operations Manager

Inbound Manager

UPS Truck Crew Supervisor

UPS Truck Crew (n = 10)

Immediate Impact

Direct Receiving staff

Receiving staff

Internal Mail staff

Secondary Impact

Product stockers

Product shippers

Distant Impact

External customers

Clients of external customers

Table 3. MU Stakeholders for the UPS Truck Crew Evaluation.

4. Resources

Financial Resources The UPS Truck Crew is funded by the organization as part of its Operations division. They receive whatever funding necessary to perform their required tasks. The upstream stakeholders stated that they are willing to spend money on changes which they deem likely to increase efficiency and/or decrease operating costs. The stakeholders will not reduce costs through staff reductions; if increased efficiency allows for a smaller UPS Truck Crew, extra crew members will be moved to other parts of Operations rather than be dismissed. Physical Resources See Figure 3 for a simplified illustration of the physical layout. The storage area assigned to the UPS Truck Crew is capable of holding between 65,000 and 75,000 packages on its shelving units. The storage shelves are permanent fixtures which cannot be removed or reconfigured.

13

Figure 3. Layout of MU UPS Truck Crew Receiving Space Knowledge Resources The supervisor of the UPS Truck Crew has held this position for 15 years, and five of the ten crew members have worked there for five or more years. Turnover is modest and not considered by management to be an issue; morale within the crew appeared to be excellent. New hires receive on-the-job training in sorting and initiating packages. Technological Resources The UPS Truck Crew inputs data from each package label into a laptop computer running proprietary inventory management software. The software has a command-line interface. It takes an experienced crew member approximately one minute to type in the data for each individual package. The laptops in use are old and slow. The shipping labels on some packages include barcodes, which are scanned using devices which connect to the laptops via a USB cable. Data incorporated into the barcode is automatically entered into the inventory management software, but additional data must still be entered manually.

14

5. Dimensions of Merit and Importance Weighting

The evaluation team met several times with the Operations Manager, Inbound Manager, and

UPS Truck Crew Supervisor to refine the focus of the evaluation. When the final dimensions of

merit were selected, the evaluation team conducted a group interview with these three

upstream stakeholders so they could weigh the importance of each dimension using a 3-level

scale (extremely important, very important and important).

The evaluation team then divided each dimension into sub-dimensions, around which it created

an assessment rubric. The dimensions, weighting, and sub-dimensions are shown in Table 4.

Dimension of Merit Dimension Weighting Sub-dimension

Instruments Important

Hardware (computerized)

Hardware (non-electronic)

Software

Workspace Very Important

Pallet Layout

Utilization of Floor Space

Space Available

Process Management Extremely Important

Dock Management

Package Labeling

Documented Standard Operating Procedures

Metrics

Table 4. MU UPS Truck Crew Evaluation Dimensions, Importance Weighting, and Sub-dimensions

All the dimensions evaluated were process dimensions. The stakeholders had initially requested

an evaluation of outcomes such as speed and accuracy, but it was determined that since no

performance standards had ever been developed or measurement methods implemented,

there was no criteria against which to rate the UPS Truck Crew’s current level of performance.

The stakeholders’ schedules did not allow for a second group interview, so they submitted their

sub-dimension weighting as individuals. The different professional priorities and viewpoints of

the stakeholders were reflected in how differently they weighted some of the sub-dimensions.

As the evaluation team was unable to obtain a consensus on the importance of the sub-

dimensions, only the weighting for the dimensions was used to calculate the final ratings. The

sub-dimension weighting is included as Appendix F for reference.

15

Sub-Evaluations

6. Process Evaluations

The entire evaluation was conducted as a process evaluation, and examined three dimensions

of merit:

Instruments

Workspace

Process Management

Instruments

Dimension Evaluation Question: How do the instruments in use by the UPS Truck Crew rate

when compared to industry best practices?

Data Sources: Interviews, observation, extant data

Overall Rating: Acceptable

The instruments dimension examined the tools in use by the UPS Truck Crew in the course of its

operations, and was divided into three sub-dimensions:

1. Hardware (Computerized) – laptop computers, handheld barcode scanners

2. Hardware (Non-Electronic) – forklifts, pallet jacks, conveyors, bins, and other equipment

3. Software – proprietary warehouse inventory system which runs on the laptops and

accepts data from the barcode scanner

An assessment rubric was created to compare the current state of the three sub-dimensions to

industry best practices. See Appendix A for the Instruments Assessment Rubric; the highlighted

text indicates the criteria which best describe the current state of Instruments used by the UPS

Truck Crew.

Workspace

Dimension Evaluation Question: How does the physical workspace in use by the UPS Truck

Crew rate when compared to industry best practices?


Overall Rating: Acceptable

The workplace dimension examined the physical workspace in use by the UPS Truck Crew in the

course of its operations, and was divided into three sub-dimensions:

16

1. Pallet Layout – how pallets are placed on the floor space for efficient sorting

2. Utilization of Floor Space

3. Space Available

An assessment rubric was created to compare the current state of the three sub-dimensions to

industry best practices. See Appendix A for the Workspace Assessment Rubric; the highlighted

text indicates the criteria which best describe the current state of the Workspace as used by the

UPS Truck Crew. Although Workspace is considered “Acceptable” under current conditions,

there is no room for expansion under the current configuration should the volume of inbound

packages continue to increase.

Process Management

Dimension Evaluation Question: How does the process management in use by the UPS Truck

Crew rate when compared to industry best practices?


Overall Rating: Mediocre

The process management dimension examined the management of operating processes in use

by the UPS Truck Crew in the course of its operations, and was divided into four sub-

dimensions:

1. Dock Management – scheduling of deliveries and commitment to unload times

2. Package Labeling – external labeling on packages to indicate shipment type, package

contents, and quantity of individual packages in shipment

3. Documentation of Standard Operating Procedures (DSOP)

4. Metrics – defined standards of performance against which actual performance can be

measured

An assessment rubric was created to compare the current state of the four sub-dimensions to

industry best practices. See Appendix A for the Process Management Assessment Rubric; the

highlighted text indicates the criteria which best describe the current state of the Process

Management in use by the UPS Truck Crew.

Of the four sub-dimensions, Dock Management and Package Labeling were rated as “Standard

Practices”. The other sub-dimensions, DSOP and Metrics, were rated as “Poor Practices” as no

standard operating procedure or performance metrics exist for this workgroup.

See Appendix B for a summary of the sub-dimension ratings and the rationale for each.

17

7. Exportability The processes in the evaluand are specific to the workgroup. There are no innovative methods in place which could be adapted for other MU departments. The processes in the evaluand are specific to the workgroup, and the following aspects of the UPS Truck Crew and the evaluation are valuable to other similar context. The greatest strengths of the UPS Truck Crew are the camaraderie and cooperation among the crew members and their loyalty to their long-time supervisor. This positive environment has contributed to the low turnover in this unskilled position, thus maintaining a high level of experience among the crew. Although the team chemistry encouraged by the UPS Truck Crew Supervisor cannot itself be exported, the supervisor’s interactions with his team could be studied to determine what elements of his supervisory style could help departments experiencing high personnel turnover rates. The evaluation team developed performance rubrics for the UPS Truck Crew based on documented industry best practices. According to the Inbound Manager, no performance rubrics or productivity standards are in place elsewhere in the receiving department. Productivity is a major concern of not only MU but the warehousing industry in general. MU must develop performance standards and measure performance outcomes to ensure that all parts of the receiving operation function at optimal levels, which should lead to improved warehouse productivity and continued revenue growth. To develop those standards, the operations management at MU should consult industry associations such as WERC, the Association for Operations Management, and the Council for Supply Chain Management Professionals for information on benchmarking and best practices.

http://www.werc.org/

http://www.apics.org/default.htm

http://cscmp.org/

http://cscmp.org/

18

Conclusions

8. Overall Significance The purpose of this evaluation was to understand the receiving process’s current level of merit

and to identify what changes to the process or resources could improve its performance. The

results obtained from each dimension were calculated and summarized in Appendix G. The

overall results (shown in Table 5) of the evaluation revealed the receiving process’s current

overall level of merit as “Acceptable” when measured on a 5-level scale (Poor - Mediocre -

Acceptable - Very Good - Excellent).

Table 5. Ratings for Overall Evaluand and Dimensions

The Instruments dimension was rated as "Acceptable" overall because the hardware and

software in use meet current needs, are properly maintained, and function correctly. All three

Instrument sub-dimensions were rated as “Standard Practices”; none are in critical need of

improvement to meet current needs, nor do any exceed industry standards.

The Workspace dimension was rated as “Acceptable” overall because under normal conditions

the floor space and layout is sufficient for the crew members to navigate easily between

locations. All three Workspace sub-dimensions were rated as “Standard Practices”; none are in

critical need of improvement to meet current needs, nor do any exceed industry standards.

Rating

Overall Evaluand

Dimension Weighting

Instruments

Important

Workspace

Very

Important

Process Management

Extremely Important

Poor Mediocre Acceptable Very Good Excellent

19

The Process Management dimension was rated as “Mediocre” overall. The Dock Management

and Package Labeling sub-dimensions were both rated as “Standard Practices”. The DSOP and

Metrics sub-dimensions were rated as “Poor Practices”; neither standard operating procedures

nor any performance metrics or measurements exist for this workgroup. Although current

needs are being met by the processes in place, future needs will be negatively impacted by the

lack of documentation and metrics.

The dimensions’ strengths and weaknesses are shown in Table 6, in order of importance within each dimension based on their potential level of impact on the evaluand.

Dimension Strengths Weaknesses

Instruments (Important)

1. Enough hardware to meet current needs.

2. Have the right tools to do the job.

3. Hardware is easy to use and in good repair.

4. Software integrates with next step in receiving process.

1. Not enough equipment to meet future demands.

2. Pallet jacks are past or near the end of their life expectancy.

Workspace (Very Important)

1. Equipment is laid out to maximize available floor space.

2. Enough space to meet current needs during periods of normal inbound volume.

1. No space to grow. 2. Lack of space causes choke-

points. 3. Lack of space makes it

difficult to walk during busy periods.

4. During busy periods floor space is very limited at best, non-existent at times.

Process Management (Extremely Important)

1. A package labeling system exists and is used (although not always followed)

2. UPS truck driver scheduled to arrive early enough for unloading to be completed in a timely manner.

3. Truck Crew is at work before UPS Truck Driver shows up.

1. No metrics exist to measure performance of the truck crew or their suppliers.

2. No SOP exists to verify Standards, train new employees, or pass on tacit knowledge.

3. UPS truck driver may be late, with no negative consequences, impacting schedule of UPS Truck Crew and other receiving areas.

20

Table 6. Strengths and Weaknesses of Evaluand Dimensions

21

9. Recommendations

Of the three dimensions evaluated in this project, Process Management was considered the

most important by the stakeholders (see Table 5 on page 17), but earned only a “Mediocre”

rating from the evaluation team. This dimension has the most opportunities for improvement in

the shortest amount of time, and at the lowest cost. Specific recommendations to improve this

dimension include:

1. Develop a standardized product labeling system. Technology exists to develop a barcode

that stores all shipment information and could update the system once scanned.

Customers could be required to print this barcode and attach it to the box or invoice

before shipment. The evaluation team understands that this step is already under

consideration by MU, and recommends that it be implemented if feasible.

2. Gather metrics to improve and manage the receiving process. By tracking metrics, the

receiving department can take appropriate actions to drive continual process

improvements. It is also good practice to report metrics to upper management and post

results where they are accessible to the UPS Truck Crew.

3. Document operating procedures as a way to standardize processes and terminology.

This will ensure that tacit knowledge is not lost as part of a succession management

plan, and will help provide consistent on-the-job training for new hires.

Workspace and Instruments both received an “Acceptable” rating. Although these dimensions

are less important than the process management dimension, both offer areas of opportunity

which can significantly improve the receiving process. These recommendations will require a

greater investment in monetary and time resources.

1. Within the Workspace dimension, the floor space should be of critical interest. The

Truck Crew is currently using its workspace to its full capacity. Because of the size and

amount of equipment required to perform the job, workspace is limited. The limited

workspace reduces the possible maximization of pallet and floor space usage due to a

lack of space to place needed pallets, and creates choke points between an Express Belt

and the Mechanical Roller. MU must seriously consider increasing floor space in

preparation for future demand or face decreased productivity and bottlenecks.

22

2. Within the Instruments dimension, the organization would need to update its

computerized and non-electronic equipment as well as its software to meet increasing

demands.

Some hardware in use is near end-of-life, and will need to be replaced soon. All

hardware should be inspected and evaluated for possible replacement or

upgrades.

MU should consider using hand-held computers with scanners rather than bulky

laptops.

Small packages are initiated by the UPS Truck Crew, and then the same data is

entered again by Direct Receiving. It would be more efficient to have Direct

Receiving initiate small packages so that the UPS Truck Crew can focus on large

packages.

23

Reporting This evaluation report was created for the benefit of the evaluand stakeholders at MU:

Director of Operations

Operations Manager

Inbound Manager

UPS Truck Crew Supervisor

Evaluation team member Chester Stevenson will e-mail a brief summary of the evaluation

team’s conclusions and recommendations, plus a copy of the full report, to the four

stakeholders and to the Vice President of Human Resources at MU. The stakeholders may

share the information with other MU personnel as desired.

Any questions about the evaluation may be directed to team member Chester Stevenson.

24

References

Brinkerhoff, R. O. (2006). Telling training’s story: Evaluation made simple, credible, and

effective. San Francisco: Berrett-Koehler.

Davidson, E. J. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. Thousand Oaks, CA: SAGE Publications.

Scriven, M. (1991). Evaluation thesaurus (4th ed.). Newbury Park, CA: SAGE Publications.

Scriven, M. (2007). Key evaluation checklist. Retrieved from http://www.wmich.edu/evalctr/checklists/kec_feb07.pdf

Vitasek, K., Harrity, C., O'Donoghue, K., & Symmes, S. (2007). Warehousing & fulfillment process benchmark & best practices guide. Retrieved from http://www.werc.org/assets/1/workflow_staging/Publications/701.PDF

http://www.werc.org/assets/1/workflow_staging/Publications/701.PDF

25

Appendix A: MU UPS Receiving Process Assessment Rubrics with Criteria Highlighting

Dimension of

Merit

Sub-

Dimension Poor Practices Standard Practices Best Practices

INSTRUMENTS

WEIGHTING:

Important

Hardware

(Non-Electronic) -

includes forklifts,

pallets, shelving,

conveyors, bins

Hardware is out of date and limited in

quantity or not in an adequate quantity.

Available hardware does not meet current

needs.

Hardware is out of repair or requires

excessive service.

Available hardware meets current needs.

Hardware is properly maintained and

functional.

Available hardware meets current and future

needs.

Backup equipment is available to replace out

of service equipment.

Hardware is efficient and does not add

additional steps to work processes.

Hardware

(Computerized)

Hardware is not mobile.

Hardware is out of date and not

compatible with newer software.

Hardware is out of date and limited in

quantity or not in an adequate quantity.

Available hardware does not meet current

needs.

Hardware is out of repair or requires

excessive service.

Hardware is mobile.

Hardware is compatible with newer

software.

Available hardware meets current needs.

Hardware is properly maintained and

functional.

Hardware is highly mobile and compatible

with the latest software.

Available hardware meets current and future

needs.

Backup equipment is available to replace out

of service equipment.

Hardware is efficient and does not add

additional steps to work processes.

Software

Software is slow, not easy to understand,

and not efficient.

Software does not interface with the next

area in the receiving process.

Software can be learned within an

acceptable time period.

Software interfaces with - and can be

retrieved by other parts of the receiving

process.

Software can be upgraded when

requirements change.

Software is easily understandable and easy to

use.

Software is easily upgradeable and easily

integrates with other company software.

Software provides metrics that can be

provided to customers.

Software is updated regularly to support best

practices.

26

Dimension of

Merit

Sub-

Dimension Poor Practices Standard Practices Best Practices

WORKSPACE

WEIGHTING:

Very Important Pallet Layout

Pallet placement is not formalized

Package placement is not clearly defined

Sorting area is not arranged to support

demand Issues meeting current volume.

Mobility around pallet area is difficult.

Pallets are laid out so that they do not hinder

movement.

Package placement is clearly defined.

Product placement is clearly defined.

Sorting area is arranged to support demand

issues meeting current and future volume.

Pallets are laid out to optimize movement,

and restrict fatigue or injury.

Utilization of

Floor Space

Floor space is poorly laid out and mobility

is difficult.

Poor layout in high traffic areas. Poor

housekeeping

Fixtures are placed so that they do not

hinder movement of product.

Areas are laid out well to support current

demand.

Floor space is well laid out to support current

and future demand.

Area is kept clean and clear of debris.

Space Available

Floor space does not meet current needs.

There is not enough room to unload an

entire truck during peak times.

Storage space is not adequate to hold

overstock during peak times.

Floor and storage space meets current

needs.

There is adequate space to unload a truck

and still allow for mobility.

Storage space is adequate to hold overstock

during peak times.

Floor and storage space is adequate to

accommodate current and future demand.

27

Dimension of

Merit Sub-Dimension Poor Practices Standard Practices Best Practices

Process

Management

Weighting:

Extremely

Important

Dock Management

Trailer deliveries not scheduled

Trailers not unloaded in a timely way

Shipper given specified delivery window

Timely unloading to avoid

detention/demurrage

Doc appointments are made and a

commitment to unload every vehicle within

two hours when the appointment is kept,

labor and dock space utilization is optimized

Package Labeling

No package labeling OR package

inconsistently or incorrectly labeled

Not all packages labeled by suppliers, but is

labeled upon receipt

Package is labeled by supplier, to

specification receipt so check-in can be

made with bar-code scan into the system

Documented Standard

Operating Procedure

(DSOP)

No process documentation exists OR

processes are written but not consistently

followed or made available

Receiving processes are documented and

available but tacit knowledge is not made

explicit

Receiving processes documented, made

readily available, and consistently followed

Metrics

Customer shipping errors are not tracked

(e.g. % of packages

received without a invoice or % of

packages received mislabeled)

No established

performance

metrics or SLAs

(service level

agreements)

established with

suppliers

Internal performance

indicators

(e.g. dock to

stock time) are

not tracked

Ad hoc process to track inbound receiving

errors or track internal functional metrics

Customer receiving errors are not shared

with customers

Basic receiving requirements are outlined and

shared with all customers

Formal process to collect Inbound receiving

metrics, but data is not shared with

customers

Basic receiving requirements are outlined

and shared with all customers

Formal process to collect inbound receiving

metrics and data is shared with customers

Internal performance metrics are collected

and posted or shared with employees

28

Appendix B: Summary of Ratings with Rationale

Dimension of

Merit Sub-Dimension Rating Rationale

Instruments

Hardware (Non-Electronic) Standard Practices

The criteria for non-electronic hardware all fall under the Standard Practices rating.

According to one stakeholder, some of the existing hardware will require replacement in

the near future but all equipment is currently adequate.

Hardware (Computerized) Standard Practices

The criteria for computer hardware all fall under the Standard Practices rating. The

laptops are not modern, but have sufficient capacity to run the necessary software.

Software Standard Practices

The criteria for software all fall under the Standard Practices rating. The software was

created in-house and continues to meet the organization’s needs.

Workspace

Pallet Layout Standard Practices

The criteria for pallet layout all fall under the Standard Practices rating. There is no

consistent layout plan for pallets, but the crew members are successfully able to create

the layout for daily conditions.

Utilization of Floor Space Standard Practices

The criteria for utilization of floor space all fall under the Standard Practices rating. Space

is limited, but the crew reports no significant problems in navigating the space under

normal volume conditions.

Space Available Standard Practices

The criteria for space available all fall under the Standard Practices rating. The space does

accommodate current normal volume, but is less able to accommodate peak volume.

There is no room for future expansion.

Process

Management

Dock Management Standard Practices

The criteria for dock management all fall under the Standard Practices rating. The main

issue with shipment delivery is related to the reliability of UPS delivery employees, which

MU cannot control.

Package Labeling Standard Practices

The criteria for package labeling all fall under the Standard Practices rating. Current

package labeling is adequate, but MU is looking at ways to improve it.

Documented Standard Operating

Procedure (DSOP) Poor Practices

The criteria for DSOP fall under the Poor Practices rating. There is no documentation of

standard operating procedures. All knowledge is tacit and communicated verbally to new

employees.

Metrics Poor Practices

The criteria for metrics all fall under the Poor Practices rating. No performance metrics

exist, nor do any methods of measuring aspects of performance.

29

Appendix C: Data Sources for Instrument Dimension

Sub-Dimension Data Sources Interviews

Observation Extant Data Truck Crew Supervisors

Hardware (non-

electronic)

(1) Extant Data –

Review equipment

inventory.

Review maintenance

records

(2) Interviews –

Management (n=3)

and Crew (n=4).

Q11: Can you think of one thing that you use now

(like a piece of equipment or a procedure) that

prevents you from working faster or more

accurately?

a. No -TC #1

b. Not really offhand. Space is the biggest hindrance.

TC #2

c. Some of the belts aren’t that good for rolling boxes.

When these belts are extended, the rollers are too far

apart and don’t roll boxes easily. TC #3

d. Nothing that gets in my way. The other guys get in

my way because of the limited space. TC #4

Q: How long do the Pallet

Jacks typically last before

they need to be replaced?

With the amount of use they

get during our receiving

process, they typically have a

lifecycle of 3 – 4 years. Two

of the Pallet Jacks have been

in use for 3 years, and two

have been in use for 6 – 7

years. Truck Crew

Supervisor

Equipment Inventory

includes:

Forklift (n=2)

Pallet Jack (n=4)

Express Belt

w/computer (n=4)

Roller (n=1)

Pallets

Bins (n=2)

Hardware

(Computerized)

(1) Extant Data –

Review equipment

inventory.

Review available

hardware

specifications.

(2) Interviews –

Management (n=3)

and Crew (n=4).

Q13: How would you improve the layout of your

work area?

d. The racks are not very sturdy. TC #4

Computerized

Inventory includes:

Computer with

scanner (n=3)

Software

(1) Interviews -

Management (n=3),

Crew (n=4),

Programmers (n=1).

Q7: How long does it take you to do the data entry

for each package? How easy is it to use the software

to do the data entry? How easy is it to make

corrections if you type something wrong?

a. 15 sec - 1.5 min (depending on how many

packages) Labels without bar codes = does not add

significant extra time to initiation To edit data entry,

search by invoice # and edit, very simple. Receiving

cannot access this software - if they find an error, they

bring it to the truck crew’s attention so the truck crew

Q1: How long have you

been using the current

software?

The software has been

unchanged for the last 15

years.

30

can make the correction. Can track which truck crew

member made the error, but not known if accuracy

problems are tracked and taken into account for

performance reviews -TC #1

b. 3-4 minutes at the longest. Very repetitive once

you learn the process should be quick (his time is

longer because he is new). Easy up to a point until

you get to the 2nd to last screen. Once it gets to that

point it would have to be corrected by supervisor -TC

#2

c. Not long, I just scan the barcode and UPS code

(these packages have standard UPS labels, I assume). I

have to open the packages occasionally to get a

packing list, but in most cases I just scan it. It’s easy to

make corrections. TC #3

d. Maybe 10 seconds, 20 seconds - I’ve been here 10

years so I can fly through it. The software is really easy

to use, and I can barely send an e-mail! Software is

not complex. Everybody has their own log-in, but not

everybody has editing permission. Supervisor and

unofficial assistant supervisor can make changes if

errors are found at the end of entry TC #4

31

Appendix D: Data Sources for Workspace Dimension

Sub-Dimension Data Sources Interviews

Observation Extant Data

Truck Crew Supervisors

Pallet Layout

(1) Interviews -

Management (n=3)

and Crew (n=4).

(2) Observation -

Observe receiving

process to determine

product placement

trends.

Q6: How easy is it to sort the large packages so that

all the packages from one sender are together? Are

the labels consistent or do you run into problems

with labels having different names for one sender?

b. It’s fairly simple, my biggest problem was spending

too much time trying to remember where I had put

things down, this became easier with practice.

Sometimes pallets are organized by #, other times by

return/guide/etc - it depends on who sets up the

pallets. TC#2

Q13: How would you improve the layout of your

work area?

a. The racks used to run the opposite way, but that

reduced the loading area so we changed the rack

orientation to give us more space. Racks are almost to

the ceiling now, at max height for forklifts -TC#1

b. A branching conveyor belt - pull off different

categories-guides, returns, and others This would

probably take up a lot of space, but would make it

easier to sort and pallet. TC #2

c. More space. Not sure offhand what would help the

layout. (moving student returns to another part of the

building - would require a different address and would

only eliminate one roller) TC#3 d. Just expand,

honestly. TC #4

Pallets are laid out in

rows but package layout

is not standardized.

32

Utilization of

Floor Space

(1) Interviews -

Management (n=3)

and Crew (n=4).

(2) Observation -

Observe receiving

process to determine

product placement

trends and floor space

usage.

Q12: Is there anything about the physical layout of

your warehouse area that gets in the way during

normal weeks? What about during high-volume

weeks?

a. High volumes SKUs that have no invoice (labeling),

having to wade through many packets -TC#1

b. The track area coming from the belt to the student

returns is a high-traffic area and is pretty tight for

space. (area around the first computer - the rack is

close to the conveyor, causes a bottleneck). Takes

more time because you have to detour with the

forklift. TC #2

c. In a normal week no, but yes during a busy week. TC

#3

There is a choke point

between the third

Express Belt and the

Roller.

Every open space is

used, with narrow paths

to walk though.

Space Available

(1) Interviews -

Management (n=3)

and Crew (n=4).

(2) Observation -

Observe receiving area

to determine space

availability.





a. The larger the load the harder it is. Space makes it

hard (after you get so much you forget where you put

it). Senders sometimes put the wrong information

(like name of individual rather than name of school) or

no information (they don’t understand they are NOT

the only vendor) -TC#1

Q8: During normal times, is it easy to move the

packages around to where they need to go?

a. Pretty easy but space is limiting -TC#1

b. Yes - TC#2

c. Pretty Easy TC # 3

d. Normal times yes but space is also an issue during

busy times. Freight (pallets) is bumping into other

shipments.

Q9: How does your work routine change during those

weeks when the number of inbound packages is

twice the usual volume?

a. It’s a lot more fast pace. Where we’re rushed we

are out of space quickly and it can be backed up for 5-

Every space is used.

33

6 weeks. When outbound is busy, inbound personnel

are diverted there to help, which slows down the

inbound processes. -TC#1

b. Between freight coming off and coming in, space

becomes very tight, hard to move around with the

forklift because there’s stuff everywhere.-TC #2

c. Space (paraphrase) TC #3

Q10: Can you think of one thing that could be added

to your work area that would make your job easier?

a. Space -TC#1

b. More space, definitely more space TC#2

c. When I scan, wondering why it can’t be done about

the people that are checking it in. Seems like

initiation is redundant - why not just put student

returns in the bins and have the direct receiving group

do the check-in? Jerry and the rest of the crew are in

favor of this. This would both remove duplication of

effort and free up space that the student return rollers

currently take up. TC#3

34

Appendix E: Data Sources for Process Management Dimension

Sub-dimension Data Sources

Observation Extant Data Truck Crew Supervisors

Dock

Management

(1) Interviews -

Management (n=3)

and Crew (n=4).

(2) Observation -

Observe unloading

process and record

times to completion.

Q4: How much time is there between the time you

start work and the time the first UPS truck arrives?

a. Anywhere from an hour to 1.5 hours, depending on

if we have the usual UPS driver. We come in at 7, the

drivers don’t come in until 8:30 -TC#1

b. When they have the normal driver, he’s usually

here about 8:30-9. When he was changing his job, the

communication over at UPS was off - the driver would

show up late, we would have to call UPS to find out

where the truck was. -TC #2

c. Depends on driver (8:15-10am)..

Sometimes there is packages left in receiving from

the day before that they can unpack until the driver is

there TC#3 The truck trailer gets there before we do

(it’s driven over and left - the truck cab goes off on

another job).

d. The truck trailer gets there before we do (it’s

driven over and left - the truck cab goes off on

another job). TC#4.

The UPS Truck is

available before the

truck crew arrives.

However, it cannot be

unloaded until the

driver is ready.

Package

Labeling

(1) Observation

(2) Interviews -

Management (n=3),

Crew (n=4), and

Programmers (n=2).





a. Senders sometimes put the wrong information

(like name of individual rather than name of school)

or no information (they don’t understand they are

NOT the only vendor) -TC#1

c. The count can sometimes be off but as long as the

labels are there. On the majority, the labels are there

-TC #3

Weaknesses in receiving

process

There are 7 to 9 different

variations for any given ISBN

(book sorted by type of

edition and physical

condition). Returns may not

be classified correctly by the

store/buyback, or returns

received may not have

originated at MBS in the

first place. Returns come in

from stores, buyback

Labels are colored,

depending on the type

of delivery.

Customers do not

always properly label

packages.

Sometimes customers

forget to put labels on

packages.

When customers ship

No package labeling

documentation exists.

No documentation exists

to inform customers of

how to properly label a

package.

35

d. It’s pretty simple after doing it for so long. Store

numbers are written on the packages, so we get used

to looking for store numbers to match up shipments.

Labels are mostly consistent, but I have seen

packages with label but no information or that have

just the sender’s first name and not the last, or boxes

with no labels (it’s a sticky tag that may accidentally

get peeled off during shipment) -TC #4

contractors, and individuals

- labeling not consistent,

may not be accurate.

Guaranteed buybacks pre-

labeled for accuracy.

Department would like to

look into more bar code

information. Right now the

receiving crew has to input

much information manually.

Inconsistency in how

bookstores label returns, so

receiving crew won’t have

accurate information (so

billing for returns may be

inaccurate because returns

from a given store may not

be processed as a group).-

Operations Manager

Problem with Invoicing of

shipments, improper

labeling, and improper

counts - weaknesses are on

the sender side. If sender

information is correct,

receiving goes like

clockwork. Biggest problem

is that improper

identification. Truck Crew

Supervisor

multiple packages, UPS

does not always deliver

them in the same

shipment.

Barcodes on the

package label only

include the UPS

Shipping number.

No customer education

process exists at any

level of the company

to inform shippers of

proper procedures.

Document

Standard

Operating

Procedure

(DSOP)

(1) Extant Data -

Review receiving and

shipping process

documentation.

(2) Interviews -

Management (n=3)

and Crew (n=4).

Q3: When you first started, was there any part of

the unloading and check-in process that took a little

more time to learn? How long did it take for you to

feel pretty comfortable doing your job?

a. Took about a week to get comfortable with the

entire process (including the computer). TC #1

b. It was all kind of new to me, so I learned one thing

at a time - there’s a lot to learn at once, so I had to

break it down. Had to learn the difference between

what’s a return and what’s a guide. I’m used to having

written directions/ guidelines, but all instruction here

New hires should be

comfortable within a 2-3

week time frame, by then

we’ll know if they can cut it.

They receive OJT. Just the

hands-on responsibility. Go

through proper lifting

methods (safety training, I

assume) . This (receiving

process) is a new beast even

if you have dock experience.

None Available

36

was verbal and each person had his own way of doing

things. It would have helped if I had something to

read that was standardized. Without that, I wasn’t

able to learn something until I actually needed to do

it. I didn’t think any particular process was more

difficult to learn than another. I started at the

beginning of a buyback rush so I was thrown into it -

learned the motions, wasn’t able to get more in-

depth until rush was over. -TC# 2

c. Took about a week to get comfortable with the

entire process (including the computer). TC#3

d. The unloading part. I’m not really computer savvy,

but it’s easy to pick up. With unloading, had to learn

the different shipment types. Took about two weeks

tops to feel comfortable. TC #4

Q14: If you were in charge, how would you improve

the efficiency of the receiving process?

a. I don’t know if there’s really any way to improve

(the people). Nothing I can think of offhand. TC #1 b.

When bringing someone in - I can pick up things

quickly because of my training but it might be hard

for others. I see it as a weeding out process to see

who they want to keep, but it would help to have a

truck crew-specific training manual and job aid.

Documented SOP - there’s isn’t one now. For

example, a standardized way of setting up the pallets.

TC #2

c. Current method is good as long as team works

together. TC#3

d. The number one thing is space, then the timing for

UPS arrival - we get here at 7 but he doesn’t get here

until later. TC #4

Can be overwhelming even

with experience. The

sorting is the hardest part of

the learning curve. (labeling,

categorizing). The guys

already do categorization on

floor. Ebb and flow of

volume would not really

make a color coded floor

labeling system feasible --

Truck Crew Supervisor

Metrics

(1) Extant Data -

Review receiving and

shipping metric

documentation.

(2) Interviews -

Management (n=3)

and Crew (n=4).

Setting benchmarks for

future evaluations;

establishing regular re-

assessments of receiving

processes; defining ideal

performance for individuals;

defining ideal performance

for the receiving team;

exporting elements of the

None Available

37

receiving processes to other

parts of the organization;

and/or developing

evaluation processes for

other parts of the

organization. -Operations

Manager

Would like to see more

metrics [in receiving

process} such as copies

checked per hour, book

shelved per hour no formal

written documentation -

Inbound Manager

38

Appendix F: Sub-dimension Weighting of Importance by Stakeholders

Sub-Dimensions Inbound Manager Operations Manager Truck Crew Manager


Extremely Important Important Extremely Important

Hardware (Computerized) Extremely Important Important Extremely Important

Software Very Important Very Important Very Important

Pallet Layout Important Important Extremely Important


Extremely Important Extremely Important Extremely Important

Space Available Extremely Important Very Important Extremely Important

Dock Management Extremely Important Important Extremely Important

Package Labeling Extremely Important Very Important Extremely Important

Documented Standard Operating Procedure (DSOP)

Important Important Important

Metrics Extremely Important Very Important Important

39

Appendix G: Scoring Rubrics for MU UPS Receiving Process Evaluation

Rubric for Ranking Importance

Rubric for Ranking Sub-Dimensions

Important Very Important Extremely Important Poor Practices Standard

Practices Best Practices

1 2 3 1 2 3

Sum of Weighted Dimensions

Dimension Weighting Weighting Value

(I) Sub-Dimension

Sub-Dimension Rating (SDR)

SDR Value (R) I x R

INSTRUMENTS Important

1 Hardware (non-electronic)

Standard Practices 2 2

1 Hardware (Computerized) Standard Practices 2 2

1 Software Standard Practices 2 2

Sum of Importance and Rating 6

WORKSPACE Very Important

2 Pallet Layout Standard Practices 2 4

2 Utilization of Floor Space

Standard Practices 2 4

2 Space Available Standard Practices 2 4


PROCESS MANAGEMENT Extremely Important

3 Dock Management Standard Practices 2 6

3 Package Labeling Standard Practices 2 6

3 Documented Standard

Operating Procedure (DSOP) Poor Practices 1 3

3 Metrics Poor Practices 1 3


Total Sum of Weighted Dimensions 36

40

Score Combinations for Dimension Rating

Sub-Dimension

Dimension

Weighting

(W)

Lower (L) Boundary

Poor (P) Mediocre (M) Acceptable (A) Very Good (V) Excellent (E)

All sub-dimensions

received a 1

Most sub-dimensions

receive a 1, and one

sub-dimension

receives a 2 for each

Dimension

Most sub-dimensions


sub-dimension


Dimension

Most sub-dimensions


sub-dimension


Dimension

Most sub-dimensions


sub-dimension


Dimension

All sub-dimensions

receive a 3


1 1 1 1 2 2 3

Hardware (Computerized)

1 1 1 2 2 3 3

Software 1 1 2 2 3 3 3

Pallet Layout 2 2 1 1 2 2 3


2 2 1 2 2 3 3

Space Available 2 2 2 2 3 3 3

Dock Management 3 3 1 1 2 2 3

Package Labeling 3 3 1 2 2 3 3

DSOP 3 3 1 2 2 3 3

Metrics 3 3 2 2 3 3 3

41

Matrix for Calculating the Lowest Possible Score for Each Rating Level

Sub-Dimension

Dimension

Weighting

(W)

Lower (L) Boundary Poor (P) Mediocre

(M)

Acceptable

(A)

Very Good

(V)

Excellent

(E)

Highest possible score for each weighted dimension

(W x L)

Highest possible score for each weighted dimension

(W x P)

Highest possible score

for each weighted dimension

(W x M)



(W x A)



(W x V)


for each weighted

dimension (W x E)


1 1 1 1 2 2 3

Hardware (electronic)

1 1 1 2 2 3 3

Software 1 1 2 2 3 3 3



2 2 2 4 4 6 6


Dock Management

3 3 3 3 6 6 9

Package Labeling 3 3 3 6 6 9 9

DSOP 3 3 3 6 6 9 9

Metrics 3 3 6 6 9 9 9

Sum of Sub-Dimensions 21 27 36 48 57 63

42

Synthesized Rubric Levels to draw Evaluative Conclusion about OVERALL Evaluand

Rating Description Rationale

Poor (21-26) Score of total weighted dimensions is between 21-26 Most or all are Poor Practices

Mediocre (27-35) Score of total weighted dimensions is between 27-35 Within each dimension, only one sub-dimension

met Standard Practices, and others are Poor

Practices

Acceptable (36-47) Score of total weighted dimensions is between 36-47 Within each dimension, only one sub-dimension is

Poor Practices and others are Standard Practices

Very Good (48-56) Score of total weighted dimensions is between 48-56 Within each dimension, at least one sub-dimension

was Best Practices and others were Standard

Practices

Excellent (57-63) Score of total weighted dimensions is between 57-63 Within each dimension, most or all sub-dimensions

were Best Practices and one was Standard

Practices

43

Instruments Scoring Rubric

Sub-Dimension

Dimension

Weighting (W) Lower (L) Boundary


All sub-dimensions

received a 1

Most sub-dimensions


sub-dimension


Dimension

Most sub-dimensions


sub-dimension


Dimension

Most sub-dimensions


sub-dimension


Dimension

Most sub-dimensions


sub-dimension


Dimension

All sub-dimensions

receive a 3


1 1 1 1 2 2 3

Hardware (Computerized)

1 1 1 2 2 3 3

Software 1 1 2 2 3 3 3

Sum of sub-dimensions 3 4 5 7 8 9

44

Synthesized Rubric Levels to draw Evaluative Conclusion about INSTRUMENTS


Poor (3) Score of total weighted sub-dimensions is 3 Most or all are Poor Practices

Mediocre (4) Score of total weighted sub-dimensions is 4

Within each dimension, only one sub-

dimension met Standard Practices, and others

are Poor Practices

Acceptable (5 or 6) Score of total weighted sub-dimensions is between 5

or 6

Within each dimension, only one sub-

dimension is Poor Practices and others are

Standard Practices

Very Good (7) Score of total weighted sub-dimensions is 7

Within each dimension, at least one sub-

dimension was Best Practices and others were

Standard Practices

Excellent (8 or 9) Score of total weighted sub-dimensions is 8 or 9

Within each dimension, most or all sub-

dimensions were Best Practices and one was

Standard Practices

45

Workspace Scoring Rubric

Sub-Dimension

Dimension



All sub-dimensions

received a 1

Most sub-dimensions


sub-dimension


Dimension

Most sub-dimensions


sub-dimension


Dimension

Most sub-dimensions

receive a 2, and

one sub-dimension


Dimension

Most sub-dimensions


sub-dimension


Dimension

All sub-dimensions

receive a 3



2 2 2 4 4 6 6



46

Synthesized Rubric Levels to draw Evaluative Conclusion about WORKSPACE


Poor (6 or 7) Score of total weighted sub-dimensions is 6 or 7 Most or all sub-dimensions are Poor Practices

Mediocre (8 or 9) Score of total weighted sub-dimensions is 8 or 9 Only one sub-dimension was Standard Practices, and others are Poor Practices

Acceptable (10-13) Score of total weighted sub-dimensions is

between 10 and 13 Only one sub-dimension is Poor Practices and others were Standard Practices

Very Good (14 or 15) Score of total weighted sub-dimensions is 14 or

15 At least one sub-dimension was Best Practices and others were Standard Practices

Excellent (16-18) Score of total weighted sub-dimensions between

16 and 18 Most or all sub-dimensions were Best Practices and one was Standard Practices

47

Process Management Scoring Rubric

Sub-Dimension

Dimension



All sub-dimensions

received a 1

Most sub-

dimensions receive

a 1, and one sub-

dimension receives

a 2 for each

Dimension

Most sub-dimensions


sub-dimension


Dimension

Most sub-dimensions

receive a 2, and

one sub-dimension


Dimension

Most sub-dimensions


sub-dimension


Dimension

All sub-dimensions

receive a 3

Dock Management

3 3 3 3 6 6 9

Package Labeling

3 3 3 6 6 9 9

DSOP 3 3 3 6 6 9 9

Metrics 3 3 6 6 9 9 9


48

Synthesized Rubric Levels to draw Evaluative Conclusion about PROCESS MANAGEMENT


Poor (12-14) Score of total weighted sub-dimensions is

between 12 and 14

Most or all are Poor Practices

Mediocre (15-20) Score of total weighted sub-dimensions is

between 15 and 20

Within each dimension, only one sub-dimension met

Standard Practices, and others are Poor Practices

Acceptable (21-26) Score of total weighted sub-dimensions is

between 21 and 26

Within each dimension, only one sub-dimension is

Poor Practices and others are Standard Practices

Very Good (27-32) Score of total weighted sub-dimensions is

between 27 and 32

Within each dimension, at least one sub-dimension

was Best Practices and others were Standard

Practices

Excellent (33-45) Score of total weighted sub-dimensions is

between 33 and 45

Within each dimension, most or all sub-dimensions

were Best Practices and one was Standard Practices

ipt 530-edubose evaluation methodology team project

Business

formative evaluation

methodologythe evaluation

process evaluations

process management dimension

inbound receiving process

upsreceiving process

inboundreceiving process

process assessment rubrics