example r&d project report

FY 2011 R&D Project Report

Research of Hiring PipelineOpportunities using Crowdsourcing

Interim Report

by Thomas F. Kaplan

ESHRI/R&D PMO

June 2011

Readers should note that the enclosed report is only an

example.

Project Manager Contact Details

Name: Thomas F. Kaplan

Address: 1900 E Street, NW, Room 3304b, Washington, DC 20415

Telephone: 202-606-4454

Email: [email protected]

Not for general distribution without express permission of the HRS R&D PMO ii

mailto:[email protected]

Table of Contents

Executive Summary...............................................................................................................1

Introduction.........................................................................................................................2

Objectives............................................................................................................................3

Methodology.......................................................................................................................4

Research Description............................................................................................................5

Results.................................................................................................................................6

Implications..........................................................................................................................7

Recommendations...............................................................................................................8

Appendices...........................................................................................................................9

Appendix A: Project Request

Appendix B: Potential Uses of Crowdsourcing (extract)

Glossary

References

Not for general distribution without express permission of the HRS R&D PMO iii



Executive Summary

Leveraging crowdsourcing for use as a potential hiring pipeline may give agency recruiters additional sources of potential candidate supply (in addition to applicant inventories). Briefly, this project attempted to research the uses of crowdsourcing for use as a potential recruitment tool. Specifically:

In 2009, crowdsourcing applications became widely available and appeared to have potential for use in as an adjunct for federal recruiting. A review of available research at that time indicated there was no ongoing investigation on the use of crowdsourcing in federal recruitment.

Since this project involved simulated use of crowdsourcing as a hiring tool, it quickly became apparent that, in general, commercial “off-the-shelf” crowdsourcing applications were not well suited for this task. On the other hand, the research indicated that customization of the crowdsourcing application could assist recruitment efforts in certain circumstances, including targeted recruitment efforts (e.g., for specialized engineering occupations such as electronics engineers with specialization in automotive and railroad industries, etc.).

Two existing customers at the Department of Transportation (notably that had difficulty in surfacing any candidates for electronics engineer positions at the National Highway Transportation Safety Administration and the Federal Railways Administration) expressed an interest. This current HRS customer was interested in pursuing a test development application based on the project’s outcome.

The project, in general, progressed without significant delay in meeting its schedule. The project scope remained unchanged from its original definition and the budget was adequate to achieve project’s original intent.

Project lessons learned include:

Engage in-house and contractor staff early in the R&D process

Spend more time presenting and discussing research goals and objectives with potential customers

Not for general distribution without express permission of the HRS R&D PMO 1

Crowdsourcingis the act of outsourcing

tasks, traditionally performed

by an employee or contractor,

to an undefined, large group

of people or community (e.g.,

a crowd), through an open

call.



Daily, spend more time documenting project actions and outcomes.

Introduction

The project objective was to explore potential recruitment pipeline enhancement and adjunct methods using crowdsourcing opportunities. This project would potentially allow agencies to develop and access large numbers of candidates differentiated by various recruitment categories (e.g., veteran, PWTD, etc.). Project outcome could lead to an additional method for identifying potential job candidates.

The idea for this research project came from an article published in the June 2009 “The American Journal for Applied Research,” by Ann Benson (Appendix B).

This research is important as it directly supports OPM’s Hiring Reform Initiative. Leveraging crowdsourcing as a potential hiring pipeline opportunity can give recruiters additional sources of candidate supply (in addition to applicant inventories). Additionally, crowdsourcing leverages emerging social media networking toward use as a potential recruitment method. Utilizing the “power” of social networking may boost recruitment opportunities not presently available through regular methods.

This project began in October 2010 and completed in September 2011. Project participants included myself and Samantha Jones, who acted as the crowdsourcing application technician.

Project beneficiaries potentially include HR staffs at two operating administrations at the Department of Transportation and other defense and civilian agency HR organizations.

Key findings include current crowdsourcing applications must be customized to realize the full potential of identifying potential candidates for specific occupations. Customization should yield a useful recruitment application that identifies job candidates not possible through contemporary methods. Additionally, agency IT operations would need to ensure maintenance of 508 compliance and IT security.

The remainder of this report describes in detail the research conducted and specific findings.




Objectives

From the Project Request Form I, Objective (Appendix A): “Use commercially available crowdsourcing sites (e.g., IdeaScale, etc.) to identify potential candidates for federal employment.” Additional objectives included 1) determine the usefulness of crowdsourcing as a recruitment tool, 2) evaluate the degree to which a crowdsourcing tool would need to be modified, if at all, for recruitment purposes (to meet Title 5 USC requirements), and 3) establish a brief process description of the information flow between potential applicant, the crowdsourcing application, and federal recruiters.

Achievement of objectives relied on a non-governmental crowdsourcing application to identify potential candidates for federal employment. As such, there was no commercial sector analogue to follow and there was only time enough to develop a limited test application to gain a proof of concept; hence, application modification was limited, but included a means to assess candidate interest and academic credentials and basic competencies.

An actual crowdsourcing application, intended to support federal hiring, would likely require extensive research and planning to be truly effective. For example, researchers completed only limited flow process charting involving a three-tiered hierarchy; this would likely need to be expanded to at least a five-tiered hierarchy process description to determine if logical flow exists between the processes.

While the crowdsourcing application would need to conform to federal IT standards (e.g., FIPS-2, etc.) the government would not need to host the application thus allowing greater leverage of cloud computing opportunities through partnering with the private sector. If successful, using a crowdsourcing application to identify potential federal job applicants could reduce agencies’ information technology investment, over time.

The next section describes the project’s methodology.




Methodology

To achieve the main and additional three objective in the previous section required the

evaluation of extant crowdsourcing providers, developing a Request for Information and its publication on FedBizOps, meeting individually with potential commercial crowdsourcing providers to

understand the uses, constraints, and limitations of the crowdsourcing applications, contracting with a crowdsourcing provider (IdeaScale) to provide a simulated

environment where potential job applicants would interact with the application, and developing interface templates for potential applicants to submit their credentials and

competencies for evaluation and agency follow-up.

Government employee researchers designed and applied several interface templates’ with specifications including

notice of user legal and compliance requirements prior to use of the application

capture of contact information

statement of academic achievement

disclosure of certifications and credentials

self-assessment of general and technical competencies, and other relevant tasks

Fifty pre-selected volunteer test subjects were given access to the crowdsourcing application and encouraged to enter at least five different fictitious profiles of academic, competency, and certification achievement for use as a proof of concept. This test generated 237 separate profiles for use in evaluating the attainment of stated objectives.

The next section presents the research description.




Research Description

After selecting a crowdsourcing provider, research staff met with the selected crowdsourcing contractor to evaluate and establish the range of actions needed to complete the project. The research staff and the contractor developed mock-up web site screens for review by staffs at the Office of the Chief Information Office and Office of the General Counsel. Research staff modified web site screens to include statements on all pages stating that the web site was a simulation for test purposes only.

Research staff provided test crowdsourcing web site information to pre-selected volunteers who were given basic instructions on the purpose and use of the web site. Input screens were shown to the volunteers and the research staff answered volunteer’s questions. The fifty volunteers were provided website access for five days to input data into the crowdsourcing application. Thirty-six of the fifty volunteers successfully entered all requested data, while the remaining 24 failed to complete the data entry tasks (due to simulated disruption of website access by the research staff). Seventy-two percent of the volunteers were able to remain logged in to the crowdsourcing application over the five days to complete required data entry tasks.

Thirty-three volunteers completed exit questionnaires describing their experience using the crowdsourcing application.

The next section presents the research results.




Results

This test application of a crowdsourcing application as a potential hiring pipeline, in this case as a precursor to applying for vacant federal positions and applicant inventories for prospective job candidates, provided the following results:

General:

Thirty-three of the 36 volunteer applicants completed the on-line crowdsourcing profile in less than 2 hours.

Twelve of the 36 volunteer applicants indicated they were able to complete the online crowdsourcing profile in less than one hour.

Five of the 36 volunteer applicants indicated they called researchers at least once for guidance on using the on-line crowdsourcing application.

Research staff reviewed the 33 completed volunteer profiles and determined that 28 of the profiles achieved 100 percent data completion.

Technical:

The crowdsourcing application remained operational 24 hours per day during the five-day test period. Researchers intentionally disrupted volunteer users at random intervals to simulate network contention often experienced by on-line system users.

All volunteers entered “unscripted” information (i.e., they entered purely individual personal information)

No volunteer applicants were able to cause the crowdsourcing application to fail to retain entered data.

Other:

Research staff was able to retrieve and analyze volunteer applicant data during the five-day access period.

A review of volunteer applicants exit questionnaires indicated an overall score of 4 out of 5 (5 indicating a high degree of satisfaction) with their experience using the crowdsourcing application.

The next section presents research implications.




Implications

Research staff and the project manager deemed the research effort as generally successful given the limitations and constraints1 of the project. While there may be a need for additional R&D efforts to fully evaluate crowdsourcing as a potential “hiring” tool, this test indicates that using a crowdsourcing application to identify potential applicants for federal job vacancies has merit. Looking forward, it may be possible to replace current agency succession planning “applicant inventories” with a crowdsourcing application that 1) quickly identifies interested job applicants, 2) reduces some agency IT costs2, and 3) gives agencies an additional source for identifying potential applicants with critically important skills.

The relatively low-cost investment involved in researching the potential use of the crowdsourcing application (less than $45K) and the use of uncompensated volunteers allowed researchers to achieve a proof of concept that crowdsourcing is useful tool in identifying potential job applicants for difficult to fill positions.

The next section presents recommendations.

1 Limitations include the small number of volunteer applicants involved and the short duration of the test period.

2 Some IT costs are borne by the crowdsourcing provider (e.g., application hosting, help desk services, etc.).Not for general distribution without express permission of the HRS R&D PMO 7



Recommendations

The following list presents tentative recommendations for evaluation against current business plans.

Provide a summary of this report to current customer agency Chief Human Capital Officers (CHCO) and determine if they would be interested in developing and implementing a customized solution.

Present the project findings at the CHCO Training Academy.

Consider promoting/sharing the results of this research at public venues (e.g., Performance Institute, Partnership for Public Service, SHRM, PIHRA, etc.).




Appendices







Project Request Part I

1. Project name (without acronyms or abbreviations) and today’s date. Research Hiring Pipeline Opportunities using Crowdsourcing. August 5, 2010.

2. Requested project description. Explore potential recruitment pipeline enhancement/adjunct using crowdsourcing opportunities. This project would potentially allow agencies to develop and access large numbers of candidates differentiated by various recruitment categories (e.g., veteran, PWTD, etc.). Project outcome could lead to an additional method for identifying potential job candidates.

3. Practice Area and Sponsor Name. Also identify other Practice Areas that could benefit from the proposed project. HRS/ESHRINNOV, Thomas F. Kaplan. Unaware of benefit for other Practice Areas.

4. When did the project begin (state month and year), or is it new? This is a new project.

5. If the request involves a legacy R&D project, what was/is the project name? NA

6. List each key characteristic of the proposed project under the headings:

Feature Function Advantage (technical/economic/social)

Wide availability World Wide Web based Internet based social media provider

3rd party hostedCommercially owned and operated hardware and software platform

Low cost alternative to equivalent government owned and operated site

Open Government compliant

Facilitates greater transparency

Helps OPM achieve Presidential mandate

Agency availability

All Executive Branch agencies can access a vast hiring pipeline

Extends OPM leadership capabilities

7. Describe any other special aspect or wider applications of the requested project. None.

8. Give a best estimate of any resources used to date (e.g., 2.0 full-time equivalents, $10,000, expendable research materials, 100sq ft lab space, 40% use of equipment, etc). 10 hours or 0.005 FTE of the sponsor’s duty time.

Not for general distribution without express permission of the HRS R&D PMO



9. Describe the requested R&D efforts in terms of objectives, requirements for human resources, risk, space, and relate these items to the funds and time required. Use commercially available crowdsourcing sites (e.g., IdeaScale, etc.) to identify potential candidates for federal employment. Government staff will maintain the site for an estimate of 1,000 hours per year. Risk in undetermined. Funds required are approximately $1,000 for an annual subscription for unlimited users, ideas, and commenters with refund for unused portion. See table.

Category Funds Risk Space Time

Labor (GS-11/5 DC) w/.284 benefits

$33.92ph x 1.284 = $43.55 x 1000 = $43,550

Moderate 1,000 hours

Non-labor $999.003 Moderate One year

Other NA NA NA NA

10. Explain if a proof of concept and prototype will result from the requested project. If not, what extra work would need to be done and by whom. Proof of concept and prototype will likely result from the project.

11. Please add any other information you consider relevant. Leveraging crowdsourcing as a potential hiring pipeline can give recruiters additional sources of candidate supply in addition to applicant inventories. Additionally, using reverse auction procurement approach would likely drive down cost of crowdsourcing subscription. Please refer to attached commercial crowdsourcing documents for additional information.

3 IdeaScale Subscription for prototyping/proof-of-concept testing.Not for general distribution without express permission of the HRS R&D PMO



Project Request Part II

1. Project name (without acronyms or abbreviations) matching the project name from Research and Development Project Request Part I and today’s date. Research Hiring Pipeline Opportunities using Crowdsourcing, August 5, 2010.

2. Is there a patentable invention/copyrighted product, or could one arise from this R&D effort? No.

3. Has there been any public disclosure about the project? If yes, please give details. The project sponsor knows of no known public disclosure.

4. Comment on any R&D or commercialization by other researchers in the field of the project. Unknown.

5. Please list any companies and organizations that could be interested in the project, details of any contacts and reasons for the likely interest. IdeaScale, Innocentive, Ninesigma, etc. No contacts as of yet. Its likely these companies would show interest in the R&D effort since it would (further) demonstrate their relevance in the federal marketplace.

6. Please name any similar services or products that already exist. Unknown.

7. Please compare the new service or product with any existing alternative that provides similar outcomes. NA.

8. Please list the types of customers and end-users that are likely to purchase or benefit from the new product or service. Customers include federal agencies and State/local agencies and others interested in identifying potential public sector job candidates.

9. Please provide an estimate of market locations and annual market size of the new service or product. Market locations are worldwide since host platform is virtual (via World-Wide Web). Annual market size is undetermined at this time; however, the market potential is vast.

10. Is there a need for Government approval for the release of the service or product? Yes.

11. Please complete a Project Cost Estimate Worksheet and attach it to this project request.




Project Cost Estimate WorksheetProject Name: Research Hiring Pipeline Opportunities using CrowdsourcingToday’s Date: August 5, 2010Planned Project Start Date: January 1, 2011Planned Project End Date: December 31, 2011

One-year subscription for IdeaScale with unlimited use is $999.


Cost Category Year 1 Year 2 Total

Salaries $42,269 Benefits 0.00 Awards 0.00Total Salaries and Benefits $42,269 Travel Transportation of Things Communications/Utilities/Misc Printing Training Contracts $999 Other Services Supplies EquipmentTotal non-Salaries and Benefits $999Total $43,268Federal Employee Hours 1,000




The following extract came from the June 2009 edition of “The American Journal for Applied Research,” by Ann Benson.

“Identifying potential government employees with critically needed skills can be a laborious effort. Searching online resume banks, conducting job fairs, communicating with college and university placement offices may not yield candidates in areas such as electronics engineering and IT architecture and design.

Recruiters need a new approach that leverages the ‘power’ of crowdsourcing. Consider the following scenario, which presents a possible adaptation of crowdsourcing to create a “job interest pipeline.”

Agency A’s succession plan identifies a future skill shortage of electronics engineers. The agency has attempted to identify potential future candidates through traditional means (including USA Jobs recruiting for future needs) with limited success.

At the same time, Bill Murphy is in his senior year at the DeForest University pursuing a degree in general engineering with an emphasis in electronics. Lately, Bill Murphy is wondering about his future employment opportunities when he learns about the “Job Opportunity Bank” (The Bank) at his university’s placement office.

Bill learns more about The Bank. The Bank allows college level students to place their personal and academic profiles online for review by government agencies. Bank students invite professors and other students to comment on the veracity of the profiled student and to add their comments. Bank students control which comments are allowed to be viewed by recruiters and hiring managers.”




Glossary

Note: A glossary is optional.

Term Definition

Crowdsourcing The act of outsourcing tasks, traditionally performed by an employee or contractor, to an undefined, large group of people or community (e.g., a crowd), through an open call.

Job Interest Pipeline An online database of candidates interested in certain occupations.




References

Identify footnotes, references, and cross-references.

Note: References is optional.




Plain English Summary

This summary format is here as a reminder and it should not be included in the TOC of the final report. All final (not interim) projects must have a plain English summary.

Project Title: Research of Hiring Pipeline Opportunities using Crowdsourcing

Researcher: Thomas F. Kaplan

Organization: U.S. Office of Personnel Management, Human Resources Solutions, center for Emerging Solutions and HR Innovations, Research and Development Program Management Office

Telephone: (202) 606-4454

Fax: (202) 606-1982

Email: [email protected] Research the use of commercially available crowdsourcing sites

(e.g., IdeaScale, etc.) to identify potential candidates for federal employment. Additionally, 1) determine the usefulness of crowdsourcing as a recruitment tool, 2) evaluate the degree to which a crowdsourcing tool would need to be modified, if at all, for recruitment purposes (to meet Title 5 USC requirements), and 3) establish a brief process description of the information flow between potential applicant, the crowdsourcing application, and federal recruiters.

Methodology To achieve the main and additional three objective in the previous section required the evaluation of extant crowdsourcing providers, developing a Request for Information and its publication on FedBizOps, meeting individually with potential commercial crowdsourcing providers, contracting with a crowdsourcing provider, and developing interface templates for potential applicants to submit their credentials and competencies for evaluation and agency follow-up.

Research Description

Research staff met with the selected crowdsourcing contractor to evaluate and establish the range of actions needed to complete the project. The research staff and the contractor developed mock-up web site screens for review by staffs at the Office of the Chief




Information Office and Office of the General Counsel. Research staff provided test crowdsourcing web site information to pre-selected volunteers who were given basic instructions on the purpose and use of the web site. Input screens were shown to the volunteers and the research staff answered volunteer’s questions. The fifty volunteers were provided website access for five days to input data into the crowdsourcing application. Thirty-six of the fifty volunteers successfully entered all requested data.

Results This test application of a crowdsourcing application as a potential hiring pipeline, in this case as a precursor to applying for vacant federal positions and applicant inventories for prospective job candidates, indicated an overall score of 4 out of 5 (5 indicating a high degree of satisfaction) with their experience using the crowdsourcing application.

Implications Looking forward, it may be possible to replace current agency succession planning “applicant inventories” with a crowdsourcing application that 1) quickly identifies interested job applicants, 2) reduces some agency IT costs, and 3) gives agencies an additional source for identifying potential applicants with critically important skills. The relatively low-cost investment involved in researching the potential use of the crowdsourcing application (less than $45K) and the use of uncompensated volunteers allowed researchers to achieve a proof of concept that crowdsourcing is useful tool in identifying potential job applicants for difficult to fill positions.

Recommendations

Provide a summary of this report to current customer agency Chief Human Capital Officers (CHCO) and determine if they would be interested in developing and implementing a customized solution. Additionally, present the project findings at the CHCO Training Academy, and consider promoting/sharing the results of this research at public venues (e.g., Performance Institute, Partnership for Public Service, SHRM, PIHRA, etc.).


example r&d project report

Documents