executive summary - dceryesmist.weebly.com file · web viewcalifornia state university. monterey...
Post on 08-Aug-2019
215 Views
Preview:
TRANSCRIPT
Running Head: YACHT CLUB SYSTEM TRAINING 1
CALIFORNIA STATE UNIVERSITY
MONTEREY BAY
Yacht Club System Training
CAPSTONE REPORT
Submitted in partial satisfaction of requirements of the degree of
MASTER OF SCIENCE in
Instructional Science and Technology
Deborah Ceryes
December 11, 2018
Capstone Approvals: (At least one advisor and capstone instructor should approve)
___________________________ _____________
Dr. Sarah Tourtellotte Signature Date
___________________________ _____________
Dr. Miguel Lara Signature Date
YACHT CLUB SYSTEM TRAINING 2
Yacht Club System Training
Deborah Ceryes
California State University, Monterey Bay
December 11, 2018
IST 699 Capstone
Professor Miguel Lara, PhD
YACHT CLUB SYSTEM TRAINING 3
Table of Contents
Executive Summary.............................................................................................................5
Yacht Club System Training...............................................................................................6
Problem Description............................................................................................................6
Target Audience...............................................................................................................7
Learner Analysis..............................................................................................................7
Context Analysis..............................................................................................................9
Current State..................................................................................................................10
Environmental Scan.......................................................................................................10
Industry Trends..............................................................................................................10
Solution Description..........................................................................................................11
Objectives......................................................................................................................11
Learning Solution..........................................................................................................12
Task Analysis.................................................................................................................12
Media and Delivery System Decisions..........................................................................13
Learning Theories..........................................................................................................13
Instructional Principles and Strategies...........................................................................15
Multimedia Principles and Strategies............................................................................16
Challenges and Breakthroughs......................................................................................17
Methods/Procedures..........................................................................................................18
Major Deliverables........................................................................................................18
Design Analysis.............................................................................................................18
Agenda.......................................................................................................................19
Activity Descriptions.................................................................................................19
Implementation Plan......................................................................................................20
YACHT CLUB SYSTEM TRAINING 4
Resources...........................................................................................................................20
Logistics and Budget.....................................................................................................21
Technical Skills and External Resources.......................................................................21
Timeline/Progress Report..................................................................................................21
Milestones Checklist......................................................................................................21
Evaluation Plans................................................................................................................22
Formative Evaluation.....................................................................................................23
Summative Evaluation...................................................................................................24
Level I —Reaction.....................................................................................................24
Level II—Learning....................................................................................................24
Level III—Behavior...................................................................................................24
Level IV—Results.....................................................................................................25
Conclusion.........................................................................................................................25
References..........................................................................................................................27
Appendices........................................................................................................................28
Appendix A: VYC Membership and ACH Reports......................................................28
Appendix B: Single Subject Testing (SST) Instrument................................................28
Appendix C: Level I – Reaction on Learning Objectives.............................................28
Appendix D: Level II – Learning Analysis using Quiz Results...................................28
YACHT CLUB SYSTEM TRAINING 5
Executive Summary The Vallejo Yacht Club (VYC), located in the San Francisco Bay Area, has operated for
over 100 years. (Evans, 2000) Governed by an elected Board of Directors (BOD), a small office
staff manage day to day operations and a host of volunteers oversee club events and programs.
To streamline and improve club operations, the BOD adopted an integrated financial system
from Club Essential, LLC, (CE) a third-party vendor that specializes in club management
systems. Due to the system implementation, customized training modules are needed for: 1)
member online payments, 2) point-of-sale (POS) transactions, and 3) back-office procedures
using the Club Management Application (CMA). Training modules provide opportunities for
members and staff to learn how to use the new system and maximize its investment value.
The learning solution includes a website developed for each subject area. Websites are
embedded with screencasts, software simulations, JavaScript/jQuery interactions, and assessment
tools. Screencasts and simulations include feedback as well as practice options to support
learner engagement and motivation. Self-paced navigation within and across pages embodies
autonomous and self-directed learning. Design components follow a watch/try/assess format and
address the adult learner’s need for active participation. Immediate application to gain
experience with new skills is provided through practice and testing tools. Learners preview
content prior to training on an actual POS device or office workstation. Quiz and survey tools
allow learners to demonstrate and reflect on their learning. Job aids and resources can also be
utilized for synchronous training.
Several forms of evaluation occur to assess usability, learning objectives and knowledge
transfer using formative and summative evaluation tools. Overall feedback on design elements
inform improvements to multimedia as well as the learner experience. Quizzes build confidence
to lift learner readiness. Self-assessments monitor learner readiness to apply new skills or
knowledge. These evaluation sources provide an environment to maximize volitional
competency and commit to changes in behavior. Since the end of formal evaluation period, the
POS modules have been requested for new volunteer onboarding. In addition, membership
onboarding will include the online payment portal site to support. Continued use of the websites
provides additional data to measure learning effectiveness and incorporate updates as needed.
YACHT CLUB SYSTEM TRAINING 6
Yacht Club System Training The Vallejo Yacht Club (VYC) established in 1900, sits on the east side of the Napa
River, facing Mare Island to the west. Within its harbor walls, the VYC membership enjoys a
history filled clubhouse, outfitted with a bar, galley and ship’s store as well as a floating harbor,
with berthing for about 195 boats. For the past decade, club operations involve management of a
million-dollar budget resulting in consistent profits to support clubhouse and harbor operations.
Sustaining efficiencies in VYC operations and financial reporting ensures the elected BOD and
flag officers execute their fiduciary responsibility as entrusted by the club membership.
In 2017, the BOD approved implementation of an integrated financial system,
encompassing POS, online payment portal and back-office functions (e.g. accounts
receivables/payables, bank reconciliation, etc.). Several organizational goals influenced the
BOD’s decision to implement a new system, such as: 1) reduce credit card fees totaling over
$17K annually; 2) streamline financial workflows from POS to the back-office; 3) implement an
inventory control process, and 4) digitize paper-based workflows to improve efficiency. The CE
system includes features that address the goals prioritized by the BOD.
Problem Description VYC operations relies primarily on a group of member-elected officers/directors and one
full-time manager. Additional part-time paid staff and rotating volunteers assist with various
office and club functions. Financial transactions are generated across digital (e.g. email,
websites, local area network (LAN)) and non-digital (e.g. paper or verbal) platforms. The CE
system integrates financial transactions (e.g. POS, general ledger, member payments) created
across mobile and wired devices. Due to the system implementation, learning modules need to
sustain volunteer training as well as promote compliant use of the system.
Specific modules provide the necessary orientation and background needed to accurately
use the new integrated financial system. One core workflow includes an interface of POS
transactions into the back-office system. POS transactions occur at the bar, galley and ship’s
store when members and guests purchase beverages, food and retail items. The POS system
allows purchases to occur via cash, charge to member accounts (i.e. “chits”), and credit cards.
Volunteers staffing the POS areas need support to learn the new system and any subsequent
updates to the workflows, as needed.
YACHT CLUB SYSTEM TRAINING 7
Another core workflow is online payment of member monthly dues, berthing (if
applicable) and any services charged to the members’ account. The online payment portal
allows members to view their monthly statement and make payments using a credit card or
electronic debit from their bank account. Online payments may require creating an account with
third party vendors to process a payment via credit card or electronic debit. Member feedback
indicates this additional account setup can be confusing and may deter members from using the
online payment portal. New and existing members need support to efficiently navigate the
online payment portal, setup their online payment method and make payments using accurate
financial information (e.g. routing numbers and bank accounts, or credit cards).
The final core workflow encompasses the back-office functions, such as membership
maintenance, accounts receivable (A/R), accounts payable (A/P), bank reconciliation, and
general ledger reporting (e.g. income statements, balance sheets, etc.). These common
bookkeeping activities are fully integrated within the Club Management Application (CMA) of
the CE system. POS and member payments are displayed in CMA as these transactions occur
via the POS devices and the online payment portal. Although the CE system provides very
detailed procedure manuals, the VYC operates differently than larger CE clients. Office staff
and the BOD need support to learn the CE workflows customized for the VYC operational
workflows.
Target AudienceThe target audience within the VYC organizational structure includes: 1) new and
existing members, 2) member volunteers, and 3) back-office staff. The VYC membership roster
includes about 337 memberships with an approximately 500 individuals eligible to use club
services. (See Appendix A) A group of 20-25 volunteers serve as officers, directors and/or staff in
the POS areas. A full-time manager oversees club operations and additional part-time employees
support bookkeeping functions, galley services and other duties as assigned.
Learner AnalysisMember information is initially gathered via a VYC membership application, including
email address. Members’ email addresses serve as a critical component to receive monthly
statements, weekly updates about club events and ongoing communication with the club office
staff. Monthly membership meetings provide feedback from members on various club services
YACHT CLUB SYSTEM TRAINING 8
and operational workflows. Ad hoc feedback to VYC officers and office staff provide additional
insight on how members and volunteers utilize and experience the new system.
The vast majority of members use some type of digital device, either wired or mobile,
including email. Members obtain and apply technology skills to support their membership in the
VYC. Door entry codes, member ID numbers, passwords, emails and the new online payment
portal require members to manage several digital components to access the clubhouse as well as
monitor communication with and payments to the club.
Some VYC officers and volunteers are familiar with the devices, software applications,
and digital use expectations needed to support the new CE system. Other
officers/volunteers/staff are either new to the environment or their technical skills require
additional support to learn the new system. The current club manager, with over a decade of
working at the VYC, possesses a broad range of technical skills needed for the various office
systems used within the VYC environment. The current part-time employee has specific
technical skills with basic accounting procedures, web-based systems, relational databases,
productivity suites and local area networks.
All learners possess a range of technology skills and prior knowledge pursuant to their
professional or individual experiences. Some learners are very adept and comfortable using
technology across various digital platforms. Based on their beliefs about technology, other
learners adopt new technology very sparingly and as such, embrace innovative technology very
slowly. Addressing motivation and volition is key to support adoption of the new system within
some learner groups.
Prerequisites specific to each learner group are based on the individual’s roles and
responsibilities at the VYC:
Members : 1) an email address, 2) experience with internet browsers, and 3)
knowledge of personal financial information to create an online payment.
Volunteers in POS areas : 1) knowledge of the VYC services and events, 2)
experience with a touch screen, and 3) ability to reconcile a cash drawer.
Office staff : 1) knowledge of VYC services and bylaws, 2) experience with
financial systems, 3) an intermediate knowledge of common business software
applications, and 4) demonstrated ability to learn and adapt to new technology.
YACHT CLUB SYSTEM TRAINING 9
Context AnalysisThe project’s design derives from: 1) membership data, 2) rejected or erroneous online
payments, 3) POS variances and 4) management’s expectation to sustain and improve financial
workflows. Although several members utilize the payment portal successfully, some members
experienced repetitive payments, ACH rejections or insufficient funds. Members onboarding
monthly require guidance on how to make a payment online to prevent errors and avoid
unnecessary merchant service fees.
When visiting the club, members primarily use cash or chits to pay for services. Credit
cards are a smaller portion of financial transactions and are often used by guests that are visiting
the club (as they cannot charge services to their member accounts) or by members for infrequent,
larger purchases. Several members use online banking to pay their current monthly statement.
Other members pay their monthly balances using cash or checks.
The Ship’s store uses the POS system consistently, whereas Bar POS usage is
approximately 2 times per week. The Galley POS system is rarely used. After initial
synchronous training was provided, Galley POS usage declined due to volunteer turnover or lack
of motivation. Quarterly, volunteers or officers onboarding necessitates new training or updated
training for a specific club event.
Related to staff use of the CMA system, entry of “chits” can be delayed which impacts
accuracy of member statements. Missed or late posting of member online payments also erodes
the integrity of member statements. Timely posting of POS batches with accurate cash
accounting reduces variances in financial reporting.
These experiences and incidents impact a broad range of operational areas at the VYC.
POS variances and system downtimes increase when volunteers are not properly trained.
Member satisfaction is negatively impacted due to varying level of volunteer competency with
the POS systems when attending club events. Finally, the need for more financial reporting to
ensure appropriate decision support for management.
Learners are located at the VYC as well as at their residences and businesses. Learning
options occur synchronously or asynchronously. Considerations for synchronous training to
occur at the VYC include: a digital device (either mobile or wired) with remote connection to
the CE system, workflow procedures, note-taking tools and table/chairs, when needed.
YACHT CLUB SYSTEM TRAINING 10
Asynchronous training requires an internet connection to access the instructional content on a
website containing various documents, videos and surveys organized by role and responsibility.
Current StateMembership data indicates about 25 members do not have an email address and continue
to request updates via phone or in person from office staff about their accounts. Approximately
216 members with gross deposits of over $370,000 have completed a payment online using the
new online payment portal. Merchant service fees total $1610 with $430 resulting from return
transactions. (See Appendix A)
Manual POS entry occurs primarily with the Galley and Bar POS batches. The Ship’s
Store reconciles exactly to all settlement types and does not experience the same variances as
other areas in the club. Data entry for member charges (i.e. “chits”) requires about four to ten
hours of staff time per week. Two to three POS batches need reconciliation per week. Alternate
staff are not trained in these functions to mitigate delays in financial workflows.
Environmental ScanThe VYC business environment indicates opportunities and challenges in the short- and
long-term. Opportunities to increase operational efficiencies and reduce costs will help address
the harbor management challenges that exist in the current economic climate. Although revenue
and profits continue to sustain from membership and harbor income, a focus on operational
efficiencies is needed to ensure expenses remain flat or decrease when possible, while improving
services to generate new as well as retain existing income.
Industry Trends The recreational marine industry is experiencing several industry pressures ranging from
decreased interest in boat ownership and increased costs for marina management. (MRA, 2013)
In addition, boaters in the Bay Area are experiencing increased costs for berthing (M. Ceryes,
Personal Communication, November 2017) and travel to/from various locations with water
access. Despite these negative trends, an increase in participation in boating related activities
(MRA, 2013) as well as club social events continues to bring new members to join the VYC.
In addition, several clubs no longer accommodate cash purchases or member chits. (M.
Ceryes, personal communication, November 2017) Instead, members provide a credit card, so all
YACHT CLUB SYSTEM TRAINING 11
club purchases are automatically charged directly to their credit card on file. Streamlining
payment options decreases accounting workflows by eliminating the need to manage cash and
member chits. An integrated system that manages front-to-back office workflow provides a
foundation for the VYC to implement more efficient accounting workflows in the future.
When researching which system to purchase, the VYC found several Bay Area clubs that
also use the CE system. A site visit to the Corinthian Yacht Club provided insights on the
benefits and risks of implementing the same system at the VYC (M. Ceryes, personal
communication, November 2017). A key feature of the CE system is the ability to integrate
many functional areas - membership, POS, inventory and the back-office. Due to limited staff
resources as well as the turnover that occurs within the volunteer groups, the VYC determined a
fully integrated system, such as CE, would benefit its operational needs. Maximizing operational
efficiencies also addresses the long-term challenges with rising harbor management costs and
intense competition to retain club memberships and sustain berthing income.
Solution DescriptionThis project’s overarching goal is to provide sustainable training resources that allows
members, volunteers and staff to use the new system efficiently and effectively. Outcomes to
measure achievement of this goal include:
Increase in members successfully using the online payment portal
Increase use of the POS system during club events
Decrease in variances when using the POS system
Increase of staff and officers posting POS batches, member payments and chits.
ObjectivesThe following objectives are aligned with the project’s goals:
1) Using written procedures and/or software simulations, members will be able to view
their statements online and make a payment using a credit card or electronic debit
transaction.
2) Using written procedures and/or software simulations, volunteers will be able to:
a. Enter POS transactions for the bar, galley or ship’s store and accept payments
via cash, credit card or member chits.
YACHT CLUB SYSTEM TRAINING 12
b. Reconcile the POS shift summary and accurately document any cash
overages/shortages.
3) Using written procedures and/or software simulations, officers and staff will be able
to:
a. Post POS batches with appropriate notes and cash adjustments.
b. Enter and post member payments using cash/check.
c. Enter member chits to charge member accounts and ensure accurate monthly
statements.
Learning SolutionTo address the performance gaps in efficient and effective use of the new CE system, the
learning solution includes:
Written procedures and/or software simulations for online payments, POS
transactions and back-office procedures.
Feedback from practice quizzes, post-tests, self-assessments using online survey tools
and variances identified in financial reports.
A web-based knowledge hub to provide consistent access to procedures, simulations
and feedback tools for continuous, ongoing improvement.
Task AnalysisBased role and responsibilities, learners display mastery of the objectives by accurately
completing the following tasks:
1) Members: View their statement online to find their monthly balance and make a
payment for the balance due via credit card or electronic debit.
2) Volunteers in POS Areas:
a. Create a “quick ticket” for a food, beverage or retail item and process a
payment via cash, member chit or credit card.
b. Print an “end of shift” summary and reconcile cash, chits and credit card
payments to the shift summary.
3) Officers and Office Staff:
a. Reconcile POS batches to “end of shift” summary reports and post to the
General Ledger with appropriate notes and adjustments, as needed.
YACHT CLUB SYSTEM TRAINING 13
b. Reconcile member payments to bank deposits and post to the General Ledger.
c. Enter a “chit” as a member charge and select the appropriate items to bill to
the members’ account.
Media and Delivery System DecisionsThe primary delivery formats are instructor led, online and self-study. Email
communication with embedded links to the online payment portal along with written procedures
and a simulation video will be sent as members join the VYC. In addition, monthly membership
events provide an opportunity for synchronous delivery of content to members. For volunteers
in the POS areas, the POS training portal may be provided prior to synchronous training.
Training for officers and office-staff occurs via synchronous and asynchronous sessions as well.
A variety of media was created and deployed to meet adult learners needs as well as the
roles and responsibilities within the VYC. Screencasts in Camtasia allow passive viewing as
well as interactive feedback. Software simulations in Captivate serve as practice to gain
competency with the POS features. Website development occurred in Dreamweaver using
JavaScript and jQuery components. Images and logos were edited in Photoshop incorporate
visual graphics. Videos are stored unlisted within YouTube to mitigate unnecessary access to
member related protocols.
Learning Theories
The primary learning theories applicable to this project are:
Adult Learning Theory (i.e. andragogy), which encompasses self-directed
learning (SDL);
Cognitivism:
o Cognitive Information Processing (CIP), as aligned with Gagne’s nine
events of instruction; and
o Schema Theory
Constructivism
YACHT CLUB SYSTEM TRAINING 14
As stated in Merriam’s (2003) article on andragogy and SDL, adult learners require a
different approach to learning. The “five assumptions underlying andragogy” applicable to the
adult learners served by this project are: 1) able to direct their own learning; 2) possess several
life experiences to relate to their learning; 3) learning needs relate to social roles; 4) problem
centered that allows immediate application of the knowledge; and 5) intrinsic motivation.
(Merriam, 2003, p. 5) Learning modules organized by role that are learner paced align well with
andragogy and its SDL components.
Cognitivism, as outlined in Harasim’s Learning Theory and Online Technology (2017),
provided additional guidance on the design of this project. Learners process varying levels of
information for retrieval and application to specific situations as needed. Schema theory
indicates “learning is easier if new subject matter is compared to existing knowledge…”.
(Harasim, Location 1382 of 6204, ebook) Activities such as paying a bill, purchasing retail
services or depositing money into an account are common financial tasks that the learner can
relate to the content included in the modules. Incorporating prerequisites into the evaluation
tools, such as ability to use a touch screen, allows learners to reflect on their current skills or
experiences needed for successful use of the system.
Gagne’s nine events of instruction create a structure for the learning strategies that can
also be utilized during development of the learning modules. Pollock, Jefferson and Wick’s The
Six Disciplines of Breakthrough Learning: How to Turn Training and Development into
Business Results (2015, p.114) cross maps Gagne’s nine events to the corporate learning
environment and ensures development focused on the adult learning audience. Demonstration
videos gain learner attention and are organized into smaller topics, to break down the larger
workflow. Interactive questions and worked examples guide learners with practice options and
knowledge checks while encouraging repetition for increased mastery. Step by step procedures
serve as job aids to support learners transfer skills and change behaviors.
Finally, constructivism’s key principles of: 1) active learning, 2) learning-by-doing, 3)
scaffolded learning, and 4) collaborative learning (Harasim, Location 1879 of 6204, ebook)
provided more direction to design the activities of the learning modules. Learners complete
specific tasks during or immediately after completing the modules, which requires the learner to
“participate and act…rather than learn passively.” (Harasim, Location 1879 of 6204, ebook)
Simulations, case scenarios and problem-based learning (PBL) (Harasim, Location 1950 of 6204,
YACHT CLUB SYSTEM TRAINING 15
ebook) incorporated into the modules provides an opportunity to learn by doing the specific tasks
learners will encounter in their roles at the club. Website resources can be access while
proficient members provide synchronous support to other learners during group collaborations,
such as scheduled meetings, as needed.
Instructional Principles and StrategiesMerrill’s “First Principles of Instruction” (2002) offer more guidance to project
development. Problem-centered learning modules demonstrate specific tasks and engage the
learner by practicing or completing the actual task presented in the content. Connecting new
knowledge to existing knowledge supports cognitivism’s schema theory. Software simulations
demonstrate the specific situations the learner will experience. Practice opportunities allow
learners to apply their knowledge. When a learner completes the task in real-time, such as an
online payment or a POS transaction, they are transferring their new skills to their everyday life.
(Merrill, 2002)
The design of the learning modules also aligns with five-step model described in
Stolovitch and Keeps’ Telling Ain’t Training (2011). Rationale, learning objectives, activities
and feedback are key components to support adult learners. In addition, learner readiness can be
measured through a self-assessment or whether the learner meets the prerequisites for each
module. Learner autonomy is a vital part of engaging and motivating the target audience, so
learners may navigate videos and website tabs at their own pace. As part of the constructivist
perspective, applying the new knowledge immediately in practice modules or a quiz also support
learners using feedback opportunities.
Motivational elements are critical to ensure learners participate in and apply the
knowledge gained from the training. Kellers’ ARCs motivational model (Pappas, 2015)
presented many options to address the conditions outlined in the learner analysis. Software
simulations, group interactions and job aids attract learner attention via active participation, use
of a variety of media and demonstration of real-world examples. The modules are relevant
activities that learners can connect to prior experiences and are useful in solving authentic
problems. Prerequisites, feedback and self-paced learning builds learner confidence, which
increases learner motivation.
YACHT CLUB SYSTEM TRAINING 16
Multimedia Principles and StrategiesBased on Clark and Mayer’s e-Learning and the Science of Instruction (2016), several
multimedia principles and strategies were applied to the learning modules due to the types of
media and delivery of content. In addition, since learner knowledge varies from novice to
expert, additional considerations influenced the development of the multimedia content.
1) Multimedia Principle: Media with both words and graphics specifically support
novice learners. Learner feedback indicates positive interest in visual elements to
support attention and sustain motivation. Decorative graphics are used primarily
within webpages for visual appeal. Topic organizers, screencasts and software
simulations use graphics to organize content while encouraging learner interaction.
(Clark, Location 2044 of 10199, ebook)
2) Contiguity Principle 1 and 2: In written procedures, words are placed near the
relevant graphic to ensure efficient cognitive processing. In videos with audio,
narration occurs at the same time as signaling occurs. (Clark, Chapter 5)
3) Modality Principle: For software simulations, audio narration helps minimize on-
screen text. (Clark, Location 2776 of 10199, ebook)
4) Redundancy Principle: Procedures and simulations adheres to the redundancy
principle, which avoids presenting words in audio and text, unless “boundary
conditions” exist (e.g. no visuals are present, learner paced, technical/unfamiliar
words and text is shorter than narration) (Clark, Location 3049 of 10199, ebook)
5) Coherence Principle: By applying a “less is more” strategy (Clark, Location 3475 of
10199, ebook), media avoids extraneous words, unnecessary topics and complex
graphics to ensure the learning modules are well organized, uncluttered and effective.
6) Personalization and Embodiment Principle: During software simulations, a human
voice and on-screen agent are employed to support deeper learning. Using
personalized text and polite speech while avoiding passive voice helps sustain a
conversational style during the narration. (Clark, Location 3878 and 3932 of 10199,
ebook)
7) Segmenting and Pretraining Principle: Learning modules are segmented by role and
responsibility. As the complexity of certain tasks increases, lessons are segmented
further to “break a complex lesson into smaller parts”. (Clark, Location 4293 of
YACHT CLUB SYSTEM TRAINING 17
10199, ebook) In addition, learners will receive materials before training occurs as
well as an outline on how to navigate the lessons.
8) Learner Engagement: Practice exercises and quizzes provide learners the opportunity
to engage with the lesson materials. As learners become experts in using the CE
system features, they will teach other members to use the system. Signaling
techniques also direct the learner’s attention to relevant content, terms and activities.
9) Worked Examples and Practice: Learners view software simulations of the
segmented lessons, which provides step-by-step examples of each workflow. For
more complex lessons, such as POS transactions, practice elements include
appropriate feedback to leverage deeper learning.
Challenges and Breakthroughs
During the initial launch of the project, human resources were redirected to support
network maintenance due to downtimes occurring with the POS devices. As such, some
multimedia for the CMA module is not as well defined as other media elements and lacks
practice options. To address this deficiency, synchronous training using the CMA website to
reference procedures strengthens the training delivery method with interactive practice directly
into an office workstation.
Accessing the CE system to create the software simulation components of the solution
was resolved using a virtual private network (VPN) license. Successful remote network
connectivity within the CE system allowed software simulation recording to occur outside the
office on a computer with recording software. Syncing narration, signaling and video recordings
required extra time and attention to audio levels, close captioning and timing of elements.
Topics organized into smaller segments along with a watch/try/assess format aligned well
with both Camtasia and Captivate media tools. Adding questions into Camtasia video
demonstrations allowed an additional level of learner interaction. Using Captivate simulation
features allows efficient creation of practice and assessment options with minimal impact to
production timelines. Reuse of website layouts using .css files from prior projects streamlines
coding within Dreamweaver. These development decisions supported meeting the milestones on
the project’s timeline.
YACHT CLUB SYSTEM TRAINING 18
Finally, additional technology requests were prioritized to allow completion of the
training modules, which rounds out the implementation for phase one of the CE system. In
addition, department managers of the store and bar already requested additional volunteer
training with the websites. Continued use of the training modules develops opportunities to
improve consistent outcomes and update content.
Methods/Procedures
Major DeliverablesList of major deliverables include website content organized by role/function for: 1) the
online payment portal; 2) POS transactions; 3) the back-office procedures for POS, member
payments and chits. Scripts and handouts were created in word processing software along with
screen shots or other images as needed. Software simulations produced in Camtasia include
signaling and questions with feedback. Captivate allows learners to practice and test workflow
procedures. Images edited in Photoshop provide continuity in graphic design of the website.
Website development in Dreamweaver provides the delivery platform, branching structure as
well as interactive elements using JavaScript and jQuery. Reference materials, such as CE
manuals, are linked for digital distribution. In addition, evaluation tools on usability, pre/post
learner readiness and knowledge transfer obtain feedback on the learner experience.
Design Analysis Characteristics of the Human Performance Technology model for Change Management
(Reiser, page 127, 2018) as well as the ADDIE system design model form the foundation to
analyze the project’s design. Organizational goals cascade with learning objectives to help
achieve business results. Iterative evaluation cycles with the client occurred during each design
phase to confirm topic content, media elements and feedback. VYC management harvested
member experiences to guide which topics are covered in the training as well as the frequently
asked questions. Capstone adviser(s) recommended prioritizing case studies and interactivity
features to enhance active learning. Based on the learner analysis, design elements were
organized by role, so learners can build their knowledge based on their level of involvement at
the VYC.
YACHT CLUB SYSTEM TRAINING 19
Agenda.
1) Online payment portal (up to 15 mins – online; at least 15 mins - in person)
a. Send website link and instructions to member for online learning.
b. Open website link during in person training to review with member.
2) POS transactions (up to 25 mins – online; plus 15 mins - in person)
a. Send website link to volunteer for practice before first POS shift
b. Observe volunteer complete POS transaction upon arrival at first shift
3) Back-office procedures (at least 35 mins – preview and in person)
a. Send website link to staff or officer for orientation before first shift.
b. Open website link and CE system to conduct in person training on CMA
modules.
Activity Descriptions.
Agenda Item Topic Description
Online Payment
Portal How to view your
monthly statement
How to make a payment using a bank account
How to make a payment using a credit card
Learners view a software simulation
video, review instructions with
screenshots or navigate through key
concepts.
For synchronous training, learners listen
to an SME and may practice using the
online payment portal.
POS Transaction How to Login
How to create a ticket in a POS system
How to accept a payment
How to end a shift
Learners view demonstration videos
with feedback, interactive simulations to
practice, review instructions with
images, and complete survey/quizzes.
For synchronous training, learners listen
to an SME and demonstrate proficiency
using the POS system.
Back-office
Procedures How to login
How to post POS batches;
Learners review written instructions
embedded with images of key buttons
YACHT CLUB SYSTEM TRAINING 20
Agenda Item Topic Description
How to enter member chits;
How to post member payments
and screenshots.
For synchronous training, learners listen
to an SME and practice using the CMA
system.
All Agenda Items Intro and Feedback surveys for learner reflection
Quiz tabs for knowledge transfer
Learners complete Intro, Feedback and
Quiz forms to assess learner readiness
and complete knowledge checks.
Implementation PlanSynchronous training requires an SME, already trained on the CE system, that is
knowledgeable about VYC membership rules, is familiar with the financial workflows, and role
models an early adoption of the new system for their personal membership payments and
position in the VYC organization. SMEs complete training on specific topics as needed to teach
other learners. Office staff manage access to various modules, performance aids and evaluation
tools to support the learning activities of the website.
ResourcesSeveral resources fully supported this project, including: subject matter experts, users,
CE data, digital devices, software and external expertise. Specific roles and resources were:
Subject Matter Experts : Officers, Directors, Manager
Users : Members, Volunteers, Staff, Officers and Directors
Data : Online Payments, POS transactions, Financial Reports from CE and ACH sources
Devices : Desktop/Laptop, Tablets, Microphone, Headphones or speakers, Digital
Camera
Software : CE System, Camtasia, Captivate, Dreamweaver, Photoshop, Microsoft Word,
Google Forms
Space : Within the VYC at POS stations or conference rooms
External Expertise : CE staff, such as the Project Manager, Implementation Specialist, or
Technical Support
YACHT CLUB SYSTEM TRAINING 21
Images : Personal photos, VYC logo, VYC burgee
Other : Prior MIST coursework, reading materials and textbooks
Logistics and BudgetSoftware simulations were produced using a VPN account to access the CE system from
a desktop computer installed with screen recording software. Procedure documentation was
drafted with word processing software. Feedback is obtained from embedded practice quizzes or
Google forms. Synchronous training occurs within the VYC and requires meeting space with
tables, chairs and digital devices for demonstrations. Asynchronous training may occur in or
outside the VYC. All training needs an internet connection. No additional expenses are
expected, unless external technical support is needed.
Technical Skills and External ResourcesThe technical skills needed for this project include: 1) knowledge of recording software
with editing features, 2) ability to create websites, and 3) familiarity with Google forms.
Additional technical support related to the CE system will be requested through the CE customer
help desk. Any technical issues (e.g. VPN, LAN) may be requested through the VYC’s
independent contractor for information technology (IT) support.
Timeline/Progress ReportThe project’s timeline spanned four months, from September 2018 through December
2018. To complete the project on time and within scope, the developer, stakeholders and
sponsors needed to sustain priority status for the project and engage member support for the
project. Any budget approvals for external expertise and additional technical support required
timely turnaround.
Milestones ChecklistTimeline Milestone Comments
September 25, 2018 Project Proposal
Storyboards
Outlines design elements to
guide development;
Share with client for
approval of scope and
YACHT CLUB SYSTEM TRAINING 22
Timeline Milestone Commentsdeliverables.
October 23, 2018 Develop software simulations and
written procedures
Create draft sample
materials to share with
client for approval
November 18, 2018 Complete media production and
website programming
Adjusted deliverables for
CMA module to written
procedures with
synchronous training.
November 20, 2018 Finish development and usability
tests with learners
Revised and modified
modules based on feedback
and results.
December 4, 2018 Finish summative evaluation with
learners
Organize data and perform
statistical analysis on results
December 11, 2018 Implementation:
Final Materials
Final Report
Share final materials with
client
After December 2018 Continued Evaluation:
Learner surveys Assessments Variances and/or downtime Decrease in fees, variances
and expenses
Share feedback with client
Evaluation PlansBoth formative and summative evaluations occurred to determine the usability of the
learning solution as well as its effectiveness in improving learner knowledge about the new CE
system. Formative evaluation was completed prior to the final release of the learning modules
for pilot testing. To ensure an optimal user experience, remedies were developed related to
erroneous content, navigation challenges or technical/logistical issues when delivering the
instruction. Summative evaluation occurred after implementing the learning modules for
YACHT CLUB SYSTEM TRAINING 23
additional testing. Additional metrics related to behaviors and results may also guide future
development and utilization of the learning modules.
Formative Evaluation Usability testing was conducted following a “single subject testing” (SST) methodology
using in person observations. (See Appendix B) One of three users participated in each SST
cycle and revisions were developed for the websites prior to the next test cycle. A final test
concluded the SST and confirmed revisions approved during prior SST cycles were included.
Participants reflected the target audience’s variation in prerequisites and experience with the CE
system.
Additional feedback collected using Google forms indicates instructions and navigations
are easy to follow. (See Figure 1) Narration, graphics and video received high marks, whereas
audio quality and engaging lesson design needs further development. Some learners shared that
the audio was difficult to hear in some sections of the video media.
Only a slim majority of learners “strongly agree” on the module’s overall educational
value, while other feedback encourages another look at the module’s content or delivery
platforms to improve learner engagement and the module’s overall value. Feedback included
embedding additional images and screenshots in the CMA modules to help learners visualize
connections from the procedures to the system features.
The lesson instructions are easy to understand.
The navigation instructions are easy to follow.
Navigating the module was easy.
The lesson design is engaging.
Graphics and video are informative.
Narration adds to the learning experience.
Audio quality is clear.
Overall lesson has educational value.
0 1 2 3 4 5 6
Yacht Club Training Usability Results
Strongly Disagree Disagree NeutralAgree Strongly Agree
Figure 1
YACHT CLUB SYSTEM TRAINING 24
Summative Evaluation Evaluating the learning modules effectiveness are based on the four levels outlined in
Kirkpatrick and Kirkpatrick’s Evaluating Training Programs (2006). Twelve evaluation tools
were created to gather data from learners as follows:
1) Introduction and Feedback (2 per website module = 6 total): These surveys collected
pre and post assessments to monitor learner readiness to complete learning objectives.
2) Quizzes (1 for the online payment and CMA modules; 4 for POS module = 6 total):
These quizzes assessed learner knowledge about features and applying system
features to case scenarios.
Level I —Reaction. A self-assessment of learning objectives based on ratings was
completed by six participants, translated into ordinal values and analyzed as the difference of
paired samples. (See Appendix C) The null hypothesis states the difference between the paired
sample measures is negative, which represents no improvement in the learner’s assessment of
skills learned during the modules. Since the Alpha (.05 significance level) is less than the p-
value, the null hypothesis is not rejected and concludes the differences in the pre and post
assessment values are not statistically significant to support learner readiness in completing the
learning objectives.
Level II—Learning. All quiz scores were analyzed together to calculate an average
score for all quizzes. Thirteen (13) quiz results were included in the test of a single sample
mean. (See Appendix D) The established criteria for a “passing” result is to achieve correct
answers on 4 of 5 questions or 80%. The null hypothesis states the average learner score is less
than or equal to 3.99. Since the average mean of the sample is 3.92 and the p-value (0.61) is
greater than the alpha (p<.05), the null hypothesis is not rejected and concludes the sample data
does not provide sufficient evidence that the alternate hypothesis (average score > 3.99) may be
true.
Level III—Behavior. The following metrics as of December 2018 also monitor usage
of the new system:
216 members using the online payment portal. (See Appendix A)
Zero POS device usage for Galley services.
YACHT CLUB SYSTEM TRAINING 25
Delayed processing of POS batches, member payments and member chits beyond
3 business days.
Level IV—Results. The BOD may consider additional metrics to assess future business
impacts and ROI such as:
Reduction in annual merchant services fees
Reduction in cash adjustments and variances in financial reports
Club operational expenses remain flat or decrease due to improved efficiencies.
ConclusionDespite positive reactions to usability and test scores, statistical analysis lacks convincing
evidence to determine the learning modules significantly improve learner scores or perceived
readiness to complete learning objectives. Several learners in the sample group possess some
familiarity or actual experience with the new system, which would influence how learner
readiness is impacted within this sample group. In addition, an extreme range of learner
prerequisites exists which may influence the inconclusive analysis of the assessment ratings and
test results. Further, the complexity of applying the appropriate statistical tests also factors into
the analysis and interpretation of the summative results.
The project’s design involves several multimedia options as well as evaluation tools. As
learners navigate the websites, they select the media per their direction. As such, not all learners
completed all evaluation tools as originally expected. Over twelve evaluation tools frame the
feedback and knowledge check options for learner interaction. Well organized, functional
navigation across websites resulted in positive usability and ease of use. Yet, engaging lessons
and audio quality are areas that need further review to continue enhancing the learner experience.
Design and development requirements heightened the need for additional support,
especially in media production. Mastery of media as well as web development, instructional
design, evaluation instruments and project management distract from the creative skills required
for truly engaging media development. A team environment would benefit this project to
address feedback related to engaging lessons as well as audio quality.
After updating additional media elements, additional volunteers will participate in using
the training modules to prepare for POS shifts. Clarifying instructions to navigate the
YACHT CLUB SYSTEM TRAINING 26
introduction, feedback and quiz elements of the module may help learners provide more
feedback to analyze effectiveness. Additional talent on the project may help produce improved
media design elements. Multiple, diverse perspectives ensure important facets of instructional
design and technology are intertwined effectively to maximize engagement and improve learner
performance.
YACHT CLUB SYSTEM TRAINING 27
ReferencesClark, R., & Mayer, R. (2016). e-Learning and the Science of Instruction: Proven Guidelines for
Consumers and Designers of Multimedia Learning. Hoboken, NJ: John Wiley & Sons,
Inc. ISBN: 9781119239086
Evans, J.M. (2000). Vallejo Yacht Club: The First 100 Years. Vallejo, CA: Vallejo Yacht Club,
Inc.
Harasim, L., (2017) Learning Theory and Online Technologies. 2nd Edition. New York, NY:
Routledge.
Kirkpatrick, D.L. and Kirkpatrick, J.D. (2006). Evaluating Training Programs: The Four Levels.
Oakland, CA: Berrett-Koehler Publishers, Inc.
Marina Recreation Association (MRA) (2013). Annual Conference. Santa Barbara, CA
Merriam, S.B., (2003). Andragogy and Self-Directed Learning: Pillars of Adult Learning
Theory. New Directions of Adult and Continuing Education, No. 89. Jossey-Bass, A
Publishing Unit of John Wiley & Sons, Inc.
Merrill, M.D., (2002). First Principles of Instruction. ETR&D, Vol. 50, No. 3. Retrieved from
firstprinciplesbymerrill.pdf
Pappas, C., (2015). Instructional Design Model and Theories: Keller’s ARCS Model of
Motivation. Retrieved from https://elearningindustry.com/arcs-model-of-motivation
Pollock, R.V.H., Jefferson, A.M., and Wick, C.W., (2015). The Six Disciplines of Breakthrough
Learning: How to Turn Training and Development into Business Results. 3rd Edition.
Hoboken, NJ: John Wiley & Sons, Inc.
Reiser, R.A., & Dewey, J.V., (2018). Trends and Issues in Instructional Design and Technology
4th Edition. New York, NY: Pearson.
Stolovitch, H.D., and Keeps, E.J., (2011). Telling Ain’t Training. Alexandria, VA: American
Society for Training & Development.
YACHT CLUB SYSTEM TRAINING 28
Appendices
Appendix A: VYC Membership and ACH Reports
Appendix B: Single Subject Testing (SST) Instrument
Appendix C: Level I – Reaction on Learning Objectives
Appendix D: Level II – Learning Analysis using Quiz Results
VYC Membership Data 12062018.pdf
VYC ACH Data 12062018.pdf
VYC ACH Customers 12062018.pdf
SST Test Instrument VYC Portals 111718.pdf
VYC Learner Objectives Stat Test 120918.pdf
VYC Quiz Stat Test 120918.pdf
top related