solving the osp reporting conundrum
TRANSCRIPT
![Page 1: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/1.jpg)
Solving the OSP Reporting Conundrum
Lynn Ward, Principal Systems AnalystIndiana University
![Page 2: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/2.jpg)
Indiana University
• 7 Campuses• 2 large research
campuses (Bloomington and IUPUI)
• 5 smaller regional campuses (East, Fort Wayne, Kokomo, Northwest, South Bend, Southeast)
• Stats• > 1K degree programs• > 5K FT, 2K PT Faculty• > 11K staff• > 100K students
• Sakai is our only CMS/LMS
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 2
![Page 3: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/3.jpg)
ePortfolio Timeline @ IU
• 1998: IUPUI campus adopts a competency or ability-based approach to general education
• 2000: IUPUI campus initiates Eport project, partnering with Ali Jafari (Epsilen)
• 2003: IU and rSmart partnership on OSP 2.0 Mellon Grant; IUPUI abandons first homegrown tool
• 2004-05: early pilots with OSP• 2006: moratorium on new portfolio projects• 2007-present: growing interest and use on all
campuses
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 3
![Page 4: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/4.jpg)
OSP Use @IU
• > 20 active portfolio projects underway, many in professional schools
• Majority focus on outcomes assessment at the program level
• Highly individualized approach to assessment
• Individual differences accommodated via customized scaffoldings and forms
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 4
![Page 5: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/5.jpg)
OSP Support @ IU
• .5 – 2 FTE developers• .10-20 FTE designer• .25 FTE combined consultant, analyst, tech
writer, trainer, QA specialist• Outsource some QA and development
work• Growing knowledge among consultants in
campus teaching centers
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 5
![Page 6: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/6.jpg)
The Conundrum
• Project coordinators need to aggregate, analyze, and summarize evaluation (and other types of) data
• Evaluation forms differ widely in terms of • # of rating fields• # rating scale • # labels assigned to each point on scale
• XML Programmer required to develop custom reports for each project – DOES NOT SCALE
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 6
![Page 7: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/7.jpg)
Our Solution
• Define local form standards • Flexible• Predictable
• Publish and promote standards• Define report definitions based on those
standards• Reports “discover” the XSD structure on
the fly and render the results
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 7
![Page 8: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/8.jpg)
Evaluation Form Standard• Field name conventions:• Rating fields (rating, rating 1, rating2, rating3 …
rating20)• Comments (comments)
• Rating fields must be represented as a selection field (drop-down menu)
• All rating fields in the form (except rating) must use the same rating scale.
• Enumeration values must be a number, “n/a”, or null
• Rating fields may not be repeatable
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 8
![Page 9: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/9.jpg)
Customization Options
• Rating fields and comments fields may use any display name
• Rating scales may have any number of points• The display label for each point on the scale
may be any valid string• Evaluation forms may have any valid name• Each matrix cell may use a different evaluation
form.
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 9
![Page 10: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/10.jpg)
SAMPLE REPORTS
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 10
![Page 11: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/11.jpg)
10th Sakai Conference - Boston, MA, U.S.A. 11
Rating Summary
![Page 12: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/12.jpg)
Rating Details
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 12
![Page 13: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/13.jpg)
Interactive Report – Main (Matrix) View
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 13
Click to see summary report
for cell
![Page 14: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/14.jpg)
Interactive Report – Summary View
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 14
Click to see detailed report for
cell
Click to return to main (matrix)
view.
![Page 15: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/15.jpg)
Interactive Report – Details View
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 15
Click to return to summary view
Click to return to main (matrix) view.
Click to open cell in new window.
![Page 16: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/16.jpg)
Matrix Cell Status Summary
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 16
![Page 17: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/17.jpg)
Matrix Cell Status Details
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 17
![Page 18: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/18.jpg)
Matrix Link Summary
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 18
![Page 19: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/19.jpg)
Next Steps
• Publish form templates and documentation on how to customize
• Refine and augment existing reports based on user feedback.
• Merge Reports tool enhancements to trunk • Post report definitions to OSP library so other
institutions can use them.
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 19
![Page 20: Solving The OSP Reporting Conundrum](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a6202b4c9054b328b5488/html5/thumbnails/20.jpg)
For More Information
• Contact…
– Lynn Ward ([email protected])– Chris Maurer ([email protected])– Bryan Holladay ([email protected])
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 20