autograder presentation

Auto-Grading Parallel ProgramsBuilding plugins for WebCAT

Team Lead: Max GrossmanTeam Member: Maha Aziz, Anant Tibrewal, Anna Chi

● Background & Motivation

● Related Work

● What is WebCAT?

● Workflow

● Feature table

● Our Progress

● Future Plans

Overview

● Current grading method & Shorts

● A tool for testing and analyzing parallel programs automatically

○ Transparency of the grading process

○ Speed up and simplify the grading process

○ Focus on application performance

○ Possibility of parallel programming MOOC to be created

Background & Motivation

● Mooshak○ Intended for hosting coding competitions online○ Simple configuration, minimal requirements (apache, linux,

tcl)○ Not very pretty UI

● Marmoset○ Not very extensive Java support

Related Work

What is WebCAT?

from May.11th till now

● Research about multiple autograding systems

Related Work

● Target on WebCat

WebCAT

● Install and set up

● Hackathon plug-in structure

● Feature table

Workflow

Description Priority (1-5, 1 being the highest)

Viability (1-5, 1 being the highest)

Workload (1-5, 1 being the highest)

Assignee

Code styling/checkstyle grading 2 1 2 Anant

Local grading of HJ-lib programs running in a single thread for correctness

1 1 1 Anant

Remote grading of HJ-lib programs running on multiple cores in a compute cluster

2 3 1 Anna

needs to commit student file to svn before/after running the autograder

1 1 5 Anna

Feature Table




Assignee

give a way for student provided tests to be graded with feedback

3/4 once we have instructor-provided, this is a 2

3

instructor provided tests with limited feedback

1 1 1

Ranking of student submissions based on performance, make it visible to students to turn every homework into a competition (with names anonymized?)

4 4 2 Maha

Statistics on how many students are passing a given test, to give feedback on the relative difficulty of different tests (maybe part of the leaderboard?)

3 4 2

Feature Table




Assignee

Student peer reviews 5 5 1

Different types of static code analysis tools

2 once we have checkstyle, this should be easier (3)

2

NetID authentication 1 3 ? Maha

Feature Table

Architecture Students

WebCAT

STIC Leaderboard

HomeworkSubmission

Performance Test Results

Student View of Leaderboard

POST Request for New Submissions

SVN

● Anant

● Maha

● Anna

Our Progress


WebCAT

STIC Leaderboard

HomeworkSubmission




SVN

● Anant

○ Local grading of HJ-lib programs running in a single thread for

correctness

○ Built HJ program with JUnit tests to test WebCAT

■ correctness, performance tests

○ Incorporated the FindBugs static analysis tool to WebCAT

■ tool to detect bugs in code

Our Progress


WebCAT

STIC Leaderboard

HomeworkSubmission




SVN

● Maha

○ “Leaderboard”

○ created a database which contains all student submissions from

the svn

■ includes test names, execution times, core count, etc.

○ database is displayed to webpage for students to see a ranking

of how well their classmates are doing

■ all information on webpage is anonymous

Our Progress

Leaderboard


WebCAT

STIC Leaderboard

HomeworkSubmission




SVN

student-runs

Handled backup of students files using SVN

student ID

before grading

num of submission

after grading

student ‘s code

Results

Remote grading on multiple cores in cluster

WebCAT STIC(cluster)

Build folder structure

Run tests

Student code & Performance test

Send performance results back

Through admin interface

Slurm personal configuration

● Feature table

● Provide documentations for students and instructors

● EduHPC

Future Plans

● Current inconvenient grading process

● Research about autograders

● Good points and properties of WebCAT

● Feature Table

● Our progress

● Next steps

Summary

autograder presentation

Documents