Is Sakai Always this Slow?

University of DaytonJune 16, 2010

University of Dayton

11th Sakai Conference - June 15-17, 2010

• Medium-Large Private School• Around 900 full/part-time faculty• 7400 undergraduate students• 3500 graduate students• About 500 faculty use Sakai

• Catholic Marianist Institution• Founded in 1850• Top ten US Catholic universities• Long basketball tradition

NIT Champions!

11th Sakai Conference - June 15-17, 2010

• 2010, our third NIT championship!

• 1903, our first team

Transitioning to Sakai

• On WebCT CE 4.1 for 5+ years• Product was end-of-lifed• It worked OK but Faculty found it difficult to use• Chose Sakai because

o Flexibility of open source solutiono Costs kept going up and up

• Branded Isidore

11th Sakai Conference - June 15-17, 2010

Our Objectives Today

1. Share a generic roadmap for performance troubleshooting2. Present our findings

11th Sakai Conference - June 15-17, 2010

Is Sakai really that slow?

11th Sakai Conference - June 15-17, 2010

The First Signs

• Reports of performance issues trickled in

11th Sakai Conference - June 15-17, 2010

• Initially dismissed/ignored claims o Pilot group hadn’t reported slownesso Not fair to compare Sakai to WebCTo Certain tools slow, but not the whole server right?

Our Application Environment

• Productiono Solariso Oracleo Files Stored in the DBo Tomcat and Apache

• Testing environment (exact copy of production)• Benchmark Site

11th Sakai Conference - June 15-17, 2010

The Speedy Environment

• Local developer’s installationso Linux (Ubuntu)o MySqlo Tomcato Walmart machine

• Matt finally saw the light!

OK, it really is that slow… But why?

11th Sakai Conference - June 15-17, 2010

What We Knew

• Independent of server load• JVM settings were good

11th Sakai Conference - June 15-17, 2010

Must be the application code…

11th Sakai Conference - June 15-17, 2010

Fixing One Thing at a Time

• Database queries• Excessive iterations

11th Sakai Conference - June 15-17, 2010

What’s the Community Say?

• Sakai Dev Groups• Conference in Boston

o Walked around with a laptopo No one else saw similar responses

11th Sakai Conference - June 15-17, 2010

The New Kid on the Block

Sakai vs. Things that are Trusted

YourKit - Java Profiler

Print Statements

Debugging the old fashioned way...

What We Knew

• Independent of server load• JVM settings were good• It's not the code• Well… it must be something in Isidore's

environment.

11th Sakai Conference - June 15-17, 2010

Check Everything Else…

11th Sakai Conference - June 15-17, 2010

jMeter Scripts

• jMeter - Open source Apache product• Not load testing• Tested 25 different environments

11th Sakai Conference - June 15-17, 2010

New Test Environments

• Kaylee: Sun Fire 280R• River: Sun Blade X6250 running Solaris• Simon: Sun Blade X6250 running Linux• Sakaitest:

o Copy of Isidore productiono Sun SPARC Enterprise T5220

11th Sakai Conference - June 15-17, 2010

• jMeter script load tools 100 times

It’s the Hardware!

11th Sakai Conference - June 15-17, 2010

Total Time to Completion

11th Sakai Conference - June 15-17, 2010

Requests > 1.5 Seconds

11th Sakai Conference - June 15-17, 2010

All Other Requests

11th Sakai Conference - June 15-17, 2010

Performance Relative to Sakaitest

11th Sakai Conference - June 15-17, 2010

Detailed Test Environments

• Kaylee: Sun Fire 280Ro An older SPARC server running Solaris 10o 2 - UltraSPARC III+ processors, 4 GB memory

• River: Sun Blade X6250o A newer x86 64-bit server running Solaris 10o 2 - Quad Core Intel Xeon processors, 16 GB memory

• Simon: Sun Blade X6250o A newer x86 64-bit server running Linuxo 2 - Quad Core Intel Xeon processors, 16 GB memory

• Sakaitest: Sun SPARC Enterprise T5220o Sun T2 series server running Solaris 10o 1 - Quad Core UltraSPARC T2 processor, 16 GB memory

11th Sakai Conference - June 15-17, 2010

Future Work

• Possibly switch hardware on database servers

• Continue monitor speed• Improve single tools

11th Sakai Conference - June 15-17, 2010

Questions?

11th Sakai Conference - June 15-17, 2010

http://www.surveymonkey.com/s/sakai10

11th Sakai Conference - June 15-17, 2010

Evaluation

Image BibliographyUD Logo - http://logo.udayton.edu/zoom.asp?whichpic=redbluep1Marianist Cross - http://portal.chaminade-stl.com/Portals/104/images/marianist%20cross.bmpFirst Basketball Team - http://digital.udayton.edu/u?/archives,6222010 NIT Champions - http://www.waitingfornextyear.com/wp-content/uploads/2010/04/Dayton-NIT-Champs.jpgIndian Star Tortoise - http://i481.photobucket.com/albums/rr176/848leanne/george%20feb%2009/georgefeb09004.jpgPenguin Team - http://www.kernel.org/pub/linux/kernel/people/paulmck/Confessions/Elephant_Team_03.jpgThe Matrix - http://images.psxextreme.com/wallpapers/ps3/the_matrix_has_you_770.jpgSpinning Wheel - http://upload.wikimedia.org/wikipedia/commons/e/e0/Irish_spinning_wheel.jpgStrike out - http://bmiller.mlblogs.com/strikeout.gifSerenity - http://www.superherostuff.com/blog/wp-content/uploads/2010/04/serenity-cast.jpgFireworks - http://www.visitdarkecounty.org/discover/enewsletter/MPj04100830000%5B1%5D.jpg Pointing - http://www.anzemc.com/p7bpassets/images/ANZEMC_FingerPoint_1.jpg

11th Sakai Conference - June 15-17, 2010