moodle performance optimizations
Post on 26-May-2015
681 Views
Preview:
DESCRIPTION
TRANSCRIPT
WelcomeITU Innovators
IT KartelletTalk
Coffee & Networking
ITU Innovators
● Student Entrepreneurial OrganizationDriven by volunteers
● VisionConnecting students with startup companies, advisors, incubators, events, networking, ...
● EventsArranging and attending talks, workshops, competitions, meetings, ...
IT Kartellet
● IT-consultancy● Software development● Hosting● Moodle/Drupal experience● ASP.NET (MVC)
Copenhagen Tech Talks
Scaling Moodle
● Independent student project fall 2013 under the supervision of Kasper Østerbye
● Collaboration with the IT department of ITU (LearnIT)
● Collaboration with ITK (Moodle consulting)
Scalability, Performance Testing & TuningIT Kartellet and friendsPresented by ITU Innovators
What is Moodle?
● Modular Object-Oriented Dynamic Learning Environment
● Open source● Open University - 700.000 users, 6.000
courses● So it does scale - but how?
Scalability basics
● Vertical / scaling up● Horizontal / scaling out
○ X-axis○ Y-axis○ Z-axis
● Combining○ Performance○ Fault tolerance○ Fault isolation
Moodle architecture
Minimum Example setup
● 20.000 students● High availability requirements -
no downtime is allowed● All components should be
fault tolerant
CBS - Learn
● 2.500 students● Permits for small
windows of downtime
● Should handle large file uploads from games and digital media students
ITU - LearnIT
Moodle as a black box
● ...or a number of black boxes● Large community● Developed over many years● Limited documentation● How can we make the best out of it?
Testing approach
● Repeatable● Automated● Well documented ● Organized● Comparable● Targeted
User request simulation
User request simulation
● Simply HTTP Requests● Realistic input and randomization● Client-side measurements● Why jMeter?
Server-side measurements
● Monitor/record during test● Compare to everyday usage● Invaluable monitoring tools
○ iostat○ inotify○ htop
Server-side measurements - iostat
● Monitoring of cpu/storage activity● Repeated measure with given interval● Most relevant columns
○ iowait, avgqu-sz, await● More details: http://linuxcommand.org/man_pages/iostat1.html
Server-side measurements - inotify
● Monitor all files access, open, write etc.● More details:
○ inotifywatch http://linux.die.net/man/1/inotifywatch
○ inotify-win https://github.com/thekid/inotify-win / JNotify http://jnotify.sourceforge.net/
Server-side measurements - htop
● When does the server give up - when is it trashing itself in terms of CPU / memory?
● Adjust the server settings accordingly - max number of threads, concurrent users etc.
Immediate visualisation
Immediate visualisation
Test and visualisation process
Shell script JMeter R script
Number of usersList of serversRamp-up
Status codeUserResponse time Diagram
Server-side
Variousmeasurements
R scriptDiagram
Case exampleFile storage and cache
● Moodle requires shared cache
File storage and cache
Data properties
● Shared vs. Local● Volatile vs. Non-Volatile
Local
Shared
Shared
Local
Strings, plugin data, config, database meta
Update Logs
File-store, Temporary files
VolatileHtmlpurifier, question data, course contacts
Non-Volatile
Baseline (single shared NFS)
MoodleHDD - File store - Language strings - Plugins data - Config - Htmlpurifier - Question data - Course contacts - Database meta - Temporary files - Update logs
Web server 1
Moodle
Web server 2
NFS
Local volatile cache
MoodleHDD - File store - Htmlpurifier - Question data - Course contacts - Temporary files - Update logs
Web server 1NFS
APC - Language strings - Plugins data - Config - Database meta
Moodle
Web server 2
APC - Language strings - Plugins data - Config - Database meta
Shared volatile cache
MoodleHDD - File store - Temporary files - Update logs
Web server 1
NFS
Memcached
RAM - Language strings - Plugins data - Config - Database meta - Htmlpurifier - Question data - Course contacts
Moodle
Web server 2
Case exampleDatabase
Database load
● Sessions● User access rights● Courses● File meta data● ...
Single instance
● Easy to set up● ACID● Single point of failure● All load on a single server● Vertical scaling
Cluster setup
● Redundancy (X-axis)● Specialisation (sharding)
○ Y-axis○ Z-axis
Scaling Open Source
● Open source code - not build with scaling in mind - will only scale “prettily” if redesigned, but we can do it!○ A lot can be achieved by configuration
● A different approach to contributing to open source!○ We were imagining to be able to walk right in and
point out where moodle developers were wrong○ However, we ended up not changing a single line of
moodle code
Questions?
Coffee & Networking
top related