perfmarks benchmarking html5 rendering performance
TRANSCRIPT
![Page 1: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/1.jpg)
PerfMarks
Benchmarking HTML5 Rendering Performance
![Page 2: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/2.jpg)
Why did we make PerfMarks?
• Spaceport: Develop games with Adobe tools, publish to cross-platform mobile apps
• Native apps AND HTML5 web-apps• What should we use?– CSS2D, CSS3D, WebGL, canvas?
• Rendering Performance matters• Which is best for mobile browsers• Benchmark it
![Page 3: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/3.jpg)
Methodology
• HTML5 games: – Translation– Scaling– Rotation
• How many images can be moved around while maintaining 30 FPS?
• Easy to measure... right?
![Page 4: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/4.jpg)
Accurate Benchmarking is hard
• Lots of complications– JIT– latency– memory– screen updates
• Getting data takes a long time• Results require scrutiny
![Page 5: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/5.jpg)
Try #1: Draw 100 frames – time itvar testStart = Date.now();var maxDraws = 100;var numDraws = 0;
function next() { for (var i = 0; i < numObjects; ++i) { renderObject(i); }
++numDraws; if (numDraws >= maxDraws) { var testEnd = Date.now(); done(testEnd - testStart); } else { setTimeout(next, 0); }}next();
![Page 6: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/6.jpg)
Expected Results
0 20 40 60 80 100 1200
20
40
60
80
100
120
140
160
180
Frame Render Time
Object Count
mill
iseco
nds
![Page 7: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/7.jpg)
Actual Results
0 20 40 60 80 100 1200
5
10
15
20
25
30
35
40
Frame Render Time
Object Count
mill
iseco
nds
![Page 8: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/8.jpg)
JIT
• Just in time engine kicks in for later runs of the test – making benchmark results depend on the order the tests are run.
• Invalid results.• Solution: warm up the JIT by running the tests
twice – take data from second run.
![Page 9: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/9.jpg)
Next Issue
var img = new Image();img.onload = function onloaded() { // img is loaded!
var copy = img.cloneNode(true); // is copy loaded?};img.src = "foo.png";
![Page 10: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/10.jpg)
DERP
![Page 11: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/11.jpg)
Image Loading Latency
• In certain browsers image not loaded– Android Browser– Mobile Firefox– Opera– Desktop Safari
• Absurdly high benchmark scores• Solution: Add more onLoad() checks – TONS of
redundant onLoad() checks...
![Page 12: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/12.jpg)
How many objects to run?
• First pass was [1, 3, 5, 10, 30, 100]• Next pass was add 25 until it's no longer
making 30 FPS – then increment by a smaller step size.
• Problems:
![Page 13: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/13.jpg)
Real world data is fuzzy!
![Page 14: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/14.jpg)
Real world data is large!
• Counting to 7000 by 25 takes a long time• Especially when you are testing 3 mobile
browsers on a dozen Android phones
![Page 15: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/15.jpg)
A Better Algorithm
• Run Test – estimate time/object/frame• Use estimate to predict the number of objects
that will run at exactly 30 FPS• Repeat until predicted number of objects has
already been tested• Ensure that there are also data-points above
and below
![Page 16: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/16.jpg)
Next problem - Memory
1. Generate Test Data2. Run Tests3. Destroy Test Data
• This kills the Garbage Collector• This will hard-crash most mobile browsers
![Page 17: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/17.jpg)
A Better Approach
• Cache test data• When more data is requested than has been
generated – add on to the cached values.• Clear out all test data when the test is
completely finished.
![Page 18: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/18.jpg)
Does the screen update?var testStart = Date.now();var numDraws = 0;
function next() { setTimeout(next, 0);
for (var i = 0; i < numObjects; ++i) { renderObject(i); }
++numDraws; var frameEnd = Date.now();
var fpsEstimate = 1000 * numDraws / (frameEnd - testStart);}next();
![Page 19: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/19.jpg)
Asynchronous Screen Updates
![Page 20: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/20.jpg)
"requestAnimationFrame"
• "requestAnimationFrame" only happens in modern browsers if it's actually rendering to the screen.
• This let's us count the number of times the screen was actually updated during a test run
![Page 21: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/21.jpg)
More Accurate Frame Countingfunction next() { requestAnimationFrame(next, element);
for (var i = 0; i < numObjects; ++i) { renderObject(i); }
++numDraws; var frameEnd = Date.now();
var fpsEstimate = 1000 * numDraws / (frameEnd - testStart);}
requestAnimationFrame(next, element);
![Page 22: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/22.jpg)
Results
• Modern Smartphone vs Modern Laptop• Macbook Pro vs iPhone 4S– iOS 5.1– Mobile Safari vs Safari 5.1 (Webkit)
• Macbook Pro vs Samsung Galaxy S2– Android 4.0.3– Mobile Chrome Beta vs Chrome 19
![Page 23: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/23.jpg)
Safari
![Page 24: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/24.jpg)
Chrome
![Page 25: PerfMarks Benchmarking HTML5 Rendering Performance](https://reader030.vdocuments.mx/reader030/viewer/2022032517/56649c995503460f9495693d/html5/thumbnails/25.jpg)
PerfMarks
• http://spaceport.io/community/perfmarks• https://github.com/sibblingz/PerfMarks