We have been troubled with some performance issues and therefor we decided to run som tests. All tests were run with the same simple application and built to the different platforms with Airplay SDK. We have done 4 different kinds of test:
Fillrate Test - drawing out a number of pictures on top of each other to see how fast the device can fill pixels on the screen. All pictures were batched and sent as one draw call. We did tests with 10 and 20 images stacked on top of each other.
Draw call Test - we tested this by placing out image in rows on the screen with one draw call per image. We did two tests with this application, one where we wanted to see how many draw calls the device could render at 30 fps, one where we rendered a set number of draw calls (images), 470 draw calls, and wrote down the fps.
Batched Test - we sent in the images as one draw call and placed them in rows and wrote down the number of images the device could render at 30 fps.
CPU Test - tested by calculating 100 000 prime numbers per frame. Performance is determined by the time each frame takes (ms). Lower is better in this test.
Devices:
Samsung Wave GT-S8500, Samsung Bada 1.0
HTC Legend, Android 2.1
iPod Touch 3rd Gen, iOS 4.1
iPhone 3GS, iOS 4.1
iPhone 4, iOS 4.1
Here are the results:
Here we see that the results doesn't change much between GLES 1.0 and GLES 2.x. There are some lower values with the iPhone 3GS and iPod Touch but the differances aren't extreme.
With 20 images the iOS devices have som trouble keeping up. The differances between iOS devices are quite small, Samsung Wave is in the lead.
Here we have some real differances, the Samsung and HTC devices are having real trouble with draw calls while the iOS devices are showing great results. The iPhone 4 is pushing out a impressive 580 draw calls.
Still some really impressive improvements with GLES 2.x. While all iOS devices are holding a firm 30fps and above the Samsung and HTC devices are struggling at a third of the performance.
In the batch test the Legend really shines. The iOS devices are quite similar with the iPhone 4 in the lead. The Samsung is clearly having trouble with this compared to the other devices, this is probably explained by the 15ms compositing delay on Samsung Bada.
To the last test where lower values are better. The iPhone 4 and Wave are both rocking a 1ghz CPU and therefor have almost the same results. The iPhone 4 is scoring a 30ms better than the Wave but it is within the limits. Since 3GS and Touch have the same CPU they are also almost the same. The device that stands out here is the Legend that, like the 3GS and Touch, have 600mhz. Could be that it is underclocked to save power and reduce heat.
Summary:
While fillrate doesn't show that much differance on our test devices we can clearly see that draw calls does. When batching all images the Legend takes the lead. The Samsung Wave has problems keeping up in all test except fillrate, this has been explained to us by Airplay who said that Bada has a 15ms compositing delay.
Conclusion:
Samsung Wave handles a quite high fillrate but the low number of maximum drawcalls makes it perform quite bad in a regular 2D game. The Legend does quite good but also has a problem with drawcalls, although it really shines in the last batch test. The iOS devices doesn't perform as good in the fillrate tests but is the clear winners when it comes to drawcalls.
So this is what we got out of these tests, the iOS devices are quite capable with drawcalls and mediocre in fillrates, the HTC can't handle as much drawcalls but are a little faster with fillrate. The HTC shows a slower CPU performance even though it should be as fast as the 3GS and Touch. The Samsung Wave can handle fillrate but this really doesn't make up for the incredibly low drawcalls and the poor performance in the Batch Test.
Also, switching to GLES 2.x helps all devices performance wise. Although this limits the number of devices the application can support since older devices doesn't support GLES 2.x.
GLES1.0 VS GLES 2.x
It is quite clear, although the fillrates doesn't improve drawcalls are clearly faster in GLES 2.x. So if the device you are developing for supports it, use GLES 2.x instead since the performance have been really improved.
These test helped us since we thought we were bound by fillrate but when reducing draw calls on the non iOS devices we got the framerate up to a more acceptable number. Since the Samsung Bada is still at its first OS version performance will probably be improved and hopefully the compositing delay will be reduced. Hopes this helps someone else as it sure helped us..
/VisualDreams
