Wednesday, November 24, 2010

Performance testing at VisualDreams

We are now in the process of getting Herman the Hermit to work on all platforms we are going to release to. So this means that we are now testing it on a number of different phones and as all developer knows (and dreads) they all work in different ways.

We have been troubled with some performance issues and therefor we decided to run som tests. All tests were run with the same simple application and built to the different platforms with Airplay SDK. We have done 4 different kinds of test:

Fillrate Test - drawing out a number of pictures on top of each other to see how fast the device can fill pixels on the screen. All pictures were batched and sent as one draw call. We did tests with 10 and 20 images stacked on top of each other.

Draw call Test - we tested this by placing out image in rows on the screen with one draw call per image. We did two tests with this application, one where we wanted to see how many draw calls the device could render at 30 fps, one where we rendered a set number of draw calls (images), 470 draw calls, and wrote down the fps.

Batched Test -  we sent in the images as one draw call and placed them in rows and wrote down the number of images the device could render at 30 fps.

CPU Test - tested by calculating 100 000 prime numbers per frame. Performance is determined by the time each frame takes (ms). Lower is better in this test.

Samsung Wave GT-S8500, Samsung Bada 1.0
HTC Legend, Android 2.1
iPod Touch 3rd Gen, iOS 4.1
iPhone 3GS, iOS 4.1
iPhone 4, iOS 4.1

Here are the results:

Here we see that the results doesn't change much between GLES 1.0 and GLES 2.x. There are some lower values with the iPhone 3GS and iPod Touch but the differances aren't extreme. 
With 20 images the iOS devices have som trouble keeping up. The differances between iOS devices are quite small, Samsung Wave is in the lead.

Here we have some real differances, the Samsung and HTC devices are having real trouble with draw calls while the iOS devices are showing great results. The iPhone 4 is pushing out a impressive 580 draw calls.

Still some really impressive improvements with GLES 2.x. While all iOS devices are holding a firm 30fps and above the Samsung and HTC devices are struggling at a third of the performance.

In the batch test the Legend really shines. The iOS devices are quite similar with the iPhone 4 in the lead. The Samsung is clearly having trouble with this compared to the other devices, this is probably explained by the 15ms compositing delay on Samsung Bada.

To the last test where lower values are better. The iPhone 4 and Wave are both rocking a 1ghz CPU and therefor have almost the same results. The iPhone 4 is scoring a 30ms better than the Wave but it is within the limits. Since 3GS and Touch have the same CPU they are also almost the same. The device that stands out here is the Legend that, like the 3GS and Touch, have 600mhz. Could be that it is underclocked to save power and reduce heat.

While fillrate doesn't show that much differance on our test devices we can clearly see that draw calls does. When batching all images the Legend takes the lead. The Samsung Wave has problems keeping up in all test except fillrate, this has been explained to us by Airplay who said that Bada has a 15ms compositing delay.

Samsung Wave handles a quite high fillrate but the low number of maximum drawcalls makes it perform quite bad in a regular 2D game. The Legend does quite good but also has a problem with drawcalls, although it really shines in the last batch test. The iOS devices doesn't perform as good in the fillrate tests but is the clear winners when it comes to drawcalls.

So this is what we got out of these tests, the iOS devices are quite capable with drawcalls and mediocre in fillrates, the HTC can't handle as much drawcalls but are a little faster with fillrate. The HTC shows a slower CPU performance even though it should be as fast as the 3GS and Touch. The Samsung Wave can handle fillrate but this really doesn't make up for the incredibly low drawcalls and the poor performance in the Batch Test.
Also, switching to GLES 2.x helps all devices performance wise. Although this limits the number of devices the application can support since older devices doesn't support GLES 2.x.

It is quite clear, although the fillrates doesn't improve drawcalls are clearly faster in GLES 2.x. So if the device you are developing for supports it, use GLES 2.x instead since the performance have been really improved.

These test helped us since we thought we were bound by fillrate but when reducing draw calls on the non iOS devices we got the framerate up to a more acceptable number. Since the Samsung Bada is still at its first OS version performance will probably be improved and hopefully the compositing delay will be reduced. Hopes this helps someone else as it sure helped us..


Thursday, October 7, 2010

iPhone 4 sucks, in a developers perspective

And why is this? Let me explain. As you know we are currently developing a game for iOS here at VisualDreams. And for sure we thought that our game would have the best performance on iPhone 4. But thats not the case. iPhone 3gs runs with about 20 more frames per second, 60 fps against about 40 fps on the iPhone 4.

All this because iPhone 4 has a higher screen resolution. The problem is that iPhone 3gs and iPhone 4 have the same GPU, and this means that when rendering 2D textures, iPhone 4 has to render 4 times as many pixels to the screen and thus have a much higher fillrate.

So now I'm working hard to optimize the rendering on iPhone 4. The thing is that there isnt that much to do when you are fillrate bound except from optimizing all the textures. As for now we dont use sprite atlases, maybe we could gain a few fps by lowering number of drawcalls.

In a worse case scenario we would have to go with the same resolution as iPhone 3gs. But hopefully this problem will be solved in a near future. I will let you know how it goes..

/Kristofer, Coder

Homepage in progress!

Yesterday Bohman and I decided that a match of Rock-Paper-Scissors will decide who's fate it will be to make our homepage, and of course I lost. David have finished the design and all I have to do now is to get the homepage to work as intended.

The homepage will probably be done next week if everything goes well. On our homepage all the lates news, videos, screenshots and games will be posted.

Here is a small preview of what the page will look like..

/Harald, Coder

Art Complete!

All the games art is now completed! Surely there will be things that will have to be added, tweaked or polished (as always with game development) but as of now everything needed is done. The coders are currently working on menues and optimisation/bugfixing while I am tweaking some gameplay.

Hopefully Kristofer will be able to post something about a problem we ran into this week as soon as possible..

Anyway, to celebrate this joyous occasion I put together a picture displaying a little bit more of the environment and also showing of our characters amazing skills..

/David, Artist

Company presentation..

First post, guess I will present the company and myself for starters. I am VisualDreams one and only artist and my name is David, and well since the coders are eagerly programming at the moment I guess I should go first..

VisualDreams is a newly started company that will specialise in mobile games and applications. We are three guys. Two coders and one artist. After finishing our 2 year education at The Game Assembly, we decided to put our skills together and throw ourselves into the wild mobile app market. Starting with the iPhone/iPad's AppStore with intention to go multiplattform and include development for Android Market and Windows Marketplace.

We started up on the 1st of September and began development of our first game around the 10th of September. Now, almost 4 weeks into development we have a playable game that are now going through the first stages of polish and gameplay tweaking.

Since we aren't really ready to announce or show off that much of the game yet we decided to present our main character. We are still fighting about what to name him so if you have any good suggestions, shoot!

To show your support and keep updated while we construct our homepage join our Facebook and Twitter feed..

Stay tuned for more!

/David, Artist