Jan
4
2011
4
On the topic of GPU acceleration in a mobile OS
John Gruber recently linked to this piece by Charles Ying, The Care and Feeding of the Android GPU. I am glad to see this issue get some attention – it’s been one of my pet peeves for a long time, and it is a defining difference between iOS and many other mobile OSes, Android included.
I remember many years ago when Apple introduced hardware accelerated UI operations into Mac OS X. At the time, I knew this would be a big deal someday. In the short term, it paid off in all kinds of smooth, cool animations in OS X. But many of these were eye candy, and the benefit to UI interaction often went unnoticed to end-users (things just worked better and smoother and nobody cared why). With a mouse-driven UI, there's a disconnect between input and display, so users are less apt to notice responsiveness on a micro level.
When they released the very first iPhone in 2007, Apple fully leveraged their experience with OS X to create a mobile OS with a hardware accelerated UI. I doubt anybody outside the developer community cared about the details, but it was obvious that the very first iPhone was benefitting extensively from GPU acceleration when you scrolled lists, panned the viewport, etc. And UI animations were fantastic. It was no longer eye-candy -- it was magic that made a touch interface functional and responsive. To this day, I give a lot of credit to Apple for getting the touch UI right, and a big part of that was GPU acceleration.
Many mobile touch screen OSes have come along in the time since, but few have offered the smooth slick feel of iOS, despite often running on devices with big guns for CPUs. I can pick up a Blackberry, WebOS device, or Android device, and immediately tell that OpenGL and GPU acceleration are not present in the UI. Once you know what hardware acceleration offers, software rendering sticks out like a sore thumb. It's slow, it's laggy, and it's at the mercy of anything else that may be using the CPU (which happens to be a lot of things on a typical smart phone with apps that use location services, camera, accelerometer, compass, gyro, etc). In short, it's just not a good way to do a touch interface because the graphics will never keep up with user inputs in a responsive, smooth, or consistent way. I think consistency is perhaps the biggest factor, and it’s the biggest problem I see when using Android. With a touch interface, consistency is as important as accuracy and precision. Things need to respond the same way all the time, or the interface feels like a kludge.
The notion that Android's solution lies in faster CPUs is wrong for many reasons. First, it's inefficient. Software rendering with a CPU is like mowing your lawn with a pair of gas-powered scissors, one leaf of grass at a time. Sure, faster scissors will improve performance, but the experience will still suck. A lot. Hardware accelerated rendering is like firing up the lawn mower, which can cut entire swaths of grass with a single sweep of the blade. You don't need to be an engineer to envision which approach will take less gas overall. GPUs are inherently more efficient at rendering, and you want to use them over the CPU whenever possible. The benefits in heat generation and battery life are significant, and compound. Every developer knows this. You just don't use the CPU for important rendering when you can access the GPU. The GPU will always be more efficient.
And that leads to the second reason that this dependence on software rendering in Android is wrong -- it shows a lack of commitment to doing things the right way from step one. Google's casual attitude towards hardware rendering is not surprising to me. If the Android UI was not designed from the ground up to take advantage of the GPU (like iOS was) then it's not going to be an easy task to put it in now. Trust me, I'm a reluctant developer sometimes, and I can spot it a mile away. Revising an OS to include low level GPU acceleration in the UI would be one hell of a grind. But it's something I believe Android must do if it really wants to go toe to toe with iOS.
Look at it this way -- CPUs are getting faster across the board, and Apple is no slouch with custom ARM hardware (not to mention that Apple no longer clings to CPU architectures courtesy of their PowerPC experience; they'll go with different silicon if that's what it takes to lead the game). iOS devices will match any increases in horsepower on Android devices. But those iOS devices will also be leveraging the GPU on top of that, and GPUs have been even farther ahead of CPUs in their pace of advancement. So this is a race that can’t be won when relying solely on software rendering. No amount of gas will make those scissors catch up to the mower.
I remember many years ago when Apple introduced hardware accelerated UI operations into Mac OS X. At the time, I knew this would be a big deal someday. In the short term, it paid off in all kinds of smooth, cool animations in OS X. But many of these were eye candy, and the benefit to UI interaction often went unnoticed to end-users (things just worked better and smoother and nobody cared why). With a mouse-driven UI, there's a disconnect between input and display, so users are less apt to notice responsiveness on a micro level.
When they released the very first iPhone in 2007, Apple fully leveraged their experience with OS X to create a mobile OS with a hardware accelerated UI. I doubt anybody outside the developer community cared about the details, but it was obvious that the very first iPhone was benefitting extensively from GPU acceleration when you scrolled lists, panned the viewport, etc. And UI animations were fantastic. It was no longer eye-candy -- it was magic that made a touch interface functional and responsive. To this day, I give a lot of credit to Apple for getting the touch UI right, and a big part of that was GPU acceleration.
Many mobile touch screen OSes have come along in the time since, but few have offered the smooth slick feel of iOS, despite often running on devices with big guns for CPUs. I can pick up a Blackberry, WebOS device, or Android device, and immediately tell that OpenGL and GPU acceleration are not present in the UI. Once you know what hardware acceleration offers, software rendering sticks out like a sore thumb. It's slow, it's laggy, and it's at the mercy of anything else that may be using the CPU (which happens to be a lot of things on a typical smart phone with apps that use location services, camera, accelerometer, compass, gyro, etc). In short, it's just not a good way to do a touch interface because the graphics will never keep up with user inputs in a responsive, smooth, or consistent way. I think consistency is perhaps the biggest factor, and it’s the biggest problem I see when using Android. With a touch interface, consistency is as important as accuracy and precision. Things need to respond the same way all the time, or the interface feels like a kludge.
The notion that Android's solution lies in faster CPUs is wrong for many reasons. First, it's inefficient. Software rendering with a CPU is like mowing your lawn with a pair of gas-powered scissors, one leaf of grass at a time. Sure, faster scissors will improve performance, but the experience will still suck. A lot. Hardware accelerated rendering is like firing up the lawn mower, which can cut entire swaths of grass with a single sweep of the blade. You don't need to be an engineer to envision which approach will take less gas overall. GPUs are inherently more efficient at rendering, and you want to use them over the CPU whenever possible. The benefits in heat generation and battery life are significant, and compound. Every developer knows this. You just don't use the CPU for important rendering when you can access the GPU. The GPU will always be more efficient.
And that leads to the second reason that this dependence on software rendering in Android is wrong -- it shows a lack of commitment to doing things the right way from step one. Google's casual attitude towards hardware rendering is not surprising to me. If the Android UI was not designed from the ground up to take advantage of the GPU (like iOS was) then it's not going to be an easy task to put it in now. Trust me, I'm a reluctant developer sometimes, and I can spot it a mile away. Revising an OS to include low level GPU acceleration in the UI would be one hell of a grind. But it's something I believe Android must do if it really wants to go toe to toe with iOS.
Look at it this way -- CPUs are getting faster across the board, and Apple is no slouch with custom ARM hardware (not to mention that Apple no longer clings to CPU architectures courtesy of their PowerPC experience; they'll go with different silicon if that's what it takes to lead the game). iOS devices will match any increases in horsepower on Android devices. But those iOS devices will also be leveraging the GPU on top of that, and GPUs have been even farther ahead of CPUs in their pace of advancement. So this is a race that can’t be won when relying solely on software rendering. No amount of gas will make those scissors catch up to the mower.