You’d be forgiven this last month or so for thinking that here in The Oven we’ve only been about B-Boy Beats, we’ll that’s not quite the case. As well as blockbusting games we love getting our hands in to some hardcore technology building and we’ve been working on some really exciting and cutting edge Augmented Reality (AR) applications.
Mobile AR is the next big thing, or so the informed media would have you believe. Put simply, AR is a blending of real and virtual worlds in real-time. Some common examples of the technology are often seen on televised sports, when displaying scores or lines marking players movements.
Despite all the hype, AR has actually been around for a while now, but only now is the technology allowing it to really break from simple passive overlaying to rich user interaction. Last year saw a number of AR apps launched on the App Store, including Layar and the AcrossAir Browser. These apps are merely the tip of the possiblity iceberg for AR as they simply overlay basic graphics onto the video stream.
So, driven by these possibilities we have been investigating how to improve on what’s currently out there and how to bring the next generation of AR to mobile platforms.
The first application of the tech we dug in to is the likely familiar navigator (see the screenshot below). This displays the direction of nearby locations and corresponding information about them. Okay, so not groundbreaking perhaps, but we’ve made huge leaps in usability when compared to the slew of currently available sub-par apps and have also increased functionality with new market-leading features.
Currently we’re getting stuck in to smoothing off the rough edges and beautifying it, allowing it to be customisable and reskinnable. We’ll follow up in the coming weeks with a demo video showing the technology in full action.
There is a lot more to AR beyond this rather simple use however. The holy grail is to create software that is intelligent enough to understand it’s surroundings and produce stunning imagery which can be blended into the environment. This is the second sort of application we’re working on – detecting items in the real environment and mixing in virtual 3D objects with the video stream.
Some of our initial work on analysing real world video and mixing in 3D
A lot of computing power needs to go into getting the software to understand the real world. One of the first steps we took towards this was simply detecting edges. Even something as trivial as this involves intensive resources – especially in real-time and especially on a mobile phone, so it required lots of programming trickery to make things run quickly and smoothly.
The image above shows the writing on the page has been detected and highlighted as an edge (it’s marked in green). The result may end up looking a lot like a cheap sci-fi B-movie effect but in fact it is cutting edge research!
3D models running over real-time video photoage
Another challenge facing mobile AR technology is getting nice 3D graphics running over video. This is key to creating rich and seemless world-merging experiences which is likely to be the future. The above screenshot shows Mobile Pie’s AR technology displaying a spinning globe over a real-time video stream on an iPhone 3GS. Unfortunately iPhone doesn’t have a huge amount of power behind it compared to the supercomputers used in Hollywood blockbusters, so lots of optimisation is required to make even this simple display run smoothly. In this demo the globe rotates depending upon which direction you’re facing.
I hope this preview of our work-in-progress AR technology has shown you a little glimpse in to the future of not only Mobile Pie but the way in which we will interact with devices and the world around us. If you would like to know more or how this technology can help your company or organisation then please feel free to contact us.
