Augmented Reality#1: Display The Live Video Feed

by CameronM 4. October 2011 05:53

There are loads of interesting Augmented Reality (AR) apps doing the rounds on all smartphone platforms and now, Microsoft has given developers all the tools they need to build great AR apps for the Mango release of Windows Phone.

The first step in creating the next big augmented reality app is to display the live video feed. Before the Mango release, the only interaction developers had with the camera was using the CameraCaptureTask, which I covered in an earlier post. The CameraCaptureTask only lets you start the built-in camera app and wait for it to send back the results to your app.

With the new features in Mango, you can access the feed directly from the camera, with only a few lines of code. All you need to do is set the source of the VideoBrush control to the devices camera.

To start, add a Rectangle to your MainPage XAML and set it’s Fill to a VideoBrush.


        <VideoBrush x:Name="videoBrush"/>

In the code behind, add a using statement for Microsoft.Devices and the in either the Loaded or OnNavigatedTo method, set the source of the VideoBrush to the PhotoCamera class.

void MainPage_Loaded(object sender, RoutedEventArgs e)


    PhotoCamera m_camera = new PhotoCamera();



Compile and run the app in the emulator and you’ll see an exciting black square moving around on a white background. Don’t worry, on a real device you will see the actual camera feed.

Now, if you are lucky enough to have a real Windows Phone 7 device and you have upgraded to Mango, you will find the results were less than ideal. In fact, the results were about 90 degrees off ideal! Yes that’s right, the code that everyone is flogging off as being able to display the live camera feed displays the resulting video 90 degrees off what it should be. It’s all very interesting and art-nouveau, but in terms of AR apps, it’s rubbish.

Thankfully it’s very easy to fix the problem. All you need to do is apply a transformation to the VideoBrush used to fill the Rectangle.

        <VideoBrush x:Name="videoBrush">
                <CompositeTransform CenterX="0.5" CenterY="0.5" Rotation="90" />

This will rotate the camera feed so that what you expect to see through the viewfinder is actually what you get.

This is of course only the first step to creating a fully fledge AR application that will combine the digital and real world in ways you only dreamed of, but it is a huge step forward compared to the old CameraCaptureTask days.

This code is not only useful for augmented reality, but with a small amount of work you can use any number of the existing launcher and chooser tasks in WP7 to create apps that show that real-world camera feed behind whatever tasks you need to perform.

In the next few posts we’ll look at building a simple email and sms tool that uses the camera feed so that users can see where they are going while writing that all important email.

Tags: ,

Windows Phone 7 | WP7