Augmented Reality#2: Is that picture level?
A while ago I wrote a post about creating a simple spirit-level using the Accelerometer in Windows Phone 7. In the Mango release a number of things have changed with the way the Accelerometer API is accessed and how its’ results are returned. There has also been one huge new feature released in Mango that takes the simple spirit-level app to a whole new level. Of course I am speaking about the VideoBrush control, which enables access to the live video feed from the device.
As I pondered potential uses of Augmented Reality, my mind drifted back to the spirit-level application and I began to think about how useful it would be to be able to stand back from an object and determine if it was level. I mean, you have to admit it is easier to stand back from your newly hung photo frame and check that it is level than to simply guess. Ok, you may be able to place your trusty spirit-level (or WP7) against a photo frame and test that it is level, but what about when you are standing in the Louvre and suddenly realise that the Mona Lisa looks a little crocked? I don’t think those burly security guards are going to let you walk up and put your phone on top of the frame.
The first thing you will notice when dealing with the Accelerometer in Mango is that there is no ReadingChanged event. According to MSDN, this has been replaced by the CurrentValueChanged event. The next thing you’ll notice is that the CurrentValueChanged event expects a SensorReadingEventArgs parameter instead of the AccelerometerReadingEventArgs parameter we utilised in our code for Windows Phone 7. In the Mango release, Microsoft have decided to standardise the way developers access the ever increasing range of sensors – including the Accelerometer, Compass, Gyroscope and Motion.
//display the video feed
PhotoCamera m_camera = new PhotoCamera();
this.videoBrush.SetSource(m_camera);
//start the Accelerometer
sensor.CurrentValueChanged +=new EventHandler<SensorReadingEventArgs<AccelerometerReading>>(sensor_CurrentValueChanged);
sensor.Start();
Thankfully, all of these changes result in only a handful of changes to the original Windows Phone 7.0 code. One thing the Mango 7.5 code takes advantage is, especially when dealing with 3D objects, is the XNA framework. This is evident by that fact that the EventArgs returned by the CurrentValueChanged event no longer refer to X,Y,Z, but to a XNA object called Vector3 (Microsoft.Xna.Framework.Vector3). Again, a small change to our previous code enables us to visualise in which direction the device is tilted, just like a real spirit-level.
void sensor_CurrentValueChanged(object sender, SensorReadingEventArgs<AccelerometerReading> e)
{
Dispatcher.BeginInvoke(() => MyReadingChanged(e));
}
void MyReadingChanged(SensorReadingEventArgs<AccelerometerReading> e)
{
//use the XNA Vector3 object to access X,Y,Z
Microsoft.Xna.Framework.Vector3 moved = e.SensorReading.Acceleration;
BallTransform.X = -e.SensorReading.Acceleration.Y * (Track.Width - Ball.Width);
}
Using the Accelerometer#2 Creating a Level
In this post we will extend upon the simple Accelerometer example I looked at in a previous post by actually graphically representing the Accelerometer data on the Windows Phone 7 device. One way to do this is to create a spirit-level – you know the thingy in your toolbox that you use to see if something is level or not!
Thanks to some XAML from Jeff Prosise's Blog we can easily add a spirit-level to the previous application and use it to track the movement of the device relative to one dimension. We add a simple rectangle and a bubble (an Ellipse) to represent how far out of level the device is.
<StackPanel Margin="0" Orientation="Vertical" VerticalAlignment="Top" Width="400">
<Button Click="Button_Click" Height="78" Content="Start" />
<TextBlock x:Name="TextBlockX" Height="50" TextWrapping="Wrap" Text="TextBlock"/>
<TextBlock x:Name="TextBlockY" Height="50" TextWrapping="Wrap" Text="TextBlock"/>
<TextBlock x:Name="TextBlockZ" Height="50" TextWrapping="Wrap" Text="TextBlock"/>
<TextBlock x:Name="TextBlockTimeStamp" Height="50" TextWrapping="Wrap" Text="TextBlock"/>
<Canvas>
<Rectangle x:Name="Track"
Width="400"
Height="80"
Stroke="#FFC0C0C0"
RadiusX="40"
RadiusY="40" />
<Line Stroke="#FFC0C0C0"
X1="0" Y1="0" X2="0" Y2="120"
HorizontalAlignment="Center"
VerticalAlignment="Center"
Margin="200,0,0,0" />
<Ellipse x:Name="Ball" Width="80" Height="80" Margin="160,0,0,0">
<Ellipse.Fill>
<RadialGradientBrush GradientOrigin="0.234,0.255">
<GradientStop Color="White"/>
<GradientStop Color="Red" Offset="0.759"/>
</RadialGradientBrush>
</Ellipse.Fill>
<Ellipse.RenderTransform>
<TranslateTransform x:Name="BallTransform" />
</Ellipse.RenderTransform>
</Ellipse>
</Canvas>
</StackPanel>
The Rectangle acts as a visual guide helping us understand which way the bubble needs to move before the device is level.
Most real spirit-levels only show the tilt in one dimension at a time, so we’ll only be looking at moving the bubble along the X axis in the example. It is also important to note that the bubble in a spirit-level moves in the opposite direction to the current tilt of the device, so we need to move our Ellipse in a negative X direction.
To graphically display the movement of the device we will use a TranslateTransform transformation named BallTransform to move the bubble to the left or right. This transformation can then be called from the code we created to handle the ReadingChanged event in the previous post.
void MyReadingChanged(AccelerometerReadingEventArgs e)
{
TextBlockX.Text = e.X.ToString();
TextBlockY.Text = e.Y.ToString();
TextBlockZ.Text = e.Z.ToString();
TextBlockTimeStamp.Text = e.Timestamp.ToString();
//New code to move the ball from left to right
BallTransform.X = -e.X * (Track.Width - Ball.Width);
}
Running the app on a WP7 device will display a red ball that moves in the opposite direction to the current tilt of the device, enabling you to find the true level of any object you lay the device on. This may be handy if you need to show the boss that your desk really does lean to the right!
Using the Accelerometer
The Accelerometer is part of the Sensors API, so to you will need to add a reference to Microsoft.Devices.Sensors. You will also want to add a using statement on the pages where you will be using the Accelerometer.
using Microsoft.Devices.Sensors;
Starting the Accelerometer and accessing its values is as simple as many of the other API calls you’ll make in developing a Windows Phone 7 application. In the code behind you declare a local variable of type Accelerometer and call its Start() method.
//declare a local variable
Accelerometer sensor;
private void Button_Click(object sender, RoutedEventArgs e)
{
sensor = new Accelerometer();
sensor.ReadingChanged += new EventHandler<AccelerometerReadingEventArgs>(sensor_ReadingChanged);
sensor.Start();
}
As with most calls to the device API’s, you will need to handle an event whenever the Accelerometer reading changes. As you can see in the code above, we are handling the ReadingChanged event which includes a parameter of class AccelerometerReadingEventArgs. AccelerometerReadingEventArgs contains a number of fields you will need to use in deciding which way the device has been moved, including the X,Y,X and a Timestamp value.
Whenever the readings from Accelerometer change, sensor_ReadingChanged is called, which invokes a background thread to handle the change of values.
void sensor_ReadingChanged(object sender, AccelerometerReadingEventArgs e)
{
//e contains X,Y,Z readings and a timestamp
Dispatcher.BeginInvoke(() => MyReadingChanged(e));
}
void MyReadingChanged(AccelerometerReadingEventArgs e)
{
TextBlockX.Text = e.X.ToString();
TextBlockY.Text = e.Y.ToString();
TextBlockZ.Text = e.Z.ToString();
TextBlockTimeStamp.Text = e.Timestamp.ToString();
}
Running this code will result in the values for X,Y,X and the Timestamp being displayed on the screen in their respective TextBlocks. The emulator can't move, so you will find the results somewhat boring. Running the app on a WP7 device however is a different story. Even when you place the device on a flat surface you will notice that the readings continually change, often many times per second. In a future post we will look at how you can smooth out these readings to obtain information about the general direction of movement.
