Programmer Payback

At MIX 2011 I had the fantastic opportunity to present on many of the cool new features recently added to the MMP Player Framework (formerly the Silverlight Media Framework). One of these new features is the ability to play stereoscopic 3D videos (think Avatar). For fun, I also demoed a little app I built that allowed you to play LIVE stereoscopic 3D using 2 webcams.

For those of you who want to try this at home, I created a new CodePlex project (liveS3d.codeplex.com) that contains all the required source code and a link to the app I demoed in my MIX session.

To understand how it works, first you need to learn a little about a special kind of plugin in MMP Player Framework called a media plugin. The Player Framework was built with extensibility in mind and many of its features are dynamically loaded as plugins. Even the rectangle used to play the actual video (as shown below) is a dynamically loaded plugin that you could replace with a custom implementation.

Note: by default, the Player Framework ships with 2 media plugins: One for progressive video and another for smooth streaming.

Step 1: Creating a webcam media plugin

To start with, I created a custom media plugin that displayed a webcam feed coming directly from the Silverlight webcam API. I did this by creating a new class (WebcamMediaPlugin) that implemented the IMediaPlugin interface found in the Microsoft.SilverlightMediaFramework.Plugins assembly. Check out the source code for the full implementation but here are some highlights…

To paint the webcam video onto a UIElement:

var captureSource = new CaptureSource() {      VideoCaptureDevice = CaptureDeviceConfiguration.GetAvailableVideoCaptureDevices().First()  };videoBrush = new VideoBrush() { Stretch = stretch };videoBrush.SetSource(captureSource); rectVideo = new Rectangle();captureSource.Start(); rectVideo.Fill = videoBrush;

Then, all I needed to do was return that Rectangle via the VisualElement property defined in the IMediaPlugin interface:

public FrameworkElement VisualElement {     get { return rectVideo; } }

Once I had my custom media plugin built, all I had to do was remove the references to the default media plugins and replace it with my own…

Violla!…

View live demo

Step 2: Showing two media plugins at the same time

The most important part about showing a stereo 3D video is to make sure you have two source (one for the left eye and one for the right). To do this, I built another media plugin that served as a wrapper around two other media plugins. Internally it just created a Canvas control as it’s own VisualElement and then added and positioned the VisualElements from each of the child media plugins so they appeared side by side.

Here’s a screenshot of this new “dual media plugin” working with 2 progressive plugin instances:

View live demo

And here’s a screenshot of this new “dual media plugin” working with 2 of my new webcam plugin instances:

View live demo

At this point, all I needed to do was to show 2 unique webcam videos from slightly different positions. I only had one webcam, so I ran down to the store and bought another identical webcam and sandwiched them together between two pieces of plastic to prevent them from moving…

Armed with 2 webcams and my new plugins, I could easily get the left webcam plugin to display the video from one webcam, and the right webcam plugin to display the video from the right webcam and show them both at the same time:

Finally, I had everything I needed. I just needed to flip the switch!

Step 3: Adding the S3D plugin

I was now ready to enable the new S3D feature in the Player Framework. This is super simple and all I had to do was add a reference to the new assembly: Microsoft.SilverlightMediaFramework.Plugins.Anaglyph3D and set the PlaylistItem.S3DProperties property.

Note: You need to calibrate the angle of your webcams and get them just right. Ideally, the two webcams should both point straight ahead and not diverge or converge. You can calibrate by making the red and cyan images align perfectly for objects very far away.

Toss on the red-cyan glasses and away you go…

…Or…

If you have the NVIDIA 3DVision Active Shutter glasses you can get an even better experience. To supoort this, remove the anaglyph plugin that comes with the Player Framework, and add a reference to the NVIDIA plugin available from the NVIDIA website.

Run Live S3D web app

Resources:

Player Framework S3D documentation

Awesome blog post by Bob Cowherd on how the new Player Framework S3D feature works.

MIX session: MMP Player Framework: Past, Present, & Future

Downloads

Source code and binaries

REQUIRED to compile source: NVIDIA 3DVision plugin

REQUIRED to compile source: Microsoft Media Platform Player Framework (formerly Silverlight Media Framework) v 2.5 or higher.