Feeds:
Posts
Comments

Posts Tagged ‘silverlight’

In my earlier post I created a high level breakdown of the APIs shared by both Silverlight and WinRT…

image

Here I’m providing a complete reference of all 6,585 public types and 15,248 members included in Silverlight 5 and/or WinRT and where they overlap.

My hope is that this will serve as a reference to Silverlight developers trying to get up to speed with WinRT.

Click here to see the WinRT Genome Project results.

image

 

Note: Please let me know if you see an errors or have requests. I’ve only begun to comb over these results myself and will be looking for ways to improve it going forward. Enjoy!

Advertisements

Read Full Post »

At MIX 2011 I had the fantastic opportunity to present on many of the cool new features recently added to the MMP Player Framework (formerly the Silverlight Media Framework). One of these new features is the ability to play stereoscopic 3D videos (think Avatar). For fun, I also demoed a little app I built that allowed you to play LIVE stereoscopic 3D using 2 webcams.

imageimage

For those of you who want to try this at home, I created a new CodePlex project (liveS3d.codeplex.com) that contains all the required source code and a link to the app I demoed in my MIX session.

To understand how it works, first you need to learn a little about a special kind of plugin in MMP Player Framework called a media plugin. The Player Framework was built with extensibility in mind and many of its features are dynamically loaded as plugins. Even the rectangle used to play the actual video (as shown below) is a dynamically loaded plugin that you could replace with a custom implementation.

image

Note: by default, the Player Framework ships with 2 media plugins: One for progressive video and another for smooth streaming.

Step 1: Creating a webcam media plugin

To start with, I created a custom media plugin that displayed a webcam feed coming directly from the Silverlight webcam API. I did this by creating a new class (WebcamMediaPlugin) that implemented the IMediaPlugin interface found in the Microsoft.SilverlightMediaFramework.Plugins assembly. Check out the source code for the full implementation but here are some highlights…

To paint the webcam video onto a UIElement:

var captureSource = new CaptureSource() {      VideoCaptureDevice = CaptureDeviceConfiguration.GetAvailableVideoCaptureDevices().First()  };videoBrush = new VideoBrush() { Stretch = stretch };videoBrush.SetSource(captureSource); rectVideo = new Rectangle();captureSource.Start(); rectVideo.Fill = videoBrush;

Then, all I needed to do was return that Rectangle via the VisualElement property defined in the IMediaPlugin interface:

public FrameworkElement VisualElement {     get { return rectVideo; } }

Once I had my custom media plugin built, all I had to do was remove the references to the default media plugins and replace it with my own…

image

Violla!…

image

View live demo

Step 2: Showing two media plugins at the same time

The most important part about showing a stereo 3D video is to make sure you have two source (one for the left eye and one for the right). To do this, I built another media plugin that served as a wrapper around two other media plugins. Internally it just created a Canvas control as it’s own VisualElement and then added and positioned the VisualElements from each of the child media plugins so they appeared side by side.

Here’s a screenshot of this new “dual media plugin” working with 2 progressive plugin instances:

image

View live demo

And here’s a screenshot of this new “dual media plugin” working with 2 of my new webcam plugin instances:

image

View live demo

At this point, all I needed to do was to show 2 unique webcam videos from slightly different positions. I only had one webcam, so I ran down to the store and bought another identical webcam and sandwiched them together between two pieces of plastic to prevent them from moving…

imageWP_000016

Armed with 2 webcams and my new plugins, I could easily get the left webcam plugin to display the video from one webcam, and the right webcam plugin to display the video from the right webcam and show them both at the same time:image

Finally, I had everything I needed. I just needed to flip the switch!

Step 3: Adding the S3D plugin

I was now ready to enable the new S3D feature in the Player Framework. This is super simple and all I had to do was add a reference to the new assembly: Microsoft.SilverlightMediaFramework.Plugins.Anaglyph3D and set the PlaylistItem.S3DProperties property.

image

Note: You need to calibrate the angle of your webcams and get them just right. Ideally, the two webcams should both point straight ahead and not diverge or converge. You can calibrate by making the red and cyan images align perfectly for objects very far away.

Toss on the red-cyan glasses and away you go…

…Or…

If you have the NVIDIA 3DVision Active Shutter glasses you can get an even better experience. To supoort this, remove the anaglyph plugin that comes with the Player Framework, and add a reference to the NVIDIA plugin available from the NVIDIA website.

image

Run Live S3D web app

Resources:

Player Framework S3D documentation

Awesome blog post by Bob Cowherd on how the new Player Framework S3D feature works.

MIX session: MMP Player Framework: Past, Present, & Future

Downloads

Source code and binaries

REQUIRED to compile source: NVIDIA 3DVision plugin

REQUIRED to compile source: Microsoft Media Platform Player Framework (formerly Silverlight Media Framework) v 2.5 or higher.

Read Full Post »

There are a lot of reasons for developers to be excited about Windows Phone 7. First and foremost, .NET developers can easily build great apps using the language and tools they already know. If you’re like me, learning a new technology is always fun, but producing great results with minimal effort is better. Smile

But there’s another thing that developers and stakeholders alike should be excited about that goes beyond the benefit of the minimal learning curve: leveraging the Silverlight eco-system.

Here’s a story, straight from the trenches, of the fantasic combination of code-reuse and redesign to maximize the benefits of the Silverlight ecosystem.

Quick history

In the summer of 2009 I created the award winning site: PuzzleTouch.com. The idea for PuzzleTouch was to take an existing and familiar concept (online jigsaw puzzles) and use Silverlight’s powerful runtime and graphics engine to create the world’s best online jigsaw puzzle app. Today, PuzzleTouch is a culmination of over a year of effort, refinement, and performance improvements.

Along came Windows Phone 7

screenshot1screenshot2

*Click images to play videos

The decision to port PuzzleTouch to Windows Phone 7 was a no-brainer. First, as a Silverlight developer I already knew 95% of what I needed to know to build a WP7 app. But more importantly, I already had an established code base to start with.

As Tony Taglialucci from Carlitos Way said, “The contract’s already down on you, pal. The guys, the guns,the lime pit’s already dug. And from in here, just one button I push.”

A little over a month ago, I sat down to see if I could get the main PuzzleTouch game play screen to run on WP7. Not to my surprise but much to my excitement, a couple hours later I was dragging puzzle pieces across the WP7 emulator and snapping them together! Here’s the significance: I’ve been dying to build a version of PuzzleTouch that ran on a phone since day 2. I’ve had many users ask if an iPhone version was in the works and for the past year I’ve been toying with the possibility of making the investment. The primary drawback was purely a matter of time… there was learning Objective C (or risking developing with MonoTouch), learning the new tools, and messing with my web services. And then there was still the question, would it even work? What if there was some insurmountable technical hurdle like no support for PolyBezier clipping or unusably poor performance. The learning process would be fun but let’s face it: my free time is precious.

With WP7, I had a working prototype in 2 hours; hot damn! Realizing the hard part was already done, I spent the rest of my effort building a “real” Windows Phone 7 app and not just a Silverlight web app port. Here were some of the challenges and considerations I faced:

Technical Considerations

Have you ever heard the phrase: “It takes 80% of the time to complete the last 20% of the project”? While my initial port to WP7 was an enormous time-saver, there was a little more to it than adding a few new phone related features and dropping a few web specific features. I won’t get into the gory details on everything that needed changing but here’s a list of the major areas that needed attention from a technical perspective.

  • Pixel shaders: Silverlight on Windows Phone 7 currently doesn’t not support pixel shaders. A big reason PuzzleTouch for the web looks so realistic has a lot to do with shadows and bevels. Fortunately, I was able to mimic dropshadows by adding a second semi-transparent puzzle piece behind the top one to get me one giant step closer to great looking puzzle pieces.

shadows

*Screen shot from actual Windows Phone 7 app.
  • Navigation: Had I used the Silverlight Navigation Framework on my website, this may have gone faster. With WP7 you really need to think in terms of pages and be aware of passing around application state. Implementing support for navigation requied modest effort but overal wasn’t not too significant in the big picture.
  • Tombstoning: Speaking of application state, it is important to understand that your app can be interrupted at any time by the user receiving and viewing a text message, hitting the search button, …etc. The app needs to be able to save its state when the user navigates away and restore it when they come back. This is not hard to do and there is a great API to help, but it is one you can’t overlook.
  • Performance: The PuzzleTouch web version was already finely tuned to support up to 1000 piece puzzles. However, WP7 devices are even less powerful than a netbook and it was necessary for me to revisit performance from top to bottom. In the end hardware acceleration done right saved the day.
  • AppBar: Another consideration was effectively using the WP7 AppBar.

appbar

The AppBar is the perfect venue for your “extra” options and help save space. Speaking of which…

  • Real-estate: Last but certainly not least was the consideration of a much smaller screen. I had to move features to subsequent screens and really think hard about how to maximize the use of the screen to play something that normally requires a considerable amount of space. After all, when was the last time you played a jigsaw puzzle on anything but your dining room table?

Design Considerations

BUT… the technical changes were only part of the story. I cannot stress enough that building a great WP7 app is much more than building a high performance Silverlight web app for a small screen. I wanted PuzzleTouch to look and feel like it was made for the phone… and not just any phone, a Windows Phone. With the help of the new panorama control I completely overhauled the part of the app leading up to the puzzle game play screen. The panorama control was a perfect fit for PuzzleTouch and the funny part is, when it was all said and done, I felt like I had ended up with something possibly even more intuitive and streamlined than in the web version. I was expecting my design choices to be full of compromises but instead it was full of gains. Personally, I think this speaks well to the Metro design paradigm in general. In fact, it makes me wonder if we will start seeing more and more webpages using the metro design in due time. I for one am considering passing on some shifts in design onto the web version. Which leads nicely to my next point…

Vice versa and versa vice

Just like how Silverlight vNext is expected to benefit from the work done on Silverlight for the Windows Phone (e.g. composition thread!), web and desktop apps are likely to benefit from some of the dev and design improvements receiving spotlight from the phone today. I would speculate to say that the average Silverlight developer is more aware of hardware acceleration, frame-rate, multi-theading, and assembly load time performance now than they were before WP7. That is awesome! Silverlight is such a high performance runtime that it is easy to be lazy, but if as developers we learn more from the phone runtime and even make some minor changes to our coding styles, it will make for even better web and desktop apps.

Another great thing in the case of PuzzleTouch was that I was not only able to share code and Xaml, but was even able to share entire assemblies. Therefore, many improvements that I made for the WP7 version have already found their way back to the web version. During the entire WP7 development process I was careful to reuse as much as possible and refactor for extensibility over the copy and alter approach. Although most of the improvements are under the hood, they are still important ones. A few that come to mind are:

  • Performance: Much of the performance tuning I did for the WP7 version was applicable to my website. Refining my bitmap caching by using features like RenderAtScale now make PuzzleTouch.com run even smoother.
  • Greater use of MVVM: One of the less mentioned benefits of MVVM is the ability to share view models with multiple views. In a single Silverlight web or desktop app it is not too common to have the need for two unique views for the same task, but when sharing code with your WP7 app, this becomes a clear win for using MVVM.
  • Styling: In cases where I had built a custom control that could be reused, I still usually needed to style it differently for WP7. Partly to adhere to themes and partly to accommodate a smaller screen size. In these cases, I was able to simply template the control or expose new styling properties. Because of this, I now have more flexible and reusable controls in my PuzzleTouch namespace.

The almighty ecosystem

Possibly the best part about being able to reuse code for the phone and the web is that they can become part of each other’s feature set. With PuzzleTouch, a user can use their phone’s camera to take a picture, turn it into a puzzle, and email it to their friends… who can click on that link and open that puzzle on the website. You might not immediatly see this as a feature of the phone app, but it is to the user. Depending on the app, being able to build an experience for the phone and the web can be invaluable combination. With Silverlight you have this potential and it’s not that hard. I urge those of you who are developing apps for the phone to not overlook this powerful potential.

To conclude, much of what you develop today for the web and desktop can be reused on Windows Phone 7 and vice versa. But don’t be afraid to redesign the UX! Make your web apps shine on the web and your phone apps shine on the phone. The good news is that by using best practices and taking advantage of all the great things in the Silverlight tool chest (Xaml, MVVM, binding, assembly sharing, MEF, …etc.), redesigning the UX while maintaining a common code base is just about as easy as it could be. Once Silverlight for WP7 and Silverlight for the Mac and PC sync up their versions (Silverlight 5!?), it will be even easier.

In the end, I was able to launch a v1.0 WP7 app in under two months in my ‘spare time’ that is arguably on par with  to a v3.0 iPhone app. Plus, the website and the phone versions now compliment each other by extending each other’s feature set and I can enjoy the holidays without slaving away at my computer. It is a win for me, win for my users, and win for the platform.

Links

Video of puzzle home page and creating a puzzle from your phone’s picture album

Video of puzzle game play screen (both regular and advanced puzzles)

PuzzleTouch Jigsaw Puzzles in the Windows Phone 7 Marketplace (zune://navigate/?appID=5309e1b0-acf3-df11-9264-00237de2db9e)
Online Jigsaw Puzzles By PuzzleTouch.com

Read Full Post »

As an amateur photographer, I have always kept my eyes open for new tools to help me search, sort, filter, and view my own photos. Microsoft Pivot (a cool new technology that lets you easily view, search, and filter data using deep zoom) seemed like a natural and obvious fit for the task.

Therefore, using Microsoft Pivot and the Pauthor open source libraries, I created  a tool that helps users create a pivot collection from the photos on their hard drive and supports filtering by all the nifty meta data embedded in those files such as shutter speed, aperture, film speed, and focal length. Plus, users end up with a great look set of deep zoom images from their photo collection!

How:

Step 1: Install Microsoft Pivot

Step 2: Install PhotoPivot beta (I’m calling it a beta because I wrote it in about 8 hours).

image

Browse for the folder containing the photos that you would like to “Pivot-ize”

Browse for the location where you would like to dump the pivot collection and all the associated deep zoom files.

Note: This can take up a lot of space

Click “Go”

Warning: If you run this on a couple hundred high resolution photos it will take ~5-10 minutes, if you run it on all 20,000 photos in your My Pictures folder… do it before you go to bed!

The Result:

In the end, assuming Microsoft Pivot is installed, it will launch the Pivot app and automatically load your collection…

image

Highlights:

  1. The end result is awesome… especially when it’s your own photos that you’re filtering and deep zooming on.
  2. This collection is totally compatible with the Microsoft Silverlight PivotViewer control which makes it super easy to publish the result on the web. All you have to do is upload the output to a web server, toss a clientconfigpolicy.xml file in the domain’s root and build a 3 minute Silverlight PivotViewer app to view them. Read my earlier blog post for more details. The only down-side: if you thought it took a long time to generate the files, wait until you try to upload them; yikes!
  3. Source code is available. Knock yourself out!

Kudos:

  • Microsoft Pivot. Seriously folks, this is where all the real work was done.
  • DeepZoom. It wouldn’t be nearly as cool without it.
  • Pauthor. And without this awesome open source library, I would never have bothered to build this. (It even uses parallel processing to generates all the deep zoom images). Great work guys!

Read Full Post »

The new Silverlight PivotViewer control provides a cool and powerful way to visualize and filter data straight out of the box. If you have some data with a few dozen records or more and have images associated with each record, I urge you to give it a spin and build a quick Silverlight PivotViewer app to show off and enable users to search and filter that data. The whole affair won’t take you more than an hour or two and in the end you’ll have a great looking, fun and powerful window into our data.

All you need to do is generate some special files that contain the data you want to show, create a Silverlight app that hosts the new Silverlight PivotViewer control, and point it at your data.

Here is a proof of concept that I built using data from PuzzleTouch, my online jigsaw puzzles application, that shows off all the stock puzzles available and allows them to be filtered by category, popularity, and difficulty (based on user stats).

Online Jigsaw Puzzle Catalog Viewer

Click here to view

Here’s a quick walk-through demonstrating how to build your very own PivotViewer app…

Step 1: Generate the data. This is where most of the work is (but that isn’t saying much). In the end you will end up with a number of Xml and deep zoom image files. Depending on the data, the easiest way may be to use the Pivot Collection Tool for Microsoft Excel. This is an Excel Addin that allows you to toss all your data in a spreadsheet and the addin will generate the Pivot data files from it. However, because I was gathering my data from a handful of different sources and because I find programmatically generating the data to be more interesting, I decided to forego Excel and generate it from code using the Pivot Collection Tool (a.k.a. Pauthor) open source C# library. This library provides an extremely simple API that has everything you need to easily generate all the data that the Silverlight Pivot control needs. All you need to do is add the PauthorLib project to your solution and away you go…

PivotCollection items = new PivotCollection();
items.Name = "Online Jigsaw Puzzles by PuzzleTouch";
items.BrandImage = new PivotImage("http://puzzletouch.com/logo.png");
items.Icon = new PivotImage("http://puzzletouch.com/favicon.ico");
items.FacetCategories.Add(new PivotFacetCategory("Category", 
PivotFacetType.String));
items.FacetCategories.Add(new PivotFacetCategory("Difficulty", 
PivotFacetType.Number));
items.FacetCategories.Add(new PivotFacetCategory("Popularity", 
PivotFacetType.Number));

foreach (var puzzle in StockPuzzles)
{
    PivotItem item = new PivotItem(puzzle.PuzzleId, items);
    item.Description = puzzle.Description;
    item.Name = StripExtension(puzzle.Filename);
    item.Href = string.Format("http://puzzletouch.com/?puzzle={0}",
 puzzle.PuzzleId);
    item.Image = new PivotImage
(string.Format("C:\\PuzzleTouch\\Images\\{0}.png", puzzle.PuzzleId));
    item.AddFacetValues("Category", puzzle.Category.ToString());
    item.AddFacetValues("Difficulty", puzzle.Difficulty);
    item.AddFacetValues("Popularity", puzzle.Popularity);

    items.Items.Add(item);
}

PivotCollectionBuffer source = new PivotCollectionBuffer(items);
LocalCxmlCollectionTarget target = new LocalCxmlCollectionTarget(
"C:\\PuzzleTouch\\PivotData\\puzzles.cxml");
DeepZoomTargetFilter targetFilter = new DeepZoomTargetFilter(target);
targetFilter.Write(source);

Here I am creating a PivotCollection object to store my data. I tell it which fields I want to support filtering by (called a FacetCategory), and then I loop through my records and create a  new PivotItem for each. Each PivotItem needs a name, href, and image, as well as a value for each FacetCategory.

Once I’m done populating my PivotCollection object, I export it to the file system (which produces all the files required for the pivot control to function). There are other formats you can export to and from, but for my purposes I just wanted to generate from code and produce a local copy of the Pivot data all ready to go. Note: The coolest part here is that all the deep zoom images will be created automatically from the source images! No separate tools or typing in command lines are necessary!

Please refer to the Pauthor documentation for more details on the Pauthor library.

Step 2: Upload. Once you have the pivot data files created, upload them. I chose to upload my data to Amazon S3 because it is cheap and scalable but you could easily put it in any HTTP accessible location. Just remember to drop a clientaccesspolicy.xml file in the root of the domain if the data is hosted somewhere other than where you will be hosting your Silverlight app from.

Step 3: Create your Silverlight app. This is almost too easy…

  • Insert the following xaml into your main page:
xmlns:pivot="clr-namespace:System.Windows.Pivot;
assembly=System.Windows.Pivot"

<pivot:PivotViewer x:Name=“PivotViewer” />

  • In the code behind, load the PivotViewer control with the url of the .cxml file that you uploaded to your web site:
PivotViewer.LoadCollection("http://puzzletouch.s3.amazonaws.com/
puzzles.cxml", string.Empty);

That’s it!!! You now have an application that can view your pivot data collection in all it’s glory. Deep zooming, filtering, searching…etc is all built in and working.

From there, you can go forth and customize the experience all you want but with very little work you’ll have a fully functional Pivot viewing application that will give your users a lot of powerful abilities straight out of the box.

Read Full Post »

Thanks to everyone who joined me for a great discussion about my new favorite technology: Reactive Extensions (Rx) for Silverlight. As promised, here is the source code for all the projects we went over as well as my PowerPoint slides…

Source code for all projects

Power point

I hope everyone grows to love Rx as much as I do! It will turn your brain inside just out a little bit, but its a good thing. 🙂

And don’t worry, I haven’t forgotten about the padlock demo. I’ll be posting a follow up with explanation and full source soon.

Read Full Post »

Silverlight 4 introduces XPath support including XPathNavigator and LINQ to XML extension methods for evaluating XPath expressions on your XElement objects! For those that aren’t familiar with XPath, it is essentially a big string that contains specially formatted text that is interpreted at runtime to describe a pattern for finding nodes in an XML document. You can use it to find nodes with specific conditions much like how regular expressions finds smaller strings in a bigger string. 

The downsides to using XPaths are 1) the string describing your XPath can get quite complicated and difficult to read, and 2) because the string must be interpreted at runtime, performance is worse than alternate methods. This is why I normally discourage using XPaths and probably why Microsoft shyed away from including XPath in earlier versions of Silverlight… thereby bolstering using the better practice of LINQ to XML for searching and parsing XML documents. However, XPaths (like regular expressions) have their place and at times come in very handy! 

One particular use that I’ve come to love over the years is the ability to create and run XPath queries at runtime. You don’t have to know very much about XPaths to easily write simple ones like “/vehicles/car” to find all the ‘car’ nodes in the ‘vehicles’ node. Or slightly more complex XPaths like: “/vehicles/car[@color=’green’]” (find all car nodes with a ‘color’ attribute of ‘green’). Because an XPath is just a string, it is very easy to assemble at runtime or even allow a power users to enter it in the UI if you dare open that can of worms 🙂 However, there is one kind of user I feel perfectly comfortable allowing in the can of worms… developers. Which is why I’ve created… 

XPathPad

How often do you just want to find out very simple information about an Xml file? Maybe you want to know how many nodes are in your document or how many have a certain attribute. Maybe you want to extract the text values from certain nodes so you can use them in another application. Maybe you want to paste all the values of a certain attribute into Excel so you can manually graph them. The possibilities are endless and if you ever have to work with Xml files (especially someone else’s), a tool to easily help without writing a program can be invaluable! 

How to use it:

  1. Run the app. Note: You can also install Out of browser.
  2. Browse for any xml file…
  3. Type in an XPath for the nodes you want to find
  4. Type in another XPath in the Output field for what stuff in each of those nodes you want to display.
  5. Click Go

 

Note: You can optionally enter a comma delimited list of XPaths in the output field if you want to get multiple things about each node found. Multiple results per node are separated by tabs so you can easily copy and paste the results into Excel and get different columns for each.

How it works under the hood:

Silverlight 4’s new XPath features did just about all of the work for me. First, I had to add a reference to the new XPath assembly:

Then, I simply load a standard XDocument from a filestream and call the new extension method that is available:

IEnumerable<XElement> elements = xdoc.XPathSelectElements(XPath);

This returns all the elements found at the provided XPath. I then loop thru each of these elements and call a second extension method to get a node relative to that element based on the output XPath:

var result = element.XPathEvaluate(outputXPath) as IEnumerable;

XObject value = result.Cast<XObject>().FirstOrDefault();

if (value is XElement)

    return ((XElement)value).Value;

else if (value is XText || value is XCData)

    return ((XText)value).Value;

else if (value is XAttribute)

    return ((XAttribute)value).Value;

else

    return value.ToString();

XPathEvaluate returns a collection of XObjects. I simply take the first one and return its value to be displayed in the results window.

To find out more about how XPathPad was written or to modify it to better fit your needs, download the source.

Read Full Post »

Older Posts »