Feeds:
Posts
Comments

Microsoft Media Platform: Player Framework for Windows 8 Metro style apps

March 11, 2012 by Tim Greenfield

Need to build an app that plays video? This week we released a new version of the MMP: Player Framework (formerly the Silverlight Media Framework) made for Windows 8 Metro style applications.

With guidance the Windows 8 and IIS teams, we (Microsoft Media Platform team & Vertigo Software) created an open source media player framework that you can use for both HTML-based and XAML-based Metro style applications.

image

Support for HTML and XAML

The player framework supports both HTML/JavaScript and XAML/C#/VB based Metro style applications. Technically, there are two different components (one for HTML and one for XAML based apps), but they share very similar APIs, feature sets, and in some cases even share the same source code. Under the hood, the XAML version is built on top of the MediaElement that ships with .NET and similarly, the HTML version is built on top of the HTML5 <video> tag.

Our primary goal was to ensure both flavors felt natural and familiar to both HTML and XAML developers. If you are an HTML/JavaScript developer, using the JavaScript version will be very similar to using the other JavaScript controls that ship with Windows 8 as well as those already familiar with the HTML5 <video> tag. If you are a XAML/managed code developer, you can expect the XAML version to adhere to the practices and patterns of .NET & Silverlight controls.

Extensibility

The framework: We refer to the player framework as a framework as opposed to a control or component because it is intended to be a mini-platform in it’s own right. While these details will be hidden to the majority of developers using it as is out of the box media player, the player framework is built on a plugin style architecture that enables the feature set to be expanded and evolve after the fact without touching the core code base. In fact, many of the features currently in the framework are implemented as plugins. For example: the main control bar that the user interacts with is technically a plugin that could be swapped in our out for a radically different implementation.

Styling: One of the most common requirements in building a media player is to be able to skin the UI to match your brand. By default, the player framework ships with a Metro style skin, but you can easily modify the look and feel of the player from CSS or XAML without having to build a custom player.

Open source: the player frameworks ships with full source code under the liberal Ms-Pl license. Crack it open and see how it ticks or make changes to suite your needs.

How similar is it to the existing HTML5 or Silverlight versions of the player framework?

You may already be familiar with the existing HTML5 for the web or Silverlight for the web/desktop and phone versions of the player framework. The Windows 8 version of the framework should feel familiar and is in fact based on much of the same code. However, you should not expect full compatibility. We wanted to make sure the player framework felt like it was made for Windows 8 Metro style apps and it’s respective development model (HTML vs. XAML). We also wanted to meld the two versions and their APIs when it made sense. Lastly, we used this opportunities to make architectural improvements that would better position the framework for a long and bright future.

Features

You’ll find that the player framework has an extensive API chocked full of many useful hooks and features. From 30,000 feet, here is a sampling of features you will find in v1 of the player framework.

  • VOD
  • Progressive video
  • Smooth streaming
  • Rich DVR (Play, pause, stop, fast forward, rewind, skip next & back, seek-able timeline, volume, slow motion, instant replay)
  • Full screen
  • Closed captions (plain text for Xaml and WebVTT for js)
  • Styling support
  • Buffering and error/retry visual states.
  • Poster image
  • Click-to-play UI to let the user choose to start watching.
  • All features already included with the MediaElement and <Video/>
  • Playlist
  • Chapters
  • Timeline markers
  • Support for localization
  • Compact UI for snapped view
  • Support for DRM (e.g. PlayReady)

What’s next?

The fun doesn’t stop here. We’re already hard at work on more features. Here are some prominent ones in the cards.

  • Adaptive streaming APIs
  • Advertising
  • Audio player
  • TTML closed captions
  • Audio track selection (multi-language audio)
  • Live video

Got feedback?

We’d love to hear your feedback and make sure the player framework offers the features and functionality you need in your apps. We encourage developers to check out the player framework on CodePlex and post feedback on the CodePlex forum.

Hope you enjoy!

Advertisement

Posted in Media, Win8 Metro | Tagged , , , , , , , , , | 9 Comments »

WinRT Genome Project Details

November 18, 2011 by Tim Greenfield

In my earlier post I created a high level breakdown of the APIs shared by both Silverlight and WinRT…

image

Here I’m providing a complete reference of all 6,585 public types and 15,248 members included in Silverlight 5 and/or WinRT and where they overlap.

My hope is that this will serve as a reference to Silverlight developers trying to get up to speed with WinRT.

Click here to see the WinRT Genome Project results.

image

 

Note: Please let me know if you see an errors or have requests. I’ve only begun to comb over these results myself and will be looking for ways to improve it going forward. Enjoy!

Posted in Silverlight tips and tricks, Win8 Metro | Tagged , , , | 8 Comments »

The WinRT Genome Project

November 17, 2011 by Tim Greenfield

What is the hardest part about learning a new programming language? I’ve often argued that learning a new language is easy, it’s learning a new framework is can be time consuming.

In case you’ve been living under a rock lately, you probably already know that Windows 8 Metro style apps can be built with Xaml & C#. This means that .NET & Silverlight developers can leverage their existing skillset to easily build Win8 Metro Xaml style apps. But just how similar is Silverlight to WinRT?

In an effort to get some real numbers, I created a program to iterate over all the default assemblies included WinRT and compare with those found in Silverlight 5. Here’s what I found:

image

 

Of those 1,582 matching types, here is a look at the members (properties, method, & events) they have in common:

image

 

Comparison details for those interested:

  • I included all Silverlight 5 assemblies installed as part of the SDK. Other assemblies such as RIA services and the Silverlight Toolkit were not included.
  • The public Silverlight 5 RC was used. IMO it is highly unlikely any numbers will change once the final version is released.
  • Differences in namespaces, type accessibility, and base classes were still counted as a match. For example, I ignored differences such as a type being sealed or implementing ISerializable.
  • Similar liberties were taken with members.
  • Property get & set methods and event add & remove methods were not counted as unique members.

My key takeaways:

  • WinRT includes a ton of new types. Some of these are certainly for new features such as the marketplace, accelerometer support, …etc. Time to start learning!
  • Approximately 1/4 of the types in Silverlight are not present in WinRT. Some of these are no longer relevant like the types associated with the DOM bridge or Out of Browser support.
  • When comparing just the shared types, SL5 starts to look very close to an actual subset of WinRT. This is good news for Silverlight developers because it means most of your existing code should easily port.

Coming soon: I also have a side-by-side comparison of the entire WinRT and SL5 API so you can explore each and every difference in their respective APIs. Check back soon.

Update: Detail now available, here’s a complete comparison of every single public API in WinRT and Silverlight 5 and where they match.

Posted in Uncategorized | 20 Comments »

//build/ is a double rainbow. What does it mean?

September 17, 2011 by Tim Greenfield

BUILD

Today concludes the last day of the Microsoft Build conference (//build/) and after four days of sessions, keynotes, developing on the Win8 tablet, and countless conversations with other developers I have formed a number of opinions (both good and bad) about the new Windows 8 development stack and what it means to the world outside of Microsoft.

Disclaimer: If anyone tells you they’re objective, they’re not being objective. I’m biased in all kinds of ways and this is a summary of my personal thoughts and opinions. Whether you agree or disagree, hopefully this will invoke a few ideas and get you thinking about the changes we witnessed this week.

//build/targets…

 

/businesses

image

/brand_visibility

What do Best Buy, the Daily Show, and Hershey’s Chocolate have in common? They all have apps in the iPhone app store. But why wouldn’t these companies just build or improve their mobile websites? Would an app somehow be a better experience than a mobile website? Doubtful. OK, maybe Best Buy can use native phone APIs to help you find a store based on your current location and maybe you can build more touch-friendly features with Objective C, but the real reason is to show up to the party. Get listed in the app store or your customers may not be able to find you. But the app store is much more than the iOS yellow pages. Users actively go look for apps they think might be useful. This is very different from how consumers use the web. How many users open their shiny new iPhone and go browse the web for sites to add to their favorites list? None. Compare that with how many users instantly go look for cool apps to install. Like it or not, Windows 8 will be used on a daily basis by millions of users. Apple has already taught the world that there is an “app for everything” and users will be stoked to find one on their shiny new PC. Whether building a metro app for the new Windows 8 marketplace is justified as an opportunity or simply a precaution, most businesses can’t afford to stay home and miss the party.

/app_marketing

Ask any developer who’s built an awesome program that never made them rich what the hardest part about building successful software is. Answer: marketing. We developers have loads of ideas and talent but usually don’t know much about marketing or like learning how to do it right. At the same time, there are many profitable apps in the iOS app store by these same kinds of developer. Enough of them have been able to rake in the cash and barely lifted a finger to market their apps. The app store is a magical jolly fat man leaving presents for those who were simply good. Go to download.com and check out the thousands of non-mobile apps out there. I’d venture to say that very few have ever made a dime regardless of their popularity; now is their chance!

/cross_platform

There’s one huge story that Microsoft didn’t exactly mention but I believe they are willing to and perhaps even subtly encouraging people be misguided about. Yes, web developers can use their skills to build first class Windows 8 apps. Yes, web developers can build those apps faster if they have an existing app or website to port, but can you really share a common code base between a website and a Win8 app? Fat chance. Besides being inappropriate to design a metro app and a website identical to one another, the code just won’t port if you do it right… especially, if you use the all new and powerful Blend designer tool to build it. Blend is built for creating Win8 apps and although it generates HTML5 compliant HTML & CSS, it also uses Win8 specific controls and references js files that rely on the WinRT. Furthermore, it is highly doubtful that its output has ever been tested on anything but IE10. The bottom line: HTML and js may save you time, but mostly because of the skills of your developers, not the reusability of the code for other platforms.

 

/consumers

image

/tablets

Tablets are the real story with Win8. Will Win8 be better than Win7 for the non-touchscreen desktop or laptop? Probably. But is that difference compelling? Probably not. Tablets are clearly the primary target for Win8 and there are some good reasons to think Win8 could own the majority of the tablet market in due time. Here’s the difference between a Win8 tablet and the competition as of today: while iPads and Android tablets are hardly toys, they certainly aren’t workstations. Every single time I’ve traveled in the last year, I came packing both an iPad and my laptop. And each and every time I desperately tried to part with one or the other. While away for pleasure, I couldn’t part with my laptop in fear that I might need it to do something important. While away on business, taking the laptop was a given but I ultimately couldn’t part with my iPad so I could get my fix of Angry Birds on the airplane. Win8 tablets will be different. Once I load up my new Win8 tablet and learn to trust what the OS can do, I’m almost certain the only devices I’ll be traveling with will be my phone and tablet. Thank baby Jesus! OK, maybe this isn’t such a horrible burden to have: bringing 3 devices with you on vacation. But it demonstrates the problem pretty well. If I can travel with just a Win8 tablet, my mother-in-law can live with just a Win8 tablet. Bottom line: Microsoft is going to seriously raised the bar for the next generation of tablets.

/netbooks+laptops

While iPads may cannibalize netbooks and laptops today, Win8 tablets will bleed into the market and all devices will simply become portable PCs. Because Microsoft relies on others to build hardware, we can expect to see tablets of all different sizes and performance abilities. Some will have built in keyboards oozing with glutinous power and others will be wafer thin, glorified phones. Keeping with tradition, Microsoft will look to the plethora of hardware makers to innovate and fill every little nook and cranny of possible use-cases humans could have for portable computers. Netbooks and laptops won’t be cannibalized, they’ll be absorbed.

/desktop

I have my own ideas on the future of the desktop PC but it has much more to do with the Kinect than it does Win8; I’ll save those ideas for another blog post. For now, I don’t see any profound impact Win8 will have on the desktop PC J

/web

The web browser is dead. OK, I intentionally over-sensationalized that statement, but let’s be honest, apps and websites more and more are serving the same function. Because of the invention of the app store, apps are gaining popularity over websites for the widening areas where they overlap. Imagine you needed a stop watch on your iPad. Would you search the web or search the app store? I’ve actually asked this question to a number of iPad-owning friends and every one of them said without hesitation that they’d search the app store. The web browser is no longer the go-to tool to extend the features of your phone or tablet and soon the same will be true for your PC. Not long ago, it was inconceivable that a business would launch an app before building a website unless their only product was the app itself. Today, I can only assume that there are companies who have already built apps and will eventually get around to making a website. Is the web dead? Absolutely not. Apps need the web. Is the web browser dead? Hardly, the web browser will remain the go-to tool for content for a long time, but the role of the web browser is diminished and this is just the beginning.

/security

The other week, my colleague Justin Angel called out the Windows Phone AVG fake anti-virus app… how can you build an anti-virus app for a platform that doesn’t support viruses? Metro apps on Windows 8 won’t be as locked down as apps on the Windows Phone but a similar principal will be at work. Windows 8 requires all apps submitted to the app store to 1) run in a security sandbox, 2) be certified (and therefore subject to being pulled and publicly reviewed) and 3) be required to declare the various limited abilities they are allowed to perform at runtime. The user can therefore be guaranteed that when they install an app, it will do no more (in terms of security) than the app claims to do and they will have ample warning before installing that app. Unless users continue the soon-to-be antiquated practice of downloading and running exes from the internet instead of installing apps from the marketplace, they will be instantly granted incredible protections that only platforms like Win8 metro can provide.

/user_experience

While Windows 8 will still support Win32 apps, metro apps are where it’s at! Metro is a new design style that is beautiful, fluid and friendly to all forms of input devices including your fingers. Everyone I’ve talked to seems to love the new metro design paradigm, but even those that want overlapping windows with lots of chrome will be able to use the desktop/classic (Win7-like) mode if they prefer. In my opinion, Windows 8 metro is a leap forward in user experience and clearly sets a new bar for usability for apps and operating systems alike.

 

/developers

image

/silverlight+wpf

I’ve been asked this question more than any other at the conference: “What do you think the future of Silverlight is?” In case you’ve been living under a rock, this is the one question Microsoft promised to answer at the Build conference and unfortunately is one of the few questions they still refuse to address. I was expecting Steven Sinofsky himself to issue a very frank and straightforward statement about the future of Silverlight on day one… even if it risked boos from the crowd; we are owed that much. Unfortunately, Build is over and all we are left with is our own speculation. You might think that Silverlight has a bright future. After all, Silverlight 5 was released as a public RC only a week or so before the conference and for legions of Microsoft developers it is still the preferred choice for building web apps. So, what is “up” with Silverlight? As a Silverlight developer I have to grit my teeth a little when I profess my honest opinion that “Silverlight for the web is dead”. Ouch, that still stings a little.

The good news: managed code and Xaml is NOT dead. And managed code and Xaml of course is after all Silverlight or WPF without a name from the marketing department. In fact, although it remains to be seen, I think managed code could even become the predominant paradigm (over C++ and javascript) for building the next generation of Windows 8 apps. At the very least it will be a very popular choice. This is great news for Silverlight and WPF developers who want to build top-shelf metro apps for Windows 8. Additionally, Silverlight in browser and out-of-browser apps will continue to work as is for Windows classic/desktop mode.

The bad news: the build conference was NOT about the web browser or Win8 desktop mode. For all the IE pumping we’ve heard in the last year, I was left a bit surprised we didn’t hear more about IE. ‘The Gu’ gave us a quick peek at some new ASP.NET features and VS 2011 near the end of the 2nd day keynote but the content felt so out of place it was almost as if the respected ‘Gu’ was only there to appease the hordes of Microsoft developers (including myself) who he helped inspire during his tenure in the developer division at Microsoft. But it makes sense: Windows 8 and the future of the Microsoft dev stack are about metro and about metro apps.

The ugly news: Silverlight does not run in the metro browser and the metro browser will most likely be the main browser from now on in Windows. This makes some sense since Silverlight would need to be refactored to use the new WinRT controls and APIs under the hood in order to correctly support input and display the way a metro app should. Microsoft wants the metro experience to be pure and consistent across the board. This means that all apps that run in the metro environment must be using WinRT to do the heavy lifting – including web pages running through the metro browser. HTML & js based web pages can do this because IE10 uses WinRT under the hood to do all the rendering. Silverlight versions 1-5 do not. Could a special version of Silverlight for IE metro be rebuilt to use WinRT? Anything is possible. Will it be done? I wouldn’t hold your breath. At the same time, Silverlight out of browser apps are in a similar boat. Silverlight OoB apps won’t run in metro and are confined to the desktop/classic mode only.

OK, back to my “Silverlight is dead” comment: managed code and Xaml running in a web browser simply does not appear to be part of Microsoft’s long-term strategy. What caused this? While I’d love to speculate, I won’t in this post. Microsoft will continue to keep its foot in the door for web development using ASP.NET but in the end, Microsoft sells a little product called Windows and everything it does in some way needs be traced back to benefiting Windows. Web development in general is only important to Microsoft for two reasons: First, the traditional reason: Users use the web and want an awesome web browser. IE is a feature of Windows and Microsoft wants developers to be able to easily build web pages that work well in it. Second (and more importantly now than 1 week ago): Microsoft wants a great selection of apps in the Windows marketplace and they want the legions of web developers to help write those apps. It is a too large and talented pool of programmers to ignore any longer. .NET developers and HTML+js developers can now all partake. Silverlight for the web isn’t going anywhere but Silverlight will continue to lose reach as Windows 8 metro will now be added to the list of platforms that won’t support it. Sorry fellow Silverlight developers, we don’t have a choice; for the long term I suggest for you to start writing Windows apps or switch to HTML & js.

/C++

C++ developers must be peeing their pants from happiness. Finally, C++ lovers are first class citizens in the Windows world by being able to take advantage of many of the modern advances in programming that has transpired since the advent of .NET. Just like VB6 paved the way to NET, .NET paved the way to WinRT. Finally C++ programmers will be able to use Xaml and a rich object oriented runtime to build apps. It must feel good.

/winforms

If C++ programmers are peeing their pants from happiness, Winforms programmers are peeing their pants from agony. Buck up Winformers, you should have switched to WPF years ago and then at least you’d have the skillset to make the metro switch.

/html+js+css+svg

These developers are the real winners. Even if you don’t plan to write a sigle Win8 app, be happy knowing you could. 🙂 HTML5, CSS3, js & SVG are all first class citizens now with native access to the WinRT and killer performance of IE10 as your rendering engine.

/azure

The Build conference pretended to include Windows Azure but like ASP.NET, content was relegated to the hind quarter of the day 2 keynote and sessions were limited. For the most part, Windows 8 doesn’t really affect the story of the back-end; not significantly anyway. There are some new roaming features in Windows 8 which Azure will be well positioned to help out with and without a built in namespace for local database access from WinRT, connecting apps to the cloud will be more important than ever. However, at the same time, Microsoft is courting the web developers who are comfortable with non-Microsoft platforms. Microsoft wouldn’t dare make them use Windows Server or Windows Azure to easily build a Win8 app and therefore, for the time being, Azure takes a back seat in order to avoid polluting the message.

/asp.net

Build was clearly all about Windows on the client and by virtue of having an app store, starts to pave the way for reduced importance of applications hosted from within the web browser. ASP.NET came up near the end of the day 2 keynote and while there were some nice improvements to debugging and website optimization, there was very little content overall and almost zero sessions. HTML based Win8 apps have nothing to do with ASP.NET. They must ship as local, pre-made html & js packages and use AJAX if communication with the web is necessary. ASP.NET, Razor, MVC, and Webforms have no place in this world aside from providing one of many possible options for implementing web services.

/wp7

Windows Phone is very much part of Microsoft’s strategy. Yet it also played virtually no roll in the Build conference. Some have speculated that Windows Phone as we know it is merely a stop gap until Windows 8 (or 9) can be made to run on those devices. Windows 8 will only support the new Metro style apps on ARM based architecture and it is easy to see how it could be a good fit for running on even smaller form factors than the tablet in the near future. Windows Phone developers should expect someday to have the same paradigm choice for building apps that the Win8 platform now enjoys.

/xna

Surprise! There’s no XNA support in Windows 8 and no way to control DirectX from managed code. While I don’t have a shred of evidence to back this next statement up, I have a strong feeling this won’t be the last we hear of XNA. There are too many awesome games built on it and it is too useful to abandon. It would be a shame if Microsoft didn’t offer an XNA option or something similar soon. For now we’ll have to hold our breaths.

 

/competition

image

/apple

Apple has made a distinct choice that a tablet is more like a phone than a personal computer. Consumers have been more than willing to live with this difference because they haven’t known any different. Will consumers continue to accept this decision after seeing a Windows 8 tablet that offers the versatility of a laptop? I expect not but it remains to be seen. I believe it is inevitable that Apple will either make OSX more like iOS or will try to follow the Win8 lead and build a radical new hybrid OS. It will be interesting to see how the world’s favorite innovator reacts long-term to the innovations seen this week.

/google

Google should have serious reason for concern! While Android is rocking the Casbah, Google pays the bills with its advertising business. Windows Phone shows us just how irrelevant Google can be if the platform has it’s own built in search feature and provides its own simple advertising component for apps to use. Google needs to get used to it and should start fearing the worst in the long term. In fact, it probably already has. Android was probably not invented because Google has a passion for building a better OS, Android was a very smart and pro-active action on Google’s part to make sure their advertising and search platforms will continued to be used. I wouldn’t count Google out of the race by any means but their profitability clearly depends on Androids ability to out compete Windows.

/mozilla

Yikes, Firefox for Windows (Chrome, Opera, & Safari as well) had better hope Windows 8 metro is a failure. Even if they build a Windows 8 metro app, they will ultimately be building a layer on top of WinRT to render web pages instead of being part of WinRT like IE10 is. This would the equivalent of building a web browser to run inside a web browser. Performance will never be as good and they will always be 2nd in class compared to IE10 for mainstream users and geeks alike.

 

Special thanks to all the awesome people I met at Build for the 4 day collective learning session and great conversations. This blog post comes from your brains more than my own.

Posted in Uncategorized | 7 Comments »

Live Stereoscopic 3D in the MMP Player Framework

April 16, 2011 by Tim Greenfield

At MIX 2011 I had the fantastic opportunity to present on many of the cool new features recently added to the MMP Player Framework (formerly the Silverlight Media Framework). One of these new features is the ability to play stereoscopic 3D videos (think Avatar). For fun, I also demoed a little app I built that allowed you to play LIVE stereoscopic 3D using 2 webcams.

imageimage

For those of you who want to try this at home, I created a new CodePlex project (liveS3d.codeplex.com) that contains all the required source code and a link to the app I demoed in my MIX session.

To understand how it works, first you need to learn a little about a special kind of plugin in MMP Player Framework called a media plugin. The Player Framework was built with extensibility in mind and many of its features are dynamically loaded as plugins. Even the rectangle used to play the actual video (as shown below) is a dynamically loaded plugin that you could replace with a custom implementation.

image

Note: by default, the Player Framework ships with 2 media plugins: One for progressive video and another for smooth streaming.

Step 1: Creating a webcam media plugin

To start with, I created a custom media plugin that displayed a webcam feed coming directly from the Silverlight webcam API. I did this by creating a new class (WebcamMediaPlugin) that implemented the IMediaPlugin interface found in the Microsoft.SilverlightMediaFramework.Plugins assembly. Check out the source code for the full implementation but here are some highlights…

To paint the webcam video onto a UIElement:

var captureSource = new CaptureSource() {      VideoCaptureDevice = CaptureDeviceConfiguration.GetAvailableVideoCaptureDevices().First()  };videoBrush = new VideoBrush() { Stretch = stretch };videoBrush.SetSource(captureSource); rectVideo = new Rectangle();captureSource.Start(); rectVideo.Fill = videoBrush;

Then, all I needed to do was return that Rectangle via the VisualElement property defined in the IMediaPlugin interface:

public FrameworkElement VisualElement {     get { return rectVideo; } }

Once I had my custom media plugin built, all I had to do was remove the references to the default media plugins and replace it with my own…

image

Violla!…

image

View live demo

Step 2: Showing two media plugins at the same time

The most important part about showing a stereo 3D video is to make sure you have two source (one for the left eye and one for the right). To do this, I built another media plugin that served as a wrapper around two other media plugins. Internally it just created a Canvas control as it’s own VisualElement and then added and positioned the VisualElements from each of the child media plugins so they appeared side by side.

Here’s a screenshot of this new “dual media plugin” working with 2 progressive plugin instances:

image

View live demo

And here’s a screenshot of this new “dual media plugin” working with 2 of my new webcam plugin instances:

image

View live demo

At this point, all I needed to do was to show 2 unique webcam videos from slightly different positions. I only had one webcam, so I ran down to the store and bought another identical webcam and sandwiched them together between two pieces of plastic to prevent them from moving…

imageWP_000016

Armed with 2 webcams and my new plugins, I could easily get the left webcam plugin to display the video from one webcam, and the right webcam plugin to display the video from the right webcam and show them both at the same time:image

Finally, I had everything I needed. I just needed to flip the switch!

Step 3: Adding the S3D plugin

I was now ready to enable the new S3D feature in the Player Framework. This is super simple and all I had to do was add a reference to the new assembly: Microsoft.SilverlightMediaFramework.Plugins.Anaglyph3D and set the PlaylistItem.S3DProperties property.

image

Note: You need to calibrate the angle of your webcams and get them just right. Ideally, the two webcams should both point straight ahead and not diverge or converge. You can calibrate by making the red and cyan images align perfectly for objects very far away.

Toss on the red-cyan glasses and away you go…

…Or…

If you have the NVIDIA 3DVision Active Shutter glasses you can get an even better experience. To supoort this, remove the anaglyph plugin that comes with the Player Framework, and add a reference to the NVIDIA plugin available from the NVIDIA website.

image

Run Live S3D web app

Resources:

Player Framework S3D documentation

Awesome blog post by Bob Cowherd on how the new Player Framework S3D feature works.

MIX session: MMP Player Framework: Past, Present, & Future

Downloads

Source code and binaries

REQUIRED to compile source: NVIDIA 3DVision plugin

REQUIRED to compile source: Microsoft Media Platform Player Framework (formerly Silverlight Media Framework) v 2.5 or higher.

Posted in Silverlight tips and tricks | Tagged , , , , , , , , , | 10 Comments »

Leveraging the Silverlight ecosystem with Windows Phone 7

November 20, 2010 by Tim Greenfield

There are a lot of reasons for developers to be excited about Windows Phone 7. First and foremost, .NET developers can easily build great apps using the language and tools they already know. If you’re like me, learning a new technology is always fun, but producing great results with minimal effort is better. Smile

But there’s another thing that developers and stakeholders alike should be excited about that goes beyond the benefit of the minimal learning curve: leveraging the Silverlight eco-system.

Here’s a story, straight from the trenches, of the fantasic combination of code-reuse and redesign to maximize the benefits of the Silverlight ecosystem.

Quick history

In the summer of 2009 I created the award winning site: PuzzleTouch.com. The idea for PuzzleTouch was to take an existing and familiar concept (online jigsaw puzzles) and use Silverlight’s powerful runtime and graphics engine to create the world’s best online jigsaw puzzle app. Today, PuzzleTouch is a culmination of over a year of effort, refinement, and performance improvements.

Along came Windows Phone 7

screenshot1screenshot2

*Click images to play videos

The decision to port PuzzleTouch to Windows Phone 7 was a no-brainer. First, as a Silverlight developer I already knew 95% of what I needed to know to build a WP7 app. But more importantly, I already had an established code base to start with.

As Tony Taglialucci from Carlitos Way said, “The contract’s already down on you, pal. The guys, the guns,the lime pit’s already dug. And from in here, just one button I push.”

A little over a month ago, I sat down to see if I could get the main PuzzleTouch game play screen to run on WP7. Not to my surprise but much to my excitement, a couple hours later I was dragging puzzle pieces across the WP7 emulator and snapping them together! Here’s the significance: I’ve been dying to build a version of PuzzleTouch that ran on a phone since day 2. I’ve had many users ask if an iPhone version was in the works and for the past year I’ve been toying with the possibility of making the investment. The primary drawback was purely a matter of time… there was learning Objective C (or risking developing with MonoTouch), learning the new tools, and messing with my web services. And then there was still the question, would it even work? What if there was some insurmountable technical hurdle like no support for PolyBezier clipping or unusably poor performance. The learning process would be fun but let’s face it: my free time is precious.

With WP7, I had a working prototype in 2 hours; hot damn! Realizing the hard part was already done, I spent the rest of my effort building a “real” Windows Phone 7 app and not just a Silverlight web app port. Here were some of the challenges and considerations I faced:

Technical Considerations

Have you ever heard the phrase: “It takes 80% of the time to complete the last 20% of the project”? While my initial port to WP7 was an enormous time-saver, there was a little more to it than adding a few new phone related features and dropping a few web specific features. I won’t get into the gory details on everything that needed changing but here’s a list of the major areas that needed attention from a technical perspective.

  • Pixel shaders: Silverlight on Windows Phone 7 currently doesn’t not support pixel shaders. A big reason PuzzleTouch for the web looks so realistic has a lot to do with shadows and bevels. Fortunately, I was able to mimic dropshadows by adding a second semi-transparent puzzle piece behind the top one to get me one giant step closer to great looking puzzle pieces.

shadows

*Screen shot from actual Windows Phone 7 app.
  • Navigation: Had I used the Silverlight Navigation Framework on my website, this may have gone faster. With WP7 you really need to think in terms of pages and be aware of passing around application state. Implementing support for navigation requied modest effort but overal wasn’t not too significant in the big picture.
  • Tombstoning: Speaking of application state, it is important to understand that your app can be interrupted at any time by the user receiving and viewing a text message, hitting the search button, …etc. The app needs to be able to save its state when the user navigates away and restore it when they come back. This is not hard to do and there is a great API to help, but it is one you can’t overlook.
  • Performance: The PuzzleTouch web version was already finely tuned to support up to 1000 piece puzzles. However, WP7 devices are even less powerful than a netbook and it was necessary for me to revisit performance from top to bottom. In the end hardware acceleration done right saved the day.
  • AppBar: Another consideration was effectively using the WP7 AppBar.

appbar

The AppBar is the perfect venue for your “extra” options and help save space. Speaking of which…

  • Real-estate: Last but certainly not least was the consideration of a much smaller screen. I had to move features to subsequent screens and really think hard about how to maximize the use of the screen to play something that normally requires a considerable amount of space. After all, when was the last time you played a jigsaw puzzle on anything but your dining room table?

Design Considerations

BUT… the technical changes were only part of the story. I cannot stress enough that building a great WP7 app is much more than building a high performance Silverlight web app for a small screen. I wanted PuzzleTouch to look and feel like it was made for the phone… and not just any phone, a Windows Phone. With the help of the new panorama control I completely overhauled the part of the app leading up to the puzzle game play screen. The panorama control was a perfect fit for PuzzleTouch and the funny part is, when it was all said and done, I felt like I had ended up with something possibly even more intuitive and streamlined than in the web version. I was expecting my design choices to be full of compromises but instead it was full of gains. Personally, I think this speaks well to the Metro design paradigm in general. In fact, it makes me wonder if we will start seeing more and more webpages using the metro design in due time. I for one am considering passing on some shifts in design onto the web version. Which leads nicely to my next point…

Vice versa and versa vice

Just like how Silverlight vNext is expected to benefit from the work done on Silverlight for the Windows Phone (e.g. composition thread!), web and desktop apps are likely to benefit from some of the dev and design improvements receiving spotlight from the phone today. I would speculate to say that the average Silverlight developer is more aware of hardware acceleration, frame-rate, multi-theading, and assembly load time performance now than they were before WP7. That is awesome! Silverlight is such a high performance runtime that it is easy to be lazy, but if as developers we learn more from the phone runtime and even make some minor changes to our coding styles, it will make for even better web and desktop apps.

Another great thing in the case of PuzzleTouch was that I was not only able to share code and Xaml, but was even able to share entire assemblies. Therefore, many improvements that I made for the WP7 version have already found their way back to the web version. During the entire WP7 development process I was careful to reuse as much as possible and refactor for extensibility over the copy and alter approach. Although most of the improvements are under the hood, they are still important ones. A few that come to mind are:

  • Performance: Much of the performance tuning I did for the WP7 version was applicable to my website. Refining my bitmap caching by using features like RenderAtScale now make PuzzleTouch.com run even smoother.
  • Greater use of MVVM: One of the less mentioned benefits of MVVM is the ability to share view models with multiple views. In a single Silverlight web or desktop app it is not too common to have the need for two unique views for the same task, but when sharing code with your WP7 app, this becomes a clear win for using MVVM.
  • Styling: In cases where I had built a custom control that could be reused, I still usually needed to style it differently for WP7. Partly to adhere to themes and partly to accommodate a smaller screen size. In these cases, I was able to simply template the control or expose new styling properties. Because of this, I now have more flexible and reusable controls in my PuzzleTouch namespace.

The almighty ecosystem

Possibly the best part about being able to reuse code for the phone and the web is that they can become part of each other’s feature set. With PuzzleTouch, a user can use their phone’s camera to take a picture, turn it into a puzzle, and email it to their friends… who can click on that link and open that puzzle on the website. You might not immediatly see this as a feature of the phone app, but it is to the user. Depending on the app, being able to build an experience for the phone and the web can be invaluable combination. With Silverlight you have this potential and it’s not that hard. I urge those of you who are developing apps for the phone to not overlook this powerful potential.

To conclude, much of what you develop today for the web and desktop can be reused on Windows Phone 7 and vice versa. But don’t be afraid to redesign the UX! Make your web apps shine on the web and your phone apps shine on the phone. The good news is that by using best practices and taking advantage of all the great things in the Silverlight tool chest (Xaml, MVVM, binding, assembly sharing, MEF, …etc.), redesigning the UX while maintaining a common code base is just about as easy as it could be. Once Silverlight for WP7 and Silverlight for the Mac and PC sync up their versions (Silverlight 5!?), it will be even easier.

In the end, I was able to launch a v1.0 WP7 app in under two months in my ‘spare time’ that is arguably on par with  to a v3.0 iPhone app. Plus, the website and the phone versions now compliment each other by extending each other’s feature set and I can enjoy the holidays without slaving away at my computer. It is a win for me, win for my users, and win for the platform.

Links

Video of puzzle home page and creating a puzzle from your phone’s picture album

Video of puzzle game play screen (both regular and advanced puzzles)

PuzzleTouch Jigsaw Puzzles in the Windows Phone 7 Marketplace (zune://navigate/?appID=5309e1b0-acf3-df11-9264-00237de2db9e)
Online Jigsaw Puzzles By PuzzleTouch.com

Posted in Silverlight tips and tricks, Windows Phone 7 | Tagged , , , , , | 3 Comments »

Turn your photos into a pivot collection with PhotoPivot

August 17, 2010 by Tim Greenfield

As an amateur photographer, I have always kept my eyes open for new tools to help me search, sort, filter, and view my own photos. Microsoft Pivot (a cool new technology that lets you easily view, search, and filter data using deep zoom) seemed like a natural and obvious fit for the task.

Therefore, using Microsoft Pivot and the Pauthor open source libraries, I created  a tool that helps users create a pivot collection from the photos on their hard drive and supports filtering by all the nifty meta data embedded in those files such as shutter speed, aperture, film speed, and focal length. Plus, users end up with a great look set of deep zoom images from their photo collection!

How:

Step 1: Install Microsoft Pivot

Step 2: Install PhotoPivot beta (I’m calling it a beta because I wrote it in about 8 hours).

image

Browse for the folder containing the photos that you would like to “Pivot-ize”

Browse for the location where you would like to dump the pivot collection and all the associated deep zoom files.

Note: This can take up a lot of space

Click “Go”

Warning: If you run this on a couple hundred high resolution photos it will take ~5-10 minutes, if you run it on all 20,000 photos in your My Pictures folder… do it before you go to bed!

The Result:

In the end, assuming Microsoft Pivot is installed, it will launch the Pivot app and automatically load your collection…

image

Highlights:

  1. The end result is awesome… especially when it’s your own photos that you’re filtering and deep zooming on.
  2. This collection is totally compatible with the Microsoft Silverlight PivotViewer control which makes it super easy to publish the result on the web. All you have to do is upload the output to a web server, toss a clientconfigpolicy.xml file in the domain’s root and build a 3 minute Silverlight PivotViewer app to view them. Read my earlier blog post for more details. The only down-side: if you thought it took a long time to generate the files, wait until you try to upload them; yikes!
  3. Source code is available. Knock yourself out!

Kudos:

  • Microsoft Pivot. Seriously folks, this is where all the real work was done.
  • DeepZoom. It wouldn’t be nearly as cool without it.
  • Pauthor. And without this awesome open source library, I would never have bothered to build this. (It even uses parallel processing to generates all the deep zoom images). Great work guys!

Posted in Programming Tools, Silverlight tips and tricks | Tagged , , , , , , | 28 Comments »

Silverlight PivotViewer app from start to finish

August 2, 2010 by Tim Greenfield

The new Silverlight PivotViewer control provides a cool and powerful way to visualize and filter data straight out of the box. If you have some data with a few dozen records or more and have images associated with each record, I urge you to give it a spin and build a quick Silverlight PivotViewer app to show off and enable users to search and filter that data. The whole affair won’t take you more than an hour or two and in the end you’ll have a great looking, fun and powerful window into our data.

All you need to do is generate some special files that contain the data you want to show, create a Silverlight app that hosts the new Silverlight PivotViewer control, and point it at your data.

Here is a proof of concept that I built using data from PuzzleTouch, my online jigsaw puzzles application, that shows off all the stock puzzles available and allows them to be filtered by category, popularity, and difficulty (based on user stats).

Online Jigsaw Puzzle Catalog Viewer

Click here to view

Here’s a quick walk-through demonstrating how to build your very own PivotViewer app…

Step 1: Generate the data. This is where most of the work is (but that isn’t saying much). In the end you will end up with a number of Xml and deep zoom image files. Depending on the data, the easiest way may be to use the Pivot Collection Tool for Microsoft Excel. This is an Excel Addin that allows you to toss all your data in a spreadsheet and the addin will generate the Pivot data files from it. However, because I was gathering my data from a handful of different sources and because I find programmatically generating the data to be more interesting, I decided to forego Excel and generate it from code using the Pivot Collection Tool (a.k.a. Pauthor) open source C# library. This library provides an extremely simple API that has everything you need to easily generate all the data that the Silverlight Pivot control needs. All you need to do is add the PauthorLib project to your solution and away you go…

PivotCollection items = new PivotCollection();
items.Name = "Online Jigsaw Puzzles by PuzzleTouch";
items.BrandImage = new PivotImage("http://puzzletouch.com/logo.png");
items.Icon = new PivotImage("http://puzzletouch.com/favicon.ico");
items.FacetCategories.Add(new PivotFacetCategory("Category", 
PivotFacetType.String));
items.FacetCategories.Add(new PivotFacetCategory("Difficulty", 
PivotFacetType.Number));
items.FacetCategories.Add(new PivotFacetCategory("Popularity", 
PivotFacetType.Number));

foreach (var puzzle in StockPuzzles)
{
    PivotItem item = new PivotItem(puzzle.PuzzleId, items);
    item.Description = puzzle.Description;
    item.Name = StripExtension(puzzle.Filename);
    item.Href = string.Format("http://puzzletouch.com/?puzzle={0}",
 puzzle.PuzzleId);
    item.Image = new PivotImage
(string.Format("C:\\PuzzleTouch\\Images\\{0}.png", puzzle.PuzzleId));
    item.AddFacetValues("Category", puzzle.Category.ToString());
    item.AddFacetValues("Difficulty", puzzle.Difficulty);
    item.AddFacetValues("Popularity", puzzle.Popularity);

    items.Items.Add(item);
}

PivotCollectionBuffer source = new PivotCollectionBuffer(items);
LocalCxmlCollectionTarget target = new LocalCxmlCollectionTarget(
"C:\\PuzzleTouch\\PivotData\\puzzles.cxml");
DeepZoomTargetFilter targetFilter = new DeepZoomTargetFilter(target);
targetFilter.Write(source);

Here I am creating a PivotCollection object to store my data. I tell it which fields I want to support filtering by (called a FacetCategory), and then I loop through my records and create a  new PivotItem for each. Each PivotItem needs a name, href, and image, as well as a value for each FacetCategory.

Once I’m done populating my PivotCollection object, I export it to the file system (which produces all the files required for the pivot control to function). There are other formats you can export to and from, but for my purposes I just wanted to generate from code and produce a local copy of the Pivot data all ready to go. Note: The coolest part here is that all the deep zoom images will be created automatically from the source images! No separate tools or typing in command lines are necessary!

Please refer to the Pauthor documentation for more details on the Pauthor library.

Step 2: Upload. Once you have the pivot data files created, upload them. I chose to upload my data to Amazon S3 because it is cheap and scalable but you could easily put it in any HTTP accessible location. Just remember to drop a clientaccesspolicy.xml file in the root of the domain if the data is hosted somewhere other than where you will be hosting your Silverlight app from.

Step 3: Create your Silverlight app. This is almost too easy…

  • Insert the following xaml into your main page:
xmlns:pivot="clr-namespace:System.Windows.Pivot;
assembly=System.Windows.Pivot"

<pivot:PivotViewer x:Name=“PivotViewer” />

  • In the code behind, load the PivotViewer control with the url of the .cxml file that you uploaded to your web site:
PivotViewer.LoadCollection("http://puzzletouch.s3.amazonaws.com/
puzzles.cxml", string.Empty);

That’s it!!! You now have an application that can view your pivot data collection in all it’s glory. Deep zooming, filtering, searching…etc is all built in and working.

From there, you can go forth and customize the experience all you want but with very little work you’ll have a fully functional Pivot viewing application that will give your users a lot of powerful abilities straight out of the box.

Posted in Silverlight tips and tricks | Tagged , , , , | 14 Comments »

Rx for Silverlight at Portland CodeCamp

May 23, 2010 by Tim Greenfield

Thanks to everyone who joined me for a great discussion about my new favorite technology: Reactive Extensions (Rx) for Silverlight. As promised, here is the source code for all the projects we went over as well as my PowerPoint slides…

Source code for all projects

Power point

I hope everyone grows to love Rx as much as I do! It will turn your brain inside just out a little bit, but its a good thing. 🙂

And don’t worry, I haven’t forgotten about the padlock demo. I’ll be posting a follow up with explanation and full source soon.

Posted in Silverlight tips and tricks | Tagged , , , | 2 Comments »

XPath support in Silverlight 4 + XPathPad

April 1, 2010 by Tim Greenfield

Silverlight 4 introduces XPath support including XPathNavigator and LINQ to XML extension methods for evaluating XPath expressions on your XElement objects! For those that aren’t familiar with XPath, it is essentially a big string that contains specially formatted text that is interpreted at runtime to describe a pattern for finding nodes in an XML document. You can use it to find nodes with specific conditions much like how regular expressions finds smaller strings in a bigger string. 

The downsides to using XPaths are 1) the string describing your XPath can get quite complicated and difficult to read, and 2) because the string must be interpreted at runtime, performance is worse than alternate methods. This is why I normally discourage using XPaths and probably why Microsoft shyed away from including XPath in earlier versions of Silverlight… thereby bolstering using the better practice of LINQ to XML for searching and parsing XML documents. However, XPaths (like regular expressions) have their place and at times come in very handy! 

One particular use that I’ve come to love over the years is the ability to create and run XPath queries at runtime. You don’t have to know very much about XPaths to easily write simple ones like “/vehicles/car” to find all the ‘car’ nodes in the ‘vehicles’ node. Or slightly more complex XPaths like: “/vehicles/car[@color=’green’]” (find all car nodes with a ‘color’ attribute of ‘green’). Because an XPath is just a string, it is very easy to assemble at runtime or even allow a power users to enter it in the UI if you dare open that can of worms 🙂 However, there is one kind of user I feel perfectly comfortable allowing in the can of worms… developers. Which is why I’ve created… 

XPathPad

How often do you just want to find out very simple information about an Xml file? Maybe you want to know how many nodes are in your document or how many have a certain attribute. Maybe you want to extract the text values from certain nodes so you can use them in another application. Maybe you want to paste all the values of a certain attribute into Excel so you can manually graph them. The possibilities are endless and if you ever have to work with Xml files (especially someone else’s), a tool to easily help without writing a program can be invaluable! 

How to use it:

  1. Run the app. Note: You can also install Out of browser.
  2. Browse for any xml file…
  3. Type in an XPath for the nodes you want to find
  4. Type in another XPath in the Output field for what stuff in each of those nodes you want to display.
  5. Click Go

 

Note: You can optionally enter a comma delimited list of XPaths in the output field if you want to get multiple things about each node found. Multiple results per node are separated by tabs so you can easily copy and paste the results into Excel and get different columns for each.

How it works under the hood:

Silverlight 4’s new XPath features did just about all of the work for me. First, I had to add a reference to the new XPath assembly:

Then, I simply load a standard XDocument from a filestream and call the new extension method that is available:

IEnumerable<XElement> elements = xdoc.XPathSelectElements(XPath);

This returns all the elements found at the provided XPath. I then loop thru each of these elements and call a second extension method to get a node relative to that element based on the output XPath:

var result = element.XPathEvaluate(outputXPath) as IEnumerable;

XObject value = result.Cast<XObject>().FirstOrDefault();

if (value is XElement)

    return ((XElement)value).Value;

else if (value is XText || value is XCData)

    return ((XText)value).Value;

else if (value is XAttribute)

    return ((XAttribute)value).Value;

else

    return value.ToString();

XPathEvaluate returns a collection of XObjects. I simply take the first one and return its value to be displayed in the results window.

To find out more about how XPathPad was written or to modify it to better fit your needs, download the source.

Posted in Programming Tools, Silverlight tips and tricks | Tagged , , , , , | Leave a Comment »

« Newer Posts - Older Posts »