Wednesday, 25 April 2012

iOS to IE10 Metro: Building Cross-Browser Plugin-Free Experiences


Rey Bango

I’ve had the good fortune of working with my friend Jonathan Sampson recently on figuring out how to help developers build plugin-free experiences. With IE10 Metro going plugin-free, it’s incredibly important to document steps to help developers provide their users with great experiences without the need for proprietary 3rd party add-ons.
If you’ve built a plug-in-free browsing experience for the iPad, a few changes will make it ready for the new IE10 plug-in-free experience on Windows 8. As more browsers adopt the plug-in-free approach, now is a good time to start thinking about it. I’ll show you how to do this in a few steps below by writing code that works well in all modern browsers.
Today we’re going to work with a MSNBC plug-in-free experience for rich media. It breaks down to two things: styles and scripts.
To modify the files of MSNBC, I will be using a proxy application known as Fiddler. You can download this tool from http://fiddler2.com. This tool allows me to modify remote files as though they were on my local machine. If you have direct access to your own site, you can ignore Fiddler, and work directly with your files. Fiddler provides a great way for testing changes without the risk of breaking your live site.

Step 1: Declare Standards mode and valid markup for modern browsers

In order to use the HTML5 elements we’ll be utilizing below, you’ll first need to ensure that you are operating in standards mode. One way to ensure this is to include the HTML5 doctype at the top of your document:
<!DOCTYPE html>

Step 2: Update your CSS vendor prefixes

The CSS language is constantly in a state of change as new features are suggested, updated, and standardized. In order to allow developers to learn how to use these new features, browser vendors typically offer experimental implementations via prefixed properties.
A key part of using vendor prefixes responsibly is to ensure that prefixes from each vendor are included in your site to allow for the broadest level of feature support. In many cases, especially when building an iPad-centric site, you may have focused solely on -webkit properties, omitting the prefixes which target other browsers such as -o, -ms, and -moz. The end result of this is that you greatly limit the target devices that can render your plugin-free site to as well as provide a degraded experience for users of other modern browsers, many of which could serve up equally engaging functionality.
For instance, we find the following on MSNBC:
background: -webkit-gradient(
  linear,
  left top,
  left bottom,
  color-stop(1, rgba(192,192,192,.6)),
  color-stop(0.5, rgba(0,0,0,.6))
);
With the growing trend towards an HTML5 plugin-free experience, it’s important to expand these rules to provide the vendor prefixes of other major browsers as well.
background: -webkit-linear-gradient(
  top, rgba( 0, 0, 0, 0.0 ) 0%, rgba( 0, 0, 0, 0.6 ) 50% );
background: -moz-linear-gradient(
  top, rgba( 0, 0, 0, 0.0 ) 0%, rgba( 0, 0, 0, 0.6 ) 50% );
background: -ms-linear-gradient(
  top, rgba( 0, 0, 0, 0.0 ) 0%, rgba( 0, 0, 0, 0.6 ) 50% );
background: -o-linear-gradient(
  top, rgba( 0, 0, 0, 0.0 ) 0%, rgba( 0, 0, 0, 0.6 ) 50% );
background: linear-gradient(
  top, rgba( 0, 0, 0, 0.0 ) 0%, rgba( 0, 0, 0, 0.6 ) 50% );
While more verbose but the benefits to broad browser feature support certainly outweigh the extra typing involved. In addition, there are a number of great tools that can break down this workload, such as SASS and Compass, -prefix-free, or even CSS Snippets in the upcoming Visual Studio 2011.
Also, if you’re working predominantly in JavaScript and would like to save time determining which features are supported by your client’s browser, review the instructions in A Best Practice for Programming with Vendor Prefixes on the IEBlog.

Step 3: Get rid of browser sniffing methods

There are two methods used to determine what the user’s browser and device are capable of. One method, which unfortunately is somewhat popular, is browser sniffing. This method consists of examining the navigator object for certain patterns or values.
if ( navigator.userAgent.indexOf("iPad") > -1 ) {
  // Load HTML5 Experience
} else {
  // Load Flash Experience
}
The above code looks at the user agent string for the value “iPad”, and if found delivers a plug-in-free HTML5 experience. Otherwise, it’s assumed you are on a device that has Flash installed. This will result in a broken experience for non-iPad users who are browsing with plug-ins disabled, even though their browser is capable of handling HTML5 features.
Here is an attempt to find the version of Internet Explorer.
if ( tests.IE ) {
  j = /msie.(\d\.\d+)/i;
  k = navigator.userAgent.match(j)[1];
}
The user agent string is tested for a pattern that attempts to target the version number. This pattern looks for a single digit, followed by a period, followed by any number of additional digits. While this test will find values like “MSIE 8.0” and “MSIE 9.0”, it will not identify the latest version of Internet Explorer, which identifies itself as “MSIE 10.0”, since only one digit is expected before the period.
These are just a couple examples of why browser sniffing is not a best practice. The user agent string is not immutable – it is a read-write value that is easily changed by plugins, or even the user. Most modern browsers include the ability to easily change this value from their development tools, which some users take advantage of to get around poorly-developed websites.
If we disable plugins, or visit MSNBC from a device/browser that doesn’t have Flash, we would expect it to attempt a plug-in-free experience. Unfortunately, this is not the case. Rather than seeing an HTML5 experience, we’re instead asked to download Flash. This is because the site puts the user in one of two categories: an iPad user, or a Flash-enabled user.

Feature Detection

Rather than trying to guess what a browser is capable of by sniffing its user agent string (which will fail you eventually), it is much wiser to actually test features directly in the browser. If you wanted to test the browser’s ability to deliver video and audio via HTML5, you could actually attempt to create these elements via JavaScript, and see if the browser understands them. This practice is called feature detection.
if ( !!document.createElement(“video”).canPlayType  ) {
  // Load HTML5 Video
} else {
  // Load Flash Video
}
In the above example, we start by testing whether the canPlayType method exists on our newly-created video tag. We’re using double-negation to cast the response to a boolean. If the browser understands what a video element is, the canPlayType method will be present. If the video element is unknown to the browser, the canPlayType method will not exist. If this test passes, we load our HTML5 video. If the test does not pass, we attempt to load Flash. Deeper feature detection could take place here, since Flash may not be on the machine, or may be disabled.
Feature detection is the preferred method of determining what a browser is capable of, since there is no guesswork involved. If the browser passes properly-constructed tests, it absolutely supports the features you would like to use.
Many great tools exist to provide feature tests for you. Once such tool, which provides over 40 tests, is Modernizr. This tool creates a global object called “Modernizr” which contains the results of your tests. With Modernizr, testing for HTML5 video support is extremely easy:
if ( Modernizr.video ) {
  // Load HTML5 Video
}
MSNBC engages in browser sniffing to see if the device accessing the page is an iPad or not. Our first step is to remove the browser sniffing code, and replace it with feature detection code.
Before we can modify browser sniffing code, we first need to locate it. While in Internet Explorer, pressing F12 will pull up our Developer Tools. Within the tools, open the Script tab and do a search for “userAgent”. This search will seek out any instance of this property name in all of the site’s script files. We’re interested in the result from line 41 of http://www.msnbc.msn.com/id/37156949/.

Now that we know what we want to edit, we can open up Fiddler and load up our traffic. Once Fiddler is opened, perform a hard-refresh (Ctrl+F5 in IE) on the MSNBC page. This results in all of the page sessions being listed in Fiddler.

Looking carefully, you’ll notice our resource is the third from the top. Next I will setup an AutoResponder for this session file so that anytime it is requested, my own custom file is substituted in the place of the server response:
  1. Right-click this session and select “Decode Selected Sessions” from the context menu.
  2. Select the AutoResponder tab on the right.
  3. Click the “Enable automatic responses” checkbox in the AutoResponder tab.
  4. Drag the selected session from the left panel into the AutoResponder tab.
At this point, you should have an entry within your AutoResponder tab with the following rules:
  • If URI matches: EXACT:http://www.msnbc.msn.com/id/37156949/
  • Then respond with: *200-SESSION_3
Right-click the entry in the AutoResponder and select Edit Response. In the popup that follows, switch to the SyntaxView tab where we will find the source for this file. As expected, line 41 contains our browser sniffing code:
if(!(navigator.userAgent.toLowerCase().indexOf("ipad")>-1)){
  // Flash Experience
}
Rather than test the contents of the userAgent, we’re going to instead look for support for the HTML5 video tag. Switch the above condition to the following:
if ( !document.createElement("video").canPlayType ) {
  // Flash Experience
}
This test checks to see if we cannot use the video element. If canPlayType comes back as undefined, it will be cast to true and the first code block will be entered, setting up the Flash experience.

Step 4: Update touch and pointer events

Safari supports both a touch event model and a mouse event model. Internet Explorer 10 groups touch, mouse, and stylus events into a single abstract item known as a pointer. In fact, Internet Explorer 10 is the first browser to work for all input types, across all devices. This abstraction cuts down drastically on the amount of effort involved to determine which event model you ought to bind to and how to detect user-interaction. This pointer is then handled through MSPointer events. If necessary, you can determine the type of pointer by accessing the pointerType property.

Due to the fact Internet Explorer doesn’t support Apple’s proprietary event model, which includes touch events like touchstart, touchmove, and touchend, MSNBC’s event listeners will need to be amended to listen for MSPointer events like MSPointerDown, MSPointerUP, and MSPointerMove.
Due to the difference in event model implementations, use a feature detection tool like Modernizr or code like this to target all major event models:
if (window.navigator.msPointerEnabled) {
  myCanvas.addEventListener("MSPointerMove", paint, false);
} else {
  myCanvas.addEventListener("mousemove", paint, false);
  myCanvas.addEventListener(“touchmove”, paint, false);
}
MSNBC only supports touch events, which we will need to change so that visitors who happen to be using a mouse can still interact with the page:
Our events are tied up in http://www.msnbc.msn.com/id/43662671/15:
document.addEventListener("touchstart", touchHandler, false);
document.addEventListener("touchmove", touchHandler, false);
document.addEventListener("touchend", touchHandler, false);
We’re going to update this to include the MSPointer events as well:
if (window.navigator.msPointerEnabled) {
  document.addEventListener("MSPointerDown", touchHandler, false);
  document.addEventListener("MSPointerMove", touchHandler, false);
  document.addEventListener("MSPointerUp", touchHandler, false);
} else {
  document.addEventListener("touchstart", touchHandler, false);
  document.addEventListener("touchmove", touchHandler, false);
  document.addEventListener("touchend", touchHandler, false);
  document.addEventListener("mousedown", touchHandler, false);
  document.addEventListener("mousemove", touchHandler, false);
  document.addEventListener("mouseup", touchHandler, false);
}
First we’re checking for the presence of pointers. Since the MSPointer covers the mouse, fingers, and pens, we don’t need anything else besides them. We fall back, if necessary, to provide both touch and mouse events.
Next we need to create cases for these event types in http://www.msnbc.com/id/44937131/. Currently, MSNBC starts with the following:
if ( event.type == "touchstart" ) {
  /* Start drag logic */
} else
if ( event.type == "touchmove" ) {
  /* Drag logic */
} else
if ( event.type == "touchend" ) {
  /* Complete drag logic */
}
We’ll modify this to listen for all of the registered event types:
if ( event.type.match( /(down|start)$/i ) ) {
  /* Start drag logic */
} else
if ( event.type.match( /move$/i ) ) {
  /* Drag logic */
} else
if ( event.type.match( /(up|end)$/i ) ) {
  /* Complete drag logic */
}
The above uses the match method and a series of regular expressions to determine which event was raised. If the event raised ends with a case-insensitive “down” or “start”, we begin our drag code. If the event ends with a case-insensitive “move”, we perform the actual drag logic itself. And lastly, if the event ends with a case-insensitive “up” or “end”, we end our dragging event. Note: other events may be caught here as well, like onresizeend and keyup. Be sure to consider this in your project.
The above is an implementation of Ted Johnson’s solution in Handling Multi-touch and Mouse Input in All Browsers.
The drag logic itself initially relies upon the event.targetTouches TouchList. This member does not exist in Internet Explorer. The drag logic attempts to gather the pageX and pageY properties from the first item in the TouchList, however in Internet Explorer these values are found directly on the event object.
var curX = event.targetTouches[0].pageX;
Using the logical OR operator, I instruct curX to hold the value of event.pageX as long as event.pageX is present on the event object. If this property is not found, look within the targetTouches list:
var curX = event.pageX || event.targetTouches[0].pageX;
If event.pageX is not found, we fall back to assigning the value of targetTouches[0].pageX to our variable.
Another important item to keep in mind is that this site initially responds to touchmove. When this event is raised while touching the playlist, the code attempts to reposition the playlist based upon your touch movement. There is no hovering when it comes to touch – you’re either touching, or you’re not.
Now that we have mouse events tied into this logic, we have introduced the possibility for hovering. So while touchmove is free to reposition our playlist when it is over the playlist, we don’t want to do the same for mousemove. In fact, we only want the mousemove event to reposition the playlist when the mouse button is pressed.
For further reading, and examples on how to target all browsers, see Handling Multi-touch and Mouse Input in All Browsers.

Testing both experiences

Recall our feature detection from earlier, how we first check to see if HTML5 video support is in the user’s browser. If it is, we give them HTML5. If it is not, we give them Flash. One easy way to test our work is to use a browser, or document mode, that doesn’t support HTML5 features. This is very easy to test with Internet Explorer:
  1. Press F12 to reveal the Developer Tools
  2. Change your Document Mode to Internet Explorer 7 Standards
  3. Refresh the page
If our feature detection condition was written properly, you should now be watching a Flash-based presentation. Switching your Document Mode back into Internet Explorer 9 Standards (or “Standards” if you’re using IE10), will return you to the HTML5 experience.

Get it Done!

Hopefully this post helps to define the types of changes that will allow your iOS site to work properly in IE10 Metro and other plugin-free environments. By including best practices such as feature detection and responsibly using vendor prefixes for great new features, you should be able to provide your users a great experience, regardless of which browser or device they’re using. To assist with testing in other plug-in-free environments, download Internet Explorer 10 (currently available only in the Windows 8 CP) and begin testing today!
Update: In the rush to get this post up, I realized that I forgot to thank and give credit to Jonathan Sampson for helping investigate and write about the great techniques mentioned above. He was a huge help in generating many of these great techniques. Thanks JS!
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)