I had a great time at Edgeconf, taking part in the Web Performance panel. We got to discuss many of the common issues that web developers face when building complex web apps, such as avoiding reflow, repaints and avoiding garbage collection.

The full video is available now HERE.

The slides are available here

All other panel videos are available HERE, or will be soon if not yet uploaded.

In many ways it’s a standard MVC application with

• declarative view definitions, so you can easily relate the code to what you see in the Web Inspector
• well defined data models which can be observed for changes
• bidirectional bindings between views and models to keep the UI in sync with the data.  That is, if a piece of data changes, the UI should just update itself.  Of course, it is also possible to add code in between any binding if some more complex processing is needed.  For example, when changing the top level objective (clicking one of the huge buttons), I wanted it to animate the change, so rather the transference of the data update is delayed.

My New Rule

One of the primary lessons I took away from this is that asynchronous data fetching is one of the leading causes of users feeling like a UI is “webby”.  That feeling where the interface suddenly, with a jerk, jumps around and changes size.

This is often caused by images loading, or by an API call returning, resulting in the UI being redrawn.

I decided on a new rule to apply to interfaces I work on:

The UI can change suddenly when the user interacts with it.  All other changes should be smooth.

What this means is if the user clicks a link or a button, it’s OK to immediately change the UI, since they can see the causation.  However if new data comes back, or a large image loads shifting the entire UI up or down, this should be smoothly animated. 

I’ve found that this both gives an interface a more polished feel, but also draws the users eye to important information.  In addition, if they were already reading something, trying to make a choice of what to click next, if that jumps even a couple of centimeters they have to start searching for it again – it may have gone below the bottom of the screen.  However if it smoothly moves there over 300ms, they continue without much interruption.

It’s important to not go overboard with this of course.  Don’t animate everything, as it’ll feel gratuitous.  I’ve found that sticking to animating only asynchronous operations leads to an application feeling both fast and professional.

Some examples in the Ads Create Flow are the large image that loads for Sponsored Stories (the bottom picture), and the entire Ad preview that loads as a result of a data callback (under Right Hand Column Preview).

One caveat is that on IE 8 & 9, you’re going to have a bad time.  I chose to not bother with janky timer based animation solutions, both in the interest of saving time, and from a healthy disdain for legacy browsers.  So, I advise to stick with CSS3 animations, and let the users of old browsers have a working but less attractive application. 

A useful command I’ve found (for the Mac and probably Linux) is the following

for k in `git branch|perl -pe s/^..//`;do echo -e `git show --pretty=format:"%Cgreen%ci %Cblue%cr%Creset" $k|head -n 1`\\t$k;done|sort -r

This prints out a list of your branches, and the length of time since you last checked them out.  It makes it really easy to find that unmemorable branch.

Of course, it’s easier if you put that command in a reusable function, e.g.

function listGitBranchesByMod() {
    for k in `git branch|perl -pe s/^..//`;do echo -e `git show --pretty=format:"%Cgreen%ci %Cblue%cr%Creset" $k|head -n 1`\\t$k;done|sort -r
}

inside some *.sh file on your PATH.

I found this somewhere on the web, and cannot find it again. If it was you, please let me know so I can link to the original. Oh, and thanks!

This is because the same Event object is reused,  with its properties being updated every time.  I presume this is to conserve memory on mobile devices, as it is both in short supply and causes too much garbage collection, the enemy of responsive UIs.

The simple solution is, rather than caching the Event object (a bad idea any as it prevents it being garbage collected on other platforms), take what information you need from it, such as touch coordinates, and cache those instead.

This is done by adding a <meta> tag to the header, like

  <meta name=”viewport” content=”initial-scale=0.5,maximum-scale=0.5,user-scalable=no,width=device-width,height=device-height,target-densityDpi=device-dpi” />

Note the 0.5 for the scale values.  This resizes the screen to take advantage of the full retina display.  Pretty.  To use the low-res version, the scale parameters are 1.

However, if you’re writing a static HTML based web app, with no backend, you can’t go sniffing the user agent to figure out whether to set the scale to 0.5 or 1 before generating the page, so you’ll have to do it in JavaScript.

Code!!

Here’s a simple script to do that, from my colleagues on the JSGameBench project at Facebook.  To use this, first put the meta tag above into the <head> of your document, with scale values of 1.

  if (window == window.top) {
        var meta_viewport = document.querySelector &&
          document.querySelector(“meta[name=viewport]“);
        if (meta_viewport && window.devicePixelRatio >= 2 ) {
          meta_viewport.setAttribute(‘content’, ‘user-scalable=no, width=device-width, height=device-height, initial-scale=0.5, maximum-scale=0.5′);
        }
  }

The script looks for the meta tag, checks if you have a hi-res Apple display, and switches the device screen scale to use all the pixels, not just one in four.

One more thing…..

If your web app uses CSS like

html, body {
  width: 100%;
  height: 100%;
}

to fit the app to the screen dimensions, and inside perhaps you have some JS based scroller, then you’ll need two further lines.  The script then becomes

if (window == window.top) {
  var meta_viewport = document.querySelector &&

    document.querySelector(“meta[name=viewport]“);
  if (meta_viewport && window.devicePixelRatio >= 2 ) {
    meta_viewport.setAttribute(‘content’, ‘user-scalable=no, width=device-width, height=device-height, initial-scale=0.5, maximum-scale=0.5′);
    document.body.style.width = ’200%’;
    document.body.style.height = ’200%’;
  }
}:

Without these extra lines, the page will only occupy one quarter of the screen, rather than all of it.

Now you have an app that looks right on older iOS devices, but still uses the hi-res screen of the iPhone4 for really sharp fonts and more.

Finally

Make sure that when designing your borders and other screen elements to use multiples of two.  E.g. have your borders be 2px, not 3px, so that when the phone divides them by two it’ll look right.

So, I decided to see what I can do about it. A little fiddling with novaterm, grep, awk and GnuPlot has helped me a lot, and hopefully it can help you too.

The image above shows the percentage memory (red) and CPU (green) usage over time of the WebAppMgr process in WebOS, which is essentially where all the apps are executed.  To get a graph like this, download the following two scripts (Mac/Linux/Cygwin only):

plotWebOS.sh

driveGnuPlotStreams.pl [Source]

You’ll probably have to make them executable, so do:

chmod +x plotWebOS.sh
chmod +x driveGnuPlotStreams.pl


Then plug in your WebOS phone or TouchPad via USB, and run

./plotWebOS.sh

and you should see a graph like the one above.  If you don’t see it, it’s likely that you don’t have GnuPlot installed, so Google how to do that for your system, it should be trivial.

For what it’s worth, I get a “Too Many Cards” error just before the process hits 60%, meaning that the other running processes are taking up the remaining ~40% of memory.

If you feel like playing around with this script and doing something else cool, let me know!

Take a look at this page:

It shows a YQL query to select the version of the app com.chofter.palmdevutils, giving the following result

{
 "query": {
  "count": 1,
  "created": "2011-10-07T14:58:53Z",
  "lang": "en-US",
  "results": {
   "p": "Version: 1.3.0"
  }
 }
}

Fetching this JSON object requires far less data that reading the entire HTML page and parsing it yourself. To see the final link that you would use, copy the link from the bottom of the Yahoo page, like this one http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22https%3A%2F%2Fdeveloper.palm.com%2Fappredirect%2F%3Fpackageid%3Dcom.chofter.palmdevutils%22%20and%0A%20%20%20%20%20%20xpath%3D’%2F%2Fdiv%5B%40id%3D%22app-price-system-version%22%5D%2Fp%5B3%5D’&format=json&callback=

Creating a regex inside and outside a loop

Declaring variables in a loop or outside it

Using hasOwnProperty or testing with undefined and void 0

Using indexOf or substring to test the end of a string

Extend Without Foreach

 

for (var i = 0; i < array.length; i++) {
array[i].match(/something/);
}


The naive assumption is that a regex is some special native thing, like a boolean. However regular expressions have a cost to construct, which you can see in this performance test – http://jsperf.com/creating-a-regex-in-and-outside-a-loop

So, if you’re using a regular expression in JavaScript more than once, declare it first and reuse it to see a dramatic performance gain


.myclass {
width: 100%;
padding: 10px;
}


then myclass will still take up 100% width of its container, instead of overflowing.

https://developer.mozilla.org/en/CSS/box-sizing

WebOS looks to be on it’s last legs, and HP has announced that it’ll never make new WebOS hardware. In the meantime, the fire sale of TouchPads has done fantastic things for us WebOS developers. The image below is a graph of the Flickr Mundo HD sales for 2011.

If only HP had anything resembling courage, rather than wimping out with the revelation that “Competition is hard? What? No one told me! Run away!!”, this sales trend might have actually continued for a long time.

Ah well, I’ll enjoy it while it lasts!

After a little digging, I found a handy script at http://forums.precentral.net/webos-development/235149-extract-build-ipkg-mac-osx.html that does the trick.

You can view the talk below [source]

The slides are embedded below.

Thanks to Dylan from SitePen for organising everything and RIM for sponsoring the event.

Now, someone has actually done it with the D3 charting library, and it’s fantastic.  Check it out at http://mbostock.github.com/d3/ex/voronoi.html

The code abstracts away most of the complexities of working with Canvas elements, and just lets you get down to what you need: your pixel manipulation algorithms.  It provides a number of test images and examples to get you started, so have fun!

Oh and yes, it’s pretty ugly

Palm/HP have a new advertising campaign aimed at getting app developers all excited, and I was very happy to be a part of it.  Here’s hoping it helps give webOS the kick it needs to show developers how simple and fun it is to develop for.

I’ve added the ability for other webOS apps to launch Flickr Mundo with a Flickr URL.  This gives a far better experience to the user than simply loading Flickr in the browser – they’ve already purchased the app, it is built natively for webOS and can do many things that the web interface cannot.

For example, loading the link

• http://flickr.com/photos/shaneosullivan will open the user ‘shaneosullivan’
• http://flickr.com/groups/palmpre will open the ‘palmpre’ group
• http://www.flickr.com/search/?q=tree will do a search for photos on Flickr

Many different patterns of Flickr URLs are supported.  However if you pass a URL that is not supported, Flickr Mundo will simply launch the browser at that URL.

To make this work just takes a few lines of code.  From within any Mojo scene assistant, simply use the followinf JavaScript code:

var url = "http://flickr.com/upload";
this.controller.serviceRequest("palm://com.palm.applicationManager", {
  method: 'open',
  parameters: {
    id: 'com.chofter.flickrmundo',
    params: {
      target: url
    }
  }
});


As you can see, you can even upload photos to Flickr using this method. So if you have some kind of photo app, would like to be able to upload to Flickr but don’t want to have to go to all the trouble of correctly authenticating a user, you can just use Flickr Mundo to do it for you.

The version of Flickr Mundo with this included, v1.4.0, was submitted for review today, Nov 14th 2010, and should hopefully be available for download in a couple of days. Many different URL patterns are supported, but if you find one that isn’t, please let me know and I’ll look into adding it.

A future version will also include the ability to specify the file to upload – v1.4.0 asks the user to pick the file, similar to the stock Photos app on webOS.

• Photo upload and download, with multiple simultaneous supported for both.
• Read & add comments to photos and photo sets
• Add/remove favourites
• Bulk editing, e.g. choose 6 photos and add tags to them all in one go, add/remove them from a set or group, add/remove person tags…. and much more.
• A great browsing experience, jump from one user to another,  open photo sets, groups and galleries.  Swipe from photo to photo.
• Auto rotated wallpapers.   All the functionality from my other app Flickr Addict is included in Flickr Mundo.  Automatically download images in the background based on many different criteria and change the phone wallpaper on a schedule chosen by you.  No more boring backgrounds!
• Context sensitive ‘Just Type’ search.  Wherever you are in the app, either hit the search icon or just start typing to search Flickr.  If you’re looking at a user, you can choose to just search that user. If looking at a photo group, you can search just that group.
• Group search, find a photo group.
• Location based search.  Find photos near you using GPS, or search for photos close to anywhere else you name.
• Fullscreen slideshows with an optional audio playlist for playing music while watching your photos.

and much, much more, all wrapped in a highly polished UI that maximises the space of a small screen to the best effect.

A special thank you goes out to Jake and all the guys on the Palm Pre group on Flickr who did huge amounts of user testing on the beta version of the app. They spotted many obscure bugs, and made many suggestions for features and tweaks, most of which have ended up in the app.  Without them Flickr Mundo wouldn’t be anywhere near as complete, functional and fun as it is.

Flickr Mundo will be submitted to the Palm App Catalog in the next 24 hours, and should be available to everyone as soon as Palm approves it!

Update: Flickr Mundo has now been submitted to the App Catalog

So what I/we learned from the exercise?

DynaTrace rocks!

To begin with, I can’t praise DynaTrace enough.  It’s a fantastic piece of profiling software that delves deep into Internet Explorer’s innards to find every little horrible Microsoft bug.  It can take a while to get used to, but that is just because it presents you with so much data, all of it useful, and anyone would have trouble finding a way to make it easily consumable.  Long story short, if you’re having performance issues with your site in IE, download DynaTrace.

Touching className makes IE cry

Pretty much every time you change the className property of a node, IE6 and IE7 reflow the page.  It doesn’t matter if the new value is the same as the old one, or whether or not the changed CSS class has any visual effect, IE will redraw the page, resulting in a lot of time being lost.

Dojo was changing className too often, for example in dojo.addClass, if the class being added was already present, it would still assign the changed value to className.  I put a fix for this into v1.5.  There were also many cases where Dojo would use a combination of dojo.removeClass and dojo.addClass to replace a class, which changed className twice instead of just once.  To fix this, I’ve added a new public api dojo.replaceClass, which only changes className once.

Dijit widgets often add CSS classes to the DOM nodes which they are creating after the nodes have been inserted into the document.  This is costly, and a large performance improvement can be gained by applying these classes earlier in the lifecycle, before inserting into the document.  Bill Keese (Dijit master) checked in the first of the fixes for this for the BorderContainer recently, and a strategy is being formulated for the many other widgets that can benefit from this.  This work should be completed for v1.6.

Risky but…..

A solution that we cannot use for Dojo, but which we found gave huge performance gains for my company’s application, was to figure out what CSS classes Dojo would apply to a node and write those classes out when generating the HTML.  For example, when creating a dijit.layout.BorderContainer, our HTML contains the CSS class dijitBorderContainer.  There is an inherent risk with this approach, as Dojo may change the CSS classes in a later release, but if you’re willing to accept the risk and the upgrade pain, you can get a nice performance boost this way.

Reading offsetLeft and offsetTop is costly

When your code reads the offsetLeft or offsetTop of a node, all pending changes to the DOM have to be performed first.  This can be very expensive – we saw a number of single operations taking over 500ms each!  Many Dijits were using the dojo.marginBox method to get the left, top, width and height properties of a node, when all they needed was the width and height.

To solve this, I introduced a new method, dojo._getMarginSize, a private method that just returns the width and height of a node.  This has resulted in huge gains in performance on IE6 & 7, especially when used with layout widgets like the BorderContainer.

Unexplained gaps in DynaTrace profiles

We hit a bit of a wall at one point when analysing the performance profiles, where there was a 1.5 second gap in the profile where it seemed like nothing was happening, but everything was paused with the CPU at 100%.

We eventually discovered that there is a horrible bug in IE7 where if you have a :hover style on something that is not an anchor tag, it causes the rendering engine to go crazy, get all confused, and push the CPU up to 100% for no reason at all.

The solution: don’t user :hover styles on anything other than <a> tags.  Ridiculous I know, but so is the IE7 rendering engine.

Give it a go!

So, the long and short of it is that Dojo/Dijit should now be a good bit snappier on IE6 and 7.  With our application we got the load time for a huge application down from 20 seconds to 5 seconds using these techniques, making IE7 far, far more usable.  Of course it just flies on any other browser, even IE8 which, all credit to MS for once, is far less buggy than the earlier incarnations.

So feel free to try out the Dojo nightlies, you should find them to be a good bit more responsive in IE6 and 7 than just a few days ago, and now that we have learned some important lessons, Dojo will be even faster for v1.6.