The other side of the moon

A few months ago, Eric Miraglia from the YUI team helped me run some analysis on the types of network connections that were coming in to YUIBlog. In this article on the YUI blog, I've published the results of my analysis.

Please leave comments on the YUIblog.

I'll be speaking at FOSDEM 2010 in Brussels, Belgium on the 6th and 7th of February. I'll be speaking about the YUI Flot library. Registration is free, so if you're in the area, just show up. If anyone's interested in a performance BoF, let's do that as well.

One of the cool things about working at Yahoo! is that you get to see and play with a lot of little toys before the rest of the world does. YUI was one such tool.

I started playing with YUI while it was still in version 1, and its API was much different from what it looks like now. Among the toys I'd made was a bookmarklet to add drag and drop to any website. I used it on our internal sites until YUI was made publicly available.

A couple of years ago, I published instructions for adding drag and drop to any website, but that still needed a little technical know-how on the user's part.

So, to remedy all that, and to bring us up to date with YUI 2.6.0, I've rewritten the bookmarklet, and hosted it on yui.bluesmoon.info (no, there's nothing else there). It's far simpler, than the earlier version, and without further ado, here it is: Make Draggable.

Simply drag that link to your bookmarks toolbar, and you're ready to use it.

Now, if you click on the bookmark (in your bookmarks toolbar) when you visit a website, most sections of the page should become draggable. Enjoy yourself rearranging your favourite pages.

Let me know if it doesn't work for you, and let me know of additional features that you'd like to see. Yes, remembering your past state would be cool, but is probably not something I want to do right now.

I use Yahoo!'s yui libraries a lot in my DHTML applications. I've found the combination of YAHOO.util.Dom.getElementsByClassName with YAHOO.util.Event.addListener to be particularly convenient for event attachment. It almost allows you to do true class based object oriented programming, though there are probably better methods to do that.

In any case, I like yui, and I use yui, and while working on one of my apps, I saw performance problems trying to add a large number of event listeners to input elements by class name. My initial implementation was something like this:

for each className:
    YAHOO.util.Event.addListener(
        YAHOO.util.Dom.getElementsByClassName( className, 'input', container ),
        'eventname',
        eventHandler
    );

The obvious bottleneck here was that getElementsByClassName was calling container.getElementsByTagName('input') and iterating through the elements once for each className. An O(n²) algorithm.

Even if getElementsByClassName cached the result of getElementsByTagName, that would not have made much difference because
that call is lightening fast compared to the iteration (over about 1200 elements).

My solution was to create a map of classes to event handlers, and write my own function that ran through the element loop only once to attach all events.

It works something like this:

var classHandlers = {
    "class1":{"click":[c1ClickHandler1, c1ClickHandler2], "change":c1ChangeHandler1}
    "class2":{"click":c2ClickHandler},
    "class3":{"keyUp":c3KeyUpHandler}
};

addListenersByClassName( classHandlers, "input", "my_container" );

function addListenersByClassName( classHandlerMap, tagName, container )
{
    container = YAHOO.util.Dom.get(container);
    if(!container)
    return false;

    var els = container.getElementsByTagName(tagName);

    for(var i=els.length-1; i>=0; i--)
    {
        if(els[i].className)
        {
            var classes = els[i].className.split(/ +/);
            for(var j=classes.length-1; j>=0; j--)
            {
                if(classHandlerMap[classes[j]])
                {
                    var handlerMap = classHAndlerMap[classes[j]];
                    for(var ev in handlerMap)
                    {
                        var handlers = handlerMap[ev];
                        if(typeof(handlers) == "function")
                            handlers = [handlers];
                        for(var k=handlers.length-1; k>=0; k--)
                        {
                            YAHOO.util.Event.addListener(els[i], ev, handlers[k]);
                        }
                    }
                }
            }
        }
    }
}

On the face of it, this looks like an O(n³) algorithm, but if you consider that the average values of j and k are 1.2 and 1.1 respectively (for my application), this really just reduces to an O(n) algorithm.

I suspect that for most applications, j (number of events to handle per element) would never cross 2, and the value of k (number of handlers per event per element) would not cross 1.

I did see significant performance improvements with this, however, IE is still dead slow in iterating through a list of elements.

I've seen a few questions about doing pagination with AJAX, and I don't like the thought of it. It smells of accessibility problems. I've addressed this issue before in my own toy pages (since I don't actually write production web code), so thought I'd share it.

This is a sort of continuation of my post on progressive enhancement.

Problem definition

Given a long list of data, display it to the user in pages to avoid scrolling. Typically you'd have a bunch of navigation links at the end with First, Last, Next, Previous links, or links to specific pages.

Six steps from vanilla HTML pages to AJAX pages

It's important to note that the problem definition does not mention AJAX, but people always like to make their apps buzzword compliant. So, forget about AJAX for the moment and concentrate on solving the problem — retrieve a database resultset in limited sized pages. Once we've done that, it's five more steps to accessible AJAXified pages:

1. Build the page as you would for html only pagination
2. When your pagination links load, attach onclick handlers to them.
3. The onclick handler makes an asyncRequest to this.href + '&js=1' (or something similar)
4. Modify your backend code to check for js=1 in the query string.
   
   If not found, then send the entire page with header and footer as before
   If found, then send one of the following:
   • The html for the paged data
   • XML for the paged data
   • A JSON object representing the paged data
5. In your callback for the asyncRequest, do one of the following:
   • Put the html into the innerHTML of your page's container object
   • Parse the XML and translate it to the DOM objects for your paged data
   • eval() the JSON and redraw the DOM for the paged data
6. Rewrite the hrefs in your paging links to point to new pages.

You now have pagination that works with and without javascript enabled.

The Code

Let's look at some of the code. I'll use the yui utilities for connection and event management since I've been playing with that.

For simplicity, I'll assume that we're representing our data as a <LI>st. A table is similar, except that you need to redraw the entire table since it's read-only in IE.

Step 1: Build HTML (PHP with MySQL)

<div id="page">
<ul>
<?php

// Fetch all results and print them
while($o = mysql_fetch_array($result, MYSQL_ASSOC))
{
?>
<li><?php print $o['name'] ?></li>
<?php
}
?>
</ul>
<?php

// Verify next/last page links
$prev_page = ($pg<=0?0:$pg-1);
$next_page = ($pg>=$num_pages?$num_pages:$pg+1);

// Display navigation links, disable (via css) links that cannot be selected
?>
<p class="navbar">
<a id="first-link" href="foo.php?pg=0"
class="<?php if($pg == 0) echo 'disabled' ?>">First</a>
<a id="prev-link" href="foo.php?pg=<?php print $prev_page ?>"
class="<?php if($pg == 0) echo 'disabled' ?>">Prev</a>
<a id="last-link" href="foo.php?pg=<?php print $num_pages ?>"
class="<?php if($pg == $num_pages) echo 'disabled' ?>">Last</a>
<a id="next-link" href="foo.php?pg=<?php print $next_page ?>"
class="<?php if($pg == $num_pages) echo 'disabled' ?>">Next</a>
</p>
</div>

Step 2: Attach onclick handlers

var nav_links = ['first-link', 'prev-link', 'next-link', 'last-link'];

YAHOO.util.Event.addListener(nav_links, 'click', navigationHandler);

Step 3: Make async call:

var callback =
{
success: gotResponse,
failure: failedResponse
}

var navigationHandler = function(e)
{
var url = this.href + '&js=1';

YAHOO.util.Connect.asyncRequest('GET', url, callback, null);

YAHOO.util.Event.preventDefault(e);
return false;
}

Step 4: Modify back end to check for js=1:

<?php
$js = $_GET['js'];

if($js)
{
header('Content-type: text/json');
}
else
{
?>
<div id="page">
<ul>
<?php
}

$json = array('n'=>$num_pages, 'p'=>$pg, 'l' => array());
// Fetch all results and print them
while($o = mysql_fetch_array($result, MYSQL_ASSOC))
{
if($js)
{
$json['l'][] = $o['name'];
}
else
{
?>
<li><?php print $o['name'] ?></li>
<?php
}
}

if($js)
{
print json_encode($json);

// nothing more to output, so quit
exit();
}
?>
</ul>

I've hilighted the code that changed, it's just a bunch of if conditions. Yeah, it's ugly, but cleaning it up is not the purpose of this article.

Step 5: Make your asyncRequest handler:

var gotResponse = function(o)
{
var json = eval("(" + o.responseText + ")") ;

var list = json['l'];
var num_pages = json['n'];
var page = json['p'];

var prev_page = (page<=0?0:page-1);
var next_page = (page>=num_pages?num_pages:page+1);

var lis="";
for(var i=0; i<list.length; i++)
{
lis += "<li>" + list[i] + "</li>\n";
}

var ul = document.getElementById('page').getElementsByTagName('ul')[0];
ul.innerHTML = lis;

Step 6: Rewrite paging urls:

var fl = document.getElementById('first-link');
var pl = document.getElementById('prev-link');
var nl = document.getElementById('next-link');
var ll = document.getElementById('last-link');

var url = fl.href.substr(0, fl.href.indexOf('pg=')+3);

pl.href = url + prev_page;
nl.href = url + next_page;
ll.href = url + num_pages;

fl.className = pl.className = (page<=0?'disabled':'');
nl.className = ll.className = (page>=num_pages?'disabled':'');

}

Steps 5 and 6 are the same function of course, so don't split them up.

A brief explanation

Well, there you have it. If javascript is disabled, the default <A> behaviour is to make a GET request to foo.php with default values for pg. On every page call, the back end changes the value of pg in the Next and Previous links, and possibly in the Last link if records in the database have changed.

If javascript is enabled, we prevent the default href from being called with our return false;, and instead make the same call using asyncRequest, but with an additional query parameter saying that we want a javascript (json) object back.

The back end php script still hits the database as usual, and gets back a result set, which it now builds into a PHP hash, and then converts to a JSON object. The JSON object is sent back to the client where it is converted into HTML to push into the <UL>.

The page and num_pages variables allow us to rewrite the hrefs so that they point to up to date paging links, that you can, in fact, bookmark.

Improvements

To make the code cleaner, you may want to build your PHP hash at the start, and then based on the value of $js, either convert it to HTML or to JSON. This of course has the disadvantage of having to iterate through the array twice. If you're just looking at 20 records, I'd say it was worth it, and a better approach if you start off that way.

This is quite a simple implementation. You could get really creative with javascript, showing funky page transitions that keep the user busy while your asyncRequest returns.


Update: json_encode is available from the PHP-JSON extension available under the LGPL.

Update 2: The cleaner way to write the PHP code that I mentioned in Improvements above is something like this:

<?php
$js = $_GET['js'];

if($js)
header('Content-type: text/json');

$list = array();
while($o = mysql_fetch_array($result, MYSQL_ASSOC))
$list[] = $o['name'];

if($js)
{
$json = array('n'=>$num_pages, 'p'=>$pg, 'l' => $list);
print json_encode($json);

// nothing more to output, so quit
exit();
}
else
{
?>
<div id="page">
<ul>
<?php
foreach($list as $name)
{
?>
<li><?php print $name ?></li>
<?php
}
?>
</ul>
<?php

// Verify next/last page links
$prev_page = ($pg<=0?0:$pg-1);
$next_page = ($pg>=$num_pages?$num_pages:$pg+1);

// Display navigation links, disable (via css) links that cannot be selected
?>
<p class="navbar">
<a id="first-link" href="foo.php?pg=0"
   class="<?php if($pg == 0) echo 'disabled' ?>">First</a>
<a id="prev-link" href="foo.php?pg=<?php print $prev_page ?>"
   class="<?php if($pg == 0) echo 'disabled' ?>">Prev</a>
<a id="last-link" href="foo.php?pg=<?php print $num_pages ?>"
   class="<?php if($pg == $num_pages) echo 'disabled' ?>">Last</a>
<a id="next-link" href="foo.php?pg=<?php print $next_page ?>"
   class="<?php if($pg == $num_pages) echo 'disabled' ?>">Next</a>
</p>
</div>
<?php
}
?>

Update: I've put up a working example on sourceforge.

Have you ever visited a website and wondered, "Man, I wish I could drag that stuff out of the way"? Well, it really shouldn't be that hard. I've been doing this for a while on my own, and thought I'd share it.

To make it easy to use, I'd suggest you make a bookmarklet out of it that can be clicked on when you get to a page.

To start, download Yahoo's yui utilities:
Zip file. Unzip it into a convenient directory on your computer.
For this particular hack, I'd suggest putting the following files into the same directory:

• YAHOO.js - from any of the subdirectories
• event/build/event.js
• dom/build/dom.js
• dragdrop/build/dragdrop.js

Next, create your own script file, that looks like this:

function add_drag_drop(element)
{
   if(!element)
      element="div";
   var divs = document.getElementsByTagName(element);
   for(var i=0; i<divs.length; i++)
   {
      var id = "div-" + i;
      if(divs[i].id)
         id=divs[i].id;
      else
         divs[i].id = id;

      var dd = new YAHOO.util.DD(id);
   }

   return "Added drag-drop to " + i + " " + element + "s";
}

I'll call it adddragdrop.js. Use a javascript prompt at the top instead if you'd like to be prompted for the element's tag name.

The only thing this function does is iterate through all elements of the specified type, adding drag drop to them. If elements don't have an id, an id is added. Notice that it takes more lines of code to add ids than it does to add drag and drop. That's why I love this library.

You now need to add these five files to the page you're viewing. You can do it via the browser URL bar like this:

javascript:void(_s=document.createElement("script"), _s.src="https://tech.bluesmoon.info/YAHOO.js", document.body.appendChild(_s);
javascript:void(_s=document.createElement("script"), _s.src="https://tech.bluesmoon.info/event.js", document.body.appendChild(_s);
javascript:void(_s=document.createElement("script"), _s.src="https://tech.bluesmoon.info/dom.js", document.body.appendChild(_s);
javascript:void(_s=document.createElement("script"), _s.src="https://tech.bluesmoon.info/dragdrop.js", document.body.appendChild(_s);
javascript:void(_s=document.createElement("script"), _s.src="https://tech.bluesmoon.info/adddragdrop.js", document.body.appendChild(_s);

Of course, use the correct path for the files in there.

Finally, call the function from your url bar like this:

javascript:alert(add_drag_drop('div'))

All <div>s on the page should now be draggable.

Ok, so it's a pain to do this over and over, so turn it into a bookmarklet or greasemonkey script. This is what
my bookmarklet looks like:

<a href='javascript:void(
_s=document.createElement("script"), _s.src="http://localhost/philip/yui/YAHOO.js", document.body.appendChild(_s),
_s=document.createElement("script"), _s.src="http://localhost/philip/yui/event.js", document.body.appendChild(_s),
_s=document.createElement("script"), _s.src="http://localhost/philip/yui/dom.js", document.body.appendChild(_s),
_s=document.createElement("script"), _s.src="http://localhost/philip/yui/dragdrop.js", document.body.appendChild(_s),
_s=document.createElement("script"), _s.src="http://localhost/philip/yui/adddragdrop.js",
_s.onload=_s.onreadystatechange = function(){alert(add_drag_drop("div"));},
document.body.appendChild(_s))'>Add drag drop to page</a>

(Whitespace added for readability)

Drag that link to your bookmarks toolbar to make it easily accessible, and of course, change the links to point to your own versions of the files.

Ok, we're ready to go now. Visit any page, click on the bookmarklet, and all divs become draggable. Is that cool or what?

If it doesn't work, let me know and I'll try and figure it out.

Anyway, now that you've got the code, play with it and show me what you can do.