Single page apps in depth

singlepageappbook.com /single-page.html T his f ree book is what I wanted when I started working with single page apps. It's not an API ref erence on a particular f ramework, rather, the f ocus is on discussing patterns, implementation choices and decent practices. I'm taking a "code and concepts" approach to the topic - the best way to learn how to use something is to understand how it is implemented. My ambition here is to decompose the problem of writing a web app, take a f resh look at it and hopef ully make better decisions the next time you make one. Update: the book is now also on Github. I'll be doing a second set of updates to the book later on. Right now, I'm working a new lightweight open source view layer implementation, which has changed and clarif ied my thinking about the view layer.

Int roduct ion

Modern single page apps - an overview

Writ ing maint ainable code Implement at ion alt ernat ives: a look at t he opt ions
T he view layer T he model layer

Medit at ions on Models & Collect ions Views - t emplat ing, behavior and event consumpt ion

1. Modern web applications: an overview

Why do we want to write single page apps? T he main reason is that they allow us to of f er a more-native-app-like experience to the user. T his is hard to do with other approaches. Supporting rich interactions with multiple components on a page means that those components have many more intermediate states (e.g. menu open, menu item X selected, menu item Y selected, menu item clicked). Server-side rendering is hard to implement f or all the intermediate states - small view states do not map well to URLs. Single page apps are distinguished by their ability to redraw any part of the UI without requiring a server roundtrip to retrieve HT ML. T his is achieved by separating the data f rom the presentation of data by having a model layer that handles data and a view layer that reads f rom the models. Most projects start with high ambitions, and an imperf ect understanding of the problem at hand. Our implementations tend to outpace our understanding. It is possible to write code without understanding the problem f ully; that code is just more complex than it needs to be because of our lack of understanding. Good code comes f rom solving the same problem multiple times, or ref actoring. Usually, this proceeds by noticing recurring patterns and replacing them with a mechanism that does the same

thing in a consistent way - replacing a lot of "case-specif ic" code, which in f act was just there because we didn't see that a simpler mechanism could achieve the same thing. T he architectures used in single page apps represent the result of this process: where you would do things in an ad-hoc way using jQuery, you now write code that takes advantage of standard mechanisms (e.g. f or UI updates etc.). Programmers are obsessed with ease rather than simplicity (thank you Rich Hickey f or making this point); or, what the experience of programming is instead of what the resulting program is like. T his leads to useless conversations about semicolons and whether we need a preprocessor that eliminates curly braces. We still talk about programming as if typing in the code was the hard part. It's not - the hard part is maintaining the code. To write maintainable code, we need to keep things simple. T his is a constant struggle; it is easy to add complexity (intertwinedness/dependencies) in order to solve a worthless problem; and it is easy to solve a problem in a way that doesn't reduce complexity. Namespaces are an example of the latter. With that in mind, let's look at how a modern web app is structured f rom three dif f erent perspectives: Architecture : what (conceptual) parts does our app consist of ? How do the dif f erent parts communicate with each other? How do they depend on each other? Asset packaging : how is our app structured into f iles and f iles into logical modules? How are these modules built and loaded into the browser? How can the modules be loaded f or unit testing? Run-time state : when loaded into the browser, what parts of the app are in memory? How do we perf orm transitions between states and gain visibility into the current state f or troubleshooting?

A modern web application architecture

Modern single page apps are generally structured as f ollows:

More specif ically: Write-only DOM . No state / data is read f rom the DOM. T he application outputs HT ML and operations on elements, but nothing is ever read f rom the DOM. Storing state in the DOM gets

hard to manage very quickly: it is much better to have one place where the data lives and to render the UI f rom the data, particularly when the same data has to be shown in multiple places in the UI. Models as the single source of truth. Instead of storing data in the DOM or in random objects, there is a set of in-memory models which represent all of the state/data in the application. Views observe model changes. We want the views to ref lect the content of the models. When multiple views depend on a single model (e.g. when a model changes, redraw these views), we don't want to manually keep track of each dependent view. Instead of manually tracking things, there is a change event system through which views receive change notif ications f rom models and handle redrawing themselves. Decoupled modules that expose small external surf aces. Instead of making things global, we should try to create small subsystems that are not interdependent. Dependencies make code hard to set up f or testing. Small external surf aces make ref actoring internals easy, since most things can changes as long as the external interf ace remains the same. Minimizing DOM dependent-code . Why? Any code that depends on the DOM needs to be tested f or cross-browser compatibility. By writing code in a way that isolates those nasty parts, a much more limited surf ace area needs to be tested f or cross-browser compatibility. Crossbrowser incompatibilities are a lot more manageable this way. Incompatibilities are in the DOM implementations, not in the Javascript implementations, so it makes sense to minimize and isolate DOM -dependent code.

Cont rollers must die

T here is a reason why I didn't use the word "Controller" in the diagram f urther above. I don't like that word, so you won't see it used much in this book. My reason is simple: it is just a placeholder that we've carried into the single page app world f rom having written too many "MVC" server-side apps. Most current single page application f rameworks still use the term "Controller", but I f ind that it has no meaning beyond "put glue code here". As seen in a presentation:

"Controllers deal with adding and responding to DOM events, rendering templates and keeping views and models in sync".

WAT ? Maybe we should look at those problems separately? Single page apps need a better word, because they have more complex state transitions than a server-side app: there are DOM events that cause small state changes in views there are model events when model values are changed there are application state changes that cause views to be swapped there are global state changes, like going of f line in a real time app there are delayed results f rom AJAX that get returned at some point f rom backend operations T hese are all things that need to be glued together somehow, and the word "Controller" is sadly def icient in describing the coordinator f or all these things.

We clearly need a model to hold data and a view to deal with UI changes, but the glue layer consists of several independent problems. Knowing that a f ramework has a controller tells you nothing about how it solves those problems, so I hope to encourage people to use more specif ic terms. T hat's why this book doesn't have a chapter on controllers; however, I do tackle each of those problems as I go through the view layer and the model layer. T he solutions used each have their own terms, such as event bindings, change events, initializers and so on.

Asset packaging (or more descriptively, packaging code f or the browser)

Asset packaging is where you take your JS application code and create one or more f iles (packages) that can be loaded by the browser via script tags. Nobody seems to emphasize how crucial it is to get this right! Asset packaging is not about speeding up your loading time - it is about making your application modular and making sure that it does not devolve into a untestable mess. Yet people think it is about perf ormance and hence optional. If there is one part that inf luences how testable and how ref actorable your code is, it is how well you split your code into modules and enf orce a modular structure. And that's what "asset packaging" is about: dividing things into modules and making sure that the run-time state does not devolve into a mess. Compare the approaches below:

Messy and random (no modules)

Every piece of code is made global by def ault Names are global Fully traversable namespaces Load order matters, because anything can overwrite or change anything else Implicit dependencies on anything global Files and modules do not have any meaningf ul connection Only runnable in a browser because dependencies are not isolated

Packages and modules (modular)

Packages expose a single public interf ace Names are local to the package Implementation details are inaccessible outside the package Load order does not matter thanks to packaging Explicitly declared dependencies Each f ile exposes one module Runnable f rom the command line without a headless browser

T he def ault ("throw each JS f ile into the global namespace and hope that the result works") is terrible, because it makes unit testing - and by extension, ref actoring - hard. T his is because bad modularization leads to dependencies on global state and global names which make setting up tests hard. In addition, implicit dependencies make it very hard to know which modules depend whatever code you are ref actoring; you basically rely on other people f ollowing good practice (don't depend on things I consider internal details) consistently. Explicit dependencies enf orce a public interf ace, which means that ref actoring becomes much less of a pain since others can only depend on what you expose. It also encourages thinking about the public interf ace more. T he details of how to do this are in the chapters on maintainability and modularity.

Run-time state

T he third way to look at a modern single page application is to look at its run-time state. Run time state ref ers to what the app looks like when it is running in your browser - things like what variables contain what inf ormation and what steps are involved in moving f rom one activity (e.g. page) to another. T here are three interesting relationships here: URL < - > state Single page applications have a schizophrenic relationship with URLs. On the one hand, single page apps exist so that the users can have richer interactions with the application. Richer activities mean that there is more view state than can reasonably f it inside a URL. On the other hand, we'd also like to be able to bookmark a URL and jump back to the same activity. In order to support bookmarks, we probably need to reduce the level of detail that we support in URLs somewhat. If each page has one primary activity (which is represented in some level of detail in the URL), then each page can be restored f rom a bookmark to a suf f icient degree. T he secondary activities (like say, a chat within a webmail application) get reset to the def ault state on reload, since storing them in a bookmarkable URL is pointless. Def inition < - > initialization Some people still mix these two together, which is a bad idea. Reusable components should be def ined without actually being instantiated/activated, because that allows f or reuse and f or testing. But once we do that, how do we actually perf orm the initialization/instantiation of various app states? I think there are three general approaches: one is to have a small f unction f or each module that takes some inputs (e.g. IDs) and instantiates the appropriate views and objects. T he other is to have a a global bootstrap f ile f ollowed by a router that loads the correct state f rom among the global states. T he last one is to wrap everything in sugar that makes instantiation order invisible. I like the f irst one; the second one is mostly seen in apps that have organically grown to a point where things start being entangled; the third one is seen in some f rameworks, particularly with regards to the view layer. T he reason I like the f irst one is that I consider state (e.g. instances of objects and variables) to be disgusting and worth isolating in one f ile (per subsystem - state should be local, not global, but more on that later). Pure data is simple, so are def initions. It is when we have a lot interdependent and/or hard-to-see state that things become complicated; hard to reason about and generally unpleasant. T he other benef it of the f irst approach is that it doesn't require loading the f ull application on each page reload. Since each activity is initializable on its own, you can test a single part of the app without loading the f ull app. Similarly, you have more f lexibility in preloading the rest of the app af ter the initial view is active (vs. at the beginning); this also means that the initial loading time won't increase proportionately to the number of modules your app has. HT ML elements < - > view objects and HT ML events < - > view changes Finally, there is the question of how much visibility we can gain into the run time state of the f ramework we are using. I haven't seen f rameworks address this explicitly (though of course there are tricks): when I am running my application, how can I tell what's going on by selecting a particular HT ML element? And when I look at a particular HT ML element, how can I tell what will happen when I click it or perf orm some other action? Simpler implementations generally f are better, since the distance f rom a HT ML element/event to your view object / event handler is much shorter. I am hoping that f rameworks will pay more attention to surf acing this inf ormation.

This is just the beginning

So, here we have it: three perspectives - one f rom the point of view of the architect, one f rom the view of the f ilesystem, and f inally one f rom the perspective of the browser.

2. Maintainability depends on modularity: Stop using namespaces!

Modularity is at the core of everything. Initially I had approached this very dif f erently, but it turned out af ter ~ 20 draf ts that nothing else is as important as getting modularization right. Good modularization makes building and packaging f or the browser easy, it makes testing easier and it def ines how maintainable the code is. It is the linchpin that makes it possible to write testable, packagable and maintainable code. What is maintainable code? it is easy to understand and troubleshoot it is easy to test it is easy to ref actor What is hard-to-maintain code? it has many dependencies, making it hard to understand and hard to test independently of the whole it accesses data f rom and writes data to the global scope, which makes it hard to consistently set up the same state f or testing it has side-ef f ects, which means that it cannot be instantiated easily/repeatably in a test it exposes a large external surf ace and doesn't hide its implementation details, which makes it hard to ref actor without breaking many other components that depend on that public interf ace If you think about it, these statements are either directly about modularizing code properly, or are inf luenced by the way in which code is divided into distinct modules.

What is modular code?

Modular code is code which is separated into independent modules. T he idea is that internal details of individual modules should be hidden behind a public interf ace, making each module easier to understand, test and ref actor independently of others. Modularity is not just about code organization. You can have code that looks modular, but isn't. You can arrange your code in multiple modules and have namespaces, but that code can still expose its private details and have complex interdependencies through expectations about other parts of the code.

Compare the two cases above (1). In the case on the lef t, the blue module knows specif ically about the orange module. It might ref er to the other module directly via a global name; it might use the internal f unctions of the other module that are carelessly exposed. In any case, if that specif ic module is not there, it will break. In the case on the right, each module just knows about a public interf ace and nothing else about the other module. T he blue module can use any other module that implements the same interf ace; more importantly, as long as the public interf ace remains consistent the orange module can change internally and can be replaced with a dif f erent implementation, such as a mock object f or testing.

The problem with namespaces

T he browser does not have a module system other than that it is capable of loading f iles containing Javascript. Everything in the root scope of those f iles is injected directly into the global scope under the window variable in the same order the script tags were specif ied. When people talk about "modular Javascript", what they of ten ref er to is using namespaces. T his is basically the approach where you pick a pref ix like "window.MyApp" and assign everything underneath it, with the idea that when every object has its own global name, we have achieved modularity. Namespaces do create hierarchies, but they suf f er f rom two problems: Choices about privacy have to be made on a global basis. In a namespace-only system, you can have private variables and f unctions, but choices about privacy have to be made on a global basis within a single source f ile. Either you expose something in the global namespace, or you don't. T his does not provide enough control; with namespaces you cannot expose some detail to "related"/"f riendly" users (e.g. within the same subsystem) without making that code globally accessible via the namespace. T his leads to coupling through the globally accessible names. If you expose a detail, you have no control over whether some other piece of code can access and start depending on something you meant to make visible only to a limited subsystem. We should be able to expose details to related code without exposing that code globally. Hiding details f rom unrelated modules is usef ul because it makes it possible to modif y the implementation details without breaking dependent code. Modules become dependent on global state. T he other problem with namespaces is that they do not provide any protection f rom global state. Global namespaces tend to lead to sloppy thinking: since you only have blunt control over visibility, it's easy to f all into the mode where you just add or modif y things in the global scope (or a namespace under it). One of the major causes of complexity is writing code that has remote inputs (e.g. things ref erred to by global name that are def ined and set up elsewhere) or global ef f ects (e.g. where the order

in which a module was included af f ects other modules because it alters global variables). Code written using globals can have a dif f erent result depending on what is in the global scope (e.g. window.*). Modules shouldn't add things to the global scope. Locally scoped data is easier to understand, change and test than globally scoped data. If things need to be put in the global scope, that code should be isolated and become a part of an initialization step. Namespaces don't provide a way to do this; in f act, they actively encourage you to change the global state and inject things into it whenever you want.

Examples of bad practices

T he examples below illustrate some bad practices.

Do not leak global variables

Avoid adding variables to the global scope if you don't need to. T he snippet below will implicitly add a global variable. // Bad: adds a global variable called "window.foo" var foo = 'bar'; To prevent variables f rom becoming global, always write your code in a closure/anonymous f unction - or have a build system that does this f or you: ;(funct ion() { // Good: local variable is inaccessible from t he global scope var foo = 'bar'; }()); If you actually want to register a global variable, then you should make it a big thing and only do it in one specif ic place in your code. T his isolates instantiation f rom def inition, and f orces you to look at your ugly state initialization instead of hiding it in multiple places (where it can have surprising impacts): funct ion init ialize(win) { // Good: if you must have globals, // make sure you separat e denit ion from inst ant iat ion win.foo = 'bar'; } In the f unction above, the variable is explicitly assigned to the win object passed to it. T he reason this is a f unction is that modules should not have side-ef f ects when loaded. We can def er calling the initialize f unction until we really want to inject things into the global scope.

Do not expose implement at ion det ails

Details that are not relevant to the users of the module should be hidden. Don't just blindly assign everything into a namespace. Otherwise anyone ref actoring your code will have to treat the f ull set of f unctions as the public interf ace until proven dif f erently (the "change and pray" method of ref actoring). Don't def ine two things (or, oh, horror, more than two things!) in the same f ile, no matter how

convenient it is f or you right now. Each f ile should def ine and export just one thing. ;(funct ion() { // Bad: global names = global st at e window.FooMachine = {}; // Bad: implement at ion det ail is made publicly accessible FooMachine.processBar = funct ion () { ... }; FooMachine.doFoo = funct ion(bar) { FooMachine.processBar(bar); // ... }; // Bad: export ing anot her object from t he same le! // No logical mapping from modules t o les. window.BarMachine = { ... }; })(); T he code below does it properly: the internal "processBar" f unction is local to the scope, so it cannot be accessed outside. It also only exports one thing, the current module. ;(funct ion() { // Good: t he name is local t o t his module var FooMachine = {}; // Good: implement at ion det ail is clearly local t o t he closure funct ion processBar() { ... } FooMachine.doFoo = funct ion(bar) { processBar(bar); // ... }; // Good: only export ing t he public int erface, // int ernals can be refact ored wit hout worrying ret urn FooMachine; })(); A common pattern f or classes (e.g. objects instantiated f rom a prototype) is to simply mark class methods as private by starting them with a underscore. You can properly hide class methods by using .call/.apply to set "this", but I won't show it here; it's a minor detail.

Do not mix def init ion and inst ant iat ion/init ializat ion
Your code should dif f erentiate between def inition and instantiation/initialization. Combining these two together of ten leads to problems f or testing and reusing components. Don't do this: funct ion FooObserver() { // ... } var f = new FooObserver(); f.observe('window.Foo.Bar'); module.export s = FooObserver; While this is a proper module (I'm excluding the wrapper here), it mixes initialization with def inition. What you should do instead is have two parts, one responsible f or def inition, and the other perf orming the initialization f or this particular use case. E.g. foo_observer.js

funct ion FooObserver() { // ... } module.export s = FooObserver; and boot st rap.js : module.export s = { init ialize: funct ion(win) { win.Foo.Bar = new Baz(); var f = new FooObserver(); f.observe('window.Foo.Bar'); } }; Now, FooObserver can be instantiated/initialized separately since we are not f orced to initialize it immediately. Even if the only production use case f or FooObserver is that it is attached to window.Foo.Bar, this is still usef ul because setting up tests can be done with dif f erent conf iguration.

Do not modif y object s you don't own

While the other examples are about preventing other code f rom causing problems with your code, this one is about preventing your code f rom causing problems f or other code. Many f rameworks of f er a reopen f unction that allows you to modif y the def inition of a previously def ined object prototype (e.g. class). Don't do this in your modules, unless the same code def ined that object (and then, you should just put it in the def inition). If you think class inheritance is a solution to your problem, think harder. In most cases, you can f ind a better solution by pref erring composition over inheritance: expose an interf ace that someone can use, or emit events that can have custom handlers rather than f orcing people to extend a type. T here are limited cases where inheritance is usef ul, but those are mostly limited to f rameworks. ;(funct ion() { // Bad: redening t he behavior of anot her module window.Bar.reopen({ // e.g. changing an implement at ion on t he y }); // Bad: modifying a built in t ype St ring.prot ot ype.dasherize = funct ion() { // While you can use t he right API t o hide t his funct ion, // you are st ill monkey-pat ching t he language in a unexpect ed way }; })(); If you write a f ramework, f or f *ck's sake do not modif y built-in objects like St ring by adding new f unctions to them. Yes, you can save a f ew characters (e.g. _(st r).dasherize() vs. st r.dasherize() ), but this is basically the same thing as making your special snowf lake f ramework a global dependency. Play nice with everyone else and be respectf ul: put those special f unctions in a utility library instead.

Building modules and packages using CommonJS

Now that we've covered a f ew common bad practices, let's look at the positive side: how can we implement modules and packages f or our single page application? We want to solve three problems: Privacy: we want more granular privacy than just global or local to the current closure. Avoid putting things in the global namespace just so they can be accessed. We should be able to create packages that encompass multiple f iles and directories and be able to wrap f ull subsystems into a single closure. CommonJS modules. CommonJS is the module f ormat that Node.js uses natively. A CommonJS module is simply a piece of JS code that does two things: it uses require() statements to include dependencies it assigns to the export s variable to export a single public interf ace Here is a simple example foo.js : var Model = require('./lib/model.js'); // require a dependency // module implement at ion funct ion Foo(){ /* ... */ } module.export s = Foo; // export a single variable What about that var Model statement there? Isn't that in the global scope? No, there is no global scope here. Each module has its own scope. T his is like having each module implicitly wrapped in a anonymous f unction (which means that variables def ined are local to the module). OK, what about requiring jQuery or some other library? T here are basically two ways to require a f ile: either by specif ying a f ile path (like ./lib/model.js ) or by requiring it by name: var $ = require('jquery');. Items required by f ile path are located directly by their name in the f ile system. T hings required by name are "packages" and are searched by the require mechanism. In the case of Node, it uses a simple directory search; in the browser, well, we can def ine bindings as you will see later. What are the benefits? Isn't this the same thing as just wrapping everything in a closure, which you might already be doing? No, not by a long shot. It does not accidentally modif y global state, and it only exports one thing. Each CommonJS module executes in its own execution context. Variables are local to the module, not global. You can only export one object per module. Dependencies are easy to locate, without being modif iable or accessible in the global scope. Ever been conf used about where a particular f unction comes f rom, or what the dependencies of a particular piece of code are? Not anymore: dependencies have to be explicitly declared, and locating a piece of code just means looking at the f ile path in the require statement. T here are no implied global variables. But isn't declaring dependencies redundant and not DRY? Yes, it's not as easy as using global variables implicitly by ref erring to variables def ined under window. But the easiest way isn't always the best choice architecturally; typing is easy, maintenance is hard. T he module does not give itself a name. Each module is anonymous. A module exports a class or a set of f unctions, but it does not specif y what the export should be called. T his means that

whomever uses the module can give it a local name and does not need to depend on it existing in a particular namespace. You know those maddening version conf licts that occur when the semantics of include()ing a module modifies the environment to include the module using its inherent name? So you can't have two modules with the same name in dif f erent parts of your system because each name may exist only once in the environment? CommonJS doesn't suf f er f rom those, because require() just returns the module and you give it a local name by assigning it to a variable. It comes with a distribution system. CommonJS modules can be distributed using Node's npm package manager. I'll talk about this more in the next chapter. T here are thousands of compatible modules. Well, I exaggerate, but all modules in npm are CommonJS-based; and while not all of those are meant f or the browser, there is a lot of good stuf f out there. Last, but not least: CommonJS modules can be nested to create packages. T he semantics of require() may be simple, but it provides the ability to create packages which can expose implementation details internally (across f iles) while still hiding them f rom the outside world. T his makes hiding implementation details easy, because you can share things locally without exposing them globally.

Creating a CommonJS package

Let's look at how we can create a package f rom modules f ollowing the CommonJS package. Creating a package starts with the build system. Let's just assume that we have a build system, which can take any set of .js f iles we specif y and combine them into a single f ile. [ [./model/t odo.js] [./view/t odo_list .js] [./index.js] ] [ Build process ] [ t odo_package.js ] T he build process wraps all the f iles in closures with metadata, concatenates the output into a single f ile and adds a package-local require() implementation with the semantics described earlier (including f iles within the package by path and external libraries by their name). Basically, we are taking a wrapping closure generated by the build system and extending it across all the modules in the package. T his makes it possible to use require() inside the package to access other modules, while preventing external code f rom accessing those packages. Here is how this would look like as code:

;(funct ion() { funct ion require() { /* ... */ } modules = { 'jquery': window.jQuery }; modules['./model/t odo.js'] = funct ion(module, export s, require){ var Dependency = require('dependency'); // ... module.export s = Todo; }); modules['index.js'] = funct ion(module, export s, require){ module.export s = { Todo: require('./model/t odo.js') }; }); window.Todo = require('index.js'); }()); T here is a local require() that can look up f iles. Each module exports an external interf ace f ollowing the CommonJS pattern. Finally, the package we have built here itself has a single f ile index.js that def ines what is exported f rom the module. T his is usually a public API, or a subset of the classes in the module (things that are part of the public interf ace). Each package exports a single named variable, f or example: window.Todo = require('index.js');. T his way, only relevant parts of the module are exposed and the exposed parts are obvious. Other packages/code cannot access the modules in another package in any way unless they are exported f rom index.js . T his prevents modules f rom developing hidden dependencies.

Building an application out of packages

T he overall directory structure might look something like this: asset s - css - layout s common - collect ions - models index.js modules - t odo - public - t emplat es - views index.js node_modules package.json server.js Here, we have a place f or shared assets (./asset s/); there is a shared library containing reusable parts, such as collections and models (./common). T he ./modules/ directory contains subdirectories, each of which represents an individually initializable part of the application. Each subdirectory is its own package, which can be loaded independently of others (as long as the common libraries are loaded). T he index.js f ile in each package exports an init ialize() f unction that allows that particular package to be initialized when it is activated, given parameters such as the current URL and app conf iguration.

Using the glue build system

So, now we have a somewhat detailed spec f or how we'd like to build. Node has native support f or require(), but what about the browser? We probably need a elaborate library f or this? Nope. T his isn't hard: the build system itself is about a hundred f if ty lines of code plus another ninety or so f or the require() implementation. When I say build, I mean something that is superlightweight: wrapping code into closures, and providing a local, in-browser require() implementation. I'm not going to put the code here since it adds little to the discussion, but have a look. I've used onejs and browserbuild bef ore. I wanted something a bit more scriptable, so (af ter contributing some code to those projects) I wrote gluejs, which is tailored to the system I described above (mostly by having a more f lexible API). With gluejs, you write your build scripts as small blocks of code. T his is nice f or hooking your build system into the rest of your tools - f or example, by building a package on demand when a HT T P request arrives, or by creating custom build scripts that allow you to include or exclude f eatures (such as debug builds) f rom code. Let's start by installing gluejs f rom npm: $ npm inst all gluejs Now let's build something.

Including f iles and building a package

Let's start with the basics. You use include(pat h) to add f iles. T he path can be a single f ile, or a directory (which is included with all subdirectories). If you want to include a directory but exclude some f iles, use exclude(regexp) to f ilter f iles f rom the build. You def ine the name of the main f ile using main(name); in the code below, it's "index.js". T his is the f ile that gets exported f rom the package. var Glue = require('gluejs'); new Glue() .include('./t odo') .main('index.js') .export ('Todo') .render(funct ion (err, t xt ) { console.log(t xt ); }); Each package exports a single variable, and that variable needs a name. In the example below, it's "Todo" (e.g. the package is assigned to window.Todo ). Finally, we have a render(callback) f unction. It takes a funct ion(err, t xt ) as a parameter, and returns the rendered text as the second parameter of that f unction (the f irst parameter is used f or returning errors, a Node convention). In the example, we just log the text out to console. If you put the code above in a f ile (and some .js f iles in "./todo"), you'll get your f irst package output to your console. If you pref er rebuilding the f ile automatically, use .wat ch() instead of .render(). T he callback f unction you pass to watch() will be called when the f iles in the build change.

Binding to global f unctions

We of ten want to bind a particular name, like require('jquery') to a external library. You can do this with replace(moduleName, st ring). Here is an example call that builds a package in response to a HT T P GET: var fs = require('fs'), ht t p = require('ht t p'), Glue = require('gluejs'); var server = ht t p.creat eServer(); server.on('request ', funct ion(req, res) { if(req.url == '/minilog.js') { new Glue() .include('./t odo') .basepat h('./t odo') .replace('jquery', 'window.$') .replace('core', 'window.Core') .export ('Module') .render(funct ion (err, t xt ) { res.set Header('cont ent -t ype', 'applicat ion/javascript '); res.end(t xt ); }); } else { console.log('Unknown', req.url); res.end(); } }).list en(8080, 'localhost '); To concatenate multiple packages into a single f ile, use concat ([packageA, packageB], funct ion(err, t xt )): var packageA = new Glue().export ('Foo').include('./xt ures/lib/foo.js'); var packageB = new Glue().export ('Bar').include('./xt ures/lib/bar.js'); Glue.concat ([packageA, packageB], funct ion(err, t xt ) { fs.writ eFile('./build.js', t xt ); }); Note that concatenated packages are just def ined in the same f ile - they do not gain access to the internal modules of each other. [1] T he modularity illustration was adapted f rom Rich Hickey's presentation Simple Made Easy http://www.inf oq.com/presentations/Simple-Made-Easy http://blog.markwshead.com/1069/simple-made-easy-rich-hickey/ http://code.mumak.net/2012/02/simple-made-easy.html http://pyvideo.org/video/880/stop-writing-classes http://substack.net/posts/b96642

3. Getting to maintainable
In this chapter, I'll look at how the ideas introduced in the previous chapter can be applied to realworld code, both new and legacy. I'll also talk about distributing code by setting up a private NPM server.

Let's start with a some big-picture principles, then talk about legacy - and f inally some special considerations f or new code. Today's new code is tomorrow's legacy code, so try to avoid f alling into bad habits.

Big picture
Use a build system and a module convention that supports granular privacy. Namespaces suck, because they don't let you hide things that are internal to a package within that package. Build systems that don't support this f orce you to write bad code. Independent packages/modules. Keep dif f erent parts of an app separate: avoid global names and variables, make each part independently instantiable and testable. Do not intermingle state and def inition. You should be able to load a module without causing any side-ef f ects. Isolate code that creates state (instances / variables) in one place, and keep it tiny. Small external surface . Keep the publicly accessible API small and clearly indicated, as this allows you ref actor and reach better implementations.

Ext ract ing modules/packages f rom your archit ect ure

First, do a basic code quality pass and eliminate the bad practices outlined in the previous chapter. T hen, start moving your f iles towards using the CommonJS pattern: explicit require()s and only a single export per f ile. Next, have a look at your architecture. Try to separate that hairball of code into distinct packages: Models and other reusable code (shared views/visual components) probably belong in a common package. T his is the core of your application on which the rest of the application builds. Treat this like a 3rd party library in the sense that it is a separate package that you need to require() in your other modules. Try to keep the common package stateless. Other packages instantiate things based on it, but the common package doesn't have statef ul code itself . Beyond your core/common package, what are the smallest pieces that make sense? T here is probably one f or each "primary" activity in your application. To speed up loading your app, you want to make each activity a package that can be loaded independently af ter the common package has loaded (so that the initial loading time of the application does not increase as the number of packages increases). If your setup is complex, you probably want a single mechanism that takes care of calling the right initializer. Isolate the state initialization/instantiation code in each package by moving it into one place: the index.js f or that particular package (or, if there is a lot of setup, in a separate f ile - but in one place only). "I hate state, and want as little as possible of it in my code". Export a single f unction init ialize() that accepts setup parameters and sets up the whole module. T his allows you to load a package without altering the global state. Each package is like a "mini-app": it should hide its details (non-reusable views, behavior and models). Rethink your inheritance chains. Classes are a terrible substitute f or a use-oriented API in most cases. Extending a class requires that you understand and of ten depend on the implementation details. APIs consisting of simple f unctions are superior, so if you can, write an API. T he API of ten looks like a state manipulation library (e.g. add an invite, remove an invite etc.); when instantiated with the related views and the views will generally hook into that API. Stop inheriting views f rom each other. Inheritance is mostly inappropriate f or views. Sure, inherit

f rom your f ramework, but don't build elaborate hierarchies f or views. Views aren't supposed to have a lot of code in the f irst place; def ining view hierarchies is mostly just done out of bad habit. Inheritance has its uses, but those are f ewer and f urther apart than you think. Almost every view in your app should be instantiable without depending on any other view. You should identif y views that you want to reuse, and move those into a global app-specif ic module. If the views are not intended to be reused, then they should not be exposed outside of the activity. Reusable views should ideally be documented in an interactive catalog, like Twitter's Bootstrap. Extract persistent services. T hese are things that are active globally and maintain state across dif f erent activities. For example, a real-time backend and a data cache. But also other user state that is expected to persist across activities, like a list of opened items (e.g. if your app implements tabs within the application).

Ref act oring an exist ing module

Given an existing module, 1. Make sure each f ile def ines and exports one thing. If you def ine a Account and a related Settings object, put those into two dif f erent f iles. 2. Do not directly/implicitly add variables under window.*. Instead, always assign your export to module.export s . T his makes it possible f or other modules to use your module without the module being globally accessible under a particular name/namespace. 3. Stop ref erring to other modules through a global name. Use var $ = require('jquery'), f or example, to specif y that your module depends on jQuery. If your module requires another local module, require it using the path: var User = require('./model/user.js'). 4. Delay concrete instatiation as long as possible by extracting module state setup into a single bootstrap f ile/f unction. Def ining a module should be separate f rom running the module. T his allows small parts of the system to be tested independently since you can now require your module without running it. For example, where you previously used to def ine a class and then immediately assign a instance of that class onto a global variable/namespace in the same f ile; you should move the instantatiation to a separate bootstrap f ile/f unction. 5. If you have submodules (e.g. chat uses backend_service), do not directly expose them to the layer above. Initializing the submodule should be the task of the layer directly above it (and not two layers above it). Conf iguration can go f rom a top level initialize() f unction to initialize() f unctions in submodules, but keep the submodules of modules out of reach f rom higher layers. 6. Try to minimize your external surf ace area. 7. Write package-local tests. Each package should be have unit and integration tests which can be run independently of other packages (other than 3rd party libraries and the common package). 8. Start using npm with semantic versioning f or distributing dependencies. Npm makes it easy to distribute and use small modules of Javascript.

Guidelines f or new projects

Start with the package.json f ile. Add a single bootstrap f unction. Loading modules should not have side-ef f ects.

Write tests bef ore f unctionality. Hide implementation details. Each module should be isolated into its own scope; modules expose a limited public interf ace and not their implementation details. Minimize your exports. Small surf ace area. Localize dependencies. Modules that are related to each other should be able to work together, while modules that are not related/f ar f rom each other should not be able to access each other.

Tooling: npm
Finally, let's talk about distribution. As your projects grow in scope and in modularity, you'll want to be able to load packages f rom dif f erent repositories easily. npm is an awesome tool f or creating and distributing small JS modules. If you haven't used it bef ore, Google f or a tutorial or read the docs, or check out Nodejitsu's npm cheatsheet. Creating a npm package is simply a matter of f ollowing the CommonJS conventions and adding a bit of metadata via a package.json f ile. Here is an example package.json { "name": "modulename", "descript ion": "Foo for bar", "version": "0.0.1", "dependencies": { "underscore": "1.1.x", "foo": "git +ssh://git @git hub.com:mixu/foo.git #0.4.1" } } T his package can then be installed with all of its dependencies by running npm inst all. To increment the module version, just run npm version pat ch (or "minor" or "major"). You can publish your package to npm with one command (but do RT FM bef ore you do so). If you need to keep your code private, you can use git +ssh://user@host :project .git #t ag-sha-or-branch to specif y dependencies as shown above. If your packages can be public and reusable by other people, then the public npm registry works. T he drawback to using private packages via git is that you don't get the benef its semantic versioning. You can ref er to a particular branch or commit sha, but this is less than ideal. If you update your module, then you need to go and bump up the tag or branch in both the project and in its dependencies. T his isn't too bad, but ideally, we'd be able to say: { "dependencies": { "foo": ">1.x.x" } } which will automatically select the latest release within the specif ied major release version. Right now, your best bet is to install a local version npm if you want to work with semantic version numbers rather than git tags or branches. T his involves some CouchDB setup. If you need a read-only cache (which is very usef ul f or speeding up/improving reliability of large simultaneous deploys), have a look at npm_lazy; it uses static f iles instead of CouchDB f or simpler setup. I am working on a private npm server that's easier to set up, but haven't quite gotten it completed due to writing this book. But once it's done, I'll update this section.

4. Testing
T DD? T he best way to make code testable is to start by writing the tests f irst - T DD style. Essentially, T DD boils down to:

T DD is a set of rules f or writing code: you write a f ailing test (red), then add just enough code to make it pass (green) and f inally ref actor where necessary (ref actor). In this chapter, we discuss how to set up testing f or your project using Mocha, how to do dependency injection f or your CommonJS modules, and how you can test asynchronous code. T he rest is best covered by some other book or tutorial; so if you haven't heard of T DD, get out f rom under that rock you've been living under and read Kent Beck's book and perhaps Michael Feather's book.

Why write tests?

Test driven development is not valuable because it catches errors, but because it changes the way you think about interf aces between modules. Writing tests before you write code inf luences how you think about the public interf ace of your modules and their coupling, it provides a saf ety net f or perf orming ref actoring and it documents the expected behavior of the system. In most cases, you don't completely understand the system when you start writing it. Writing something once produces just a rough draf t. You want to be able to improve the code while ensuring that existing code does not break. T hat's what tests are f or: they tell you what expectations you need to f ulf ill while ref actoring.

What to test?
Test driven development implies that tests should guide the development. I of ten use tests as T ODO's when developing new f unctionality; no code is written until I know how the code should look like in the test. Tests are a contract: this is what this particular module needs to provide externally. I f ind that the greatest value comes f rom testing pure logic and otherwise-hard-to-replicate edge cases. I tend not to test internal details (where you test the actual implementation rather than the

public interf ace). I also avoid testing things that are hard to set up f or testing; testing is a tool, not a goal in itself . T his is why it is important to have good modularization and f ew dependencies: the easier your code is to test, the more likely it is that someone will want to write tests f or it. For views, I'd test the logic (easy to test/easy to have errors in) and try to make it so that it can be tested separately f rom any visual properties (hard to test without a human looking at stuf f ).

Test f rameworks
Use any test f ramework/runner except Jasmine, which is terrible f or asynchronous testing due to the amount of boilerplate code it requires. Test runners basically use one of three dif f erent styles f or specif ying tests: BDD: describe(foo) .. before() .. it () T DD: suit e(foo) .. set up() .. t est (bar) and exports: export s['suit e'] = { before: f() .. 'foo should': f() } I like T J's Mocha, which has a lot of awesome f eatures, such as support f or all three specif ication styles, support f or running tests in the browser, code coverage, Growl integration, documentation generation, airplane mode and a nyan cat test reporter. I like to use the "exports" style - it is the simplest thing that works. Some f rameworks require you to use their assert () methods, Mocha doesn't. I use Node's built-in assert module f or writing my assertions. I'm not a f an of the "assertions-written-out-assentences" -style; plain asserts are more readable to me since they translate trivially to actual code and it's not like some non-coder is going to go poke around in your test suite.

Setting up and writing a test

Let's set up a Node project with mocha and write a test. First, let's create a directory, initialize the package.json f ile (f or npm) and install mocha: [~] mkdir example [~] cd example [example] npm init Package name: (example) Descript ion: Example syst em Package version: (0.0.0) Project homepage: (none) Project git reposit ory: (none) ... [example] npm inst all --save-dev mocha I like the exports style f or tests: var assert = require('assert '), Model = require('../lib/model.js'); export s['can check whet her a key is set '] = funct ion(done) { var model = new Model(); assert .ok(!model.has('foo')); model.set ('foo', 'bar'); assert .ok(model.has('foo')); done(); };

Note the use of the done() f unction there. You need to call this f unction at the end of your test to notif y the test runner that the test is done. T his makes async testing easy, since you can just make that call at the end of your async calls (rather than having a polling mechanism, like Jasmine does). You can use bef ore/af ter and bef oreEach/af terEach to specif y blocks of code that should be run either bef ore/af ter the whole set of tests or bef ore/af ter each test: export s['given a foo'] = { before: funct ion(done) { t his.foo = new Foo().connect (); done(); }, aft er: funct ion(done) { t his.foo.disconnect (); done(); }, 'can check whet her a key is set ': funct ion() { // ... } }; You can also create nested test suites (e.g. where several sub-tests need additional setup): export s['given a foo'] = { beforeEach: funct ion(done) { // ... }, 'when bar is set ': { beforeEach: funct ion(done) { // ... }, 'can execut e baz': funct ion(done) { // ... } } };

Basic assertions
You can get pretty f ar with these three: assert.ok(value, [message]) assert.equal(actual, expected, [message]) assert.deepEqual(actual, expected, [message]) Check out the assert module documentation f or more.

Tests should be easy to run

To run the f ull test suite, I create a Makef ile:

TESTS += t est /model.t est .js t est : @./node_modules/.bin/mocha \ --ui export s \ --report er list \ --slow 2000ms \ --bail \ $(TESTS) .PHONY: t est T his way, people can run the tests using "make test". Note that the Makef ile requires tabs f or indentation. I also like to make individual test f iles runnable via node ./pat h/t o/t est .js . To do this, I add the f ollowing wrapper to detect whether the current module is the main script, and if so, run the tests directly (in this case, using Mocha): // if t his module is t he script being run, t hen run t he t est s: if (module == require.main) { var mocha = require('child_process').spawn('mocha', [ '--colors', '--ui', 'export s', '--report er', 'spec', __lename ]); mocha.st dout .pipe(process.st dout ); mocha.st derr.pipe(process.st derr); } T his makes running tests nice, since you no longer need to remember all those def ault options.

Testing interactions between modules

Unit tests by def inition should only test one module at a time. Each unit test excercises one part of the module under test. Some direct inputs (e.g. f unction parameters) are passed to the module. Once a value is returned, the assertions in the test verif y the direct outputs of the test. However, more complex modules may use other modules: f or example, in order to read f rom a database via f unction calls (indirect inputs) and write to a database (indirect outputs).

You want to swap the dependency (e.g. the database module) with one that is easier to use f or testing purposes. T his has several benef its: You can capture the indirect outputs (dependency f unction calls etc.) and control the indirect inputs (e.g. the results returned f rom the dependency). You can simulate error conditions, such as timeouts and connection errors. You can avoid having to slow/hard to set up external dependencies, like databases and external APIs. T his is known as dependency injection. T he injected dependency (test double) pretends to implement the dependency, replacing it with one that is easier to control f rom the test. T he code

being tested is not aware that it is using a test double. For simple cases, you can just replace a single f unction in the dependency with a f ake one. For example, you want to stub a f unction call: export s['it should be called'] = funct ion(done) { var called = false, old = Foo.doIt ; Foo.doIt = funct ion(callback) { called = t rue; callback('hello world'); }; // Assume Bar calls Foo.doIt Bar.baz(funct ion(result )) { console.log(result ); assert .ok(called); done(); }); }; For more complex cases, you want to replace the whole backend object. T here are two main alternatives: constructor parameter and module substitution.

Const ruct or paramet ers

One way to allow f or dependency injection is to pass the dependency as a option. For example: funct ion Channel(opt ions) { t his.backend = opt ions.backend || require('persist ence'); }; Channel.prot ot ype.publish = funct ion(message) { t his.backend.send(message); }; module.export s = Channel; When writing a test, you pass a dif f erent parameter to the object being tested instead of the real backend: var MockPersist ence = require('mock_persist ence'), Channel = require('./channel'); var c = new Channel({ backend: MockPersist ence }); However, this approach is not ideal: Your code is more cluttered, since you now have to write t his.backend.send instead of Persist ence.send; you now also to pass in that option though you only need it f or testing. You have to pass that option through any intermediate objects if you are not directly using this class. If you have a hierarchy where Server instantiates Channel which uses Persistence; and you want to capture Persistence calls in a test, then the Server will have accept at channelBackend option or to expose the Channel instance externally.

Module subst it ut ion

Another way is to write a f unction that changes the value of the dependency in the module. For

example: var Persist ence = require('persist ence'); funct ion Channel() { }; Channel.prot ot ype.publish = funct ion(message) { Persist ence.send(message); }; Channel._set Backend = funct ion(backend) { Persist ence = backend; }; module.export s = Channel; Here, the _set Backend f unction is used to replace the (module-local) private variable Persist ence with another (test) object. Since module requires are cached, that private closure and variable can be set f or every call to the module, even when the module is required f rom multiple dif f erent f iles. When writing a test, we can require() the module to gain access to setBackend() and inject the dependency: // using in t est var MockPersist ence = require('mock_persist ence'), Channel = require('./channel'); export s['given foo'] = { before: funct ion(done) { // inject dependency Channel._set Backend(MockPersist ence); }, aft er: funct ion(done) { Channel._set Backend(require('persist ence')); }, // ... } var c = new Channel(); Using this pattern you can inject a dependency on a per-module basis as needed. T here are other techniques, including creating a f actory class (which makes the common case more complex) and redef ining require (e.g. using Node's VM API). But I pref er the techniques above. I actually had a more abstract way of doing this, but it turned out to be totally not worth it; _set Backend() is the simplest thing that works.

Testing asynchronous code

T hree ways: Write a workf low Wait f or events, continue when expectations f ulf illed Record events and assert Writing a workf low is the simplest case: you have a sequence of operations that need to happen, and in your test you set up callbacks (possibly by replacing some f unctions with callbacks). At the end of the callback chain, you call done(). You probably also want to add an assertion counter to verif y that all the callbacks were triggered. Here is a basic example of a workf low, note how each step in the f low takes a callback (e.g.

assume we send a message or something): export s['can read a st at us'] = funct ion(done) { var client = t his.client ; client .st at us('it em/21').get (funct ion(value) { assert .deepEqual(value, []); client .st at us('it em/21').set ('bar', funct ion() { client .st at us('it em/21').get (funct ion(message) { assert .deepEqual(message.value, [ 'bar' ]); done(); }); }); }); };

Waiting f or events using EventEmitter.when()

In some cases, you don't have a clearly def ined order f or things to happen. T his is of ten the case when your interf ace is an EventEmitter. What's an EventEmitter? It's basically just Node's name f or an event aggregator; the same f unctionality is present in many other Javascript projects - f or example, jQuery uses .bind()/.t rigger() f or what is essentially the same thing. Node.js EventEmitter Attach a callback to an event Trigger an event Remove a callback Add a callback that is triggered once, then removed .on(event, callback) / .addListener(event, callback) .emit(event, data, ...) .removeListener(event, callback) .once(event, callback) jQuery .bind(eventType, handler) (1.0) / .on(event, callback) (1.7) .trigger(event, data, ...) .unbind(event, callback) / .of f (event, callback) .one(event, callback)

jQuery's f unctions have some extra sugar on top, like selectors, but the idea is the same. T he usual EventEmitter API is a bit awkward to work with when you are testing f or events that don't come in a def ined sequence: If you use EE.once(), you have to manually reattach the handler in case of misses and manually count. If you use EE.on(), you have to manually detach at the end of the test, and you need to have more sophisticated counting. EventEmitter.when() is a tiny extension to the standard EventEmitter API: Event Emit t er.when = funct ion(event , callback) { var self = t his; funct ion check() { if(callback.apply(t his, argument s)) { self.removeList ener(event , check); } } check.list ener = callback; self.on(event , check); ret urn t his; };

EE.when() works almost like EE.once(); it takes an event and a callback. T he major dif f erence is that the return value of the callback determines whether the callback is removed. export s['can subscribe'] = funct ion(done) { var client = t his.client ; t his.backend.when('subscribe', funct ion(client , msg) { var mat ch = (msg.op == 'subscribe' && msg.t o == 'foo'); if (mat ch) { assert .equal('subscribe', msg.op); assert .equal('foo', msg.t o); done(); } ret urn mat ch; }); client .connect (); client .subscribe('foo'); };

Recording events and then asserting

Recording replacements (a.k.a spies and mocks) are used more f requently when it is not f easible to write a f ull replacement of the dependency, or when it is more convenient to collect output (e.g f rom operations that might happen in any order) and then assert that certain conditions are f ulf illed. For example, with an EventEmitter, we might not care in what order certain messages were emitted, just that they were emitted. Here is a simple example using an EventEmitter: export s['doIt sends a b c'] = funct ion(done) { var received = []; client .on('foo', funct ion(msg) { received.push(msg); }); client .doIt (); assert .ok(received.some(funct ion(result ) { ret urn result == 'a'; })); assert .ok(received.some(funct ion(result ) { ret urn result == 'b'; })); assert .ok(received.some(funct ion(result ) { ret urn result == 'c'; })); done(); }; With the DOM or some other hard-to-mock dependency, we just substitute the f unction we're calling with another one (possibly via the dependency injection techniques mentioned earlier). export s['doIt sends a b c'] = funct ion(done) { var received = [], old = jQuery.foo; jQuery.foo = funct ion() { received.push(argument s); old.apply(t his, Array.prot ot ype.slice(argument s)); }); jQuery.doIt (); assert .ok(received.some(funct ion(result ) { ret urn result [1] == 'a'; })); assert .ok(received.some(funct ion(result ) { ret urn result [1] == 'b'; })); done(); };

Here, we are just replacing a f unction, capturing calls to it, and then calling the original f unction. Check out MDN on what arguments is, if you're not f amiliar with it.

Addit ional reading

5. What's in a View? A look at the alternatives

In this chapter, I will look at the concepts and dif f erences of opinion between various f rameworks when implementing views. I actually started writing this chapter with a code comparison (based on TodoMVC), but decided to remove it - the code you write is mostly very similar, while the underlying mechanisms and abstractions used are dif f erent. T he view layer is the most complex part of modern single page app f rameworks. Af ter all, this is the whole point of single page apps: make it easy to have awesomely rich and interactive views. As you will see, there are two general approaches to implementing the view layer: one is based around code, and the other is based around markup and having a f airly intricate templating system. T hese lead to dif f erent architectural choices. Views have several tasks to care of : Rendering a template. We need a way to take data, and map it / output it as HT ML. Updating views in response to change events. When model data changes, we need to update the related view(s) to ref lect the new data. Binding behavior to HT ML via event handlers. When the user interacts with the view HT ML, we need a way to trigger behavior (code). T he view layer implementation is expected to provide a standard mechanism or convention to perf orm these tasks. T he diagram below shows how a view might interact with models and HT ML while perf orming these tasks:

T here are two questions: How should event handlers be bound to/unbound f rom HT ML? At what granularity should data updates be perf ormed? Given the answers to those questions, you can determine how complex your view layer implementation needs to be, and what the output of your templating system should be. One answer would be to say that event handlers are bound using DOM selectors and data updates "view-granular" (see below). T hat gives you something like Backbone.js. T here are other

answers. In this chapter, I will present a kind of typology f or looking at the choices that people have made in writing a view layer. T he dimensions/contrasts I look at are: Low end interactivity vs. high end interactivity Close to server vs. close to client Markup-driven views vs. Model-backed views View-granular vs. element-granular vs. string-granular updates CSS-based vs. f ramework-generated event bindings

Low-end interactivity vs high-end interactivity

What is your use case? What are you designing your view layer f or? I think there are two rough use cases f or which you can cater: Low-end interactivity Example: Github Pages are mostly static. You take a document that represents a mostly static piece of inf ormation already processed, and add a bit of interactivity via Javascript. Changing data usually causes a f ull page ref resh. High-end interactivity Example: Gmail Pages are mostly dynamic. You have a set of data which you want the user to interact with in various ways; changes to the data should be ref lected on the page immediately. Changing data should update the views, but not cause a page ref resh - because views have many small intermediate states which are not stored in the database.

State and data can be stored in HT ML, because if data is altered, the page is ref reshed. Because a majority of the state is in the HT ML, parts of the UI do not generally interact with each other. If complex interactions are needed, they are resolved on the server.

Storing state and data in HT ML is a bad idea, because it makes it hard to keep multiple views that represent the same data in sync. Complex interactions are more f easible; data is separate f rom presentation. Interactions don't need to map to backend actions, e.g. you can paginate and select items in a view without writing a new serverside endpoint.

Both Github and Gmail are modern web apps, but they have dif f erent needs. Github's pages are largely f orm-driven, with disconnected pieces of data (like a list of branches) that cause a f ull page ref resh; Gmail's actions cause more complex changes: adding and starring a new draf t message shows applies the change to multiple places without a page ref resh. Which type of app are you building? Your mileage with dif f erent app architectures/f rameworks will vary based on what you're trying to achieve. T he way I see it, web apps are a mixture of various kinds of views. Some of those views involve more complicated interactions and benef it f rom the architecture designed f or high-end interactivity. Other views are just simple components that add a bit of interactivity. T hose views may be easier and cleaner to implement using less sophisticated methods.

If you never update a piece of data, and it can reasonably f it in the DOM without impacting perf ormance, then it may be good candidate f or a low-end approach. For example, collapsible sections and dropdown buttons that never change content but have some basic enhancements might work better as just markup without actually being bound to model data and/or without having an explicit view object. On the other hand, things that get updated by the user will probably be best implemented using high-end, model-and-view backed objects. It is worth considering what makes sense in your particular use case and how f ar you can get with low-end elements that don't contain "core data". One way to do this is to maintain catalogue of low-end views/elements f or your application, a la Twitter's Bootstrap. T hese low-end views are distinguished by the f act that they are not bound to / connected to any model data: they just implement simple behavior on top of HT ML.

Close to server vs. close to client

Do you want to be closer to the server, or closer to the user? UIs generated on the server side are closer to the database, which makes database access easier/lower latency. UIs that are rendered on the client side put you closer to the user, making responsive/complex UIs easier to develop.

Data in markup/HT ML manipulation Data is stored in HT ML; you serve up a bunch of scripts that use the DOM or jQuery to manipulate the HT ML to provide a richer experience. For example, you have a list of items that is rendered as HT ML, but you use a small script that takes that HT ML and allows the end user to f ilter the list. T he data is usually read/written f rom the DOM. (examples: Twitter's Bootstrap; jQuery plugins). Specif ic HT ML+CSS markup structures are used to to make small parts of the document dynamic. You don't need to write Javascript or only need to write minimal Javascript to conf igure options. Have a look at Twitter's Bootstrap f or a modern example. T his approach works f or implementing low-end interactivity, where the same data is never shown twice and where each action triggers a page reload. You can spot this approach by looking f or a backend that responds with f ully rendered HT ML and/or a blob of Javascript which checks f or the presence of particular CSS classes and conditionally activates itself (e.g. via event handlers on the root element or via $().live()). PJAX. You have a page that is generated as HT ML. Some user action triggers code that replaces parts of the existing page with new server-generated HT ML that you f etch via AJAX. You use

PushState or the HT ML5 history API to give the appearance of a page change. It's basically "HT ML manipulation - Extreme Edition", and comes with the same basic limitations as pure HT ML manipulation. Widgets. T he generated page is mostly a loader f or Javascript. You instantiate widgets/rich controls that are written in JS and provided by your particular f ramework. T hese components can f etch more data f rom the server via a JSON API. Rendering happens on the client-side, but within the customization limitations of each widget. You mostly work with the widgets, not HT ML or CSS. Examples: YUI2, Sproutcore. Finally, we have markup-driven views and model-backed views.

Markup-driven views vs Model-backed views

If you could choose your ideal case: what should people read in order to understand your application? T he markup - or the code? Frameworks f all into two dif f erent camps based on this distinction: the ones where things are done mostly in markup, and ones in which things are mostly done in code. [ Dat a in JS models ] [ Dat a in JS models ] [ Model-backed views ] [ Markup accesses models ] Model-backed views. In this approach, models are the starting point: you instantiate models, which are then bound to/passed to views. T he view instances then attach themselves into the DOM, and render their content by passing the model data into a template. To illustrate with code: var model = new Todo({ t it le: 'foo', done: false }), view = new TodoView(model); T he idea being that you have models which are bound to views in code. Markup-driven views. In this approach, we still have views and models, but their relationship is inverted. Views are mostly declared by writing markup (with things like custom attributes and/or custom tags). Again, this might look like this: {{view TodoView}} {{=window.model.t it le}} {{/view}} T he idea being that you have a templating system that generates views and that views access variables directly through a f ramework-provided mechanism. In simple cases, there might not even be a directly accessible instance of a view. Instead, views ref er to variables in the global scope by their name, "App.Foo.bar" might resolve to a particular model. Views might ref er to controllers or observable variables/models by their name.

Two tracks
T hese two approaches aren't just minor dif f erences, they represent dif f erent philosophies and have vastly dif f erent complexities in terms of their implementation. T here two general modern single page app (view layer) approaches that start f rom a dif f erence of view in what is primary: markup or code.

If markup is primary, then one needs to start with a f airly intricate templating system that is capable of generating the metadata necessary to implement the f unctionality. You still need to translate the templating language into view objects in the background in order to display views and make sure that data is updated. T his hides some of the work f rom the user at the cost of added complexity. If code is primary, then we accept a bit more verbosity in exchange f or a simpler overall implementation. T he dif f erence between these two can easily be at least an order of magnitude in terms of the size of the f ramework code. View behavior: in view object vs. in controller? In the model-backed views approach, you tend to think of views as reusable components. Traditional (MVC) wisdom suggests that "skinny controller, f at model" - e.g. put business logic in the model, not in the controller. I'd go even f urther, and try to get rid of controllers completely replacing them with view code and initializers (which set up the interactions between the parts). But isn't writing code in the view bad? No - views aren't just a string of HT ML generate (that's the template). In single page apps, views have longer lif ecycles and really, the initialization is just the f irst step in interacting with the user. A generic component that has both presentation and behavior is nicer than one that only works in a specif ic environment / specif ic global state. You can then instantiate that component with your specif ic data f rom whatever code you use to initialize your state. In the markup-driven views approach, ideally, there would be no view objects whatsoever. T he goal is to have a suf f iciently rich templating system that you do not need to have a view object that you instantiate or bind a model to. Instead, views are "thin bindings" with the ability to directly access variables using their names in the global scope; you can write markup-based directives to directly read in those variables and iterate over them. When you need logic, it is mostly f or special cases, and that's where you add a controller. T he ideal is that views aren't backed by objects, but by the view system/templating metadata (transf ormed into the appropriate set of bindings). Controllers are a result of non-reuseable views. If views are just slightly more sophisticated versions of "strings of HT ML" (that bind to specif ic data) rather than objects that represent components, then it is more tempting to put the glue f or those bindings in a separate object, the controller. T his also has a nice f amiliar f eeling to it f rom server-side f rameworks (requestresponse f rameworks). If you think of views as components that are reusable and consist of a template and a object, then you will more likely want to put behavior in the view object since it represents a singular, reusable thing. Again, I don't like the word "controller". Occasionally, the distinction is made between "controllers specif ic to a view" and "controllers responsible f or coordinating a particular application state". I'd f ind "view behavior" and "initialization code" to be more descriptive. I would much rather put the "controller code" specif ic to a view into the view object, and make the view generic enough to be reusable through conf iguration and events. Observables vs. event emitters Once we have some view behavior, we will want to trigger it when model data changes. T he two major options are observables and event emitters. What's the dif f erence? Basically, in terms of implementation, not much. In both cases, when a change occurs, the code that is interested in that change is triggered. T he dif f erence is mostly syntax and implied design patterns. Events are registered on objects:

Todos.on('change', funct ion() { ... }); while observers are attached through global names: Framework.regist erObserver(window.App.Todos, 'change', funct ion() { ... }); Usually, observable systems also add a global name resolution system, so the syntax becomes: Framework.observe('App.Todos', funct ion() { ... }); Or if you want to be an asshole, you can avoid typing Framework. by extending the native Function object: funct ion() { ... }.observe('App.Todos'); T he markup-driven approach tends to lead to observables. Observables of ten come with a name resolution system, where you ref er to things indirectly via strings. T he reason why a global name resolution system - where names are strings rather than directly accessing objects - is of ten added f or observables is that setting up observers without it becomes complex, since the observers can only be registered when the objects they ref er to have been instantiated. Since there are no guarantees whether a distant object is initialized, there needs to be a level of abstraction where things only get bound once the object is instantiated. T he main reason why I don't particularly like observables is that you need to ref er to objects via a globally accessible name. Observables themselves are basically equivalent to event emitters, but they imply that things ought to be ref erred by global names since without a global name resolution system there would be no meaningf ul dif f erence between event listeners and observables with observers. Observables also tend to encourage larger models since model properties are/can be observed directly f rom views - so it becomes convinient to add more model properties, even if those are specif ic to a single view. T his makes shared models more complex everywhere just to accomodate a particular view, when those properties might more properly be put in a package/module-specif ic place. Specif ying bindings using DOM vs. having f ramework-generated element ID's We will want to also bind to events f rom the DOM to our views. Since the DOM only has a element-based API f or attaching events, there are only two choices: DOM-based event bindings. Framework-generated event bindings. DOM-based event bindings basically rely on DOM properties, like the element ID or element class to locate the element and bind events to it. T his is f airly similar to the old-f ashioned $('#foo').on('click', ...) approach, except done in a standardized way as part of view instantiation. Framework-generated event bindings allow you to bind event handlers to HT ML without explicitly providing a element ID or selector f or the view. You don't have to give elements classes. Instead, you write the event handler inside the markup, and the templating system generates an ID f or the element, and tracks the lif ecycle of the element (e.g. attached to the DOM/not attached to the DOM etc.), making sure that the event handler is attached.

What update granularity should be supported? View-granular, element-granular and stringgranular T his is a subtle but important part of the view layer, since it determines basically how a lot of the rest of the f ramework code is written. "Update granularity" ref ers to the smallest possible update that a particular f ramework supports. Interestingly, it is impossible to visually distinguish between the dif f erent approaches just by looking at code. T his snippet: <p>Hello {{name}}</p> ... can be updated at any level of granularity. You actually have to look at f ramework code in order to know what the update granularity is: View-granular frameworks allow you to update a single view, but nothing smaller. Internally, the view is represented as a element ref erence and template f unction that generates/renders a HT ML string. If the {{name}} changes, then you re-render the HT ML and change the innerHT ML of the top-level element that is bound to the view. Element-granular frameworks make it possible to update the value directly inside the DOM, but they require that each individually updateable part is represented in the DOM as an element. Internally, elements are added around each updateable piece, something like this: <p>Hello <span id="$0">foo</span></p> Given this compiled result and some metadata, the f ramework can then select "$0" and change it without altering the rest. String-granular frameworks allow you to update any part of the template, and do not require that updateable parts are wrapped inside elements. Instead, they use script tags or comment tags to delimit updateable content (mostly, because the Range API doesn't work on IE). T hat template might compile into: <p> Hello <script id="met amorph-0-st art " t ype="t ext /x-placeholder></script > foo <script id="met amorph-0-end" t ype="t ext /x-placeholder"></script >. </p> T his is almost the same thing as element-granular updates, except that the DOM contains two nonvisual elements f or each updatedable part; and conceptually, the f ramework's binding engine works with string ranges between the two elements rather than with single elements. What are the benef its and disadvantages of each of these approaches? View-granular updates mean that a value update causes the inner HT ML of each view interested in that update to be re-rendered and inserted into the DOM. View-granular updates are simple: each view corresponds to a single element (and its innerHT ML) and only one DOM element needs to be tracked per view. T he disadvantage is that since the view cannot render parts of itself individually, doing a redraw might reset things like text in input elements and keyboard f ocus if they are inside the view markup and in a non-def ault state. T his can be worked around with a bit of coding, however.

Element-granular updates mean that af ter a view is rendered once, parts of it can be updated separately as long as those parts can be wrapped in an element. Views have bound elements that represent values f rom some model/data that in the resulting markup are wrapped in f rameworkgenerated elements with DOM ids. T he disadvantage is that there is much more to track (both in JS and in the DOM), and using CSS is not necessarily straightf orward since bound values are wrapped inside elements, meaning that the CSS path to the element is not what you might expect (e.g. p span instead of p). String-granular updates are the most complex. T hey provide the same f unctionality as elementgranular updates, but also allow you to specif y a bindings that do not correspond to elements, such as a f oreach without a container element: <t able> <t r> <t h>Names</t h> {{#people}} <t d>{{name}}</t d> {{/people}} </t r> </t able> T his could not be done using a element-granular approach, because you cannot insert an element other than a <th> or <td> inside a <tr>. If you place an invalid wrapper element like a <div> where the {{#people}} strings are, the browser will relocate it outside the table as a way recover f rom invalid markup. But without an element, you cannot ref er to that particular part of the DOM in a manner that works in IE. So you need some other way to make that part of the DOM accessible and hence replaceable in a more granular manner. T here are two known techniques f or this: <script> tags and <!-- comment --> tags stay in all DOM locations, even invalid DOM locations, so they can be used to implement a string-range-oriented rather than element-oriented way to access data, making string-granular updates possible. Script tags can be selected by id (likely f aster) but inf luence CSS selectors that are based on adjacent siblings and can be invalid in certain locations. Comment tags, on the other hand, require (slow) DOM iteration in old browsers that don't have certain APIs, but are invisible to CSS and valid anywhere in the page. Perf ormance-wise, the added machinery vs. view-granular approaches does incur a cost. T here are also still some special cases, like select elements on old IE version, where this approach doesn't work.

Conclusion
T he single page app world is f airly conf using right now. Frameworks def ine themselves more in terms of what they do rather than how they accomplish it. Part of the reason is that the internals are unf amiliar to most people, since -- let's f ace it -- these are still the early days of single page apps. I hope this chapter has developed a vocabulary f or describing dif f erent single page app f rameworks. Frameworks encourage dif f erent kinds of patterns, some good, some bad. Starting f rom a f ew key ideas about what is important and what should def ine a single page app, f rameworks have reached dif f erent conclusions. Some approaches are more complex, and the choice about what to make easy inf luences the kind of code you write. String-granular bindings lead to heavier models. Since model properties are directly observable in views, you tend to add properties to models that don't represent backend data, but rather view state. Computed properties mean that model properties can actually represent pieces of logic.

T his makes your model properties into an API. In extreme cases, this leads to very specif ic and view-related model properties like "humanizedName" or "dataWithCommentInReverse" that you then observe f rom your view bindings. T here is a tradeof f between DRY and simplicity. When your templating system is less sophisticated, you tend to need to write more code, but that code will be simpler to troubleshoot. Basically, you can expect to understand the code you wrote, but f ewer people are well versed in what might go wrong in your f ramework code. But of course, if nothing breaks, everything is f ine either way. Personally, I believe that both approaches can be made to work.

6. The model layer: an overview

Let's examine the model layer in more detail. In the introduction chapter, a model was shown as something that simply queries and writes to storage. T he diagram below shows more details of the model layer:

T he model layer looks f airly similar across dif f erent single page app f rameworks because there just aren't that many dif f erent ways to solve this problem. You need the ability to represent data items and sets of data items; you need a way to load data; and you probably want to have some caching in place to avoid naively reloading data that you already have. Whether these exist as separate mechanisms or as a part of single large model is mostly an implementation detail. T he major dif f erence is how collections are handled, and this is a result of choices made in the view layer - with observables, you want observable arrays, with events, you want collections.

Data source
Common way of instantiating models f rom existing data Fetching models by id Fetching models by search A data source (or backend proxy / API) is responsible f or reading f rom the backend using a simplif ied and more powerf ul API. It accepts JSON data, and returns JSON objects that are converted into Models. Note how the data source reads f rom the data store/cache, but queries the backend as well. Lookups by ID can be f etched directly f rom the cache, but more complicated queries need to ask

the backend in order to search the f ull set of data.

Model
A place to store data Emits events when data changes Can be serialized and persisted T he model contains the actual data (attributes) and can be transf ormed into JSON in order to restore f rom or save to the backend. A model may have associations, it may have validation rules and it may have subscribers to changes on its data.

Collection
Contains items Emits events when items are added/removed Has a def ined item order Collections exist to make it easy to work with sets of data items. A collection might represent a subset of models, f or example, a list of users. Collections are ordered: they represent a particular selection of models f or some purpose, usually f or drawing a view. You can implement a collection either: As a model collection that emits events As an observable array of items T he approach you pick is dependent mostly on what kind of view layer you have in mind. If you think that views should contain their own behavior / logic, then you probably want collections that are aware of models. T his is because collections contain models f or the purpose of rendering; it makes sense to be able to access models (e.g. via their ID) and tailor some of the f unctionality f or this purpose. If you think that views should mostly be markup - in other words, that views should not be "components" but rather be "thin bindings" that ref er to other things by their name in the global scope - then you will probably pref er observable arrays. In this case, since views don't contain behavior, you will also probably have controllers f or storing all the glue code that coordinates multiple views (by ref erring to them by name).

Data cache
Caches models by id, allowing f or f aster retrieval Handles saving data to the backend Prevents duplicate instances of the same model f rom being instantiated A data store or data cache is used in managing the lif ecycle of models, and in saving, updating and deleting the data represented in models. Models may become outdated, they may become unused and they may be preloaded in order to make subsequent data access f aster. T he dif f erence between a collection and a cache is that the cache is not in any particular order, and the cache represents all the models that the client-side code has loaded and retained.

7. Implementing a data source

In this chapter, I will look at implementing a data source.

Def ining a REST-based, chainable API f or the data source

Let's start of f by writing some tests in order to specif y what we want f rom the data source we will build. It's much easier to understand the code once we have an idea of what the end result should look like. Given the f ollowing f ixture: var xt ure = [ { name: 'a', id: 1, role: 2 }, { name: 'b', id: 2, role: 4, organizat ion: 1 }, { name: 'c', id: 3, role: 4, organizat ion: 2 } ]; var db.user = new Dat aSource(); ... here are tests describing how I'd like the data source to work:

Can load a single item by ID

'can load a single it em by ID': funct ion(done) { db.user(1, funct ion(user) { assert .equal(xt ure[0], user); done(); }); },

Can load multiple items by ID

'can load mult iple it ems by ID': funct ion(done) { db.user([1, 2, 3], funct ion(users) { assert .deepEqual(xt ure, users); done(); }); },

Can load items by search query

T he data source should support retrieving items by conditions other than IDs. Since the details depend on the backend used, we'll just allow the user to add search terms via an object. T he parameters are passed to the backend, which can then implement whatever is appropriate (e.g. SQL query by name) to return the result JSON. 'can load it ems by search query': funct ion(done) { db.user({ name: 'c'}, funct ion(user) { assert .deepEqual(xt ure[2], user); done(); }); },

Can add more search conditions using and()

We'll also support incrementally def ining search parameters: 'should allow for adding more condit ions using and()': funct ion(done) { db.user({ role: 4 }) .and({ organizat ion: 1 }, funct ion(users) { assert .deepEqual(xt ure[1], users); done(); }); },

Implementing the chainable data source API

T he f ull implementation f or a chainable data source API is below. It almost f its on one screen. f unction Search(options) { this.uri = options.uri; this.model = options.model; this.conditions = []; } Search.prototype.and = f unction(arg, callback) { if (!arg) return this; this.conditions.push(arg); return this.end(callback); }; Search.prototype.end = f unction(callback) { if (!callback) return this; var self = this, params = {}, urls = []; f unction process(arg) { if (typeof arg == 'number') { urls.push(self .uri(arg)); } else if (Array.isArray(arg)) { urls = urls.concat(arg.map(f unction(id) { return self .uri(id); })); } else if (arg === Object(arg)) { Object.keys(arg).f orEach(f unction(key) T he data source accepts two parameters: url, which is a f unction that returns a URL f or a particular id model, an optional parameter; if given, the results will be instances of that model instead of plain Javacript objects (e.g. JSON parsed as a JS object). T he idea behind chainable APIs is that the actual action is delayed until a callback is passed to the API. condit ions is a simple array of all the parameters (model ID's and search parameters) passed to the current data source search. Also note how all the f unctions return t his . T hat allows f unction calls to be written one af ter another. T he end() f unction is where the conditions are processed and stored into url and params . We call process() on each condition in order to extract the inf ormation. process(arg) looks at the type of each argument. If the argument is a number, we assume it's a model ID. If it is an array, then it is considered an array of IDs. Objects are assumed to be search parameters (key: value pairs). For numbers, we map them to a url by calling t his.uri() on them. T hat parameter is part of the resource def inition.

{ params[key] = arg[key]; }); } } this.conditions.f orEach(process); (urls.length == 0) && (urls = [ this.uri() ]); this._execute(urls, params, callback); }; Search.prototype._execute = f unction(urls, params, callback) { var self = this, results = []; urls.f orEach(f unction(url) { Client .get(url).data(params) .end(Client.parse(f unction(err, data) { if (err) throw err; results.push((self .model ? new self .model(data) : data)); if (results.length == urls.length) { callback((urls.length == 1 ? results[0] : results)); } })); }); }; Search.prototype.each = f unction(callback) { return this.end(f unction(results) { results.f orEach(callback); }); }; module.exports = f unction(options) { return f unction(arg, callback) { If .each(funct ion() { ...}) is called, then we take the callback, and wrap it in a f unction that iterates over the results array and calls the callback f or each result. T his requires ES5 (e.g. not IE; since we rely on Array.f orEach to exist). For IE compatibility, use underscore or some other shim. Finally, how do we def ine a datasource? We return a f unction that accepts (arg, callback) and itself returns a new instance of Search. T his allows us to def ine a particular data source and store the conf iguration in another variable. Every search is a T his is where the magic happens (not really). We call the HT T P client, passing each URL and set of parameters. Once we get each result, we store it in the results array. When the results array is f ull, we call the original callback with the results. If there was only one result, then we just take the f irst item in the array.

return f unction(arg, callback) { return new Search(options).and(arg, callback); } };

new instance of Search. See the f ull usage example at the end of the chapter f or details.

Making ajax a bit nicer: Client

Since I wanted the same code to work in Node and in the browser, I added a (chainable) HT T P interf ace that works both with jQuery and Node.js. Here is a usage example: Client .get ('ht t p://www.google.com/') .dat a({q: 'hello world'}) .end(funct ion(err, dat a) { console.log(dat a); }); And the f ull source code: f or jQuery (~40 lines; below) and f or Node (~70 lines; w/JSON parsing). var $ = require('jquery'); funct ion Client (opt s) { t his.opt s = opt s || {}; t his.opt s.dat aType || (t his.opt s.dat aType = 'json'); t his.opt s.cache = false; }; Client .prot ot ype.dat a = funct ion(dat a) { if(!dat a || Object .keys(dat a).lengt h == 0) ret urn t his; if(t his.opt s.t ype == 'GET') { t his.opt s.url += '?'+jQuery.param(dat a); } else { t his.opt s.cont ent Type = 'applicat ion/json'; t his.opt s.dat a = JSON.st ringify(dat a); } ret urn t his; }; Client .prot ot ype.end = funct ion(callback) { t his.opt s.error = funct ion(j, t , err) { callback && callback(err); }; t his.opt s.success = funct ion(dat a, t , j) { callback && callback(undened, dat a); }; $.ajax(t his.opt s); }; module.export s.parse = Client .parse = funct ion(callback) { ret urn funct ion(err, response) { callback && callback(undened, response); }; }; ['get ', 'post ', 'put ', 'delet e'].forEach(funct ion(met hod) { module.export s[met hod] = funct ion(urlSt r) { ret urn new Client ({ t ype: met hod.t oUpperCase(), url: urlSt r }); }; });

Putting it all together

Now, that's a f airly usef ul data source implementation; minimal yet usef ul. You can certainly reuse it with your f ramework, since there are no f ramework dependencies; it's all (ES5) standard Javascript. Defining a data source Now, let's create a page that allows us to use the datasource to retrieve data. For example, you might want to use the datasource with a model. You may have noticed that I slipped in support f or instantiating models f rom the result (see the this.model parameter in implementation). T his means that we can ask the data source to instantiate objects f rom a given model constructor by passing the model option: // Find inst ances of Todo using Todo.nd() Todo.nd = new Dat aSource({ uri: funct ion(id) { ret urn 'ht t p://localhost :8080/api/t odo/' + (id ? encodeURIComponent (id) : 'search'); }, model: Todo }); As you can see, the uri f unction simply returns the right URL depending on whether the search is about a specif ic ID or just a search. T he code also demostrates composition over inheritance. T he inheritance-based way of setting up this same f unctionality would be to inherit f rom another object that has the data source f unctionality. With composition, we can simply assign the DataSource to any plain old JS object to add the ability to retrieve JSON data by calling a f unction. Building a backend . T he server-side f or the datasource can be f airly simple: there are two cases - reading a model by ID, and searching f or a model by property.

var ht t p = require('ht t p'), url = require('url'); var t odos = [ { id: 1, t it le: 'aa', done: false }, { id: 2, t it le: 'bb', done: t rue }, { id: 3, t it le: 'cc', done: false } ], server = ht t p.creat eServer(); var idRe = new RegExp('^/api/t odo/([0-9]+)[^0-9]*$'), searchRe = new RegExp('^/api/t odo/search.*$'); server.on('request ', funct ion(req, res) { res.set Header('cont ent -t ype', 'applicat ion/json'); if(idRe.t est (req.url)) { var part s = idRe.exec(req.url); // ret urn t he ID if(t odos[part s[1]]) { res.end(JSON.st ringify(t odos[part s[1]])); } } else if (searchRe.t est (req.url)) { var dat a = ''; req.on('dat a', funct ion(part ) { dat a += part ; }); req.on('end', funct ion() { var search = JSON.parse(dat a); // search t he t odos array by key - value pair res.end(JSON.st ringify( t odos.lt er(funct ion(it em) { ret urn Object .keys(search).every(funct ion(key) { ret urn it em[key] && it em[key] == search[key]; }); }) )); }); } else { console.log('Unknown', req.url); res.end(); } });

8. Implementing a model
What's a model? Roughly, a model does a couple of things: Data. A model contains data. Events. A model emits change events when data is altered. Persistence . A model can be stored persistently, identif ied uniquely and loaded f rom storage. T hat's about it, there might be some additional niceties, like def ault values f or the data.

Def ining a more usef ul data storage object (Model)

f unction Model(attr) { this.reset(); attr && this.set(attr); };

Model.reset()
_data: T he underlying data structure is a object. To keep the values stored in the object f rom conf licting with property names, let's store the data in the _dat a

}; Model.prototype.reset = f unction() { this._data = {}; this.length = 0; this.emit('reset'); }; Model.prototype.get = f unction(key) { return this._data[key]; }; Model.prototype.set = f unction(key, value) { var self = this; if (arguments.length == 1 && key === Object(key)) { Object.keys(attr).f orEach(f unction(key) { self .set(key, attr[key]); }); return; } if (!this._data.hasOwnProperty(key)) { this.length++; } this._data[key] = (typeof value == 'undef ined' ? true : value); }; Model.prototype.has = f unction(key) { return this._data.hasOwnProperty(key); }; Model.prototype.remove = f unction(key) { this._data.hasOwnProperty(key) && this.length--; delete this._data[key]; }; module.exports = Model;

property Store length: We'll also keep a simple length property f or quick access to the number of elements stored in the Model.

Model.get(key)
T his space intentionally lef t blank.

Model.set(key, value)
Setting multiple values: if only a single argument Model.set ({ foo: 'bar'}) is passed, then call Model.set () f or each pair in the f irst argument. T his makes it easier to initialize the object by passing a hash. Note that calling Model.set (key) is the same thing as calling Model.set (key, t rue). What about ES5 getters and setters? Meh, I say. Setting a single value: If the value is undef ined, set to true. T his is needed to be able to store null and f alse.

Model.has(key), Model.remove(key)
Model.has(key): we need to use hasOwnProperty to support f alse and null. Model.remove(key): If the key was set and removed, then decrement .length. T hat's it! Export the module.

Change events
Model accessors (get/set) exist because we want to be able to intercept changes to the model data, and emit change events. Other parts of the app -- mainly views -- can then listen f or those events and get an idea of what changed and what the previous value was. For example, we can respond to these: a set() f or a value that is used elsewhere (to notif y others of an update / to mark model as changed) a remove() f or a value that is used elsewhere We will want to allow people to write model.on('change', funct ion() { .. }) to add listeners that are called to notif y about changes. We'll use an EventEmitter f or that. If you're not f amiliar with EventEmitters, they are just a standard interf ace f or emitting (triggering) and binding callbacks to events (I've written more about them in my other book.) var util = require('util'), events = require('events'); f unction Model(attr) { // ... }; util.inherits(Model, events.EventEmitter); Model.prototype.set = f unction(key, value) { var self = this, oldValue; // ... oldValue = this.get(key); this.emit('change', key, value, oldValue, this); // ... }; Model.prototype.remove = f unction(key) { this.emit('change', key, undef ined, this.get(key), this); // ... }; T he model extends event s.Event Emit t er using Node's ut il.inherit s() in order to support the f ollowing API: on(event, listener) once(event, listener) emit(event, [arg1], [...]) removeListener(event, listener) removeAllListeners(event) For in-browser compatibility, we can use one of the many APIcompatible implementations of Node's EventEmitter. For instance, I wrote one a while back (mixu/miniee). When a value is set (), emit ('change', key, newValue, oldValue). T his causes any listeners added via on()/once() to be triggered. When a value is removed(), emit ('change', key, null, oldValue).

Using the Model class

So, how can we use this model class? Here is a simple example of how to def ine a model: funct ion Phot o(at t r) { Model.prot ot ype.apply(t his, at t r); } Phot o.prot ot ype = new Model(); module.export s = Phot o; Creating a new instance and attaching a change event callback: var badger = new Phot o({ src: 'badger.jpg' }); badger.on('change', funct ion(key, value, oldValue) { console.log(key + ' changed from', oldValue, 't o', value); }); Def ining def ault values: funct ion Phot o(at t r) { at t r.src || (at t r.src = 'default .jpg'); Model.prot ot ype.apply(t his, at t r); } Since the constructor is just a normal ES3 constructor, the model code doesn't depend on any particular f ramework. You could use it in any other code without having to worry about compatibility. For example, I am planning on reusing the model code when I do a rewrite of my window manager.

Dif f erences with Backbone.js

I recommend that you read through Backbone's model implementation next. It is an example of a more production-ready model, and has several additional f eatures: Each instance has a unique cid (client id) assigned to it. You can choose to silence change events by passing an additional parameter. Changed values are accessible as the changed property of the model, in addition to being accessible as events; there are also many other convenient methods such as changedAttributes and previousAttributes. T here is support f or HT ML-escaping values and f or a validate() f unction. .reset() is called .clear() and .remove() is .unset() Data source and data store methods (Model.save() and Model.destroy()) are implemented on the model, whereas I implement them in separate objects (f irst and last chapter of this section).

9. Collections
What's in a collection? A collection: contains items (or models) emits events when items are added/removed is ordered; can be accessed by index via at () and by model ID via get ()

In this chapter, we'll write an observable array, and then add some additional niceties on top of it to make it a collection (e.g. something that is specif ic to storing models).

Storing Models and emitting events

Let's start with the constructor. We want to mixin EventEmitter to add support f or events f or the collection. funct ion Collect ion(models) { t his.reset (); models && t his.add(models); } ut il.inherit s(Collect ion, event s.Event Emit t er); To support passing a set of initial models, we call this.add() in the constructor. Resetting the collection . Self -explanatory, really. We will use an array to store the models, because collections are ordered rather than indexed; and we will maintain a length property directly f or convenience. Collect ion.prot ot ype.reset = funct ion() { t his._it ems = []; t his.lengt h = 0; t his.emit ('reset '); }; Adding items. We should be able to call add(model) and emit/listen f or an "add" event when the model is added. Collect ion.prot ot ype.add = funct ion(model, at ) { var self = t his; // mult iple add if(Array.isArray(model)) { ret urn model.forEach(funct ion(m) { self.add(m, at ); }); } t his._it ems.splice(at || t his._it ems.lengt h, 0, model); t his.lengt h = t his._it ems.lengt h; t his.emit ('add', model, t his); }; To support calling add([model1, model2]), we'll check if the f irst parameter is an array and make multiple calls in that case. Other than that, we just use Array.splice to insert the model. T he optional at param allows us to specif y a particular index to add at. Finally, af ter each add, we emit the "add" event. Removing items. We should be able to call remove(model) to remove a model, and receive events when the item is removed. Again, the code is rather trivial.

Collect ion.prot ot ype.remove = funct ion(model){ var index = t his._it ems.indexOf(model); if (index < -1) { t his._it ems.splice(index, 1); t his.lengt h = t his._it ems.lengt h; t his.emit ('remove', model, t his); } }; Retrieving items by index and retrieving all items. Since we are using an array, this is trivial: Collect ion.prot ot ype.at = funct ion(index) { ret urn t his._it ems[index]; }; Collect ion.prot ot ype.all = funct ion() { ret urn t his._it ems; };

Iteration
We also want to make working with the collection easy by supporting a f ew iteration f unctions. Since these are already implemented in ES5, we can just call the native f unction, setting the parameter appropriately using .apply(). I'll add support f or the big 5 - f orEach (each), f ilter, map, every and some: ['lt er', 'forEach', 'every', 'map', 'some'].forEach(funct ion(name) { Collect ion.prot ot ype[name] = funct ion() { ret urn Array.prot ot ype[name].apply(t his._it ems, argument s); } });

Sorting
Implementing sorting is easy, all we need is a comparator f unction. Collect ion.prot ot ype.sort = funct ion(comparat or) { t his._it ems.sort (comparat or || t his.orderBy); }; Array.sort is already implemented in ES3 and does what we want: you can pass a custom comparator, or set collect ion.orderBy to set a def ault sort f unction.

Using our observable array

T he code above covers the essence of an observable array. Let's look at f ew usage examples bef ore moving on to a making it a collection. var it ems = new Collect ion(); it ems.on('add', funct ion(it em) { console.log('Added', it em); }); set Int erval(funct ion() { it ems.add(Mat h.oor(Mat h.random() * 100)); console.log(it ems.all()); }, 1000);

Creating a collection
A collection is a more specialized f orm of an observable array. Collections add the ability to hook into the events of the models they contain, and add the ability to retrieve/check f or item presence by model id in addition to the position in the array. get(modelId). Let's implement get (modelId) f irst. In order to make get() f ast, we need a supplementary index. To do this, we need to capture the add() and remove() calls: Collect ion.prot ot ype.add = funct ion(model, at ) { var self = t his, modelId; // ... modelId = model.get ('id'); if (t ypeof modelId != 'undened') { t his._byId[modelId] = model; } }; Collect ion.prot ot ype.remove = funct ion(model){ var index = t his._it ems.indexOf(model), modelId; // ... modelId = model.get ('id'); if (t ypeof modelId != 'undened') { delet e t his._byId[modelId]; } }; Now get() can make a simple lookup: Collect ion.prot ot ype.get = funct ion(id) { ret urn t his._byId[id]; }; Hooking into model events. We need to bind to the model change event (at least), so that we can trigger a "change" event f or the collection: Collect ion.prot ot ype._modelChange = funct ion(key, value, oldValue, model) { t his.emit (key, value, oldValue, model); }; Collect ion.prot ot ype.add = funct ion(model, at ) { // ... model.on('change', t his._modelChange); }; And we need to unbind when a model is removed, or the collection is reset: Collect ion.prot ot ype.remove = funct ion(model){ // ... model.removeList ener('change', t his._modelChange); }; Collect ion.prot ot ype.reset = funct ion() { var self = t his; if(t his._it ems) { t his._it ems.forEach(funct ion(model) { model.removeList ener('change', self._modelChange); }); } // ... };

Using the Collection class

10. Implementing a data cache

T here are three reasons why we want a data store: To have a central mechanism f or saving data. To retrieve cached models quickly. To prevent duplicate instances of the same model being created. T he f irst two are obvious: we need to handle saving, and when possible, use caching to make unambiguous retrievals f ast. T he only clearly unambigous type of retrieval is f etching a model by id. T he last reason is less obvious. Why is it bad to have duplicate instance of the same model? Well, f irst, it is inef f icient to have the same data twice; but more importantly, it is very conf using if you can have two instances that represent the same object but are separate objects. For example, if you have a data cache that always returns a new object rather than reusing an existing one, then you can have situations where you change the model data, or add a model data listener, but this change does not actually work as expected because the object you used is a dif f erent instance. We'll tackle this af ter looking at saving and caching.

Implementing save()
Serializing models into JSON. In order to send the model data, we need the ability to transf orm a model into a string. JSON is the obvious choice f or serializing data. We need to add a additional method to the model: Model.prot ot ype.json = funct ion() { ret urn JSON.st ringify(t his._dat a); }; Mapping to the right backend URL. We also need to know where to save the model: Model.prot ot ype.url = funct ion(met hod) { ret urn t his.prot ot ype.urlRoot + (met hod == 'creat e' ? '' : encodeURIComponent (t his.id)); }; T here are three kinds of persistence operations (since reads are handled by the data source): "create": PUT /user "update": POST /user/id "delete": DELET E /user/id When the model doesn't have a id, we will use the "create" endpoint, and when the model does have id, we'll use the "update"/"delete" endpoint. If you set Model.prototype.urlRoot to "http://localhost/user", then you'll get the urls above, or if your URLs are dif f erent, you can replace Model.prototype.url with your own f unction. Connecting Model.save() with the DataStore. Reading is done via the data source, but create,

update and delete are done via the data store. For the sake of convenience, let's redirect Model.save() to the DataStore: Model.prot ot ype.save = funct ion(callback) { Dat aSt ore.save(t his, callback); }; And do the same thing f or Model.dest roy: Model.prot ot ype.dest roy = funct ion(callback) { Dat aSt ore.delet e(t his, callback); }; Note that we allow the user to pass a callback, which will be called when the backend operation completes.

Managing the model lif ecycle

Since the data store is responsible f or caching the model and making sure that duplicate instances do not exist, we need to have a more detailed look at the lif ecycle of the model. Instantiation . T here are two ways to instantiate a model: new Model(); T he cache should do nothing in this case, models that are not saved are not cached. Dat aSource.nd(condit ions, funct ion(model) { ... }); Here, the models are f etched f rom the backend using some conditions. If the conditions are just model IDs, then the data source should check the cache f irst. When models are instantiated f rom data with an ID, they should be registered with the cache. Persistence operations: create, update, delete. Model.save(); // model.id is not set Once the backend returns the model id, add the model to the data cache, so that it can be f ound by id. Model.save(); // model.id is set Add the model to the data cache, so that it can be f ound by id. Model.delet e(); Remove the model f rom the data cache, and f rom any collections it may be in. Data changes. When the model ID changes, the cache should be updated to ref lect this.

Reference counting . If you want an accurate count of the number of models, you must hook into Collection events (e.g. add / remove / reset). I'm not going to do that, because a simpler mechanism -- f or example, limiting model instances by age or by number -- achieves the essential benef its without the overhead of counting. When ES6 WeakMaps are more common, it'll be much easier to do something like this.

Implementing the data store / cache

DataStore.add(), DataStore.has(), DataStore.save(), DataStore.delete(), DataStore.ref erence(). The implementation section is still a work in progress, my apologies.

11. Implementing associations: hasOne, hasMany

Defining associations. Associations / relationships are sugar on top of the basic data source implementation. T he idea is that you can predef ine the associations between models, f or example, that a post hasMany comments. T his might be described as: funct ion Post (args) { Model.apply(t his, args); t his.denit ion = { t ags: Tags, comment s: Comment s }; } We can f etch stuf f manually without assocation support. For example, assume that posts.comment_ids is an array of ids: db.t ag(post .comment _ids, funct ion(t ags) { t ags.forEach(funct ion(t ag)) { // ... }); }); But given several levels of nesting (post has comment has author), this gets old pretty f ast. It's the age-old problem of dealing with callbacks - which turns out to be pretty trivial once you add a couple of control f low patterns to your repertoire. T he f undamental ones are "series", "parallel" and "parallel but with limited concurrency". If you are unf amiliar with those, go read Chapter 7 - Control Flow of my previous book. Don't pretend to have a blocking API. Some f rameworks have taken the approach that they pretend to provide a blocking API by returning a placeholder object. For example: var comment s = post .get ('comment s'); // we do not have t he dat a for comment s, // but we'll ret urn a placeholder object for it T his is a very, very leaky abstraction. It just introduces complexity without really solving the issue, which is that you have to wait f or the database to return results. I'd much rather allow the user to set a callback that gets called when the data has arrived; with a little bit of control f low you can easily ensure that the data is loaded - or build a higher level mechanism like we will be doing.

APIs that appear not to incur the cost of IO but actually do are the leakiest of abstractions (Mikeal Rogers). I'd much rather opt f or the simple callback, since that allows me to explictly say that a piece of code should run only when the required data has arrived.

Building a nicer API f or f etching associated records

Now, I don't want to do this either: post .get ('t ags', funct ion(t ags) { post .get ('comment s').each(funct ion(comment ) { comment .get ('aut hor', funct ion(comment s) { // ... }); }); }); Instead, I think the right pattern (as advocated in my previous book) is to tell the system what I want and pass a single callback that will run when the data is loaded: post .wit h(['t ags', 'comment s.aut hor'], funct ion(post ) { // post .t ags; post .comment s and post .comment s[0..n].aut hor should now be loaded }); Basically, you tell the API what you want as the input, and give it a callback to run when it has done your bidding. Implementation . How can we build this? It is basically an API that takes a bunch of paths, looks up the metadata, makes data source calls to f etch by ID, and stores the data on the model, and then calls the continuation callback. The implementation section is still a work in progress, my apologies.

12. Views - Templating

What's in a template?
I would classif y templating systems not based on their input, but based on their output: as simple f unctions as f unctions and metadata as objects with lif ecycles T he simplest systems make string interpolation and array iteration more convenient. More complicated ones generate metadata that can be used as an input f or other systems. T he simplest templating system A template is the part of the view object that is responsible f or generating HT ML f rom input data. In other words, a template is a f unction which takes a single argument: base (context) and returns a string of HT ML.

funct ion it emTemplat e(base) { ret urn [ '<li>', '<div class="t odo', (base.done ? ' done' : ''), '">', base.t ext , '</div>', '</li>' ].join(''); } Of course, writing templates with this syntax is generally not pref erred. Instead, templating libraries are used in order to get the best of both worlds: the nicest possible template def inition syntax, and the perf ormance of using native JS operations. Templating syntax should have no perf ormance impact - you should always precompile your templates into their optimal JS equivalents. T he optimal output f or simple templates In theory, unless a templating library does something extremely unusual, all of the templating libraries should have similar perf ormance: af ter all, they only perf orm string interpolation on an input and ought to compile to similar compiled JS output. Sadly, in the real world very f ew templating languages actually compile to the optimal markup. Have a look at the results f rom this benchmark: Resig Micro-templating: 3,813,204 (3813 templates per ms; 61,008 in 16ms) (76 templates per ms; 1216 in 16ms) (46 templates per ms; 736 in 16ms) (15 templates per ms; 240 in 16ms)

Underscore.js template: 76,012 Handlebars.js: ejs: 45,953 14,927

I'm not discussing the causes here, because even with the slowest templating engine, the rendering itself doesn't have a signif icant impact in terms of total time (since even the slowest engines can cope with hundreds of template renders per 16 ms). In other words - despite large dif f erences (up to two orders of magnitude) in microbenchmarks - generating HT ML f rom a compiled template is unlikely to be a bottleneck no matter how slow it is, except on mobile browsers. Outputting metadata / objects with lif ecycles As I noted in the overview chapter f or the view layer, the key dif f erence between view layer implementations is their update granularity: whether views are redrawn as a whole (view-granular) or can be rendered at element-granularity or string-granularity. View-granular systems can just use the simple output where a compiled template is represented as a f unction that takes a set of data and returns a string. Element-granular and string-granular view layers need more metadata, because they need to convert the bindings into code that keeps track of and updates the right parts of the view. Hence, element-granular and string-granular rendering requires a templating system that outputs objects / metadata in addition to strings. Notice that this doesn't generally af f ect what f eatures are supported in the templating language: it just af f ects how granular the updates are and the syntax f or def ining things like event handlers.

Templating language f eatures

Let's have a look at some common templating language f eatures. Sadly, I don't have the time right

now to write a templating system - as cool and f un that would be, I'm pretty sure it would be a low payof f in terms of writing a book. String interpolation allows us to insert values into HT ML. Dependending on the update granularity, the tokens can be updated either only by re-rendering the whole view, or a single element, or by updating the content of the element with string-granular updates. <div> Hello {{ name }}! </div> Escaping HT ML . It is generally a bad practice not to escape the values inserted into HT ML, since this might allow malicious users to inject Javascript into your application that would then run with the privileges of whomever is using the application. Most templating libraries def ault to escaping HT ML. For example, mustache uses {{name}} f or escaped HT ML and {{{name}}} ("triple mustache") f or unescaped strings. Simple expressions. Expressions are code within a template. Many templating libraries support either a f ew f ixed expressions / conditions, or allow f or almost any JS code to be used as an expression. <li><div class="t odo {{ done? }}">{{ t ext }}</div></li> I don't have a strong opinion about logic-in-views vs. logicless views + helpers. In the end, if you need logic in your views, you will need to write it somewhere. Intricate logic in views is a bad idea, but so is having a gazillion helpers. Finding the right balance depends on the use case. Generally, templating engines support {{if expr}} and {{else}}f or checking whether a value is set to a truthy value. If the templating library doesn't support logic in views, then it usually supports helpers, which are external f unctions that can be called f rom the template and contain the logic that would otherwise be in the template. Displaying a list of items. T here are basically two ways, and they correspond to how sets of items are represented in the model layer. T he f irst option corresponds to observable arrays: you use an expression like each to iterate over the items in the observable array: {{view App.TodoList }} <ul> {{each t odos}} {{view App.TodoView}} <li><div class="t odo {{ done? }}">{{ t ext }}</div></li> {{/view}} {{/each}} </ul> {{/view}} T he second option corresponds with collections of models, where the view is bound to a collection and has additional logic f or rendering the items. T his might look something like this: {{collect ionview App.TodoList t ag=ul collect ion=Todos}} <li><div class="t odo {{ done? }}">{{ t ext }}</div></li> {{/collect ionview}}

Observable arrays lead to less sophisticated list rendering behavior. T his is because each is not really aware of the context in which it is operating. Collection views are aware of the use case (since they are components written f or that specif ic view) and can hence optimize better f or the specif ic use case and markup. For example, imagine a chat message list of 1000 items that is only updated by appending new messages to it. An observable array representing a list of messages that contains a thousand items that are rendered using a each iterator will render each item into the DOM. A collection view might add restrictions about the number of items rendered (e.g. only showing the most recent, or implementing incremental rendering by only rendering the visible messages in the DOM). T he observable array also needs to keep track of every message, since there is no way of telling it that the messages, once rendered, will never be updated. A collection view can have custom rendering logic that optimizes the renderer based on this knowledge. If we choose the "each" route f or collections, then optimizing rendering perf ormance becomes harder, because the mechanism most f rameworks provide is based on rendering every item and tracking every item. Collection views can be optimized more, at the cost of manually writing code.

Nest ed view def init ion

Templating libraries usually only support def ining one template at a time, since they do not have an opinion about how templates are used in the view layer. However, if the output f rom your templating system is a set of views (objects / metadata) rather than a set of templates (f unctions that take data arguments), then you can add support f or nested view def inition. For example, def ining a UserInf o view that contains a UserContact and UserPermissions view, both of which are def ined inside the App.UserInf o view: {{view App.UserInfo}} <ul> <li>User informat ion</li> {{view App.UserCont act }} ... {{/view}} {{view App.UserPermissions}} ... {{/view}} </ul> {{/view}} T his means that the output f rom compiling the above markup to object/metadata inf o should yield three views: UserInf o, UserContact and UserPermissions. Nested view def inition is linked directly with the ability to instantiate and render a hierarchy of views f rom the resulting object; in the case above, the UserInf o view needs to know how to instantiate and render UserContact and UserPermissions in order to draw itself . In order to implement this, we need several things: A template parser that outputs objects/metadata A view layer that is capable of rendering child views f rom templates Optionally, the ability to only render the updated views in the hierarchy T he f irst two are obvious: given markup like the one in the example, we want to return objects f or each view. Additionally, views that contain other views have to store a ref erence to those views

so that they can instantiate them when they are drawing themselves. What about the ability to only render the updated views in the hierarchy? Well, imagine a scenario where you need to re-render a top-level view that contains other views. If you want to avoid rerendering all of the HT ML, then you have two choices: Write the render() f unction yourself , so that it calls the nested render() f unctions only when relevant Af ter the initial render, only perf orm direct updates (e.g. via element-granular or stringgranular bindings) T he f irst option is simpler f rom a f ramework perspective, but requires that you handle calls to render() yourself . T his is just coordination, so not much to discuss here. T he second option relies on adding metadata about which pieces of data are used in the views, so that when a model data change occurs, the right views/bound elements can be updated. Let's have a look at how this might be done next.

Adding met adat a t o enable granular (re)-rendering

T he basic idea here is to take one set of strings (the names/paths to the model data in the global scope), and translate them into subscriptions on model changes (e.g. callbacks that do the right thing). For example, given this templating input: {{view}} Hello {{ window.App.current User.name }}! {{/view}} ... the output should be a view object, a template and a event subscription that updates the piece of the DOM represented by the {{window.App.current User.name}} token. Ref erences to items can be considered to be dependencies: when a observed value changes, then the element related to it should change. T hey might result in a subscription being established like this: Framework .observe('window.App.current User.name') .on('change', funct ion(model) { $('#$1').updat e(model); }); Where $('#$1') is an expression which selects the part to update. I am glossing over the implementation of the DOM selection f or the piece of DOM. One way that might be done - in the case of a element-granular view layer - would be to create a templating f unction that wraps those updateable tokens with a span tag and assigns sequential ID numbers to them: <div id="$0"> Hello <span id="$1">Foo</span>! </div> T he id attributes would need to be generated on demand when the view is rendered, so that the code that subscribes to the change can then ref er to the updateable part of the string by its ID. For string-granular updates, the same would be achieved by using <script > tags, as discussed in the overview chapter f or the view layer. To avoid having to type the f ully qualif ied name of the model data that we want to bind to, views

can add a def ault scope in the context of their bindings: {{view scope="window.App.current User"}} Hello {{ name }}! {{/view}} T his addition makes the subscription strings less verbose. T his is the gist of granular re-rendering. T here are additional things to consider, such as registering and unregistering the listeners during the view lif e cycle (e.g. when the view is active, it should be subscribed; when it is removed, it should be unsubscribed). Additionally, in some cases there is an expression that needs to be evaluated when the observed value changes. T hese are lef t as an excercise to the reader, at least until I have more time to think about them.

13. Views - Behavior: binding DOM events to HTML and responding to events
In this chapter, I will discuss the things that need to happen in order to respond to user events: attaching listeners to DOM nodes in the HT ML in order to react to user events handling cross-view communication abstracting common behavior in views

Dif f erent kinds of UI interactions

Additing interactivity is about taking a DOM event and responding to it by doing something in the model layer and the view layer. Let's look at a f ew dif f erent kinds of interactions using Gmail as an example, and see how they might af f ect the state of the application (e.g. the model/view layers). Model data change. Here, the user interaction results in a model property being set. For example, in Gmail, click a message to star it. T his might result in message.starred being set to true. Assuming that the view layer receives change events f rom the model, any views showing that message can then update themselves. Single view state change. Here, it is less clear which model is associated with the change. For example, in Gmail, click a collapsible section to show/hide it. T his is naturally expressed as a property of the view instance.

Multiple view state change. In this case, we want a single action to inf luence multiple views. For example, in Gmail, change the compactness of the app display density. T his will cause all views to adjust their display density, making them visually more compact. T here are two ways this might be implemented: by sending a transient message to which all views react, or by having a setting in the global scope that all views poll/subscribe to. Page state transition. What makes page state transitions dif f erent f rom the others is that it involves a wholesale change in the page. Views might be destroyed or hidden, and new views swapped in place of them. For example, in Gmail, click on "Compose" to start writing a new message, which loads up the message editor.

Binding DOM events to the View

What the examples above try to show is that in order to respond to user actions, we need to do two things: Listen to DOM events Given the event, f igure out what action makes sense Listening to DOM events is all about the lif ecycle of our view. We need to make sure that we attach the DOM listeners when the element containing the view is inserted into the DOM and removed when the element is removed. In essence, this requires that we delay event registration and make sure it each handler is attached (but only once), even if the view is updated and some elements within the view are discarded (along with their event handlers). Figuring out what action makes sense is part app programming, part f ramework capabilities. Whether we are using model-backed views or markup-driven views, we still want to make the most common operations simple to do by providing access to the related inf ormation. T he rest is appspecif ic.

Options f or specif ying the event-to-handler relations

Since the DOM only has a element-based API f or attaching events, there are only two choices: DOM-based event bindings. Framework-generated event bindings. DOM-based event bindings basically rely on DOM properties, like the element ID or element class to locate the element and bind events to it. T his is f airly similar to the old-f ashioned $('#f oo').on('click', ...) approach, except done in a standardized way as part of view instantiation. Here is an example: View.t emplat e = '<div>\ <input t ype="checkbox" class="select ">\ <img class="t oggleSt ar">\ <a class="hide">Hide</a>\ </div>'; View.event s = { 'click .select ': funct ion() { Emit t er.emit ('int ent :message:select ', t his.model); }, 'click .t oggleSt ar': funct ion() { t his.model.t oggleSt ar(); }, 'click .hide': 'hide', }; Framework-generated event bindings allow you to bind event handlers to HT ML without explicitly providing a element ID or selector f or the view. Here is an example: {{#view Foo}} <div> <input t ype="checkbox" {{onclick="Emit t er.emit ('int ent :message:select ', t his.model);"}}> <img {{onclick="t his.model.t oggleSt ar();"}}> <a {{onclick="t his.hide();"}}>Hide</a> </div> {{/view}}

Both of these are obviously just ways to call the DOM API to add event listeners to elements. T he dif f erence is that DOM-selector-based event bindings can be implemented much more simply. T he f ramework-generated event bindings require that the f ramework generates selectors to attach the event bindings, and that the metadata f or what events to listen to and what to do needs to be extracted out of the template into metadata or a view object. I f ind it f airly hilarious that we've basically come f ull circle: when JS launched, it began with onclick handlers inside HT ML. T hen people started hating on JS within HT ML. And now, we're back to def ining onclick handlers; except that now those onclick handlers actually get compiled into JS objects that represent the DOM that are managed by a f ramework. Of course, things are a bit dif f erent on this iteration: the patterns used are more sophisticated, we pass a custom scope to the event handler, the bindings f or the onclick handlers are more intelligent, and binding/unbinding events is handled by the f ramework. T here is a certain declarativeness to writing onclick handlers this time around. But still, I f ind it f unny.

Implementing event bindings

T here is a reason why many f rameworks make a view represent a single element: it makes binding events a lot easier if you can instantiate the element early on. T he initial render workf low looks something like this: var el, buer, view; if(rerender) { view.unBindEvent s(el); dest royElement (el); } buer = view.renderToBuer(); el = view.at t achToDom(buer); bindEvent s(el); I'm sorry I need to gloss over the details a bit, I will update this section later on. T he gist of the render process is that views go through a number of states: Not instantiated Instantiated and rendered to buf f er Attached to the DOM Destroyed Event bindings need to keep track of the view's state to ensure that the events are bound when a DOM element exists (since the only way to bind events is to have a DOM element f or it). How would this look in terms of code? For DOM-based bindings, something like this:

View.prot ot ype.render = funct ion() { var t emplat e = Templat eEngine.compile(t his.t emplat e); t emplat e(t his.dat a); t his.at t ach(el, View.event s); }; View.prot ot ype.at t ach = funct ion(el, event s) { event s.forEach(funct ion(select or) { var part s = select or.split (' ', 2), callback = event s[select or]; $(el).on(part s[0], part s[1], callback); }); }; Here, we are simply taking View.event s which is presumed to be a hash of event selectors and their associated callbacks, and using jQuery to attach those events. T he implementation is essentially identical f or the f ramework-generated bindings. Instead of using CSS classes, the event selectors are based on the markup that the template engine generated. T he only dif f erence is that the hash of event selectors comes f rom the templating engine rather than f rom the View object: View.prot ot ype.render = funct ion() { var met a = Templat eEngine.compile(t his.t emplat e); met a.t emplat e(t his.dat a); t his.at t ach(el, met a.event s); }; View.prot ot ype.at t ach = funct ion(el, event s) { // ... see above };

14. Consuming events from the model layer

T here are two areas of interest in the chapter, both of which f all under the larger issue of generating and consuming events: Re-rendering views in response to data changes. When data changes, we get a change event f rom the model layer. In response, we would like to re-render all the views that were af f ected by the change. Communication between views. Some actions - like the example in Gmail where you change the display density - require that multiple views change in response to the same user event. We need to pick a nice way to represent these changes, and trigger the right changes. T hese are both coordination problems. We want to specif y that when A happens, B and C should also happen. In essence, we are trying to bridge the gap between concrete DOM events and event handlers spread across multiple views: [Event s] < - > [Event handlers / act ions] Let's assume that we are trying to implement the f ollowing interaction: "when a user selects a message, the top menu should change and the message should appear as selected in the message list view". In this scenario, we are changing a piece of data and asking f or that change to be communicated to multiple views. T here are several ways we could do this:

Directly in the select event handler. T he naive and obvious way would be to write code in the list that explicitly calls the interested parties. MessageView.onSelect = funct ion() { message.set Select ed(t rue); list .check(message.id); menu.updat e(message); // one call for each ot her view t hat cares about t his operat ion }; However, the problem is that this is highly brittle since the views are tightly coupled: the message view knows about the message model, the list view and the menu view. Using a mediating controller One way is to use a mediating controller, which ref ers to the objects directly. T his looks something like this: MessageView.onSelect = funct ion() { cont roller.select Message(); }; Cont roller.select Message = funct ion(message) { message.set Select ed(t rue); list .check(message.id); menu.updat e(message); // one call for each ot her view t hat cares about t his operat ion }; Now, instead of views knowing about each other, they only need to know about a controller. Putting the code in a controller centralizes the coordination, but the code is still ugly and f ragile: since that code explicitly ref ers to each view, removing or breaking one view can potentially break the whole re-render. It's still the same code, you just moved it into a dif f erent object; this is just as f ragile as without a mediating controller (since the controller can't work without both views), though it is a bit more reusable since you can swap the controller. Using observables. Another alternative is to use observables. When someone selects a message, we can ref lect that either as a property of the message ("selected") or as part of a collection ("selectedMessages"): Observable properties. Selection is ref lected as a property on the model. Views subscribe to changes on that particular property and update themselves based on changes to that property. Observable collections. Selection is ref lected as a collection on the current page, or a property on a controller. Views subscribe to changes on that particular collection or controller property to update themselves. Here is how this might look as code:

MessageView.onSelect = funct ion() { AppModule.FooCont roller.set ('current Foo', t his); // current Foo is a observable propert y // each ot herView observes it , and performs // act ions based on change event s }; // init is called when t he ot her view is creat ed Ot herView.init = funct ion() { Framework .observe('AppModule.FooCont roller.current Foo') .on('change', funct ion(model) { Ot herView.updat e(model); }); }; While the views don't know about each other, they still know about the controller. Furthermore, the properties of the controller become an implicit API between the views. I say implicit, because the controller doesn't know it's being used f or this purpose. So instead of having an explicit controller f unction that knows about the views, you now have a controller property that is the API f or making calls between views and f or passing state between views. You haven't gotten rid of "views knowing about the controller"; it's just that the views are now also responsible f or registering callbacks on the controller properties. Of course, in the case of one view, another dependent view and one controller this isn't too bad. But the problem is that as the number of views, controllers and interrelationships increase, the number global state properties and dependencies on various pieces of state increases. Using global events. We can also implement this using a global event dispatcher / event aggregator. In this case, selection is ref lected as an global event. When the user selects a message, a global event is emitted. Views subscribe to the selection event to be notif ied and can publish messages via the global event emitter instead of knowing about each other. MessageView.onSelect = funct ion() { global.emit ('message_select ed', t his); }; Ot herView.init = funct ion() { global.on('message_select ed', funct ion(model) { message.set Select ed(t rue); }); }; T he global event emitter is the single source of events f or the views. Views can register interest in a particular event on the global event emitter, and models can emit events on the global event emitter when they change. Additionally, views can send messages to each other via the global event emitter without having an observable property change.

Observables and event emitters: two dif f erent schemas f or specif ying interest
Basically, the choice boils down to either having views send message to each other, or having views observe models. T hese options look dif f erent, but f undamentally they are the same thing, just using a dif f erent schema. Observables: funct ion() { ... }.observe('App.Todos');

Event emitters: Todos.on('change', funct ion() { ... }); But really, the only dif f erence is what the schema is. With observables, we specif y interest in a change by using the name of the source object in the global scope. With event emitters, we specif y interest by the type of the message. Let's look at those two again: Observables: global.observe('App.Todos:change', funct ion(model) { /* ... */ }); Event emitters: App.Todos.on('change', funct ion(model) { /* ... */ }); Now, with a minor change, the two patterns look a lot more similar. T he dif f erence is that in one, the standard way to say we want to be inf ormed about a change is to use the name of source object vs. in the other, we subscribe via the type of the message. T he "views observe models directly" and "models publish to a global EventEmitter" both introduce a level of indirection between model change events and view re-renders. T his is good, because it means that there is no code that specif ically ref ers to particular views - if a view is removed or doesn't work, then only it will be af f ected. However, these two approaches have dif f erent implications.

Observables vs. event emitters

Observables Observables are the way in which markup-driven views (e.g. view systems with a templating system emphasis) implement re-rendering on demand. T here are two parts to this system: the name resolution / event attachment system which, given a input like this: Framework.observe('App.Todos', funct ion() { ... }); ... will attach the callback f unction to 'App.Todos' when App.Todos is instantiated (and re-attach the callback f unction when that object is replaced). Once the events are attached, the second part of the system needs to determine which views have changed in response to the data change event. T his is basically a process of taking the event and matching it against the currently active observers on a global level, and then triggering the observer callbacks where appropriate. Global events. Here, we introduce a single shared eventemitter which acts as a broker f or events. Each view expresses interest in a particular set of events by subscribing on the global event emitter. When an event which needs to be handled by multiple views occurs, a global event is emitted. T he advantage here is decoupling, since none of the views need to know about each other: they just know about the global eventemitter and how to handle a particular event.

Three choices: transient, model, and collection events

Whether you pick event emitters or observables, you still are lef t with the choice between three ways in which user actions can be represented in your app. You can represent an action - like changing the display density or selecting an item - as: A transient event. In this case, there is no property or collection associated with the change. Any views that are subscribed to notif ications about the event will have their event handlers triggered. A model change event. Here, the result of the action is that a property on a model changes. T his causes change events f or that property, which then trigger the event handlers/callbacks f or interested models. A collection change event. You might represent the user action as a change on a collection or an observable array. T he set of items is changed by the user action, and this triggers interested listeners in the view layer to update their contents. Using a collection makes it easy to f ind out which model(s) are selected, which is usef ul in cases (like the selection example). Observables generally do not support triggering transient events, since they are based on the idea that everything is a property of a model or observable array. While model and observable array changes generally cover most actions, it may be that some actions are best represented as transient events. For example, clicking a user in a list to bring up an edit dialog might be better implemented as a transient event rather than, say, a list click event handler that is tightly coupled to a particular dialog. With observables, every interaction, event if it is limited to a single activity, will exist as a property on the model or collection (unless the developer goes outside the f ramework). T his is a disadvantage: each interaction adds to the global complexity of models. With event emitters, you tend to perf orm the same model/collection binding by having views that are bound to either a collection or a model: Model bound views Collection/observable array bound views Model-bound views take a single model, and represent it in the DOM. Change events f rom that view trigger updates in the DOM. Collection-bound views represent a set of models or data items in a more complex set of markup. T hey may implement additional f unctionality that allows them to ef f iciently render a set of items. For example, updating a row in a table should only update that row, rather than the whole table. T his allows the collection-bound view to potentially be more ef f icient at the cost of requiring the programmer to think about perf ormance more.

Which one should I pick?

T he observables basically make the choice to have the data accessed by its global name (which is used to establish subscriptions). T he global events approach makes the choice of having the data be pushed via a global intermediary; each view only knows about this intermediary and instead of ref erring to things by their name in the global scope, the event data is received directly with the event. Observables abstract out the instantiation of the object they ref er to, since you can start observing on an object even if it does not exist. However, the same applies to the global event emitter: you can register a event listener f or an event even if that event is not triggerable.

Basically, these are equivalent approaches, using dif f erent naming schemes. I would pref er a naming scheme that is based on the event type, rather than one that is based on the location of the data in the global scope. Ref erring to things by their location in the global scope creates a dependency: that particular variable has to be set in order to receive data. Worse yet, a view that explicitly accesses the state (e.g. via some global name) now depends on that global state (e.g. MyApp.MyModule.currentFoo) to be set up. While this is easy to write, it makes the view progressively harder to test and harder to reuse. < < Previous Chapter | Next Chapter > >