A bit of a show ‘n’ tell tonight at the fortnightly Async JavaScript meetup, hosted at The Skiff in Brighton… notes below.

Links for the talks ought to be added to the Async blog post (http://asyncjs.com/showntell3/) by the speakers.

The next Async features Kyran Dale (session 7 below) on module loading (‘requireJS’), in two weeks time, also @ The Skiff.

1. Mike de Boer (from ajax.org) – Cloud9 IDE

Written in JS (as a node app) for dev’ing JS. Nice. “npm install cloud9″

Mounts filesystem/WebDAV server.

You can write a node app, which is run as a separate app and has the output logged to a console. They’ll have an online version up soon.

Does live debugging, and you can variables real-time via breakpoints. You can also edit the code when execution is paused. i.e. hot-code swapping. He mentioned node inspector (what’s that?).

The code editor can cope with 134k lines of code. Which textmate can’t. Nice.

You get a command-line too, for using git (other version control systems coming later).

http://github.com/ajaxorg

2. Glenn Jones (Madgex) – Draggables

HTML5 experiment for dragging data between systems.

Lets you drag across browsers, which is cool. Also you can drag html files (and zipped html files!).

Really nice little thing with a bookmarklet injecting a draggable icon into a page (well, on Firefox).

Glenn is also working on a microformats parser that works cross-browser, which he will open-source.

3. Phil Hawksworth (The Team) – nodejs & ndistro – “multiple projects. quickly.”

I.e. how Phil works.

Problem with node is deployment (i.e. build) can be hard. Plus, the codebase is changing frequently. So how to cope?

ndistro – a node distribution toolkit (github.com/visionmedia/ndistro)

Note: installing involves running install script as root (sudo su Phil recommends)

You write ndistro files for each project, where you specify the version of node and any modules from github

Phil like ejs as a templating language

Phil has some very nice presentation techniques – videos of terminals and a funny, eased, scrolling of a webpage. Guess keynote == good?

Seeing as ndistro files are just bash scripts, you can nest another running of ndistro inside the ndistro file

expressjs = “almost” MVC for node

After a Q: Not sure if npm would play nicely with this.

4. Prem (@premasagar) & Graeme Sutherland (@grasuth) – Sqwidget

http://github.com/premasagar/sqwidget

The problem: widget developers have a really big problem developing something that runs in other people’s environments (which is like cross-browser development, but worse).

One solution: iFrame

Their solution: sqwidget.

It sandboxes CSS and JavaScript. This is cool, because you can do things like lightboxes, which you can’t do with iFramed widgets.

The widget is embedded using a DIV and its data attributes to do settings

Q: option to swap in different templating? A: yes. Built-in one is a hand-built library called Tim.

Q: licence? A: MIT

5. Mark Wubben, from Copenhagen (who needs a place to live in London) – Chrome2iPad

Problem: how do I get my tabs I have open in Chrome into my iPad?

Solution: Chrome2iPad.

There is a Chrome extension using Socket.io (a node module) which hotlinks to a webpage on the iPad to the Chrome browser via a Chrome extension

Not released on github yet. When it is available, it will be on http://github.com/novemberborn

6. Paul Downey (@psd, Osmosoft) – TiddlySpace

TiddlyWiki is a single-page HTML app.

I’m writing these notes in a TiddlyWiki…

TiddlyWiki is interesting because it can save itself when on a file URI.

All the plugins are written in JavaScript.

TiddlySpace is like TiddlyWikis in the sky. It’s slightly different from file-based TiddlyWikis, since you can follow people.

TiddlySpace is based on TiddlyWeb, which is an open-source hosting platform for tiddlers, with a RESTful API.

A tiddler in a TiddlyWiki is a div; a tiddler on TiddlySpace can be HTML, JavaScript, SVG – anything.

7. Kyran Dale (academia) – JavaScript physics

Impressed by JavaScript’s performance to do simulations.

Shows Craig Reynold’s Boids running 300 objects, drawn in canvas. Chrome on Linux. Real-time variable changing.

Now a robotics demo – 100 robots (Brightenbergs) with light-sensors and lights. They go mad. But without slow-down.

Next a magnetic controller. This one does some weird stuff, but again, in a peformant way.

Q: source? A: will be available, probably on a Mercurial repo (http://bitbucket.com/keirandale)

8. Morgan Roderick (@mrgnrdrck) – PubSub in JavaScript

Writes really large systems in JavaScript.

PubSub is messaging in JavaScript. It is an alternative to the “observer pattern” (see Java, apparently).

Keeps different bits of your system nicely decoupled.

Don’t use PubSub when you have 1:1 communication, or need accurate control.

http://github.com/mroderick/pubsubjs

Q: is is threaded? A: all messages are passed asynchronously. In terms of performance, it’s about 10x faster than jQuery.

Q: how is it different to custom events in jQuery? A: it doesn’t depend on the DOM

Q: how do you wake listeners up? A: They just get called. If a call fails, the exception is caught and it is called again.

thanks to flowerhouse on flickr

The idea of running JavaScript on the server has been around for a while now (think Jaxer back in 2008), but it recently got a big boost with the featuring of Node.js at JSConf in November 2009. Node.js found immediate fame by demonstrating blinding performance as a web server, and by building on the hotly hyped V8 JavaScript engine Google bundled with Chrome.

I have a big interest in server-side JavaScript (SSJS) because I generally code in the browser and would enjoy being able to make things happen on a server without having to resort to another language. The ideal would be to take client-side code and run it unchanged on a server. The last step is pain-free application hosting – call it “Platform as a service” for want of a less boring name.

Over a year ago, Joyent bought Reasonably Smart, a cloud provider of SSJS, and subsequently transmuted it into Smart Platform, which runs on their distributed infrastructure. This followed the shining example of Heroku and Google App Engine, in that application deployment became a matter of pushing some new code to a Git repository online. For me, this was fantastically attractive, as I avoid touching servers as much as realistically possible. They came out with Smart Platform shortly before the demise of AppJet’s free service, which I had been happily trying for a while, as it did much the same thing.

Nearly a year on, and Smart Platform hasn’t really taken off, if Google Trends is anything to go by. And my ambitions to run my entire web estate on identical server-side and client-side code have rather fallen by the way-side. I am happy to see that Smart Platform now has a permanent full-time coder (@konobi), who is making Smart Platform play nicely with the CommonJS spec for server-side JS, but I don’t yet know when Joyent are going to drop a new release, or where they’re going with it.

Enter Node.js. The interest in Node.js has stayed high since its November launch, with developers building an assortment of frameworks and tools on top of it or with it, in effect legitimising it as a serious development environment for software running on a server.

Of significant interest to me is the jsdom project, which brings the browser environment to Node.js in a similar way to how Env.js brought it to Rhino (and Ruby). Env.js has supported jQuery since the get-go, and there are rumblings of jQuery support in Node.js via jsdom. Both these situations inspire joy as, if they pan out successfully, I’ll be able to use one methodology for writing and testing JavaScript applications, whether they run in the browser or on the server. Unfortunately, I’ve not seen Env.js working on any hosted platform (although Smart Platform may get it soon).

The number of cloud providers for Node.js has increased from 0 to 2 very recently: Heroku has announced a very experimental Node.js stack, and ElusiveHippo is in development, releasing accounts slowly. I should probably also mention that Ryan Dahl (who created Node.js) works for Joyent, so I wouldn’t be suprised if a Smart Platform-style service came out for Node.js.

Now then, if Coda starts supporting Git, or I start doing all my code editing in Bespin, I think with a bit of fairy-dust I’m going to have a pretty slick JavaScript-only setup fairly soon…

I’ve recently been putting together an Adobe AIR application using HTML, CSS and JavaScript (aka Web Standards), which is very empowering and everything, but turned out to be a little tricky to write tests for. All the interesting stuff I wanted to do through AIR’s window.runtime object, such as opening native windows and messing with dock icons, is missing from the browser if you’re trying to test the JavaScript components in isolation. The ideal would be able to run your QUnit tests within the AIR runtime itself.

Fortunately, it’s straightforward to create a test app that wraps the QUnit test runner in the AIR runtime and exercises your code. Here’s how I got going:

Setting up the file structure

The folder structure I am using for the AIR app is like this:

/myApp
   myApp.html
   myApp-app.xml
   /js
      myApp.js
   /css
      styles.css

The myApp-app.xml file is the “application descriptor” that AIR uses to create the .air package when you build the app.

I added two folders at the same level as /myApp, to contain the test framework and test code; the complete structure looks like:

/myApp
   myApp.html
   myApp-app.xml
   /js
      myApp.js
      AIRAliases.js
      jquery-1.3.2.min.js
   /css
      styles.css

/myAppTests
   runner.html
   tests.js
   myAppTests-app.xml

/qunit
   testrunner.js
   testsuite.css

Configuring the test app

The application descriptor for myAppTests is not very different from the descriptor for myApp; here they are:

myApp-app.xml:
<?xml version="1.0" encoding="UTF-8"?>
<application xmlns="http://ns.adobe.com/air/application/1.5">
    <id>examples.html.myApp</id>
    <version>0.1</version>
    <filename>myApp</filename>
    <initialWindow>
        <content>myApp.html</content>
        <visible>true</visible>
        <width>400</width>
        <height>800</height>
    </initialWindow>
</application>

myAppTests-app.xml:
<?xml version="1.0" encoding="UTF-8"?>
<application xmlns="http://ns.adobe.com/air/application/1.5">
    <id>examples.html.myAppTests</id>
    <version>0.1</version>
    <filename>myAppTests</filename>
    <initialWindow>
        <content>myAppTests/runner.html</content>
        <visible>true</visible>
        <width>400</width>
        <height>800</height>
    </initialWindow>
</application>

The main structural difference is that myApp-app.xml has an initialWindow.content property set to myApp.html, whereas myAppTests-app.xml refers to the test runner from a directory above: myAppTests/runner.html. This is needed because the test app needs to have its working directory set to the top-level of the file structure, otherwise runner.html won’t be able to get at all the files it needs in other directories (more below).

Putting the correct files in runner.html

Here’s a runner.html that works with the folder structure described above:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html lang="en">
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
		<title>Test Suite</title>
		<link rel="stylesheet" type="text/css" href="../qunit/testsuite.css">
		<script src="../myApp/js/AIRAliases.js" type="text/javascript"></script>
		<script src="../myApp/js/jquery-1.3.2.min.js" type="text/javascript"></script>
		<script src="../qunit/testrunner.js" type="text/javascript"></script>
		<script src="../myApp/js/myApp.js"></script>
		<script src="tests.js" type="text/javascript"></script>
	</head>
	<body>
		<h2 id="banner"></h2>
		<h2 id="userAgent"></h2>
		<ol id="tests"></ol>
		<div id="main"></div>
	</body>
</html>

Running the test suite in debug mode

The easiest way to run the test suite is to use the built-in Adobe Debugger, which lets you test your app without having to build it into an AIR package:

adl myAppTests/myAppTests-app.xml .

Don’t forget the ‘.’ character on the end – this tells the debugger to run with the top-level directory as the working directory – without it, the default is the same directory as XML file.

It ought to be self-explanatory. I’m thinking about wrapping this up in an iPhone app so I can pay people for free. Rather than use PayPal or something.

Sorry it’s a bit blurry.

Source (hopefully changing lots) on github.

It’s not as simple as hitting an edit button yet, but it’s still worth having a play. The use of Sammy to handle and dispatch incoming requests, along with a single “bootstrap.js” file that controls all the SSJS, means that it ought to be fairly easy to port over single-page AppJet apps to Smart Platform. Here’s some notes I’ve gathered whilst porting over an application that acts as a gateway between tarpipe’s REST connector and a Yahoo! Pipe, allowing you to send data to a Yahoo! Pipe for processing.

Request object

In AppJet, you get the POSTed variables and the query-string parameters both in request.params. In Smart, the query-string parameters are in request.query with the raw string in request.queryString; POSTed variables are in request.body with the raw datastring in request.content.

Dispatching

In AppJet, you have the option to define different functions for handling different types of request e.g. GETs, POSTs and requests to different resources e.g. /this, /that. Smart uses Sammy, which gives you a similar mechanism to choose which functions respond to different incoming requests, although you also have the option to use a single function main(request) {...} to handle all responses. A concrete example is:

AppJet:
    function get_resource() { ... }
Smart:
    GET("/resource", function() { });

Writing a response

AppJet provides a default chrome that wraps an application, which you can remove by setting page.setMode('plain'); then you use functions like print(response) to send the respons. Smart does it slightly differently, where whatever you return from a function is sent as the response. It doesn’t seem possible to send HTTP header information.

Making HTTP requests

AppJet provides functions like wpost and wget to make HTTP calls. Smart provides the system.http.request function to do the same sort of thing. The difference in the returned response is that AppJet gave you a raw string of the response body, whereas Smart returns an object containing headers and content properties.

Deployment

The lack of an in-browser edit experience has been my complaint of late, but doing the Git bit to deploy my app to Smart was straightforward, and now it’s up at http://f7e3dbf.smart.joyent.com.

Try it out with:

curl -d 'pipeId=lAUEoB1R3hGTqt6hggSecQ&data={"items":[{"title":"A title","description":"Any description","link":"http:\/\/example.com"}]}' http://f7e3dbf.smart.joyent.com

Code

For the sake of completion, here’s the source code. I’ve gone from this:

/* appjet:version 0.1 */
import("lib-json"); 

var p = request.params;
var result = "";
var output = ""; 

function serialize(obj) {
    var str = "";
    for(var i in obj) {
        str += "&"+encodeURIComponent(i)+"="+encodeURIComponent(obj[i]);
    }
    return str.substring(1);
}
if(p && p.data && p.pipeId) {
    var data = JSON.parse(p.data);
    var pipeId = p.pipeId;
    var items = data.items;
    if(items) {
        var url = "http://pipes.yahoo.com/pipes/pipe.run?_id="+pipeId+"&_render=json";
        // tarpipe only POSTs 1 item
        var result = wpost(url,{title:items[0].title,description:items[0].description});
        if(result) {
            result = JSON.parse(result);
            if(result) {
                output = result.value;
                if(!output.items[0].title) {
                    output.items[0].title = "returned from Yahoo! Pipe at: "+pipeId;
                }
                output = JSON.stringify(output);
            } else {
                result = "bag result from Yahoo! Pipe at: "+pipe;
            }
        } else {
            result = "no result returned";
        }
    } else {
        result = "no items in data input";
    }
} else {
    result = "no data or no pipe input";
} 

page.setMode('plain');
output ? print(output) : print(result);

To this:

system.use("com.joyent.Resource");
system.use("org.json.json2");

function serialize(obj) {
    var str = "";
    for(var i in obj) {
        str += "&"+encodeURIComponent(i)+"="+encodeURIComponent(obj[i]);
    }
    return str.substring(1);
}

function main(request) {
	var p = request.content!="" ? request.body : request.query;
	var result = "";
	var output = "";
	/* for debug input
	for (var i in p) {
		output += i+", "+p[i]+"\n";
	}
	return output;*/
	if(p && p.data && p.pipeId) {
	    var data = JSON.parse(p.data);
	    var pipeId = p.pipeId;
	    var items = data.items;
	    if(items) {
	        var url = "http://pipes.yahoo.com/pipes/pipe.run?_id="+pipeId+"&_render=json";
	        // tarpipe only POSTs 1 item
	        var toPost = "title="+items[0].title+"&description="+items[0].description;
	        var result = system.http.request("POST",url,null,toPost);
	        if(result && result.content) {
	            result = JSON.parse(result.content);
	            if(result) {
	                output = result.value;
	                if(!output.items[0].title) {
	                    output.items[0].title = "returned from Yahoo! Pipe at: "+pipeId;
	                }
	                output = JSON.stringify(output);
	            } else {
	                result = "bag result from Yahoo! Pipe at: "+pipe;
	            }
	        } else {
	            result = "no result returned";
	        }
	    } else {
	        result = "no items in data input";
	    }
	} else {
	    result = "no data or no pipe input";
	}
	return output || result;
}

(The most interesting difference is that Streams combines Tarpipe’s event-driven model with Yahoo! Pipe’s modular data processing model. For more on this project and its ambition to open-sourcing, see LShift’s and the BBC’s blog posts.)

I have been using WireIt, an open-source JavaScript library, to aid with the interface prototyping. Fortunately for me, WireIt’s latest version (v0.4), announced shortly before I started this project, includes a fully-featured “Wiring Editor”, which gets you up and running with the familiar Yahoo! Pipes-style editor.

The brief for the prototype was to be able to complete a round-trip of loading up a Stream (stored as a document in CouchDB), edit it, save it (back to CouchDB) and be able to load it up again.

I thought it would be helpful to bring to your attention the things you need to be aware of if you want to use WireIt’s Wiring Editor and adapt it to work with an independent source of module and workflow data.

Stage 1 – different data formats

It is hardly worth saying that there is no point trying to persuade your data store owners to change their data format so your job is made easier. Apart from this obvious need to convert from one data format to another, you also have to deal with the fact that WireIt only keeps hold of the data it needs to display and configure its modules.

The best place I could find to hook into the loading process was by overriding the function called after the load button is pressed:

WireIt.WiringEditor.prototype._onLoad = WireIt.WiringEditor.prototype.onLoad;
WireIt.WiringEditor.prototype.onLoad = function() {
    this.service.listWirings({group:true}, {
        success: function(result) {
            var responseText = result.responseText;
            if(responseText) {
	        this.pipes = convertFromStreams(responseText);
                this.pipesByName = {};
                this.renderLoadPanel();
                this.updateLoadPanelList();
                this.loadPanel.show();
            }
        },
        failure:function() {
            console.log('failure when loading, args:',arguments);
        },
        scope: this
    });
};

The data format of a combination of blocks and wires in WireIt looks like this:

var modules = [
	{
		name: ,
		working: {}
	}
]
where <em>working</em> is defined as:
working = {
	"modules":[
		{
			"name":"FormContainer",
			"config":{
				"position":[115,58]
			},
			"value":{
				"email":"",
				"firstname":"",
				"happy":true,
				"lastname":"Dupont",
				"title":"Mr",
				"website":""
			}
		}],
	"properties":{
		"description":"",
		"name":"teste"
	},
	"wires":[
		{
			"src":{
				"moduleId":0,
				"terminal":"br"
			},
			"tgt":{
				"moduleId":1,
				"terminal":"tr"
			}
		}
	]
}

The convertStreams function changes each stream document available in CouchDB into the format above.

WireIt doesn’t keep hold of any properties you attach to blocks that aren’t used explicitly. This means you need to keep hold of any properties you need to save back to your data store. I did this by keeping a globally available variable, WireIt.customFields:

...
if(node.type) {
    WireIt.customFields.nodes[i].type = node.type;
} else {
    WireIt.customFields.nodes[i].terminal = node.terminal;
}
...

Stage 2 – loading the list of blocks from your data store

As well as loading up descriptions of a full combination of blocks and wires, I wanted to load all the available blocks in CouchDB into WireIt so they’d be available for making workflows with.

The WireIt Wiring Editor doesn’t load its blocks from a remote system, you have to provide all the definitions in a JSON “language” file when you create the Wiring Editor. I hooked into the setup of the Wiring Editor after it had finished loading its backend-describing SMD file (see below for explanation of this SMD malarkey):

WireIt.WiringEditor.prototype.onSMDsuccess = function() {
	WireIt.customFields = {};
	this.populateModules();
};

The populateModules function is responsible for turning the description of the block from CouchDB into the format WireIt uses for defining available blocks, which looks like this (note it is different to the format used to describe a block that is part of a workflow, described above):

var modules = [
	{
		"name": "detect_iplayer",
		container: {
			xtype: "WireIt.FormContainer",
			type: "regexp_replace",
			title: "Detect iplayer and remove",
			icon: "images/detect_iplayer.png",
			fields: [
				{"inputParams": {"name": "caseinsensitive", "label": "case insensitive?", "value":true}, "type": "boolean"},
				{"inputParams": {"name": "regexp", "label": "regular expression", "value": "(iplayer)"}},
				{"inputParams": {"name": "dotall", "label": "dot matches all?", "value": false}, "type": "boolean"},
				{"inputParams": {"name": "multiline", "label": "multiline?", "value": false}, "type": "boolean"},
				{"inputParams": {"name": "replacement", "label": "replacement", "value": "\\1"}}
			],
			width:250,
			terminals: [
				{"name": "input", direction: [-1,0], offsetPosition: {left: 110, top: -15}},
				{"name": "positive", direction: [1,0], offsetPosition: {left: 60, bottom: -15}},
				{"name": "negative", direction: [1,0], offsetPosition: {right: 60, bottom: -15}}
			]
		}
	},
	...
]

Stage 3 – adding default layout

The Streams data store doesn’t contain any information about how the blocks in a stream should be laid out, as this is not important to the execution of the streams. However, for loading a stream into WireIt, I wanted to give the blocks a default layout so they didn’t end up on top of each other.

I picked a layout that put the blocks equally spaced around a ring. This algorithm gives you the coordinates of a set of count blocks. It assumes the blocks are going to be square, and sets the radius of the ring so that the blocks would just touch if they were square.

function calculateLayout(count,width) {
	var coords = [];
	var n = count || 4;
	var h = width || 100;
	var radius = (n/Math.PI + 1.5)*h/Math.sqrt(2);
	var theta = 0;
	for (var i=0; i<n; i++) {
		theta = 2*Math.PI*i/n;
		coords[i] = [];
		coords[i][0] = radius*(1+Math.cos(theta)); // x
		coords[i][1] = radius*(1-Math.sin(theta)); // y
	}
	return coords;
}

Stage 4 – different load/save API (and the limitation of yui-rpc)

The Wiring Editor is hard-wired to use an implementation of JSON-Schema called “YUI-RPC”, along with a defining Service Mapping Description (SMD – think WSDL for JavaScript). The down-side to this is that the implementation does not currently support HTTP verbs other than GET and POST, nor is it obvious how to extend the implementation in line with the spec. If your API requires the use of PUT (as CouchDB does to update or create named documents), then you will need to do something about this.

YUI-RPC supported what I needed for loading, so supporting CouchDB was only as hard as changing one of the method specifications:

"listWirings": {
    "transport": "GET",
    "envelope": "URL",
    "target": "couchdb/_design/feeds/_view/join",
    "parameters": [
        {"name":"group", "type":"boolean"}
    ],
    "description": "Get a list of streams"
}

To support PUT for saving, I overrode WireIt.WiringEditor.prototype.saveModule and, deciding to go with what I know, stuck in a block of jQuery that I’d had hanging around from a test working with CouchDB’s API.

General comments on prototyping with single-page applications

I made the prototype as an application that could just be served as a single HTML page from a file: URI. I strongly support this over trying to make your prototype work with some web framework or other, as you will lose valuable time setting up the framework in the first place.

Working with HTML files loaded as files gives you a couple of special properties: you can read and write from the file system and make cross-domain XMLHttpRequests (on many browsers). I had to modify YUI’s Connect object a bit to support the cross-domain XHR requests. For more information on the special properties of HTML pages viewed over file: URI’s, see these sections of a W3C position paper about TiddlyWiki written by Osmosoftonian Paul Downey.

function log() {
   if(console) {
      console.log.apply(this,arguments);
   }
}

However, this function breaks Safari (at least it does on version 3.1.2) – causing a helpful message TypeError: type error. At the moment, Safari’s console.log only takes one argument, so a way to fix the above function is as follows:

function log(s) {
   if(console) {
      if(navigator.userAgent.toLowerCase().indexOf("applewebkit") != -1) {
         console.log(s);
      } else {
         console.log.apply(this,arguments);
      }
   }
}

I’ve had a look. The clearly exciting thing, from my point of view, is that SproutCore produces static HTML, CSS and JavaScript. And it does this by asking you to write JavaScript. So, that’s a good thing. Static web applications are easy to scale and are looked on favourably by search engines. The interaction with any server-side (for data storage or web service access) is done using their simple REST API.

It almost looks like SproutCore has been designed as my web development framework of choice – the problem is that SproutCore uses a Ruby-based templating language called “Erubis” to create what it terms “views” (in the Model-View-Controller tradition); these are essentially the structure for the web pages. So the development behaviour becomes: do your coding in JavaScript and Ruby, compile the whole thing down to HTML, CSS and JavaScript and then upload to the server. I would have thought it would make more sense to write your templates in something that resembled HTML in the first place, and perhaps made use of JavaScript.

Notwithstanding the small deviation from the HTML, CSS, JavaScript toolset, SproutCore takes the idea of smart clients and dumb servers seriously, and does so more than any other web development framework out there.

There are several ways of getting around this security restriction, known as Javascript’s same-origin policy, but they generally involve using iframes and other files on your own server. I wanted to keep everything in one self-contained file, so I pretty much ruled these out. The idea of using a local proxy was thrown out for the same reasons.

A found a some info that covered “dynamic Script tags”, which requires the remote server to serve you a tailored javascript file that called a function you supplied the name of, with the data you wanted returned as a parameter to that function, but that was really just a facade on top of another proxy story.

Then I hit upon this wonderful website, which gave an insight into an API architecture being adapted by, amongst others, Yahoo!. The idea revolves around the API itself returning JSON encapsulated in a function call, so the body of the response is essentially the parameter to the function. You add a Script tag to your page with the API call as the src attribute and when the browser loads the script, it ends up calling the function, which you write support for in your code.

// adapted from http://www.hunlock.com/blogs/Howto_Dynamically_Insert_Javascript_And_CSS
var headID = document.getElementsByTagName("head")[0];         
var newScript = document.createElement('script');
newScript.type = 'text/javascript';
newScript.onload=scriptLoaded;
newScript.src = 'http://api.flickr.com/services/feeds/photos_public.gne?tags=sunset&format=json';
headID.appendChild(newScript);

jsonFlickrFeed = function(feed) {
   // deal with the feed!
};

// when the script is finished, remove the script node we added
// removeNode is a custom function defined in the TiddlyWiki source code: http://svn.tiddlywiki.org/Trunk/core/js/Dom.js
// It ensures that the eventhandler is removed before deletion
scriptLoaded = function() {
	removeNode(newScript);
};

It’s simple, elegant and easy to work with. There are two alternative implementations that I’ve seen today: the first and simplest for the server to support, is exemplified by Flickr – the function call wrapped around the response data is fixed and you have to know what it is to write support for it [edit - they do support the next method too]; Yahoo! Pipes (and more generally, Yahoo!’s other developer API’s) supports the second method – the API takes the name of a callback function as a parameter and then wraps the data in a call to this function, which takes away the dependency on knowing the specific function to write support for.

This isn’t the perfect solution, as it leaves you wide open to exploitation from cross-site scripting, so I don’t recommend using this with API’s that you don’t trust. I feel pretty safe using Yahoo!’s API’s, but someone less salubrious could inject whatever they wanted in there. You don’t really want that when the script has access to the global object and, therefore, everything on your page.

This takes me one step closer to making it easier to replicate your offline TiddlyWiki environment online… Hurrah…

The confusion arises because there are various ways to get hold of the content of an element: nodeValue, value, data, textContent, innerText and innerHTML all seem to do the job in different circumstances. In the hope that I’m not the only one with this problem, I looked into the matter…

In a nutshell:

• nodeValue returns attributes or text (read/write)
• value returns the text content of an attribute (read)
• data is used to access text content of a node (read/write)
• textContent returns the text content of a node and its descendents (read/write)
• innerText returns the text content of a node and its descendents (read/write)
• innerHTML returns the HTML contained by an element (read/write)

nodeValue, counter-intuitively, does not return the text content of a node such as a paragraph i.e. it is not a W3C analogue of innerHTML. nodeValue is in the DOM Core specification and is well supported by browers. It is a read/write attribute.

This table from developer.mozilla.org shows the return value of accessing nodeValue on various nodes:

value returns the text value of a node’s attribute. This does not work with text nodes. value is in the DOM Core Specification and it is very poorly supported in Windows IE, although Firefox does support it. It is a read-only attribute.

data is used to address the content of a text node. It is the same as nodeValue for that node. data is in the DOM Core Specification and it is well supported by browsers. It is a read/write attribute.

textContent/innerText returns the text contained in a node and its descendants, ignoring any contained tags. Whilst textContent is in the DOM Level 3 Core Specification and it is not really supported at all except in Firefox, innerText is well supported by browsers, the exception being buggy support in Mozilla 1.75. Both are read/write attributes.

innerHTML returns the markup contained in a node and its descendants. Whilst innerHTML is not in a W3C specification, most browsers support it and it is often faster than W3C DOM methods. It is a read/write attribute.

I’m heavily indebted to quirksmode.org and the W3C DOM specifications for this information.