Web designers

January 27, 2010 – 2:48 pm

(I was thrilled to read Asana.com’s description of a web designer, which inspired this post.)

by Dreamtargets on flickr - /photos/dreamtargets/1085206378/

It’s not every industry where you can be a 30-year old and legitimately claim to have been working in the industry since it’s birth. The field of web design is one of these industries, and the nature of the job has changed significantly since the days when animated gifs were the height of sophistication. I can’t claim any knowledge of what it was like to be a web designer in the late 90’s – the closest I can come is that I had a subscription to .Net magazine. However, I do know what it’s like to be a web designer now and I know that it is a badly named job.

People in the web industry tend to be labelled either “web designers” or “web developers”, the assumption being that you either draw the pictures or write the code. This may have been a reasonable separation at one point. The truth as I see it now is that the web designer has progressed from writing HTML and CSS to taking responsibility for everything that happens within the browser. Web developers have retreated to the increasingly complex job of moving and processing data, around and between servers.

This means that web design, in practice, is a combination of typography, human-computer interface design, graphic design, rapid prototyping, JavaScript programming, HTML/CSS, agile product development, to name a selection. The modern web designer understands the significance of REST and HTTP, and why they make her job easier. She may even know regular expressions. If there is another word to replace “designer”, which captured the spirit of experimentation, creation and science that goes into modern web design, I am stuck to think of it.

Web design has flourished as a design profession – matured enough for an average project to bear the hallmarks of the kind of “design thinking” you might find amongst architects or industrial designers. This is all for the good. But web design is hard – there are barbed-wire boundaries between the work of web designers and those that surround them: the web developers, the content authors, the artists, the project managers, the business development consultants. Web design is relatively straightforward in isolation; meshed with all the other people involved in a project, it quickly becomes unstable. In my opinion, what holds the industry back from developing beyond a craft – that is, a complex and manual task – is the absence of good tools.

I overheard a conversation recently about why Flash became so popular with designers and has remained so, despite a strong feeling opposing its indiscriminate application. The problem boils down to one of tooling – Flash applications are built in beautiful, assistive development environments, which produce cross-browser and cross-platform applications. Web sites, on the other hand, are usually built in text editors and involve a significant investment of time to tweak and test for all the browsers in which they are likely to be run. Adobe produce compelling videos demonstrating how their tools simplify workflows and allow multiple professions to work together. Web designers generally have to “throw things over the wall” or duck things being chucked in their direction.

Specifically, I’m interested in seeing solutions to these sorts of problems:

  • Why isn’t it easy for content authors to decouple their stuff from the ongoing design of a website?
  • Why do I design a website in a different environment from the one it is going to be deployed to?
  • Why do I have to cut up PhotoShop comps into web pages and then cut up those into templates?
  • Why can’t I change the use of a single colour across a website by changing it once in the stylesheet?
  • Why can’t two web designers work on the same web page without it being a pain to maintain?
  • Why does everyone have to diagnose, fix and learn the same cross-browser tricks?
By Jonathan Lister | Posted in web | Tags: , , , | Leave a Comment

Testing local TiddlyWeb applications in virtual machines

January 15, 2010 – 2:19 pm

This is particular to VirtualBox, but I imagine the principles are the same for VMWare, Parallels and all those other virtual doohikies.

I’ve been developing a TiddlyWeb application on a local server on my Mac. This is fine for when I want to test in Firefox, Safari, Chrome, Webkit and others, but not so hot when I want to test in IE6/7/8. Virtualisation makes the general problem a lot easier, and I run VirtualBox for that purpose.

Unfortunately, accessing a locally running web server, rather than plain ol’ Internet, gave me some problems, so I thought I’d write down how I managed it.

Generally, to start up a TiddlyWeb instance, I’ve been using:

twanager server localhost 3000

I can point a browser to http://localhost:3000/ and I’m away. If I want to run on port 80, I can use:

sudo twanager server localhost 80

and then I can point a browser to http://localhost/.

The key thing to getting my virtual machines talking to my local server is to run TiddlyWeb on an IP address, rather than on “localhost”, which I had been doing. The IP address to run the application on is the IP address of your machine on the network. So how do you get this?

Given I’m using a mac, the non-confusing way is to open up System Preferences, hit the “Network” icon and read off the IP address in the “Status” section.
Network options in System Preferences

The more confusing way is to open up a Terminal, type in ifconfig and read off the IP address:

Running ifconfig on the Mac Terminal to get your IP address
With your IP address in hand, you can now use twanager to start up TiddlyWeb on an address accessible to your virtual machines:

twanager server 192.168.1.71 3000

Then you’ll be in IE testing hell in no time!

By Jonathan Lister | Posted in technology | Tags: , , , , , , , | Comments (1)

Why time booking isn’t just for grown-ups

November 26, 2009 – 11:27 am

According to the testimony of friends who work as lawyers, the worst part of the job (apart from putting paper into colour-coded files) is the hassle of accounting for every minute of every day. They do this so their practice can charge its outrageous fees to the right clients – it seems that anything not being directly charged for isn’t worth doing (unless you can get someone commanding a lower hourly rate to do it). I feel palpable relief that I have never been subject to such officious management.

It has therefore been something of a shock that, over the last six months, I’ve become a convert to time booking. This transformation has been so complete that I book time to all the projects I work on, even if I’m not billing a customer. The reason I do this is simple – far apart from the homicidal minute-by-minute accounting of the lawyer, keeping a record of how I spend time brings a number of benefits, the most pronounced of which is a change of behaviour to spend time doing what I know I want to be doing. This might sound frankly un-earth-shattering, but as I hope to make clearer, knowing you’re doing what you want to be doing feels good, especially if you can get there without it being burdensome or annoying.

Why I book time

This breaks down into three explanations, which build on top of each other: knowing what I’m doing, knowing I’m doing what I want to be doing and doing what I’ve said I’m going to do.

Knowing what I’m doing

I recently carried out an experiment on myself, where I kept a diary of everything I spent time on for three months and then analysed the data to see where my time was going (I’ve yet to write this up, although the sister experiment on what I spend money on is here). In advance, I’d predicted what I spend time on – things like sleeping, eating, hanging out with friends – to see how accurate a picture I had of my own doings. The outcome showed that, although I’d been able to make a decent gut estimate of how I spent my time on habitual activities (such as sleeping), there was a wide gap of 30 or so hours (nearly 20% of the week) where I had no idea in advance what I was going to do.

It is not such a terrible thing not to be able to say what you’ve been doing, but it feels like we are under a lot of pressure to increase efficiency, live life faster, sleep when you’re dead, etc. etc. Even if you don’t buy into this nonsense, it’s likely you’ve wished there were more hours in the day at least once. How can you improve something if you can’t measure it? Writing down what I’m up to obviously means I have a much better idea of what I’m up to.

Knowing I’m doing what I want to be doing

I don’t think I’m alone in finding it easy to get to the end of a day and think “very good, martini time. Now then, what exactly have I achieved today?”. It’s a somewhat more pronounced feeling if I get to the end of a week and think similar thoughts. It’s easy to brush off the question and determine to be more memorably constructive the next week, but at some point it starts to feel like there’s a lack of control over your own time.

I do think it’s important to feel that you’re doing what you want to be doing, rather than just reacting to things, bouncing about. Having a written record of where my hours have gone means I can’t just kid myself into thinking I’ve been terribly productive when I haven’t – initially this record is an uncomfortable kind of crutch. However, I think the discomfort is worth it, as after a short time, making myself aware of what I’m doing means I change my behaviour to be happy with that record.

Another thing worth noting is that once you feel confident that you’re doing things you want to be doing, all the other things that previously seemed so pressing now seem less so. In other words, you’ll end up choosing how you spend time.

Doing what I’ve said I’m going to do

Diets, fitness programmes, personal development plans, business development – many things take more than a week to come to fruition, but since it seems natural to think of time in a rhythm of the week and the weekend, it’s easy to become dispirited if looked-for effects take longer than a single week to surface.

One way of making it harder to stop doing things you’ve previously decided you want to be doing is to have the fact that you’re doing them written down – it feels way harder to stop something I’ve started when I can look back and see what progress I’ve made. It’s also easier to succumb to continuity and copy across activities from an earlier week than it is to commit to new ones: if I can see I’m on the way to finishing something, that makes it harder to start something shiny and new. Which brings me on to a bonus benefit…

Avoiding kittening

My habit of writing down what I’m doing means that I often have to expend effort to change what I’m doing. I like to avoid effort, so I’ve tended to want to avoid changing what I’m doing – my behaviour has drifted naturally towards spending longer periods of time on one thing rather than multi-tasking fruitlessly.

How I book time

Now you’re totally convinced of the need to do some time booking of your own, you will be worrying that it is going to take you forever to keep up to date. Fear not! Even with a mild tendency towards data-OCD, it takes me somewhere between ten minutes and an hour per week. As I have said, I like to avoid effort, so I don’t go much detail unless there’s a good reason to (such as wanting a detailed record for billing).

At the start of the week, I take the time to write down what I want to be doing that week and an estimate of how long I think each thing will take. At the end of the week, I write down whether each thing has been finished or not and compare my estimate of how long something would take with how long it actually did.

At the moment, I’m using two methods to book time – a broad estimate of time-spent vs. an hour-by-hour account.

Broad

At the end of a day, if I know I’ve spend a chunk of time on something, I’ll write that next to the thing, with a brief note of what I did. This is pretty rough, and I’m only bothered about the accuracy being to the nearest 30 minutes or even hour. I think this probably takes ten minutes in total over a week.

Detailed

If I’m doing something that I want to keep detailed notes for, I’ll write down each day when I start spending time on it and what I’m doing as I go along. When I change to something else, I’ll record the end of that period. The accuracy is to the nearest five minutes. I usually have one or two projects running like this at any one time and over a week it might take closer to an hour than ten minutes to make the record.

What time booking is not for

I do not advocate the measurement of time-spent as some sort of performance metric. It is not right to reward yourself or someone else just because a lot of time has been spent on something: reward outcomes, not efforts.

However, the ability to accurately forecast how long things will take you and to do what you say you will are both measurable traits you ought to encourage.

By Jonathan Lister | Posted in life | Tags: , , , , , , , , | Leave a Comment

Maemo Summit 2009, Amsterdam – a bit of an open-source eye-opener

November 12, 2009 – 6:40 pm

Last month, Tager Communications (a customer) took me out to the Nokia-sponsored Maemo Summit in Amsterdam to help figure out whether Maemo could play a part in the national message Nokia’s marketing division puts out about its new N9xx series.

The most striking thing about being a visitor at the summit was how large the community was – at least 300 people turned up. It was later made doubly striking when I found out that almost all of those people don’t get paid to work on or with Maemo – in fact, the N900 pre-production loan we were treated to was only available to people not on Nokia’s payroll.

This was my first experience of a cohesive open source community outside of the TiddlyWiki community – it gave me much more faith in the ability of large numbers of people to work together to produce something, without needing a centrally-enforced power structure coercing them to do so. Everyone we spoke to had a very different reason for participating in the project, often because their participation gave them experience or tools that helped them sell services to other companies. Others contributed simply because they found it interesting (or earnt them wuffie).

The whole community seemed to be attached to Nokia through a single employee – Quim Gil – and, almost surprisingly, there was ’nuff luv felt for the man. It really seems as if Nokia have done a good job “stewarding” the Maemo community, filling such gaps as needed filling (graphic design, workshop hosting) and ultimately, being rewarded with a piece of software that represents an entirely different understanding of what it means to be a smart phone OS.

And that is killer with Maemo – Nokia’s N900 is obviously a competitor to the iPhone, but the experience of using it is nothing short of using a tiny computer. The “app” metaphor is nowhere near as prominent as on the iPhone or on Google Android – applications can also surface in deep integrations with other system software. As an example, the N900 ships with Skype, although you’d never know it – there is no app icon to click on. To make a Skype call you first add your account as a “VOIP and IM” account and then choose “Skype call” instead of “Cellular call” from the dialer. (Cellular? I’m sure there will be a UK localisation by the time the device is released in Blighty.)

Another departure from the standards set by both the iPhone and Android is the acceptance of web technologies as the building blocks for Maemo applications. Launching at some point in November, Nokia Web Runtime will allow packages, suspiciously similar to W3C widgets, to be installed on the phone. HTML and CSS control an application’s appearance and JavaScript controls the behaviour and taps into the native abilities of the device, such as the camera and accelerometer. The fact that these applications can also make it into Nokia’s Ovi Store will mean that the huge financial appeal of Apple’s App Store is extended to thousands times more people than the Objective-C massive.

[Update: Palm took a big move toward the web app model with the release of the Palm Pre and webOS]

The N900 isn’t going to convert many Mac fan-boys (self included), but it’s a damn fine start at a different approach. Keep an eye out for the N910.

By Jonathan Lister | Posted in reactions | Tags: , , , , , , , , , | Comments (1)

Testing Adobe AIR applications with QUnit

November 4, 2009 – 5:29 pm

I’d like to say I always write tests before I write any JavaScript code… Those times when I do write tests, I use the QUnit framework, which is used to test jQuery, so pretty well-tested itself.

I’ve recently been putting together an Adobe AIR application using HTML, CSS and JavaScript (aka Web Standards), which is very empowering and everything, but turned out to be a little tricky to write tests for. All the interesting stuff I wanted to do through AIR’s window.runtime object, such as opening native windows and messing with dock icons, is missing from the browser if you’re trying to test the JavaScript components in isolation. The ideal would be able to run your QUnit tests within the AIR runtime itself.

Fortunately, it’s straightforward to create a test app that wraps the QUnit test runner in the AIR runtime and exercises your code. Here’s how I got going:

Setting up the file structure

The folder structure I am using for the AIR app is like this:

/myApp
   myApp.html
   myApp-app.xml
   /js
      myApp.js
   /css
      styles.css

The myApp-app.xml file is the “application descriptor” that AIR uses to create the .air package when you build the app.

I added two folders at the same level as /myApp, to contain the test framework and test code; the complete structure looks like:

/myApp
   myApp.html
   myApp-app.xml
   /js
      myApp.js
      AIRAliases.js
      jquery-1.3.2.min.js
   /css
      styles.css

/myAppTests
   runner.html
   tests.js
   myAppTests-app.xml

/qunit
   testrunner.js
   testsuite.css

Configuring the test app

The application descriptor for myAppTests is not very different from the descriptor for myApp; here they are:

myApp-app.xml:
<?xml version="1.0" encoding="UTF-8"?>
<application xmlns="http://ns.adobe.com/air/application/1.5">
    <id>examples.html.myApp</id>
    <version>0.1</version>
    <filename>myApp</filename>
    <initialWindow>
        <content>myApp.html</content>
        <visible>true</visible>
        <width>400</width>
        <height>800</height>
    </initialWindow>
</application>

myAppTests-app.xml:
<?xml version="1.0" encoding="UTF-8"?>
<application xmlns="http://ns.adobe.com/air/application/1.5">
    <id>examples.html.myAppTests</id>
    <version>0.1</version>
    <filename>myAppTests</filename>
    <initialWindow>
        <content>myAppTests/runner.html</content>
        <visible>true</visible>
        <width>400</width>
        <height>800</height>
    </initialWindow>
</application>

The main structural difference is that myApp-app.xml has an initialWindow.content property set to myApp.html, whereas myAppTests-app.xml refers to the test runner from a directory above: myAppTests/runner.html. This is needed because the test app needs to have its working directory set to the top-level of the file structure, otherwise runner.html won’t be able to get at all the files it needs in other directories (more below).

Putting the correct files in runner.html

Here’s a runner.html that works with the folder structure described above:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html lang="en">
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
		<title>Test Suite</title>
		<link rel="stylesheet" type="text/css" href="../qunit/testsuite.css">
		<script src="../myApp/js/AIRAliases.js" type="text/javascript"></script>
		<script src="../myApp/js/jquery-1.3.2.min.js" type="text/javascript"></script>
		<script src="../qunit/testrunner.js" type="text/javascript"></script>
		<script src="../myApp/js/myApp.js"></script>
		<script src="tests.js" type="text/javascript"></script>
	</head>
	<body>
		<h2 id="banner"></h2>
		<h2 id="userAgent"></h2>
		<ol id="tests"></ol>
		<div id="main"></div>
	</body>
</html>

Running the test suite in debug mode

The easiest way to run the test suite is to use the built-in Adobe Debugger, which lets you test your app without having to build it into an AIR package:

adl myAppTests/myAppTests-app.xml .

Don’t forget the ‘.’ character on the end – this tells the debugger to run with the top-level directory as the working directory – without it, the default is the same directory as XML file.

By Jonathan Lister | Posted in javascript | Tags: , , , , , , | Leave a Comment

A short jog with Eddie Izzard

September 16, 2009 – 3:08 pm

About 1:45 yesterday afternoon, Nick and I stood in the rain near Stratford station, waiting to see if Eddie Izzard would run round the corner, so we could join him for the last 10 miles of a marathon finishing in Trafalgar Square. A marathon is never something to be sniffed at, but in this case, it was Eddie’s 43rd marathon in just 51 days, and the final leg of more than 1100 miles running around the British Isles for Sport Relief. (It’s not too late to donate! £200k so far…)

I had my Flip camera with me and took some videos as we ran round. Some of them are a little shakey (some of them are nausea-inducing). Nevertheless, I’m happy to have a record of Eddie’s gutsy sprint down the Mall for a 5-hour personal best, knowing literally how far he’d come but only guessing what must have been going through his mind.

Meeting Eddie

As we waited in the rain for Eddie to get to Stratford, I couldn’t help but notice that he has over 1 million Twitter followers.

Pit stop

You run 1000 miles and your crew can’t even keep up…

Eddie meets a fan

It was amazing how many people shouted encouragement or beeped their horns as we ran through London. One guy actually leaned out of his van to give Eddie a donation.

The rickshaw

The other side of the lens, this guy pedaled the camera crew smoothly through London traffic, displaying some fine maneuvers weaving between cones.

Catching up

Quite early on, I ran in the opposite direction to tell the producer we were moving on. I didn’t realise Eddie was going to set such a punishing pace all the way from the Olympic stadium to Aldgate East, where I finally caught them.

A chat with a doctor

Eddie’s put up pictures of his battered feet (I’m not providing a link!) to show the physical effects of all his running, but this doctor thought his shins must be taking a serious beating. Eddie did admit to all the muscles being “badly bruised”. Ouch.

Legging it

Eddie had an unnerving habit of picking up the pace just as you were getting comfortable. You wouldn’t know he’d been running for 50 days…

Over London Bridge

The traffic was blocked off from our side of the road and a helicopter flew overhead – it felt a lot like we were in a 5-man London Marathon.

Approaching St. Paul’s

Could seeing St. Paul’s and knowing there was only three miles left to go failed to have had an uplifting effect?

A new supporter

Danny joined at St. Paul’s. I was really expecting there to be a lot of people running in support, but there was just the three of us. I guess the torrential rain made a difference…

Embankment

Pit stop – the camera-crew needed directions once again.

South Bank

Eddie ran past the crowds by the London Eye and straight into a tourist, providing them with a pretty good story to tell back home.

Parliament Square

More speed from Eddie as St. James’ Park swung into view and we neared the last mile.

Minor drama

You can’t run 1100 miles and not have your sprint-finish captured on film, right? So where are the crew?

Buckingham Palace

Crew located, we sped on – just one straight, long road to go.

Down The Mall

This is where the temptation to attach “Chariots of Fire” gets very strong. I haven’t, so if you are that way inclined, please imagine it playing in your head.

Eddie’s Ices

Free ice-cream is never a bad thing.

Gracious End

Personally, I would have collapsed at this point, but Eddie had a nice long chat with the press and then hobbled down the steps of Trafalgar Square signing autographs and talking to people. He posed for some photographs with us. :)

By Jonathan Lister | Posted in life | Tags: , , , , , , | Comments (8)

Baselining my life (part 1, money)

September 10, 2009 – 5:28 pm

Introduction

I began to seriously contemplate the idea of going freelance back in May, and realised I wasn’t very prepared. For one thing, I didn’t really know how much money I spent each month or what I spent it on. Nor did I have a good handle on how I spent my time. I wanted to see how I was behaving financially, so as to know how much I’d need to earn as a freelancer to maintain my lifestyle. I also wanted to know what made up this lifestyle to see how much time I’d have available for doing this freelance thing. I did go freelance at the end of June and carried on measuring things for a while so I’d have enough data for a meaningful result.

On the 6th June, I started keeping a record of my spending; on the 1st July, I began to keep a diary of what I was doing all day. At the end of August, I stopped measuring. This means I have four months of records of how I’ve spent money and three of how I’ve spent time.

Having this information is very useful, I think, so I’ve published all the data and analysis spreadsheets for you to play with. If you wanted to do something similar, they might be helpful. (The data are all published as Google Spreadsheets – see the links at the bottom of this post.)

Of course, it would be remiss of me, as a one-time scientist, if I were to look at data and deduce hypotheses after the fact. So, without prejudicing my judgement too much by figuring out a collection of perhaps-useful things in advance, I’ve written down some questions that I want answering and hypotheses to test for each.

Money

  1. How much money do I spend in a month?
    • Hypothesis: I think I spend £2400 a month
  2. What am I spending most of my money on?
    • I think the majority is being spent on rent, dining and boozing
  3. How much do I spend on booze?
    • I think I spend £400 a month on booze

Time

  1. How much time do I spend sleeping?
    • I think I spend 1/3 of my time sleeping
  2. How much time do I have free in a week after I’ve done all the habitual activities?
    • I estimated I have 31 hours a week to use (estimate recorded in Habits)
  3. What is the most expensive activity per time spent on it?
    • I think it is boozing (although dining will be a close second)

This post is going to cover the questions about spending money.

Constraints

  1. The Hawthorne effect – will I change my behaviour by measuring it?
  2. Not splitting money and time spending when on holiday – it just goes down as ‘holiday’

Measuring money

Answering the questions

How much money do I spend in a month?

Hypothesis: I think I spend £2400 a month
The data shows: Between May and August 2009, I spent an average of £2492.65 each month

What am I spending most of my money on?

Hypothesis: I think the majority is being spent on rent, dining and boozing
The data shows: rent 21.7%, holiday 20.6%, dining 10%, boozing 7.7%

How much do I spend on booze?

Hypothesis: I think I spend £100 a week or £400 a month on booze
The data shows: Between May and August 2009, I spent an average of £191.20 on booze each month

The data illuminates

The images below are generated from the summary spreadsheet, which contains live graphs you can play with – clicking on pie segments, for example, reveals the percentage and value attached to each.

Methods

Collecting data – iPod Touch & Notes

Online banking is a boon to the would-be data collector, as you can generally download a transaction history in csv format. Unfortunately, plenty of entries for “cash” creates a big unknown in the results. I decided to keep a record of when I spent cash during the day, what I spent it on and how much I spent. I didn’t start monitoring expenditure until the 6th May, so approximately £140 cash is unaccounted for over the four month period (I haven’t been worried sufficiently to alter any calculations to take this into account, as my monthly spending has a standard deviation of more than this).

I’d done something like this before, when I was having a crisis about where all my money was going shortly after I started working for BT in 2005. Back then, I used a pencil and notepad, and the prospect of copying all the records into a computer was so daunting that I never did. This time, I had the benefit of an iPod Touch (thanks Dad) and its “Notes” application. You can mail yourself a note, which I did every month or so.

Dealing with the data after it fell into my inbox was a matter of some search-and-replace in a text editor, to make the fields tab-separated so that they could be copy-and-pasted into Google Spreadsheets.

Analysing data – Google Spreadsheet

The first thing to do was enhance the data by tagging each expense with a chunky category like “rent” or “boozing”.

Using Google Spreadsheets to analyse an amount of information is a task with a fairly steep learning curve, since the examples for using the more complicated functions are not always that enlightening (I imagine if you are already an Excel functions master, it won’t be so hard). Nevertheless, the Google Docs forum is replete with the writings of two characters in particular – Otávio Alves Ribeiro, the Spreadsheet Ninja from Brazil, and “ahab“, the mysterious Google Docs Guru.

The performance of a Google Spreadsheet is generally quite good, although there is often slowdown in processing. I found, mainly through trial and error, that a good way to understand what is going on and get decent performance out of Google Spreadsheets is to separate worksheets between data collection and analysis; then perform any inter-spreadsheet aggregation by importing the plain data from each spreadsheet into a single worksheet and make all your calculations based on that.

Further work

I ended up with just over 20 different tags; if I were to do this again, I would split the ‘holiday’ tag into its constituent parts and I would split “dining” into “breakfast”, “lunch” and “dinner”.

It will be interesting to compare my monthly spend (which is the simplest data to measure) to the relatively steady pattern I have discovered over these four months to see how much effect measuring my spending had on dampening it; indeed, I could measure the historical monthly spends and perhaps see an effect that way.

Data

All the data for this experiment is published on various Google Spreadsheets. See the links below.

By Jonathan Lister | Posted in life | Tags: , , | Comments (10)

Coloured-in jaytoon

August 27, 2009 – 12:50 pm

Nick’s been playing around with Illustrator and is pretty bowled-over by how easy it is to manipulate images. He did me this to show off his skills. I’m feeling most appreciative.

Here’s the original cartoon: Celebration

By Jonathan Lister | Posted in cartoons | Leave a Comment

How-to log into HSBC online banking from the command-line

August 14, 2009 – 3:18 pm

This is a description of the HTTP calls you need to make to achieve the log-in and account information retrieval I showed in my recent video.

Note that the content of cookies and session variables will be different each time, but examples are included here to illustrate what it going on.

Please let me know if you have similar information about a different online bank or find that this doesn’t work for you for some reason!

Getting to the login page

You can skip the HSBC homepage altogether and make a GET to:

http://www.hsbc.co.uk/1/2/HSBCINTEGRATION/

even without any cookies set. The response sets these cookies:

Set-Cookie: HSBC_COOKIEMI=e7f408d0-75f9-11de-8a43-000408000305;Domain=;Expires=Sun, 20-Jul-2014 13:25:06 GMT;Path=/
Set-Cookie: CAMToken=GCPyfe4/i1TFoVS3L/z4qRhzbNs=;Path=/1; Secure
Set-Cookie: piba=73472940.21248.0000; path=/

Submitting your Internet Banking number

Now POST to:

https://www.hsbc.co.uk/1/2/;jsessionid=0000EzVqNO88EsWt4lhQhIBMHVE:12c58sh8k?idv_cmd=idv.CustomerMigration

Note the jsessionid=... on the end of the URL, which is different for each session; the POSTed content:

userid=IBxxxxxxxxxx&StartMigration=

The response is a 302 to (note the doubled jsessionid):

https://www.hsbc.co.uk/1/2/!ut/p/kcxml/04_Sj9SPykssy0xPLMnMz0vM0Y_QjzKLN4o38nYFSZnFm8cbm-pHmsWbxjt7QkQM4h3hAkH6BbmhEeWOiooAtvLMKw!!;jsessionid=0000EzVqNO88EsWt4lhQhIBMHVE:12c58sh8k;jsessionid=0000EzVqNO88EsWt4lhQhIBMHVE:12c58sh8k

that sets this cookie:

Set-Cookie: CAMToken=ItaH8ksgEuKY6vv6YC5nRJSTiEg=;Path=/1; Secure

The bits of your PIN that the system wants are contained in this structure in the content:

<div class="extPibRow hsbcRow">
<div class="logonPageAlignment">

         <strong>The</strong>
         <span class="hsbcTextHighlight">&#160;<strong>FIRST</strong></span>
         &nbsp;<strong>and</strong>
         <span class="hsbcTextHighlight">&#160;<strong>NEXT TO LAST</strong></span>
         &nbsp; <strong>and</strong>
         <span class="hsbcTextHighlight">&#160;<strong>LAST</strong></span>

Submitting login details

POST to:

https://www.hsbc.co.uk/1/2/;jsessionid=0000PPUs1fHtg7XOIHMqjvIsoHW:11j74lld0;jsessionid=0000PPUs1fHtg7XOIHMqjvIsoHW:11j74lld0?idv_cmd=idv.Authentication

with content:

userid=IBxxxxxxxxxx&memorableAnswer=xxxxxx&password=xxx

memorableAnswer is date-of-birth formatted ddmmyy
The response is a 302 with this location:

https://www.hsbc.co.uk/1/2/!ut/p/kcxml/04_Sj9SPykssy0xPLMnMz0vM0Y_QjzKLN4w38nYFSZnFm8cbm-pHoggZxDuiiZjEm7khhIL0C3JDQyPKHRUBimrSIQ!!;jsessionid=0000PPUs1fHtg7XOIHMqjvIsoHW:11j74lld0;jsessionid=0000PPUs1fHtg7XOIHMqjvIsoHW:11j74lld0

which sets these cookies:

Set-Cookie: CAMToken=1HOPXe7zOn2qQ/dp28qb1yeozS0=;Path=/1; Secure

Continuing with login

An additional step happens due to the JavaScript in the page. There is a GET to:

https://www.hsbc.co.uk/1/2/personal/internet-banking;jsessionid=0000PPUs1fHtg7XOIHMqjvIsoHW:11j74lld0;jsessionid=0000PPUs1fHtg7XOIHMqjvIsoHW:11j74lld0?BlitzToken=blitz

The response is a 200.

What you can get once you’re logged in – example – accounts’ balance

The account summary information on the left-hand column is in a structure like this:

<div id="jsAccountDetails" class="hsbcDivletBoxContent">
<div class="hsbcDivletBoxTitle" id="jsHideAccounts" style="display: none;">
<div class="hsbcDivletBoxRow">
			<a title="Hide my balances summary" onkeypress="this.onclick" onclick="hideAccounts(); return false;" href="#" class="hsbcDivletBoxRowText">
				<span class="hsbcDivletBoxRowImage">
					<img border="0" title="Hide my balances summary" src="/1/themes/html/hsbc_ukpersonal_ib/icon_balances.gif"/>
				</span>
				<span>Hide balances</span>
			</a></div>
<div class="hsbcSeparator"/></div>
<div class="hsbcDivletBoxRow hsbcDivletBoxRowNoLink">
			<span class="hsbcDivletBoxRowText">
				<strong>GRAD PLUS</strong>
			</span></div>
<div class="hsbcDivletBoxRow hsbcDivletBoxRowNoLink">
			<span class="hsbcDivletBoxRowText">LISTER JR</span></div>
<div class="hsbcDivletBoxRow hsbcDivletBoxRowNoLink">
			<span class="hsbcDivletBoxRowText">			40-27-16 91466410			</span></div>
<div class="hsbcDivletBoxRow hsbcDivletBoxRowNoLink">
			<span class="hsbcDivletBoxRowText">
				<strong> £ xx.xx  C</strong>
			</span></div>
<div class="hsbcSeparator"></div>
</div>
</div>
By Jonathan Lister | Posted in web | Tags: , , , | Comments (2)

Online banking with JavaScript – first working thing

August 6, 2009 – 10:49 pm

I recorded this at Islington Hackspace tonight, figured it would be cool to share visually rather than have a big blag-rant, (see my last post for one of those).

It ought to be self-explanatory. I’m thinking about wrapping this up in an iPhone app so I can pay people for free. Rather than use PayPal or something.

Sorry it’s a bit blurry.

Source (hopefully changing lots) on github.

By Jonathan Lister | Posted in javascript | Tags: , , , | Comments (8)