Our friend Mark Alexander is in prison, wrongly convicted of murdering his father. While he fights to clear his name, we keep in contact with him, and like to keep him up to date with what's going on in "normal life" outside prison.
A wonderful service, email-a-prisoner provides a very convenient way of sending Mark a letter. You log into the site, type your letter into a web form, hit send, and the letter is sent to the mail room of the prison, gets printed out (either on a fax or a printer), and is put into an envelope by prison mail-room staff, and is delivered to Mark in his cell about a day after it's sent. A one-page letter sent through email-a-prisoner costs 30p.
Mark has to use snail mail to reply to our letters - he has no access to a network-connected computer.
Email-a-prisoner also run an "SMS-a-prisoner" service - you register your mobile with their service and when you text their special number, it gets sent as a letter to Mark in prison.
When I saw this, I *obviously* thought: "wouldn't it be cool to be able to 'tweet-a-prisoner'", but quickly thought of all sorts of reasons why this probably wasn't a cost-effective solution - 30p for 140 characters didn't seem like something you'd use for the lightweight, day-to-day, ambient messages that fuel our twitterverse.
One day, Matt Whitehead (@matthew101) had a great idea - wouldn't it be cool if you could link twitter to email-a-prisoner. My previous objections turned into just the kind of challenges that we love to overcome when we're innovating together, and soon we had the outline of a solution.
The idea we came up with is that there would be a twitter account which is a "proxy" for Mark. We'd set it up to follow a list of Mark's friends on twitter, enabling them to @mention or DM (direct message) this account. When we had a page-full of these tweets, we'd package them up as a single message and drive the user-interface of email-a-prisoner as if the tweets were being typed-in to the web form. Then we'd click "send" and the page of tweets would go through email-a-prisoner and be printed out and delivered to Mark as a paper letter. All of the accumulation of tweets and the packaging and sending through email-a-prisoner would be completely automated, with no human intervention (apart from me and Matt keeping an administrative watchful eye over the process to make sure it didn't go nuts and send a million letters to him in one go!).
We argued quite a lot over the name of the account, but settled on @tap_MA - "tweet-a-prisoner - Mark Alexander".
I already had some perl code which reads the timeline of a twitter account and publishes incoming messages to the MQTT messaging system. This had been used for an earlier "tweetject" project - controlling Christmas tree lights from twitter. This application uses the Net::Twitter perl library, giving convenient access to the twitter API and sorting out all the oAuth authentication gorpiness for me. The application periodically reads the timeline, mentions and direct messages for the @tap_ma twitter account, parses the resulting array of tweets (which come complete with a big chunk of metadata), and formats and publishes the text and sender information to MQTT.
[andysc] @tap_MA greetings from twitter! :)
I ran into a problem with authentication using the twitter API to read Direct Messages. Incredibly serendipitously, I happened to mention the problem to @andypiper, who just happened to have spotted a recent change to the twitter API permissions which was causing the problem. I can safely say I would never have found this, had he not pointed it out to me, so thank you, Andy.
My twitter-to-MQTT bridge publishes messages on three topics - mentions (messages which contain @tap_ma), timeline (all the tweets from the people @tap_ma follows), and direct (Direct Messages sent to @tap_MA). The messages are published to an MQTT message broker. As is the purpose of messaging middleware, this gives an elegant and convenient decoupling between my part of the system, and the part Matt was working on.
Matt's app is the brains of the operation: it takes incoming tweets (mentions and DMs), formats them on a page, and when it has a page-full, logs into email-a-prisoner as if it were a person sending a message to Mark via the web interface, posts the page to the web site, and hits the "send" button (and spends 30p on our behalf!). As the different types of tweets (mentions, DMs, hashtags) come into his app on separate topics, he can process them separately, for example, prioritising Direct Messages over messages on the ordinary timeline if there's a conflict for space on the page; or marking DMs distinctively, so Mark knows they were just for him, rather than for general consumption. This also gives an elegant separation of "business logic" from "presentation logic" which proved useful during the iterative usability design.
I specified the format of the interface - the topics and the message format - first, so we both had a common interface to code to. This made it easy for our development work to proceed in parallel, and by publishing messages to test topics, and subscribing to MQTT topics in an interactive MQTT client, it was easy to stub out and debug our apps before we hooked them together.
When we linked the sub-systems together, we didn't let it send messages to email-a-prisoner at first. We just published the final formatted message to an MQTT topic so we could subscribe to it and see what it would have sent if we'd linked it up to the final sending step. Another very useful debug trick we used, was to publish the partially-built page to a "retained" MQTT topic each time the content was added-to by a new tweet. By connecting to the MQTT broker and subscribing to this debug topic, we could at any time see the current state of the semi-compiled email, and watch it clear down when the sending criteria were met and the message had been sent on its way to Mark.
During testing it became apparent that if there weren't many messages being sent to Mark, he could wait a very long time before he received a letter, or "twemail" as they came to be called (twitter-email). Matt added a timeout feature, so after a few days, even if the page isn't full, it sends whatever tweets have been received.
I also got fed-up of typing "@tap_ma" in my tweets to copy Mark in on a message. So I implemented the same feature that a lot of people use who have twitter and facebook accounts, and use the hashtag "#fb" to tell twitter to cross-post this tweet to their facebook status. I added parsing for the "#ma" hashtag in messages on @tap_MA's timeline (i.e. only the people who were likely to be using #ma to refer to Mark, rather than all the other #ma hashtags in the twitterverse, for example, to mean "Massachusetts"!). These tweets are published on a separate topic (hashtag), so Matt's app can deal with them as a separate stream if he so wishes (or just merge them in with the @mentions, as he did in the end). I like the #ma feature a lot - it makes it really trivial to make any tweet visible to Mark, and it has become a natural and frequently-used suffix to many tweets.
After a few days of testing, we agreed it was time to turn on the connection to email-a-prisoner and send our first "twemail" to Mark. It was an exciting and nervous time for us both. We were watching the debug topic pretty much all day as the tweets rolled in. Finally it got to the high water-mark and the debug topic cleared-down, indicating that the page had been sent. Matt logged into email-a-prisoner to look in his outbox of sent messages. There it was - our first twemail had been sent!
Now we had to wait.
One of the tweets in that first twemail was to tell Mark to phone me to tell me he'd received the letter. We weren't sure if someone might not like what we were doing - would email-a-prisoner object to the computerised driving of their web interface? Would the prison guards not appreciate the weird layout of the letter and block it from going to Mark? We just didn't know.
The next evening, the phone rang and Mark was on the line, so excited, burbling about his twemail that had arrived that day. He was so thrilled: not only to receive this snapshot of real, everyday life outside the prison, but was very flattered that we'd spent a big chunk of our spare time writing quite a complex software application just for him. Hearing the happiness in his voice endorsed the view that Matt and I had, that this was a Really Cool Project!
I think this is just amazing. In particular, having spent a big chunk of the last four years working with @clarejhooper on mixed modality interactions, and the re-envisaging of digital experiences in non-digital media, this just struck me as a great example where we were forced to use non-digital media for parts of the system, but where the externals were identical to a fully digital interaction. I'm sure Clare will explain that much better than I just did in a subsequent blog post!
So there you have tweet-a-prisoner. If you'd like to send a tweet to Mark in prison, he'd love to hear from you, so mention @tap_MA in a tweet, or if you want to get stuck-in, follow @tap_MA, wait for "him" to follow you back, and then you can DM Mark, and use the magical "#ma" hashtag to include him in a tweet.
A footnote on an unexpected benefit of using a publish-subscribe messaging system like MQTT to link the components of the system together: I had some ideas about the formatting and layout of the twemail that was sent to Mark which I wanted to try out "offline" before I proposed them to Matt for inclusion in the live system. Because MQTT is a "one-to-many" messaging system, I was able to subscribe to the same set of topics from my twitter-to-MQTT bridge that he was using, and write my own app to process the different types of tweet (DMs, mentions, etc), and format them my way, and then we could see how we liked the new ideas. As I was using perl, it was very rapid to implement new ideas, and whilst discussing some layout ideas with @lauracowen, I was able to implement those while we were talking and see the result of the changes immediately. An exciting and productive way to use rapid prototyping to try out new ideas alongside a running "production" system.
Trackback: As predicted, @clarejhooper explained that much better than I just did in a subsequent blog post!