We Like IndieWeb Software 2015-01-16

Recently on Twitter Anil Dash and I had a bit of a back and forth:

Hi, it's been two years, will somebody go build this set of tools and go make millions of dollars please? http://t.co/jLX0Dp5DxI

— Anil Dash Dot Com (@anildash) January 14, 2015

@anildash all that's been shipped as #indieweb tools, except we use html instead of json because web. Try it.

— Kevin Marks (@kevinmarks) January 14, 2015

(told for tools there is an autocorrect failure)

@kevinmarks I want products, not toolkits. Cobbling stuff together is too time-consuming & this stuff isn't all in one place.

— Anil Dash Dot Com (@anildash) January 14, 2015

@anildash then use @withknown and send feature requests. Build tools, not specs.

— Kevin Marks (@kevinmarks) January 14, 2015

Marco Rogers chided me for being short with Anil:

@kevinmarks @anildash these are feature requests. From a potential customer. Listening is a good strategy. Berating, less so.

— Marco Rogers (@polotek) January 14, 2015

This is a fair point, and so here is a post going into more detail.

The quoted parts are from Anil's blog post - I'll respond inline:

So, my contribution is to collect some of the notes I've been gathering for the last few years about what I'd like to see in a blogging tool. I know there are apps with many, perhaps even all, of these features, but I'd like to see one emerge as a leading platform for doing innovative work.

(emphasis added)

Here Anil is explicitly calling out for a monoculture, rather than a set of interoperable tools and protocols. As Anil mentions, he used to work at Six Apart, which built several blogging tools in that way, each hoping to be the one. Indeed seeing the success of silos, their monolithic nature can be seen as contributing to this; their widespread adoption coming from focusing on innovating and improving user experience rather than interoperability with others.

With indieweb we are trying a different approach by working through our own wishlists, reusing common components, and making sure we interoperate along the way. This gives us a composable set of tools that do plug together - the toolkit Anil both is and isn't asking for.

My blogging features wishlist:

• I enter markdown in plain text files; these are stored on Dropbox/Google Drive/Skydrive and/or S3 and/or GitHub.
• The system renders those plain files into JSON assets in a documented format.

There are lots of indieweb tools that work in this kind of way; but Anil is very focused on the specifics of formats. Markdown is one popular way of writing text for posts; indeed many indieweb tools support it. JSON is handy as an interchange format between programming languages, but as Anil says, having a documented common format is useful. What we realised working on indieweb is thet we already have a lingua franca for webpages and blogs, and that is HTML. Ultimately all sites publish in this format, so using that and adjusting it minimally to make interop easier is the approach we took.

Our documented format is Microformats 2, which for blog posts involves h-entry and h-feed as common structure, with additional microformats to label other reusable features. Generating HTML from Markdown is relatively straightforward, if not always deterministic, as that was its design goal. Generating JSON from HTML formatted with microformats 2 is also straightforward and more deterministic.

Storing source files in various cloud services is practical and indeed done by many indieweb tools, but requiring a specific cloud service's file system is less flexible and general than using HTML on the web itself, so that's what we do.

• A Bootstrap-themed reading client app lives at my site, on my domain, and reads a single simple config file to learn how to display and navigate between those JSON assets. This client app would also have to handle URL routing and persisting states, while ideally also keeping preferences and reading history for readers.
• The default theme offers a YouTube-style browsing view of all my content, where people can make playlists of posts (this is equivalent to navigating my archives by tag), embed my posts on their own sites, and easily explore by traditional groupings like category or date.

This is a lot of fragility - the default Blogger template switched to a JSON model like this and consequently fails to render a lot of the time. Twitter too used an all-JSON web app for a while, before reverting to HTML+json enhancements. The fragility comes from JSON being much less resilient to encoding or writing errors than HTML - HTML5 specifies how to consistently handle even invalid or poorly marked-up HTML pages, whereas with JSON you will get a parse error and lose the whole page, just like XML. By using HTML instead for your format, the page can be read on every platform by default. Now, behind the scenes JSON can be useful - indeed Known uses a JSON store for its posts database, but exposing this publicly will likely lead to incompatibility over time.

We are beginning to see indieweb reader apps that work by parsing the h-feed and h-entry markup published, and give the remixability that Anil mentions, but they are an augmentation to the basic page, not a required path along the way.

• There might be an optional administration interface separate for me, just for editing the markdown files through a plain text in-browser editor; In this case, it should be a responsive app that works in all my browsers.

On the indieweb, this is handled by two protocols: Micropub and IndieAuth.

IndieAuth lets you prove that you are the owner of the site by logging in, so you can get editing privileges or other enhanced versions.

Micropub is a simple protocol, based on HTML forms, to let you edit and publish posts and notes. By using these two it is possible to have multiple different tools to create and edit posts, independent of the mechanics of their storage, as they also use HTML for interchange.

• Ideally comments are handled as small messages in a documented json format, sent between instances of this blogging application. Of course in the short term I would just embed Disqus/Facebook/Google-style comments until that infrastructure was further along.

For comments we use the same documented HTML format, but add the idea of a webmention. This is a simple protocol that enables you to send a link to your post or comment that responds to (mentions) another post. The webmention receiver can parse this, check it does indeed link to it, and interpret it as a comment, a reply, a repost, a like or an RSVP depending on the microformats markup used.

• Having a documented format for the json objects which represent posts and comments would permit transclusion and sending of posts between sites, in a manner analogous to how Fargo does this for outliners, and in a way that would bring back some of the positives of TrackBack in the early blogosphere.

Exactly! Webmention does this, but with verification and context, which means that it can be used for reposting, but also for threaded comments, per-paragraph annotations and other things we haven't thought of yet.

• "Themes" would largely be implemented as Bootstrap CSS stylesheets, with some affordance for separate content modules. By default, themes are public so I would just be able to tell an admin app to import a theme from your site so I could remix it.

By using HTML as the source interchange format, styling can be done by CSS directly; also it is relatively easy to process and parse HTML and inject it into a site - see the webmentions below that are added by an indieweb service.

• The API endpoint for discovering the json representations of content would double as the API for others to access my data to build around it; Eventually a posting app which saved POSTs of that json format as fiels in dropbox would allow a write API.

The webpage itself is the HTML representation of the content; you get the json version by parsing that, using existing parsers - for example, this is the json version of this page.

A posting app can look for the micropub endpoint in the page and the indieauth endpoint and work with that, as tools like Quill, ownyourgram and Postly do now.

I think that's it for now. Let me know if somebody's got all these boxes checked on their platform today, but I suspect the hardest part is the client app for readers, which works in a way analogous to an RSS reader or email client, but would have to support a new format and would be optimized for clean reading and subsequent discovery, rather than the three-pane model which has dominated those apps for the last decade or two.

Anil's assumption that the reader app is harder is shrewd, though it has also been less necessary as the browser enables reading of pages in any case. Indieweb readers are being built now with varying UI based on these underlying protocols, so we should have all the pieces soon.

The closest thing to a single platform that supports this is Known, an open source indieweb-friendly blogging tool that can be installed on your own site or hosted by Known for you.

There's even a Pro version of Known if you want to help make Anil's prediction come true.

If these indieweb ideas sound interesting, come along to the next Homebrew Website Club, or join us in IRC.