danirod

Roadmap for 2018 for danirod.es: embrace the open web

Dani Rodríguez • • Permalink

This year, I want to make danirod.es my main online identity and stop relying so much on social networks for hosting my content, such as my photos or my status updates. This will not change the use I make of social networks, so if you follow me on social networks, not much will be noticeable. This blog post will help me control the list of changes and features I’ll add to my website through the year in order to make it open, federated, and independent.

Context: social networks and the closed web

Mainstream social networks are used because they are convenient and easy to set up. The hassle required for installing software, patching servers and doing database backups, is replaced with a simple interface where you just have to write a status update or select a picture to upload, and press a button to share it with the world. Some non-technical users may not have the skills to install a GNU Social node in a VPS, and then set up a DNS record to make a domain name point to the VPS server. However, filling a form on a mobile device or a web browser is an easier abstraction that more people can cope with.

Despite of what it looks like, third party platforms and services are nothing but storing data on a computer we don’t own. We upload data to these services because they promise us they will do their best to make our pictures and thoughts reachable to people we are connected with. People who are subscribed to our updates on YouTube will be able to see our new content in their subscription stream. However, as outsiders, we don’t actually control the way they store, present or distribute our content. We just feed them with new data to keep the system spinning. So they behave like data silos.

A few examples on the problems this has:

  • Storage: Remember Vine? Years after being bought by Twitter, their technology (playing short videos in a loop) was added to the Twitter platform, and they got rid of the site. It’s read only, so you cannot upload new content, although existing content can still be browsed, but who knows for how long. What happens when a service dies? Fortunately, most sites these days have a backup system that allows user to download their data once a site announces its shutdown. However, permalinks rot, they cannot be backed up.
  • Distribution: A lot of social networks made improvements to their feeds in the recent years. It’s not enough to subscribe for updates from people or brands we care about. An algorithm will decide whether an update status is worthy enough to have it on our feed or not. Some social networks now have features like bells, stars or favourites, to signal the algorithm we actually want to receive updates from these people. So, you have to subscribe twice to get content from your network.
  • Presentation: Why does every Medium site have to look the same? Nothing else to say here.

A lot of these problems would not be an issue if we could control the software that drives these platforms. By owning a fully featured website, like in the old days, storage, distribution, and presentation, is under our control. I’m not trying to romanticise how the web was before social networks or the Web 2.0, but it’s impossible to deny that, while social networks have lowered the barrier for having an identity online for sharing thoughts and information with people around us, they’ve also removed some of the internet magic we used to have before.

IndieWeb: let’s make the web open and fun again

I first learned about IndieWeb last year and got interesed by the ideas they propose. IndieWeb is an alternative movement to the closed and corporate web, based in a few ideas. You should own your content and post it on a site that you own, because it cannot be forever safe when it’s managed by a company or third party service that it’s outside of your control. The platform could dissappear or your account could get locked or banned by some arbitrary reason.

Plus, having a site means more freedom on how to add and manage your content. You don’t need Twitter threads in your site, because there is no arbitrary character limit on posts made on your site. You can control how your site looks. You can control the distribution of your content by using reliable permalinks that will last as long as you want, ideally forever considering that hosting is cheap.

However, you don’t have to stay out of the loop of social networks (unless you want). There are models on how to publish both to your site and to social networks, to make it available to your friends who would still want to use those sites1. You can post a photo or a status update to your site, and then cross-post it to a third party site like Twitter or Instagram. Just make sure to add links on your tweets or photos that point back to the same content in your site to set up the connection between both copies of the same content.

This is known as POSSE: Post on your Own Site, Syndicate Elsewhere. There are tools for doing this automatically, such as automatically sending your status update to Twitter once you post it to your site. Sites that have a public API make this process easier. In fact, you probably know of a few services that automatically do this, such as YouTube sharing a link on Twitter when you upload a new video or Twitter sending status updates to your Facebook profile.

However, in order not to rely forever on POSSE, the W3C Social Web Working Group has been releasing in the last couple of years a few recommendations for protocols that add a social layer to websites. This social layer is neutral, not owned by a corporation, and not associated to a particular social network. Instead, these are open and federated protocols that once added to an open website, allows it to talk to other open websites in order to have some social features usually present in mainstream social networks2. For instance:

  • Webmentions: this is the evolution of pingbacks. You can use them to tell other websites you’ve linked them in your posts, maybe as a cite, as a reply or as a like or a repost. The server operator will be notified of the URL that made the mention, and it can register it to display it under the original blog post, similar to how other comment systems work.
  • WebSub: You may know how Atom or RSS works. An user agent fetches a document hosted in a different server and reads it to find if there is new content available (like blog posts or photos). WebSub works the same but it’s the opposite direction. The owner of a feed will take care of notifying subscribers whenever the feed is updated. Clients subscribe and unsubscribe to the content source. This is more efficient because it avoids the need of polling a server all the time to check for new content. Instead, it’s the server who will tell the world about new content.
  • Micropub: it defines an open protocol for creating, updating and deleting posts using third party clients. How do you add content to your website? Do you have a WordPress admin panel, a self-crafted dashboard, or you use a command line or email to write posts? It doesn’t matter. As long as it talks Micropub, any other third party (authorized) client can create posts.
  • IndieAuth: it’s an identity system built on top of OAuth 2.0 that allows users to identify themselves using their site, similar to how many other services let users identify themselves using a Facebook or Twitter account. It’s similar to OpenID.

There are a few services, like webmention.io, that make easy to set up a working implementation in a breeze. Also, in case you’re building your own website from scratch, there are a few libraries that already implement this protocol. However, it may also be fun or as a coding challenge, to write your own implementation. Some protocols have test suites to verify they implement properly the spec. I started working on an Elixir Plug that adds support for webmentions, although it’s not finished yet and I haven’t actually worked on it for the last month, but it’s a good project to practice what I’m learning about Elixir and to know more about Elixir Plugs.

Okay, so what’s the plan this year?

This year, I want to reunite in my site content previously spread in my multiple social networks, being YouTube the only exception I’ll make at the moment, because the bandwidth and storage costs would be too high. This means making my old Twitter, Instagram, Flickr and Google+ content available through my website.

Then, I’ll make sure that new content posted on social media is posted instead on my site. You won’t notice nothing if you are following me on Twitter or Instagram because the content will be syndicated at the same time through POSSE, plus I’ll keep using those social networks and liking and retweeting memes, but there will be a canonical permalink and a copy of the content I make on my website.

As a coding challenge, I want to implement some of the recommendations of the W3C Social Web WG and add them to my website.

Post types: notes, photos and workouts

I want to close the gap between some social networks I use and my decentralized website this year. Just like sites such as Facebook have multiple types of publications (status updates, photos, videos, links…), you can do the same with regular websites. In fact, I’m already doing this in my site. Some posts are blog posts, some other are linklog bookmarks.

This year I’ll tweak the source code that makes my website work in order to add support for the following things:

  • Notes: they are similar to tweets. Short status updates, implemented as title-less blog posts. They have no character limits, unlike their Twitter or Mastodon equivalents, but they are intended to be used for short posts, such as a few sentences or lines. They are the easiest to cross-post to Twitter.
  • Photos: I seriously dislike how most social networks only let posters add a description to photos. Flickr is the exception, allowing photos to have both a title and captions. Since the whole point of using open web is setting my own rules instead of accepting rules of other platforms, I can have both and be happy about it. I want titles and I want descriptions, and if possible, I want alternative text added to my photos.
  • Workouts: I capture a lot of data of my workouts using Runtastic. I’m not probably sharing maps data because of the privacy implications that would have, but I think having a public log of the numbers I make may be useful, in case Runtastic ever dissapears.

I’m also interested in additional posts. You can tell your friends on Facebook shows and movies you’ve watched or music albums you’ve heard. I could add support for scrobbles (similar to the scrobbling system made by Last.fm), to keep track of podcasts, albums and other AV content I’ve seen, but it’s not a priority at the moment, since that would be a lot of content to keep track of.

Pull more backlog archives

In 2016 I added a backlog of blog posts from old websites into my site. Once I have support for more post types, I’d like to import a backlog of content coming from other social networks, for backup purposes.

  • Adding photos from my Instagram and Flickr accounts into the photos section. Maybe some photos shared in other social networks such as Twitter or Google+.
  • Adding some old workouts from old fitness apps I used. CardioTrainer became Noom, and then it changed their business model. In fact, I replaced it with Endomondo years ago, so I don’t even know if they’re still running or if the site is dead. However, I downloaded before leaving a stash of running and biking workouts using their export assistant, so there’s that. All those files are lying somewhere in my hard drive. If I found them, I could store them here.
  • Pull most of my Twitter status updates into my site. Tweets with photos would be considered photos, tweets with links would be considered bookmarks, other tweets would be considered regular notes. I’m leaving replies, retweets, and probably quotes outside of this import because that’s a lot of noise and probably it’s copyright infringement.

This has actually been already done. You can test this by looking at the source code of this website.

Microformats is a semantic web system for adding metadata into pages designed for humans. Designed for humans means that it’s an HTML document with valuable content for humans, such as blog posts, news articles, a personal website or some event information. An RDF or a JSON document is not designed for humans.

Microformats is similar to other formats such as RDFa, OpenGraph, JSON+LD or Schema.org, and it works by adding some invisible and simple HTML markup to these documents to tag properties about things. However, Microformats are designed to be extensible and they produce valid HTML 5 and even HTML 4 or XHTML 1.0 code.

Take the following HTML code:

<p>My name is John Doe.</p>

You could add some invisible markup to make easier to non-human user agents to detect that my name is John Doe:

<p class="h-card">My name is <span class="p-name">John Doe</span>.</p>

h-card is a microformat that represents a contact card. It’s used to describe information about a person or an organization. Each microformat has a list of properties. One of them is p-name, which is used to describe a person name. This document is still readable by humans, but computers can extract information as well without the need of heuristics or other kinds of guesses.

If you’re reading this blog post inside danirod.es, you’ll notice this blog post content is wrapped inside a tag with the class h-entry applied, which is the class used for entries (such as blog posts). This text is wrapped inside an e-content tag (it’s the contents of the blog post), and the header and footer has several other class names added to the title, the publication date, the URL and the tags. All these items help a computer detect the metadata for this blog post and present it.

h-entry can be used by a webmentions server to extract information about an URL that is linking to a blog post, such as the title or the author name, to pretty print it. It’s also used by micropub for providing the contents of the blog post that it’s being created.

Additionally, I added rel=me tags to the <head> node of the template used in my site. rel=me is a relationship value from XFN, used in links (<a>, <link> and <area> tags). When I tag a link using rel=me, it means that the destination of such link is another web address of mine. For instance, I use rel=me tags on this site to mark my profiles on social networks, since those profiles are also mine:

<link rel="me" title="Twitter" href="https://twitter.com/danirod93">
<link rel="me" title="LinkedIn" href="https://linkedin.com/in/danirod">
<link rel="me" title="GitHub" href="https://github.com/danirod">

Implementing some SocialWG Recommendations

For the sake of programming stuff, I want to write a few endpoints or microservices implementing SocialWG recommendations. I could use a hosted service for this, or use an already existing library, but I think it will be more of a challenge if I write the tools by myself.

I want to write a microservice or implementation for the following recommendations:

  • Webmentions: easy if I start working on Menta again. Might not take a lot of time.
  • Micropub: I don’t want to write a dashboard or an admin panel if I’ll be the single user of the system. Instead, decoupling the admin panel and having a micropub endpoint and a micropub client for writing such content would be much better.
  • WebSub: looks extremely useful; may even reuse it for other projects.
  • IndieAuth: probably required to implement a micropub endpoint.

Conclusion

I know this has been a long blog post (heh, that’s why this is not a note), but I wanted to provide some additional context on my roadmap in order to help readers understand what I’m talking about. If I’m able to inspire other people to do a similar effort and to start building an open web, it would be better.

Tim Berners-Lee, creator of the World Wide Web, has been seen in articles in the last couple of years slightly uncomfortable with the way the web is going. The web today is closed, and it’s controled by a few corporations. Things such as net neutrality and geo blocking are not making things easier.

Those who have the knowledge and the resources to do their small contribution on fixing the currently broken internet should work on fix it. This is my small contribution.

  1. Even if on the long term is probably bad for them, you won’t be able to convince your entire social graph to drop third party platforms. It’s easier (and more practical) to focus on yourself and build tools for you instead of forever preaching. ↩

  2. If you’re a GNU Social or Mastodon user, you may have already seen this, as users of a GNU Social or Mastodon node can follow users on a different node, and the servers will exchange data to make sure you get new status updates from those users and replies or interactions that you do are also sent to them. ↩

Más