A distributed Web

Posted on Jul 22, 2019

This is not a real, well contemplated article. It’s just two posts on SSB in which I shared my vision of a distributed web (I know, the irony is that this blog is not available via dat yet).


%bh7GPg6oaLAc3eZw/TNrNvRWNM0LRtJy8Ycni3YfPEM=.sha256 , posted on Jun 30.

Thoughts on how to build a gossip-web

Abstract

Motivation

I think we can all agree that building a decentralized web, based on p2p technology is one of our major goals when it comes to internet-ish applications. Not only do we object the idea of central authorities, nor do we accept the fragility of the current technology.
Right now, the building blocks for the next-generation web are at our hands, enabling us to build a network that reflects and embraces the strength of our social connections.
If we want to declare the Cyberspace as independent, we must pursuit the technology that makes independence become reality.

In the future, resilience will be of the essence. While governments and cooperations are putting more and more effort in fighting the free internet, both for different reasons, we as a community cannot re-do the effort that has been taken to build the net as it is right now.
In a self-sustaining future, we may have a number of networks that eventually won’t be interconnected by hi-speed fiber-links or won’t have global reachability. This is where gossip protocols come in play. Since people travel around, visiting a number of different communities on the way, there is a way to implement a sneaker-internet connecting local networks by transporting data from one place to another in an automated fashion.
While we already have scuttlebutt as a gossip-based social-network, we’ll need a similar approach to form a gossip-based web.

Goals

The main goal of the gossip-web is to establish the tools we’d need to keep information flow freely, even without the regular Internet underneath.

An Idea for an Implementation

To explain my thoughts on the gossip-web, let me explain how I thought one could implement it using current tech.

Content distribution

If we want to distribute and consume content we need a protocol that takes care of replicating, locating and distributing it. I found the dat-protocol to fit well for our needs. It let’s you access content from other computers in reach and it let’s you replicate it, so others can access it from your machine.
Note: From here on, I may use some dat-related terminology.

Content propagation

Without permanent global interconnection, we’ll need a way to propagate content. Here the gossip-model comes in play by publishing links and metadata (more on this later) to every site you seed into ssb. Ideally, you would build a local search-index containing links, descriptions, etc. This search-index would then be published via dat and a link to it would be posted on ssb. As your node finds links to your friends’ search-indexes on ssb, it can download them (if currently available) and merge them into it’s own index. Additionally, you could publish subscriptions to specific topics, keywords, authors, etc. on ssb as well. Everyone would then download websites from the local search-index that match its own or their friends’ subscriptions.
This models how we would propagate links as human beings. When we recommend books, magazines or articles to each other that’s basically what we’re doing.


%IujUftuYJ/4h0KPCP2YhRPXFmYS3b/glCgILSHY69oU=.sha256, posted on Jul 17.

More thoughts on the distributed Web

As I imagine a utopian future of the web based on the technology existing/emerging at the time I consider #dat and #ssb having two distinct but important roles:

  • SSB modeling us as individuals, including our social graphs

  • DAT transporting content that may or may not be associated with an individual but in any case has a role on it’s own (websites etc…)

What we miss here is something replacing the traditional concept of namespaces and domains as provided by traditional DNS. In the past I have played around a bit with GNS (from #GNUNet) and it’s concepts. Domains are something tied to an entity or person, so maybe it might be a cool Idea using SSB to manage namespaces mapping human-readable names to dat-links. (e.g. dat://blog.vi.ssb/index.html mapping to a dat-link I posted in a special-formatted message). All it would take is a message-format for SSNS (Secure Scuttlebutt Name Service, I just came up with this..) and a local dns-proxy resolving .ssb tlds locally by querying the flumedb.