Beaker Browser (mentioned as one of many possibilities in this article) is the real deal. If it doesn't get you fired up about the possibilities, then - well let me explain I guess:
* The interface is dead simple - share this folder, done.
* It is a read-write browser. Netscape (and other browsers) used to be this way - they had some limited HTML creation tools. Beaker brings this back in the form of making an "editable copy" of a website. It's a choice in the address bar.
* Making an "editable copy" doesn't have to mean you're now editing raw HTML. An editable copy can direct how it is edited through JS. (See the recently released "dead-lite" for an example of this.)
All these attempts are exciting but I'm actually starting to use Beaker because it's so useful even without adoption.
I checked out Beaker Browser, and apparently it's based on the Dat project [1], which seems to be very similar to IPFS. Then apparently it follows that, just like IPFS, you can't throw random things onto the network and expect it to stick; you need to pay someone for hosting and bandwidth (that someone could be yourself) to have it pinned, and in order to have it available worldwide at all times you still need to pay for a CDN of sort — the Linux box in your closet, or worse, your laptop that sometimes goes offline just won't cut it. Eventually it's just another protocol to copy stuff around, where stuff originates from various servers (your browser basically embeds a server, capable of serving stuff), with the possible benefit of popular stuff may be p2p'ed (but if you're a business you probably can't rely on that anyway). I fail to see how it's radically different.
(Also, I'm not even sure how you could p2p private user data, unless you expect everyone to carry around one or more yubikeys, or implant chips into fingers or something; plus all devices need into buy into that. But I haven't given that much thought.)
Some things in p2p hypermedia (dat) that's not possible with http/s:
* You can generate domains freely using pubkeys and without coordinating with other devices, therefore enabling the browser to generate new sites at-will and to fork existing sites
* Integrity checks & signatures within the protocol which enables multiple untrusted peers to 'host'. This also means the protocol scales horizontally to meet demand.
* Versioned URLs
* Protocol methods to read site listings and the revision history
* Offline writes which sync to the network asynchronously
* Standard Web APIs for reading, writing, and watching the files on Websites from the browser. This means the dat network can be used as the primary data store for apps. It's a networked data store, so you can build multi-user applications with dat and client-size JS alone.
I'm probably forgetting some. You do still need devices which provide uptime, but they can be abstracted into the background and effectively act as dumb/thin CDNs. And, if you don't want to do that, it is still possible to use your home device as the primary host, which isn't very easy with HTTP.
This is a very interesting topic, thanks for working on it and answering questions here.
The first concern I had/have is about security. If everybody runs their own server, isn't this a security nightmare waiting to happen?
I understand from the presentation that these websites won't run php or other server side scripts which at least take some concern away.
Tara also showed how easy it was to copy a website, while pretty cool, that is also a nightmare scenario for most companies. If your competitors can clone your websites and pretend to be you, how do users know who's data they are looking at?
Not OP, but I believe when it comes to website copies, you can identify which one you are actually using by the url. So if someone makes a copy of dat://mylocalbank.com, their url would be just the hash (Eg. dat://c6740...)
> You can generate domains freely using pubkeys and without coordinating with other devices, therefore enabling the browser to generate new sites at-will and to fork existing sites.
Not entirely sure what you mean,
- We can generates HTTP sites at will (all you need is an IP address);
- We have existing protocols for mirroring sites (not implemented universally, but nor is dat://);
- When you talk about pubkeys with coordination, there are obvious problems like the last paragraph of my original comment, right? Again, I'm probably misinterpreting what you're saying.
> Integrity checks & signatures within the protocol which enables multiple untrusted peers to 'host'.
Basically subresource integrity? Granted, with this protocol you can in theory retrieve objects from any peers (provided that they actually want to cache/pin your objects), not just the ones behind a revproxy/load balancer, so that's a potential win from decentralization.
> Versioned URLs
We can have that over HTTP, but usually it's not economical to host old stuff. In this case, someone still needs to pin the old stuff, no? I can see that client side snapshots could be more standardized, but we do have WARC with HTTP.
(EDIT: on second thought, it's much easier to implement on the "server"-side too.)
> Protocol methods to read site listings and the revision history
> Standard Web APIs for reading, writing, and watching the files on Websites from the browser.
You can build that on top of HTTP too.
My takeaway is it's simply a higher-level protocol than HTTP, so it's unfair to compare it to HTTP. Are there potential benefits from being decentralized? Yes. But most of what you listed comes from being designed as a higher-level protocol.
> We can generates HTTP sites at will (all you need is an IP address);
That's not really so easy from a consumer device with a dynamic IP.
> - When you talk about pubkeys with coordination, there are obvious problems like the last paragraph of my original comment, right? Again, I'm probably misinterpreting what you're saying.
You do need to manage keys and pair devices, yeah.
> My takeaway is it's simply a higher-level protocol than HTTP, so it's unfair to compare it to HTTP. Are there potential benefits from being decentralized? Yes. But most of what you listed comes from being designed as a higher-level protocol.
The broader concept of Beaker is to improve on the Web, and we do that by making it possible to author sites without having to setup or manage a server.
Decentralization is a second-order effect. Any apps that use dat for the user profile & data will be storing & publishing that data via the user's device. Those apps will also be able to move some/all of their business logic clientside, because theyre just using Web APIs to read & write. Add to that the forkability of sites, and you can see why this can be decentralizing: it moves more of the Web app stack into the client-side where hopefully it'll be easier for users to control.
Its not a higher level of http, its more like lets use torrents instead of http because they are distributed and scale better. But the web is more than http, its dns and email and logins and all of that stuff, it all scales poorly, it can all be improved with distribution, lets not replace http with torrents lets replace it all with distributed stuff.
As an example you talk about needing a special device to manage keys which presents problems. It centralises your identity to your yubi key (instead of email), lose your yubi key and you lose your identity, what if it gets wet, crushed, corrupted, your fucked. Instead we encrypt the key and distribute it across the net, if a copy is deleted or corrupted there are other copies and its available to you anywhere anytime. Currently your identity is centralised to your email, if your email goes down you lose your identity, if its distributed and a copy goes down you just use it like normal.
Distribution solves pretty much all the problems centralisation creates, its just really complicated so we generally don't bother.
Of course it's not higher level of HTTP, I never said that. I said higher level than HTTP. HTTP is just a stateless transport protocol, of course dat is higher level, and as I said much of the benefits described can be built on top of HTTP (and have been, just not standardized or not widespread).
> it all scales poorly, it can all be improved with distribution
Pretty sure it all does NOT scale poorly, as has been proven over the past thirty years. What's being solved here is not a problem of scale. "It can all be improved with distribution" is very hand wavy and doesn't really say anything. DNS and many other protocols are already distributed, btw.
> Instead we encrypt the key and distribute it across the net, ..., if its distributed and a copy goes down you just use it like normal.
There are two kinds of crypto, symmetric key and public key. Symmetric key is easily out of the window. For public key crypto, you always need a secret key and that has to be prior knowledge, not something negotiated on the fly, and of course prior knowledge has to be kept somewhere and presumably synced if you need it elsewhere, and it definitely can be lost. "Distributed secret keys solving everything" sounds like nonsense to me; there's always a secret key that is the starting point (call it the master key, if that makes more sense) and can't be distributed.
That indeed is a fundamental difference. But on second thought I got confused. Source independence, content addressable are nice and all, but we don't build static websites that always have same hashes; "Ubuntu Server 18.04.1 ISO" could be ipfs://<static_hash>, but even "latest Ubuntu Server 18.04.x ISO" couldn't be that. You still need to query the origin server (or client, whatever you call it), the central authority, to get those addresses. So, frequently changing websites/webapps don't benefit from this; they may even be penalized by the overhead. Only aggressively cacheable objects could benefit, but the vast majority of those probably won't be popular enough to be cached/pinned by peers anyway, so you still end up getting whatever you need from the origin server (or paid CDNs).
Btw, I skimmed through Beaker docs, and it seems they resolve names through DNS (what else can they do) and even use HTTP for discovery.
I'd say that most websites are static enough to be pinned. With the others, the main problem is content determinism. If the same website renders differently on different platforms, they will have different hashes. The only reliable way to store them is in "unrendered" form. Which prevents the inclusion of external resources, something that most single-page interactive websites rely on.
Naming is a consensus problem. The key here is having the freedom of choice between trusted providers. The central source could be provided by a single cryptographic key, by many keys, m-of-n schemes or other arbitrary contracts, even in P2P form.
I'm really interested in what kind of user interfaces the Beaker people come up with when it comes to their "editable cloned websites" (forks).
What is pinned would be the content that doesn't change. It may mean the site architecture would need to be changed to accommodate one of these decentralized models.
You are right, a p2p web wont solve the barrier to entry, but web hosting costs $20 a year, so its not much of a barrier.
The real cost is scale, $20 year will cover a few thousand users but if you want googles scale it will cost you in bandwidth and complexity. p2p like torrents radically reduces the cost of bandwidth by distributing it, but more importantly it reduces complexity by standardising it.
Once the complexity is standardised budget web hosting can provide google scale for dirt cheap, and there are millions of budget hosting companies too many to shutdown them all giving you censorship resistance.
The original vision for the web was that editing/creation had the same status as viewing/consumption, and that websites were writable as well as readable. This is what Amaya implemented. It never gained wide adoption, but is served as a reference implementation of the W3C's vision of the web. (In my experience Amays is not particularly usable because it regularly crashes, but that could be fixed.)
Is Beaker similar to Amaya extended to use transport layers beyond http, such as ipfs?
I'd hazard to say that the concept of "wiki" is what successfully implemented that early vision of editable WWW.
Wiki markup is different from HTML markup, but it represents many of the same (early) text-formatting and resource-linking concepts, while limiting the excessively powerful features of arbitrary layout, scripting, etc.
Have a look at the federated wiki [0], it allows someone to easily fork someone else's page, edit it, and host it. History is tracked at the paragraph level.
I can't really answer this yet, because I've never heard of Amaya and I don't want to answer based on a cursory glance - but I really appreciate this link, as I love bits of Web history and I am going to do some homework on this. So, thankyou, femto!
So - there is no server-executed code - it all runs in the browser and the folder can only access itself anyway, which can't happen unless you have the private key.
Access control in Beaker is through that private key - you need it in order to edit the 'dat' (name for a synced folder). So, no, there aren't a lot of complex permissions available - but you can also separate an app into several dats and use a master one to manage the permissions of those. Not terribly complex, but it's actually surprising how much you can do. (It's tough to wrap your head around not having a server - but it's actually true.)
But help me out - I think alot of the Web falls into this category:
* User logs in to edit their data (has private key to their dat).
* User shares their data (blog, photo feed, whatever) with others (who don't have the key).
* Those others merge all incoming feeds into a single master feed.
You could replicate YouTube, Facebook, Twitter this way - usually there are not complex permissions in these apps, are there? (Not that you'd want to replicate them...)
Umm, having everyone’s data as a flat directory as opposed to an aggregated database sounds terribly inefficient... You need to somehow build a distributed, decentralized database on top of that flat structure, right? Otherwise your Twitter is just a microblog publishing tool plus a direct crawl RSS reader...
Maybe Twitter is too specialized an example. What about any kind of search? You do need an index, and someone still has to own that index, and “donate” computing power to update that index. You own your self-hosted data, like many of us already do, but there will still be gatekeepers, e.g. Google for our current web.
EDIT: I realized that with a clever enough architecture and probably much more computing power than necessary in a trusted environment, no one needs to own the index. But it seems way more advanced than this protocol. (I’m completely new to this stuff so please excuse my naive skepticism.)
No, no - I understand on several fronts: first, there is just so much technology these days, it's tough to find anything that isn't just a fleeting thing; also, you're absolutely right that you can't just solve everything with a distributed filesystem.
I also am not sure what yourself (or newnewpdro) are looking for in the web or what appeals to you - for me, Google simply doesn't work for me - sure for technical issues it does, but it is basically Stack Overflow search in that department. If I'm looking for personal blogs, I can't just type "personal blog" into Google and find anything worthwhile - it's all clickbait of a fashion. The best way I've found of finding blogs is either to look through Pinboard tags or to click around on other blogs until I eventually get somewhere. It's horribly inefficient - but it's rewarding when I get there. I'm making a personally-edited blog directory to try to aid discovery - and yeah I actually think there's a lot we can do if we all did more grassroots search and directories. Anyway, that's my perspective - wondering what you're looking for in this thread. Have enjoyed your other questions above (below?)
> Access control in Beaker is through that private key - you need it in order to edit the 'dat' (name for a synced folder).
You're referring to write access, which is a small subset of access control.
How do you restrict read access to a group of specific people? Encrypt the data and distribute keys to the privileged parties? How does revocation work?
Yeah, so in my case, I wrote a blogging thing that runs in Beaker. I can run it from any of my machines and it has no server. This is great because it is insanely portable - I can setup the software on a new machine by just going to the URL. It is just "software as a service" but there is no server. It's somewhat similar to TiddlyWiki but Beaker adds automatic synchronization. (I still manage the JS code itself in git, but the blog posts are managed by Beaker.) Thank you for the question, styfle.
Look up Tara Vancil's talk on "A Web Without Servers" if you need a crystal clear explanation - don't know if I'm doing adequately. And my blog is at kickscondor.com if you're curious why I had to write my own blog warez. Also, there is a resurgence in blogging happening right now with the shakedown of social media. It's great.
1. A (current?) limitation I've noticed with Beaker is that you can only edit a site implementation (for a specific address) on a single machine. What would you do if you had multiple computers / locations that would want to make updates (JS or User Content)
2. What about errors. Don't they persist in the address's history? What if there's something undesirable that got added by accident
3. What about mobile? How could someone visit/browse on mobile without a non-distributed proxy http address
Yeah, cool - thank you for the questions, kingnight.
1. So this answer is a bit convoluted because I am still learning, wish I could keep it short. So my setup might be a bit 'naughty' because I'm currently saving the 'key' in my JS in a seperate dat. Right now I have one dat that acts as the 'admin' and one that is the actual blog - but I am going to move to hyperdb (the new solution for multi-writer support.) There are actually a couple libraries cropping up for doing this sort of thing and I'm not up on all of them. So this is more of a 'need to make up my mind' thing than a capability thing with Beaker. There are a TON of libraries and a TON of possibilities - (there's an 'awesome dat' page that just goes on and on and on...) - but I am still researching a better way (and, who knows, maybe my way is fine.) I want something that could be in place ten years from now - because I do think JS and HTML will.
2. No, files can get replaced. Not sure if that is your question. Yes the undesirable will persist in the history, but you can overwrite it. I can prevent bad content, though, by checking it in my JS code and allowing a preview first.
3. Yeah this is a problem - there is a Bunsen Browser in development, but I haven't tried it. I am honestly okay with everyone browsing through HTTPS, though - I like Beaker for the admin tool. Again, I have a browser-based blog software without needing to run a server anywhere at all.
Thanks much for the replies. I’m very interested in this and am wrapping my head around how it works and these differences so it’s great to hear perspectives from people using it.
I'm not sure how I would talk about the CSS and JavaScript - which was the most work. I will think about 'if'/'how' I can describe that. I really appreciate the encouragement. Please send me a link to your blog, if you have one. I am collecting links to personal blogs. See you around, kingnight.
If you visit kickscondor.com be sure to turn down volume first. You may find out as I did that if you visit at night its bleeps and bloops can wake up a partner :O
Thankyou for saying something tomcam - I've turned down the volume on the sample - please apologize to your person in bed. I might also consider closing down in the evenings.
I love your blog @kickscondor. The article you have about using Twine with the kids and the way they used it for building linked stories, reminded me how Twine is such a natural and intuitive tool for authoring linked chunks of content, much more than typing in HREFs.
If you were able to build a visual map and bridge to someone else's paragraph... that would make the distributed system's easier to grasp.
Hey thankyou! This is very encouraging. It's true - kids really love Twine and it is one of the very few tools that comes automatically to them. It is better than code.org at what code.org is 'attempting' to do.
Really cool point about extending Twine! That had never occurred to me. Amazing.
Hi, I've been playing with making something similar myself -- although using IPFS rather than Dat. I notice you've married up Webmentions and Dat; how have you done that? The only real solution I've come up with is to run a bunch of regular services at an HTTP endpoint (for Webmentions, ActivityPub, Webfinger etc.) that then generate new builds of the static site, which gets added to IPFS.
Oh believe me - that's exactly what I'm doing :D Webmentions are very much made for HTTP - since they require an HTTP POST. A variation would have to be made for Dat.
(As an aside, I originally didn't like using Webmentions on a static site - I had planned on making a cron to periodically check for Webmentions and republish. But now I really appreciate that it goes hand-in-hand with moderating comments. I look over the incoming Webmentions, nuke any spam, and republish. No bad feelings about comments that sat in the queue for a day - they are still out on the web at their original URL.)
* The interface is dead simple - share this folder, done.
* It is a read-write browser. Netscape (and other browsers) used to be this way - they had some limited HTML creation tools. Beaker brings this back in the form of making an "editable copy" of a website. It's a choice in the address bar.
* Making an "editable copy" doesn't have to mean you're now editing raw HTML. An editable copy can direct how it is edited through JS. (See the recently released "dead-lite" for an example of this.)
All these attempts are exciting but I'm actually starting to use Beaker because it's so useful even without adoption.