What is remoteStorage?

remoteStorage is a simple and brilliant way to free our data. I'll just post an explanation I just gave on #erights here.

(07:04:07 PM) msimoni: zooko put remoteStorage this way yesterday on tahoe-lafs: "The point is that a human can type in "zooko@example.com" and the app contacts example.com and somehow persuades example.com to authorize it to use storage there and then the app uses storage at http://example.com/storage/zooko"
(07:04:26 PM) msimoni: the zooko@example.com is a remoteStorage account identifier, not an email address
(07:05:02 PM) msimoni: you use web apps, like say a document editor, and "bring your own storage", and the web app uses the storage to store your documents
(07:05:23 PM) msimoni: it basically uses GET and PUT on a fairly flat hierarchy at http://example.com/storage/zooko
(07:05:31 PM) dash: nice
(07:05:39 PM) msimoni: (the form of the URL could be anything)
(07:06:31 PM) msimoni: and you have a bunch of private "categories" e.g. "documents", "pictures", which are totally private to you
(07:06:52 PM) msimoni: plus there's one special category "public" that's world-readable, but not world-enumerable, so you can use unguessable identifiers in there

That's it!

P.S. To try it out for yourself, you can get free storage at a remoteStorage-enabled provider such as owncube.com, and then try out this tutorial.

CCN@PARC update

As Graydon Hoare says:

CurveCP + CCNx = future. Hurry fast new-topology-net, we need you.

(Via neuraxon77.)

Serverless Social Web Apps

This post is in response to Mark Boas' P2P Web Apps - Brace yourselves, everything is about to change.

Mark writes:

The old client-server web model is out of date, it’s on its last legs, it will eventually die. We need to move on, we need to start thinking differently if we are going to create web applications that are robust, independent and fast enough for tomorrow’s generation. We need to decentralise and move to a new distributed architecture

I've come to the conclusion that something like Git is the right architecture to move forward. This post tries to explain why and how.

It's somewhat disorganized, but I want to get this out of the Drafts folder.

A Git overview

Git is extremely simple: you have:

• an object store, mapping content addresses to content objects
• a set of references, mutable pointers to certain objects

Git uses trees to create filesystem-like hierarchical structures.

By knowing the root of the tree, we know the whole tree. (The connection to persistent data structures is interesting.)

Commits are just another kind of content object, which point to a particular tree. Commits also point to their parent commits, creating a chain of commits, recording the history of the repository over time.

Git's architecture has huge benefits:

• The integrity of arbitrarily large and complex data structures can be verified with a single hash comparison.
• Synchronization between two repositories is made extremely efficient: the repo that wants to sync with a remote repository sends the IDs of commits it has to the remote, which tells the remote what objects the repo is missing. These are then sent to the repo in a compressed packfile.

Git for P2P Web Apps

What's a P2P web app? I'm thinking of things like del.icio.us, wikis, Twitter - but serverless.

Well, there may be servers, but they're not the central component anymore. They can act as caches, and as always-on storage.

How could this work?

• Local storage: A user's data is stored locally in the browser.
• Browser-2-browser communications: There's a way to talk to other users' browsers. (Initially this will have to run through a in-cloud relay server, but I'm sure we'll get browser-2-browser comms soon. See for example IETF's Real-Time Communication in WEB-browsers.)
• Efficient sync: Use Git-like protocol to retrieve other users' data and cache it locally.
• Cloud-based caches/storage: Servers are just peers in the network, that are usually online.
• Indexing: Peers index interesting data using IndexedDB to provide views like timelines and search.
• Security: This is really a huge issue, but I think it can be solved with a liberal sprinkling of public-key cryptography. ;)

One problem Git has is with very large directories. But we can easily work around this using something like a date-based YYYY/MM/DD hierarchical structure.

The main thing to get is that integrity and efficient transfer of changes falls right out of the Git model (or any similar content-based model.)

The great aesthetic which will inaugurate the twenty-first century will be the utterly invisible quality of intellectual integrity; the integrity of the individual dealing with his scientific discoveries; the integrity of the individual in dealing with conceptual realization of comprehensive interrelatedness of all events; the integrity of the individual dealing with the only experimentally arrived at information regarding invisible phenomena; and finally integrity of all those who formulate invisibly within their respective minds and invisibly with the only mathematically dimensionable, advanced technologies, on the behalf of their fellow men.

- Buckminster Fuller, 1973

App reviews as a new art form

Towards independent web apps

Werner Vogels is stripping away dependencies from his weblog app in a way that all Winerians will find pleasing.

So now we're serving static HTML like it's the 1990's all over again. (Only this time, we're saving them to the file server via HTTP, not FTP.)

This has the nice effect of not tying us into some weird, proprietary server - the only thing it needs to understand is GET and PUT, basically.

But this time, it's the 2010's, and we now have browsers powerful enough to run whatever apps we desire.

So: what about running all the HTML generation stuff in the browser, and using the server only as dumb storage? Just an idea.

The Intimate Internet: You're gonna need a thick skin

Due to social networking, we're all neighbors now. (And it's only getting worse.)

I find that this deprives me of a lot of options for jokes. The people (and their projects) I want to joke about are just one hyperlink away. I myself am quite sensitive to criticism, so I also hold back criticism of others.

But criticism is important, lest we all become a bunch of homogeneous circlejerkers.

In the old days, artists, and everybody who put out stuff into public view needed a very thick skin. It came with the territory. If you were an artist, you had to live with those blood-sucking critics.

But because the internet is so low-key, we're all just normal persons now, putting our projects out into public view.

But that shouldn't stop us from criticizing each other. I think we all need to grow thicker skins, so that we can allow the criticism that invariably comes from public attention.

Get ready to be dissed!

Content-centric networking and microdata

PARC's content-centric networking (CCN), aka named data networking, is a "master idea" about how to re-architect the internet, given what we've learned since the WWW took off.

Like Git, CCN uses content-based addressing. A piece of content has a symbolic name (e.g. /com/nytimes/front-page/2011/02/19), which is cryptographically bound to a signature of the creator of the content (e.g. NYT Corp), and to a hash of the data.

Symbolic names have hierarchical structure, and are late-bound: a common example is /room/projector which is an interface to the projector in the current room. (Plan 9, anyone?)

There are two kinds of packets used in the CCN protocol: Interest and Data packets.

• An interest packet is analogous to an HTTP GET, and expresses the client's interest in a piece of content given its symbolic name.
• A data packet is analogous to an HTTP response, and satisfies an interest. It contains arbitrary content, cryptographically bound to its creator via a signature and to its symbolic name. The receiver can thus easily verify that the content is legit.

CCN can not only be used for classic WWW-style applications, but also for multimedia protocols like VoIP. Results show adequate performance compared to stock protocols. From CCN's content-based architecture we get a lot of benefits for free (e.g. caching is much easier).

Content-centric architectures seem ideally suited to the problems of our current, heavily fragmented social web experience. We still need to figure out the details, though.

See also: camlistore, Content-Addressable Multi-Layer Indexed Storage, a "gitified" database, by the Google frat pack.

Dazed and slightly confused notes on the future of web platforms

First, there's the Locker Project, motto: "I am the platform".

Even though it's by the creator of Jabber, it's not about streaming XML stanzas, thank you very much.

Locker uses JavaScript connectors, that know how to talk to a service to retrieve and sync your stuff (Flickr, Twitter, ...). Think Emacs modes for different services.

Locker seems to run on the server, using node.js, so until everyone of us has their FreedomBox plugged in, this will not really give us freedom.

What if we ran this in the browser, using client-local storage? And then, as Dave Winer would tell us, we push that stuff as RSS, err, Atom feeds, maybe connected using Atom Paging and Archiving, to a dumb cloud store, hopefully free as in freedom? (This is where Content-centric networking comes in, as a simple way to dedup, among other awesomeness.)

Doing the crawling of a user's feeds in the browser has the big benefit that we have cycles to burn and bandwidth to waste there. Scalable following is damn hard, which can be seen from the fact that it's one of the few algorithms that's not provided as a shrink-wrapped, commoditized package. Yet.

The second piece I find wildly interesting is homomorphic encryption. As far as I can tell, the promise of homomorphic encryption re cloud storage is that the cloud store can sort and search encrypted data. I.e., data it can't even look at can still be subjected to the Big Data treatment, aka Google's infinitely-scalable b-tree. (Links: Order-preserving Encryption, Cryptographic Constructions for Secure, Privacy-preserving Distributed Information Sharing.)

Exciting times!

Of course, if we want to do to microdata and the web what Emacs did to plain text and Unix, first we need a real Lisp in the browser. Thankfully, that's exactly what I'm working on at the moment. Stay tuned.

Balm for the soul

Dean Bubley:

Tablets - I've been somewhat surprised by the iPad's success, but I still don't see tablets as a major game-changing trend, beyond the "Apple Effect". I still think that the iPad is primarily a nice (and for some groups of people also very useful) gadget which complements their PC/Mac and smartphone usage. I don't see massmarket Android platforms fulfilling the same roles and I certainly don't see tablets heralding some sort of mythical "post-PC" era. I do seem them as becoming important for "social TV" use cases in the home, though. I expect to see a declining % of tablets with embedded/activated 3G radios going forward - the bulk will be WiFi-only. 2011 success rating: 5/10

Let me repeat:

I certainly don't see tablets heralding some sort of mythical "post-PC" era.

I really like these obnoxious new FSF banners

Abandoning del.icio.us?

I understand that people are upset, but I don't get the point of moving from one problematic provider to another equally problematic provider.

It's the whole concept that's rotten, not just particular bad apples.

And in the meantime, I lose easy access to your much-valued data.

Thus, for 2011, we should all work hard on two issues:

• Easier, but still hard: Decentralization: it shouldn't matter which problematic provider you're signed up to - I still want to browse your data in my problematic provider.
• Hard: Make this sh*t work: We should found a democratic trust that hosts our data on a free software stack, so that we can get rid of problematic providers altogether.
  

Our data is worth much more than gold!

The cloud's 9/11

I think the del.icio.us situation was a wake up call for a lot of people.

Keeping it short, here are my suggestions for the future:

• Let us pay: As they say if you don't pay for a product, you are the product.
  
• A company ain't enough: Classical capitalism isn't fit to provide the service we need. I'm thinking about some sort of netizen's trust.
  
• It takes an architecture: The whole stack has to be free software, and good one at that. Everything else won't cut it.

Our data is worth much more than gold!

Institutional innovators welcome!

I'm afraid of sociotards

Exhibit N:

See that cute girl or boy sitting over there? Imagine seeing what you have in common at a glance.

Exhibit N+1:

you will be able to "see" all your "friends" all the time and know which movie or TV show they're watching and watch it "with" them if you want

The Desktop is Back

Sony is back to OpenStep (!) with SNAP:

Sony’s Networked Application Platform is a project designed to leverage the open source community to build and evolve the next generation application framework for consumer electronic devices.

The developer program gives access to a developer community and resources like SDK, tools, documentation and other developers.

The foundation upon which this project is base comes from the GNUstep community, whose origin dates back to the OpenStep standard developed by NeXT Computer Inc (now Apple Computer Inc.). While Apple has continued to update their specification in the form of Cocoa and Mac OS X, the GNUstep branch of the tree has diverged considerably.

Wayland is talking about phasing out X11:

The problem with X is that... it's X. When you're an X server there's a tremendous amount of functionality that you must support to claim to speak the X protocol, yet nobody will ever use this. For example, core fonts; this is the original font model that was how your got text on the screen for the many first years of X11. This includes code tables, glyph rasterization and caching, XLFDs (seriously, XLFDs!) Also, the entire core rendering API that lets you draw stippled lines, polygons, wide arcs and many more state-of-the-1980s style graphics primitives. For many things we've been able to keep the X.org server modern by adding extension such as XRandR, XRender and COMPOSITE and to some extent phase out less useful extension. But we can't ever get rid of the core rendering API and much other complexity that is rarely used in a modern desktop. With Wayland we can move the X server and all it's legacy technology to a optional code path.

Jean-Louis Gassée:

Someday, the progress in HTML implementations and better, thicker pipes might move the boundary between local and Cloud applications. But for the time being, conventional desktop “productivity” apps such as word processors, spreadsheets, and presentation programs have an advantage over their Cloud competitors.

Anil Dash:

Cloudtop apps are delivered as native code on nearly every supported platform, from desktop computers to smart phones, with an interface that scales appropriately.

While the app may have a web interface, that's largely a convenience and is not usually the primary way in which you interact with the app.

Oh, and let's not forget RIM's acquisition of QNX.

OpenID is a nightmare:

We already have the ability to sign up with username/password - but we're going to make that the *only* way to register for our site from now on.

Apps On!

New Post-it notes by 3M for mobile dev:



(Although I find the image of hipster mobile developers scribbling on them while sipping their lattes deplorable.)

It's bad

They looked in my iPhone's address book.

... it's not like it could be anyone's "business model" to use that data. But now, as far as I know, some unknown startup in California has all my data.

This is bad and ugly on two levels: first, that a company would slurp your private data without asking; second, that your computer lets it do this.

there is hope

the use of location-based services is actually declining in America, from 5% of the online population in May to 4% last month

trotsky on HN:

If facebook can really deliver a rock solid web mail client, I think they will really be a company to watch in the rest of 1996 and 1997.

Dear Google

If you read this - please let me turn off that instant preview thingy.

When I look at a web page, I like to massage the text by clicking around a lot on the page. (It's a subconscious thing...) Now, this brings up preview images.

While we're at it: what's the star icon next to each result for?

Thanks,
Manuel

(via copyranter)

Windows Phone 7

I can't believe I'm saying this, but Microsoft did something interesting, and maybe even great, with the design of WP7:



I like this typographical approach, it seems great for small screens. And it's a welcome change from Apple's infantile eye candy.

Cloudtop Apps

Anil Dash is certainly onto something with Cloudtop Apps:

Cloudtop apps are delivered as native code on nearly every supported platform, from desktop computers to smart phones, with an interface that scales appropriately.

While the app may have a web interface, that's largely a convenience and is not usually the primary way in which you interact with the app.

Given the arcane techniques required to build interactive web apps, and the puny results, I can certainly see the appeal of writing a bunch of native apps instead.

Support costs should be lower, it's probably easier to charge for native apps than web apps, and heck, even development costs may actually be lower.

Umair Haque firing on all cylinders

101 Let's take a sec to clear up a few misconceptions about my perspective.

Five Reasons I Wouldn't Have Invested in Zynga Selling virtual goods is interesting, right? Wrong. It’s just about as interesting as selling physical “product” – an industrial age revenue stream in disguise

MyGengo and the Power of Markets, Networks, and Communities if you want to be disruptive, here’s the single strategy that will take you the furthest. Pick a zombified, moribund industry, and use a market, network, or community to disrupt it, by altering the structure and intensity of search, monitoring, and transaction costs.

21st Century Capitalism vs FoxMartWorld Like I sometimes say: want fries with that Ponzi zombieconomy?