Unix in 14 lines of Ruby (it's trivial)

Anyone under 17 knows that Ruby is so extremely dynamic, that Google's complete Map.Reduce framework can be implemented in one line of code:

data.map.reduce

(Note the expressive and elegant object-orientedness that lies at the core of Ruby's efficient syntax...)

But now R is giving Ruby a run for it's money: Map/Reduce has been ported to R with just one line of code, too!

So clearly, we need to raise the bar here: I will show you how to implement Unix in a couple of lines of Ruby.

Yes, you heard that right. Ruby is so elegant and dynamic, that Unix takes just a couple of lines of code. Take that, C.

Other languages, like LISP, have lambdas and closures, but Ruby not only has these, but also continuations, blocks, procs, scopes, dos, and bacon! Mmmmh, bacon. (And btw, Ruby can implement Common LISP in a couple of lines.)

(There are a lot of articles that explain the differences between these, but as of Ruby 2.1.003m the schemantics of most of these have changed slightly, so the articles are out of date. As a rule of thumb: a continuation is a block that has been captured inside a scope where a do is in effect, and cannot be passed to a proc, only to a lambda. It's really quite simple. Matz has recently proposed a ten-year moratorium on Ruby development, so that we'll finally be able to understand all these.)

So here it is!



A sample run of this Unix for you're pleasure:

You have no mail.
$ uname
Runix 1.0
$ halt

All it takes to implement Unix in Ruby is 14 lines of elegant code.

It doesn't do processes yet, and it has no filesystem, but apart from that, it's a fairly complete Unix.

Ruby is a butterfly.

Steps Towards an Acceptable Lisp

Houston, we have a problem

I think that these two quotes capture perfectly the dilemma Lisp is in:

Common Lisp: The king of programming languages. – Slava Pestov

Lisp is Not an Acceptable Lisp. – Steve Yegge

On the one hand, Lisp is totally lovable, on the other hand my heart aches when I look at current Lisps.

Here are my thoughts about what an acceptable Lisp would be:

Generic

An acceptable Lisp would be generic, or protocol-based, meaning that all data types are defined by interfaces, rather than concrete implementations. Think STL, not conses.

Orthogonal

An acceptable Lisp would be built from orthogonal primitives.

One example is control flow. An acceptable Lisp could for example include call/ec and unwind-protect, and be done with it. No need for Common Lisp's salad of catch, throw, tagbody, go, block, and return.

Modular

An acceptable Lisp would include a serious module system, like O'Caml's, that really allows for programming in the large.

Object-oriented

An acceptable Lisp would be as object-oriented as Smalltalk, which means that the only way to get at an object's internals is via messages.

CLOS's generic functions show that OO integrates beautifully with Lisp's semi-functional approach to programming.

Extensible

An acceptable Lisp would be as extensible as existing Lisps, which means at least macros and reader macros, and maybe even OMeta-style completely extensible parsing.

Cutting-edge

An acceptable Lisp would have to bite the bullet, and do some language research, and/or adopt some of the novel results of the Scheme community.

One example is clear staging, aka phase separation. eval-when is simply ass-backwards.

Another example is a hygienic defmacro. It's the Lisp spirit, as evidenced by PLOT's use of hygienic macros.

Proven

Lisp is still lightyears ahead of most other dynamic languages, and an acceptable Lisp would have to carry forward these proven features: rich arithmetic, meta-object protocol, restartable conditions, optional dynamic scope, compiler-at-runtime, generalized references, etc, etc, etc.

Realistic

An acceptable Lisp would have a BDFL, a single implementation, focus on UNIX only, include lotsa batteries (or even better, power generators!), etc, etc, etc.

Unfriendly

I think an unfriendly community is a great feature, because it keeps idiots out, and there's no place for idiots in the world of Lisp.

This is (a wee bit) tongue-in-cheek of course, but the community should demand from n00bs that they do their homework, and do it well.

I'd much rather have a plainspoken, occasionally angry Linus-type character as BDFL, than a Guido-type, who tries to be n00b-friendly and forthcoming all the time.

Conclusion

Designing an acceptable Lisp is no walk in the park.

The designer of the Next Big Lisp has to keep a large area of the Lisp design space in his head, and make tasteful design decisions.

So, go ahead and just do it!

Python Community in Anguish, Pain, Despair Over Web Server

First they had the guts to punk TechCrunch, now Facebook picks a fight with the Python community by releasing a web server, called Tornado, developed by recently acquired Friendfeed.

Terry Jones, CEO of Fluidinfo, probably best expressed the pain felt by Pythonistas: "Words fail me on this one. I’ve spent some hours today trying to put my thoughts into order so I could put together a reasonably coherent blog post on the subject. But I’ve failed."

Facebook's Bret Taylor explains the reasons for such a highly unpopular move: "We ended up writing our own web server and framework after looking at existing servers and tools like Twisted because none matched both our performance requirements and our ease-of-use requirements."

The performance myth was quickly dismissed by an article which shows that Twisted performs almost as well as Tornado.



Pwpwp looked at the Twisted documentation, to dispel the ease-of-use myth. The wiki page Twisted Web Plan (aka What's Going On With Twisted Web) explains:

Currently there is a lot of confusion as to what to do and where to go to get a good, supported twisted.web server. Users are confronted with 3 options and an infinite number of permutations of those options: twisted.web, nevow, and twisted.web2. This confusion is made manifest in the lengthy explanation of web development with twisted hosted here on this wiki.

This is mostly a problem of perception, but there are some real issues. For example, there is a lot of redundant maintenance going on in, for example: twisted.web.static, twisted.web2.static, and nevow.static; twisted.python.urlpath and nevow.url; nevow.appserver and twisted.web.server.

As you can see clearly, it's mostly a problem of perception. The page continues with inspiring optimism:

However, at some point in the future, there will be one supported, good web server in the Twisted community, and that will be twisted.web.

A second page, titled Web Development With Twisted functions as a "roadmap to the wilderness that the landscape of web development with Twisted has become."

In conclusion, Twisted performs almost as well as Facebook's "cobbled together" newfangled Tornado, and Twisted developers have provided a clear roadmap to the "wilderness that the landscape of web development with Twisted has become".

Why Facebook/Friendfeed decided to create a new web server is completely beyond us.

Why I don't want lightweight threads in my programming language

1. Integration with your virtual machine, be it Linux, the JVM, or Python, is integral to success. None of these platforms support lightweight threads, so you'll always be "foreign", and not "native" (cf. Erlang C nodes).
2. Scheduling should be done in the OS. If your OS' scheduler sucks, make it better, instead of implementing your own scheduler.
3. RAM/SSD storage means that we'll probably be CPU-bound in the future.
4. Real programming languages let you implement your own concurrency models. Try that with Erlang.
   

Zero-Latency Internet GUIs Using "Multiple Worlds"

I have discovered an interesting way to write zero-latency user interfaces for internet services.

It's similar in spirit to Alice ML's first-class futures, promises.

The basic idea is that every user operation that hasn't been acked yet by the server, results in a new "possible world" in some part of the user interface, that may have to be rolled back (if the server fails, for example).

The model (if you're a MVC person) is represented as a tree of operands, which are basically futures, containers for variables that reside on the server. GUI widgets subscribe to operands as listeners, and GUI operations are represented as first-class objects that change an operand.

/** A usually high-latency action that changes the states of one or
   more operands. */
interface Operation {
   public String getDescription();
}

interface OperandListener<OP extends Operand> {
   public void onOperandChange(OP operand);
   public void onOperandFailure(OP operand);
}

/** An observable variable with a last known state (usually retrieved
   from the server), and optionally a pending operation and a
   tentative state (a state that the pending operation would like the
   operand to have, but that hasn't been acked by the server yet).

   Clients of the operand should usually just call getState(), which
   returns the operand's tentative state if there is one, or the last
   known state otherwise.  This guarantees that the user sees an
   up-to-date image of the operation's progress.  (There is a danger
   however, in that the tentative state of an operand may be invalid
   from the client's perspective.)

   There can be only one pending operation on an operand at one time,
   and trying to start a new operation on an operand with a pending
   operation will result in an error. */
abstract class Operand<T> {
   protected T lastKnownState;
   protected T tentativeState;
   protected Throwable currentFailure;
   protected Operation pendingOperation;
   protected Set<OperandListener> listeners;
}

A GUI operation simply puts an operand in a tentative state, which locks the operand, so that it cannot be changed (by another operation) until the server has acked the change.

This means that a user can edit multiple objects on the screen, and each of these changes results in a zero-latency screen update to the new tentative state, and a background thread that tries to update the state on the server.

(I haven't implemented this yet, but it would also be possible to merge commutative operations.)

Storing JSON as Protocol Buffers

I love JSON (and I think that my HyperJSON format is the best data model in existence) but for some reason or other, I always have trouble with JSON libraries for mainstream programming languages.

As a way out, I have defined my own data model, similar to JSON's, but I'm serializing it as Protocol Buffers using this .proto file:

message Data {

    optional Text text = 1;
    optional Link link = 2;
    optional Entry entry = 3;
    optional Feed feed = 4;
    optional Date date = 5;
    optional Bool bool = 6;

    message Text {
        required string string = 1;
    }

    message Link {
        required string href = 1;
        optional string rel = 2 [default = ""];
        optional bool rev = 3 [default = false];
    }

    message Feed {
        repeated Data element = 1;
    }

    message Property {
        required string key = 1;
        required Data value = 2;
    }

    message Entry {
        repeated Property property = 1;
    }
    
    message Date {
        required int64 millis = 1;
    }

    message Bool {
        required bool bool = 1;
    }

}

There are a lot of different ways to store JSON in PB, and the details are pretty much irrelevant. An important point though is how the JSON objects (Entry) are stored: simply as a list of Property objects.

It's the same as with object-oriented programming: you never want to model extensible records (business objects, documents, etc) directly as objects (Protocol Buffers in this case). Instead, you want to compose your business objects from your programming language's native objects.

See Being poor at modeling is essential to OOP.