Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The gist—as I understand it—is that Unix programs are essentially all equivalent to functions : ([String], TextStream) -> (TextStream, TextStream) which limits how one can tie them together and causes a proliferation of text-processing tools (awk, sed, grep, cut, &c &c) which is not as elegant as functional programming. There are already some Unix command-line tools that work on structured data as mentioned, e.g. jsawk[1] is a tool inspired by awk whose scripting language is JavaScript and which works on JSON.

As an aside, I'd like to see people experiment with these concepts on an OS level while consciously targeting Xen or other virtualization systems. Already, Haskell can run barebones on Xen using HaLVM[2] and the successor to Plan 9, Inferno[3], was a virtual machine that could run either on bare metal or inside another OS. I can imagine an entirely new OS would meet some resistance—like Plan 9 did—but supplying an OS intended to be virtualized would let people experiment freely within their existing OS.

[1]: https://github.com/micha/jsawk [2]: http://halvm.org/wiki/ [3]: http://code.google.com/p/inferno-os/



Just because data is in a text stream doesn't mean it is not structured. Awk is designed to handle structured text for example, the output of most (traditional) Unix tools is 'structured' too (eg., ls and ps).

I recommend you read The Unix Programming Environment which gives many illustrations of how the Unix 'text streams' model works in practice.

One of the authors of that book and other folks that created Inferno (and Plan 9 and Unix) are now at Google working on Go.

Go channels are obviously not the same as Unix pipes, but there are similarities.


Traditional Unix stream processing assumes a specifically line-oriented structure, though, and is somewhat awkward in other cases. The Plan 9 people recognized that, and made some attempts to improve it, for example with Rob Pike's concept of "structural regular expressions" to describe streams that have a different "shape" than an array of lines: http://doc.cat-v.org/bell_labs/structural_regexps/


Not really sure how you jumped from structured data in Unix to a bare metal OS, but this project came up a few weeks ago, OCaml programs on Xen with no OS:

http://www.openmirage.org/wiki/papers (down now, doh)

Seems to have some similarities to halvm.


It's back up now; the ML TCP stack was running a pcap dumper for debugging, which didn't cope well with a hackernews link.

Hacking is going pretty well on Mirage. The longer-term plan is to generalise the support libraries to work with other languages (particularly HalVM and GuestVM for Java), but it's far simpler to work with just OCaml for getting the first cut out. The Xen Cloud toolstack (also written in OCaml) is currently being adapted to support low-latency microkernel establishment, which will remove much of the hassle of coordinating multiple Mirage 'processes'.

Another interesting performance-related aspect has been the heavy IPC workloads that result from using many VM-VM interconnects. Some early benchmarks in http://anil.recoil.org/papers/2012-resolve-fable.pdf .


The (inadequately developed) point was that this idea could be built into an entirely new operating system with e.g. a different notion of a process—as opposed to building new tools on top of Unix—and that this "operating system" could be a lightweight layer like HaLVM or Mirage, so such ideas could be explored without committing to building an entire OS.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: