You can now testdrive streamline without compilation. I created a vmware virtual machine image that contains the latest streamline (SVN 730). Download the compressed archive ubuntu-sl-20100120.tbz2 (Warning: 1.1 GB).

The tarball expands to the preconfigured .vmx configuration file and a 4GB .vmdk disk image that contains a small Ubuntu 9.04 distribution with Linux 2.6.24.2 kernel. I chose this old configuration, because it is the basis for evaluation in the main streamline paper.

Anonymously subversion checkout is now possible. Nightly tarballs are no longer supported (or needed). Thanks to Philippe for pointing out that this was broken up until now.

After 18 months of silence, today we released version 1.8.1. This is primarily a maintenance update, comprising tens of bugfixes and usability updates. New major features include filter access control to allow unprivileged users to safely run fast IO code in the kernel (think BPF) and an optimizer that uses linear programming to map IO tasks onto hardware.

Read the README for important notes on system support (at least up to Ubuntu 10.04 and Linux kernel 2.6.31) and known issues. One particularly irritating cleanup bug persists in this release. I postponed rolling a release to fix squash this bug, but after 18 months have to admit that I lack the time at the moment.

The PhD thesis based on Streamline received the 2010 Roger Needham PhD Prize, an annual award for a thesis that "is regarded to be an exceptional, innovative contribution to knowledge in the systems area".

The award is established by Eurosys, the European ACM SIGOPS chapter, and donated by Microsoft Research Cambridge.

We've been silent as the project went into a period of maintenance mode. Work has continued over the summer, however. At the moment I suggest using the SVN tarball over the latest release. Send me an email on the mailing list if you want me to roll a new release. Also, I'm happy to say that development is ramping up again as new users are picking up the code.

Update: the code has seen quite some updates since 1.8.0.2, thanks mainly to Reggie Cushing and Mihai Cristea. The note on release on demand still holds.

The new release is mainly a bugfix update. It has been verified to execute all the examples correctly. Also, it adds deflate/inflate filters and allows simpler expressions: "a --expression=$EXPR" can now be written as "a $EXPR".

As I ran the 1.8.0.2 version on a computer I hadn't tried before I found some compilation issues. The system ran Debian Etch and may affect more Debian users. Note that I develop on a Ubuntu system, so users of that distribution should not see these issues.

In the left menu you will find a new item called videos. Here I will upload screen captures of Streamline in action. I've uploaded the first two, which show only very basic use. Also, I put a new set of slides online that talks about Streamline.

The examples have been updated to work with the latest version (1.8.0.2) of Streamline.

I just released the first update to the 1.8 line, which fixes some issues in the new grammar and improves Unix process handling. Specifically, it resolves the bug where Unix processes could not be called with parameters (a fairly serious one).

If you want help or have feedback, be sure to send us a message on the mailing list. Just remember to register as user first, otherwise your message will end up in my spambox.

Today we release a new version that breaks interface compatibility with the 1.7 line. Specifically, it replaces the request language with one that is a superset of Unix pipelines and it gives the buffer API a (hopefully) unique prefix, so that the calls do not clash with application symbols.

Functionally, 1.8.0 is similar to 1.7.4.5. The only change is that it builds on kernel 2.6.27. The new interface has one known bug: even though Unix processes can be integrated into streamline pipelines, parameter passing confuses the parser. I'll roll an update that fixes this issue shortly.

Note that because the interface is new, this release not as stable as the last of the 1.7 line.

The Team from the University of Amsterdam Interactive Networks research project have put the movies of their demo online. Interactive Networks uses Streamline as the reconfigurable component. See the videos here.

Last few days the site suffered from some hosting problems. My previous provider closed shop, which forced me to move both DNS and eventually hosting. Things should be fine from now on.

Streamline is currently being demoed at SC'08 as building block for the Interactive Networks project of the University of Amsterdam.

This again is mainly a bug-fix release, specifically for PipesFS. It also adds an interface for runtime modification of active filters.

Whoops. The latest release fixed regressions in sockets and pipes, but I apparently failed to test PipesFS. The new, more efficient, update mechanism proved so efficient that it never wakes up processes listening on PipesFS nodes. I will fix this in the next version (and start saving for an automated test setup). For now, please use the previous version 1.7.4.3.

This is mainly a bug-fix release. It bumps kernel support up to 2.6.26, fixes regressions down to 2.6.19, reduces signalling overhead (increasing small packet performance) and fixes a host of issues that resulted from switching to blocking as default stream behavior in 1.7.4. The socket, pipe and pcap interfaces have all been verified to work.

I updated the overview page and added an architecture picture, to (try to) explain the relationship between Streamline, PipesFS and Beltway Buffers.

Documentation is lacking, especially for the new PipesFS interface. If you came here after reading the OS Review paper looking for more details, please have a look at the readme file in the SVN repository for now. Or, read the sourcecode itself. If you still have questions, contact us through the forum or mailing list.

Yesterday I spent an afternoon fixing build issues with Linux 2.6.26. Linux kernel interfaces had changed quite a bit since my last kernel (2.6.24.2). Read the svn changelog for more details. I have not explicitly tested 2.6.25. Here, YMMV.

This version fixes the bug discussed below. No new Ubuntu packages are needed, as they do not use Streamline's installation process.

In all recent versions, make install and make uninstall can lead to severe data loss if ./configure is passed parameters. For now, do not pass parameters to the script or do not install. I'll release a safe update ASAP.

Today we released a major functional update of Streamline. Version 1.7.4 adds PipesFS: a Linux virtual filesystem for I/O. PipesFS presents kernel I/O operations as directories and exports live streams through Unix pipes. The FS allows users to quickly construct kernel tasks using the 40+ Streamline operations using mkdir, ln, etc. and to interact with kernel I/O using cat, grep, gzip, etc.

1.7.4 also updates our shared-memory version of Posix I/O: Beltway Buffers. Version 1.7.4 was used for application benchmarks (bind9, mplayer, tcpdump) and increases performance and Posix compliance over 1.7.3

Also new is support for Sun RPC (beta).

Finally, this is the first version that ships with precompiled distro packages. Download the sourcecode,
the i386 .deb file (for Ubuntu 8.04, kernel 2.6.24-19-virtual) or
the amd64 .deb file (for Ubuntu 8.04, kernel 2.6.24-19-generic).

Since the last update I have benchmarked (and optimized) Streamline and implemented a new interface, Pipesfs, that fits in better with existing tools because it exports I/O through a virtual filesystem. As always, the bleeding edge is available from our subversion repository. I also plan to roll a new release within the week.

We were receiving too much spam on the mailing list. From now only emails from registered members will get through. Sorry for the inconvenience.

As alternative, I've reinstated forums for technical support.

With RC2, Streamline 1.7.3 binaries have been added to the Catapulta LiveCD. Catapulta is a Ubuntu derived distribution that bundles network-monitoring applications. It is the easiest and safest way to try out Streamline (and other tools, such as PF_RING).

Based on the .deb ubuntu packages created by the Catapulta team we will soon start making binary Streamline versions available for vanilla Ubuntu as well.

On March 25th we released version 1.7.3 of Streamline. Download the distribution or read the release notes first.

February 25th, 2008

...

(older stories went missing ...)