Bye bye drawfcall

Devwsys no longer uses an internal Wsysmsg representation. As a matter of fact, drawfcall support has been completely removed. Getting rid of this level of indirection devwsys can become a simpler program. It will only support one interface: a 9p file system.

The drawfcall infrastructure in p9p's devdraw has been verey helpful to get started, but it was starting to get in the way. We want to run devwsys in different environments, any unnecessary complication should be avoided. Mixing the draw and wsys file systems is complex enough already.

Preview release of devwsys

A preliminary version of devwsys is publicly available:

hg clone http://bitbucket.org/yiyus/devwsys-prev/

This version does not have a functional drawing device yet, but can be used to create windows and read input events. Libixp and mk are the only external dependencies required to build. The Inferno drawing libraries are included in the repo.

Running devwsys a 9p server is started (by default in /tmp/ns.$USER.$DISPLAY/wsys). When mounted, a new window is created in the X display, and the following files can be accessed:

• winid: an unique window id.
• label: read or write it to get or set the label of the window.
• mouse: to get mouse and resize events.
• cons: to get keyboard events. Alt is the compose key.
• consctl is only a dummy file. cons is always on rawon mode.
• wsys/n/ gives access to the file system of all the open windows.
• draw/ and draw/new are not functional yet.

Devwsys still needs much more work, but is progressing. There are still many aspects to improve, and more files need to be implemented for rio(4) compatibility. The drawing device is the top priority now.

Libdrawp release

The libdrawp repository is now publicly accessible:

hg clone http://bitbucket.org/yiyus/libdrawp/

For more information about libdrawp, see this post. Let me know if you find the libraries useful or have any problem with them.

Devwsys

Devwsys will be a new program. Its main components will be some libraries from Inferno (libdraw, libmemdraw, libmemlayer and also lib9 and some pieces from other libraries), libixp and a considerable amount of p9p's devdraw code, as well as from the Inferno kernel devices.

The objective of devwsys is simple in its conception: serve a 9p file system similar to that one of rio(4), with the difference that new windows are not created in a rio instance, but in the windowing system of the host (X11). In fact, a devwsys window will be more limited than a rio window, since it will not include any text editing capabilities, they will only be graphical windows (eg. consctl will only be a dummy file, cons will always be on rawon mode).

In the future, it could be convenient to use an alternative server, for example to work with the p9p's devdraw protocol in stdin/out or to replace libixp with something else. In a similar way, although it is not part of this project, it should be possible to use other windowing systems (like OSX, Windows or, if it takes off, Wayland). In order to achieve this goal, devwsys will use the internal message representation of devdraw. 9p messages will be converted into Wsysmsg's and a runmsg function will take care of interpreting this message and perform the relevant (window system agnostic) function calls. If we want to replace libixp we will only need to build the corresponding Wsysmsg from any form the requests take and supply a reply function, and if we want to support other windowing system only the functions used by runmsg will have to be implemented.

Of course, we are not there yet, but are getting a bit closer with every line of code. A very preliminary version of devwsys will be made public at the end of this week, with the basic functionality to create new windows and some support for input events (mouse and cons files), but not drawing yet.

(Not so) portable drawing devices

The quick results obtained with the drawing libraries led me to think that it could be possible to port one of the existing devices without much work. I was wrong.

The devices in 9vx and Inferno are kernel devices and they make use of all the facilities a real kernel gives you. In particular, they use kprocs and chans, which are not obvious to get working in an Unix environment (not without having to add too much code). There is a kproc implementation based in pthreads I have not got my hands on yet, but anyway we want to stay away from threads, because multi-threaded programs and X11 do not play well together. Extracting only the useful bits from these devices is certainly doable, but not an easy job (not particularly difficult, but is not the kind of thing you can try in an afternoon).

Plan9port's device, however, is not a kernel device. It is an external program. It looks like that could make things easier but, in practice, not so much. I have tried but, after writing some sed and ed I realized the task was much larger than I had originally thought (or, should I say, hoped).

So, what is the plan for the wsys device? Writing our own. However, we have nice examples available, so it is not a rewrite from scratch, not at all. Some portions can be taken from p9p, some other portions from Inferno or 9vx. Surely some parts will have to be reworked, refactored or totally rewritten. The idea is to start with something very basic and add features one at a time, always having something we can compile and test. This way, not only we have a better control of the code, it will also help me to understand better how it works and all the involved problems. It will take some time but I hope the result will be worth all that work and, to be honest, being able to compile one of the existing devices with only a few changes would have been too good to be true.

For the moment, this was the last experiment. The following posts will discuss the development of the new device.

Portable drawing libraries

A drawing device needs some Plan 9 libraries: draw(2), memdraw(2) and memlayer(2). These libraries have been ported to Unix as part of p9p, Inferno and drawterm. Another option, used by 9vx, is to use the original Plan 9 libraries.

The C code in Plan 9 is not ANSI C, so it cannot be compiled without modification with standard compilers (although gcc has support for the extended features of Plan 9 in its latest versions). Therefore, the .c files in 9vx are not exactly the same ones that are found in /sys/src. However, the changes are few enough to be easily applied as a set of .ed files.

If you try to autogen the .c files in 9vx from current Plan 9 sources (for example, from sysfromiso) you won't get too far. Unfortunately, those scripts have not been kept up to date with the latest changes in sources. Another problem is that some of the last bugfixes have not been included in the Plan 9 distribution (yet?).

As part of the experiments performed to find a version of the drawing libraries suitable for the wsys device a repository with portable versions of libdraw, libmemdraw and libmemlayer has been set up in bitbucket. The .ed files and the autogen scripts have been updated to work with the latest versions (from the beginning of June) and other new bugfixes are applied as additional ed patches (only memdraw32bit, actually). Make, and not mk(1), is used to build the libraries.

The libdrawp repository (private for the moment) will be made public soon, just in case somebody is interested. It just needs a README and the LPL license.

However, these libraries will not be used in the wsys device. The libraries in Inferno are not going anywhere any time soon (they are needed for the native implementations and in other host environments), they are ANSI C, the build process uses mk(1) and include some bugfixes not present in Plan 9. That's what we are using for the wsys device. I don't think I'm going to bother maintaining libdrawp in the future, but it was another interesting experiment.

9win

9win was the first of a series of experiments. It consisted on adding a 9p fileserver to p9p's devdraw using libixp.

The whole file system required by a wsys device was not implemented. In fact, only the most basic functionality made it into 9win. An attach created a new window with an unique id and a mouse event caused a message to be printed (to stdout, not even using the 9p file system!). Although 9win was just  small hack, I consider the experiment quite successful: it showed that libixp was a good candidate to get the project running (certainly faster than writing our own 9p implementation), that 9p can be served and X events read without using dedicated threads (9win was not a multi-threaded program), and that p9p's devdraw design allowed to be easily extended to support multiple windows and other protocols.

A devdraw device which servers 9p is basicly what we are after, but in order to compile 9win you needed p9p, and p9p is not an acceptable dependency for 9vx, drawterm or Inferno. That's why the next series of experiments consisted on looking for a version of the libraries which was portable enough for our purposses.

GSoC2011: Unification of X11 code and wsys device

The main objective of this project is to reduce the duplication of code between the different Plan 9-like drawing devices which run on X11. Currently, there are four similar (but subtly different) implementations. Charles Forsyth (mentor of this project) made a very clear explanation of which is the current situation, I've taken the freedom to reproduce it here:

1. The existing devdraw program runs on x11 and osx. It implements the plan 9 graphics protocol, but embeds that in a novel non-9p protocol. Only p9p's programs can talk to it, through a modified libdraw.
2. Inferno and plan 9 use the original libdraw (more or less) to access draw's primitives through a name space that's implemented by draw(3), a kernel driver. draw(3) uses libmemdraw for graphics. Applications access the name space by binding a local device or mounting a remote 9p connection.
1. Plan 9's kernel driver port/devdraw.c gets */screen.c to do the work, typically in flushmemscreen (or hwdraw). 
2. Inferno's kernel driver uses x11, osx, windows and native variants of win.c or screen.c (roughly speaking), with most of the work done in attachscreen and flushmemscreen.
   
3. Drawterm and 9vx implement draw(3), drawing using a libmemdraw that devolves a lot of the work to special x11 and osx variants, and using a few geometry primitives from libdraw. 9vx applications are just plan 9 applications, and they use plan 9's libdraw.

Except for minor differences that can get in the way in practice, Inferno, Plan 9, and 9vx can all draw on each other's screens. They can also all draw on a drawterm screen (again, if not prevented from doing that by minor differences).

Inferno and drawterm are the only ones that run on Windows. The others, except for Plan 9, run on Unix-y systems like Linux, and on MacOSX.

Only p9p programs can talk to the devdraw program directly, and (I think this is right) they can't talk to any of the other systems directly. Generally, p9p is the odd one out. On the other hand, p9p uses a separate program to do the graphics, and that has advantages in hosted environments.

In order to improve this state of things, a new wsys device will be written. This device will be a unix program which uses xlib and serves 9p. Being a 9p server, it will be possible to mount it from Inferno, 9vx and drawterm without any modificaions. It is still to be decided how the new device will be used from p9p.