Irritation-driven-development

Back in the past I wrote a couple of Java apps. These were written using the Java Swing framework, and have worked nicely on both Windows and Linux X11 desktops for a long time.

More recently I switched away from X11 & Budgie to pure Wayland for my, desktop on the assumption that it’s over 10 years old now, and is the default technology underlying current Gnome and KDE desktops.. everything will be fine right? Kind of..

It turns out Java GUI technology has been somewhat abandoned by Oracle, in particular when running Swing (or JavaFX) applications on Wayland, Java relies on the XWayland adpater to bodge X11 API calls back to Wayland proper. This is not great (read: doesn’t work out of the box), so I wondered if someone had implemented a native Wayland backend for Swing - and they have, multiple times - super, but wait.. Oracle have not merged either project into mainline Openjdk, and indeed the only way to make either of them work is by hideous bodging, which also puts paid to any thoughts of writing my own backend, as the hacks required to load it at run time (read: in every application) are not sustainable. Grrr! So what to do?

I can of course switch back to an X11 desktop (ie: Budgie), but this irritates me, plus I have already ported OpenSceneGraph + Flightgear to Wayland, it wasn’t that difficult :)

Swing, like most medium-large frameworks has documented public API and internal implementations, so I could choose to re-write Swing, retaining the public API (thus existing applications can re-compile against my library) but rewrite Swing internals from the ground up on top of Wayland APIs. Sounds sensible right? However Swing (and the earlier AWT it relies on) have several hundred documented classes, with many more undocumented internal classes and a fair amount of native code (in C) to connect to X11, Win32 or Cocoa (macOS), plus there are no Java bindings to Wayland, only the libwayland C API. How brave do I feel?

Looking at my existing apps (see above), I have only used a small number of Swing features, so I should be able to create a spike that supports just the features I need to prove my proposal.. still seems pretty mad, but I have time (the glory of retirement!), so let’s go!

Wayland API in (mostly) Java

Given I will be using most of the Wayland protocol to achieve anything (it’s minimal) and Wayland is well specified then the ‘start at the bottom and build up’ design pattern fits. Wayland has a wire protocol based on a Unix socket, and usefully Java has supported Unix sockets since release 16, so I can write everything in Java to (de)serialise messages. This gets me going quickly, providing positive feedback that I’m on the right track..

Unfortunately Wayland also uses two Unix features that Java doesn’t have: SYS V shared memory buffers; file descriptor passing over Unix sockets. I then discover a bug (or misunderstand the documentation!)..

Shared memory solution

Helpfully Java Native Interface (JNI) does allow memory buffers to be passed from native code back to Java, and vice versa using DirectByteBuffer objects - so I only have to write enough JNI/C code to allocate and destroy shared memory, which I can then efficiently access from the Java code.

Passing file descriptors

This is a less obvious solution, since Java does not provide public access to the native file descritor(s) for any operating system APIs, however the source code is available identifying where a socket stores this native value (which appears to be stable over many releases), and from JNI/C I can easily access the required object field without the security challenges of doing so in managed Java code. I’ll live with this slightly hacky solution for now (until Java has accessors for native descriptors, it’s been proposed a few times!)

Checking for input

Wayland operates asynchronously between client and server, there are only a couple of occasions when a client needs to wait for input and we can simply block in a read call, at other times we need to poll for input using a check for data being available to read. This should work via the java.nio.channels.Selector mechanism, but sadly it does not. Thus I extend my JNI/C a little to provide a working data availability check, using the above hack to obtain the native file descriptor and calling ioctl(FIONREAD), which does work!

Testing ensues..

As soon as I have enough of the protocol implemented to connect and display something, then testing ensues - gotta see something on screen :) The test program will now evolve along with the protocol components as I add features. Oh, and it works nicely - this is 11 days in!

Swing API in many steps

With the underlying Wayland protocol up and running, I move on to a ‘baby steps’ exploration of the Swing API, working down from one of my simple applications (the powermonitor) and creating ‘just enough code to pass the tests’. This takes considerably longer than I hoped, as Swing is rather tightly coupled, and small refactorings (such as hoisting state to a super class) result in a lot of changes to coupled code - bah!

Along side the real application, I create a Swing test application which provides scope for short cuts and internal testing during development without hacking my real app about (always good to have something you can run after each change!)

After another 5 days work, I get to see the first Swing app run, with almost no useful features, but API compatible with the real thing. Neat!

Features, then more features

So from here I start picking off features to add and working through the stack to ‘make it so’ for each: first up a single 8x16 MONOSPACED font that I steal borrow from The Ultimate Old Skool PC Font Pack, the original IBM VGA text mode font. Retro computing FTW.

This is followed up with input handling: first keyboard then mouse, both rather buggy; then more high level components (Dialog, JLabel, JButton) over the next few days, followed by improvements to Graphics primitives.. and lots of debugging, particularly around the interaction with my tiling window manager (sway) and how it chooses to position new windows.

Cursor support bogs me down for a bit, especially the interaction between an asynchronous protocol, and the need to change the cursor on demand while the pointer is within a single Wayland window, also the increasingly complex logic required to flow input events through the component stack, which I refactor a couple of times.

Help arrives!

About a month into the project, my eldest son arrives for a family visit, and gets interested in the madness.. he pulls the repository, builds the code and it runs first time, on his Pinebook, which is both an ARM (not x86) and big-endian by default - I’m almost impressed this just works!

He also find and fixes a couple of protocol bugs for me - thanks Martin!

Dogfooding time

At this point I decide to write an app that both uses Swingland and supports it, a font editor for the custom file format I have created to store simple bitmap fonts - this allows me to create a proper cursor font without hand-editing in a hex editor (which I have been doing until now!)

The process of creating this real application finds more bugs (of course!)..

Image support

The next feature is the ImageIO library that allows bitmap images to be loaded. Because I have contributed to it and I think it’s cool, I start with the Quite OK Image format, then Portable Network Graphics as most images will be in PNGs. Transparency and alpha blending follow as both image formats support alpha channels.

Window manager interaction

Switching an application to full screen (and back) under program control is explicitly supported in Swing, and useful for games or similar, so this goes in next, and requires yet another refactor of the way Swing interacts with Wayland

Menus!

At this point we’re approaching the 80% useful mark of the Pareto Principle, Swingland is looking like an actually useful libray (tm), although it really needs application menus.. cue another round of refactoring, popup window handling and event processing re-design, and we have it! Looking good:

Getting braver with testing

Having exhausted my own applications as test beds, I decide to move on to the Oracle-provided Swing demo apps, in particular the TopLevelDemo and ButtonDemo ones that should now work with a simple change of imports and a re-compile.. well almost ;) More debugging and fiddling with the detail of button processing ensues.

More fonts..

The latest feature to go in is support for X11/PCF font files, which opens up a world of reasonable fonts for text. I had looked at TrueType (TTF) fonts earlier and decided that they were far too much work for now (a whole virtual machine and binary executable format to just fiddle with point positions during layout.. err nope!).

Ongoing development

I hope to maintain Swingland as an actually useful library for some time, and would particularly welcome other contributors who need other Swing features that they could develop. Come get some hot source