More recently I switched away from X11 & Budgie to pure Wayland for my, desktop on the assumption that it’s over 10 years old now, and is the default technology underlying current Gnome and KDE desktops.. everything will be fine right? Kind of..
It turns out Java GUI technology has been somewhat abandoned by Oracle, in particular when running Swing (or JavaFX) applications on Wayland,
Java relies on the
XWayland adpater to bodge X11 API calls back to Wayland proper. This is not great (read: doesn’t work out of the box), so
I wondered if someone had implemented a native Wayland backend for Swing -
and they have,
multiple times - super, but wait.. Oracle have not merged either project into mainline Openjdk,
and indeed the only way to make either of them work is by hideous bodging, which also puts paid to any thoughts of writing my own backend, as the hacks
required to load it at run time (read: in every application) are not sustainable. Grrr! So what to do?
I can of course switch back to an X11 desktop (ie: Budgie), but this irritates me, plus I have already ported OpenSceneGraph + Flightgear to Wayland, it wasn’t that difficult :)
Swing, like most medium-large frameworks has documented public API and internal implementations, so I could choose to re-write Swing, retaining the
public API (thus existing applications can re-compile against my library) but rewrite Swing internals from the ground up on top of Wayland APIs.
Sounds sensible right? However Swing (and the earlier AWT it relies on) have
several hundred documented classes, with many more undocumented
internal classes and a fair amount of native code (in C) to connect to X11, Win32 or Cocoa (macOS), plus there are no Java bindings to Wayland, only
libwayland C API. How brave do I feel?
Looking at my existing apps (see above), I have only used a small number of Swing features, so I should be able to create a spike that supports just the features I need to prove my proposal.. still seems pretty mad, but I have time (the glory of retirement!), so let’s go!
Wayland API in (mostly) Java
Given I will be using most of the Wayland protocol to achieve anything (it’s minimal) and Wayland is well specified
then the ‘start at the bottom and build up’ design pattern fits. Wayland has a
wire protocol based on a Unix socket, and usefully Java has
supported Unix sockets since release 16, so I can write everything in Java to (de)serialise messages. This gets me going quickly, providing
positive feedback that I’m on the right track..
Unfortunately Wayland also uses two Unix features that Java doesn’t have: SYS V shared memory buffers; file descriptor passing over Unix sockets. I then discover a bug (or misunderstand the documentation!)..
Shared memory solution
Helpfully Java Native Interface (JNI) does allow memory buffers to be passed from native
code back to Java, and vice versa using
DirectByteBuffer objects - so I only have to write enough JNI/C code to allocate and destroy shared memory,
which I can then efficiently access from the Java code.
Passing file descriptors
This is a less obvious solution, since Java does not provide public access to the native file descritor(s) for any operating system APIs, however the source code is available identifying where a socket stores this native value (which appears to be stable over many releases), and from JNI/C I can easily access the required object field without the security challenges of doing so in managed Java code. I’ll live with this slightly hacky solution for now (until Java has accessors for native descriptors, it’s been proposed a few times!)
Checking for input
Wayland operates asynchronously between client and server, there are only a couple of occasions when a client needs to wait for input and we can
simply block in a read call, at other times we need to poll for input using a check for data being available to read. This should work
java.nio.channels.Selector mechanism, but sadly it does not. Thus I extend my JNI/C a little to provide a working data availability check,
using the above hack to obtain the native file descriptor and calling
ioctl(FIONREAD), which does work!
As soon as I have enough of the protocol implemented to connect and display something, then testing ensues - gotta see something on screen :) The test program will now evolve along with the protocol components as I add features. Oh, and it works nicely - this is 11 days in!
Swing API in many steps
With the underlying Wayland protocol up and running, I move on to a ‘baby steps’ exploration of the Swing API, working down from one of my simple applications (the powermonitor) and creating ‘just enough code to pass the tests’. This takes considerably longer than I hoped, as Swing is rather tightly coupled, and small refactorings (such as hoisting state to a super class) result in a lot of changes to coupled code - bah!
Along side the real application, I create a Swing test application which provides scope for short cuts and internal testing during development without hacking my real app about (always good to have something you can run after each change!)
After another 5 days work, I get to see the first Swing app run, with almost no useful features, but API compatible with the real thing. Neat!
Features, then more features
So from here I start picking off features to add and working through the stack to ‘make it so’ for each: first up a single 8x16
steal borrow from The Ultimate Old Skool PC Font Pack, the original IBM VGA text mode
font. Retro computing FTW.
This is followed up with input handling: first keyboard then mouse, both rather buggy; then more high level components (
over the next few days, followed by improvements to
Graphics primitives.. and lots of debugging, particularly around the interaction with my
tiling window manager (
sway) and how it chooses to position new windows.
Cursor support bogs me down for a bit, especially the interaction between an asynchronous protocol, and the need to change the cursor on demand while the pointer is within a single Wayland window, also the increasingly complex logic required to flow input events through the component stack, which I refactor a couple of times.
About a month into the project, my eldest son arrives for a family visit, and gets interested in the madness.. he pulls the repository, builds the code and it runs first time, on his Pinebook, which is both an ARM (not x86) and big-endian by default - I’m almost impressed this just works!
He also find and fixes a couple of protocol bugs for me - thanks Martin!
At this point I decide to write an app that both uses Swingland and supports it, a font editor for the custom file format I have created to store simple bitmap fonts - this allows me to create a proper cursor font without hand-editing in a hex editor (which I have been doing until now!)
The process of creating this real application finds more bugs (of course!)..
The next feature is the
ImageIO library that allows bitmap images to be loaded. Because I have contributed to it and I think it’s cool, I start
with the Quite OK Image format, then Portable Network Graphics as most images will
be in PNGs. Transparency and alpha blending follow as both image formats support alpha channels.
Window manager interaction
Switching an application to full screen (and back) under program control is explicitly supported in Swing, and useful for games or similar, so this goes in next, and requires yet another refactor of the way Swing interacts with Wayland
At this point we’re approaching the 80% useful mark of the Pareto Principle, Swingland is looking like an actually useful libray (tm), although it really needs application menus.. cue another round of refactoring, popup window handling and event processing re-design, and we have it! Looking good:
Getting braver with testing
Having exhausted my own applications as test beds, I decide to move on to the Oracle-provided Swing demo apps, in particular the
ButtonDemo ones that should now work with a simple change of imports and a re-compile.. well almost ;) More debugging and fiddling with the detail
of button processing ensues.
The latest feature to go in is support for X11/PCF font files, which opens up a world of reasonable fonts for text. I had looked at TrueType (TTF) fonts earlier and decided that they were far too much work for now (a whole virtual machine and binary executable format to just fiddle with point positions during layout.. err nope!).
I hope to maintain Swingland as an actually useful library for some time, and would particularly welcome other contributors who need other Swing features that they could develop. Come get some hot source