Running Wine in a 32-bit sandbox on 64-bit NetBSD

Disclaimer: this is completely mad science, and mostly a learning exercise.

I recently updated the Wine package to a recent stable version, and now want to play some old Windows games on NetBSD.

It's current year, my machine runs amd64, and Wine needs to be built with 32-bit libraries to run a lot of older Windows applications.

"Mainline pkgsrc" can't do strange multi-arch Wine builds yet, so a 32-bit sandbox seems like a reasonable way to use 32-bit Wine on amd64 without resorting to running real Windows in NVMM. We'll see if this was a viable alternative to re-reviewing the multi-arch support in pkgsrc-wip...

We're using sandboxctl, which is a neat tool for quickly shelling into a different NetBSD userspace. Maybe you also don't trust the Windows applications you're running too much - sandboxctl creates a chroot based on a fresh system image, and chroot on NetBSD is fairly bombproof.

I learned a little bit more about how X11 works while trying this, too, which was nice!


First, install the sandboxctl package:

# pkgin install sandboxctl

You'll need to enable some sysctl variables.

This one will allow multiple system users to play audio, and will save you some debugging if you're wondering why things are mysteriously silent:

# sysctl -w hw.audio0.multiuser=1

This one will allow Wine to map the null page (which it needs to do for cursed reasons):

# sysctl -w vm.user_va0_disable=0

Add them to /etc/sysctl.conf to make them permanent.

Configuring sandboxctl

Download some i386 NetBSD sets into a directory:

$ mkdir -p ~/netbsd-i386/binary/sets && cd ~/netbsd-i386/binary/sets
$ ftp
$ ftp
$ ftp
$ ftp
$ ftp

Create a new config file for sandboxctl in /usr/pkg/etc/sandboxctl/wine.conf:


NETBSD_RELEASE_SETS="base etc xbase xetc xfont"

Let's create the sandbox and Wine user:

# sandboxctl -c wine create
# sandboxctl -c wine run useradd -m -d /home/wine wine

Kernel config abuse

Wine needs to abuse the system, because it needs to run Windows applications. You'll need to reconfigure NetBSD to allow this abuse, because by default NetBSD is fairly strict.

Running NetBSD 10 or -current? Then you don't need to do this.

Running NetBSD 9.x? Make sure your kernel has the necessary security features disabled to run Wine. You'll need fresh netbsd-9 sources.

$ vi sys/arch/amd64/conf/GENERIC

These options matter:

# USER_LDT. You need to disable SVS to use it.
options 	USER_LDT	# user-settable LDT; used by WINE
no options	SVS

Build a new kernel. Shouldn't take too long :D

$ ./ -U -j4 tools
$ ./ -U -j4 kernel=GENERIC

Back up the old kernel, copy the new one into place and reboot:

# mv /netbsd /onetbsd
# cp sys/arch/amd64/compile/obj/GENERIC/netbsd /netbsd
# shutdown -r now

Making X11 play nicely with the sandbox

We need some way to allow X11 applications inside the sandbox to communicate with the X server.

Attempt 1 (X11-over-TCP)

This is required if /tmp is not on the same filesystem as the sandbox. It took me far too long to realize this.

Start the X server on the host machine with TCP connections and indirect GLX enabled:

$ startx -- -listen tcp +iglx

Once you're in X11, disable access control, and hopefully you're either behind a firewall or on a trusted network...

$ xhost +

Attempt 2 (/tmp on the same filesystem as the sandbox)

I gave up with TCP and removed my tmpfs mount in /etc/fstab so that /tmp could reside on the same filesystem as the sandbox. I actually don't like this at all.

This allows us to use hard links to the X11 Unix socket once X is started:

# mkdir -m 777 -p /var/chroot/wine-i386/tmp/.X11-unix
# ln -f /tmp/.X11-unix/X0 /var/chroot/wine-i386/tmp/.X11-unix/X0
# chmod 777 /var/chroot/wine-i386/tmp/.X11-unix/X0

Then use xauth to grant access to your user's X server to the sandbox:

$ xauth extract /var/chroot/wine-i386/home/wine/.Xauthority :0


I quickly discovered Direct Rendering from a 32-bit sandbox isn't all that viable. In the sense that glxgears immediately segfaults, regardless of whether the connection to X11 is over TCP or Unix socket. I honestly wonder why, but debugging this further is probably too much pain for today.



(Makes OpenGL use llvmpipe instead of the GPU, might be slow depending on the age of the software and your hardware. You do not need +iglx for this. llvmpipe is surprisingly good even on my 5 year old CPU.)



(Seems to work briefly, but then my X server crashes - ymmv.)

The Xserver(1) man page has some interesting things to say about +iglx and LIBGL_ALWAYS_INDIRECT=1...

Indirect GLX is of limited use, since it lacks support for many modern OpenGL features and extensions; it's slower than direct contexts; and it opens a large attack surface for protocol parsing errors.


Using your sandbox

Shell into your sandbox:

# sandboxctl -c wine shell

Install the Wine package, and any others you desire in your 32-bit sandbox:

# export PKG_PATH=
# pkg_add pkgin
# pkgin update
# pkgin install wine

(I prefer to always use pkgin even for simple cases like this, it's just better.)

su into your user account:

# su -l wine

From outside the sandbox, copy some files, maybe. In this case, I'm copying the DRM-free version of Red Faction from

# cp ~/Downloads/setup_red_faction_2.0.0.7.exe /var/chroot/wine-i386/home/wine/

Inside the sandbox, I can now run my 32-bit executable:

$ export DISPLAY=:0
$ winecfg
$ wine setup_red_faction_2.0.0.7.exe

The Red Faction installer

When you're done, exit the sandbox with ^D.



Making sure the sandbox user has the same uid as your normal user account probably helps with permissions nonsense.

In my case, it was the first ordinary user created on both systems, so that was automatic.