v86/docs/archlinux.md

(This document partly also applies to other Linuxes)

Choosing an installer ISO

The last ISO installer version of Archlinux that supports 32-bit is 2017.02.01. Later versions of the archisos don't work on the v86 emulator because the installer only supports x86_64, not x86 anymore. For existing Archlinux installations, updates and patches will be done until somewhere around 2018.

In the future the community might come up with an alternative distribution based on Archlinux to maintain support for x86. At this point in time archlinux32 seems to work.

Basic installation using QEMU

Installing Archlinux like this will result in a raw disk image that can be booted by v86.

# Create a 10 gigabyte disk image. If you intend to pacstrap only 'base' then 1.5G should be fine also.
qemu-img create arch.img 10G

# Follow the normal installation process (you can add accel=kvm if your system supports it to speed up the installation)
qemu-system-x86_64 -m 256 -drive file=arch.img,format=raw -cdrom archlinux-2017.02.01-dual.iso

For keyboard support it is necessary to open /etc/mkinitcpio.conf and edit the following line:

MODULES="atkbd i8042"

For the changes to take effect you need to regenerate the RAMdisk with mkinitcpio -p linux

The resulting arch.img is a bootable disk image for v86.

Scripting image creation for v86

Installing the ISO by hand takes a long time if you intend to recreate the image many times. There are various reasons why you might want to do this more than once. For example: because the emulator is slow you might want to compile any new software release in QEMU which is much faster and then use the resulting image in v86 instead of making the emulator compile the software. Another reason is that the build progress potentially takes long and if you want to do automated builds in parallel to find out what configurations do and don't work you can just throw more computing power at the problem in order to solve it. This example requires that you have packer, qemu and kpartx installed.

Creating a packer template

Packer is a tool that lets you boot an ISO in any of multiple emulators (so QEMU in our case) and send pre-scripted keystrokes to bootstrap and SSH server. Once the SSH connection is established a script can be started for further provisioning.

Create a template for automating the base installation

mkdir -p packer
cat > packer/template.json << 'EOF'
{
  "provisioners": [
    {
      "type": "shell",
      "override": {
        "qemu": {
          "scripts": ["scripts/provision.sh"]
        }
      }
    }
  ],
  "builders": [
    {
      "type": "qemu",
      "boot_command": [
        "<enter><wait10>",
        "dhcpcd<enter><wait10>",
	"usermod --password $(echo toor | openssl passwd -1 -stdin) root<enter><wait10>",
        "systemctl start sshd<enter>"
      ],
      "headless": true,
      "boot_wait": "10s",
      "disk_size": 1500,
      "disk_interface": "ide",
      "http_directory": "http",
      "iso_url": "https://mirror.archlinux32.org/archisos/archlinux-2017.04.01-i686.iso",
      "iso_checksum": "aa4718837c95e607233aecca43824c08c798b1a4",
      "iso_checksum_type": "sha1",
      "ssh_wait_timeout": "120s",
      "ssh_pty": true,
      "ssh_username": "root",
      "ssh_password": "toor",
      "ssh_port": 22,
      "format": "raw",
      "vm_name": "Archlinux-v86"
    }
  ]
}
EOF

You can tweak the options a bit to match your situation. For debugging you can set headless to false. That will show you the vnc instead of running the boot_command in the background. For a base pacstrap using a 1.5G disk should be sufficient. The raw disk format is important. v86 does not read qcow2 images, only raw disk images. If your system does not support kvm (the default accelerator), you can add "accelerator": "none" to the settings. Other accelerator options can be found here.

After gaining SSH connectivity to the VM, packer will run the scripts/provisioning.sh script in the guest.

Creating the Archlinux installation script

Create a script for your Archlinux installation. This runs in the ISO booted Archlinux environment, so you need to partition, pacstrap and install a bootloader.

mkdir -p packer/scripts
### Write your own or copy paste the example below
vim packer/scripts/provision.sh

An example script to install Archlinux with the root mounted using the 9p network filesystem:

#!/bin/bash
echo "Creating a GPT partition on /dev/sda1"
echo -e "g\nn\n\n\n\nw" | fdisk /dev/sda

# In case you might want to create a DOS partition instead. It doesn't really matter.
#echo "Creating a DOS partition on /dev/sda1"
#echo -e "o\nn\np\n1\n\n\nw" | fdisk /dev/sda

echo "Formatting /dev/sda1 to ext4"
mkfs -t ext4 /dev/sda1

echo "Mounting new filesystem"
mount -t ext4 /dev/sda1 /mnt

echo "Create pacman package cache dir"
mkdir -p /mnt/var/cache/pacman/pkg

# We don't want the pacman cache to fill up the image. After reboot whatever tarballs pacman has cached are gone.
echo "Mount the package cache dir in memory so it doesn't fill up the image"
mount -t tmpfs none /mnt/var/cache/pacman/pkg

# Install the Archlinux base system
echo "Performing pacstrap"
pacstrap -i /mnt base --noconfirm

echo "Writing fstab"
genfstab -p /mnt >> /mnt/etc/fstab

# When the Linux boots we want it to automatically log in on tty1 as root
echo "Ensuring root autologin on tty1"
mkdir -p /mnt/etc/systemd/system/getty@tty1.service.d
cat << 'EOF' > /mnt/etc/systemd/system/getty@tty1.service.d/override.conf
[Service]
ExecStart=
ExecStart=-/usr/bin/agetty --autologin root --noclear %I $TERM
EOF

# This is the tricky part. The current root will be mounted on /dev/sda1 but after we reboot 
# it will try to mount root during boot using the 9p network filesystem. This means the emulator
# will request all files over the network using XMLHttpRequests from the server. This is great 
# because then you only need to provide the client with a saved state (the memory) and the 
# session will start instantly and load needed files on the fly. This is fast and it saves bandwidth.
echo "Ensuring root is remounted using 9p after reboot"
mkdir -p /mnt/etc/initcpio/hooks
cat << 'EOF' > /mnt/etc/initcpio/hooks/9p_root
run_hook() {
    mount_handler="mount_9p_root"
}

mount_9p_root() {
    msg ":: mounting '$root' on real root (9p)"
    # Note the host9p. We won't mount /dev/sda1 on root anymore, 
    # instead we mount the network filesystem and the emulator will
    # retrieve the files on the fly.
    if ! mount -t 9p host9p "$1"; then
        echo "You are now being dropped into an emergency shell."
        launch_interactive_shell
        msg "Trying to continue (this will most likely fail) ..."
    fi
}
EOF

echo "Adding initcpio build hook for 9p root remount"
mkdir -p /mnt/etc/initcpio/install
cat << 'EOF' > /mnt/etc/initcpio/install/9p_root
#!/bin/bash
build() {
	add_runscript
}
EOF

# We need to load some modules into the kernel for it to play nice with the emulator
# The atkbd and i8042 modules are for keyboard input in the browser. If you do not
# want to use the network filesystem you only need these. The 9p, 9pnet and 9pnet_virtio
# modules are needed for being able to mount 9p network filesystems using the emulator.
echo "Configure mkinitcpio for 9p"
sed -i 's/MODULES=""/MODULES="atkbd i8042 virtio_pci 9p 9pnet 9pnet_virtio"/g' /mnt/etc/mkinitcpio.conf

# Because we want to mount the root filesystem over the network during boot, we need to 
# hook into initcpio. If you do not want to mount the root filesystem during boot but
# only want to mount a 9p filesystem later, you can leave this out. Once the system
# has been booted you should be able to mount 9p filesystems with mount -t 9p host9p /blabla
# without this hook.
sed -i 's/fsck"/fsck 9p_root"/g' /mnt/etc/mkinitcpio.conf

echo "Writing the installation script"
cat << 'EOF' > /mnt/bootstrap.sh
#!/usr/bin/bash
echo "Re-generate initial ramdisk environment"
mkinitcpio -p linux

echo "Installing the grub package"
pacman -S os-prober grub --noconfirm

echo "Setting grub timeout to 0 seconds"
sed -i 's/GRUB_TIMEOUT=5/GRUB_TIMEOUT=0/g' /etc/default/grub

echo "Installing bootloader"
grub-install --target=i386-pc --recheck /dev/sda --force

echo "Writing grub config"
grub-mkconfig -o /boot/grub/grub.cfg
sync
EOF

echo "Chrooting and bootstrapping the installation"
arch-chroot /mnt bash bootstrap.sh

umount -R /mnt

With the packer template and the script you have enough to create an image that can be booted by v86. But because this example script installs an Archlinux that wants to mount root over the network with 9p, we need to host that filesystem first. If you do not want to use 9p, you can just run (cd packer && packer build -force template.json) to build the image.

Creating the 9p filesystem

Now that we have an image that contains a filesystem, we can convert that filesystem into something we can host on the webserver together with the v86 library.

To do so, we need to mount the image once and create a json mapping of the filesystem. This can be done with the fs2json python script.

Get the script:

wget https://raw.githubusercontent.com/copy/fs2json/master/fs2json.py

The following script shows how to map the filesystem in an automated fashion.

Create a script to builds the image and then creates v86 compatible artifacts:

vim build.sh

Example script:

#!/bin/sh

# build the boxfile from the iso
(cd packer && packer build -force template.json)

# test if there is a boxfile where we expected it
if [ ! -f packer/output-qemu/Archlinux-v86 ]; then
    echo "Looks like something went wrong building the image, maybe try again?"
    exit 1
fi;

# clean up any previous loops and mounts
echo "Making sure mountpoint is empty"
sudo umount diskmount -f || /bin/true
sudo kpartx -d /dev/loop0 || /bin/true
sudo losetup -d /dev/loop0 || /bin/true

# mount the generated raw image, we do that so we can create 
# a json mapping of it and copy it to host on the webserver
mkdir -p diskmount
echo "Mounting the created image so we can convert it to a p9 image"
sudo losetup /dev/loop0 packer/output-qemu/Archlinux-v86
sudo kpartx -a /dev/loop0
sudo mount /dev/mapper/loop0p1 diskmount

# make images dir
mkdir -p output/images

# map the filesystem to json with fs2json
sudo python fs2json.py --exclude /boot/ --out output/images/fs.json diskmount

# copy the filesystem and chown to nonroot user
echo "Copying the filesystem to output/arch"
mkdir output/arch -p
sudo rsync -q -av diskmount/ output/arch
sudo chown -R $(whoami):$(whoami) output/arch

# clean up mount
echo "Cleaning up mounts"
sudo umount diskmount -f
sudo kpartx -d /dev/loop0
sudo losetup -d /dev/loop0

# Move the image to the images dir
mv packer/output-qemu/Archlinux-v86 output/images/arch.img

An example repository with these scripts can be found here

Using the created artifacts in v86

Now that we have everything we need to host a server that serves an Archlinux environment over the network.

Create a checkout of v86 and run make build/libv86.js. Then edit examples/arch.html and replace the hda and filesystem section with something like this:

hda: {
    url: "http://localhost:8000/images/arch.img",
    async: true,  # For this option we need to run a webserver that supports the Range header
    size: 1.5 * 1024 * 1024 * 1024,  # This needs to be the size of the raw disk. 
    # See the `disk_size` item in the packer template.
},
filesystem: {
    baseurl: "http://localhost:8000/arch/",
    basefs: "http://localhost:8000/images/fs.json",
},

Then copy the created image, filesystem and json file to the v86 directory:

cp -f /your/packer/dir/output/images/arch.img /your/v86/dir/images/arch.img
cp -f /your/packer/dir/output/images/fs.json /your/v86/dir/images/fs.json
cp -r /your/pacher/dir/output/arch /your/v86/dir/

Next, we need a webserver that supports the Range header. For example this extension of the SimpleHTTPServer.

cd /your/v86/dir
wget https://raw.githubusercontent.com/smgoller/rangehttpserver/master/RangeHTTPServer.py
python2 RangeHTTPServer.py

Now that the webserver is running, point your browser to http://localhost:8000/examples/arch.html. Wait for the Linux to boot. When the system is up, click 'Save state to file'. Your browser will download a v86state.bin file. Copy that file to /your/v86/dir/images. You can then edit examples/arch.html again and add a 'state' key to the V86Starter array.

initial_state: {
    "url": ""http://localhost:8000/images/v86state.bin,
},

If you refresh http://localhost:8000/examples/arch.html you will see that the state is restored instantly and all required files are loaded over the network on the fly.

Networking

The emulator can emulate a network card. For more information look at the networking documentation. To set up networking in the VM, add the following item to the V86Starter array in the examples/arch.html file:

network_relay_url: "ws://localhost:8080/",

This will make the emulator try to connect to a WebSockets proxy. Running the proxy is very easy if you use the Docker container.

sudo docker run --privileged -p 8080:80 --name relay benjamincburns/jor1k-relay:latest

You can check if the relay is running correctly by going to http://localhost:8080/ in your browser. There you should see a message that reads Can "Upgrade" only to "Websocket"..

Now you should be able to get network connectivity in the virtual machine. If you are restoring from a saved state, you might need to first run:

ip link set enp0s5 down
rmmod ne2k-pci

To bring the network up, run:

modprobe ne2k-pci
ip link set enp0s5 up
dhcpcd -w4 enp0s5

It might take a while for a carrier to become available on the interface. If the dhcpcd command fails shortly after booting, wait a bit and try again a bit later. If you are using the 9p network filesystem you can use the developer tools networking tab (in chrome) to get a sense of what is going on by looking at the files that are being downloaded.

When the network is up you should be able to curl a website. To check, run curl icanhazip.com. There you should see the public IP of the machine running the proxy.

You can't do inbound traffic into the VM with the websockproxy Docker container because it uses a basic NAT. To SSH into the VM running in the browser, you can create a reverse SSH tunnel to expose the SSH port of the sshd in the VM to the outside world.

# This will create a port 1122 on the example.com server 
# which forwards to the SSH in the VM
ssh root@example.com -R 1122:localhost:22

Now on the example.com server you should be able to SSH into your browser tab by running ssh root@localhost -p 1122.