For some time now I’ve been looking for a way of keeping projects I’m working on in a separate VM without much setup. There are numerous solutions to do this kind of thing using namespaces (various container technologies, firejail, flatpak, bubblewrap, …) but I wanted something that uses real virtualization. I recently stumbled upon incus and it seemed like it could be a solution to my problem. Incus is a manager for both containers and virtual machines. It is possible to create VM images that can be used as templates, meaning that other machines can then be based on them on creation. Virtual machines can be configured using “profiles”.

For my use case, I have created a profile that defines how my template image should look like. In this profile I use cloud-init to configure the template machine on startup:

config:
  cloud-init.user-data: |
    #cloud-config
    apt:
      conf: |
        APT {
          Install-Recommends 0;
        }
    package_update: true
    package_upgrade: true
    package_reboot_if_required: true
    packages:
      - file
      - man-db
      - bat
      - eza
      - git
      - git-delta
      - neovim
      - zsh
      - tmux
      - curl
      - ncurses-term
      - hx
      - nix-bin
      - nix-setup-systemd
      - rustup
      - npm
      - podman
    users:
      - name: user
        lock_passwd: false
        # mkpasswd 'incus password'
        hashed_passwd: $y$j9T$p9XBP4Ox3SBUnyu.sqv5m1$f1rxcPJyVs05w./dZvIzu6.4dVWBvzu99nMBZRNlamD
        groups: sudo, nix-users
        shell: /usr/bin/zsh
        uid: 1000
    write_files:
      - path: /home/user/.config/nix/nix.conf
        content: |
          experimental-features = nix-command flakes
        permissions: 0600
      - path: /home/user/.zshrc.local
        content: |
          export PATH="$PATH:$HOME/.nix-profile/bin"
        permissions: 0600
    runcmd:
      - [curl, -sSfL, -o, /home/user/.tmux.conf, https://grml.org/console/tmux.conf]
      - [curl, -sSfL, -o, /home/user/.vimrc, https://grml.org/console/vimrc]
      - [curl, -sSfL, -o, /home/user/.zshrc, https://grml.org/console/zshrc]
      - [chown, -R, user:user, /home/user/]    
description: Base template for dev VMs
...

and I use a script to regenerate the template image when I want to adapt the config:

#!/bin/sh

#set -e
set -x

NAME=dev-vm-template

# adapt the profile before rebuilding the template image
incus profile edit cloud-dev-template
incus delete --force ${NAME} || true
incus create images:debian/13/cloud --vm --quiet --profile default --profile cloud-dev-template ${NAME}
incus start ${NAME}
incus wait ${NAME} agent
incus exec ${NAME} -- cloud-init status --wait || true
incus stop ${NAME}
incus publish --alias cloud-dev-template --reuse ${NAME}

This script uses the debian/13/cloud image together with my cloud-dev-template profile to set up a machine, configure it using cloud-init and store the result as an image called cloud-dev-template.

inker

To use this setup, I created a small python tool called inker that talks to the incus REST API. Using inker create creates a new VM based on my template. Using inker attach attaches the current working directory to a VM as a readonly mount. It is possible to combine those commands like inker create | inker attach to do those two steps in one command. inker login logs into the VM that has the current directory attached. This way I can create VMs that are connected to directories on my filesystem.

I am currently playing around with this setup, but I think this could help me to have a throwaway development machine, similar to dev containers. I’m not sure yet how happy I am with incus, though. I was also looking at systemd-vmspawn, but there are still some features missing that I would need. But I might rewrite inker at some point to use a different backend.