Paul

Me hacking away at things.

4 minute read

I replaced Debian with NixOS on a Hetzner VPS using nixos-anywhere, fully remotely. It sshs into the running box, kexecs into a NixOS installer in RAM, runs disko to partition, nixos-installs the flake, and reboots.

Files I had to add to the flake:

  • hetzner/disko.nix - declarative disk layout. Confirm the disk device and boot mode on the box first (lsblk, [ -d /sys/firmware/efi ]). Mine was /dev/sda, legacy BIOS: 
    {
  disko.devices.disk.main = {
    device = "/dev/sda";
    type = "disk";
    content = {
      type = "gpt";
      partitions = {
        boot = { size = "1M"; type = "EF02"; };   # BIOS-boot (GRUB core.img)
        root = {
          size = "100%";
          content = { type = "filesystem"; format = "ext4"; mountpoint = "/"; };
        };
      };
    };
  };
}
  • hetzner/hardware-configuration.nix - minimal qemu/virtio guest. disko owns the filesystems, so declare none here: 
    { modulesPath, lib, ... }:
{
  imports = [ (modulesPath + "/profiles/qemu-guest.nix") ];
  boot.initrd.availableKernelModules =
    [ "virtio_pci" "virtio_scsi" "virtio_blk" "sr_mod" ];
  nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux";
}
  • hetzner/configuration.nix - the system. sshd on 80 (proxy) + 22, key-only, and the root key baked in so I am not locked out after the wipe: 
    boot.loader.grub.enable = true;
services.openssh = { enable = true; ports = [ 22 80 ]; };
users.users.root.openssh.authorizedKeys.keys = [ "ssh-ed25519 AAAA... me" ];
system.stateVersion = "26.05";

    Note: do not set boot.loader.grub.device here. disko already sets boot.loader.grub.devices from the disk holding the EF02 partition; setting it again fails eval with “You cannot have duplicated devices in mirroredBoots”.

  • flake.nix - a disko input and the machine, wiring disko in as a module: 
    hetzner = nixpkgs.lib.nixosSystem {
  system = "x86_64-linux";
  modules = [ disko.nixosModules.disko ./hetzner/configuration.nix ];
};

Then the install itself, after a local nixos-rebuild build --flake .#hetzner to catch eval errors:

nix run github:nix-community/nixos-anywhere -- \
  --flake .#hetzner --ssh-port 80 --target-host root@IP

Note: the host key changes on reinstall, so afterwards ssh-keygen -R '[IP]:80' before reconnecting.

Troubleshooting: it stalled after kexec

The VPS is an SSH jump host that listens on port 80 precisely because the networks I admin it from block port 22. That block broke the install:

  • --ssh-port 80 sets only the initial connection. After kexec it reconnects on --post-kexec-ssh-port, which defaults to 22 -> blocked -> stalled.
  • Note: --post-kexec-ssh-port 80 only changes which port it connects to, not what the installer listens on. The stock kexec installer’s sshd is on 22, so pointing it at 80 just hit a closed port.

Since the reconnect never happened, disko never ran and Debian was intact - the kexec installer lives in RAM, so a reboot would have recovered the old OS. The fix, with the installer still up (VNC console + internet, disk untouched):

  • Start a second sshd on port 80 inside the installer. sshd refuses a relative invocation, so expand the absolute path: 
    $(command -v sshd) -p 80
  • Drive nixos-anywhere at the running installer, skipping kexec: 
    nixos-anywhere --flake .#hetzner --ssh-port 80 \
  --phases disko,install,reboot --target-host root@IP

You May Also Enjoy