Ben's notes

This is an investigation into pure UEFI isolated multi-boot. Multiple operating systems on an external drive with separate boot managers chainloaded from GRUB.

A few take-home points:

• UEFI USB booting will look for /EFI/BOOT/BOOT<arch>.EFI, in this document BOOTX64.EFI. This is a PE32+ executable starting the GRUB2 bootloader.
• Some UEFI Firmware does not work well, or not at all with multiple EFI System Partitions (ESPs) on the same disk, or even machine.
• The UEFI firmware's bootmenu is local to the machine, new entries will not be available on other machines.
• Booting an encrypted Arch Linux OS from another machine, it might not be possible to enter the pass phrase because the keyboard isn't recognised. For example, using a USB-keyboard on the one and PS/2 on the other machine. You can fix this with running mkinitcpio again so it detects the keyboard, or add the atkbd hid_generic ohci_pci usbhid modules to mkinitcpio.conf and generate a new initramfs.
• Passphrase iteration is determined by cryptsetup depending on CPU power. Decrypting on other hardware might be slower (too slow, or faster). Lookup with cryptsetup luksDump.
• If you are creating a new ESP partition using parted, then you also need the boot flag to set the correct partition type.
• Use filesystem or partition UUIDs where possible. This persistent naming solves problems when names differ between machines.
• Secure Erase an SSD over USB does not work properly!

• External drive (SSD over USB with UAS) is /dev/sdX.
• GPT Partitioning.
• Linux host (using Fedora 26 here) with GRUB2.
• Install packages: grub2-efi-x64-modules grub2-efi-x64.x86_64

In this section we will create the required ESP and the first partition for Arch Linux.

First, lets secure erase the SSD. If you trust the vendor then follow along. Else well, don't store anything in the clear before encrypting the device. Also, don't rely on this procedure if the SSD is USB attached.

• Make sure the device is not frozen:

  # hdparm -I /dev/sdX
  ...
  Security: 
  	Master password revision code = 65534
  		supported
  	not	enabled
  	not	locked
  --->	not	frozen
  	not	expired: security count
  		supported: enhanced erase
  	2min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
  ...

• Set password, any password will do, it will be reset to NULL after erasing.

  # hdparm --user-master u --security-set-pass randomDae6up6A /dev/sdX
  security_password: "randomDae6up6A"
  
  /dev/sda:
   Issuing SECURITY_SET_PASS command, password="randomDae6up6A", user=user, mode=high

• Check that password is enabled:

  # hdparm -I /dev/sdX
  Security: 
  	Master password revision code = 65534
  		supported
  --->		enabled

• Secure erase SSD:

  # time hdparm --user-master u --security-erase randomDae6up6A /dev/sdX
  security_password: "randomDae6up6A"
  
  /dev/sda:
   Issuing SECURITY_ERASE command, password="randomDae6up6A", user=user

• Verify: check that master password is supported, and not enabled:

  # hdparm -I /dev/sdX
  Security: 
  	Master password revision code = 65534
  		supported

• Verify: make sure old data is removed:

  partprobe /dev/sdX
  lsblk
  dd if=/dev/sdX bs=512 count=63 | hexdump -C

Write a GUID Partition Table (GPT), ESP partition and first OS-partition to the disk.

• Partiton the drive using the parted command:

  # Start at sector 65535, for correct alignment of my SSD.
  # 512MiB UEFI ESP
  # 32GiB partition for Arch Linux
  parted --script /dev/sdX              \
      mklabel gpt                       \
      unit mib                          \
      mkpart ESP fat32 65535s 1114095s  \
      toggle 1 boot                     \
      mkpart Arch 1179630s 68287470s

• Format the ESP:

  mkfs.fat -F32 -n ESP /dev/sdX1

• Mount the ESP:

  mount /dev/sdX1 /mnt

• Install GRUB:

  grub2-install --removable --boot-directory=/mnt/boot --efi-directory=/mnt

• The ESP should look like this:

  /mnt
  ├── boot
  │   └── grub2
  │       ├── fonts
  │       │   └── unicode.pf2
  │       ├── grubenv
  │       ├── locale
  │       │   ├── ast.mo
  <snip>
  │       │   └── zh_TW.mo
  │       └── x86_64-efi
  │           ├── acpi.mod
  │           ├── adler32.mod
  │           ├── affs.mod
  │           ├── afs.mod
  <snip>
  │           ├── zfscrypt.mod
  │           ├── zfsinfo.mod
  │           └── zfs.mod
  └── EFI
      └── BOOT
          └── BOOTX64.EFI
  

To test the concept, lets create a simple first entry starting the UEFI Shell. If you want to program your own UEFI binaries see Rodericks Hello World example: http://www.rodsbooks.com/efi-programming/hello.html

• Download the EFI Shell. https://github.com/tianocore/tianocore.github.io

  mkdir /mnt/EFI/UEFI-Shell
  cd /mnt/EFI/UEFI-Shell
  curl -OL https://github.com/tianocore/edk2/raw/master/ShellBinPkg/UefiShell/X64/Shell.efi

• Lookup the ESP filesystems UUID, here's two ways:

  # lsblk -f /dev/sdX1
  NAME FSTYPE LABEL UUID                                 MOUNTPOINT
  sdX1 vfat   ESP   7A65-AEC7                            /mnt

  # grub2-probe -t fs_uuid -d /dev/sdX1
  7A65-AEC7

• Edit /mnt/boot/grub2/grub.cfg.

  timeout=5
  
  menuentry 'UEFI Shell' {
    insmod chain
    insmod search_fs_uuid
    search --set=root --fs-uuid 7A65-AEC7
    chainloader ($root)/EFI/UEFI-Shell/Shell.efi
  }

• To test, unmount the ESP and boot the drive!

Let's install our next OS onto the external drive. We're going to install a full encrypted Arch linux OS, even /boot will be encrypted! For more information read the Arch Linux installation guide

• Since we're installing from Fedora, we'll need some tools:

  dnf install squashfs-tools

• Download the Arch Linux LiveCD image:

  curl -O http://ftp.nluug.nl/os/Linux/distr/archlinux/iso/2018.05.01/arch/x86_64/airootfs.sfs

• Download the hash:

  curl -O http://ftp.nluug.nl/os/Linux/distr/archlinux/iso/2018.05.01/arch/x86_64/airootfs.sha512

• Verify the download:

  sha512sum -c airootfs.sha512
  airootfs.sfs: OK

• Extract the image

  unsquashfs airootfs.sfs

• chroot into the install image:

  mount --bind squashfs-root squashfs-root
  mount -t proc none squashfs-root/proc
  mount -t sysfs none squashfs-root/sys
  mount -o bind /dev squashfs-root/dev
  mount -o bind /dev/pts squashfs-root/dev/pts  ## important for pacman (signature check)
  cp -L /etc/resolv.conf squashfs-root/etc      ## this is needed for networking within the chroot
  
  modprobe efivarfs
  mount -t efivarfs efivarfs squashfs-root/sys/firmware/efi/efivars
  
  chroot squashfs-root bash

• Initialize pacman:

  pacman-key --init
  pacman-key --populate archlinux

• Select a mirror in /etc/pacman.d/mirrorlist and refresh and update the packages

  pacman -Syyu

• Setup encryption:

  cryptsetup luksFormat /dev/sdX2
  cryptsetup open /dev/sdX2 cryptroot

• Create the filesystem:

  mkfs.ext4 /dev/mapper/cryptroot

• Mount the new filesystem:

  mount /dev/mapper/cryptroot /mnt

• Install Arch linux onto the new filesystem:

  pacstrap /mnt base

• Create a new fstab file:

  $ blkid /dev/mapper/cryptroot
  /dev/mapper/cryptroot: UUID="3ca1ba61-388f-452c-886b-a36d1a88d785" TYPE="ext4"
  
  vi /mnt/etc/fstab
  UUID=3ca1ba61-388f-452c-886b-a36d1a88d785	/         	ext4      	defaults	0 1

• chroot into the new Arch Linux install:

  arch-chroot /mnt

• Configure the time zone:

  ln -sf /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime

• Set the Hardware Clock from the System Clock, and update the timestamps in /etc/adjtime.

  hwclock --systohc

• Select locales in /etc/locale.gen and generate the locales:

  locale-gen
  localectl set-locale LANG="en_US.UTF-8"

• Configure the hostname in /etc/hostname and /etc/hosts
• Change the root password and create new users:

  passwd root
  useradd ...

• Add the encrypt to hook in /etc/mkinitcpio.conf:

  HOOKS="base udev autodetect modconf keyboard keymap block encrypt filesystems fsck"

• Generate a new initramfs:

  mkinitcpio -p linux

• Install GRUB:

  pacman -S grub

• Configure GRUB:

  # blkid /dev/sdX2
  /dev/sda2: UUID="628f049e-c5ea-4ef4-8876-50392a5ca1f6" TYPE="crypto_LUKS" PARTLABEL="Arch" PARTUUID="096a076a-e3fd-474e-864d-09aa78fad601"
  
  vi /etc/default/grub
  GRUB_ENABLE_CRYPTODISK=y
  GRUB_CMDLINE_LINUX="cryptdevice=UUID=628f049e-c5ea-4ef4-8876-50392a5ca1f6:cryptroot"
  
  grub-mkconfig -o /boot/grub/grub.cfg

• If you want to use the Gnome desktop, install the following packages and services:

  pacman -S gnome gnome-extra gdm
  systemctl enable gdm
  systemctl enable NetworkManager

• Exit the chroots:

  exit    # arch-chroot
  exit    # livecd chroot

• Mount the ESP and add the new Arch Linux install the boot menu. Note: modify the UUIDs:

  mount /dev/sdX1 /mnt
  vi /mnt/boot/grub2/grub.cfg 
  
  menuentry 'Arch Linux' {
    insmod gzio
    insmod part_gpt
    insmod cryptodisk
    insmod luks
    insmod gcry_rijndael
    insmod gcry_sha256
    insmod ext2
    cryptomount -u 628f049ec5ea4ef4887650392a5ca1f6
    set root='cryptouuid/628f049ec5ea4ef4887650392a5ca1f6'
    configfile /boot/grub/grub.cfg
  }

• Unmount the filesystems mounted from the external drive and test your new Arch Linux install!

• Add a new 32GiB partition:

  parted /dev/sdX
  unit mib
  mkpart Debian 76807020s 143914861s
  quit
  
  partprobe /dev/sdX

• Setup encryption:

  cryptsetup luksFormat /dev/sdX3
  
  cryptsetup open /dev/sdX3 cryptroot

• Create an ext4 filesystem:

  mkfs.ext4 /dev/mapper/cryptroot

• Mount the filesystem:

  mkdir /mnt/debianroot
  mount /dev/mapper/cryptroot /mnt/debianroot
  mkdir -p /mnt/debianroot/boot/efi

• Install debootstrap:

  dnf install debootstrap

• Install Debian:

  # Path is needed when installing from Fedora...
  PATH="$PATH:/bin:/sbin:/usr/sbin" debootstrap --arch amd64 stretch /mnt/debianroot http://ftp.nl.debian.org/debian
  ..
  ..
  I: Base system installed successfully.

• Enter the chroot:

  mount -t proc none /mnt/debianroot/proc
  mount -t sysfs none /mnt/debianroot/sys
  mount -o bind /dev /mnt/debianroot/dev
  mount -o bind /dev/pts /mnt/debianroot/dev/pts  ## important for pacman (for signature check)
  cp -L /etc/resolv.conf /mnt/debianroot/etc      ## this is needed to use networking within the chroot
  
  modprobe efivarfs
  mount -t efivarfs efivarfs /mnt/debianroot/sys/firmware/efi/efivars
  
  XTERM=xterm-color LANG=C.UTF-8 PATH="$PATH:/bin:/sbin:/usr/sbin" chroot /mnt/debianroot bash

• Edit the fstab:

  lsblk -f /dev/sdX3 # run outside chroot, UUID of the cryptroot.
  
  vi /etc/fstab
  UUID=cbd33fa1-9d89-4308-a24f-b3128671874a /     ext4    defaults        0 1

• Configure Debian:

  apt install locales
  dpkg-reconfigure locales tzdata
  
  # Install kernel, efibootmgr, grub
  apt install linux-image-amd64 efibootmgr grub-efi-amd64 cryptsetup initramfs-tools
  # don't install the grub bootloader yet...
  
  vi /etc/crypttab # lookup UUID with blkid /dev/sdX3
  cryptroot UUID=<UUID> none luks
  
  Modify "CRYPTSETUP=y" in /etc/cryptsetup-initramfs/conf-hook
  
  update-initramfs -ut
  
  vi /etc/default/grub
  GRUB_ENABLE_CRYPTODISK=y
  
  grub-mkconfig -o /boot/grub/grub.cfg
  
  
  set hostname in /etc/hostname
  
  passwd root
  useradd...
  
  tasksel install desktop
  tasksel install laptop
  
  exit
  umount -R /mnt/debianroot

• Add Debian to grub Note: change the UUID's to the blkid found earlier:

  mount /dev/sdX3 /mnt
  vi /mnt/boot/grub2/grub.cfg 
  
  menuentry 'Debian' {
    insmod gzio
    insmod part_gpt
    insmod cryptodisk
    insmod luks
    insmod gcry_rijndael
    insmod gcry_sha256
    insmod ext2
    cryptomount -u 63a124c1f7744493a20ab5a45bc5e06f
    set root='cryptouuid/63a124c1f7744493a20ab5a45bc5e06f'
    configfile /boot/grub/grub.cfg
  }
  

• Add a new 32GiB partition:

  parted /dev/sdX
  parted /dev/sdX
  unit s
  mkpart Kali 143980395s 211088235s
  align-check optimal 4
  quit
  
  partprobe /dev/sdX

• Setup encryption:

  cryptsetup luksFormat /dev/sdX4
  
  cryptsetup open /dev/sdX4 cryptroot

• Create an ext4 filesystem:

  mkfs.ext4 /dev/mapper/cryptroot

• Mount the filesystem:

  mkdir /mnt/kaliroot
  mount /dev/mapper/cryptroot /mnt/kaliroot
  mkdir -p /mnt/kaliroot/boot/efi

• Install debootstrap:

  dnf install debootstrap

• Install Kali:

  # Path is needed when installing from Fedora...
  curl "http://git.kali.org/gitweb/?p=packages/debootstrap.git;a=blob_plain;f=scripts/kali;hb=refs/heads/kali/master" > kali-debootstrap
  
  PATH="$PATH:/bin:/sbin:/usr/sbin" debootstrap --arch amd64 kali-rolling /mnt/kaliroot http://http.kali.org/kali ./kali-debootstrap
  ..
  ..
  I: Base system installed successfully.

• Enter the chroot:

  mount -t proc none /mnt/kaliroot/proc
  mount -t sysfs none /mnt/kaliroot/sys
  mount -o bind /dev /mnt/kaliroot/dev
  mount -o bind /dev/pts /mnt/kaliroot/dev/pts  ## important for pacman (for signature check)
  cp -L /etc/resolv.conf /mnt/kaliroot/etc      ## this is needed to use networking within the chroot
  
  modprobe efivarfs
  mount -t efivarfs efivarfs /mnt/kaliroot/sys/firmware/efi/efivars
  
  XTERM=xterm-color LANG=C.UTF-8 PATH="$PATH:/bin:/sbin:/usr/sbin" chroot /mnt/kaliroot bash

• Edit the fstab:

  lsblk -f /dev/sdX4 # run outside chroot, UUID of the cryptroot.
  
  vi /etc/fstab
  UUID=cbd33fa1-9d89-4308-a24f-b3128671874a /     ext4    defaults        0 1

• Configure Kali:

  apt install locales
  dpkg-reconfigure locales tzdata
  
  # Install kernel, efibootmgr, grub
  apt install linux-image-amd64 efibootmgr grub-efi-amd64 cryptsetup initramfs-tools
  # don't install the grub bootloader yet...
  
  vi /etc/crypttab # lookup UUID with blkid /dev/sdX4
  cryptroot UUID=<UUID> none luks
  
  Modify "CRYPTSETUP=y" in /etc/cryptsetup-initramfs/conf-hook
  
  update-initramfs -ut
  
  vi /etc/default/grub
  GRUB_ENABLE_CRYPTODISK=y
  
  grub-mkconfig -o /boot/grub/grub.cfg
  
  set hostname in /etc/hostname
  
  passwd root
  useradd...
  
  tasksel install desktop
  tasksel install laptop
  
  exit
  umount -R /mnt/kaliroot

• Add Kali to grub Note: change the UUID's to the blkid found earlier:

  mount /dev/sdX4 /mnt
  vi /mnt/boot/grub2/grub.cfg 
  
  menuentry 'Kali' {
    insmod gzio
    insmod part_gpt
    insmod cryptodisk
    insmod luks
    insmod gcry_rijndael
    insmod gcry_sha256
    insmod ext2
    cryptomount -u 333324c1f7744493a20ab5a45bc5e06f
    set root='cryptouuid/333324c1f7744493a20ab5a45bc5e06f'
    configfile /boot/grub/grub.cfg
  }