systemd services used:

• systemd-tmpfiles
• systemd-resolved
• systemd-networkd
• systemd-timesyncd

• Host PC user (for Fedora) needs to be member of dialout and the disk group to be able to access the serial port and write to the USB drive. Or else use sudo.
• Connect to the PC Engines APU's serial port.

  screen /dev/ttyUSB0 115200 # to select: boot from USB
  screen /dev/ttyUSB0 38400  # to continue Arch Linux installation

• Connect the APU to Ethernet / internet for updates and access to the repo's. The first network port is next to the serial port.
• If you want, you can update the BIOS first: pcengines_apu2_update_firmware_rom

• Download the latest ISO image from https://www.archlinux.org/download/.
• Verify the download:

  Hashes from sha1sums.txt
  ce7bbccc246b6c4c6ff13eadbcdf18e44bb85d82  archlinux-2017.11.01-x86_64.iso
  5b6c6f8fd11281e2f01224478b35283e481dec26  archlinux-bootstrap-2017.11.01-x86_64.tar.gz
  
  $ sha1sum archlinux-2017.11.01-x86_64.iso
  ce7bbccc246b6c4c6ff13eadbcdf18e44bb85d82  archlinux-2017.11.01-x86_64.iso

• Write image to USB:

  sudo dd bs=4M if=archlinux-2017.11.01-x86_64.iso of=/dev/sdX status=progress

• Boot the APU2 en press F10, select USB boot.
• Switch console to 38400 baud. Press 'ctrl-l' to redraw the screen.
• Select the Boot Arch Linux option and press TAB.
• Add console=ttyS0,38400 to the kernel line and press enter
• Log in with user root (no password).
• If you connected the network cable after booting, request an IP-address

  # dhclient enp1s0

• Install and run SSHd to complete the installation over SSH:

  select nearby mirror in: /etc/pacman.d/mirrorlist
  
  # pacman -Sy
  # pacman -S openssh
  # passwd root
  # systemctl start sshd

The next steps will install Arch Linux on the SSD.

• Partition the SSD:

  (
  echo o     # Create a new empty DOS partition table
  echo n     # Add a new partition
  echo p     # Primary partition
  echo 1     # Partition number
  echo       # First sector (Accept default: 1)
  echo +256M # Last sector (Accept default: varies)
  echo n     # Add a new partition
  echo p     # Primary partition
  echo 2     # Partition number
  echo       # First sector (Accept default)
  echo       # Last sector (Accept default, rest of the drive)
  echo w     # Write changes
  ) | sudo fdisk /dev/sdX

• You might reboot if you cannot use the new partitions yet:

  # partprobe /dev/sda                                                                                  :(
  Error: Partition(s) 2 on /dev/sda have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use.  As a result, the old partition(s) will remain in use.  You should reboot now before making further changes.

• Create the /boot and root filesystems:

  # mkfs.ext4 /dev/sdX2
  # mount /dev/sda2 /mnt
  
  # mkfs.ext4 /dev/sdX1
  # mkdir /mnt/boot
  # mount /dev/sdX1 /mnt/boot

• Copy Arch Linux to the new filesystems:

  # pacstrap /mnt base

• Generate a fstab:

  # genfstab -L /mnt >> /mnt/etc/fstab

• Chroot into the new system:

  # arch-chroot /mnt

• Set root password:

  # passwd root

• Setup system clock:

  # hwclock --systohc --utc

• Set the hostname:

  # echo MYHOSTNAME > /etc/hostname

• Generate locale, uncomment wanted locale:

  # vi /etc/locale.gen
  # locale-gen

• Generate new initramfs:

  # mkinitcpio -p linux

• Install bootloader:

  # pacman -S grub
  # grub-install /dev/sdX
  # grub-mkconfig -o /boot/grub/grub.cfg

• Configure serial output for the Linux kernel:

  # vi /etc/default/grub   # add options below
  GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"

• Configure grub and serial:

  # vi /etc/default/grub   # add options below
  
  ## Serial console
  GRUB_TERMINAL=serial
  GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"

• Make new grub config:

  # grub-mkconfig -o /boot/grub/grub.cfg

• Exit chroot:

  # exit

• Reboot and reconnect with 115200 baud.

• Configure timezone:

  # timedatectl set-timezone Europe/Amsterdam

• Configure network (DHCP example):

  #  systemctl enable systemd-resolved --now
  
  #  cat > /etc/systemd/network/50-wired.network <<EOF
  [Match]
  Name=enp1s0
  
  [Network]
  DHCP=ipv4
  EOF
  
  # systemctl enable systemd-networkd --now

• Add users
• Enable SSH:

  # pacman -S openssh
  # systemctl enable sshd --now

• Configure simple firewall:

  # pacman -S ufw
  # ufw default deny
  # ufw allow SSH
  # ufw enable

• Configure timekeeping:

  # vi /etc/systemd/timesyncd.conf
  # timedatectl set-ntp true

• CPU Temperature, power reading seems not valid:

  # pacman -S lm_sensors
  # sensors-detect
  # sensors
  k10temp-pci-00c3
  Adapter: PCI adapter
  temp1:        +55.6 C  (high = +70.0 C)
                         (crit = +105.0 C, hyst = +104.0 C)
  
  fam15h_power-pci-00c4
  Adapter: PCI adapter
  power1:        6.63 W  (interval =   0.01 s, crit =   6.00 W)

• HDD/SSD temperature:

  # pacman -S hddtemp
  # hddtemp /dev/sda
  /dev/sda: SATA SSD: 33 C
  

• Control LEDs:

  # pacman -S git gcc cmake
  
  # git clone https://github.com/pcengines/apu_gpio_lib.git
  # cd apu_gpio_lib/
  # mkdir build
  # cd build/
  # cmake ..
  # make
  # cd example
  # ./blinky

• Kernel module from: https://github.com/trick77/apu-led-button:

  # pacman -S linux-headers
  # git clone https://github.com/trick77/apu-led-button.git
  # make
  # make install
  
  # Arch linux specific module install
  gzip apuled-button.ko
  cp apuled-button.ko.gz /usr/lib/modules/`uname -r`/extramodules/
  
  echo "apuled-button" > /etc/modules-load.d/apuled-button.conf
  modprobe apuled-button
  
  ### Test
  # modprobe ledtrig_default_on; modprobe ledtrig_heartbeat
  # echo heartbeat > /sys/class/leds/apu2:1/trigger
  
  # apu2led eth0 -c nrt -f

• Config LED at boot (with systemd-tmpfiles):

  # printf "ledtrig_default_on\nledtrig_heartbeat\n" >> /etc/modules-load.d/apuled-button.conf
  # echo "w /sys/class/leds/apu2:1/trigger - - - - heartbeat" > /etc/tmpfiles.d/apuled.conf

• Configure the third LED as network activity LED for enp1s0:

  # cat > /etc/systemd/system/apuled.service << EOF
  [Unit]
  Description=APU2 LED
  After=network.target
  
  [Service]
  Type=simple
  ExecStart=/usr/local/sbin/apuled enp1s0 -c nna -f
  Restart=on-abort
  
  [Install]
  WantedBy=multi-user.target
  EOF
  
  # systemctl enable apuled.service --now

• Due to a bug, … disable i2c_piix4, if you don't need it.

  cat > /etc/modprobe.d/noi2c.conf <<EOF
  # Disable i2c_piix4 in favour of watchdog device. Due to a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1406844
  blacklist i2c_piix4
  EOF
  

• But then….

  dmesg |grep tco
  [    3.223577] sp5100_tco: SP5100/SB800 TCO WatchDog Timer Driver v0.05
  [    3.223965] sp5100_tco: PCI Vendor ID: 0x1022, Device ID: 0x780b, Revision ID: 0x42
  [    3.223997] sp5100_tco: failed to find MMIO address, giving up.
  

• see http://pcengines.info/forums/?page=post&id=F1D33B75-58E7-4A5A-9A80-1E8CA35AC84D&cachecommand=bypass&pageindex=1
• After mkinitcpio -p linux and another reboot:

  $ dmesg |grep tco
  [    3.221835] sp5100_tco: SP5100/SB800 TCO WatchDog Timer Driver v0.05
  [    3.222200] sp5100_tco: PCI Vendor ID: 0x1022, Device ID: 0x780b, Revision ID: 0x42
  [    3.222310] sp5100_tco: Using 0xfeb00000 for watchdog MMIO address
  [    3.222346] sp5100_tco: Last reboot was not triggered by watchdog.
  [    3.222562] sp5100_tco: initialized (0xffffae8d40a29000). heartbeat=60 sec (nowayout=0)

• Enable watchdog:

  systemctl enable watchdog --now

• Config file: /etc/watchdog.conf