Ben's notes

—-====[Work in progress]====—-

This page describes the installation of CentOS 7 on Supermicro hardware. I will be using this server for virtualization (KVM).

Mainbord model: X9SCL, version: 2.10 (01/09/2014)
IPMI Firmware: 3.38

• Quiet Boot: Disabled
• Restore on AC Power Loss: Last State
• Intel VT-d: Enabled
• Active State Power Management: Disabled
• SATA Mode: AHCI
• SOL (Serial Over LAN): Enabled
  • Terminal type: VT100+
  • Bits per second: 115200
• Fan speed control mode: Optimal

• Connect to the server via IPMI (remote console) and connect the installation ISO via the iKVM viewer.
• Boot the server, press F11 and select: IPMI Virtual CDROM
• Date & Time: Europe/Amsterdam
• Keyboard: English (US)
• Language support: English (United States)
• Installation source: Local media
• Software selection: minimal install
• Installation destination: software RAID1 /boot, swap, /, /var.

• Edit grub settings /etc/default/grub

  ## Serial console settings
  GRUB_TERMINAL="serial console"
  GRUB_TERMINAL_OUTPUT="serial console"
  GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=2 --word=8 --parity=no --stop=1"
  
  ## Remove "rhgb quiet" from GRUB_CMDLINE_LINUX, add: console=tty0 console=ttyS2,115200n8

• Rebuild the grub.cfg file with following command:

  grub-mkconfig -o /boot/grub/grub.cfg

• Reboot and test the serial console via IPMI over SSH.

• SSH to ipmi address.
• Open SOL:

  cd system1/sol1
  start

• To exit, press: <enter> <esc> <T>.

• Set hostname:

  hostnamectl set-hostname <enter fqdn>

• Install tools:

  yum install bridge-utils tcpdump

• At the time of writing, there are some bugs in NetworkManager (nmcli) when enslaving VLAN devices into a bridge. So this configuration will not use NM.
• Disable NM, enable network service:

  systemctl stop NetworkManager.service
  systemctl disable NetworkManager.service
  systemctl enable network.service
  systemctl start network.service

• Create bond0 device:

  vi /etc/sysconfig/network-scripts/ifcfg-bond0
  DEVICE=bond0
  BOOTPROTO=none
  ONBOOT=yes
  USERCTL=no
  BONDING_OPTS="mode=4 miimon=100"
  NM_CONTROLLED=no

• Add bond slaves:

  vi /etc/sysconfig/network-scripts/ifcfg-eno1
  DEVICE=eno1
  MASTER=bond0
  SLAVE=yes
  USERCTL=no
  ONBOOT=yes
  BOOTPROTO=none
  NM_CONTROLLED=no
  
  vi /etc/sysconfig/network-scripts/ifcfg-enp2s0f0
  DEVICE=enp2s0f0
  MASTER=bond0
  SLAVE=yes
  USERCTL=no
  ONBOOT=yes
  BOOTPROTO=none
  NM_CONTROLLED=no
  
  vi /etc/sysconfig/network-scripts/ifcfg-enp2s0f1
  DEVICE=enp2s0f1
  MASTER=bond0
  SLAVE=yes
  USERCTL=no
  ONBOOT=yes
  BOOTPROTO=none
  NM_CONTROLLED=no
  
  vi /etc/sysconfig/network-scripts/ifcfg-enp4s0
  DEVICE=enp4s0
  MASTER=bond0
  SLAVE=yes
  USERCTL=no
  ONBOOT=yes
  BOOTPROTO=none
  NM_CONTROLLED=no

• Create VLAN and bridge devices, do this for every VLAN you have:

  vi /etc/sysconfig/network-scripts/ifcfg-br50
  DEVICE=br50
  ONBOOT=yes
  SLAVE=bond0.50
  TYPE=Bridge
  VLAN=yes
  NM_CONTROLLED=no
  
  vi /etc/sysconfig/network-scripts/ifcfg-bond0.50
  DEVICE=bond0.50
  VLAN=yes
  ONBOOT=yes
  USERCTL=no
  NM_CONTROLLED=no
  BRIDGE=br50

• If needed, configure IP-addresses on bridge devices.
• Restart networking:

  service network restart

• Test:

  brctl show
  cat /proc/net/bonding/bond0

• If virbr0 isn't in zone internal, configure it so:

  firewall-cmd --permanent --zone=internal --change-interface=virbr0

• Add local (sources) subnets to trusted zone:

  firewall-cmd --permanent --zone=trusted --add-source=192.168.2.0/24

• Reload firewall:

  firewall-cmd --reload

• Install and configure chrony
• Install and configure apcupsd
• Set tuned profile:

  tuned-adm profile virtual-host

• Configure postfix

• Install libvirt etc:

  yum groupinstall "Virtualization Host"

• Enable guest service so that guests are suspended before reboot/shutdown and auto started after boot:

  systemctl enable libvirt-guests.service