Cluster Provisioning: CentOS 7.9 PXE Automated Deployment

When building AI/HPC clusters with tens or thousands of bare-metal servers, manual OS installation is impractical. This document details the complete process of setting up a fully automated provisioning environment using PXE (Preboot Execution Environment) combined with DHCP + TFTP + HTTP services.

1. Environment Preparation

Note: Assume server IPs are uniformly planned. Example:

• Deployment Node (Server) IP: 192.168.1.100
• Subnet: 192.168.1.0/24

1.1 Configure Local YUM Repo

To ensure speed and stability, configure a local YUM source using the ISO image.

# 1. Backup existing repos
cd /etc/yum.repos.d/ && mkdir backup && mv *.repo backup/

# 2. Create local repo config
cat > /etc/yum.repos.d/local.repo <<EOF
[centos79]
name=RedHat7.9
baseurl=file:///mnt
enabled=1
gpgcheck=0
EOF

# 3. Mount ISO (Recommend permanent mount for production)
mount -o loop /root/centos-server-7.9-x86_64-dvd.iso /mnt

1.2 Disable Security Restrictions

PXE involves communication across multiple services (UDP 67/69). It is recommended to disable Firewalld and SELinux during deployment.

systemctl stop firewalld && systemctl disable firewalld
setenforce 0
sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config

---

2. TFTP Service (Transfer Bootloader)

TFTP is used to send the bootloader and kernel files to the client.

2.1 Install & Configure

yum install -y xinetd tftp tftp-server

# Edit config: Enable TFTP
vim /etc/xinetd.d/tftp
# Change 'disable = yes' to 'disable = no'
# Ensure 'server_args = -s /var/lib/tftpboot'

2.2 Start Service

systemctl start xinetd tftp
systemctl enable xinetd tftp
# Verify UDP 69 port
netstat -anu | grep 69

---

3. DHCP Service (IP Allocation & Guidance)

DHCP allocates IPs and informs the client of the "Next Server" (TFTP) location.

3.1 Install DHCP

yum install -y dhcp

3.2 Configuration Details

Edit /etc/dhcp/dhcpd.conf. Modify IPs according to your actual subnet:

authoritative;
ddns-update-style none;
default-lease-time 600;
max-lease-time 7200;

# PXE Options
option space PXE;
option PXE.mtftp-ip code 1 = ip-address;
option arch code 93 = unsigned integer 16; # Architecture check

subnet 192.168.1.0 netmask 255.255.255.0 {
    range 192.168.1.200 192.168.1.250;  # IP Range
    next-server 192.168.1.100;          # TFTP Server IP
    
    class "pxeclients" {
        match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
        # 00:07 = UEFI, 00:00 = Legacy BIOS
        if option arch = 00:07 {
            filename "BOOTX64.efi";
        } else {
            filename "pxelinux.0";
        }
    }
}

3.3 Start Service

systemctl start dhcpd && systemctl enable dhcpd

---

4. HTTP Service (Image & Kickstart Hosting)

HTTP service provides the full OS image download and the Kickstart automation script.

4.1 Install & Mount

yum install -y httpd
systemctl start httpd && systemctl enable httpd

# Mount ISO
mkdir -p /var/www/html/centos79
mount -o loop /root/centos-server-7.9-x86_64-dvd.iso /var/www/html/centos79

Verify: Access http://192.168.1.100/centos79/ via browser.

---

5. PXE Boot Configuration (Core)

5.1 Prepare Boot Files

Extract necessary files from ISO or syslinux package to the TFTP directory.

# Copy Kernel and Initrd
cp /var/www/html/centos79/images/pxeboot/{vmlinuz,initrd.img} /var/lib/tftpboot/

# Prepare UEFI Bootloader (shim.efi or grubx64.efi needed)
# cp /path/to/BOOTX64.efi /var/lib/tftpboot/
chmod +x /var/lib/tftpboot/BOOTX64.efi

5.2 Edit Boot Menu (UEFI)

File: /var/lib/tftpboot/pxelinux.cfg/efidefault (or grub.cfg)

default=10
timeout=15

label 10
    title Install CENTOS-7.9-x86_64 (UEFI)
    root (nd)
    # ksdevice=link: Use the linked NIC
    # ks=...: Kickstart file location
    kernel /vmlinuz ksdevice=link ks=http://192.168.1.100/centos79.cfg ip=dhcp
    initrd /initrd.img

---

6. Troubleshooting

Symptom	Possible Cause	Solution
Stuck at "DHCP..."	DHCP service down / Firewall blocking	Check systemctl status dhcpd and UDP 67
Error "TFTP Open Timeout"	Client cannot reach TFTP	Check next-server IP correctness
vmlinuz download fail	Permission denied	Ensure read permissions for /var/lib/tftpboot
HTTP 404/403	Kickstart unreachable	Test URL via browser manually

7. Maintenance Commands

# Check all ports
netstat -nutlp | egrep "67|69|80"

# Restart all services
systemctl restart xinetd tftp dhcpd httpd