Part 6: Kernel Mechanics, Boot Recovery & Security Hardening

01: The 6-Stage Linux Boot Sequence

Understanding the boot sequence allows you to diagnose and fix non-booting machines rapidly:

Plaintext

[ 1. Firmware ] ──► [ 2. GRUB2 ] ──► [ 3. Kernel ] ──► [ 4. initramfs ] ──► [ 5. systemd ] ──► [ 6. Target ]
   (BIOS/UEFI)       (Bootloader)     (Decompress)    (Early Storage)       (PID 1)        (e.g. multi-user)

1. Firmware (BIOS or UEFI): Executes Power-On Self-Test (POST), initializes hardware, and loads the registered boot loader from disk, NVRAM, or MBR.

2. GRUB2 Bootloader: Displays the OS boot menu, loads the selected kernel image (vmlinuz) and initramfs image into RAM, and passes control to the kernel.

3. Kernel Initialization: Decompresses itself, initializes core CPU/memory architectures, and executes kernel initialization threads.

4. initramfs (Initial RAM Filesystem): A temporary root filesystem loaded into memory containing minimal device drivers required to mount the real root filesystem (e.g., LVM, RAID, NVMe drivers).

5. systemd (PID 1): The kernel mounts the real root partition, executes /sbin/init (symlinked to systemd), and starts user-space services.

6. Default Target: systemd processes unit dependencies and brings the system up to its default target (e.g., multi-user.target).

02: Emergency Boot Recovery (/etc/fstab Failure)

A corrupted or missing entry in /etc/fstab will cause systemd's local-fs.target to time out during boot, dropping the machine into an emergency rescue shell.

Rescue Recovery Steps:

Bash

# Step 1: Remount the root filesystem as read-write (it defaults to read-only in emergency mode)
mount -o remount,rw /

# Step 2: Edit /etc/fstab to correct or comment out the failing device entry
vi /etc/fstab

# Step 3: Test mounting all filesystems defined in /etc/fstab to verify zero syntax errors
mount -a

# Step 4: Regenerate systemd's internal mount unit cache from the updated fstab
systemctl daemon-reload

# Step 5: Exit emergency shell and resume normal boot
reboot

03: Container Primitives: cgroups v2 and Namespaces

Containers are not virtual machines—they are ordinary Linux processes isolated by two native kernel primitives:

1. Namespaces (Isolation - What a process can SEE)

Namespaces restrict a process's view of system resources:

• pid: Provides an independent process tree (the container process sees itself as PID 1).

• net: Provides an isolated network stack (own IP addresses, routes, and iptables rules).

• mnt: Provides an isolated filesystem mount table.

• user: Maps container root (UID 0) to an unprivileged user ID on the host system.

2. cgroups v2 (Resource Control - How much a process can CONSUME)

Control Groups limit and monitor resource utilization (CPU, RAM, Disk I/O, process counts) for a process group.

Bash

# View active cgroup allocations for running services
systemd-cgtop

# Dynamically set resource limits on a running systemd service without modifying unit files
sudo systemctl set-property httpd.service MemoryMax=500M CPUQuota=20%

04: Security Hardening

System security relies on defense-in-depth: layering mandatory access control, authentication controls, and system auditing.

Mandatory Access Control: SELinux

SELinux tags every file and process with a security context label formatted as: user:role:type:level.

Security enforcement relies on Type Enforcement: a web server process running under httpd_t can only interact with files labeled httpd_sys_content_t.

Bash

# View SELinux context labels on files
ls -Z /var/www/html/index.html
# Output: unconfined_u:object_r:httpd_sys_content_t:s0 /var/www/html/index.html

# Check recent Access Vector Cache (AVC) denial logs
sudo ausearch -m avc -ts recent

# Restore default SELinux context recursively across a directory after moving files
sudo restorecon -Rv /var/www/html/

PAM (Pluggable Authentication Modules)

PAM configures authentication policies for system applications under /etc/pam.d/.

To protect against brute-force password attacks, configure pam_faillock to lock accounts after repeated failed logins:

Bash

# Check failed login lock status for a user
faillock --user jdoe

# Manually unlock a locked user account
faillock --user jdoe --reset

Server-Side SSH Hardening (/etc/ssh/sshd_config)

Apply these production security defaults inside /etc/ssh/sshd_config:

Ini, TOML

# Disable password-based logins (forces key-based authentication only)
PasswordAuthentication no
KbdInteractiveAuthentication no

# Disable direct remote login as root
PermitRootLogin no

# Restrict SSH access strictly to specified users
AllowUsers deployer jdoe

Bash

# Test SSH configuration file syntax before restarting the daemon
sudo sshd -t

# Apply new configuration
sudo systemctl restart sshd