Kernel Panic VFS Error and TPM Bypass (Part 2)

calendar Apr 20, 2024 · 3 min read  ·
Share on: linkedin copy

Solution 2: Fixing Incomplete Kernel or System Package Installation

  1. Create an EBS Snapshot:

  2. Open Amazon EC2 Console:

    • Ensure you're in the correct Region.

  3. Stop Impaired Instance:

    • Navigate to Instances, select the impaired instance.
    • Choose Instance State, Stop instance, and confirm.

  4. Detach Root Volume:

    • In the Storage tab, under Block devices, select the Volume ID for /dev/sda1 or /dev/xvda.
    • Choose Actions, Detach Volume, and confirm. Note the Availability Zone.

  5. Launch Rescue Instance:

    • Launch a rescue EC2 instance in the same Availability Zone.

  6. Attach Detached Volume:

    • Once the rescue instance launches, go to Volumes, select the detached root volume of the impaired instance.
    • Choose Actions, Attach Volume.
    • Choose the rescue instance ID and set an unused device (e.g., /dev/sdf).

  7. Connect to Rescue Instance:

    • Use SSH to connect to the rescue instance.

  8. Mount Root Partition:

    • Run lsblk to view available disk devices.
    • Mount the root partition of the mounted volume to /mnt: mount -o nouuid /dev/xvdf1 /mnt.

  9. Set Up Chroot Environment:

    • Mount /dev, /run, /proc, and /sys of the rescue instance to the same paths as the mounted volume: for m in dev proc run sys; do mount -o bind {,/mnt}/$m; done.
    • Enter the mount directory: 
      1   chroot /mnt

  10. Update GRUB Bootloader:

  • Follow the appropriate procedure based on your OS:

For GRUB1 (Legacy GRUB) - Red Hat 6 and Amazon Linux 1: Use sed command to replace the corrupt kernel with the stable kernel in "/boot/grub/grub.conf":

1  sed -i '/^default/ s/0/1/' /boot/grub/grub.conf

For GRUB2 - Ubuntu 14 LTS, 16.04, and 18.04:

  • Replace corrupt GRUB_DEFAULT=0 default menu entry with stable GRUB_DEFAULT=saved value in /etc/default/grub.
1  sed -i 's/GRUB_DEFAULT=0/GRUB_DEFAULT=saved/g' /etc/default/grub
  • Run "update-grub" command.
1 update-grub
  • Run "grub-set-default" command, so that the stable kernel loads at the next restart :
1  grub-set-default 1

For GRUB2 - RHEL 7, Amazon Linux 2:

  • Replace corrupt GRUB_DEFAULT=0 default menu entry with stable GRUB_DEFAULT=saved value in /etc/default/grub.

    1  sed -i 's/GRUB_DEFAULT=0/GRUB_DEFAULT=saved/g' /etc/default/grub

  • Update GRUB to regenerate /boot/grub2/grub.cfg.

1   grub2-mkconfig -o /boot/grub2/grub.cfg
  • Run grub2-set-default command.
1  grub2-set-default 1

For GRUB2 - RHEL 8 and CentOS 8: GRUB2 in RHEL 8 and CentOS 8 utilizes blscfg files and entries in /boot/loader for boot configuration, departing from the previous grub.cfg format. It's advisable to employ the grubby tool for managing blscfg files and retrieving information from /boot/loader/entries/. If these files are missing or corrupted, grubby may not yield any results, necessitating their regeneration for functionality restoration. Kernel indexing depends on .conf files in /boot/loader/entries and kernel versions, prioritizing the latest kernel with the lowest index.

  • View Current Default Kernel:
1grubby --default-kernel
  • List all Available Kernels and Indexes:
1grubby --info=ALL

Example Output:

1  index=0
2  kernel="/boot/vmlinuz-4.18.0-305.el8.x86_64"
3  ...
4  index=1
5  kernel="/boot/vmlinuz-0-rescue-0c75beb2b6ca4d78b335e92f0002b619"
6  ...
7  index=2
8  kernel="/boot/vmlinuz-4.18.0-305.3.1.el8_4.x86_64"
9  ...

  • Set Default Kernel:

    • Identify the desired kernel path (e.g., /boot/vmlinuz-4.18.0-305.3.1.el8_4.x86_64).
    • Execute below command:
    1 grubby --set-default=/boot/vmlinuz-4.18.0-305.3.1.el8_4.x86_64

    Note: Replace "4.18.0-305.3.1.el8_4.x86_64" with your kernel's version number.

  • Verify Default Kernel:

    1grubby --default-kernel
  1. Unmount Volumes and Detach Root Volume:
    • Exit chroot:
    1  exit
    • Unmount /dev, /run, /proc, and /sys:
    1umount /mnt/{dev,proc,run,sys}
    • Stop the rescue instance and detach the root volume from it.
    • Attach the root volume to the impaired instance as the root volume (/dev/sda1), then start the instance.

The stable kernel should now load, and your instance should start successfully.