On the train to work I opened my Purism Librem 13 v3 laptop, and the screen did not wake up. Normally I have to hold the power key for 2-3 seconds to enable it in such a case, but this time that did not work. ( I decided to reboot.
The boot logo appeared, grub flashed, and then I saw:
cryptsetup: Waiting for encrypted source device UUID=d21486a-2088-4949-9e9f-deadbeefcafe
cryptsetup: Waiting for encrypted source device UUID=d21486a-2088-4949-9e9f-deadbeefcafe
[...]
Gave up waiting for suspend/resume device
cryptsetup: Waiting for encrypted source device UUID=d21486a-2088-4949-9e9f-deadbeefcafe
Gave up waiting for root file system device. Common problems:
- Boot args (cat /proc/cmdline)
- Check rootdelay= (did the system wait long enough?)
- Missing modules (cat /proc/modules; ls /dev)
ALERT! /dev/mapper/boo--vg-root does not exist. Dropping to a shell!
BusyBox v1.35.0 (Debian 1:1.35.0-3+b1) built-in shell (ash)
Enter ´help´ for a list of built-in commands.
(initramfs) _
I could not enter the password to decrypt the hard disk, and so the laptop could not boot.
I am using Debian unstable, and this is the first time this was a problem: I was bitten by Bug #1023492 . And yes, I am running kernel 6.0.6-2. And I had purged the 5.x kernels a couple of days earlier when cleaning up the system :(
The bug report said that the bug was fixed with Debian kernel 6.0.8-1. But how do I upgrade the kernel on a system that does not boot?
Booting live disk
I downloaded a live image and copied the .iso file to a USB flash drive with cp. After plugging it into the laptop I could boot from it and had a shell.
Decrypting
The live CD did not have cryptsetup on board, so I had to install it. Luckily I have an USB ethernet adapter, so I simply plugged it into the laptop and ran:
# see the available network devices $ ip addr # who had the idead to use the mac address in the device name? # that one did not think of the people that have to manually type them! $ dhclient enx000ec6e25b11 # check if the device is up now $ ip addr $ apt update && apt install -y cryptsetup
Now I could finally decrypt the harddisk and mount the partition. I had a LVM group so mounting was not as straight-forward as I had hoped:
$ cryptsetup open /dev/sda5 sda5_crypt $ mount /dev/mapper/crypted_sda5 /mnt/ mount: /mnt: unknown filesystem type "LVM2_member" $ vgdisplay --- Volume group --- VG Name boo-vg [...] $ mount /dev/boo-vg/root /mnt/ $ ls /mnt/ bin boot dev etc home ...
cryptsetup warning
At first I used "crypted_sda5" as last parameter for cryptsetup. This worked, but when updating the kernel I got an error:
update-initramfs: Generating /boot/initrd.img-6.0.0-4-amd64 cryptsetup: WARNING: target 'crypted_sda5' not found in /etc/crypttab
It is better to look into /etc/crypttab to see the name you have to use.
chroot
Now that I could mount the harddisk, I only had to chroot into the system and update the kernel.
$ mount /dev/boo-vg/root /mnt/ $ mount /dev/sda1 /mnt/boot $ mount -t proc proc /mnt/proc $ mount -t sysfs sys /mnt/sys $ mount -o bind /dev /mnt/dev $ mount -t devpts pts /mnt/dev/pts $ chroot /mnt $ dhclient enx000ec6e25b11 $ apt update && apt upgrade -y # manually force regenerating initramfs when you run into # the "target 'crypted_sda5' not found in /etc/crypttab" error # https://www.debian.org/doc/manuals/debian-kernel-handbook/ch-initramfs.html $ dpkg-reconfigure linux-image-6.0.0-4-amd64 $ exit $ reboot
With the updated kernel I could enter the harddisk password at boot, and the system started up normally.
The process of fixing this problem took ~ 2.5 hours.