Piece of Unix Hints

Imagine that your linux system is running on a 1To drive. At one point you needed 1To but nowaday you only use 70Go on it. You may want to transfer your system to a smaller drive either to :

• Reuse this 1To physical drive in another system
• Free up some space on your host if this 1To drive is a thick provisioned virtual hard drive

As a reminder, if this 1To drive is a virtual thin provisioned hard drive, you can reclaim the space by ensuring that your hypervisor is correctly configured to support discard on this drive, then run the fstrim command in the linux VM. The discard option in fstab would trim automatically

In the following example, the system is a debian VM using LVM and the hard drive will be changed from 32G to 10G.
My filesystem type on the root partition I’ll shrink is ext4 and I boot using UEFI

Before starting be sure you have a backup of your data. You could loose all your data if something goes wrong.
To be able to perform this operation, your filesystem should not be mounted.
In my case I’m shrinking the root filesystem so I’ll boot on a debian system installed on a second disk.
Another option would be to boot in the debian installation media and choose Advanced -> Rescue mode in the installer to get a shell and perform the commands. If you do so, ensure to answer “do not use a root file system” when the installer will ask what partition to use a the root filesystem.

Let’s start by identifying the drives and filesystem to reduce

root@debian:~# lsblk 
NAME                  MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sda                     8:0    0    7G  0 disk 
├─sda1                  8:1    0  512M  0 part /boot/efi
├─sda2                  8:2    0  5.5G  0 part /
└─sda3                  8:3    0  976M  0 part [SWAP]
sdb                     8:16   0   32G  0 disk 
├─sdb1                  8:17   0  512M  0 part 
├─sdb2                  8:18   0  488M  0 part 
└─sdb3                  8:19   0   31G  0 part 
  ├─debian--vg-root   254:0    0 30.1G  0 lvm  
  └─debian--vg-swap_1 254:1    0  976M  0 lvm  
sdc                     8:32   0   10G  0 disk 
root@debian:~# fdisk -l /dev/sdb
Disk /dev/sdb: 32 GiB, 34359738368 bytes, 67108864 sectors
Disk model: QEMU HARDDISK   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 0C445618-2828-451C-BBD8-3105C88F2D06

Device       Start      End  Sectors  Size Type
/dev/sdb1     2048  1050623  1048576  512M EFI System
/dev/sdb2  1050624  2050047   999424  488M Linux filesystem
/dev/sdb3  2050048 67106815 65056768   31G Linux LVM  

In my case the original hard drive is sdb and target is sdc.
sdb have 3 parition, sdb1 is the EFI parition, sdb2 is the boot partition and sdb3 is the LVM partition where my root and swap partition is located.

We will shrink “debian–vg-root” and move everything from sdb to sdc

Let’s see/confirm the current root filesystem size.

root@debian:~# mount /dev/debian-vg/root /mnt
root@debian:~# df -h | grep root
/dev/mapper/debian--vg-root   30G  1.7G   27G   6% /mnt
root@debian:~# umount /dev/debian-vg/root

To be able to reduce the logical volume root of the debian-vg volume group we must start be reducing the filesystem size.

We start by a forced fsck check on the filesystem

root@debian:~# e2fsck -f /dev/debian-vg/root 
e2fsck 1.47.0 (5-Feb-2023)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/debian-vg/root: 38223/1970416 files (0.2% non-contiguous), 590657/7879680 blocks

Then reduce your filesystem with the resize2fs command. The -M option will reduce as much as possible. This command could be very long to execute depending to the size of the filesystem

root@debian:~# resize2fs -M /dev/debian-vg/root 
resize2fs 1.47.0 (5-Feb-2023)
resize2fs 1.47.0 (5-Feb-2023)
Resizing the filesystem on /dev/debian-vg/root to 784869 (4k) blocks.
The filesystem on /dev/debian-vg/root is now 784869 (4k) blocks long.

Check the result

root@debian:~# mount /dev/debian-vg/root /mnt
root@debian:~# df -h | grep root
/dev/mapper/debian--vg-root  2.8G  1.7G  1.1G  62% /mnt
root@debian:~# umount /dev/debian-vg/root

Now let’s resize the LVM logical volume. Stay safe between the filesystem size and the lv size. A lv smaller than the filesystem will destroy it. Add at least ~1G to be safe. Here I resize to 6G as i want to keep enough free space on the root filesystem

root@debian:~# lvresize -L 6G /dev/debian-vg/root 
  WARNING: Reducing active logical volume to 6.00 GiB.
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce debian-vg/root? [y/n]: y
  Size of logical volume debian-vg/root changed from <30.06 GiB (7695 extents) to 6.00 GiB (1536 extents).
  Logical volume debian-vg/root successfully resized.

You can extend the filesystem to 100% of the logical volume if you want

root@debian:~# e2fsck -f /dev/debian-vg/root
e2fsck 1.47.0 (5-Feb-2023)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/debian-vg/root: 38223/196224 files (0.2% non-contiguous), 474182/784869 blocks
root@debian:~# resize2fs /dev/debian-vg/root 
resize2fs 1.47.0 (5-Feb-2023)
Resizing the filesystem on /dev/debian-vg/root to 1572864 (4k) blocks.
The filesystem on /dev/debian-vg/root is now 1572864 (4k) blocks long.

root@debian:~# mount /dev/debian-vg/root /mnt
root@debian:~# df -h | grep root
/dev/mapper/debian--vg-root  5.8G  1.7G  3.9G  30% /mnt
root@debian:~# umount /mnt 

Let’s run lsblk again

root@debian:~# lsblk 
NAME                  MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sda                     8:0    0    7G  0 disk 
├─sda1                  8:1    0  512M  0 part /boot/efi
├─sda2                  8:2    0  5.5G  0 part /
└─sda3                  8:3    0  976M  0 part [SWAP]
sdb                     8:16   0   32G  0 disk 
├─sdb1                  8:17   0  512M  0 part 
├─sdb2                  8:18   0  488M  0 part 
└─sdb3                  8:19   0   31G  0 part 
  ├─debian--vg-root   254:0    0    6G  0 lvm  
  └─debian--vg-swap_1 254:1    0  976M  0 lvm  
sdc                     8:32   0   10G  0 disk 

The sdb3 parition is still 31G but it’s content is only 6G+976M + 512M of sdb1 + 488M of sdb2 = ~8G. It will fit in sdc. We can start to move our data to the new disk.

Let’s see again the partition table of sdb

root@debian:~# fdisk -l /dev/sdb
Disk /dev/sdb: 32 GiB, 34359738368 bytes, 67108864 sectors
Disk model: QEMU HARDDISK   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 0C445618-2828-451C-BBD8-3105C88F2D06

Device       Start      End  Sectors  Size Type
/dev/sdb1     2048  1050623  1048576  512M EFI System
/dev/sdb2  1050624  2050047   999424  488M Linux filesystem
/dev/sdb3  2050048 67106815 65056768   31G Linux LVM

Then we need to replicate this on sdc. Of course the LVM partition will be smaller.
I start by creating a GPT disklabel (as I’m using EFI boot. If you are using “Legacy bios” you need to create a MBR disklabel with o instead of g and making the new system bootable will require extra step not explained here)

root@debian:~# fdisk /dev/sdc

Welcome to fdisk (util-linux 2.38.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS (MBR) disklabel with disk identifier 0x9cf3f81a.

Command (m for help): g
Created a new GPT disklabel (GUID: 3D38E71C-613D-5345-AA1B-8979A377E738).

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

Now we will replicate the partition table of sdb to sdc.

root@debian:~# fdisk /dev/sdc

Welcome to fdisk (util-linux 2.38.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): n
Partition number (1-128, default 1): 
First sector (2048-20971486, default 2048): 2048
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-20971486, default 20969471): +1048576

Created a new partition 1 of type 'Linux filesystem' and of size 512 MiB.

Command (m for help): t
Selected partition 1
Partition type or alias (type L to list all): 1
Changed type of partition 'Linux filesystem' to 'EFI System'.

Command (m for help): n
Partition number (2-128, default 2): 2
First sector (1050625-20971486, default 1052672): 1052672
Last sector, +/-sectors or +/-size{K,M,G,T,P} (1052672-20971486, default 20969471): +999424

Created a new partition 2 of type 'Linux filesystem' and of size 488 MiB.

Command (m for help): n
Partition number (3-128, default 3): 3
First sector (1050625-20971486, default 2054144): 2054144
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2054144-20971486, default 20969471): 

Created a new partition 3 of type 'Linux filesystem' and of size 9 GiB.

Command (m for help): t
Partition number (1-3, default 3): 3
Partition type or alias (type L to list all): lvm

Changed type of partition 'Linux filesystem' to 'Linux LVM'.

Command (m for help): p
Disk /dev/sdc: 10 GiB, 10737418240 bytes, 20971520 sectors
Disk model: QEMU HARDDISK   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 3D38E71C-613D-5345-AA1B-8979A377E738

Device       Start      End  Sectors  Size Type
/dev/sdc1     2048  1050624  1048577  512M EFI System
/dev/sdc2  1052672  2052096   999425  488M Linux filesystem
/dev/sdc3  2054144 20969471 18915328    9G Linux LVM

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

Let’s copy sdb1 to sdc1 and sdb2 to sdc2 (EFI and boot partition)

root@debian:~# dd if=/dev/sdb1 of=/dev/sdc1 bs=4k
131072+0 records in
131072+0 records out
536870912 bytes (537 MB, 512 MiB) copied, 3.80218 s, 141 MB/s
root@debian:~# dd if=/dev/sdb2 of=/dev/sdc2 bs=4k
124928+0 records in
124928+0 records out
511705088 bytes (512 MB, 488 MiB) copied, 4.3996 s, 116 MB/s

We will now extend the lvm physical volume

root@debian:~# vgextend debian-vg /dev/sdc3 
  Physical volume "/dev/sdc3" successfully created.
  Volume group "debian-vg" successfully extended

Then move the volume group from sdb3 to sdc3.

root@debian:~# pvmove -v /dev/sdb3 /dev/sdc3
  Executing: /sbin/modprobe dm-mirror
  Creating logical volume pvmove0
  activation/volume_list configuration setting not defined: Checking only host tags for debian-vg/root.
  Moving 1536 extents of logical volume debian-vg/root.
  activation/volume_list configuration setting not defined: Checking only host tags for debian-vg/swap_1.
  Moving 244 extents of logical volume debian-vg/swap_1.
  activation/volume_list configuration setting not defined: Checking only host tags for debian-vg/root.
  activation/volume_list configuration setting not defined: Checking only host tags for debian-vg/swap_1.
  Archiving volume group "debian-vg" metadata (seqno 5).
  Creating debian--vg-pvmove0
  Loading table for debian--vg-pvmove0 (254:2).
  Loading table for debian--vg-root (254:0).
  Loading table for debian--vg-swap_1 (254:1).
  Suspending debian--vg-root (254:0) with device flush
  Suspending debian--vg-swap_1 (254:1) with device flush
  Resuming debian--vg-pvmove0 (254:2).
  Resuming debian--vg-root (254:0).
  Resuming debian--vg-swap_1 (254:1).
  activation/volume_list configuration setting not defined: Checking only host tags for debian-vg/pvmove0.
  Creating volume group backup "/etc/lvm/backup/debian-vg" (seqno 6).
  Checking progress before waiting every 15 seconds.
  /dev/sdb3: Moved: 0.39%
  /dev/sdb3: Moved: 52.36%
  /dev/sdb3: Moved: 86.29%
  /dev/sdb3: Moved: 100.00%
  Polling finished successfully.

The last step is to remove sdb3 from the volume group

root@debian:~# pvs
  PV         VG        Fmt  Attr PSize   PFree  
  /dev/sdb3  debian-vg lvm2 a--  <31.02g <31.02g
  /dev/sdc3  debian-vg lvm2 a--   <9.02g   2.06g
root@debian:~# vgreduce debian-vg /dev/sdb3
  Removed "/dev/sdb3" from volume group "debian-vg"
root@debian:~# pvs
  PV         VG        Fmt  Attr PSize  PFree 
  /dev/sdb3            lvm2 ---  31.02g 31.02g
  /dev/sdc3  debian-vg lvm2 a--  <9.02g  2.06g

You can now halt the system, remove the sdb disk and boot on the new disk.
Depending on your OS and the UEFI, you may need to tell the EFI to boot on the right file on the new disk. For example, you use debian and proxmox, you will need to create a new boot entry

I was looking to upgrade the firmware of a MX500 SDD from M3CR043 to M3CR046

Crucial only provide update software for Windows. If you are using another system like Linux or macOS, you will have to create a bootable media from the ISO file provided by Crucrial (M3CR046_ISO.iso for this specific firmware version which can be downloaded from https://www.crucial.com/support/ssd-support/mx500-support (the ISO is inside the zip file))

I did not tried to burn the ISO to a CD (who still have CD-ROM in 2024 ? Not me.) may be it would have work directly. I took the option to create a bootable USB flash drive. It SHOULD be simple and easy. It was not (at least for me..)

I first created an USB key with belena etcher (it’s easy to use, work great and is multi plateform) but when I started my computer on the key, I got a “grub >” prompt

I then followed crucial documentation on how to create a bootable usb flash drive line by line, including using the Windows software they recommend to create the usb key. Guess what, it was not booting better.

I lost a bit of time to try many things I’ll not detail here but like another USB key, EFI vs MBR boot, some adjustment in the content of the USB key, etc…

So I decided to manualy load the linux os that is on the USB key with grub commands. This sounds a bit scary as if the software is not run correctly it’s unsure what can happen to the disk but:

• I was ok to lose data on this disk as it was part of a RAID 5 system and I have backup
• I deeply looked the content of the USB key (if you are curious, decompress corepure64.gz) and was confident of what I was going to do
• I made boot test without any disk connected

So inside the key, the boot/isolinux/isolinux.cfg has the following content

DEFAULT tc
PROMPT 0
TIMEOUT 0
LABEL tc
KERNEL /boot/vmlinuz64
INITRD /boot/corepure64.gz
APPEND libata.allow_tpm=1 quiet base loglevel=3 waitusb=10 superuser mse-iso rssd-fw-update rssd-fwdir=/opt/firmware    

If you never used grub, it difficult to understand, but if you are a bit familiar with grub you see that the last 3 lines give us the information to give to grub. What we miss is the identifier of the USB key to boot on.

If you run ls in grub you will see an output like

grub> ls
(hd0)

If you SSD is connected (which is needed at some point if you want to update the firmware) you will see more then 1 entry when doing the ls command. To know which “hd” is the right to boot on run ls (hd0)/ on each entry you see, so that may also be ls (hd1)/, ls (hd1,1)/, ls (hd0,msdos1)/, etc. Most likely if you see an entry like (hdX,…) (entry with a “,”) this is likely not the good entry. The output of the right entry should look like this

grub> ls (hd0)/
boot/ efi/

I highly recommend to first try to boot without any disk inside your computer. This way you will only see the USB key when running ls and you will be sure to run ls (hd...)/ on the USB key. When your SDD will be in the computer (hd0) which was likely your USB key before may now be your SDD and the USB key may be (hd1) or something else. But by running ls (hd...) on all entry and by comparing the result with the result when only the USB key was connected, you are sure to identify easily the right (hd…).

To boot, enter the following line. Replace hd0 by the right hdx for you. Of course do not type “grub >”

grub> linux (hd0)/boot/vmlinuz64 root=ram1 libata.allow_tpm=1 base loglevel=3 waitusb=10 superuser mse-iso rssd-fw-update rssd-fwdir=/opt/firmware 
grub> initrd (hd0)/boot/corepure64.gz
grub> boot

If it doesn’t boot correctly, you may try to change root=ram1 by another ram number like root=ram0, root=ram2, etc..

If you are looking a way to install ESXi with less than 4GB of memory, you’re on the right place.

Before starting, please note that ESXi 7 will use about 1.4GB for itself, so if you have less than 4GB of memory, you will end up with a very limited amount of RAM to run VMs. But that may be enough for a test system.

Like me, when googling on how to solve the issue, you probably ended up on this page https://open-sourced.be/installing-esxi-with-less-than-4gb-of-ram/

The problem is that with ESXi 7 you will not be able to edit update_precheck.py because of filesystem restriction preventing to alter files.
Please note that even if the file is called update_precheck.py it apply to fresh installation.
It’s interesting to see in update_precheck.py that the actual requirement is not 4GB but 4GB minus 3.125% = 4096M – 128MB = 3968MB
My system do have a 4GB of RAM but 192MB is used for the integrated video card, leaving me with 3804MB seen by the OS.

def memorySizeComparator(found, expected):
    '''Custom memory size comparator
    Let minimum memory go as much as 3.125% below MEM_MIN_SIZE.
    See PR 1229416 for more details.
    '''
    return operator.ge(found[0], expected[0] - (0.03125 * expected[0]))

def checkMemorySize():
    '''Check that there is enough memory
    '''
    mem = vmkctl.HardwareInfoImpl().GetMemoryInfo()
    if hasattr(mem, 'get'):
       mem = mem.get()
    found = mem.GetPhysicalMemory()

    MEM_MIN_SIZE = (4 * 1024) * SIZE_MiB
    return Result("MEMORY_SIZE", [found], [MEM_MIN_SIZE],
                  comparator=memorySizeComparator,
                  errorMsg="The memory is less than recommended",
                  mismatchCode = Result.ERROR)

If we can’t modify the file during the installation, let’s do it on the installation media.
To do that, you will need another computer (likely the one with which you created the installation media), a tool to compress/decompress the gzip format and an hexadecimal editor.

In my case I did the modification on macOS. On the root directory of the installation media, you will found a file called WEASELIN.V00, this file is a tar file compressed with gzip. Let’s uncompress it

$ file WEASELIN.V00 
WEASELIN.V00: gzip compressed data, max compression, original size modulo 2^32 2548792
$ mv WEASELIN.V00 WEASELIN.V00.gz
$ gzip -d WEASELIN.V00.gz 
$ file WEASELIN.V00 
WEASELIN.V00: tar archive

For a reason I did not look for, if you try to untar the file, you will get errors

$ tar xf WEASELIN.V00 
tar: Damaged tar archive
tar: Retrying...

If we can’t untar the file to directly edit upgrade_precheck.py, let’s edit the tar archive directly as a tar file is a collection of file “as is” (I mean there is no compression, no encoding, etc.)

So open WEASELIN.V00 in your favorite hexadecimal editor, in my case i used Hex Friend

Search for MEM_MIN, you will see MEM_MIN_SIZE = (4 * 1024) * SIZE_MiB, edit 4 by something fiting your need. In my case I changed 4 by 3

Save your change and compress WEASELIN.V00

$ gzip WEASELIN.V00 
$ mv WEASELIN.V00.gz WEASELIN.V00

Your are good to eject the installation media (close your terminal windows if you’re in your installation media directory) and retry the installation. Enjoy

Normaly when you use PHP as CGI, $_GET and $_POST should work as usual.
But in certain configuration, they may not be set. In this case this code is the base to get args

foreach (split('&',$QUERY_STRING) as $tmp) {
$tmp2 = split('=',$tmp);
$$tmp2[0]=$tmp2[1];
}