ext keeps going read only

During backups, I get this, and the root filesystem goes read only.

I’ve replaced disks, rebuilt filesystems, arrays, LVM, etc.

 

dmesg shows:

[19501.355932] EXT4-fs error (device dm-7): ext4_get_verity_descriptor_location:295: inode #6032: comm dsmc: verity file doesn’t use extents
[19501.403496] Aborting journal on device dm-7-8.
[19501.414969] EXT4-fs (dm-7): Remounting filesystem read-only
[19501.414974] fs-verity (dm-7, inode 6032): Error -117 getting verity descriptor size

 

Find the file by inode:

find / -xdev -inum 6032 -print

 

This showed that there is junk in lost+found from when I had FS corruption.  I rebuilt the root filesystem, but never cleaned out lost+found.  Deleting the damaged files should solve the problem.


Convert EXT3 to EXT4

### Change to EXT4 mount mode (OKAY before conversion)
vi /etc/fstab

### Reboot into single user mode
shutdown -r now
LILO: linux S

### Unmount or read-only every filesystem
umount -a
mount -oremount,ro /usr
mount -oremount,ro /

### Convert all ext4 into new metadata formats
grep ext4 /etc/fstab | tr -s [:space:] | cut -f 1 -d \ | tune2fs -O extents,uninit_bg,dir_index

### Build the directory index and verify metadata
grep ext4 /etc/fstab | tr -s [:space:] | cut -f 1 -d \ | fsck.ext4 -yfD

### Reboot back to multiuser mode
shutdown -r now

### Covert all files in EXT4 filesystems to extent mode (was bitmap)
for dir in `mount | grep ext4 | cut -f 3 -d \ ` ; do LC_ALL=C find $dir -xdev -type d -print0 | LC_ALL=C xargs -r0 -P3 chattr +e ; done
for dir in `mount | grep ext4 | cut -f 3 -d \ ` ; do LC_ALL=C find $dir -xdev -type f -print0 | LC_ALL=C xargs -r0 -P3 chattr +e ; done

### References
* https://debian-administration.org/article/643/Migrating_a_live_system_from_ext3_to _ext4_filesystem
* http://unix.stackexchange.com/questions/131535/recursive-grep-vs-find-type-f-exec-grep-which-is-more-efficient-faster