0x8007003B timeout copying large file to Samba server

2023-03-15Posted by xaminmo

PROBLEM: 0x8007003B timeout copying large file to Samba server

SOLUTION: This is an SMB.CONF issue, solved / fixed with this line:

strict allocate = no

DESCRIPTION: I had this issue for a long time, and mostly the web mocks people, tells them to do stupid things, or generally is unhelpful. Lots of 2GB, or “your network” or “your firewall” or “turn off DPI” or whatever, none of it applicable to me. I just accepted it, but decided to dig a little deeper today.

The exact amount of data written before it fails would vary, but the size from LS would always be the full file size. Higher performance filesystems such as XFS, EXT4, JFS, all of them on NVMe arrays, I found I could get about 55GB allocated before timeout. On spinning disk, it was much less, which is probably why many people fell down the rabbit hole of claiming 2GB limits, etc.

Strict Allocate = YES tells it to allocate the whole file upon request, which is what Windows does. Samba says “OK, hold on”, and then times out. Some people used powershell on a client to change the smbclient timeout to 600 seconds, or whatever, but that’s not really ideal, since it does not scale.

Strict Allocate = NO says to use normal UNIX semantics, where the file has no pre-allocated blocks, and allocates blocks only as the data comes in. This starts with a fully sparse file, and data copy status on the windows client shows it processing immediately. This is what we want for large files. If it was only small files, then we don’t care.

I made this a global change. I don’t need fully pre-allocated, non-sparse files on my file server. It’s possible someone writing databases might need this, and you’d want to make sure you didn’t feed data faster than the kernel can allocate blocks. Another one of those multiple filesystems kind of solutions.

When you play with tunables, you run into things that people don’t really know how to troubleshoot. That’s what this is for, just so it shows up in web searches.

AIX JFS2 autoresize

2018-01-06Posted by xaminmo

computersarefun put in a request for AIX to auto-grow/shrink filesystems.
Ref: https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=114789

This seems more like a monitoring thing than an operating system thing.
Also, handling this as a thin LUN is probably better where possible.
Here is an example script.

Potential improvements:
* Notifications on exceptions
* Config file to track different settings per filesystem
* Also check iused / ifree to handle tiny-files
* Run as a daemon vs from cron.
* Explicit lists of filesystems, or include/exclude lists

#!/bin/ksh
###########
# Run this from cron every minute to automatically resize JFS2 filesystems
# Incorrect limits could cause size flapping for small filesystems.
# We skip things we cannot reduce.

MINFREEPCT=10
MAXFREEPCT=70
MINPPFREE=10

LVLIST=`mount | grep jfs2 | grep /dev/ | awk ‘{print $1;}’ | cut -f 3 -d /`
for lv in $LVLIST ; do
df -gv 2>/dev/null | grep $lv | read device size used free pct iused ifree ipct mountpoint || continue
FREEPCT=$(( $used * 100 / $size ))
VG=`lslv $lv 2>/dev/null | grep “VOLUME GROUP:” | awk ‘{print $6;}’`
PPSIZE=`lsvg $VG 2>/dev/null | grep ‘PP SIZE’ | awk ‘{print $6;}’`
[[ $PPSIZE -gt 0 ]] || continue
#
if [[ $FREEPCT -lt $MINFREEPCT ]] ; then
FREEPPS=`lsvg $VG | grep FREE | awk ‘{print $6;}’`
[[ $FREEPPS -gt $MINFREEPPS ]] && chfs -a size=+1 $mountpoint
continue
fi
#
[[ $FREEPCT -gt $MAXFREEPCT ]] && chfs -a size=-$PPSIZE $mountpoint
#
done

Convert EXT3 to EXT4

2017-01-05Posted by xaminmo

### Change to EXT4 mount mode (OKAY before conversion)
vi /etc/fstab

### Reboot into single user mode
shutdown -r now
LILO: linux S

### Unmount or read-only every filesystem
umount -a
mount -oremount,ro /usr
mount -oremount,ro /

### Convert all ext4 into new metadata formats
grep ext4 /etc/fstab | tr -s [:space:] | cut -f 1 -d \ | tune2fs -O extents,uninit_bg,dir_index

### Build the directory index and verify metadata
grep ext4 /etc/fstab | tr -s [:space:] | cut -f 1 -d \ | fsck.ext4 -yfD

### Reboot back to multiuser mode
shutdown -r now

### Covert all files in EXT4 filesystems to extent mode (was bitmap)
for dir in `mount | grep ext4 | cut -f 3 -d \ ` ; do LC_ALL=C find $dir -xdev -type d -print0 | LC_ALL=C xargs -r0 -P3 chattr +e ; done
for dir in `mount | grep ext4 | cut -f 3 -d \ ` ; do LC_ALL=C find $dir -xdev -type f -print0 | LC_ALL=C xargs -r0 -P3 chattr +e ; done

### References
* https://debian-administration.org/article/643/Migrating_a_live_system_from_ext3_to _ext4_filesystem
* http://unix.stackexchange.com/questions/131535/recursive-grep-vs-find-type-f-exec-grep-which-is-more-efficient-faster