Hi,

We're running an Xserve with Xserve RAID in a mixed Solaris/Linux
Environment. Version Mac OS X Server 10.3.2 plus all patches up to
yesterday.

When copying large files (more than hundred MB up to 3 GB) via NFS
over the network or from DVD to the RAID or even on the same disk the
files get screwed up (MD5 checksums differ).

See:

xserve01:/Volumes/NFS/sw/OLU/PLATFORM/1.12.0.0 root# cp
OLU-MONPAT_4.2.0.10.gtar.gz OLU-MONPAT_4.2.0.10.gtar.gz-1
xserve01:/Volumes/NFS/sw/OLU/PLATFORM/1.12.0.0 root# cp
OLU-MONPAT_4.2.0.10.gtar.gz OLU-MONPAT_4.2.0.10.gtar.gz-2
xserve01:/Volumes/NFS/sw/OLU/PLATFORM/1.12.0.0 root# cp
OLU-MONPAT_4.2.0.10.gtar.gz OLU-MONPAT_4.2.0.10.gtar.gz-3
xserve01:/Volumes/NFS/sw/OLU/PLATFORM/1.12.0.0 root# cp
OLU-MONPAT_4.2.0.10.gtar.gz OLU-MONPAT_4.2.0.10.gtar.gz-4
xserve01:/Volumes/NFS/sw/OLU/PLATFORM/1.12.0.0 root# cp
OLU-MONPAT_4.2.0.10.gtar.gz OLU-MONPAT_4.2.0.10.gtar.gz-5

xserve01:/Volumes/NFS/sw/OLU/PLATFORM/1.12.0.0 root# ls -al
OLU-MONPAT_4.2.0.10.gtar.gz*
-r-xr-xr-x 1 root adnstaff 264891835 25 Feb 10:43
OLU-MONPAT_4.2.0.10.gtar.gz
-r-xr-xr-x 1 root adnstaff 264891835 25 Feb 11:22
OLU-MONPAT_4.2.0.10.gtar.gz-1
-r-xr-xr-x 1 root adnstaff 264891835 25 Feb 11:22
OLU-MONPAT_4.2.0.10.gtar.gz-2
-r-xr-xr-x 1 root adnstaff 264891835 25 Feb 11:22
OLU-MONPAT_4.2.0.10.gtar.gz-3
-r-xr-xr-x 1 root adnstaff 264891835 25 Feb 11:22
OLU-MONPAT_4.2.0.10.gtar.gz-4
-r-xr-xr-x 1 root adnstaff 264891835 25 Feb 11:23
OLU-MONPAT_4.2.0.10.gtar.gz-5

xserve01:/Volumes/NFS/sw/OLU/PLATFORM/1.12.0.0 root# md5
OLU-MONPAT_4.2.0.10.gtar.gz*
MD5 (OLU-MONPAT_4.2.0.10.gtar.gz) = aa4fc2117b34fe4d5e7ced16ae5c7dab
MD5 (OLU-MONPAT_4.2.0.10.gtar.gz-1) = be1d8b8bf5d5ff14ac1330446be1e7a7
MD5 (OLU-MONPAT_4.2.0.10.gtar.gz-2) = dc1b5f52e2622230f273b38434836466
MD5 (OLU-MONPAT_4.2.0.10.gtar.gz-3) = aa4fc2117b34fe4d5e7ced16ae5c7dab
MD5 (OLU-MONPAT_4.2.0.10.gtar.gz-4) = 930a5770fc584d08738ab92eb5e76018
MD5 (OLU-MONPAT_4.2.0.10.gtar.gz-5) = aa4fc2117b34fe4d5e7ced16ae5c7dab

This results in CRC errors when gunzipping the files. Smaller files up
to a few MB seem to get less corrupted.

Can someone else confirm this behaviour? Does somebody know a fix for
this? Thanks in advance.

Regards,
Bernd

The file corruption happens even on local disk by just reading the
file. In the /tmp directory on the root disk I created several files
containing the character 'x' and file sizes by the power of two up to
4GB.

xserve01:/tmp root# echo -n x >2^00
xserve01:/tmp root# cat 2^00 2^00 >2^01
xserve01:/tmp root# cat 2^01 2^01 >2^02
xserve01:/tmp root# cat 2^02 2^02 >2^03
....
xserve01:/tmp root# cat 2^31 2^31 >2^32

xserve01:/tmp root# ls -al
....
-rw-r--r-- 1 root wheel 536870912 26 Feb 08:32 2^29
-rw-r--r-- 1 root wheel 1073741824 26 Feb 08:33 2^30
-rw-r--r-- 1 root wheel 2147483648 26 Feb 08:35 2^31
-rw-r--r-- 1 root wheel 4294967296 26 Feb 08:38 2^32

Then I calculated two times five MD5 checksums of the 500MB file at
once:

xserve01:/tmp root# md5 2^29 2^29 2^29 2^29 2^29
MD5 (2^29) = 6a8a2c385690377f8198dc85e1859fad
MD5 (2^29) = 6a8a2c385690377f8198dc85e1859fad
MD5 (2^29) = 6a8a2c385690377f8198dc85e1859fad
MD5 (2^29) = 6a8a2c385690377f8198dc85e1859fad
MD5 (2^29) = 6a8a2c385690377f8198dc85e1859fad

xserve01:/tmp root# md5 2^29 2^29 2^29 2^29 2^29
MD5 (2^29) = 6a8a2c385690377f8198dc85e1859fad
MD5 (2^29) = 6a8a2c385690377f8198dc85e1859fad
MD5 (2^29) = 6a8a2c385690377f8198dc85e1859fad
MD5 (2^29) = 6a8a2c385690377f8198dc85e1859fad
MD5 (2^29) = 6a8a2c385690377f8198dc85e1859fad

They are identical as they should be. Same procedure with the 1GB
file:

xserve01:/tmp root# md5 2^30 2^30 2^30 2^30 2^30
MD5 (2^30) = 2b7e1ada49850d0da6180167ae78b4d8
MD5 (2^30) = b401e84f99ef1841320f08df07a97923
MD5 (2^30) = fcf63edefb38bf86984badee25b382d7
MD5 (2^30) = c07816c7d758e30ac0bef2e877c25f71
MD5 (2^30) = d7496bb9aff1b72d290c83d2d7f6dc5a

xserve01:/tmp root# md5 2^30 2^30 2^30 2^30 2^30
MD5 (2^30) = 2b13477852f8831f497b18593690ebd0
MD5 (2^30) = 701269930107061185ea77f37b8ec8e3
MD5 (2^30) = e5a3668daf758b25abe664f14325739a
MD5 (2^30) = c7fdbfd3bc370f52f7112691799cbdd2
MD5 (2^30) = 98e91bd869f66f43425b680cc46cc2ef

Cool random number generator ... :-)

One could think that the md5 program is broken (we had this once on
Solaris) but we also get CRC errors when unzipping/untarring a tarball
and when I hexdump the file I get:

xserve01:/tmp root# od -x 2^30
....
1173554740 7878 7878 7878 7078 7878 7878 7878
7878
1173554760 7878 7878 7878 6878 7878 7878 7878
7878
1173555000 7878 7878 7878 6078 7878 7878 7878
7878
1173555020 7878 7878 7878 7878 7878 7878 7878
7878
....

There should be everything hex 78. And it happens again at the same
file position, so it's not only a reading fault:

xserve01:/tmp root# od -x 2^30 |grep 1173554760
1173554760 7878 7878 7878 6878 7878 7878 7878
7878
^C
xserve01:/tmp root# od -x 2^30 |grep 1173554760
1173554760 7878 7878 7878 6878 7878 7878 7878
7878

xserve01:/tmp root# od -x 2^30 |grep 1173554760
1173554760 7878 7878 7878 6878 7878 7878 7878
7878


Regards,
Bernd