Which linux file system would checksum every cluster
it writes onto the disk (thus allowing you to
verify the saved data easily)?

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (Xubuntu 8.04) Linux 2.6.25.5
^ ^ 19:07:01 up 5:37 1 user load average: 1.01 1.04 1.13
? ? (CSSA):
http://www.swd.gov.hk/tc/index/site_...ub_addressesa/

"Man-wai Chang ToDie (33.6k)" <toylet.toylet******.com> wrote in message
news:484a6c3a$1@127.0.0.1...
>
> Which linux file system would checksum every cluster
> it writes onto the disk (thus allowing you to
> verify the saved data easily)?

Do you mean in addition to the ECC done by the disk drive which works on all
the common file systems?

You can set a read after write verification check on some file systems but
that only checks that the data in the system buffer is written correctly,
not that the application has written the data correctly.

If the data is really critical a read after write check by the application
is what is needed.
Be sure to flush the cache before the verify or you will just check the
contents of RAM and not what's on the disk.

Consider a database to store the data, they tend to be able to handle errors
and checking better.

What is the application?

> Do you mean in addition to the ECC done by the disk drive which works on
> all the common file systems?

Even those cheap consumer-grade hard disks have this ECC thing?

> If the data is really critical a read after write check by the
> application is what is needed.
> Be sure to flush the cache before the verify or you will just check the
> contents of RAM and not what's on the disk.

I meant both the data and the checksum are stored in the
file system so that you don't need the original file
to verify the correctness of the data after a copy
process.

> What is the application?

Nothing, just backing up data.

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (Xubuntu 8.04) Linux 2.6.25.5
^ ^ 20:36:01 up 7:06 1 user load average: 1.00 1.02 1.00
? ? (CSSA):
http://www.swd.gov.hk/tc/index/site_...ub_addressesa/

"Man-wai Chang ToDie (33.6k)" <toylet.toylet******.com> wrote in message
news:484a815f@127.0.0.1...
>> Do you mean in addition to the ECC done by the disk drive which works on
>> all the common file systems?
>
> Even those cheap consumer-grade hard disks have this ECC thing?

Yes. They wouldn't work very well without it.
You can find the error rate in the disk specs and compare them if you need
to.

>
>> If the data is really critical a read after write check by the
>> application is what is needed.
>> Be sure to flush the cache before the verify or you will just check the
>> contents of RAM and not what's on the disk.
>
> I meant both the data and the checksum are stored in the
> file system so that you don't need the original file
> to verify the correctness of the data after a copy
> process.
>
>> What is the application?
>
> Nothing, just backing up data.

I think you need to change your question to

"Which backup application does a proper verification and keeps checksums in
its image?" so I have.

You can do it with tar if you write a script,
but there is bound to be something better that I haven't seen.

tar is an ancient utility that has been used to create archives on unix
since the year dot so it is known to be reliable.

I use a utility (Memeo) that backs up changed files on the fly on windows, I
don't know if there is a similar application for linux.

"Man-wai Chang ToDie (33.6k)" <toylet.toylet******.com> writes:

>> Do you mean in addition to the ECC done by the disk drive which works on
>> all the common file systems?

>Even those cheap consumer-grade hard disks have this ECC thing?

Oh yes, masses of it. It can correct huge native error rates. That is one
way they squeeze every last bit out of those drives.


>> If the data is really critical a read after write check by the
>> application is what is needed.
>> Be sure to flush the cache before the verify or you will just check the
>> contents of RAM and not what's on the disk.

>I meant both the data and the checksum are stored in the
>file system so that you don't need the original file
>to verify the correctness of the data after a copy
>process.

Do you want to just check it, or correct it as well. The latter is much
harder and more extensive in space.


>> What is the application?

>Nothing, just backing up data.

That is probably something in your backup program-- eg storing an MD4 sum
with every file of that file. Of course what do you do if the checksum says
the file is corrupted?

In <484a6c3a$1@127.0.0.1>, on Sat, 07 Jun 2008 19:09:09 +0800, Man-wai
Chang ToDie (33.6k), toylet.toylet******.com wrote:
>
> Which linux file system would checksum every cluster
> it writes onto the disk (thus allowing you to
> verify the saved data easily)?

Sounds like zfs would probably fit your bill.
http://www.sun.com/software/solaris/ds/zfs.jsp
Available on Solaris, FreeBSD, Mac, and probably a
couple others. Not available for Linux though (yet?).

Steve Ackman wrote:
> In <484a6c3a$1@127.0.0.1>, on Sat, 07 Jun 2008 19:09:09 +0800, Man-wai
> Chang ToDie (33.6k), toylet.toylet******.com wrote:
>> Which linux file system would checksum every cluster
>> it writes onto the disk (thus allowing you to
>> verify the saved data easily)?
>
> Sounds like zfs would probably fit your bill.
> http://www.sun.com/software/solaris/ds/zfs.jsp
> Available on Solaris, FreeBSD, Mac, and probably a
> couple others. Not available for Linux though (yet?).

There is a Linux port, look for zfs-on-fuse.

--

//Aho

> That is probably something in your backup program-- eg storing an MD4 sum
> with every file of that file. Of course what do you do if the checksum says
> the file is corrupted?

For now I am using 7-zip....

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (Xubuntu 8.04) Linux 2.6.25.5
^ ^ 11:36:01 up 22:06 1 user load average: 1.01 1.01 1.00
? ? (CSSA):
http://www.swd.gov.hk/tc/index/site_...ub_addressesa/

> "Which backup application does a proper verification and keeps checksums
> in its image?" so I have.
> You can do it with tar if you write a script,
> but there is bound to be something better that I haven't seen.

Don't think tar could handle double-byte big5 file names...

> I use a utility (Memeo) that backs up changed files on the fly on
> windows, I don't know if there is a similar application for linux.

I am using 7-zip now (store mode).

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (Xubuntu 8.04) Linux 2.6.25.5
^ ^ 11:36:01 up 22:06 1 user load average: 1.01 1.01 1.00
? ? (CSSA):
http://www.swd.gov.hk/tc/index/site_...ub_addressesa/

>> Sounds like zfs would probably fit your bill.
>> http://www.sun.com/software/solaris/ds/zfs.jsp
>> Available on Solaris, FreeBSD, Mac, and probably a
>> couple others. Not available for Linux though (yet?).
> There is a Linux port, look for zfs-on-fuse.

Does XFS have similar functionality?

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (Xubuntu 8.04) Linux 2.6.25.5
^ ^ 11:35:01 up 22:05 1 user load average: 1.04 1.01 1.00
? ? (CSSA):
http://www.swd.gov.hk/tc/index/site_...ub_addressesa/

Man-wai Chang ToDie (33.6k) wrote:
>>> Sounds like zfs would probably fit your bill.
>>> http://www.sun.com/software/solaris/ds/zfs.jsp
>>> Available on Solaris, FreeBSD, Mac, and probably a
>>> couple others. Not available for Linux though (yet?).
>> There is a Linux port, look for zfs-on-fuse.
>
> Does XFS have similar functionality?
>
Read this: http://oss.sgi.com/projects/xfs/papers/ukuug2003.pdf

If you want another industrial file system, then check here:
jfs.sourceforge.net/

--

//Aho

Man-wai Chang ToDie (33.6k) wrote:
>> "Which backup application does a proper verification and keeps
>> checksums in its image?" so I have.
>> You can do it with tar if you write a script,
>> but there is bound to be something better that I haven't seen.
>
> Don't think tar could handle double-byte big5 file names...

Works fine with UTF-8 and works like a charm using the fonts downloaded
from the HK-administrations homepage.


>> I use a utility (Memeo) that backs up changed files on the fly on
>> windows, I don't know if there is a similar application for linux.
>
> I am using 7-zip now (store mode).

There are loads of backup tools and a good number of those are multi
platform, so you can backup stuff on one system and then restore on
another, just find the one which full fills your criteria.
http://freshmeat.net/search/?q=backu...&Go.x=0&Go.y=0

--

//Aho

>>>> Sounds like zfs would probably fit your bill.
>>>> http://www.sun.com/software/solaris/ds/zfs.jsp
>> Does XFS have similar functionality?
> Read this: http://oss.sgi.com/projects/xfs/papers/ukuug2003.pdf
> If you want another industrial file system, then check here:
> jfs.sourceforge.net/

It would be fun if all 3 file systems are available
to Window$. :)

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (Xubuntu 8.04) Linux 2.6.25.5
^ ^ 15:16:01 up 1 day 1:46 1 user load average: 1.03 1.05 1.01
? ? (CSSA):
http://www.swd.gov.hk/tc/index/site_...ub_addressesa/

"Man-wai Chang ToDie (33.6k)" <toylet.toylet******.com> wrote in message
news:484b53b6$1@127.0.0.1...
>>> Sounds like zfs would probably fit your bill.
>>> http://www.sun.com/software/solaris/ds/zfs.jsp
>>> Available on Solaris, FreeBSD, Mac, and probably a
>>> couple others. Not available for Linux though (yet?).
>> There is a Linux port, look for zfs-on-fuse.
>
> Does XFS have similar functionality?

Don't make the mistake of thinking such functionality is a substitute for a
proper backup.
You need to do a proper verify of the archive *after* its creation.
There were many backups lost on tape drives who thought the read after write
check on some of them was good enough.

> Don't make the mistake of thinking such functionality is a substitute
> for a proper backup.
> You need to do a proper verify of the archive *after* its creation.
> There were many backups lost on tape drives who thought the read after
> write check on some of them was good enough.

Thanks for the reminder.

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (Xubuntu 8.04) Linux 2.6.25.5
^ ^ 19:48:01 up 1 day 6:18 1 user load average: 1.10 1.05 1.01
? ? (CSSA):
http://www.swd.gov.hk/tc/index/site_...ub_addressesa/