Hi I have been looking for some software that is similar to teracopy (which is only for win) for linux. More importantly something that can recursively copy and check (like CRC/Size/etc) as it copies (or after it copies). Any suggestions would really be appreciated!
Well the Pausing part is nice but really i was focusing more on the ability to do CRC checks (and for whatever reasong, in win at least, teracopy is much faster than win copy)
Thanks, i took a look and the GUI part of it is nice. Forgive the noob question but does rsync have some sort of robust copy verification option? I saw it had something about checksum but it was a bit ambigous (I was hoping for a robust algorithms like CRC, or Fletcher’s checksum or Adler-32)
I knwow its a long time ago but have you found something like teracopy for Linux?
I’m also searching for this copy-program.
For people who don’t know teracopy for windows: It takes over the windows copy completely (copy-past, drag-drop everything). It is a lot faster then normal windows actions. It has pause-resume (not very important) but the most important thing it does is: It checks the copied files after copying if it is exactly the same with a checksum or hash sum.
If something does not exists: is there a program that can check files if they are exactly the same with a checksum or hash sum or md5 for whole directory’s?
On 01/31/2013 07:46 AM, sir chris pecked at the keyboard and wrote:
> I knwow its a long time ago but have you found something like teracopy
> for Linux?
> I’m also searching for this copy-program.
>
> For people who don’t know teracopy for windows: It takes over the
> windows copy completely (copy-past, drag-drop everything). It is a lot
> faster then normal windows actions. It has pause-resume (not very
> important) but the most important thing it does is: It checks the copied
> files after copying if it is exactly the same with a checksum or hash
> sum.
>
> If something does not exists: is there a program that can check files
> if they are exactly the same with a checksum or hash sum or md5 for
> whole directory’s?
>
>
>>> rsync can do all of that.
>>
>> I don’t see a verify write option in rsync. :-?
>
> How do you think it decides whether files need updating?
Just to be pedantic, by default it does it by modify timestamp and
filesize (both must match in order for the file to be declared the same)
which is a good, easy, fast start. Be sure to enable the checksum
argument, though, to actually verify the checksum. From the manpage:
-c, --checksum skip based on checksum, not mod-time & size
On 2013-01-31 14:42, Dave Howorth wrote:
> Carlos E. R. wrote:
>> On 2013-01-31 14:19, Ken Schneider wrote:
>>> rsync can do all of that.
>>
>> I don’t see a verify write option in rsync. :-?
>
> How do you think it decides whether files need updating?
That’s not the same.
–
Cheers / Saludos,
Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)
On 2013-01-31 15:15, ab wrote:
>>>> rsync can do all of that.
>>>
>>> I don’t see a verify write option in rsync. :-?
>>
>> How do you think it decides whether files need updating?
>
> Just to be pedantic, by default it does it by modify timestamp and
> filesize (both must match in order for the file to be declared the same)
> which is a good, easy, fast start. Be sure to enable the checksum
> argument, though, to actually verify the checksum. From the manpage:
>
> -c, --checksum skip based on checksum, not mod-time & size
Right.
But what it does not do is, after writing the file, verify again that
the copy matches the original. That’s what “verify” means.
And of course, the verify means an actual read from the disk, not from
the kernel memory cached copy.
–
Cheers / Saludos,
Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)
A second run, then, this time with checksums for verification. Trivial.
But it’ll be slower, you might argue.
I doubt it; one way or another the data get there and are checksummed;
doing it separately probably just has a better chance of verifying it
comes from disk vs. disk cache in memory.
The zealot side of me would toss out something like, "Linux does all of
this copying-without-corruption by default.
On 2013-01-31 16:13, ab wrote:
> A second run, then, this time with checksums for verification. Trivial.
Yes, that’s what I do when I can. It also catches some of the files that
were open on the first run.
> But it’ll be slower, you might argue.
No, a verify run has to be done after the backup has finished, and the
cache flushed. It is about the same speed.
> The zealot side of me would toss out something like, "Linux does all of
> this copying-without-corruption by default.
In MsDOS times my preferred backup software came from PCtools
(commercial). It did compression, verification, and forward error
correction. Plus, it was astonishingly fast. I have not seen anything
comparable in Linux, yet.
It was so reliable that backups sets of 90 floppies made about 1995
still work without errors - last time I tried was few years ago. I need
time to recover those and transfer to some other media before the
hardware breaks. But one is so lazy when I think of it…
–
Cheers / Saludos,
Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)
I’m don’t think that rsync does the same: check the file after copy with an hash code. Wikipedia says something over hashes but is is used to check if parts of the file had been changed and then just copy that part of a file. This is not the same…?
http://en.wikipedia.org/wiki/Rsync
I don’t think that checking the size and the timestamp is enough. In windows 7 I have seen a lot of different checksums (in files that were not changed of course). Both the files could be opened and seemed the same… But something has changed.
On Thu, 31 Jan 2013 18:16:01 +0000, sir chris wrote:
> I’m don’t think that rsync does the same: check the file after copy with
> an hash code. Wikipedia says something over hashes but is is used to
> check if parts of the file had been changed and then just copy that part
> of a file. This is not the same…?
>
> Code:
> --------------------
> http://en.wikipedia.org/wiki/Rsync
> --------------------
>
> I don’t think that checking the size and the timestamp is enough. In
> windows 7 I have seen a lot of different checksums (in files that were
> not changed of course). Both the files could be opened and seemed the
> same… But something has changed.
From the rsync man page:
–append-verify
This works just like the --append option, but the existing
data on the receiving side is included in the full-file checksum
verification step, which will cause a
file to be resent if the final verification step fails
(rsync uses a normal, non-appending --inplace transfer for the resend).
Note: prior to rsync 3.0.0, the --append option worked
like --append-verify, so if you are interacting with an older rsync (or
the transfer is using a protocol
prior to 30), specifying either append option will initiate
an --append-verify transfer.
Thanks, i ran the command: rsync -r -t -v --progress -c -s path1 path2 (with help from Grsync).
It was 33.2 GiB in 45 minutes. The second time (without -c) took 6 seconds.
Its a nice tool for backups. It is not the same as teracopy but close.
FWIW
It’s why my preferred choice for transferring very large files is bit torrent. “Better” torrent clients will automatically checksum every individual transferred “chunk” as well as the final file when put together (not all clients do this, read the small print). So, transfers can be very accurate and efficient. It becomes almost impossible for even a small, altered part of a file to be injected or otherwise become part of the transferred copy, and discarding bad chunks is fast without using significant resources.
Normal file transfers on the other hand might wait until an entire file is transferred before performing an integrity check, and that means that a problem is discovered only after an entire file transfer has been attempted, and then would have to be attempted again.
Implementing torrent should be very easy, almost any torrent client can create a torrent file that defines how the file is chopped up into chunks (size of chunks and number) and how source copies of the file can be found (tracker, DHT, Peer, etc), you can then make that file available to any client you wish, not just by tracker but other means like http, smb, ftp, email, etc.
Of course, beware the fact that many networks block bit torrent use because of its widespread use to distribute pirated content.