tar, cp, rsync

English Programming/Scripting
#1

Hi OS_Friends,

I’m an absolute beginner at the command line of Linux. Since about one year I have been using Suse 11.1, always using the desktop (KDE4.1). Over all I’m quiet satisfied with this OS on my productivity system. And after the first month’s I can do all the work I have to do without any significant problems: I’m experienced in the usage of the network-printing, Avidemux, Kino, Konquerer, VirtualBox, OpenOffice, MonoDevelop and so on. Before changing to Linux, I worked 7 years as a C++ software-developer on MS-Windows platforms.

For several reasons now I have decided to learn the “real linux-basics” on my own. That means I want to work much more from the command line, in order to better understand the system from its base.

Therefore I have planed, for about the next three month, to do as much jobs as possible from the command line. Currently I’m not quiet sure, if this was a good idea. But I’m not one of those who give up with the occurrence of the first problems and so I’ll keep on trying to realize the complete backup-strategy of my personal computer (Laptop HP-Compaq67615s) and the two other computers within our home-network (all old Laptops) from the command line. The plan is to find out the right commands and the ways to use them and on a second step to bring them to shape with shell-scripting. The only help I have for this, are the man-pages (claro) and some books, the two ‘bibles’ from Addison-Wesley: Kofler Linux and the Linux administrations handbook from Nemeth, Snyder and Hein. But only with this books I can’t solve the following problem:

From the VirtualBox-environment there is a file of 25 GB (I named it WinXP_1.vdi) which I’d like to backup on an external USB-drive (what normally is not necessary or doesn’t make sense, but I wanted to know whether my backup-strat. works with it).

Therefore I tried the use of cp, tar and rsync. Without any success. Every command can only transfer 4096MB (4GB) of the required 25 GB. That means to me: 2³². As I have installed the 64-Bit Version of the Suse Linux 11.1 I don’t understand whats going on there.

Well, that’s it. Now I hope I’m “welcomed” in the scene and there is somebody who can help me with a “tiny workaround” for solving this problem.

I’m sure, that this question is not the last one. And if there is anybody out there who wants to be my personal guide on my way to the Linux-Sky… please don’t hesitate to tell me :-)))

Greetings to u all, Ralf

#2

Hi
A good one to look at would be the iBackup script here for a start;
http://gnu.ethz.ch/linuks.mine.nu/ibackup/

In the past I have used tar/rsync for some eg;


/bin/tar cjf \
/media/sata_backup/config_backups/crossover_office_data_`date+\%m%d%y%H%M%S`.tar.bz2\
..cxoffice/*

or

FILENAME=/tmp/backup.`date +\%s`
/usr/bin/rsync -rlLtuzi --stats --progress --delete \
/home/malcolml/* /media/sata_backup/sled/ \
> $FILENAME
/usr/bin/tr -d '\015' < $FILENAME | /usr/bin/mailx -s \
'HOME Backup via RSYNC to USB Disk' malcolml@localhost 
# Delete backup file
rm $FILENAME

–
Cheers Malcolm °¿° (Linux Counter #276890)
SUSE Linux Enterprise Desktop 11 (x86_64) Kernel 2.6.27.37-0.1-default
up 1 day 0:37, 3 users, load average: 0.09, 0.05, 0.09
GPU GeForce 8600 GTS Silent - CUDA Driver Version: 190.18

#3

Hi
Forgot to add what format is the external device?

–
Cheers Malcolm °¿° (Linux Counter #276890)
SUSE Linux Enterprise Desktop 11 (x86_64) Kernel 2.6.27.37-0.1-default
up 1 day 0:46, 3 users, load average: 0.01, 0.05, 0.07
GPU GeForce 8600 GTS Silent - CUDA Driver Version: 190.18

#4

I also think that is important info, because the size of the file seems to be the problem, not the method of copying.

What is the fs type of the “external USB drive” (well not the drive, but the partition on the disk).

#5

elgitanito wrote:
> The only help I have for this, are
> the man-pages (claro) and some books, the two ‘bibles’ from
> Addison-Wesley: Kofler Linux and the Linux administrations handbook from
> Nemeth, Snyder and Hein.

to those you should add these online resources:

Rute User’s Tutorial and Exposition
http://rute.2038bug.com/

Bash Guide for Beginners http://tldp.org/LDP/Bash-Beginners-Guide/html/

Advanced Bash-Scripting Guide
http://tldp.org/guides.html

and, you may have gold on your hard drive, at:
file:///usr/share/doc/packages/bash/bashref.html

> Every command can only transfer 4096MB (4GB) of the required 25 GB.

sounds like a (vfat?) file system limit to me…

–
palladium

#6

Hi Malcolm,

thanks for your reply.
With your helb I guess the problem found.
The following information is from a hidden directory /.udev/db within the dev-dir. I guess its something like a database of the devices, connected to the computer. I’m sure you know how to handle this info.

The fs is a fat32 or a vfat, as it can be seen from the info below. No I guess I have to format the drive with another file-system specification. Which one do you recommend for my purposes ? NTFS ? Or better a Linux fs ?

N:sdb1
S:disk/by-id/usb-Generic_External_K44GT7B2643D-0:0-part1
S:disk/by-path/pci-0000:00:13.5-usb-0:1:1.0-scsi-0:0:0:0-part1
S:disk/by-uuid/0150-DC63
S:disk/by-label/TREKSTOR
M:8:17
E:ID_VENDOR=Generic
E:ID_MODEL=External
E:ID_REVISION=1.04
E:ID_SERIAL=Generic_External_K44GT7B2643D-0:0
E:ID_SERIAL_SHORT=K44GT7B2643D
E:ID_TYPE=disk
E:ID_INSTANCE=0:0
E:ID_BUS=usb
E:ID_PATH=pci-0000:00:13.5-usb-0:1:1.0-scsi-0:0:0:0
E:ID_FS_USAGE=filesystem
E:ID_FS_TYPE=vfat
E:ID_FS_VERSION=FAT32
E:ID_FS_UUID=0150-DC63
E:ID_FS_UUID_ENC=0150-DC63
E:ID_FS_LABEL=TREKSTOR
E:ID_FS_LABEL_ENC=TREKSTOR
E:ID_FS_LABEL_SAFE=TREKSTOR

Thanx a lot and bye bye from Germany, ralf

#7

Hi palladium

as I replied yet to Malcolm, it is vfat and fat32… sometimes the solution is so clear, but as in germany it is used to be said: sometimes due to all the trees, the forrest can’t be seen :slight_smile:
Thanks for the links you have sent me.

Greets, Ralf

#8

Hi
Glad your starting to sort things out. You need to use ext3 as it will
keep all the permissions and ownerships or the files.

On windows you can install this driver http://www.fs-driver.org/ but
run a test first to ensure it maintains your data integrity!

I would in the long run get another drive for your windows system to
use.

You could also shrink the vfat one and create a second partition on the
external drive. This can all be done via YaST partitioner.

–
Cheers Malcolm °¿° (Linux Counter #276890)
SUSE Linux Enterprise Desktop 11 (x86_64) Kernel 2.6.27.37-0.1-default
up 1 day 17:39, 3 users, load average: 0.15, 0.24, 0.23
GPU GeForce 8600 GTS Silent - CUDA Driver Version: 190.18

#9

OK,

now i’m a little bit upset of myself due to the fact, that I didn’t join this forum earlier. It could have saved a lot of time during the past few month.

On my posting from yesterday I 've received much more information than I have expected.
Thanx a lot to the experts.

Henk ?, If I understand your reply correctly, with your suggestion it should be possible to have access to the external drive from both OS’s, Linux and Windows as well. Did I get that right ?
I’ll keep you posted about the success of my trial…

greetings, Ralf

#10

Well, I think it is Malcolm who offered you suggestions how to use ext3 from windows. Not me, because I know less to nothing from Windows lol!

#11

in Texas they say: Can’t see the forest for the trees.

-welcome-

–
palladium

#12

Hi Ralf
Yes, if you install the windows ext2 driver (which works with ext3). In
linux just use the partitioner to reformat, you will need to remove your
data first as it will wipe the drive!!

–
Cheers Malcolm °¿° (Linux Counter #276890)
SUSE Linux Enterprise Desktop 11 (x86_64) Kernel 2.6.27.37-0.1-default
up 2 days 12:53, 3 users, load average: 0.51, 0.29, 0.42
GPU GeForce 8600 GTS Silent - CUDA Driver Version: 190.18

#13

jeje,

a little bit later I recognized my fault and pleased the Sysadmin to change the text. I was quiet confused that day. To much to do. In the meantime I solved the problems, with the help of you all.

At first I tried to format the drive only from the command line and so I learnt a lot about fdisk, mkfs (and its sisters and brothers), file systems and so on. I tried to have at the end the drive devided into two partitions, one as ext3, the other as NTFS. But I think i wanted to much. The combination of Linux, Virtualbox, Network and XP was to much for me. So at the end I only had partwise-success. When I could work with the drive on Linux-base, it didn’t work with VBox, and vice versa. And so on. Now I have the whole drive formated with ext3 and work according to my whole backup-problems under VBox, Linux and the other Laptops within the net, with shared files under Linux-control. Thats the most stable solution, I can reach with my current level of Linux-skills. But thanks to your information about online and offline e-Infos, I’ll become better from time to time, so god will.

Tanks to u all until the next problems !!

Greetings, Ralf

#14

Hi Malcolm,

did you read my answer above ? (or below, or before, or what ever :-)) BEFORE I HAVE READ YOUR ANSWER WITH THE EXT2 INFO, I wrote I had problems to get the drive run under VBox-XP-Linux with always the same configuration. The main point was, under VBox the Win-Expolorer couldn’t open the USB-Drive, although I had a NTFS-Sys partitioned (apart to ext3) and the drive worked fine with Linux. So I thought it might be any serious problem with the VBox.

But if there is an windows-ext2-driver, which should be installed in order to get the data-stream parsed, I guess there is the real point. don’t you think so ?
Maybe I should keep on trying to get the perfect solution:
So I ask you: Where can I get the ext2-driver for Windows and how do I have to install it ? Do I have to look at any special configurations with the NTFS -partition ?, is it OK to use the first sector or cylinder and all the default settings that are offered from the mkntfs-command ? The size of the files I have to transfer is from very small to very big and to me it doesn’t make sense, to distinguish between different file-systems or ajustments to handle different file-sizes in a perfect manner on a private-system with little thruput (I hope everybody is able to understand my bad DENGLISCH = deutschenglisch we say in Germany).

So, let’s see what comes next… ! Thanx MALCOLM (now I’m right with the name, I hope :-))

#15

OH NOOOOOOOO !!!

I’m really confused, Malcolm. And you are a victim of this confusion :slight_smile:
We, or I didn’t talk about the same matter. The ext2-driver doesn’t help solving problems with NTFS. And I guess with the VBox, as I use it, this ext2-driver must already be in use. That means used by the VBox-environment. Otherwise it shouldn’t be possible to access the shared-files from the linux-side and the XP-side as well.
So I only would have to install the ext2 driver, if there wasn’t VBox but two partitions with Linux and XP instead and the necessity to access from both sides the same USB-drive. I think you talked about this situation. Did I get that right ? If yes, I think the solution which I found it last night, with shared files controlled by linux and served to the VBox for the use by windows, is a good solution at the end (for my purposes).
Mh… learning by writing postings :slight_smile:

Thanks once more and greetings, Ralf

#16

Hi Ralf
Yes, were were thinking the same thing. I’m glad you have now sorted out
the solution that works for you :slight_smile:

Enjoy :slight_smile:

–
Cheers Malcolm °¿° (Linux Counter #276890)
SUSE Linux Enterprise Desktop 11 (x86_64) Kernel 2.6.27.37-0.1-default
up 2 days 20:58, 4 users, load average: 0.08, 0.07, 0.09
GPU GeForce 8600 GTS Silent - CUDA Driver Version: 190.18

#17

Hi Palladium

how can that be ? “Can’t see the forest for the trees.” I always thought there was no forest at all in Texas. jejeje :slight_smile:
Only oil-piplines… or is there now a plan, thanks to obama, to grow forest in texas in order to reduce co2-exhaust in the US ?
LOL, I’m so glad for having found the solution with my backup-problem, that I’m in the mood to make bad chokes with people from Texas… please forgive me !!!
Peaceful greets from old Europe to Texas, Ralf

#18

> how can that be ? “Can’t see the forest for the trees.” I always
> thought there was no forest at all in Texas. jejeje :slight_smile:

well, see all you know about Texas comes from Hollywood, take a
peaceful look at Sabin National Forrest or Angelina National Forrest
in http://maps.google.com/

> Peaceful greets from old Europe to Texas

this Texan lives in Denmark and enjoys it, a lot!

–
palladium

#19

Hi Palladium,
you’re absolutly right, I’ve never been to Texas and all my imaginations about it are from hollywood. So, yesterday I thought can’t be right, what I wrote, because there must be forests, at least forests from cactuses:-)

Well I believe in you, Texas might be wonderful and faszinating, but I’ll never get there, because according to my personal convinience about the todays climate situation, it doesn’t make sence, to fly around the globe only to be on holiday in a region, that I also can see on tv or on google earth or that I can have just around the corner in southern Europe, maybe a little bit diferent, but worth to see it (to get there is bad enough for the climate). So I guess I’ll never get out of Europe for private pleasure, maybe one time for professional reasons, but then I won’t have time for sightseeing. For personal reasons not, for sure, at least until there’s no better way to get there, than by plane. For the same reason I love sailing more, than being in a desert :slight_smile:
Are you danish or texan ? I didn’t understand, what are you enjoying, beeing in denmark as Texan, or having been to Texas as danish ?
But maybe this is not the right place to discuss issues others than linux based.
My email is: elgitanito@gmx.net, and now I’ll take a look at the link you have sent me.

Byby, greets from Ludwigshafen (BASF-Town), Germany