Help setting up rsync (I think) please

I have a collection of CDs which I am trying to set up on a upnp music server.
No problems with mp3 for pop music as tagging is very simple.
However with classical this is not true and in my experience classical music tagging requires a considerable investment in time to get it right, mainly because the public databases are not very consistent.

What I do is rip and encode on my computer (grip/lame or grip/flac) to my main database where I edit the tags using easytag. I then copy the files to a NAS box which has the music server (twonky).
What I would like to do is set up rsync (I think) so that when I have completed a batch of CDs I can run the script and the NAS will be updated.

Am I on the right track and please could somebody help. It would be neat to do this using ssh in due course but first things first.

Please could somebody advise steps to take etc.
Regards,
Budgie2

i see you have had no responses in over 12 hours and i guess that is
because you have asked such a basic question, it appears you just need
a pointer in the right direction…i hope this is it, for you:

you are on the right track, rsync is really easy to use and it will do
exactly as you tell it, and should make you very happy with keeping
your music server stuffed with goodies…

no set up required, just open a terminal and copy paste in the same
‘canned’ command each time you need it, and push enter…easy, simple…

to help make that ‘canned’ command, there are about a jillion
tutorials and hints on how use rsync on the web, you can look at a few
of those from here:

http://www.google.com/search?q=how+to+use+rsync

to get a feel for it, and then do NOT miss the chance to look on your
own hard drive, by using the terminal to access the built-in manual
and info goodies, try it, do this in a terminal


man rsync

press ‘q’ to quit the man, or use the up down arrows to scroll.

then try


info rsync

again, ‘q’ to quit, arrows to scroll but aslo much more available, hit
‘?’ for help, ‘m’ for menu

REPEAT: do NOT read the man on your hard drive, because the tutorial
you read on the net might be 10 years old, and the info there might
not match YOUR rsync…over time rsync has been improved and as it is
the manual changes…you MUST use the command switches for YOUR rsync
(or, install exactly the same rsync used by the person who wrote the
tutorial you really like–but, don’t even try!)

by the way, no not miss the chance to look at how really powerful the
man system is, do this:


man man
info man

enjoy the power and the freedom of free and open source software…


DenverD
CAVEAT: http://is.gd/bpoMD [posted via NNTP w/openSUSE 10.3]
Be it ever so humble, there is no place like 127.0.0.1.

DenverD wrote:

> http://www.google.com/search?q=how+to+use+rsync

STOP, i just looked at the first hit on that LONG list of articles and
see it tells how to install rsync…

do NOT follow that advice, because your openSUSE was born with rsync
installed…

NOTE therefore what i already hinted at: most everything you read on
the internet about Linux could be SO old as to be not only useless,
but DANGEROUS to your system…

actually you probably are better off looking though THIS list which
is, at least, pruned to only show openSUSE info:

http://tinyurl.com/5r33e8k

note the google site specifier or “site:opensuse.org” in the search
string!! use it and that will help you NOT find DANGEROUS help from
(say Ubuntu or 1997 Debian)


DenverD
CAVEAT: http://is.gd/bpoMD [posted via NNTP w/openSUSE 10.3]
Be it ever so humble, there is no place like 127.0.0.1.

Hi DenverD,
Many thanks for the reply.

I am hesitant on two issues. First using ssh. I understand that I could use the rsync -e option. Would that make the command rsync -r -e ssh <sources> <destination> ? Or I read I could set up my environment variables to ensure rsync uses ssh by putting the line RSYNC_RSH=ssh. Problem is that there is a caveat in openSUSE saying not to edit profile file (which is great because I cannot follow all of it,) but to use profile.local. Does that mean I create file profile.local with one line in it or is there a gui which ensures I do not **** things up?

The second issue is that I have the remote directory to which I wish to send the files mounted using NFS. On the other hand I read that sources and dest are in the form user@host: port/filename. From my point of view it would be easiest to use the NFS dest which appears on my system. Does it make any difference?

Sorry if I am being thick. Grateful if you could please clarify when you have a moment.
Regards,
Budgie2

I read that it should be set to

Does your NAS box support ssh? When you can ssh to the NAS box, rsync will use ssh by default. Just try to transfer one file with rsync. What happens? Do you get any errors?

P.S. I would avoid using NFS. Pure rsync is much more reliable.

On Fri, 21 Jan 2011 21:06:02 +0000, vodoo wrote:

> P.S. I would avoid using NFS. Pure rsync is much more reliable.

Well, the two are really quite different.

rsync is used for synchronizing data between two directories or systems.

NFS is a network-based filesystem that lets you access files on a remote
system.

Two different use cases, and two different goals.

:slight_smile:

(Clarification mostly for the OP, since I know you, vodoo, know these are
two different things)

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

I’m surprized, the easy ones are usually answered quicker. rsync is what you want, don’t see a need for ssh. it’s exactly how I send everything to my server. i’m running gnome (you didn’t specify gnome/kde/other) and use grsync. (I think it will work under kde, etc but others will have to verify that) It’s a gui front end to rsync which might be useful especially in the beginning. So, assuming suse sees your nas, just fill in the source / destination, preserve time/owner (it will make sence when you run it) nd do a “dry-run”. when output looks correct, do a full run. The other nice thing about the dry-run is it shows the rsysnc command being executed, so you could use that info to swith to a script if you ever wanted.

Hi all,
Thanks for the detailed advice.
I shall be trying command line with --dry-run and various other experiments with test files & directories until I am confident.
I am not sure what is meant by pure rsync other than just running it from command line and using for destination user@host:/port/filename and getting all my permissions and firewall right. I will only achieve this over some time based on experience to date. Having relevant remote directory mounted locally using NFS keeeps things simple but I have had some problems copying over database this way so will try all options and call for help again when I get stuck.
Regards,
Budgie2

I’m back and confess to cheating by using rsync between the local directory and the NFS mounted directory and it works OK but I really want to do it correctly.

One of my problems is that my network does not have a name server so I am having problems with syntax for correct way to address remote target. I can set up NFS using IP address rather than machine name and I can also see remote directory as samba share. What I am not sure of is how to address the remote directory using user@machine format. For example who is “user” on shared NAS directory? Is it “guest” or “public” or “budgie2” if budgie2 has necessary permissions? Can I use IP of machine if no dns available and do I have to specify port also? As you can tell I am struggling with basics still. Happy to learn but a gui would be great in the interim. Is there a gui in KDE I wonder. Will search but any more guidance on these points would be appreciated. The man pages and rsync --help are all good and a great help but do not address the above gaps in my understanding.
Thanks again,
Budgie2

On 02/02/2011 01:36 AM, Budgie2 wrote:
>
> I’m back and confess to cheating

hmmmm, if it is secure and works, how is it ‘cheating’? too easy??

> The man pages and rsync --help are all good and a great help but do not address the above gaps in my understanding.

this might help: http://transamrit.net/docs/rsync/

it may not fill every gap, but it should do a pretty good job on
most/some of them…

i’ve not vetted it for you, instead i just googled it up with this
string: rsync basic usage

other hits might ‘fit’ your knowledge level better, maybe:
rsync basics

please don’t get the idea i’m saying you are “on your own” with the
wild internet as your only help…but, instead i just give you some
reading material while you wait for a real guru to happen by…


DenverD
CAVEAT: http://is.gd/bpoMD
[NNTP posted w/openSUSE 11.3, KDE4.5.5, Thunderbird3.0.11, nVidia
173.14.28 3D, Athlon 64 3000+]
“It is far easier to read, understand and follow the instructions than
to undo the problems caused by not.” DD 23 Jan 11

I’m on NNTP and haven’t see anything except this message.

Budgie2 wrote:
> One of my problems is that my network does not have a name server so I
> am having problems with syntax for correct way to address remote target.
> I can set up NFS using IP address rather than machine name and I can
> also see remote directory as samba share. What I am not sure of is how
> to address the remote directory using user@machine format. For example
> who is “user” on shared NAS directory? Is it “guest” or “public” or
> “budgie2” if budgie2 has necessary permissions?

Which user will depend on what is set up on your client and server.
Remember that for NFS it is the UID rather than the name that actually
matters, so you need to make sure that budgie2 has the same numerical
uid on all machines.

> Can I use IP of machine
> if no dns available and do I have to specify port also? As you can tell
> I am struggling with basics still. Happy to learn but a gui would be
> great in the interim. Is there a gui in KDE I wonder.

Don’t know about KDE (I use Gnome) but YaST offers three GUIs for NFS!

Under “Network Services” there’s “NFS Server” and “NFS Client”.
Under “System” there’s “Partitioner” and one of the entries in its tree
is “NFS”.

If you don’t have DNS you can specify all the host names and addresses
explicitly in /etc/hosts. Make sure it’s the same on every machine. But
why don’t you have a DNS server set up?

Cheers, Dave

Hi and thanks for the further links.
I conclude my problem is not with rsync but with my basic networking knowledge. I do not have a clue how to point to the remote directory location.
I can use Dolphin to find it and that goes something like Network>Samba Shares>Domain>NAS_name>directories etc. but my feeling is I should have my network set up correctly with dns etc and do not have the time to do this now.
Will plod on.
Many thanks,
Budgie2

On 02/02/2011 01:23 PM, Dave Howorth wrote:
> I’m on NNTP and haven’t see anything except this message.

i’m not sure what that means?

did you just add now add this fora and (what?) not elect to download
any messages which preceded your arrival? or ??

what i’m saying is the OP has five posts in this thread and there are
four different responders…why you can’t see those is not easily
apparent to me, but it sounds like you have a set up problem of some
sort…


DenverD
CAVEAT: http://is.gd/bpoMD
[NNTP posted w/openSUSE 11.3, KDE4.5.5, Thunderbird3.0.11, nVidia
173.14.28 3D, Athlon 64 3000+]
“It is far easier to read, understand and follow the instructions than
to undo the problems caused by not.” DD 23 Jan 11

On 02/02/2011 02:06 PM, Budgie2 wrote:
>
> Network>Samba Shares>Domain>

oh! samba, i know less than zero about that…that has something to do
with non-Linux machines, right? :wink:

see if anything on this page is helpful: http://opensuse.swerdna.org/
it is the site of one of the moderators here and he knows all about
samba…

if you don’t get more help here i’d suggest you begin another thread
using both samba and rsync in the subject…or, send a PM to user
“swerdna” and ask him to look in on this thread…

oh, he lives down under and may be in a hole waiting for a BIG storm!


DenverD
CAVEAT: http://is.gd/bpoMD
[NNTP posted w/openSUSE 11.3, KDE4.5.5, Thunderbird3.0.11, nVidia
173.14.28 3D, Athlon 64 3000+]
“It is far easier to read, understand and follow the instructions than
to undo the problems caused by not.” DD 23 Jan 11

Hi Dave,
Sorry I didn’t make it clear. I can use NFS no problem.

What I want to do is define remote target using user@host/directory format where “host” is a NAS box. My question boils down to who is the “user” on the NAS shared directory; Public, guest, admin or what?

I know I should have a DNS server set up. I shall use /etc/hosts meanwhile and that sorts out resolving hosts. It is the “user” however that has me stuck.

As soon as I have sorted this out and finished my multimedia project I shall revert to setting up DNS server, probably on NAS box since it runs 24/7 more reliably than other machines.

Regards,
Budgie2

@Budgie2: can you tell us what operating system is running on your NAS box? That would increase your chances to get some useful help here.

Hi Vodoo,
It is a linux based system, cannot say exactly which version. If you need more info I can probably get it but that it is linux based should be enough.
Regards,
Budgie2

A quick note to say thanks to all those who have helped on this and to give a bit of feedback just in case somebody else might find it helpful.

I found that if I mounted the remote directory with NFS and then ran rsync between the source with the rsync server being the NFS mounted directory I did not get entirely satisfactory results. It might have been my command syntax but the --delete option did not work well and some minor changes to files were not picked up.

I then tried rsync using the correct address notation for the remote directory. I was advised to try "admin@<IP address> for the NAS and this worked. I still am struggling with the correct syntax for some of the options but will continue to experiment with non critical stuff.

Thanks again all.
Budgie2

Unlike DenverD I find rsync not that easy to use, as it has tons of different options that strongly affect each other.

My way of getting a deeper understanding of the rsync syntax happened kind of accidentally; after having struggled with the man pages of rsync for a while (which are well written, only… huge!) I (admits) gave up and looked for a GUI-frontend for rsync, which brought me to a pretty cool application named →luckyBackup. lB does not only offer a nice GUI that lets you manage your tasks by point&click, the real nice feature is a “validate”-button that will show you the exact rsync command based on your settings. This is pretty useful, because then you can filter the man pages for the exact options and read about them

Hi and thanks for this suggestion. It is what I need because the syntax is NOT clear from the man pages and there are insufficient examples to enable me to deduce where I am going wrong.
I have now confirmed the correct manner to address my remote machine which is "admin@<IP_address>:/…
I have found one or two strange things happening. I have recently added a new subdirectory to my maser database and removed a couple too.

The command I used was:-
rsync -r -del -e ssh /home/alastair/mastermedia/music/ admin@192.168.XXX.XXX:/share/Qmultimedia/Music.
I have found this rather slow even though the two machines have gigabit NICs and are connected using gigabit switch but there are 42 Gigs of data to process.
What I have found is that the -del option does not delete directories even though they have been removed from the master database. The second problem is that the new directory has been created on the server but I cannot access it. In other words the permissions have changed.
I recall there being an option to preserve permissions but need to research and read more. Meanwhile why does the delete not work?
If you have time I would be grateful for further insights but will now try the suggested gui. Just one question having read the info from the luckybackup link. It refers to KDE release 46. Does that mean I msu upgrade my KDE desktop. I am reluctant to do this at present as I hated some features of KDE that came with openSUSE11.3. Grateful for some guidance on this.
Regards,
Budgie2