Dedupe two folders into one

Not sure this is the right forum, but…

I have two music folders (mine & the wifes). They both stem from one original music folder but have since then evolved in line with our individual tastes :slight_smile:

I’d like to merge them both back into one folder, but not have duplicate songs (as much as possible).

My problem is, my folder structure is like this:


/Artist/Album/Track No. - Song Name.mp3

Hers is


/Album/Artist - Song Name.mp3

(these are two identical songs)
And additionally, she is less anal than I, so her files may be named anything at all!

I thought I could maybe use an md5 hash to deduplicate this lot?

Any ideas?

EDIT: Or, perhaps just copy both dirs into one, dupes and all, and point say Amarok at it & get it to reorganise the files for me? Each folder is roughly 20gigs big though, so the combined one would be 40gigs of music - not sure Amarok is up to the task tbh… Also, the id3 tags may not be correct all the time.

Ok, solved I think.

Found fdupes - works even when the filenames are different. Bonanza!

weighty foe wrote:
> Not sure this is the right forum, but…
>
> I have two music folders (mine & the wifes). They both stem from one
> original music folder but have since then evolved in line with our
> individual tastes :slight_smile:
>
> I’d like to merge them both back into one folder, but not have
> duplicate songs (as much as possible).

Might I ask why you want to merge them back together? It sounds like
either you or your wife would then have to use the other’s indexing
system. If you’re just trying to save space, you could use hard links
instead. So there would only be one copy of each track, but both
directory structures would still be in place.

You could write some sort of script to do the job, but if you choose the
hard-link solution, you could just run a program called
‘faster-dupemerge’ on the two top-level directories. There’s a
description at

<http://furryterror.org/~zblaxell/projects/dupemerge/dupemerge.html>

and the program itself is available from

<http://www.hungrycats.org/~zblaxell/dupemerge/faster-dupemerge>

Cheers, Dave

Well I want to put our collection on a nas (freenas) to share to the stereo, ps3 & netgear “digital entertainer” that I was given. I’d rather not have duplicate songs. I’m moving videos etc as well.

On 2012-01-28 13:46, weighty foe wrote:
>
> Well I want to put our collection on a nas (freenas) to share to the
> stereo, ps3 & netgear “digital entertainer” that I was given. I’d
> rather not have duplicate songs. I’m moving videos etc as well.

Hard links do not duplicate files, just the names.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)