Page 1 of 2 12 LastLast
Results 1 to 10 of 14

Thread: need a duplicate mp3 file finder

  1. #1

    Default need a duplicate mp3 file finder

    Is there a program for linux which can show me a list of all duplicate music files in a directory? This will allow me to delete all duplicate files without searching for them manually. The more advanced the better.

  2. #2
    Join Date
    Dec 2008
    Location
    GMT-7
    Posts
    382

    Default Re: need a duplicate mp3 file finder

    Duplicate by tag info, or just file name? In other words, if you have a VanHalen-Jump.mp3 and Track01.ogg file with the same song...?
    Desk: AMD Phenom II X4 945 8GB RAM Radeon HD 3300 Arch Linux Xfce 4.8
    Lap: Intel 2.13GHz Core i3 M330 8GB RAM nVidia GeForce 310M Arch Linux KDE 4.7.3

  3. #3
    palladium NNTP User

    Default Re: need a duplicate mp3 file finder

    there are several duplicate file finders available...any should
    do...if you just wanna check the mp3s you would just read the
    documentation to see how to restrict it to certain directories or file
    types..

    462 hits at http://www.google.com/linux?q=duplic...inder+opensuse

    i can't recommend any particular one over another (have never used any
    of them)

    --
    palladium

  4. #4
    Join Date
    Jan 2009
    Location
    Switzerland
    Posts
    1,529

    Default Re: need a duplicate mp3 file finder

    There was a similar thread about finding duplicate images here: Tool to find duplicate images? - openSUSE Forums. Maybe this can help.

    To me it seems you are looking for an Iron Monk listening to all your music files and taking notes of what he heard.
    Technology is 'stuff that doesn't work yet.' -- Bran Ferren

  5. #5

    Default Re: need a duplicate mp3 file finder

    if you're just looking for duplicate files (and not for similar contents), a simple shell script like that one could do it, provided you don't use "@" in filenames!

    Code:
    #! /bin/bash
    # recursively find duplicate files (same size and same md5sum), optionaly with the given extension
    
    dir=$1
    ext=$2
    
    if [ "x$*" == "x" ] ; then
            exec echo "syntax : $0 directory [extension]"
    elif [ ! -d $dir ] ; then
            exec echo "directory $dir not found"
    elif [ "x$2" == "x" ] ; then
            allfiles=(`find $1 -type f -ls | awk '{ print $7"@"$11 }' | sort -n`)
    else
            allfiles=(`find $1 -type f -name "*.$2" -ls | awk '{ print $7"@"$11 }' | sort -n`)
    fi
    
    i=0
    j=0
    
    while [ $i -lt  ${#allfiles[*]}  ] ; do
            j=$(($i+1))
            if [ $j -lt ${#allfiles[*]} ] ; then
                    e1=${allfiles[$i]} ; e2=${allfiles[$j]}
                    f1=${e1##*@} ; f2=${e2##*@}
                    s1=${e1%%@*} ; s2=${e2%%@*}
                    if [ $s1 -eq $s2 ] ; then
                            m1=`md5sum $f1 | awk '{ print $1}'`
                            m2=`md5sum $f2 | awk '{ print $1}'`
                            echo "$f1 = $f2" 
                    fi
            fi
            let i++
    done

  6. #6

    Default Re: need a duplicate mp3 file finder

    Thx for the help, but the anwsers don't quite fit the bill. Maybe I should have been more specific. I have a large collection (7000 mp3s) which contains duplicates no doubt. The problem is that chances are slim that these duplicates have the same filename, or even the same file size (eg bitrate 128 vs bitrate 320).

    I need something that looks at the mp3 tags: if artist and title are the same: they are duplicates. I now that iTunes has an option to give me a list of all duplicates. But I do not want to install windows just for that reason. I looked if amarok could to this, but it only allows me to remove all duplicates in a playlist. I need the opposite: remove all non-duplicates.

  7. #7

    Default Re: need a duplicate mp3 file finder

    There are several python scripts to display and manipulate mp3 tags.

    zypper in python-eyeD3

  8. #8

    Default Re: need a duplicate mp3 file finder

    Quote Originally Posted by please_try_again View Post
    There are several python scripts to display and manipulate mp3 tags.

    zypper in python-eyeD3
    But how will this help finding the duplicates? It seems that with this program I would have to do it manually. For this amarok would suffice, but it is just to much work since I have too many mp3's. I find it hard to believe no decent tool for linux exists. And yet I haven't find one.

  9. #9

    Default Re: need a duplicate mp3 file finder

    Quote Originally Posted by suskewiet View Post
    But how will this help finding the duplicates? It seems that with this program I would have to do it manually.
    Assuming your files are in /data/music (and subdirectories), you can use a command line like the following one to write a list where file names, size, mp3 title and author would appear on the same line ( = one line per file ). Then, it should not be too difficult to parse the list ( which get written in the file /data/music/mp3.list in the example below), sort the files and remove duplicates.
    find /data/music -type f -name "*.mp3" | xargs eyeD3 --no-color | grep -e '.mp3' -e 'title' | sed 's/^title/=&/' | sed -e :a -e '$!N;s/\n=/ /;ta' -e 'P;D' > /data/music/mp3.list

    But I known that's not what you're looking for.

  10. #10

    Default Re: need a duplicate mp3 file finder

    Quote Originally Posted by please_try_again View Post
    Assuming your files are in /data/music (and subdirectories), you can use a command line like the following one to write a list where file names, size, mp3 title and author would appear on the same line ( = one line per file ). Then, it should not be too difficult to parse the list ( which get written in the file /data/music/mp3.list in the example below), sort the files and remove duplicates.
    find /data/music -type f -name "*.mp3" | xargs eyeD3 --no-color | grep -e '.mp3' -e 'title' | sed 's/^title/=&/' | sed -e :a -e '$!N;s/\n=/ /;ta' -e 'P;D' > /data/music/mp3.list

    But I known that's not what you're looking for.
    It seems your command does not support filenames with spaces:
    if a file is found: "this is musicfile.mp3"

    it gives errors:
    Code:
    File not found: this
    File not found: is
    File not found: musicfile.mp3
    I'm starting to give up hope. Still can't believe a music player like amarok doesn't support this. iTunes has this probably for years...

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •