Results 1 to 6 of 6

Thread: [BASH]read line by line, and compare

  1. #1
    Join Date
    Dec 2010
    Location
    Ankara, Turkey
    Posts
    250

    Default [BASH]read line by line, and compare

    I want to read a file looks like this:

    Code:
    [a md5checksum][a space character][name of the file]
    After reading it, I want to split it by space, and compare md5checksum of first line to second, second to third etc. I will sort the data beforehand, so doing this, I am aiming to achieve checking for dublicate files, even if name, modification date etc. are different. After finding them, I will probably execute some other code on them.

    Is there a good example you know that will teach me how I could do this, or can you provide that example for me?

    Yaşar Arabacı
    The only problem with troubleshooting is that sometimes trouble shoots back.
    My blog (in Turkish): http://yasar.serveblog.net

  2. #2
    Join Date
    Jun 2008
    Location
    UTC+10
    Posts
    9,683
    Blog Entries
    4

    Default Re: [BASH]read line by line, and compare

    Look at the options of sort involving duplicate lines and restricting the comparison fields.

  3. #3
    Join Date
    Nov 2008
    Location
    Germany
    Posts
    963

    Default Re: [BASH]read line by line, and compare

    Do you want to write this in bash? I think in this case it will be better to use python, for example.

  4. #4
    Join Date
    Aug 2008
    Location
    Behind the 8 ball
    Posts
    116

    Default Re: [BASH]read line by line, and compare

    I'm not sure what you are trying to accomplish, are you just wanting a list of the filenames with no duplicates?

    Try this
    Code:
    for FILE in $(cat some_file | awk '{print $2}' | sort | uniq)
    do
         echo ${FILE}
    done
    Hope this gets you started.

    Good luck,
    Hiatt

  5. #5
    Join Date
    Jun 2008
    Location
    UTC+10
    Posts
    9,683
    Blog Entries
    4

    Default Re: [BASH]read line by line, and compare

    Er, why the loop to read each line from the pipeline which just prints to stdout anyway? Why not just run the pipeline?

  6. #6

    Default Re: [BASH]read line by line, and compare

    Code:
    sort -u -k 1.1,1.32 < yourFile

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •