Idiot's guide to Konsole and fdupes

Hi Everyone,

I realised a couple of months ago that my OS was no longer supported. (What can I say it was working and life has been busy). I have 1 Gb of ram and I am running OpenSuse 12.1. So when I have spare moments I have been thinking about what I am going to do. Do I get a new PC or try a laptop as it may use less energy. Do I run OpenSuse on my present machine with a ‘lighter’ environment or do I change to a lightweight distribution. All those questions aside I need to tidy all my data up.

In my desktop I have two internal hard drives. I backup parts of my /home directory and sometimes parts of my /home directory to a partition on the second hard drive, then every so often I back up my data from the second hard drive onto an external hard drive. Then once a year I also copy my photos onto a DVD and these go off to a godparent. So I think I have lots of copies of emails, photos, documents, video on both internal hard drives and my external hard drive.

I am not very good with terminal, but I did some research and fdupes appear to be the best I have available to me. I used Konsole. I decided to start with my second internal hard drive with

fdupes -r /mydisk

I did that last night before I went to bed as the drive has 43 gb’s on it. This morning I have come down to prompt and lines and lines of text. When I scroll back, I cannot find my original command. So my questions are:

  1. Is it all there? I don’t think it is, how do I scroll back to where I typed the original command?
  2. What do the lines of text mean? Most are in twos, but some are 3 or 4 lines together and are not identical.
  3. I have looked at fdupes man document, but I am not sure I understand it. What I would like to do is when I have removed the duplicates from the both internal drives and the external drive. I would like to check for duplicates against the second hard drive and the external harddrive, so how do I do this with fdupes?
  4. I realise I can delete the duplicates, but I am not confident that I am doing the right thing, so will probably open two Konquorer windows and delete that way. A website with simple instructions and examples of output would be great?

Thank you very much.

On 2013-10-25 16:26, nappy501 wrote:
>
> Hi Everyone,
>
> I realised a couple of months ago that my OS was no longer supported.
> (What can I say it was working and life has been busy). I have 1 Gb of
> ram and I am running OpenSuse 12.1.

11.4 is still supported by evergreen. You could wait a bit and install
13.1 which will also have long term support by evergreen. Then you can
forget installs for a long time :wink:

I have 12.3 on a laptop with only 500 MB of RAM, with LXDE. Works fine.
Of course, there are applications that will run slow and use swap, they
take a lot of memory. Firefox, libreoffice…


Cheers / Saludos,

Carlos E. R.
(from 11.4, with Evergreen, x86_64 “Celadon” (Minas Tirith))

About the konsole. As you may understand, keeping all lines of what is shown in a session somewhere so you can scroll back to see them obviously has it’s limits. In the Settings menu you will find Configure current profile, which has the Scrolling tab. Look there, there is even an Unlimited scrollback, but that is of cource limited by the amount of disk available.

In general, when you want to keep output of a command for later reference, you use redirection to put all of it in a file of your choice. You can then later consult that file. An example coming close to your case:

fdupes -r /mydisk >fdupesoutput 2>&1

will store all standard and error output of the fdupes command in the file fdupesoutput (of course in the working directory of that moment in time).

And about fdupes. As most of us are not clairvoyant, we are unable to comment on output lines of fdupes we never have even seen. Thus I invite you to copy/paste some of them in a post (of course between CODE tags, you get those tags by clicking on the # button in the toolba of the post editor).

I am not fluent with fdupes at all (it is not even on my system), but I found a man page on the Internet and it says the synopsis is:

fdupes options ] DIRECTORY

This means that you can give several directories to it. While you are talking about “internal” and “external” disks, that is completley irrelevant. At the level you are addressing, there are only directories in the directory tree. Thus you could use the mount points of the several file systems involved in the fdupes command.

Right, I shall go and look at Evergreen as I have never heard of it.

Hi,
Thank you very much for your response. I am not sure I understand and I think I did something wrong. I pasted the above into konsole and got ‘permission denied’, so I then ‘su’ and entered my password. Although nothing happened on the screen the box appeared to be working. When I looked at it this morning it was back to the prompt. I went to mydisk to look for fdupesoutput, but I couldn’t find it.

I then tried kfind fdupesoutput ( I assume * = wildcard) and set mydisk for the search. Nothing was found. I am not sure what I did wrong?

And about fdupes. As most of us are not clairvoyant, we are unable to comment on output lines of fdupes we never have even seen. Thus I invite you to copy/paste some of them in a post (of course between CODE tags, you get those tags by clicking on the # button in the toolba of the post editor).

I have looked at the man page again. I think it is showing me duplicates in groups, but as it was not showing my personal data it is hard for me to judge. I am hoping when I can see what it shows for my personal data, it will make more sense.

I am not fluent with fdupes at all (it is not even on my system), but I found a man page on the Internet and it says the synopsis is:

This means that you can give several directories to it. While you are talking about “internal” and “external” disks, that is completley irrelevant. At the level you are addressing, there are only directories in the directory tree. Thus you could use the mount points of the several file systems involved in the fdupes command.

I think I understand what you are saying in principle. However I have no idea what to type. I will go and look at the man page again.

Please don’t reply to my last post yet, I think I might have sussed it.

Hi Again,

Thank you so much.

I needed to ‘cd /mydisk’ for it to work.

I actually understand the fdupesoutput file now. I found it very interesting.

First you get:

Building file list - 
                                        
fdupes: could not chdir to /mydisk/lost+found

Then

Building file list \ 
Building file list | 
Building file list / 
Building file list - 

For a long time, followed by:

Progress [347/77551] 0% 
Progress [348/77551] 0% 

and more

Progress [29236/77551] 37% 
Progress [29237/77551] 37% 

Progress [77549/77551] 99% 
Progress [77550/77551] 99% 

And then finally the duplicates

/mydisk/home/Pictures/12 sept/100DICAM/DSCI0005.JPG
/mydisk/homebackup/nappyPictures/12 sept/100DICAM/DSCI0005.JPG
/mydisk/homebackup/oldfolders/2012/12 sept/100DICAM/DSCI0005.JPG

I checked all of these and they are all the same photograph. I don’t think I am comfortable using the delete function in fdupes. So although it will take some time I will go through and manually delete the duplicates. I believe a lot will be complete folders so it may not take as long.

Another question if you please, if I have to be in the directory to check for duplicates. How do I check two different directories? That would be checking files in different partitions on different drives.

I think I may have found the answer to my question here:
Linux fdupes: Get Rid (Delete) Of Double Duplicate Files In Directory

How Do I Find Dupes In Two Directories?

Type the command as follows:
 # fdupes  /dir1 /dir2
 OR
 # fdupes -r /etc /data/etc /nas95/etc

I may also try this:

 -q --quiet hide progress indicator

to reduce the size of the fdupesoutput file.

Thank you again.

P.S Just for my own learning what do

>fdupesoutput 2>&1 

What do the > and 2>&1 do or mean on either side of the fdupesoutput?

Thank you in advance.

The default for Konsole, I think, is a scroll back of 1000 lines. You can make it infinite, but that might run out of memory if you try hard enough.

Here’s a neat little command for you that would have solved your problem:

script

You type “script”. That creates a file called “typescript” (you can give it a different name as a command line option to “script”). And, in that file, there will be a complete transcript of everything in your session from when you started “script” until you exit from it.

If I plan to run a command that will produce a lot of output, and if I want to be able to peruse that output later, I use “script”. Then I can later use “less” to browse through the “typescript” file. It is even possible to open an additional terminal session, and browse through the “typescript” file, while you are still doing stuff in that “script” session. The last few lines of output might not yet have been recorded in the file, but most of the session will be there.

Give “script” a try. It is simple, and it is exactly what you needed.

The “2>&1” tells the shell (which starts the command), that error messages should go to the same place as normal output.

Using “2>filename” would redirect file descriptor 2 (normally “stderror” to write its output to the indicated file. Using, instead, “2>&1” says that file descriptor 2 output should be written to file descriptor 1 (normally “stdout”).

Note that I used quotes above for highlighting. DO NOT use them for things like “2>&1” in a shell command, because quoting suppresses the special meaning of that string.

It is a bit difficult to explain everything if you do not undestand the terms used in the explanation. That is of course not your fault. The learning curve is rather steep in the beginning. Try to get hold of some basic Unix/Linux course that tells you about the shell.

>something

is short for

1>something

. Where 1 is standard output, what you normaly see on the terminal (except for errors, those should go to 2, standard error). The > means re-direction to a file (in this case, there are of course many other meanings of > when it appears in other places). And something is a path to a file of your choice. When you use a relative path (like fdupesoutput I used as an example and which you choose to use unaltered) it is relative to your working directory. That is the place where you are in the directory tree. When you login as user, that is your home directory, but you can change it with the cd command.

A few other remarks:

The first time you did this the file fdupesoutput must be created in the working directory you were in. Even iff you can not find it, it is still there. A command to search for it:

find / -name 'fdupesoutput'

I do not understand why the cd /mydisk is needed. You mention /mydisk in the command, that is an absolute path and should be sufficient.

As I said earlier, the man page tells you that more then one directory can be given to the command. Like

fdupes -r /mydisk $HOME/Pictures /mybackup/pictures

or whatever.

And a forum technical remark:
I see you try to make your computer texts in the post readable by using HTML tags. That is very helpfull indeed. Next time you can use CODE tags by clicking the # button left of the <> button.

On 2013-10-26 01:46, nappy501 wrote:
>
> robin_listas;2593493 Wrote:
>> On 2013-10-25 16:26, nappy501 wrote:
>>
>> 11.4 is still supported by evergreen. You could wait a bit and install
>> 13.1 which will also have long term support by evergreen. Then you can
>> forget installs for a long time :wink:
>>
>
> Right, I shall go and look at Evergreen as I have never heard of it.

Here: openSUSE:Evergreen


Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

Thank you so much for this, but my brain is not big enough yet to understand. It may be better next time I visit.

It will grow on you, as you need it.

Right now you have something at the back of your head giving you hints. As you need to do those things, you will explore more. And, eventually, it will seem natural.

Try the following set of commands and then read the files that are made.

ls -a

This prints to the screen.

ls -a 1>myfolderls

Now the output is saved to myfolderls. The 1 before the > is implied when you type only the >.

ls -z

You should see an error message on the screen.

ls -z >wrongoption

The same message should print to screen and wrongoption should be empty.

ls -z 2>oopsididitagain

Should give no message to screen, but the error message is in oopsididitagain. Once this makes sense, reread the earlier explanation of the 2>&1 bit. Alternatively you could follow your command with

>whatworkedright 2>crapthisscrewedup

and have two files - this might make it easier to find the errors sometimes, and keeps the good stuff in one file by itself if you ever want to do something else with that information someday. As always, please choose your own filenames as desired.