Wrong estimate of hard-drive usage?

I just recently discovered something I found a bit peculiar. If I call for a calculation of the directory size through Dolphin it shows that /proc is 128 TiB large. I am not familiar with Tibibyte and Gibibyte but the other directories on the system are estimated to be in GiB approximately the size I expect them to be in GB. I know /proc is a special directory so I suppose there is a logical explanation to why this ‘abnormal’ disk usage is proposed which I don’t understand.

Any views on this?

The system seems normal, it runs just fine.

olav@DDAmt:~> df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          118G  6.5G  105G   6% /
devtmpfs        7.9G   36K  7.9G   1% /dev
tmpfs           7.9G     0  7.9G   0% /dev/shm
tmpfs           7.9G  520K  7.9G   1% /run
/dev/sdb1       118G  6.5G  105G   6% /
tmpfs           7.9G     0  7.9G   0% /sys/fs/cgroup
tmpfs           7.9G  520K  7.9G   1% /var/run
tmpfs           7.9G     0  7.9G   0% /media
tmpfs           7.9G  520K  7.9G   1% /var/lock
/dev/sda1       908G  361G  501G  42% /home
olav@DDAmt:~> 

Thanks,
Olav

F Sauce wrote:
>
> I just recently discovered something I found a bit peculiar. If I call
> for a calculation of the directory size through Dolphin it shows that
> /proc is 128 TiB large. I am not familiar with Tibibyte and Gibibyte but
> the other directories on the system are estimated to be in GiB
> approximately the size I expect them to be in GB. I know /proc is a
> special directory so I suppose there is a logical explanation to why
> this ‘abnormal’ disk usage is proposed which I don’t understand.
>
> Any views on this?
>
> The system seems normal, it runs just fine.
>
> Code:
> --------------------
> olav@DDAmt:~> df -h
> Filesystem Size Used Avail Use% Mounted on
> rootfs 118G 6.5G 105G 6% /
> devtmpfs 7.9G 36K 7.9G 1% /dev
> tmpfs 7.9G 0 7.9G 0% /dev/shm
> tmpfs 7.9G 520K 7.9G 1% /run
> /dev/sdb1 118G 6.5G 105G 6% /
> tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
> tmpfs 7.9G 520K 7.9G 1% /var/run
> tmpfs 7.9G 0 7.9G 0% /media
> tmpfs 7.9G 520K 7.9G 1% /var/lock
> /dev/sda1 908G 361G 501G 42% /home
> olav@DDAmt:~>
>
> --------------------
>
>
> Thanks,
> Olav
>
>
The ones with 'i’s are the actual binary system of measuring stuff.
Everything is calculated as a multiple of 1024 instead of 1000.
The ones with 'i’s are more accurate or something like that.

Refer :- http://en.wikipedia.org/wiki/Mebibyte


GNOME 3.6.2
openSUSE Release 12.3 (Dartmouth) 64-bit
Kernel Linux 3.7.10-1.16-desktop

My guess: Dolphin isn’t calculating/estimating anything.

It gives a size for file systems. Presumably, there is a data structure that the kernel provides when asked, and the 128TiB is just a fake meaningless value put into the file system size slot in that data structure.

On 2013-10-13 02:06, F Sauce wrote:
>
> I just recently discovered something I found a bit peculiar. If I call
> for a calculation of the directory size through Dolphin it shows that
> /proc is 128 TiB large. I am not familiar with Tibibyte and Gibibyte but
> the other directories on the system are estimated to be in GiB
> approximately the size I expect them to be in GB. I know /proc is a
> special directory so I suppose there is a logical explanation to why
> this ‘abnormal’ disk usage is proposed which I don’t understand.

TiB, GiB, MiB are the classical memory units, expressed as powers of
two, whereas TB, GB, MB, are powers of 10 (or 1024 and 1000, if you
prefer). The wikipedia article is quite good. This is the new
measurement standard. Ie, the K in KB means the same as the K in Km or
Kg, that is, 1000.

1000 kB kilobyte (1000 bytes)
1000² MB megabyte (1000000 bytes)
1000³ GB gigabyte (1000000000 bytes)
1000⁴ TB terabyte (1000000000000 bytes)

1024 KiB kibibyte (1024 bytes)
1024² MiB mebibyte (1048576 bytes)
1024³ GiB gibibyte (1073741824 bytes)
1024⁴ TiB tebibyte (1099511627776)

As to the /proc directory, the contents are virtual files, not real
files. Don’t worry if you have files bigger than your hard disk, they do
not exist. They can be accessed, though, the kernel takes care of that.


Cheers / Saludos,

Carlos E. R.
(from 11.4, with Evergreen, x86_64 “Celadon” (Minas Tirith))

Carlos E. R. wrote:
> As to the /proc directory, the contents are virtual files, not real
> files. Don’t worry if you have files bigger than your hard disk, they do
> not exist. They can be accessed, though, the kernel takes care of that.
>
>

Now that you mention it. Nautilus reports that /proc is of size
“totalling 140.7 TB”
My harddisk is not even the size of 1 terabyte :slight_smile:

GNOME 3.6.2
openSUSE Release 12.3 (Dartmouth) 64-bit
Kernel Linux 3.7.10-1.16-desktop

/proc is a virtual file system it contains APIs to the kernel processes. In Unix/Linux all things are files.

OK, thanks, guys, for helpful inputs!

Just had a look on this.

There is a 128TiB big “file” in /proc (the exact size may vary depending on your system):

# ls -lh /proc/kcore
 -r-------- 1 root root 128T 14. Okt 09:40 /proc/kcore

For an explanation, see f.e. here: Support | Why is /proc/kcore so big?

So dolphin doesn’t calculate wrong… :wink:

‘Note: On 64-bit systems the size of /proc/kcore is even 128TB because that’s the absolute limit of what 64-bit systems can allocate.’, Support | Why is /proc/kcore so big?.

Thank you wolfi, that explain my /proc dir size. I’m a bit curious to why vazhavandan’s proc is 140.7 TiB though, if 128 is the limit; are there any other files similar to kcore?

F Sauce wrote:
>
>> ‘Note: On 64-bit systems the size of /proc/kcore is even 128TB because
>> that’s the absolute limit of what 64-bit systems can allocate.’,
>> ‘Support | Why is /proc/kcore so big?’
>> (http://www.novell.com/support/kb/doc.php?id=7004153).
>
> Thank you wolfi, that explain my /proc dir size. I’m a bit curious to
> why vazhavandan’s proc is 140.7 TiB though, if 128 is the limit; are
> there any other files similar to kcore?
>
>
GNOME terminal in human readable format(command 2) reports 128T

$ls -l /proc/kcore
-r-------- 1 root root 140737486266368 Oct 15 10:04 /proc/kcore
$ls -lh /proc/kcore
-r-------- 1 root root 128T Oct 15 10:04 /proc/kcore
$

But Nautilus reports space occupied by kcore as

140.7 TB (140,737,486,266,368 bytes)

Only explanation is Nautilus is diving by 1000’s

140737486266368 ==> 140737486266.368 ==> 140737486.266368 ==>
140737.486266368 ==> 140.7374862664

and terminal is dividing by 1024’s

140737486266368 ==> 137438951432 ==> 134217726.007813 ==>
131071.998054504 ==> 127.9999981001


GNOME 3.6.2
openSUSE Release 12.3 (Dartmouth) 64-bit
Kernel Linux 3.7.10-1.16-desktop

On 2013-10-15 06:47, vazhavandan wrote:

> Only explanation is Nautilus is diving by 1000’s

Nautilus is correctly stating Terabytes, but terminal is incorrectly
stating the units. It is using “tebibytes” (TiB) but not saying so, only
“T”.

The man page for ls says:

SIZE may be (or may be an integer optionally followed by)
one of following: KB 1000, K 1024, MB 10001000, M 10241024,
and so on for G, T, P, E, Z, Y.

It doesn’t follow current standards.


Cheers / Saludos,

Carlos E. R.
(from 11.4, with Evergreen, x86_64 “Celadon” (Minas Tirith))

vazhavandan wrote:
> Only explanation is Nautilus is diving by 1000’s
>
> 140737486266368 ==> 140737486266.368 ==> 140737486.266368 ==>
> 140737.486266368 ==> 140.7374862664

i.e. it is measuring how many TB

> and terminal is dividing by 1024’s
>
> 140737486266368 ==> 137438951432 ==> 134217726.007813 ==>
> 131071.998054504 ==> 127.9999981001

i.e. it is measuring how many TiB

http://en.wikipedia.org/wiki/Tebibyte

Yes it does, at least according to https://wiki.ubuntu.com/UnitsPolicy

Exception

The application can keep their previous behavior for backwards compatibility if the following points apply. The application may add an option to display the sizes in base-10, too.

  • is a command-line tool
  • is often parsed by machine (for example, the output is used in scripts)
  • only the prefix is displayed and not the unit (for example, M instead of MB)

Some applications which fall under this rule are:

  • df
  • du
  • ls

:wink:

On 2013-10-15 12:56, wolfi323 wrote:
>
> robin_listas;2591428 Wrote:
>>
>> It doesn’t follow current standards.
>>
> Yes it does, at least according to https://wiki.ubuntu.com/UnitsPolicy

Irrelevant. Look up standards at the IEEE, for instance.


Cheers / Saludos,

Carlos E. R.
(from 11.4, with Evergreen, x86_64 “Celadon” (Minas Tirith))

Well, I wouldn’t call that “irrelevant”. You have to retain compatibility as well (those “standards” don’t exist that long, in the IT world that is).
And “ls” does have options to provide output according to those standards.

Anyway, I find it irritating that nautilus only shows the sizes in measures of 1000’s.
And I can’t find an option to change that, but that’s GNOME I guess… :wink:
In KDE at least you can choose between showing the size in KiB, kB and the old pseudo standard of KB.

On 2013-10-15 16:56, wolfi323 wrote:
>
> robin_listas;2591475 Wrote:
>> Irrelevant. Look up standards at the IEEE, for instance.
>>
> Well, I wouldn’t call that “irrelevant”. You have to retain
> compatibility as well (those “standards” don’t exist that long, in the
> IT world that is).

Well, then irrelevant because it is from Ubuntu :stuck_out_tongue:
If it were from openSUSE…

> And “ls” does have options to provide output according to those
> standards.

Not in the names. It can show multiples of 1024 or 1000, but it uses
names of its choosing, for saving space in the output. Unless I have
missed a setting, or unless this 11.4 doesn’t have it.

> Anyway, I find it irritating that nautilus only shows the sizes in
> measures of 1000’s.
> And I can’t find an option to change that, but that’s GNOME I guess…
> :wink:

Gnome has a lot of hidden options, in a regedit style (they hate been
reminded of that). It could be there.

> In KDE at least you can choose between showing the size in KiB, kB and
> the old pseudo standard of KB.

It is a point, yes.

The old “de facto standard” was acceptable as long as memory sizes were
small and the difference did not matter much. With terabytes, it is
highly visible :frowning:

I have been using that de facto standard all my life and defended it,
and explained it to people and some students. No longer. The new
standard is the correct one to use, although ugly.


Cheers / Saludos,

Carlos E. R.
(from 11.4, with Evergreen, x86_64 “Celadon” (Minas Tirith))