Guys,
I have a OpenSuse 15.5 machine with several ext4 partitions.
How do I make a partition into a hash-indexed partition ? I want to make a directory can have an unlimited number of files / subfolders ( no 65k limit. )
Pls advise.
dir_index exists since over 20 years. Are you sure it is not enabled for your filesystems?
The number of files is limited by the number of inodes. That’s a setting you can make when formatting the partition. There isn’t an “unlimited” option.
Hi. How do i check ? a cli approach is preferred ?
Hi. How do i check ? a cli approach is preferred ?
Repeat post - sorry. I cannot delete it. Pls ignore this
A buddy setup this server for me…I wasnt paying attention …
I have about 14TB on this partition. I want to make sure its hash-indexed and not ‘linear’. Forgive me if the terms are not technically correct.
Study the mam page of tune2fs . E.g. the -l option.
BTW, this is all about the file system, not about the partition.
This is the output of command dumpe2fs /dev/sda5
Filesystem volume name: <none>
Last mounted on: /storage
Filesystem UUID: 5b7f3275-667c-441a-95f9-5dfdafd09e75
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery
extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 481144832
Block count: 3849149243
Reserved block count: 192457462
Overhead clusters: 30617806
Free blocks: 3748257100
Free inodes: 480697637
First block: 0
Block size: 4096
Fragment size: 4096
Group descriptor size: 64
Reserved GDT blocks: 212
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 4096
Inode blocks per group: 256
Flex block group size: 16
Filesystem created: Wed Jan 31 18:25:23 2024
Last mount time: Mon Jul 1 21:57:47 2024
Last write time: Mon Jul 1 21:57:47 2024
Mount count: 16
Maximum mount count: -1
Last checked: Wed Jan 31 18:25:23 2024
Check interval: 0 (<none>)
Lifetime writes: 121 GB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 32
Desired extra isize: 32
Journal inode: 8
Default directory hash: half_md4
Directory Hash Seed: a3f0be94-84c1-4c1c-9a95-e9fc53040195
Journal backup: inode blocks
Checksum type: crc32c
Checksum: 0x874e658e
Journal features: journal_incompat_revoke journal_64bit journal_checksum_v3
Total journal size: 1024M
Total journal blocks: 262144
Max transaction length: 262144
Fast commit length: 0
Journal sequence: 0x0000fb3e
Journal start: 172429
Journal checksum type: crc32c
Journal checksum: 0x417cec36
Group 0: (Blocks 0-32767) csum 0xeed3 [ITABLE_ZEROED]
Primary superblock at 0, Group descriptors at 1-1836
Reserved GDT blocks at 1837-2048
Block bitmap at 2049 (+2049), csum 0xaf2f641b
Inode bitmap at 2065 (+2065), csum 0x47b1c832
Inode table at 2081-2336 (+2081)
26585 free blocks, 4085 free inodes, 2 directories, 4085 unused inodes
Free blocks: 6183-32767
Free inodes: 12-4096
.
.
.
.
.
Group 117466: (Blocks 3849125888-3849149242) csum 0x10bf [INODE_UNINIT, ITABLE_ZEROED]
Block bitmap at 3848798218 (bg #117456 + 10), csum 0x2f8086f1
Inode bitmap at 3848798229 (bg #117456 + 21), csum 0x00000000
Inode table at 3848800790-3848801045 (bg #117456 + 2582)
23355 free blocks, 4096 free inodes, 0 directories, 4096 unused inodes
Free blocks: 3849125888-3849149242
Free inodes: 481140737-481144832
So the max number of files seems to be 481144832. That’s not unlimited, but is a lot higher than the 65k that you worried about. And it seems to be using a hashed directory.
And it seems to be using a hashed directory.
Could you share how you ascertained this ? Which is the particular entry ?
Hi. before i tune the fs, dow do i definitely check if ‘hash-tree’ indexing has not been enabled ?
It is in the feature list of the file system you show.
And of course you can switch it on again (and see if something changes in the feature list). Does not hurt, special when you want it on anyway.
By reading some documentation and comparing with your output, you have already hash tree directories enabled. Nothing to do…
https://www.kernel.org/doc/html/latest/filesystems/ext4/directory.html
This is the article on the 64k limit - A directory on ext4 can have at most 64000 sub directories - What is the meaning of "ext[3/4]_dx_add_entry: Directory index full!"? - Red Hat Customer Portal
Is there any command that I can run to verify that ?
I found this excerpt from - ext4 - Wikipedia
Unlimited number of subdirectories
ext4 does not limit the number of subdirectories in a
single directory, except by the inherent size limit of
the directory itself. (In ext3 a directory can have
at most 32,000 subdirectories.)[17][obsolete source]
To allow for larger directories and continued performance,
ext4 in Linux 2.6.23 and later turns on HTree indices
(a specialized version of a B-tree) by default, which allows
directories up to approximately 10–12 million entries
to be stored in the 2-level HTree index and 2 GB directory size limit for 4 KiB block size, depending on the filename length.
In Linux 4.12 and later the large_dir feature enabled
a 3-level HTree and directory sizes over 2 GB,
allowing approximately 6 billion entries in a single directory.
Just wished i could get a better source than wikipedia on the above but from man ext4, we get the following;
dir_index
Use hashed b-trees to speed up name lookups in large
directories. This feature is supported by ext3 and ext4 file
systems, and is ignored by ext2 file systems.
dir_nlink
Normally, ext4 allows an inode to have no more
than 65,000 hard links. This applies to regular files as
well as directories, which means that there can be no more
than 64,998 subdirectories in a directory (because each of
the '.' and '..' entries, as well as the directory entry
for the directory in its parent directory counts as
a hard link). This feature lifts this limit by causing ext4
to use a link count of 1 to indicate that the number of
hard links to a directory is not known when the
link count might exceed the maximum count limit.
So looks like ‘dir_nlink’ and ‘dir_index’ are the key flags that i need to look for from command tune2fs -l /dev/sdX