Extendible hashing is a type of hash system which treats a hash as a bit string and uses a trie for bucket lookup. Because of the hierarchical nature of the system, re-hashing is an incremental operation (done one bucket at a time, as needed). This means that time-sensitive applications are less affected by table growth than by standard full-table rehashes.
Extendible hashing was described by Ronald Fagin in 1979. Practically all modern filesystems use either extendible hashing or B-trees. In particular, the Global File System, ZFS, and the SpadFS filesystem use extendible hashing.[1]
Assume that the hash function
h(k)
i
i
Keys to be used:
\begin{align} h(k1)=100100\\ h(k2)=010110\\ h(k3)=110110 \end{align}
Now, if k3 were to be hashed to the table, it wouldn't be enough to distinguish all three keys by one bit (because both k3 and k1 have 1 as their leftmost bit). Also, because the bucket size is one, the table would overflow. Because comparing the first two most significant bits would give each key a unique location, the directory size is doubled as follows:
And so now k1 and k3 have a unique location, being distinguished by the first two leftmost bits. Because k2 is in the top half of the table, both 00 and 01 point to it because there is no other key to compare to that begins with a 0.
The above example is from .
h(k4)=011110
Now, k4 needs to be inserted, and it has the first two bits as 01..(1110), and using a 2 bit depth in the directory, this maps from 01 to Bucket A. Bucket A is full (max size 1), so it must be split; because there is more than one pointer to Bucket A, there is no need to increase the directory size.
What is needed is information about:
In order to distinguish the two action cases:
Examining the initial case of an extendible hash structure, if each directory entry points to one bucket, then the local depth should be equal to the global depth.
The number of directory entries is equal to 2global depth, and the initial number of bucketsis equal to 2local depth.
Thus if global depth = local depth = 0, then 20 = 1, so an initial directory of one pointer to one bucket.
Back to the two action cases; if the bucket is full:
Key 01 points to Bucket A, and Bucket A's local depth of 1 is less than the directory's global depth of 2, which means keys hashed to Bucket A have only used a 1 bit prefix (i.e. 0), and the bucket needs to have its contents split using keys 1 + 1 = 2 bits in length; in general, for any local depth d where d is less than D, the global depth, then d must be incremented after a bucket split, and the new d used as the number of bits of each entry's key to redistribute the entries of the former bucket into the new buckets.
Now,
h(k4)=011110
h(k2)=010110
If had been 000110, with key 00, there would have been no problem, because would have remained in the new bucket A' and bucket D would have been empty.
(This would have been the most likely case by far when buckets are of greater size than 1 and the newly split buckets would be exceedingly unlikely to overflow, unless all the entries were all rehashed to one bucket again. But just to emphasize the role of the depth information, the example will be pursued logically to the end.)
So Bucket D needs to be split, but a check of its local depth, which is 2, is the same as the global depth, which is 2, so the directory must be split again, in order to hold keys of sufficient detail, e.g. 3 bits.
Now,
h(k2)=010110
h(k4)=011110
h(k2)
h(k4)
Below is the extendible hashing algorithm in Python, with the disc block / memory page association, caching and consistency issues removed. Note a problem exists if the depth exceeds the bit size of an integer, because then doubling of the directory or splitting of a bucket won't allow entries to be rehashed to different buckets.
The code uses the least significant bits, which makes it more efficient to expand the table, as the entire directory can be copied as one block .
class Page: def __init__(self) -> None: self.map = [] self.local_depth = 0
def full(self) -> bool: return len(self.map) >= PAGE_SZ
def put(self, k, v) -> None: for i, (key, value) in enumerate(self.map): if key
def get(self, k): for key, value in self.map: if key
def get_local_high_bit(self): return 1 << self.local_depth
class ExtendibleHashing: def __init__(self) -> None: self.global_depth = 0 self.directory = [Page]
def get_page(self, k): h = hash(k) return self.directory[h & ((1 << self.global_depth) - 1)]
def put(self, k, v) -> None: p = self.get_page(k) full = p.full p.put(k, v) if full: if p.local_depth
p0 = Page p1 = Page p0.local_depth = p1.local_depth = p.local_depth + 1 high_bit = p.get_local_high_bit for k2, v2 in p.map: h = hash(k2) new_p = p1 if h & high_bit else p0 new_p.put(k2, v2)
for i in range(hash(k) & (high_bit - 1), len(self.directory), high_bit): self.directory[i] = p1 if i & high_bit else p0 def get(self, k): return self.get_page(k).get(k)
if __name__
import random random.shuffle(l) for x in l: eh.put(x, x) print(l)
for i in range(N): print(eh.get(i))