saberbud/parallel-hashmap
1
0
Fork 0
mirror of https://github.com/greg7mdp/parallel-hashmap.git synced 2026-08-30 00:50:38 +08:00
Code Issues Packages Projects Releases Wiki Activity

update "Parallel Hashmap" writeup

Browse Source
This commit is contained in:
greg
2019-03-31 16:35:03 -04:00
parent fa8c63c006
commit 2677210b5a
1 changed files with 42 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
html/parallel_hashmap.md
Vendored
+42 -37
View File
@@ -1,7 +1,7 @@
# The Parallel Hashmap
or Abseiling from the shoulders of giants - © Gregory Popovitch - March 10, 2019
[tl;dr] We present a novel hashmap design, the Parallel Hashmap. Built on top of Abseil's *flat_hash_map*, the Parallel Hashmap has lower space requirements, is nearly as fast as the underlying *flat_hash_map*, and can be used from multiple threads with high levels of concurrency.
[tl;dr] We present a novel hashmap design, the Parallel Hashmap. Built on a modified version of Abseil's *flat_hash_map*, the Parallel Hashmap has lower space requirements, is nearly as fast as the underlying *flat_hash_map*, and can be used from multiple threads with high levels of concurrency. The [parallel hashmap](https://github.com/greg7mdp/parallel-hashmap) repository provides header-only version of the flat and node hashmaps, and their parallel versions as well.
### A quick look at the current state of the art
@@ -53,11 +53,20 @@ providing an index between 0 and 15.
The benefit of this approach would be that the internal tables would each resize on its own when they reach 87.5% capacity, and since each table contains approximately one sixteenth of the values, the memory usage peak would be only one sixteenth of the size we saw for the single *flat_hash_map*.
The rest of this article describes my implementation of this concept that I have done inside the Abseil library (I have submitted a pull request in the hope it will be merged into the main Abseil codebase). The current name for it is *parallel_flat_hash_map* or *parallel_flat_hash_set*. It does provide the same external API as Abseil's other hash tables, and internally it uses a std::array of N *flat_hash_maps*.
The rest of this article describes my implementation of this concept that I have done in my [parallel hashmap](https://github.com/greg7mdp/parallel-hashmap) repository. This is a header only library, which provides the following eight hashmaps:
- phmap::flat_hash_set
- phmap::flat_hash_map
- phmap::node_hash_set
- phmap::node_hash_map
- phmap::parallel_flat_hash_set
- phmap::parallel_flat_hash_map
- phmap::parallel_node_hash_set
- phmap::parallel_node_hash_map
I was delighted to find out that not only the *parallel_flat_hash_map* has significant memory usage benefits compared to the *flat_hash_map*, but it also has significant advantages for concurrent programming as I will show later.
This implementation requires a C++11 compatible compiler, and provides full compatibility with the std::unordered_map (with the exception of *pointer stability* for the `flat` versions. C++14 and C++17 methods, like `try-emplace`, are provided as well.
The names for it are *parallel_flat_hash_map* or *parallel_flat_hash_set*, and the *node* equivalents. These hashmaps provide the same external API as the *flat_hash_map*, and internally use a std::array of 2**N *flat_hash_maps*.
> I will use the names *parallel_hash_map* and *parallel_flat_hash_map* interchangably. They refer to the same data structure. The name used in my Abseil fork is *absl::parallel_flat_hash_map*, as it may be desirable to also provide a *absl::parallel_node_hash_map*.
I was delighted to find out that not only the *parallel_flat_hash_map* has significant memory usage benefits compared to the *flat_hash_map*, but it also has significant advantages for concurrent programming as I will show later. In the rest of this article, we will focus on the *parallel_flat_hash_map*, but similar results are seen for the *parallel_node_hash_map*, and the *set* versions of course.
### The Parallel Hashmap: memory usage
@@ -68,7 +77,7 @@ So, without further ado, let's see the same graphs graphs as above, with the add
![stl_flat_par_zoomed comparison](https://github.com/greg7mdp/parallel-hashmap/blob/master/html/img/stl_flat_par_mem_zoomed.PNG?raw=true)
We see that the *parallel_hash_map* behaves as expected. The memory usage matches exactly the memory usage of its base *flat_hash_map*, except that the peaks of memory usage which occur when the table resizes are drastically reduced, to the point that they are not objectionable anymore. In the "zoomed-in" view, we can see the sixteen dots corresponding to each of the individual submaps resizing. The fact that those resizes are occuring at roughly the same x location in the graph shows that we have a good hash function distribution, distributing the values evenly between the sixteen individual submaps.
We see that the *parallel_flat_hash_map* behaves as expected. The memory usage matches exactly the memory usage of its base *flat_hash_map*, except that the peaks of memory usage which occur when the table resizes are drastically reduced, to the point that they are not objectionable anymore. In the "zoomed-in" view, we can see the sixteen dots corresponding to each of the individual submaps resizing. The fact that those resizes are occuring at roughly the same x location in the graph shows that we have a good hash function distribution, distributing the values evenly between the sixteen individual submaps.
### The Parallel Hashmap: speed
@@ -82,27 +91,27 @@ The first step (compute the hash) is the most problematic one, as it can potenti
![index computation cost](https://github.com/greg7mdp/parallel-hashmap/blob/master/html/img/idx_computation_cost.PNG?raw=true)
As for the hash value computation, fortunately we can eliminate this cost by providing the computed hash to the submap functions, so that it is computed only once. This is exactly what I have done in my implementation of the *parallel_hash_map* within the Abseil library, adding a few extra APIs to the Abseil internal raw_hash_map.h header, which allow the *parallel_hash_map* to pass the precomputed hash value to the underlying submaps.
As for the hash value computation, fortunately we can eliminate this cost by providing the computed hash to the submap functions, so that it is computed only once. This is exactly what I have done in my implementation of the *parallel_flat_hash_map*, adding a few extra APIs to the internal raw_hash_map.h header, which allow the *parallel_flat_hash_map* to pass the precomputed hash value to the underlying submaps.
So we have all but eliminated the cost of the first step, and seen that the cost of the second step is very minimal. At this point we expect that the *parallel_hash_map* performance will be close to the one of its underlying *flat_hash_map*, and this is confirmed by the chart below:
So we have all but eliminated the cost of the first step, and seen that the cost of the second step is very minimal. At this point we expect that the *parallel_flat_hash_map* performance will be close to the one of its underlying *flat_hash_map*, and this is confirmed by the chart below:
![stl_flat_par comparison](https://github.com/greg7mdp/parallel-hashmap/blob/master/html/img/stl_flat_par_speed.PNG?raw=true)
Indeed, because of the scale is somewhat compressed due to the longer times of the std::unordered_map, we can barely distinguish between the blue curve of the *flat_hash_map* and the red curve of the *parallel_hash_map*. So let's look at a graph without the std::unordered_map:
Indeed, because of the scale is somewhat compressed due to the longer times of the std::unordered_map, we can barely distinguish between the blue curve of the *flat_hash_map* and the red curve of the *parallel_flat_hash_map*. So let's look at a graph without the std::unordered_map:
![flat_par comparison](https://github.com/greg7mdp/parallel-hashmap/blob/master/html/img/flat_par_speed.PNG?raw=true)
This last graph shows that the *parallel_hash_map* is slightly slower especially for smaller table sizes. For a reason not obvious to me (maybe better memory locality), the speeds of the *parallel_hash_map* and *flat_hash_map* are essentially undistinguishable for larger map sizes (> 80 million values).
This last graph shows that the *parallel_flat_hash_map* is slightly slower especially for smaller table sizes. For a reason not obvious to me (maybe better memory locality), the speeds of the *parallel_flat_hash_map* and *flat_hash_map* are essentially undistinguishable for larger map sizes (> 80 million values).
### Are we done yet?
This is already looking pretty good. For large hash_maps, the *parallel_flat_hash_map* is a very appealing solution, as it provides essentially the excellent performance of the *flat_hash_map*, while virtually eliminating the peaks of memory usage which occur when the hash table resizes.
But there is another aspect of the inherent parallelism of the *parallel_hash_map* which is interesting to explore. As we know, typical hashmaps cannot be modified from multiple threads without explicit synchronization. And bracketing write accesses to a shared hash_map with synchronization primitives, such as mutexes, can reduce the concurrency of our program, and even cause deadlocks.
But there is another aspect of the inherent parallelism of the *parallel_flat_hash_map* which is interesting to explore. As we know, typical hashmaps cannot be modified from multiple threads without explicit synchronization. And bracketing write accesses to a shared hash_map with synchronization primitives, such as mutexes, can reduce the concurrency of our program, and even cause deadlocks.
Because the *parallel_hash_map* is made of sixteen separate submaps, it posesses some intrinsic parallelism. Indeed, suppose you can make sure that different threads will use different submaps, you would be able to insert into the same *parallel_hash_map* at the same time from the different threads without any locking.
Because the *parallel_flat_hash_map* is made of sixteen separate submaps, it posesses some intrinsic parallelism. Indeed, suppose you can make sure that different threads will use different submaps, you would be able to insert into the same *parallel_flat_hash_map* at the same time from the different threads without any locking.
### Using the intrinsic parallelism of the *parallel_hash_map* to insert values from multiple threads, lock free.
### Using the intrinsic parallelism of the *parallel_flat_hash_map* to insert values from multiple threads, lock free.
So, if you can iterate over the values you want to insert into the hash table, the idea is that each thread will iterate over all values, and then for each value:
@@ -173,48 +182,48 @@ And the graphical visualization of the results:
![mt_stl_flat_par comparison](https://github.com/greg7mdp/parallel-hashmap/blob/master/html/img/mt_stl_flat_par_both_run2.PNG?raw=true)
We notice in this last graph that the memory usage peaks, while still smaller than those of the *flat_hash_map*, are larger that those we saw when populating the *parallel_hash_map* using a single thread. The obvious reason is that, when using a single thread, only one of the submaps would resize at a time, ensuring that the peak would only be 1/16th of the one for the *flat_hash_map* (provided of course that the hash function distributes the values somewhat evenly between the submaps).
We notice in this last graph that the memory usage peaks, while still smaller than those of the *flat_hash_map*, are larger that those we saw when populating the *parallel_flat_hash_map* using a single thread. The obvious reason is that, when using a single thread, only one of the submaps would resize at a time, ensuring that the peak would only be 1/16th of the one for the *flat_hash_map* (provided of course that the hash function distributes the values somewhat evenly between the submaps).
When running in multi-threaded mode (in this case eight threads), potentially as many as eight submaps can resize simultaneaously, so for a *parallel_hash_map* with sixteen submaps the memory peak size can be half as large as the one for the *flat_hash_map*.
When running in multi-threaded mode (in this case eight threads), potentially as many as eight submaps can resize simultaneaously, so for a *parallel_flat_hash_map* with sixteen submaps the memory peak size can be half as large as the one for the *flat_hash_map*.
Still, this is a pretty good result, we are now inserting values into our *parallel_hash_map* three times faster than we were able to do using the *flat_hash_map*, while using a lower memory ceiling.
Still, this is a pretty good result, we are now inserting values into our *parallel_flat_hash_map* three times faster than we were able to do using the *flat_hash_map*, while using a lower memory ceiling.
This is significant, as the speed of insertion into a hash map is important in many algorithms, for example removing duplicates in a collection of values.
### Using the intrinsic parallelism of the *parallel_hash_map* with internal mutexes
### Using the intrinsic parallelism of the *parallel_flat_hash_map* with internal mutexes
It may not be practical to add logic into your program to ensure you use different internal submaps from each thread. Still, locking the whole *parallel_hash_map* for each access would forego taking advantage of its intrinsic parallelism.
It may not be practical to add logic into your program to ensure you use different internal submaps from each thread. Still, locking the whole *parallel_flat_hash_map* for each access would forego taking advantage of its intrinsic parallelism.
For that reason, the *parallel_hash_map* can provide internal locking using the `absl::Mutex` (the default template parameter is `absl::NullMutex`, which does no locking and has no size cost). When selecting `absl::Mutex`, one mutex is created for each internal submap at a cost of 8 bytes per submap, and the *parallel_hash_map* internally protects each submap access with its associated mutex.
For that reason, the *parallel_flat_hash_map* can provide internal locking using the `std::mutex` (the default template parameter is `phmap::NullMutex`, which does no locking and has no size cost). When selecting `std::mutex`, one mutex is created for each internal submap at a cost of 8 bytes per submap, and the *parallel_flat_hash_map* internally protects each submap access with its associated mutex.
| map | Number of submaps |sizeof(map) |
| :--- | :---: | ---: |
| std::unordered_map (vs2017) | - | 64 |
| absl::flat_hash_map | - |48 |
| absl::parallel_flat_hash_map, N=4, absl::NullMutex | 16 |768 |
| absl::parallel_flat_hash_map, N=4, absl::Mutex | 16 | 896 |
| phmap::flat_hash_map | - |48 |
| phmap::parallel_flat_hash_map, N=4, phmap::NullMutex | 16 |768 |
| phmap::parallel_flat_hash_map, N=4, phmap::Mutex | 16 | 896 |
It is about time we provide the complete parallel_flat_hash_map class declaration (the declaration for parallel_flat_hash_set is similar):
```
template <class K, class V,
class Hash = absl::container_internal::hash_default_hash<K>,
class Eq = absl::container_internal::hash_default_eq<K>,
class Allocator = std::allocator<std::pair<const K, V>>,
class Hash = phmap::container_internal::hash_default_hash<K>,
class Eq = phmap::container_internal::hash_default_eq<K>,
class Allocator = phmap::container_internal::Allocator<std::pair<const K, V>>, // alias for std::allocator
size_t N = 4, // 2**N submaps
class Mutex = absl::NullMutex> // use absl::Mutex to enable internal locks
class Mutex = phmap::NullMutex> // use std::mutex to enable internal locks
class parallel_flat_hash_map;
```
Let's see what result we get for the insertion of random values from multiple threads, however this time we create a *parallel_hash_map* with internal locking (by providing absl::Mutex as the last template argument), and modify the code so that each thread inserts values in any submap (no pre-selection).
Let's see what result we get for the insertion of random values from multiple threads, however this time we create a *parallel_flat_hash_map* with internal locking (by providing std::mutex as the last template argument), and modify the code so that each thread inserts values in any submap (no pre-selection).
![no_preselection](https://github.com/greg7mdp/parallel-hashmap/blob/master/html/img/no_preselection.PNG?raw=true)
If we were to do a intensive insertion test into a hash map from multiple threads, where we lock the whole hash table for each insertion, we would be likely to get even worse results than for a single threaded insert, because of heavy lock contention.
In this case, our expectation is that the finer grained locking of the *parallel_hash_map* (separate locks for each internal submap) will provide a speed benefit when compared to the single threaded insertion, and this is indeed what the benchmarks show:
In this case, our expectation is that the finer grained locking of the *parallel_flat_hash_map* (separate locks for each internal submap) will provide a speed benefit when compared to the single threaded insertion, and this is indeed what the benchmarks show:
![flat_par_mutex_4](https://github.com/greg7mdp/parallel-hashmap/blob/master/html/img/flat_par_mutex_4.PNG?raw=true)
@@ -230,10 +239,10 @@ This is indeed what we see:
| map | Number of submaps |sizeof(map) | time 100M insertions |
| :--- | :---: | ---: | ---: |
| absl::flat_hash_map | - |48 | 14.77s |
| absl::parallel_flat_hash_map, N=4, absl::Mutex | 16 | 896 | 8.36s |
| absl::parallel_flat_hash_map, N=5, absl::Mutex | 32 | 1792 | 7.14s |
| absl::parallel_flat_hash_map, N=6, absl::Mutex | 64 | 3584 | 6.61s |
| phmap::flat_hash_map | - |48 | 14.77s |
| phmap::parallel_flat_hash_map, N=4, std::mutex | 16 | 896 | 8.36s |
| phmap::parallel_flat_hash_map, N=5, std::mutex | 32 | 1792 | 7.14s |
| phmap::parallel_flat_hash_map, N=6, std::mutex | 64 | 3584 | 6.61s |
There is still some overhead from the mutex lock/unlock, and the occasional lock contention, which prevents us from reaching the performance of the previous multithreaded lock-free insertion (5.12s for inserting 100M elements).
@@ -247,22 +256,18 @@ We have seen that the novel parallel hashmap approach, used within a single thre
1. It would be beneficial to provide additional APIs for the *parallel_flat_hash_map* and *parallel_flat_hash_set* taking a precomputed hash value. This would enable the lock-free usage of the *parallel_flat_hash_map*, described above for multi-threaded environments, without requiring a double hash computation.
2. We may consider providing *parallel_node_hash_map* and *parallel_node_hash_set* in Abseil, for the cases when pointer stability is required for keys and/or values. This would be a simple addition.
### Thanks
I would like to thank Google's *Matt Kulukundis* for his eye-opening presentation of the *flat_hash_map* design at CPPCON 2017 - my frustration with not being able to use it helped trigger my insight into the *parallel_hash_map*. Also many thanks to the Abseil container developers - I believe the main contributors are *Alkis Evlogimenos* and *Roman Perepelitsa* - who created an excellent codebase into which the graft of this new hashmap took easily, and finally to Google for open-sourcing Abseil. Thanks also to my son *Andre* for reviewing this paper, and for his patience when I was rambling about the *parallel_hash_map* and its benefits.
I would like to thank Google's *Matt Kulukundis* for his eye-opening presentation of the *flat_hash_map* design at CPPCON 2017 - my frustration with not being able to use it helped trigger my insight into the *parallel_flat_hash_map*. Also many thanks to the Abseil container developers - I believe the main contributors are *Alkis Evlogimenos* and *Roman Perepelitsa* - who created an excellent codebase into which the graft of this new hashmap took easily, and finally to Google for open-sourcing Abseil. Thanks also to my son *Andre* for reviewing this paper, and for his patience when I was rambling about the *parallel_flat_hash_map* and its benefits.
### Links
[Github repository for the benchmark code used in this paper](https://github.com/greg7mdp/parallel-hashmap)
[Repository for the Parallel Hashmap, including the benchmark code used in this paper](https://github.com/greg7mdp/parallel-hashmap)
[Swiss Tables doc](https://abseil.io/blog/20180927-swisstables)
[My fork of Google Abseil repository, with the parallel_flat_hash_map implementation](https://github.com/greg7mdp/abseil-cpp)
[Google Abseil repository](https://github.com/abseil/abseil-cpp)
[Matt Kulukindis: Designing a Fast, Efficient, Cache-friendly Hash Table, Step by Step](https://www.youtube.com/watch?v=ncHmEUmJZf4)