wip

html/parallel_hashmap.html

@@ -112,7 +112,7 @@
 <p>If you haven't been living under a rock, you know that Google open sourced late last year their Abseil library, which includes a very efficient flat hash table implementation. The <em>absl::flat_hash_map</em> stores the values directly in a memory array, which avoids memory indirections (this is referred to as closed hashing).</p>
 <p><img src="https://github.com/greg7mdp/parallel-hashmap/blob/master/img/closed_hashing.png?raw=true" alt="closed_hashing" /></p>
 <p>Using parallel SSE2 instructions, the flat hash table is able to look up items by checking 16 slots in parallel, which allows the implementation to remain fast even when the table is filled to 87.5% capacity.</p>
-<p>The graphs below show a comparison of time and memory usage necessary to insert up to 100 million values (each value is composed of two 8-byte integers), between the default hashmap of Visual Studio 2017 (std::unordered_map), and Abseil's flat_hast_map:</p>
+<p>The graphs below show a comparison of time and memory usage necessary to insert up to 100 million values (each value is composed of two 8-byte integers), between the default hashmap of Visual Studio 2017 (std::unordered_map), and Abseil's flat_hash_map:</p>
 <p><img src="https://github.com/greg7mdp/parallel-hashmap/blob/master/img/stl_flat_both.PNG?raw=true" alt="stl_flat comparison" /></p>
 <p>On the bottom graph, we can see that, as expected, the Abseil <em>flat_hash_map</em> is significantly faster that the default stl implementation, typically about three times faster.</p>
 <h3 id="the-peak-memory-usage-issue">The peak memory usage issue</h3>
@@ -121,6 +121,9 @@
 <p>In the case of Abseil's <em>flat_hash_map</em>, the values are stored directly in a memory array. The memory usage is constant until the table needs to resize, which is why we see these horizontal sections of memory usage.</p>
 <p>When the <em>flat_hash_map</em> reaches 87.5% occupancy, a new array of twice the size is allocated, the values are moved (rehashed) from the smaller to the larger array, and then the smaller array, now empty, is freed. So we see that during the resize, the occupancy is only one third of 87.5%, or 29.1%, and when the smaller array is released, occupancy is half of 87.5% or 43.75%.</p>
 <p>The default STL implementation is also subject to this higher peak memory usage, since it typically is implemented with an array of buckets, each bucket having a pointer to a linked list of nodes containing the values. In order to maintain O(1) lookups, the array of buckets also needs to be resized as the table size grows, requiring a 3x temporary memory requirement for moving the old bucket array (1x) to the newly allocated, larger (2x) array. In between the bucket array resizes, the default STL implementation memory usage grows at a constant rate as new values are added to the linked lists.</p>
+<blockquote>
+<p>Instead of having a separate linked list for each bucket, <em>std::unordered_map</em> implementations often use a single linked list (making iteration faster), with buckets pointing to locations within the single linked list. <em>absl::node_hash_map</em>, on the other hand, has each bucket pointing to a single value, and collisions are handled with open addressing like for the <em>absl::flat_hash_map</em>.</p>
+</blockquote>
 <p>This peak memory usage can be the limiting factor for large tables. Suppose you are on a machine with 32 GB of ram, and the <em>flat_hash_map</em> needs to resize when you inserted 10 GB of values in it. 10 GB of values means the array size is 11.42 GB (resizing at 87.5% occupancy), and we need to allocate a new array of double size (22.85 GB), which obviously will not be possible on our 32 GB machine.</p>
 <p>For my work developing mechanical engineering software, this has kept me from using flat hash maps, as the high peak memory usage was the limiting factor for the size of FE models which could be loaded on a given machine. So I used other types of maps, such as <a href="https://github.com/greg7mdp/sparsepp">sparsepp</a> or Google's <a href="https://code.google.com/archive/p/cpp-btree/">cpp-btree</a>.</p>
 <p>When the Abseil library was open sourced, I started pondering the issue again. Compared to Google's old dense_hash_map which resized at 50% capacity, the new <em>absl::flat_hash_map</em> resizing at 87.5% capacity was more memory friendly, but it still had these significant peaks of memory usage when resizing.</p>
@@ -222,7 +225,7 @@
 <p>Using multiple threads, we are able to populate the <em>parallel_flat_hash_map</em> (inserting 100 million values) three times faster than the standard <em>flat_hash_map</em> (which we could not have populated from multiple threads without explicit locks, which would have prevented performance improvements).</p>
 <p>And the graphical visualization of the results:</p>
 <p><img src="https://github.com/greg7mdp/parallel-hashmap/blob/master/img/mt_stl_flat_par_both_run2.PNG?raw=true" alt="mt_stl_flat_par comparison" /></p>
-<p>We notice in this last graph that the memory usage peaks, while still smaller than those of the flat_hast_map, are larger that those we saw when populating the <em>parallel_hash_map</em> using a single thread. The obvious reason is that, when using a single thread, only one of the submaps would resize at a time, ensuring that the peak would only be 1/16th of the one for the <em>flat_hash_map</em> (provided of course that the hash function distributes the values somewhat evenly between the submaps).</p>
+<p>We notice in this last graph that the memory usage peaks, while still smaller than those of the <em>flat_hash_map</em>, are larger that those we saw when populating the <em>parallel_hash_map</em> using a single thread. The obvious reason is that, when using a single thread, only one of the submaps would resize at a time, ensuring that the peak would only be 1/16th of the one for the <em>flat_hash_map</em> (provided of course that the hash function distributes the values somewhat evenly between the submaps).</p>
 <p>When running in multi-threaded mode (in this case eight threads), potentially as many as eight submaps can resize simultaneaously, so for a <em>parallel_hash_map</em> with sixteen submaps the memory peak size can be half as large as the one for the <em>flat_hash_map</em>.</p>
 <p>Still, this is a pretty good result, we are now inserting values into our <em>parallel_hash_map</em> three times faster than we were able to do using the <em>flat_hash_map</em>, while using a lower memory ceiling.</p>
 <p>This is significant, as the speed of insertion into a hash map is important in many algorithms, for example removing duplicates in a collection of values.</p>
@@ -319,7 +322,10 @@ class parallel_flat_hash_map;
 <h3 id="in-conclusion">In Conclusion</h3>
 <p>We have seen that the novel parallel hashmap approach, used within a single thread, provides significant space advantages, with a very minimal time penalty. When used in a multi-thread context, the parallel hashmap still provides a significant space benefit, in addition to a consequential time benefit by reducing (or even eliminating) lock contention when accessing the parallel hashmap.</p>
 <h3 id="future-work">Future work</h3>
-<p>I believe it would be beneficial to provide additional APIs for the <em>parallel_flat_hash_map</em> and <em>parallel_flat_hash_set</em> taking a precomputed hash value. This would enable the lock-free usage of the <em>parallel_flat_hash_map</em>, described above for multi-threaded environments, without requiring a double hash computation.</p>
+<ol>
+<li><p>It would be beneficial to provide additional APIs for the <em>parallel_flat_hash_map</em> and <em>parallel_flat_hash_set</em> taking a precomputed hash value. This would enable the lock-free usage of the <em>parallel_flat_hash_map</em>, described above for multi-threaded environments, without requiring a double hash computation.</p></li>
+<li><p>We may consider providing <em>parallel_node_hash_map</em> and <em>parallel_node_hash_set</em> in Abseil, for the cases when pointer stability is required for keys and/or values. This would be a simple addition.</p></li>
+</ol>
 <h3 id="thanks">Thanks</h3>
 <p>I would like to thank Google's <em>Matt Kulukundis</em> for his eye-opening presentation of the <em>flat_hash_map</em> design at CPPCON 2017 - my frustration with not being able to use it helped trigger my insight into the <em>parallel_hash_map</em>. Also many thanks to the Abseil container developers - I believe the main contributors are <em>Alkis Evlogimenos</em> and <em>Roman Perepelitsa</em> - who created an excellent codebase into which the graft of this new hashmap took easily, and finally to Google for open-sourcing Abseil. Thanks also to my son <em>Andre</em> for reviewing this paper, and for his patience when I was rambling about the <em>parallel_hash_map</em> and its benefits.</p>
 <h3 id="links">Links</h3>