saberbud/parallel-hashmap
1
0
Fork 0
mirror of https://github.com/greg7mdp/parallel-hashmap.git synced 2026-08-29 16:40:39 +08:00
Code Issues Packages Projects Releases Wiki Activity

wip

Browse Source
This commit is contained in:
greg
2019-03-03 10:31:13 -05:00
parent fcd3d85d1c
commit 40eeed0c19
5 changed files with 193 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
Vendored
+3 -1
View File
@@ -9,7 +9,9 @@
### A quick look at the current state of the art
If you haven't been living under a rock, you know that Google open sourced late last year their Abseil library, which includes a very efficient flat hash table implementation. The flat hash table stores the values directly in a memory array, which avoids memory indirections. Using parallel SSE2 instructions, the flat hash table is able to look up items by checking 16 slots in parallel, which allows the implementation to remain fast even when the table is filled to 87.5% capacity.
If you haven't been living under a rock, you know that Google open sourced late last year their Abseil library, which includes a very efficient flat hash table implementation. The absl::flat_hash_map stores the values directly in a memory array, which avoids memory indirections. Using parallel SSE2 instructions, the flat hash table is able to look up items by checking 16 slots in parallel, which allows the implementation to remain fast even when the table is filled to 87.5% capacity.
![closed_hashing](https://github.com/greg7mdp/parallel-hashmap/blob/master/img/closed_hashing.png?raw=true)
The graphs below show a comparison of time and memory usage necessary to insert up to 100 million values (each value is composed of two 8-byte integers), between the default hash map of Visual Studio 2017 (std::unordered_map), and Abseil's flat_hast_map:
diagrams/closed_hashing
Vendored
+28
View File
@@ -0,0 +1,28 @@
+----------------+
|"(keyi, valuei)"| "(key, value) pairs are stored directly"
+----+-----------+ "into the array (no pointers)"
| +--------+---------------------+
| | | |
| | | |
| | | |
| | | |
| | | |
| +--------+---------------------+
+---------------> | keyi | valuei |
hasher(keyi) +--------+---------------------+
| | |
| | |
| | |
+--------+---------------------+
| | |
+--------+---------------------+
| | |
| | |
| | |
+--------+---------------------+
absl::flat_hash_map
diagrams/closed_hashing.svg
Vendored
+133
View File
@@ -0,0 +1,133 @@
<svg class="bob" font-family="arial" font-size="14" height="448" width="560" xmlns="http://www.w3.org/2000/svg">
<defs>
<marker id="triangle" markerHeight="8" markerWidth="8" orient="auto" refX="4" refY="2" viewBox="0 0 8 4">
<polygon fill="black" points="0,0 0,4 8,2 0,0"/>
</marker>
<marker id="clear_triangle" markerHeight="10" markerWidth="10" orient="auto" refX="1" refY="7" viewBox="0 0 20 14">
<polygon fill="none" points="2,2 2,12 18,7 2,2" stroke="black" stroke-width="2"/>
</marker>
<marker id="circle" markerHeight="5" markerWidth="5" orient="auto" refX="10" refY="10" viewBox="0 0 20 20">
<circle cx="10" cy="10" fill="black" r="8"/>
</marker>
<marker id="square" markerHeight="5" markerWidth="5" orient="auto" refX="10" refY="10" viewBox="0 0 20 20">
<rect fill="black" height="20" width="20" x="0" y="0"/>
</marker>
<marker id="open_circle" markerHeight="10" markerWidth="10" orient="auto" refX="10" refY="10" viewBox="0 0 20 20">
<circle cx="10" cy="10" fill="white" r="4" stroke="black" stroke-width="2"/>
</marker>
<marker id="big_open_circle" markerHeight="20" markerWidth="20" orient="auto" refX="20" refY="20" viewBox="0 0 40 40">
<circle cx="20" cy="20" fill="white" r="6" stroke="black" stroke-width="2"/>
</marker>
</defs>
<style type="text/css">
line,path {
stroke: black;
stroke-width: 2;
stroke-opacity: 1;
fill-opacity: 1;
stroke-linecap: round;
stroke-linejoin: miter;
}
line.dashed {
stroke-dasharray: 5;
}
circle.solid {
fill:black;
stroke: black;
stroke-width: 2;
stroke-opacity: 1;
fill-opacity: 1;
stroke-linecap: round;
stroke-linejoin: miter;
}
circle.open {
fill:none;
stroke: black;
stroke-width: 2;
stroke-opacity: 1;
fill-opacity: 1;
stroke-linecap: round;
stroke-linejoin: miter;
}
tspan.head{
fill: none;
stroke: none;
}
</style>
<rect fill="white" height="448" width="560" x="0" y="0"/>
<g>
<line x1="52" x2="52" y1="56" y2="88"/>
<line x1="52" x2="188" y1="56" y2="56"/>
<line x1="52" x2="92" y1="88" y2="88"/>
<line x1="92" x2="92" y1="88" y2="216"/>
<line x1="92" x2="188" y1="88" y2="88"/>
<line marker-end="url(#triangle)" x1="92" x2="220" y1="216" y2="216"/>
<line x1="188" x2="188" y1="56" y2="88"/>
</g>
<g>
<line x1="236" x2="236" y1="104" y2="200"/>
<line x1="236" x2="308" y1="104" y2="104"/>
<line x1="236" x2="236" y1="200" y2="232"/>
<line x1="236" x2="308" y1="200" y2="200"/>
<line x1="236" x2="236" y1="232" y2="296"/>
<line x1="236" x2="308" y1="232" y2="232"/>
<line x1="236" x2="236" y1="296" y2="328"/>
<line x1="236" x2="308" y1="296" y2="296"/>
<line x1="236" x2="236" y1="328" y2="392"/>
<line x1="236" x2="308" y1="328" y2="328"/>
<line x1="236" x2="308" y1="392" y2="392"/>
<line x1="308" x2="308" y1="104" y2="200"/>
<line x1="308" x2="484" y1="104" y2="104"/>
<line x1="308" x2="308" y1="200" y2="232"/>
<line x1="308" x2="484" y1="200" y2="200"/>
<line x1="308" x2="308" y1="232" y2="296"/>
<line x1="308" x2="484" y1="232" y2="232"/>
<line x1="308" x2="308" y1="296" y2="328"/>
<line x1="308" x2="484" y1="296" y2="296"/>
<line x1="308" x2="308" y1="328" y2="392"/>
<line x1="308" x2="484" y1="328" y2="328"/>
<line x1="308" x2="484" y1="392" y2="392"/>
<line x1="484" x2="484" y1="104" y2="200"/>
<line x1="484" x2="484" y1="200" y2="232"/>
<line x1="484" x2="484" y1="232" y2="296"/>
<line x1="484" x2="484" y1="296" y2="328"/>
<line x1="484" x2="484" y1="328" y2="392"/>
</g>
<g>
<text x="57" y="76">
(keyi, valuei)
</text>
</g>
<g>
<text x="97" y="236">
hasher(keyi)
</text>
</g>
<g>
<text x="193" y="428">
absl::flat_hash_map
</text>
</g>
<g>
<text x="241" y="76">
(key, value) pairs are stored directly
</text>
</g>
<g>
<text x="241" y="92">
into the array (no pointers)
</text>
</g>
<g>
<text x="249" y="220">
keyi
</text>
</g>
<g>
<text x="321" y="220">
valuei
</text>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.9 KiB

diagrams/index_computation
Vendored
+29 -26
View File
@@ -1,26 +1,29 @@
+---------------+
| "(Key, Value)"|
+------+--------+
|
| hasher "Parallel Hash Map"
v "(with 8 submaps)"
+--------+-------------+ +----------------+
| h=0x7d84ea13707f4657 | | submap0 |
+---------+------------+ +----------------+
| | submap1 |
| "(h ^ (h >> 3)) & 0x7" +----------------+
v | submap2 |
+----+----+ +----------------+
|"idx = 5"| | submap3 |
+----+----+ +----------------+
| | submap4 |
| +----------------+
+------------------------------->| submap5 |
+----------------+
| submap6 |
+----------------+
| submap7 |
+----------------+
"parallel_hash_map with 8 submaps, each submap is an absl::flat_hash_map"
+----------------+
|"(keyi, valuei)"|
+------+---------+
|
| hasher(keyi) "Parallel Hash Map"
v "(with 8 submaps)"
+--------+-------------+ +----------------+
| h=0x7d84ea13707f4657 | | submap0 |
+---------+------------+ +----------------+
| | submap1 |
| "(h ^ (h >> 3)) & 0x7" +----------------+
v | submap2 |
+----+----+ +----------------+
|"idx = 5"| | submap3 |
+----+----+ +----------------+
| | submap4 |
| +----------------+
+------------------------------->| submap5 |
+----------------+
| submap6 |
+----------------+
| submap7 |
+----------------+
"parallel_hash_map with 8 submaps, each submap is an absl::flat_hash_map"
img/closed_hashing.png
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.9 KiB