From b795215260482ffedce2c996b28d3bb3197ffb05 Mon Sep 17 00:00:00 2001 From: greg Date: Sat, 9 Mar 2019 15:26:27 -0500 Subject: [PATCH] wip --- README.md | 6 ++++++ img/lock_various_sizes.PNG | Bin 0 -> 42193 bytes make_chart_data.py | 19 +++++++++++++++---- pdf/parallel_hashmap.html | 2 ++ 4 files changed, 23 insertions(+), 4 deletions(-) create mode 100644 img/lock_various_sizes.PNG diff --git a/README.md b/README.md index c7b9b5f..36c6e87 100644 --- a/README.md +++ b/README.md @@ -208,10 +208,16 @@ Let's see what result we get for the insertion of random values from multiple th ![no_preselection](https://github.com/greg7mdp/parallel-hashmap/blob/master/img/no_preselection.PNG?raw=true) +If we were to do a intensive insertion test into a hash map from multiple threads, where we lock the whole hash table for each insertion, we would be likely to get even worse results than for a single threaded insert, because of heavy lock contention. +In this case, our expectation is that the finer grained locking of the parallel_hash_map (separate locks for each internal submap) will provide a speed benefit when compared to the single threaded insertion, and this is indeed what the benchmarks show: ![flat_par_mutex_4](https://github.com/greg7mdp/parallel-hashmap/blob/master/img/flat_par_mutex_4.PNG?raw=true) +If we increase the number of submaps, we should see more parallelism (less lock contention across threads, as the odds of two seperate threads inserting in the same subhash diminishes)m and this is indeed what we see: + + + ### In Conclusion We have seen that the novel parallel hashmap approach, used within a single thread, provides significant space advantages, with a very minimal time penalty. When used in a multi-thread context, the parallel hashmap still provides a significant space benefit, in addition to a consequential time benefit by drastically reducing (or even eliminating) lock contention when accessing the parallel hashmap. diff --git a/img/lock_various_sizes.PNG b/img/lock_various_sizes.PNG new file mode 100644 index 0000000000000000000000000000000000000000..123c423672020e8e98bf466615fbc4b77d0d56c6 GIT binary patch literal 42193 zcmdqJ1yq#Z*EUQFh#=iVcZuZC&482$igcHB=YWJrw{$2&ihy(pNH<7Ghjb1|OTRbj zub%&Jz0b4W?|av`)_1SPat7u;bI!i^Is3ZyzP6!{RUlY*$?hT`AYduT%RE6qKvqIP zKukct1N`QWefBKy52EuEh!jG}AjKx|2E{^BSrP%EEE@CD2o-pb;UKT&jDUdMcKd_a zWnW;5fRJOZAS0>ap$~6FO`{N~soq|2y6!$}g3dQCOxbrYwmUAxwC!*SrMwJOSt2o( z-${1t?B_Knp}fz-qdxjUUGeGH;_N&`S5{pU^i6@*-p4OmgK5L(%Y=pCwY(SPgQ_=SE-1Q2PW_oEK<@Ho~6W7hur{KXG z1pTx2@4`#4UYw3()9;DY!=~zN{Q{zu=E%zB4Wt4)3!ba052Q#_OnVKeTT#JQ&_!DP2(f$XWu8CI(TC@| zp`6;L15{=Nvbxs7_SHH#+gg(|j7etO>g$I*Tld$sB)#V-E@bu=!1f$7o}lGgzfsBv zX@MgrlPT#OU&Si9tZt~0eZC?O$a8i_NzM-T^*Z8UV4YPw&T{C9bV4E2y<>!k*!!41 z%&@$QY1+Q)AVbkYt5hKe3am(C4Y7q6f{xa)ruzyPJA0?H7OWhMPkI7vjnZicmMCE% zi2Is_HiFiS+%unPk_>EF}>;a6I~hdWk3B}<1*fQ1vrDeE!NTXiA#Qu3$^Qd&zgLP z%Sdt56O^LE?KOFBqORV7MQ6v4A{k{|zrrxHIp zLvY&0v&8EnUC$6sKH+=f5w$xBmuoe*Sjx0<72F%mg(iw6sM8K%0+a2gpiW*x7g+Cf ziP~9c5KiNVuwpaK4Jfz_EL$cG-#PRacs7oHfNi$V9J9&gX=`$lhuYDK6&Zv$L)0dOWl9WO#jN} zc3x5Il&vzJnf$`|r=~@Ho>_JQEZ6Hl&wJoew++^vJWBoFbqcMdFY*8lE&p+D3oM>I4 zYuAQ!;uVN_bjq!Vu4+K7IW3++_`@CS*Q5UW%{%vmKq5MVc7q=qMdc_Vbtm3VM}7qB zyb-{L&z-{%!BiAn5?k?h7Fs5906(H8C~CcZ4(cjzpp4ly7-G4b00 z=8o{i-!PSKHa!i!KAv9qVZA!O0$a&sI*%sP_`no*-?+NZ1y@g0esEeFQ&YuSIGld_ z7)+s;mcO=J+Y5Jg+}-iCfx@@CUu?MU^UcHE8pcO@NSA;mM;5gow zlY0z37d0~}I+R^ioI~?bxdSB%&HAMaQ&!n^8f&H9k@R#>?lmf`fM*g*qxi zcOBZ^-QpIBJfs#B35t-|z`AExotD97l`TK)22)#Mbeyo;Fol4Sf&`!f$9I-F9b#to zoE>q=l!-!g)<6Hm`qJlj~b4ns{5WG_4Zml zj5=CI#ngTB3|rPoVZ}oDpm+5*R{3o6uqdF10HTJfqKIu~e}ozW({NONUwnr!V7dd3&`#s3m#M^JjmJi)YYFlgRjW-%HOv zc8oj47UIwg6vg7)6CQJL0+< zkzqxscoe?Ue!NXU~rh(9a%=&T;kx+t5>pPy(zH*>5p-{=__AExJdRRT9 z)k*$}Bp1Vd{S^t+kHT+}CP9x8cd!c(0(t_HNJ5=8x=gU*w_T%He4X?gL5Y?=t=LDOv(V3FzPayQj>ebrYdFO{CK7YMWzEqm<8RQ z+ad8++7%d4C-jIY>IdIsvHzUf?HJU_FOkvnR$9U-3K0G=YeWEp_5>0O{+ibxe=t{S6nuw&w-R-GZ2G^? zoC*DFd4D}MYPPUjH=zbms(R$r032!GkB1?A^G z+Eu+j5=g#6C56!e|ALu?GcD* ztSF<&evi;|6_v$U;n*QVuQCvqZO##Xj-Dha!R4=n_RFpm1vpCXUI9|{%6E>`7>E>o zMBbPODmm!hE;$7m1;5+n)(dcyKKtKo8RH?sLn(i`y#njz-N#?}yDe5qEgOFM;dZBcC=V60oMizVk(Nl&s~saN$kFMjalqo;f>cb&adn|Oom};@ ze5vm77_R-ykx)5)Q!EL+OBPEo!y@ zder~tMna^CTHz%RU}+YmmWxz+4$k;+ND7hZ&^u4f-7O=fe42{c((X!vU9+$tA&8^o zED=6FH0Xx8-`2{J?AiAU z_2b#Xo@*N;PnW?)LC7`{-$UpHM}XS~qta_aNFBwTICqM)gMtPM-weJ2xzKSm8xbp= zS2Xy#%w|vziKXwFnN}$jTh=eH$UWQ-Lvt)1zf*U=PNlE0$M$Fg z)A13Tbdi9;3k!{5D`KKWp-2mkta!&wcRenv39F$2ErA-rG^c*I6zH;-J#hPycD@Hr*NO8mS2}d?1;{9Fi~EagBn_<$=-KdcBn^M7ac8aS za+6HmqBB`1G<|C+*#=|>jD39xDCsJuOIrUdU$9;*Zley8*8DlnoA3lLoI7$a( zdBH7jLxQ`yEViI7jZY0mHKv4ltdov#k+rxK28-o>eA?NV4s{#Eo-i6Cc7BvOOPRJz z37T_s2alJ_ms790Zw7L4o|39NB!f%dpvI|atQ**AR<4^;K)!h&`tmDaFe-{V8O&VF z$$qesX&90}8@zT*EX9R;dIATc|rD|lUW75khX`|;6k3B^|(XbVU=v|L6 zwYw^L$o+%)bDN12PYw-!2x?JGkrFC(GE3J&gLyQ=DCJywIf1z?y8-Nqnv z#&3psDC&KWLpb@*Hsq^RKWK47l9ZxCsuW`r#>g0nAu~$|4d4kCG(oA!sZ5NP6~>U2 zliG~a_DvheZd>2xiPO&UWi0=ozhT%{;!u%%k}Lj(0EI1K0eT&-GI6GD06q1rrnDpd z({a%q>GM-?l%SmjBb#1S8TMm=qLtXfsA}5|F|1z5D~fJsadX2&U8!w0LdBhqlL~}` z1&eQmDAapAmD{b}q?-aZ=;hkVvX5Puhd4Fp?mC~Ks3$ai6lY4hi2S_% z!8S2PuO)|Odh3(@u$#95`Keh|SGVWzvx$RT?*H6N7%eVor+V+hb{bqB;j4HmS1gaE zs8~!sd;`+H{s2v)TMY%T23OeVvt?C)fAHr|?Ou|*et!qWim&n+z~B!G&`(sJl-?0AQ;PZZ5#ri>`+o~rcSfptTG z=X~yd=;yzjN)IG2bQtAnU42`O2U!i55#9bCR}$zK{TQk-7_TQ*QEnW;tH~HGyrlFr zaVzaDRBOTxy7BaCEeSavg^r?#77P2zVO4 z;TZbyBj1mjS9a0<;T0@uyi6Qj-!0Rv8F*hq@Rbg<@l+ubP1kKsAB2mdTyf3y+}62C zEqLzoFeG#1f=d7()XvR)X{cn^IbH~ zVcsVOs#ryu>SyI zcHALFwDAgHq11@vnybICt9PBy(>QO%T)F{ypO9FLpt)HhBwOExb-7;=%2{DqBF3)r ze6t4Xx#V0QGZN7AbfZ?K)R{?Fe~eGz(D{-6#4t>XK9cPC8T0^y@PQVsSU&zgBfeot z!6v24Q<+AsKG(3Z@yZ8L`IjxHQ8pc5vQGz)_;h&0VTJXgn=g zIa}{ocR`9`1K=O$A7Q4SFn{|>ihf@#`wVoxj_(@ZIW@+3oWDYv-IH!iY)jwW^sFaU zllXD$;W#eSMU^?vQYs@!;1r%xd~xyE(R!9fjWH@#QUb|>TrG3~D#c4Ec3$wq&q5{2 zkrDcG`7VG<`~C>d6{z<9quc~Jt%C>vPI(1+bIv3+CN9PRM7Cf|-SA}WMBRK*lz!->=eG!2FyUmwXBBo~a5!(vtn=4^yZ4l0K>tmjPev`!iK zniG^~mWjUULOxyb>5M+YuN%^k-V!&eusdCxvwsNZ#PBA%`9ZrQQG+^Uw|er(4UB(d z8PB^v&r~BQiiHX|naD?Bu6UQ|R~ss-FFASTP~uXspKUIb$uG5M*;k`Ru&I*+q~DMq zMSZ5)R*L?tzM;dhr()>IDugO@=w3AA{@E@*Q-r-aIyz${N>EVkp1fbj-lY_mGTQFC zLjzXJO680xYCKE8a*YKPMfi!IhK?enCC6yfny4Y99G=!RG z^_lX>8zS+ir>Itj#$i(G>PwR%Ls@4HE(ifu*Z?pXt2YbBk-2UXbJRVUb=Ek$zF6XE zL!^9U;MD_ugtJIrCT+>sW5kW3&M)lP=^5Te>ZWo`;P#e1Dro?>M$l}>!)$ehT!n=x zt&J!bx)Wk=?{skKSWw`RE_Q`VatDgC{Z|Vj{KWac46C@jHk8zBR#3+tn%2FtG zJ;qV0KnSN390)c%I6gveX)oN^@9MbeTkwgWt`fgqwQrOahzIH6sYLJalB+aszDyl_ z1`gcaxSOx8%O+@XP|UbaOuu*Q|VWIxk}rFWZo33};QKsgmkUiYCV5$+u>_I()oY;5h;e-eDZ4isC@ zZbo`azfcOZnMzz;i2Gb?zf5V?F#L#n=h2Y;g9*?3UcS*?apk`mhA~avlxu@(g|ON- zz9dEJYboB{Xurx3=&}paFUKJuA3~2 zueJS@x>x0MW?Kk<1wyDXazwEK#TDZO-12q9oCB$TbC&!WryIu`s-X(^NZum$h$Q^Si*SP=WU6!}Um*~Hd$`!t5BTkgvMs+&^EuAC{`d%{EGoq9%6-rz zzq8+M&w{u z)nWu5)A5SUT3ZjdaIh2~b2$pGVKEpEMQoUC>dT5x@Ab5gKfq3l-L9^YrbxT8P)@w4 zTwgxW+f&$8y-L}ds9o^BJc$+h{VYAgbe%m?Gn@$&lq$Bz^QQ*J;^=-tI#m2<&N=h5Z3A^Uv`>6USD<{xeO61trGmNK!^ZJL^ z914}8LyETF-!Bka7+9iugzvceln zM-hIp2=4piB6=n?tfMkt&N}6B?(k$3gez5~CFfGR0j#)12_x^@B#GYLtGqToXO^3(Yk28yeEee1#(7rf^O$#+biI)uRRKglIf8$v#l16iSa(PIhbe zJD&HTiJK$*H_cc4$C-jAK55i$kTDkmWu|U0wy;<0JMQ#6%RwEDuD&pbZxNqb9pB6l zz@XT#Is7jZt!2gP8p{L8=UA$!cZHuc*EqkYf^4lU|2XfP=11Oaf@9arH_KgH4)qVq z=}^gbcydf^Thb+&*$0Bx_{px!DPLoTGyXVoc4}PoijW40W_2Vo=;p8VfX_LEvWIf{ zLwpn9Pddl-FAgil;3L=;Q?V9Ce!gDw@@M0R=2V-jGo^&^%DF&Tu@%CvbjBgRbLt%Y z1OeopV($KKSQSWrjPV7I?%)9y0RupFtlG5*M*+XMcx4-&J99cKeJp-svy!Z(^je6W zJ;=lqY!=KzjBWCPVQTT7u;Ga_QP<#_(oDM8+-6)Gvks`$67CFkCEH)CG1&HeaB>2X zp4DkDtOO@ygUrunDI0YwC@bddup8fO1oYUH`cP%toHg05yQZIRjyLauOIL+Jwl21T z<95ULoyWmYQwv4UY!}~3FLn~w=Lcl;?8!S_oXJD#s>27VLyALmB^6>p@_LlV+S_qi z(lX3pIT>Fr=FMSiu7XZy)z+Znt#z14Y68m7xZZGvxM$J6l>b3k-fnb9W&Et*Eu931 zG)?L~GGZ3sBfpsvdldLQBXqLhnRXj*Rt%^;If^a$rkzHPJ>+scHd;oZb64A`eA#~4 zC+I?y^I?Yeuk#_PX=8LJpk#ChTtCML_l|NC;D!8U)AmugM^NaiCsReGVMAA*o^nxV zs3`a!V+ssM49QamflOD96xWWV*GcFL9JgDZSr3 z)7N1UD;+YhPgkb;r={=$U)>Ld^26AiugyThf> z9lcB!=rrauZPSMr4^MaNHFI2RCV)ghk)t`;cVD)iod*eH`a%_ZwPB)B5m=Dbq2WbQ z-=(>nidRnM{a2yoRt=S18ZLNK5cyIQxl=JbdZ(kqp~>bbzl`zYo28q`K-2_98S~OGt=xjdIK(s6(5hgJHmVfYW{RF?wm?TfaBnRB^@35vs0%k+<8_C*|%Sv6NFVdoLf^~$m(Nkf7*E!&w-D1HTd z+Iu0EZiw}>O9b+e_PjD**4=wwT3EO1>V2V=8146OQ5|_e;ir7v2x_R~BIY^#-2Qz1 z(BN^K1oD;Mk^2Emv2`;x!@R9f0Q7it>aN8Xklvzvm^H(?l2~`DZ(*6_tn>9yka2A7sbpn2!(gVnlb@dPf zDe;Phb%4TAMQkfPH1dk{xy?Y-)ckjv3F1uwMXiGhUWWvZnes~=Jd#sT6XdK(sbS=B zrq6XgFmqwKEQkG;=Jc30kl$BQ8+{yNS91|r==zMN-8zBQD=oNe2u_$&(@CAkO7o+nY&d5Aw(N#B5} ztgJR+8S;qlU7PyW4pOKJYVCe3m>7&w|j?4T4R(dr-;M1OQT-;>nH3U)v7B*vdkcfRK?#XP$>npquhM+ zl)M?Ug&=k0b+F&3b2`zF7l`|U4{23mHx0GYsb9wz(NH)R{clnIe3Zk7Lp)%qP&(n} zC7zlAo6_9q+NdAriVa-#hIm>}cWCd|DDR>*Hm+Ui#QES89c_!MzILo2KDD6!SfyD_>hBYMa&f#xjK4g8 z)7h06mslH-VB(5Pzq81VzaFdjAs=a-^5r=ZQ!?#4QB zcA}V*vqqzXRq0WBdUMOhsf8`D-vGW{I84vZRTk&eR+tp-*|1O(%H#J3Z?h1a1che_iy|uD6GR;e zomS|bXjiXb%w|L)k$pr+olM_OWemUnzD*(S1_5#v_#tWWxJz6wnWd#YcJDd5<5HC( zbQ!wjjZgF>de>Vc{bu*TeT;wBJY4~yfsQq*YTZ&uveWTLO4z!E2n-+q5$o)*naE5{ zaEVizaplnBM~0XMr{Zp=8cjC`m~bvpI~1~Y3gmv4W4i)2>~Esj~Wr!DvvuQyK4 zP+>XR^5VO&Q)`q$xN9ydet33L&{e#sxu}W=ks-L>X$zyZo5r3Ckof#5l=to+RERlcmGOo*-p6g{?H zWUFeNK^$#H)RO6_sb;FcQ!h%l39b!kJU?TOW@G@4J-}L%8R9a=u*ufJm+1Y}nDi>^ zWIh_xptpm=ILeZAxj=1LscG%8qMzX$MI8F@fn`)6WAc%o?RdO}mRnbg)6R#srJ(2O zidYx>yeSwh?Jz~f`w#e`?s0 zAKf>`g*Sag)}F)ZtP_4lYvo`+k!(9JfHu@mdZ z_j&8E8);~{s$dwsTdNCh;trJ;kyIHTIcjrEoO1K^d6C06-%pn?oJZ;956nlu|M>XB zw<(vrugaq0GI+Z#;Stuw#sYR)+^AFiqMKeoq0vIKUTZn5O%1csB;^KhMLqz0wsjYjcv|6rsO3fM!96A&a34y z@p*fS0l&$2h=wKpb7t5iMo1}xy3?R&?0x3pT=R9_1{j}M{nT21a7zq{Ez%)5FyL;0 z0x@q{{$%Z4(8Q+eZcx6PEa{tr7t2cA@c}HOe^0~nq`3*?VPwfEBg~BieJRtn_;^P@ zt9?7+yptw|8w(hU6R>I$c!JL3!>_u6bIjou939Qm(Db&jpzt9jB=|+km?jmkST5Dy zs}klTyHcsUi}J3ybCNDFS3PORJwj-xSOLA^ne z=1n=vjzpQ7VAVR@LSl8A7g{oH9=8dAi7LE(#I;&Xmdkgu1rxtkrGopJgb4MHJV>}l zp$TYI^wt1^!JV-=J@;8Ao~yS+MlF!o2_Y>3HmvSo6x5%^?cyaS!a80>qJbe=Iwc z+2Fhc0m1p+EsUKBuyUGY2@qmKlA_kr6PK$Z_NiRV^c6bCkV{4gLHoV1H;X+n=(P@I z^;z1%#V|?f_&=)xmTBf)U79+VOP6n--9hO2rO+7#lplT8otAUW7q0;Qx~SAL^0URt zRkbUMe2!@B$A`s7&PabFm!bERl!nX5ig4zzwy`MNtF);cB@)TmGy37zyX^XaBZu0MDYa>7hc>giAo(iC)Qv?lS^ z#G7hR#CC}T>M+=}t*-4e2Aspp7C|z*JYQqZRVP2pKs`>=lID8SOu9hu(a&DCKgn8o zTMD`eP>8va~5P*Y^b&8c6zY3{7Ft<$uP|vR)ptmxL$aJEljv$-jC&Gw9kr zVX*lBq{tHb;Ny&Rz+{$x$M{Y%9E-|HnD)w?MbS6$PE`47b@))Z9_gvAOPGy-wnDJp z<{5ldXisuzZ~yrdB6LQsERu9|HD!axC}!PNp)fi=v4{0zTsS#u3j}jE(Mlc564>a- z*lT#d{8Qc9Uo6bU_zeB4!HHVOLudlaX@*h#_wc5?&8@6hKIPGtC6YRD)Z(Grx(?&C zAUQ5%`ZCFPs?3$PW@xiraKD%8Y-jgJ=;4y$Tu(&bLjnNCJ~*4FczzzT9a7p{iJ5VQ z|FtFQe^hB9G2-sSIW77s2|VArk9%;Ue(CsdW0CSuDOmc9~slbUTCufX?Yk%0dVD3a0R+{ zcpj4d&h3i9!!egL4k`XzXYa>4-O;)qt#vvc#67bWJ9CqHDjg87p}4NA{SR4M)&GR0 zDKtmRlSiJ+oIn0%rOJ6_Uf5M2&C4#?(rz1PCx7&+QI9f2EOD1_s|p~blMz09C%!mc zv+jeYb*kQCoC{T|tJh;$g}#xZSt`E)yh~t*s;vEEd}$Sbm+W!K8ORrxQDj~;_LnG6 zA_OQaRvH2-8qRAnEtqME39C520-OO++fP#*bitkiP5L`eOIP^c^R)k_`jRr}Tl;^) z)9`9~^n<%r!>$??PzpVz-mh$6Aj=TzTSmRHeAK z5!VxCai`-nef+X0THB5ZmuBc^?Zsz)jb7F{8zyYB+l`%V@<>18L~uGz;<|gCdGG_Z zK154;);*`0_H=D2MEksZK<&6Sw{gUINvQY~@9O%4ZRLDE%um!&f7|l)nT-vUJ_g=D z;I?u|qIeeu2SbqR=A{s?2DU~RJsTl*yI`so8Y+F&;ug=U>HP+fAE#Ot=Nd$tlk z!zNriobdz`py0>e!ujWpIVxgNrOh8~d17{Fb{)2_tXHB>`YCKHkJU8NC!89i_?@&) z_(M$WV8ixnjSIp;9)y2#q*17yLjpO6`-i=;< zBv-2*zIJAi%vV=uo1S#5IzwsDoFx0^XO*LF_kX8a2uae@vs16%l5lWK`=)I=9MjKP z9;>$*$_L%avifWFjG6jfqdQ7&q5?7vN!Vl%o^S`Rptko6ZZZEKmtbl!7(0C!7v0Df z(rjH=SR;<7Ej5j|6m>o`RZ(8Yhia6{ zkAV7x1{Y~M-gd-YA!ORh8ul3Qt;E$+^`+MuwG-$Mu_Z=JLbHif1S{Xdfx35LS+Lb= za;%9jjc|y56k9j-5)K`ajITmJy(G8WL|i<`k9?hOcF!?%a|~cC{e+FFKMKFd#MJG@ zra$L(%xOq`hogLFjoWMiGmoIKY)yg630ho;jwjNC(KG^9~$$!X!|&4`U^ zhjV6?k3MGaryqqoKLs);Q^x)FcdWmU4wJ1j$B};U&tEP?`7I&_m=!l5M1_^NjXsAZ z+{uYF#EbeQDB8oJY@g894k)u?2|NmyNb9OPfXdTs93w^xJ&IE6gv`swsyLY^$x$y9 zkb!vnKZ<%S(=J3L?e^;ohWp~~+i@!=5w`uMZ~v?4^Aa?_U0}q;oHq(8EW1mu zV*WAMKlF%HHtf#tM|&$N|EnsZq>SqtvC%>X<_grK&anC(3LFE;kA}(2(SJ+Lx%65p zZKk;E9TuhJ@^~x;Yki}Gc+)TdwcX~$UbDb_<(B`lko!Gl6#xsoEUEqhY9YhcR2?~f z){KrZ2va@;{WVTP&tK~N0m`Ot1Xb9t3*NTvk$qT{f<^Bhp^UITZ&O12EnWVHlAY7h zBd4HJ7)M`|ezYhwD{P)x&&KI4J?@8hbP@nrNxj#9qn2a&TXTy8aSHgmZXPK6Y&tR7 zz+~6Sp)T=(d>T5ac=_@lgZqi(-F`iw(YFjteP%Tt^JRXB99&PHmFt|cp#UPf@3d$? zCGDSg@nG2k)Im6q3KJTaPvfka;Fr>l7B7Y5qzQuL4t}{|0=*P_8FM*$9#L;koNOFt zCPgZ1hyOn&GI{SJub^6-Ie>3PLj~PPBOk=$85!^L);KOq=!!Rc6?y;H>K`Ql3UfRd z1n4QT#aB9;TJjDLass{L#yO*dsg2vz|G1kq)H~c1ibG^WYc{NeZTgR1i;Q>)%dlyP zR4V%2FDyxhSmr_tID!KA>HLORf9hcKWQI}HqiAdnYy%WRNYXw!;;SLie4>A6B3K=* zF|}2mwD}Yg_m75z43!%hd8M+3*?yS(Go$v=T_6=-XvHj#kCu&x(r!&{LKOK2+)QDILu_tPWzTT4ba zeZ%%w6cykgJstIy#hjCJ^s)x z*YW^RH~thjDiOLyx)kLF4r<>2vdAj>JTS$n$|dhbOv!|cnwD++_A$^fbI2yirHtHQ z=!6d`fvHdi^@X#Rb1^)4vq(jV;EV&Lh9 z1Aj96X~;y?-gedDr$u&o(D$ZWp?~Tfm6rC6)~>%~iWAyV+ud(m=52z#)9{f0=E^x~ zbEr}>@HWZY%IBunI`haYx#ke#3DMwJ@`d+)+3EA&J89(68pxB;)t=hFTLHxNM#&3XH|Igi$lRbA9Aj3g%&@iXUq{|;QV11x2tBPRMI7;$q z?>|~{Vp!cK1jHC*zH~6=OW@Ho3xikO&Ii$bNxlys zoBB`B@vnP%8pH=lDtU2ZaUguOT)s=~8cF*cIT`w&22ED(yi_;tor_IX5TcG`%=i%Q zL)n>1&jtDIGkj(@(7(9igvUeZrb@wSC>(jPY8b3qq* z&inA`C};K(B3IJC45|10WDE;s6M*6)bBiyawsD%@@BHd<>t|4Jp~fO4_|kB?@IWjd z^SAo)uLC-UT|^4VsO7-;aNoJSA7sj#>bQ%H{j;0Szs@+j>dm7h&RiY$IR-qxwY?EU zKxzBe-tw2(Tt8L-`G1sOaw(irEpFR~)CU5+L!Wm)>#801Fi9~#_5JQIe_3}P{ve{F z5DVz`|EpC@;#1{)G+B8EEoNZjs1H3~qG>Vu*& zUR53YE!uEH!)uusm2NXL8}mM4ud-cGIc~m`BBb(>=uk`yLXFj|hYpD@-K_Q{hPG$1 z&3g|q3swOwugHIeeg_kg&)e}xF8$?V3ipvIll?`_f6bX1p8}LyHm$wY?DR_LKE>O- zD(jBQy7DA_K+d^WgH8O#hn11?!Gs0id>fTv(+DY1EQxCPobBNrqi>pFpVo;=MM}J;wasc`#-(5q2HucH%o)9 zSoQs>nyZ1K+@CW~^7GO zrL0GFK-xJcj{X*`eQdDe!rEWl-iE~ryBD0!3ci@BizoA#o_$UAWS?iKHjG*@)XKT^ zlN6tUFKG(DJDe=?>%`DTW!)42=+v~Gm=*CQr_MUdu!x@PuaBVq*&8Zwvq^$cx06Ni zc2`R~i{Spxu2sFIkMb*e* zcWDIQh+KRC7F)AW0G%iS7JKar3qAr`F=(`znXC~xzH7+%h=ngvJ}!@x(+cO?;vSlj zs|1v6Io9|EJb!ha&Rym@a>=Zd7jMUhH6Zbd^!@3FiV@YtI-sRsh_^Ps{doRB<5e|| z`iaO%yC>Mb*bU2I26|bW9MGk{KZ`Qy1B86spwB&wdx$wcz2hkO(Gn@foz6m;B!zK7 zQLE316%VS6U%V|W*nx*km9+oWW_lRj^NrwX$W=ToOo3_JN8>9wb+a4i(_3cpd3P4$ z-u>{akaqOpcha<%{FvE31NkN`qh z0HsRd`Ebkf#*E;waM=yGo4)kX#h%9Tl*M^M9U7Gjp`_E6m}7ufCD5F(;GXsT^_cs$ zNMvGCd~@kg0fq-+3&DfGnxIV)O*8D6m`V(YjVkI??gHNSO#%T}*+|-;+WNJEz%p6PIBO8KCsh?MBBY-sHRWu@H zf}V7s=)K>C9n_moTmlHwD*1g;;aVrwdlBFCUiY21!l}tDXKUoFBkhk{DuzbjU?kS> z`ZB1hqO!8~O$r?bK9Xk5*N;|jE~VeRzOVi@S5ZUf*_wV$?5-u!hmGTjLc`4`tXG>% z6C(1%Knt^Oy8zbU?>3uHDlA*6^h!O7WML8XxqjG@O0P)WG% z)(6!vG$6J~zMlfwX9~R!r7dG&i^jH{l&D(pRRpTj_@^6>7fqtormRahVbazUVaIZf z#GD~q^G%tQFvhod=y|mv|LFMEBhbp;@1gDGEQrhC-1y0j{Sg43l`9q)KX|MFRGh|m z3s?$xZ>a=Vk{j{CnPr@SDFNS&QM@&(T6pw|nEB@VLJu{DVa_=ky-x3} z1x@4TaLCx?+5D{Q_i>I?%NJ4-uXx{>@5Ri$11H&PK;mJQC`}QC=fxu%web zsd_a2ss3eHdQB8~s+2G7o24q=ZCs;3zpbuqwQk3h2#+(J6sD25%!IX!|y^9iO5*lBO z!XUiywb04l1mra_J+g1&7QN&7J&6RPxQ&Y~o)a7UFnI z8?n&nAV{JRM96AZTqPRr7m?ZTIl|2FCkVhs0s6C4I7D7(jBV!6zVEJVIE)l~*vp&c zKj%NyzA#D`KzWoY3~`_LQAekkkEmi}V%i;kH?p3U-y1EKeQw}840)}R{+34BN10c^ z#Kn@5n^#Z<3mHq}i<*SR_Rz@uL1U$upJ;LIrvNeVAtHz9rv%FBi^sPc^;VvFc~cZ_ zFsw;@G@R$I(r7Z4I4!|7fW#iPf&!dZ4%D~*L&1v;GUpQ2rDcr;aVsEyRqn%K7$Dxj^;mSfuocpCPjyRz=K&(o;hLnT+*6fi$PmaYigNPmrR;s5zE@woS;82!cT+R*)4Xv5w9LA zXtVAdjeZmP74C4bh&b5w&pCB*0t$@2vcA9G;bDdD%hoqKypzSLW5BNe{H|`h=_d2N z*fsc|5%+xq&_frQDp5ZoJ`5PU#%rm>l}i7QXQ$+`Lzk5^1$&m$R3%=BzMD+r}@@x+>i_`59XkQjqE%w^L$Xe zw{zB3Dt)yh#%S?i#1a>pIumaKdW>z`9%{y?cmlqrx zjSk+hA+qe@%xCix?GV2kgIwcMagA~IC5t5FC4BM|@}K*yN4&BZi9aAnJ7@5(LnqNA zwC2vN)11|xUrvP2n3#Wm#;@k#SNi#RZb%xIYP!fWk8Vc5Y!7}MWpd8QmI32GRqUTd z)VhBcaP#djUM&A;o$1_paDvr%M1{g@Yk-Nc1uM^3+-ZOpvYyWw6sb5$oPX~M6`_}|xd6}TC7#2onGj|?? zLH8hb{mZ6W@|SJ)@S#D;R;w~S8BkNoy)J8#KemBn-FPW6{P6+5eGo>eA-w#=6$=)A zzv_q3$NHm$q@+8ouZ}ww>w>;}$Ok>RIO|IbTpoH^HjsrD$}CQ=TW3Auv1h-DFy0+r zZ?GsYefxGyiC@Pp+UoOEPSOR*D(O}><3_W6rwE6XZ;9CWd21xT{3SCx-E_NY zipRFOe=pyEI;u}o1~%VI-9^%UqR$2Ih8bOf_iAcbS-wzNMoe@8MUup-qcH%SeUyYvvkOX>$?Zce#W^~ms2vAg2EZW^yf(>%C`r& zS&&PU)q&w8sPL@L`;I@TmaFxcl{?0p(Tws|a8z#0ew`@)NWRP*(-w}36=CmuXJx(Q z`zcxsnb=QTZ1zhf)$N=-6jQSd@3;)TSfo9$DBXTU_kw!#>|+8-uSyy&LEP+*dnAs3 zGvIdwjRn2_25e;AKHNdo#feYftnOi}zBSXNq{#DQZ(+gR%u$VfZz1Fc?cT36%#OO8 zh#t?Jz+To9J<0I;0e30%*S`v!e-H9bs|R&|C|kz0x*3pVWhX`QUv@K0oxBxCcQC1&g-_Q#2*;DY zz^x=aH9hyr;ZhiX``?>W&-Z+v50fY4ElB zP*_=gvUFGX{>O>agWzkonC-h}Jd-h44fCRj4ef&-@Sz$Nag946tU<%2gx>d^g$7I< z_8W*K%VnkN=Naso6+-w!mo3h9%FCI`kh4|7WYWpL55D=(>drYxk(qeYbe7bWhd9P= z;=A2N?$rc%W00gNxcj{vHlk=hMAbyhOA=xB&kC`7E3)wWSUXxn<7Cg^?hBZxW?q`@b~dEX%v@c z3G%Nhrfy10n>1v4V^~dQY-Y<8yxx6Nc!`uLbiVXpayihcb2ywTr4Axcj(W-TfrX%x zIfM5VPiP?smP;~NUw%2$e)5>~N7WSacOjPt7j*@hjO%>g2$|sQr`_~mMYZ`<>Q3`O zUXvTv-%h3$w@f@PT*nQY(tCK);L)Xbpg&e?dZ=EX9#Oa)X>Syq+zVOf2$cr4Xa$W+{-}EZnj752-=Cm2hHnxEY79FTEE5~8D zVa0SW=lg*p2`gjf^ryE49wV$~+Ewy1zRPi1?HauG5Zk99M$gForsVuU5Bj=vwtA*@ z>Br-cbl0wY!uF?%`r$}Z*X<_p)a+D^VSQ<0QZ_XuDO^kDQP~;8}L*@@#yU*X zYdiQJGBFucNXFWL&YzSDCO#$EEtu5p!0O-qsV3L4K|wjlIVyRR{HA@~Mdj-dGV9p7 zqc6r`0*gihMt`Bk8dvvZCNkzoHoo8h&3>j$z^!b((+=Yg<(V&`5kT-Ppwwm#3mN?@bm~sj=b%@() zR(23M6D>*RQ6Gt-F+NF!)CvinGw8J3XP}AsWn$w;*A+8W5pkgB$#*NG;u>PI2slISFuWr9n1f?>*IGge8 z@{l1`BHdacHurEGZ8hnXZ^$z_jTTewS+SM@=BSq%4PN5KVrFbOeJV04#fnK0e`y=2 zM`F&*5_ILIQyH+;@J+CQHecVNBP>WG$lR?K>RDTYfSlx{!o^9|$6~rC|cF)_^y0agKVSeo(p&(+4Pi8~RuD0zdBSolh zb4RQvtoc2g(Phc27E0XBk+}9f82+u?*%;bk93mCRCj}d!LDuEicZet%t7{T5I1qbi zBnimzTu{KVo3{sRpyP*3GBDD2VGx^bv;Y||XEv&1GMU7AbKccS(JGS2wG{z z?+O#Cqd`?o27Y6|q2v@}v(Pl>8;l8zpg&OXYDMkpDH3dLr&07hJ~m!pL>7-k!8I*~|1jEA*q5LN+AK{RYINr;2os*^1UY!2SLmC=|hP2>Ambvp# zQH%e0W@R1fNz+p9B)3dk9hxh}=%Lrk)lIGg=`$n9h`9BdntU5o$%Ny2Myt~dVfGW` z7jsw12mW)$<->s^XWQeH251^5JKf)Js*QdWD;Arpk|i@EjavMTLG+MCf`z(5&p@4m zyjYOIhpf1vDY~g<`(~rfS$G1C=s|l{@FCWm5jL3^OlqjKLf^jPAZPT8gW<7oCszXn zV_d(K$~Sy5Pp0pQ`2BPC&;_A8cOTh!Ho?l?a&#=()IJv;Y1_AH{2ioX9%ZP|8Pm-m zu+aG8F^aKtg{p(lW%y`(s=o+kEL_10VQ{d5uER*6>%bwA23;35yR&ngFHXQ*5maE% zl!5}BRNhYTzRPWOH!Gk6CzOn@SIT)T7Rh^d@)+j!p3ES)3XHXWo>PATA~NuG0BiQ zC>@2VErD2YD`~~_I6vzF-yDvv*j2&|2w|xAE@NQxsJHrl3+#}O|M|vUS!y5sWW)6s zKZxUs;#T4l@kxz_Ly9V^DfwIuZbKY;n#dcUHkW3^g8TC4&vLlC`Lp+Q&H@ zTE;aX6-n{X8$SE`WIV$j6*fn0v8Y>CTUq}Oy*};QBG)lPbr%ziHqHHb_{yDVyV$^Sic-wL+lKW9u}R03-X3P;)YLB|+I9?-ZljUH zMN%2`dY3k5314mVcIe`dsI26#mwbeRCniTvwuI2L+SGe_`@ESocwLjf|60fx9SQU$ zPAL(2G`#WgBdXJ4EHS7BYr2E$h;Kc&Y)i%hE4!TS!mFh6@@0tcUUIDwF69|woD`x$ z$F_FtQlCEB&2z(KrZ(Anbdr=IDTQCQ^#a;XEl(_#jV}h2_CGX7ptsz7V(R3IUXM@c zh~6$6l`Pt2ey|2zmzG$-YYk3N-rlSEyfM*X62%z6OK3KiKRaS5g#l;l<^;4rd z-inw;QA@8>O-lKu1I#h6gF+mkea?a&@&XDXhf{_+zg2yhKl!W3?wI zkMt;K$LpNmKf5&CVziy6HWBxm3xM1R2EMYt7@1&yB2?(y4Dg78IMT1B%q^=7#{Hzs4uRre8wk&}(>mRHZJ&W2KeL1*2VQt*@Odu0!Qz&R7F4^3z;q0mlQqe&~ zool|%3js3kcwceLxXqi|Fk7`6zm31Rtg0RkS#xO{+~#9p;r|$9Aaqcut+$65N;grT zr7MqGJr3B_7*&-G_YDV$;fJM)ubt-yqDt`{G%*Th3)8zIltSL43Fg8MkNCE~;#(rn zdr#``=n=H93{bwbcbh;jIj1}yFY;QY4uVfr#h+05d?^O$F49w_JP1()6EX7!EgHj5ocC+n&u1=76xZx-q9YVy(iO3Zm_Oze zrx4A*y5g_mD;Sl|Ub`J{?pk2VF00P%(bm~GuWsx`QwDBm4#ZI+$cU%x{?POqS!(HC zZagz}feF&s?0I8Sz|L-X3Ni||RnKvGj|-1GG~cEH7MNX+OcR+|@Sw@7vH9ie<}^)8 z^rZhz;OOs0UtV3ZGQ2rxW=amcD_2G`yKSQ(_AT(eoB_rR)@wS)BIS+=?CQ$O(L!1G9KC%j<{Ppu@mRxRIQ( z`YC2m_g==u*9VHY^ao9*O^d{@H(aYicdB1fQn6Qqb}swz7&e1Ic{3j)Cg_F(n_=yO zlG7JzPZTM^@}Y_3$6dXiM7ao~CE^?AgLvMYGxtmKGd?t5<_!=Vd<@BpD(SAF*hG2cPPse%qQOy6rli0sl$#D48bFqy9Tvlvh0~lFT6< ztx!b`<)PH9vjcZ1!;4WKi@_9L%|^{EW`nn!q#yBLh_$xnoD?fV;)>Qf{YH(hw}}Vb z46kKnm~L+kzB$<96zHE(S7G%WQe21-HXBCQvep-cd>q5@dg~3yw{K#B6Rj(&1~V2# zvCHguEk~oN!#BDLWX zM~B2SL#dhrLqft*5HCxoof_it%Vu7bw~G6AV29oVtAklumq(Z8UQMpdfe`fp{-4^A zF!T{_>QCd)HU~`j8*AU^1_(P4P#Hh_b`3Oum|6`xeQ6aAZl)(xIC>yz5?UP>=3j8+ zT6Smq8iyn&k0Rv-{jE9?zQ*$8$|d!cu3X(i2=q)CumYBlFZukkl7FH-O*g zC&O`U$*sKNkCDNT$;)VEy6JC`cbBt%STJ{4F!L_R_v`B#Hhc1sQkme7a}aD~Ro&CMeOa`_@1D*buykVQi$+)z>120X z2zgN(CGWX2cLS^)qzcOxLHM? znYpY-MFAsEx+l?}hcm9DtEJ2LYa-+g!$z_t&x+*6t#N}2eT#s)vsPQz^P@Fte4ppi zmF;e~Y%*@*)_n0M=Dmi7Y5eQ`ZL$P!G|rqm1mo79%cw+Ij<1)q<{K@c>z}-g{VYAd+#h2P=$?> zECIwiNR=i&ieb%OfUn$WKx)*YH0F=Zv&9}M;>)?=Ou znlC2o^|T)Mhjl<`Xh#qh1N~q>vYo*R65O z{MI{L)m(o@Uear*eER5sAaMD&$`n~evV;?8wR=d%)~tCliR1-#H65+xrLt-8_zqTJ?Fjr%(((@r3M%1usd^&V$B4F_h=3c*^fQ!3?k{OL zlGibeJyAiKy&OLzZj|B3-|s3sS^PQCYS6y_c=bBJ8kZGRBn4LshNvyP9tAJ{+))*_ zGfr2hIUz+$u@p5RqDdj5lsn_4e5@?^0xEZ$WWscQ{I-za?(2SCq7pn*89^#Djc@!` zW(60?vsk^>T#J>{c_1v!M(!RwEpj02;dY>RGCxOIxpBhHZ-0uv824M%Mzh?U+%DCu zXsuxV4ccP5RgU?o-tt-AGk&BUDfwBJ??Ka$lwSH~9uBq}8C6MkuEXWT(`3r6%v3gq z{tG<*<>l%Pdsl}?utCJ0}l(hVSd%a@?HaQglo&KzNs(vR_DbSb!fnSX8D zN5;SK^oD)O5Zi%31@=SeT)jRXQi%8dzh2{S4GR`i!bs)ra5ND1?W@ z@)0ho=|s4do&(m^l^;&o)Lj#$ODNf?z4f)V?x->uEv(HlU7v%Ax)=OZM$)cu355qj z!z-+@TOLoTZx=OP z4&6i~C8gZ9Jrrc&FnEWD?IuB}=-eLYG?+@XV7RU==zZAosyXqs*!o{pdPoi?;yO{1}`jPL$cP1M@@$mZ6h%u!qYj=%xClm0w_v1Z47cv9rJ3sfD zDi}Et>>FRMyn?2A9Fj479jxQBksc+{TxF~o)Jr8aN(bhOmal{J;vZhED=xq9+_qWx zh#z-|zM>WrVQV=TM6l)E4&4NhJ!XkUHeBh8WDbD-) zjZZ+q^b!UC8OF1zvDAUsp!z4V>QNGkzKUN7A4*5!YLRD_RhMSQRMCf9!!bdPfIl3L zcux5$&*5?t?nQ(%a4`}fb)VRaG<#x#PBMqL)OU?!s}me+(Enwn>?H$j&cMyz_|pw9 ziuK8C_(w9fya&0-vVy;?^S%98zh?{%V`fJiJWTHE97VFO`Z4;%T`$CnC

qU{fVU^ z4?x6CeWgVXIw-nMzSqyb;bOxui=!d+Z5@um*g47*BomM2xRMOt$tO^@nO9|C_xiU* zbBT@FSvSxZMT6R>=rAh##CpEoPv0KS@&S~1&cN~~R zfB;3aw5~F(u7nc$1n6hVArF<#$kXS;^YzHMvg5S*vcDy z7xu*(GK;`}WOsCY5{A6O_?Ij4`0RzC=%C;qN~z_$8MGC(Plk4qXaF%u9p~2Gx~Uck z0u#-yajRYziC7e#qtww|D5|8>W)AYJG|ldhw&Y3!vpUJR-A{>2K+0RVdkJ(O%y7PUj^{Dmoc+9h!Oa2=RC#wIYR zn}yQ37p(E2>+96PG*dS10x)-Cylf$=LjJ{Ul#U=0!6#v5ZTMIg)nR9e(CuAp5_%PT z?aR9FAIt7v23$BR@KP0S;9YS8${b)1$PrB-=Joc$&tZA6+I)!YmGK~EQ`-n3w0D~o z50x!+KGa{Gtd5}5)eE0|@^s6YEwxZU>Je*MM&O~a1ynZFjtJ3Z{-|SGt|3s388k5m z-*qU*U);;p?_HshvOQUA@#Np%XPDySm-DrZl5(M$q5{Qf*k`tlJ7Q!$n;ewa5(xk0 z#(_6uEk0c73k;FhA2+Sc)|#_5!k{pqw@3H1sf z_X4j(l7%&oZ7(yZlZ{`FIw?1rLuFH@`H@8D>SuUy4)OPEv)f2(rzZuKFsJX?LUO>8 zkxfGxt}>f;Yqb>Ch{XS6O+`(0_{$}1_6zx-o332*nN54}2rc1{zdnE?L{ULytJdzy z3wxU81`awqzIgdh=mY5l1i%&Qlr-{TE)V)acq87pY?O7$d$qHKgT)r;_T3syBH+5b06h{E+@;A z2006+8}(0Nr8F|_K~QW;A{m8u7GbWvX}RNN07mUJV@W7S=_#9vi8t^{%OW$H0W|;Y zKkfiD)CD%!CMx%~)pVWQ$N>OJt_46}R|ENdjORzxXTpMnV1O?q;bf@GMRy>z6Rx!7 zbd@^+?xz1S&b!zuV7NqNMo?@Y^f=wCNO?EL(Fb$37@XHb?%BA2Z}6!@K9 zAH&p#Iw{phX4@9lU?OLZ4R;-g`tw>9_g2fb$f;MdKT-Bi3%G&V4JV)W!xdEZ(;{{} zNwdkHrmRJBi#!&6+eDh#6(bC7b)!R#LipF<4nZ5|7%iQn?IaqHn9)dl(SGjW*Vl&t z{LIUhO)QBU=qLRo#j&cWF5VW{lMo;vVM_?>xpe2se|G2=`L znt+Nkh8JlRQ-}Fd4}@%f-8wlQ??pk*a1^D3`FW%UqqpE|q@AWSMLiC(Yg?=E>~vi9 z?-z2bMd?#hF@LPbD3bi&t!cKM_Sx+z=J%!M;=l@jicv1O2!l@}?+T`cz-?njHP9Vp zP>5NM5=tP<;j3=|=Vo3VrB_Y}o^Zz8+jIwXvtNVTYh`oe!zFqv!(JB>L+M^h7Z;w$ zDFxLjEyemS$9^~X%qOsx1D{6B=eyRK1Ep#-)dKo)*O^B7!~&-&^Qta4o(IP69^g++ zK33YuEl1<}@$mn8wmR{w?{?EjG6(f=E!zaURIV7sHPNlee-feP309Q)T|Z^x$B&B; z($JP|ge$9`!pH}YfXDKD?5)PZPB%=^xKXWtuEQ8jhe8a?>u@1vw)gpkJ@Tf9ebpk< zxy>$~>%p&2MBbI}M#N86=d|o|P9P3p_V=`S?_1!gj?K#P9}h8D>MlPItaMw!M@c>0 z!8i(qrIqvU(;~~@R z&!025dIX*gGrVhHm1|0auh2f7kR>HA6;?1bIE&t#a7dSWDCD%vr|V(PrEguUbdllq zuny_XSSY-dy(<#YKir|@UFi_j^ul6jMUG_cY3eZ?wOjt{S)G(S)DDv8yl&scJq45S znz6v^`^mjVl8ex7Iwm@B`z-gmei!d{pNRw4zF{L1^$#xVaY=_q&w_9aLgSgsgNVx7 zliNNBUJ7b@?m>qx&z@9SPRUPoMj(D|OmI$*%C_{YW`Ayn;Nl^Eg_FvaXfkaw=QU$r z=8eD_-+(m`DEd?zwVwjbPR+w7pNS_~nbpl^7p;9Y=?(1>#Yx^I%>YV$KBtopM$%g+ z@&Ws+a__GM zLIAf#2sDNzlHRTG*@#0Dfz=M$ng^7YuI5Wmt?RFpnNxtyoUN-U%Vj+&n_64TnQ^O_ zL($|WXG`5g3I+Z1(9o+JE)XO|Xo+Mf7j#!ylDL(&0HW%R!8Pb( zGKKW9kM#wjq6)7`Ss9_~6XQ->d0&T$3a*^*`mMP)nm26o%__}o1USg{ zQ$>_PWWg9QQV^+Ie{bI{LJ0|hM%+aPMS-v^^4pyVcBE_!>3O&qkV!J;(*LrXl8BieJW!(u+FPo%uxs2P{WhDG{Y3%Z>&t~ zABJ$rH#{qFs2O6QZ8fnDVp3Kd7~7ID?S2W?yNNWi4D3gp`M(7MZl6}^QvQiiB`#yt{Mx~IH{@O7iOD;{(x@>nWL z$OuiQcHYE^Y#)|A^ZkfEFCR2)SxYFgQ+xSFsLuCAMN+#cas_1QO!GZFVWqAAF>?r< zOPv>hxBLh4xGkriASWfiq?Q5M%I*O7GX&6wiPtSe&ws0la!#e1tLD@ru z9&t`qBZ2+iJoIGOD|Iari1(5_7bNgupS_LVl|RFiu9A7E`boL2>6;6|SkeWGNn0vY zH`-464TDpLuuT?0wXl1u{Bce&(UTx?#Ojd7`C`01aAct}MP$a`Sdev@1H^`?y%6Hn znJt-@u*z%yEuW+D-T{vKyeJK!{oTIAlPBU0#r2J_rpCRJd;7CoQioejGgPh(ca*E< zxlmp}AJ6&=QcIRGChoAs6^F&C#XV%Tmri>xCZ%;|K!8;t?`Ae)1~xE*2l#UU&A_;I z$N{XYG&A=+@&4-t%|V1VSPM!n3S9>fpu2JaE=1ja$a@eGjtyg2T3gSV9Z*{p@-mS0 zdHQBo6SwW>DURm-Qyk=YHt^eY=sq-vXo68Z1GU>W%*sg!sUVt zP(ot)WszDb>$5ZaaI?hv^`_84tQqwOAW`<}tYA(pJ;3RgziyzZorY$F!c2&i+*!(Yi+ zC|O1evtT74ok>|q4bCN@C#%#vIRsLILT98Ia2VrH*lzHzq1b^j8z zB4y4>Wq!{yr3vXQOmU6y`g;b?(0Lw%n;7 z^ik~#j_-iOiE5`TrXmJi__%|(HzUc$EmdX9$Rz9Ig4M8ykVD0RDx8=0n{ZYBgB8ez zn?ulNmSOZh{hk85@_{6;mCKov_G|k3JkoB4_wRHHMx)PR{&U~|TA}6qiH{S!m>PnM z!CsYWz87rjd`^&9e&OSfg-t}n%50?lZP3&wz+VxGP)1lPF4tf^&;ynqzpfdVE0Yj} ziRc;gM-7?SqV0U(jGBsxczyJoHGH)wg@@ajgYt%_KpVPuUgy^Gu3GBt7=U%)TG0@AFZ{ zktO7;d4(hV4LPsxjj^N1W}ij4mg~No7q{P7-U`Gp8QJx4wV4D^)k%LIvM{( z<67BKK{m6fgeBEZ`08h`y_e=W0z^g}Z+5Pe{?}uW?;3@t|16Y znwz5M;BRvG zvz;c%BUi@L9f5Qt0)2J5)8sv{56XgC2=Tp4TJod6-es8jSSDa??W(%Ty~ofM=j2N{ z+hrVoA|}H*3zSPyTgA&+#x($^>&4uPhp-xhyC|GJ#i?gJsirUW#AaA@YLpBG9uL zDe!vfHKHP>+r*~FnCjh!%eus#b9VW$nf1)hi(m-7+uXinLursde%X_`eQ81JjQvSc+t{8vktN5Ck^~ZhL3PYsK-n%C!oDVAdcf}n@1njuD;}q zHRa6{&>mZd-5eIo=qNY@gHIv{1=3KNzjoYjM$^~-XL{pZG$K)K3#_1KJ4!CI%oJv8 zj>QM7-w5Lg`Klj0k~yUlf2#Ds-tvaSS5#&CV=vU5nq4Z~R}9!HCnXPVTT@SvXG|ma zwxxBxgvG`M?~63R@)v5C+UbgzJ>@^J61q714keExH%>Q0EgI^~;*QcJOm+iGnJb)q zwT2Q}ps0?!+bLNQpR|CyQ;?!YbN!w^uc$aRV)jS78^ora*MPDeF?aFfQr)7@welBp zYV$K$t)AijjuL(Gn&M+Ek<{}Sv9{C0BMWCPT4g~8bfL-4cQBQIJ-}-C5a`SF?%hq? z3Y7+HpTZsmweF4gkk&6P_rxI&FggFZ*>|LgA8-0FrdM#a#1axcR?V9qlPP5G?BQo_ zRfLpSlyIW~AShr_q$N5>;VGl&C6TPPR@7_RW*d_#Bo@>H(|xGwT6KPP$e4}#6at0c z_&K~2zT(huzv#YS=F1u3?AY3u@~08-R<7HeL5yY%Z|JKZ$A7+S300qMYB-{8wA&#^ zor`bzcmGD6j9q{KdUB=vN*9uGCy(4G&;ORGF69#I)aYlGVxBh<=*H-^#QV3x?JJ12 zN2ct%1{b*~(_dId%;7eQFpu}r$QI@7dRy+#-MvuiYqF3l$jcQxLo%37$!|GICWQ0)B5hKZ;>J9`1&zF`PKrp9QSyC|wx%B= zu5Ajoi0t9!_2qtUj#O`ZU`?G|NBw;UA#tu%8WjGl2>2wgTD)OD+FJ{d5hAFlGgW>< z|Ed`XD~)E=FYkCyJzF;GT%%>*dYInV9 zrNf^~XBDHy-vT=5i(6GxVyG`2Ma*YStp|MO1fLaOMGPwpY&ePlB#oLa!5BzaYty%9 zi<0TU+U)9sX{K}H;L{EiybpYsofWg^q2##IMp)1^Poi+>Oi%)r*082DFie`CJH(=* zLtNFveIR9*d8pjXme-2B5UKbQOke)abz{_=x{gfQeX#Tp5!DdYF6#r z*8|vz=02jO>sFC$AzALzp+*gjiB;Eg*{7`kS_w}<8NZXHc^x{TN}H(ZN*AdS>*rOzK>zdFjo-`pI{dVvss`>eXZ z-+E%@OJHi{S{&j#_tk6R8c>8XKckc1s(#o9YZ|q6k=FerwR?}rk zh>p*?>vVHjdx$~bWG~b~`UglMn4S6pvPdljAYDWN;5=ufI0DW)$Zh>?yup+JfEKPV zifq3x5@qr_wS@MU9{#vbVr}-g6c(X+nLj$ZhcbU03{;DcHXsj|G%B?SqTHYb)H@I~ItMP(D?HOo?_9E(10{(V3KwU{;8SFM|Lj@x|W~OB$ZU~Ng1lwe{k}Qv2 z?sW;BkK>9MG)V`22*b3PZC|3Q?JA-Acc(4O_-}A7*^5q`{#U>M-@~Ad-?i8I20SpM zz)~~SdcYj%OfaUmaec#WTJl-Rh|n{AMSU5V%6V5Ds>fM6mv7Qf4ehE2`C^h9mzW#A zzF&v>Zw2@mM`Wryz)_KhljU`pG_dHlP67{OE{F&AN3GISt~T#)k1lyp5K@&K?|u!e zZ>NKkZYe6Dmt2$ypPY&*q6>-0qVgY4z#%lw;iX269)TF#ebBvIxzXY4wCFB`7yr+c z>%TZz*IMqTHUarX=@OHO#Hmc)M>XZ!W zXAWTp&7~3Ss7_JS$qyp2;A@yS{u%!bD4maW*a4vkHU4D{7nq)w(qvP5xQ`0*)zjb5 zmq`H}aE5Q(JPLT3JN1d?L!P*9Dk0&Z!*V3PNW)lBGB1;ELjnrPfl@kK&fKXAvp6;> z;X7%zP~O>yWUnAbJA>p;wtc`9^KtWlpWl7)#v-|C##*w>O8X9Uy!=%kb-+U9=FEBJ zVJ=m ze92e-Zq7l@vW+S!tlVW?q4^uVu5SL}x{jG{H4Jr{vjU-b39`i*FA>d#%cca+k@R?( z7;^HjL!0&y4v6q!(#j$W;*y#Q@pYXAx-ecLuOpWe{w?!yR~O1uUR_Dz%auDmf&O^d z4|JBl1^W|V=j-WxEnw|c_GRfGxebo&9sJozcEYr(zX!k(#8ze#{;VsZ~MumJ<( zEuFAXEE4+y9lXOSE}|?WpPo~DgM|z4>m&^}zT=-Q3A~B1D7_}BV7zgSCrn&tCyyK` z1*EgNxV{c4i6`ykQ|7}z?Y}ie9kx-y;dXEO=l;+wYg8?@Tpem|(%WxmZ>z_Ws>Any zx=#wb<)`|FbJx_6!%JfV&FNQMoIFpU-su0nn7j$fJ_VrH1E5jB0Rbw0U=7fnCs=Zx z$uVHrvJXC#qS!e#1=g?CAK#%=^-qEMOOp)D9|cemFGVoTv`+qDjWmqoa_Xb{@t}HuV(q^AaLh1ejtvX#gv98nMDXiD2{L ze6GzWpIaL<_>3&$`wNBSr-c9Y8j00m(ZR;2gov^L115X4ZKwzO(7@2X=qKBO_*{Qp zq92bfi}QgFheIi-M(6d&rzfip_e{-nKS^7htkS`-Dm}~2mTAm@gKxFjJ<)w?>g&)m z-9Wt++uhgWx)pI1yA><761}xpG)iog9*LfU6(2rt3d+Y?;hv5|`aH zMhZ36t{tFy&m?l}S15cQi$&qmZUa*vTPstaFVq~gn?nbX7E)TN?VBDNLwdQv4Fr`3 z!@5wDSL4T>k&DdQB(hZN-?}uj=8+%M9=C1P}jImE0nWvcb zoaQ&HuBk+R_!Ch6d(U|9t3jyn5)6;xxWSncdM6_@=aFbhn^;m1G~-3xk(*lwCm$~2 zryEH*WL3y6AZet`DJVPoZF=tbRuch@a$9ch-J}#w`TiC1wcLq~YSXRM#w+b2F1xG- zT)f^5uc)T6au|1sh;n^XNuH23n6?x$DG7!D9Iy4=!bL0&z0~luhz9xBy9u-XTp2|F z^}a+h`IwIKfb$ns*U~i{2PH5NBDedeC)3ugkjCg6pE&Kpc;*W{_p1yWwDPowC3qJImBLaaiQQVsNe&N4f49I;SL3#?Nijy z+C|X*2a-kq=iYd2zaCKBW`<69hbzY&(H zF}xGf_ipHkXxqc7g=;uyVk9YkzEQn5-&>AF# zFw-(b1$6zT^X_N70E`<9>*<^xG|YS?fL6^l9ts|E&Dozeowi>{I{@sIrCzn#j6XK# zGrgxR!M9c^w@{l7N4S;Km=%XEAb$XWg$CYK<2;4y{J8^e`R<8~`EqFbMV?4gjY6_P zQ7#UigTPJ#$~Mh_;%`@)g>8~0=D;eh9e>RT6LtUZw+74>nJ3G5!7B-*f6xu@TLYqvx%nERe*-rY`EYD8%}c(!<@$Ua<)dfMdIjD92z^y zveX!^sWD;rR4b{>WTPt+g|vt#ZocbHjIvg($wJ|nm%5UnT8&~`XJ^>w2O=BdM=8fv zex)UpP7vxR=ZQngZiW-j0)SoeBA=PdRg0Q*>``Z#w|KQegO@e`4lbdaU~pQMP56S;eN= zchae6y3y=LOO@kwvFmjz>9)ptY!Y`cHSV?F|D8={d01o~m3{Xu!Iw62e2;ehkAnG^ z-y%Zj0$o$%8L7k&rOrvHIug-a$jo*He^5k?_8k%%ErM~4Vol$SShPF34p1Ze) zN7Hhv%lS}wB5k0vz3NQsu2fr2`eu&#wy0p(F?In~ExIc{w6^ImjH62YP z*u1^=gZ1B!veB%o`KuE$uQ$BkM0<~7gOoZ|S8>gaWr<``lm?GEk#{rr6(w;LoV&9N zj3U31k``Qc5t_|Zg|2#?1b5B2hGKC_*b_?Wc8&DtyIpeepLZ@*dW|^k9U>{0;h$#u zI^POsE#V3+F$GZ^opQL&MT{)hZVAIjM0BLOX9Q$Lnq5<$iReg}Z*F{qP?oj3k)pcg zN}gQeTEpR>r}~iV`F`*h?3nCfH!>64fnj=#wh@YpccZ*$U)KL?=XzmPB`^XA8HqWQ zOlwXlN+aN11Hlooa>MA=Uj13M_EM==Im!%>Q}y*M<7bU z{l`iE(68(^16RgtmSCfBPXjTVW(ja7!-DYX;l2{l)jUc}zEmAR3>TQysb%!rW8Q_qX{1}AfvuBTIq zdy7T%%Wych$&fEj<%wZAyu0KQ(COSARl@e3Kj)J^7Wi6lQXDb<^qh12iJK1wn=L>m z{Z+UAS&>^)>NTX|)7+y!H#=9$o*mm8&>s7$qlkm&T4iddX+nb^ZF7S-wT`~gyb!bB zn66(wns@3i&2>VIS8wGJc5h`zTsnstOjQI?%$O2N9$xN3gqluXyB?e>uf+8}8>p!M zJg`Ho)UO;GVlvdoQhMWC8u!!;STRVl<_*L87P&x(*4k&4?KB@wCrcnPt$yPzX~Q>f zfbaRwqZh=^_Smh`xov81`ztLqa%o;!M6`OVQ7b455iu#{mR_Gc+-yU9euSLbL-IHw z%!`Nz=Da9ALb3ucbQMCq++sd>?h-a$^qaYERq0orC4SmNWWTOtD*CB;?YrqdSO`9J zVl012maspWfYvpcK3`WvodVH*Qepk*qx<~ny1~LfpXN`^@8@y~$6W5FXZ82T_VTnx zpzRHw)YJHwbT;mNJ9)cYU^tzN+N7)-tG$KYT|_);c_TbsGkl(%KRP&}0?f42=$2)9pG%eCyXC6w%+6VcT=P#5 z!n1o|1h4$#N-0**ms^e;?ckV(R(ZMqf%AWx%t0b zh2u5G>5o+rN?trncfj)61Vmj9xf1{DBxcwHcd6gV-27jjxECGDC8iZ&4{eynd0s5s zK>jbEjfe@RJg{_W7k$8LPMj`g{0F7-r*ZPf)$=CBz02U^d!l=ckKi3g!X8H1zo7Jy zyv|W*SGJ$OA0J5G{ai4i^Ho$%C`x5vSNw-s^3NRwNvmj-67$OZF(c@{6R~WoDNuX8 z;_OONBsQGgy~^cC*3O!LGNHq)_y>Ub&jkgxc?k3+0k;*kd|$MLPDNii*@R4C%{|3? ziGUHAr$sLJ4@K|KNgU+@1D~zk;s48F{vW>>;&ybT(Ext7^w&y$oohPqiSNNT5$k24 z|Ke$bKYP`z06*LpD|k!iK{>uH3;mBhbOP*5?cnemxBlm!obpQh8PMpboMeZ(fk1Gq#LugjRO~;$cb#wV( z;3oap>0FHU0b}d0e2M?Q5COf_{tIUIg95kzbqxX?+49!{xT0pe{J{2R(FQ2LVeVQ~ zEqu#<&{takt_QviEdb%3Z^Lcch2KVPvVZ3qc{b6^(7jr_bWNISP?G;hsn$L^#ApNvv7xqtZ+QTZjl%meo);^{^HHnnN+v2or2$_1 zXUI>wFA1;^SK=`#9Q|tU2F?%fmc|%tya>o2$aM7}QhT#cS!rR>_3>cqPF{S&oQyG5 zv#&&z&OLfjVd5qQU~aELM3z0P1_5D~ zfC`RF4YYVIIoy2r8fk~M?W}!ZLXkSYd@WhF4KI{q2_vQjns`*FsxBo(OelG0@D51= z?N0xcC$#s8Gn#dQ=n4#d&x;H4VzR`Cd117Y%`Ny42(kbKRTlfv2OVfF5;Z6S`b9<4T@S#j{Ogb9;lI(cjgLZt)g=)j4epX~@lMrQJ6uAZH z0#05zPV370{GcIGe|>vEyMNr(xsgV%ZCB8`H~3TF<`UAW7lGl9u&M(GkV%yzvfH~3 zUK)$sX>rStj1Qun8#l=czE~xv{aX6gX~7mvmS7ZGAtZt2r@7;$e$wKn%0m=`mXeIg zRBG>5 zqlnMffue|%ur7!Wk-;+Cp$QE$ST3i0tGX#~7)eU|@y!V9YD7#Vp!`+n*}1KK-u4vD zTb!>&VASl4nJR#6+L10d-OKp6$^G4!&zepZ8f8}ekG4+sQFmq%7GBrV`T^Zvv-xz5 ztca|tHh6(+(oG(z(`OtrnsYzvhsdP@U)2Dx2=v=w+9g1(m;Nu z;vD+rX=~v#4iUEeYvj+#Q5v%2X=A;Nz+!A%6twx4myPk&1MV!;qBQfcrn4p4EC}bw z$IfgZ$esVjSDzCaG#TIdwnSh@1Z2X2Voh=}wctD2({9CBjk4UY3(7YC-75JXEU5qc zt~bAPb;gM0ZxcL&G!E?#)ootrkk{!8EM7Fj{BAK(3$JAu#nhc6c4tx5l~ZBSn7pRj z>o#V37f$k8>Ya;=Q~>KB|7m?R@R%qfRq2m z=9RJ)p33nVSUwo!rcVu7N+meW`Y@gJFMv-(KakXOT_JRs>0Doc{m9fS{QTgZX-&$Z zkzQ7Y#EWuW2)3^~VLom7My1)%_c{YDU$&s=A#Hz^U&WSm){lN#qqBeSO?X3QI*3Jw zJ8u{mAg$x{*};?2I-ec2PA^*OZg6aQeY{-==DV)uP)$}ax&QKmtP9Oe7~nna{GsC& zC>apw5)_b?kFy9X+2|(um0GrNH|9-^w`9h7$?w*7;cL|`coFfvW9u}}0nIi9{ zHSTiFz2@8nQ}*c)v2SarVqEcxd*z8Fe5vK^l*jy7V5rh7$?)iwX9@?i53anm*!vC| zuOadP;8%DS*egH^3f=vjOGH?k8)^t_tbur0Fraa8aQ6M^$9gWL8b-m#YBml%WIH1q zbGFwjkjMoh8X?+~z%?C?4bW$BtJU45%tq@>_w=+6-LLDZcbjD!KG%7F(#MT+lqT3)SCswVun4@^RD$UB!bNukGcO6-J3V60bM~+0mIf<%30=NL>J_tmBY9y3H zksmsHL{74R;bX%HVFpZ8m3VR0DDDx*NXvL%M8mWSH7HyM5U|6XTgX zQLtD>&Wld4KGw4$;dSG3vBAqFk@%NqQ!_Tl=IyD&VK6)5WJ~r(n&+5`6o^7#H6`sA zv>(2Y;=Y!ETAdUBqXn8j~FTFC?R7XfOipet829q3!5%zxce z8X06>7;gX@MYOk59u4KxajA2(LKO6g9X(%XQ)z9;{_^c%FBtNkFx8y#h}KHm;LI57 z*S{Tmr?m=>sEYNXSVb6iZ`Ym=K~*H|W?XF;RvW2p7{@Xb?|txB{oXllvE*8}igDmS zeFXdQ4}LxNn%S_HSI>GLDKZFw@TvW7b577P

Let's see what result we get for the insertion of random values from multiple threads, however this time we create a parallel_hash_map with internal locking, and modify the code so that each thread can insert values in any submap (no pre-selection).

no_preselection

+

If we were to do a intensive insertion test into a hash map from multiple threads, where we lock the whole hash table during each insertion, we would be likely to get even worse results than for a single threaded insert, because of heavy lock contention.

+

In this case, our expectation is that the finer grained locking of the parallel_hash_map (on internal submaps) will provide a speed benefit when compared to the single threaded insertion, and this is indeed what the benchmark shows:

flat_par_mutex_4

In Conclusion

We have seen that the novel parallel hashmap approach, used within a single thread, provides significant space advantages, with a very minimal time penalty. When used in a multi-thread context, the parallel hashmap still provides a significant space benefit, in addition to a consequential time benefit by drastically reducing (or even eliminating) lock contention when accessing the parallel hashmap.