From 594a56260f0af9f1d32f0e7217d90c98b109911a Mon Sep 17 00:00:00 2001 From: Adam Sawicki Date: Tue, 21 Nov 2017 15:01:29 +0100 Subject: [PATCH] Added VmaAllocationCreateInfo::memoryTypeBits. Wrote more documentation, including new page "Choosing memory type". --- bin/VulkanSample_Release_2015.exe | Bin 101888 -> 102400 bytes docs/html/choosing_memory_type.html | 103 ++++++++ docs/html/functions.html | 3 + docs/html/functions_vars.html | 3 + docs/html/index.html | 1 + docs/html/search/all_2.js | 1 + docs/html/search/all_6.js | 1 + docs/html/search/pages_1.js | 1 + docs/html/search/variables_4.js | 1 + ...ct_vma_allocation_create_info-members.html | 11 +- .../struct_vma_allocation_create_info.html | 41 +++- docs/html/vk__mem__alloc_8h.html | 9 +- docs/html/vk__mem__alloc_8h_source.html | 219 +++++++++--------- docs/html/vk_khr_dedicated_allocation.html | 8 +- src/vk_mem_alloc.h | 203 ++++++++++++++-- 15 files changed, 445 insertions(+), 160 deletions(-) create mode 100644 docs/html/choosing_memory_type.html diff --git a/bin/VulkanSample_Release_2015.exe b/bin/VulkanSample_Release_2015.exe index 1fd8967cc646a37e2175cbcd9ec747b5b301bfd1..ae08c334fbb99cb942e4aee2ee79b52011da7a3f 100644 GIT binary patch delta 40539 zcmeFad0bT0|37~3g^^Ji2b2+T0}&L(g+&y_1(eZ?qLR5Ug*)bk3W{5U!Wh$S3_a#p zS53!>)d_ zIqc-O6mZ9;tNw)l*|RI>n>V|w|A!oJ?>86F)$^=B!9RO;;rPhe(=y4%`nExx(#?V} zeXWmRjR}|_XjiomCN!3s2ttr32rE_MsHQAToY^HMQxv>G6-q^+ieRxQjPljw2%Hs# z9RSgTMImc_+m>Q6`&?`$RW?}Pw#5cX-(y7}g#5rjs=Z0TbA<{*M%P8tr({nNgz7P% zM}ub^U@O*7>d-M8Sgk7wK6K4exw!}MM1ar2O14hwAU@4Lk~%P*S5x+j)LdN08hG8@ z;61ImEnXCik%kQf{B2OaFjAKjXa$Q4^>5}b+rY2GN-DuI>R763} zKN5LkwN4aH%K5h=o!9?x^SAZz$7=i%NSyzX8veIy_-pAOS)<>g3AC*dm{lXN9@aI7 z7HI;D+z$PU?ob`-P{Y5>!@oh}pWx;%=47Ks0qHy=Hm~Ge}L@`s&IHSB- zT!Sj(v2ao*5|`|Xh+|e90Ms{8J;04Y2ndw z+)3xG2P{yZAhu=c`u>r9|3&14Z!M7o5-IWzDKv219= zwo=9+Hoswesq#ymRRm!dJpt=_I>dvTj z1x(ji@$CadYsD$HrE!$6&Om?s@S$h@0`ODSjA1HW)Kyh$_PIR{C@K=g|#kIicmCiOkY);H50DHtQ&R6zSzR8OG!lV`-v)z6X((n7(C2(e)gu6YX9s6rkc(lSgn1d*2^E?PH(&_woI7UyZ`brS-t(!ty8yMUF_g6r&YQ>^sYF~&?&q{=<)uBn4G zqMU^^?`c@Fhr|a{mw5%UY0cY8@*ehU^JeV%=8?XQJ(c6wspetQmEEZ8TRM(un(jwXSHWnN$1UuPn)P1sp zN1$EHR+4=OOK#cAccjLWX4n1B*0xOPD1nfw4C6CUG@pTvI*aa;o9M-C$L!GS*La9e19_03D_* zf$}b|vv*q!mM(q3es9&;@csui@*|CGZmX8TR!(s%nCw>2q?Bc}j*^l;VD{Frz6Ouj zhwMb_R_%X$pY#k=&HB`sozKU4TDVLz;X>ywLz|;s(({*Czp$tcVSS|NOS}4oW%x)h zpJT-xGo(J}nAGVh$!iZ&Iz1=N-Oa+H!o;9m=BOrOlZKcMW#xiwADZdxvTN6r&L2x+ z`mWa9Zc1V()5p#i&$Feml~R}Yc0Cd|TNFcgecb&wRPFkpXNl;4_cjTCCEM2(3WDJD zd6O;eHC)>J1}pEiUJPL4d%rIZVm^uMr0cISN8dG8h8ZvHgA9N8Gl1G^~(qbHgK#ri4d}sY(vdgjD5}bM`CnFocEoYZLQy zEpE4$KmV+H+3&L#r;xDrhk!I%}(}fAzgchIs5fBz5R^G3Afr;SnvKVq}*rN znEsEN+WiYd2>S#KcNF$@|3uSXIWAiwGd>%~$f`Yf| zMfPiQl(fUf#a3}HE*8cn4rmoTniB@=)nQa4_00>riUth#l3p%lRpu|HD~H*hLCZTu zofL$)FOo|VykN_&M2bkWK+sUmY@k*N5@=p0ca0zXBcwlONkii;W3g=u&09u`#h3al<;feS-ZR(xb@X|@mOY6P%~Xt)}K!G3OBjl$p{H?GEE zaHtzsBQZGAjjOR39PP#_8fgFE1UG>K0zAo$s}UJ2yKyxpgVWu(8kNE0+_)N-!5MB` zjm+RoH%_sE{wz0+Fj$C?>n89>D3h?$%s&$P_+i=u+a`qvjTxIfR$hHOkB6rqyaI5T za$h+RewI^yL&6yS{EU9G-)~00q|lkk#wR~F`i}}v|IO&15}uV}OAAjo`lp3wB;PIb z&Te+M;Qn;n%$|>LDZ5e01DDHn z!-&L|m4fgp*iKfCrJz{J-DoP?QZUT}FC`pElB~dd*$rKC*$pZ0J2B0+GCUy}1*(iL zObd_1b|?ifG(|a+<~$Cu8(30>6eT~8vfs8XyclSjZFl%qKv~(wt;+V4@bnaAH@BOn zoTiB}I_`@UTQ(FzbEvbL{WZL0%TmZF$Ej9^e(sQ2Im&%vF(V=(!!@17fN4;t+Sn3~ zRalT$AISqqe*E#r4)4kv(Ne^Ye@lA4c!h;N(lMpwIYB^DHdHEHrk*Gl$;$a5$}@jt z`e1-xdz}c@OWDoQI+y9&FFm}Ci8rn^&Auf(k5yd=W$!<-Ph87pjqD(GZO@8EW{KOF z_-KfJO9aW}EMo7DY{gO@jcwE8PnQb@2N`$$f!h-0LYn;-EebMF{+!C{|UQ4Dpnft3F|VtpJe=m%^TfcOk(ekjxs!R#^c*o z_T%VIXrJF0lc8HJ`a~8xW{lzTY0jyq7T??T+?Y>9gLuj#++|nBxOh?Wd4HE@iQ(d+hs(sfLcVD5F`zq;IO)1QAnB!b#<$8*3)!vj|`!XzPF?>+x{Q zFS2d&A0FR#e&(@y3gx0yrQko58c_YMoS*9_7z?iH1i@Tz*J#B?TR3WVVBc`Ynf2a7 zOWjX%sBLw39#ha4#Qy3#D!ZlD*5i|U{G2iPVZEDb@vV@{&Df=pALZa*8brHH**-$= z^jif>cHbqKqp;x!D8%$){uGmnq?pTX1N7v?!FOGj`um^WwUoPe3`dbI#Q*bFmn&I0 zCo4whmv24PT%n#M7n%6(LwuD_c)~K#0{_pT58?lXy8e$``LO?IRR1r)|1{gG>$2kR zleMp5hqK>sc*Vm!<1EiaMW1Pj}O^1&J?P*)ZKlTW!7sNY_HO6=R9nCLp^r#G5M}JM9l8* zy!LPirqyE?lk+V7F`%)Phu1uC{xEXmN#M@6AC|}RS~u`%O1EKH{p`ArDg?30>*+GPw$5>j192jumM7IN+mv);{aN4Ee&>;X8xIpwr`K1;iA^g2BZ z3!Htlk28bcn%221c@JE*yBtS>mLMTJ3xgXY-ZzN`dEUyxzlEI1^?Gw*Ey+e{iY17W zB_TEc7co0_rZFU}fCd0`yUj^-6V#Ur@!jGiDoj9tNDg?vh>7}KZw&PqW%8b1!#*cCwo=M+^8PvTc*n@IWvg=5SZN^@6;+uTE zSW|T0<-&~5UZ2@bIzEG)n;B(zX$E=Bj{*lAWw)odvT(vce#1eLk(EdIz572)U^e!u zS@|P5{~)Qu`V)kk33J>}XjBSq#IUa!wmr?>rz}-@itabkdTrfId<7jaRLliejn?fz zlXoLjXPm3vp!kN!%3^*WiG7Om&*|{Rg8O&6#Z4)k6^T*vY&rRN#<`txu`c^bVz1#| zfQ{T4XXL|L-}JDpNVCm*C8coQ+p_XF^6gP|g$YN?kbD!>6~>}2G4-a&&tHGY{BVz6 z_bhqosOqd2oK05vl;M2kAwg3u!IVPs`$DSnECtN@=a*Q?l3fU-d3#FY4z<_b!X|;D zSMV01m)2`RQ1*h_nV6C=xA5s-5#~FqmBpiFRqbTBl1(%FyzRHV(iA6>G1YD@#BLcYS8^V$*G6+?-Tg?x0Z(cc zqg*shIp~B0pGlcElaA$m>>j~1pe)XFQ>h(wHy^p!&`@U=7E^3%^E=)PrA`9k}b5lGKsF_m| zQ(b4sZrscogC03P_r1i$_>3~(5VnZ}Y~C?65#?Zg={%$bbwu;q34@3!KwV zdM%6fnG@N#@H$xRxLb4fV`Xy#hVOgK?YvhQ&ZT5pqL>NOIO;u`f zrxa%6PSO6{*(Pr8d3Yo@Cqn#)`OgoS5mSrRt@y`MH|q~Cx?D8A&p3}gU)RL_$#osl zJQ?}YzclR>%bFW0rm)00?OKw%x4FA3sr??GXH90^7PJzBS@*f2EMQ)9%Nox0B|Qxa zbL^^+D_vVMKsTd@qb5CmIOz7EY;)cAf5#>*2$ssX zu%!#4MI9-yTwH6caM5!Pf!E!9uGi0Putf>4D?JIO=}EPK36wr_r}(37#YyJ4GeX#*B5s1 zJ%kBSwV3-HJHIfjv7eja)+Xk)sDtmK8it_fSdT>^%(S?@B#vc$7kBG>7LW9`T+hL+ z_`m1+dz`6$uE%GxcNVuYU*|5@YvxpsL`~}U*yr{$U}N1@b|1qgF9?!aY-ZuvUGM;J zWOk>%rK8C)zJ$#9h&$Fx&cDS6;bYpIdfNFSXBtF8qS?8R_(tmcdUOBNWYDov_ z_y=s%l7!&A4@kj1TC$8dp^}x+#j;Z2OniyGxukQ`p{T{0Wv7*gmaWF?*bht0mL$Ja z9Akcsl}&0_X;#wpW@VhOSxNFv&UcF0;q+}R_yXgR<9^Bc_r#p0$@#~nB;R5E9|wU zNx_My7wc_-)??X;%CD|Nsdz2G>{Xih)uN5;pQVxdfzTCKdh0Ta&K+#j;q?%g$%x}4 zSurF8v9-DV!b-5JV%CsFcu6OW=`NG{WX$jlYFws-?d)1^j5N6+3trZxZ>QaYaJO<; z%hKfhDhU&E3*flh%W@8zi2zaD?4z?}yPZy`oHv&9pq-`MPI4+ES^lz)Jxksb@M49M zd{0g+Tjb+1ZRL1Wg`9t1zZm#tj_2R+ump&rGFLdiXFn|KpD>`FY-~N*u-dXFx9r?? zY{K#$-L8J5p92op)16+pXVF|whwDY;`YX%Wr^}m59p(yh#AMaP8h@78TcILAHh2;jKZ4j+}d!ugn;Wv)n^Q1(t;;hh}!2*1j4O?acq zT;aS$kE!gV;UwbeGKIelR2iq2y@Ilr%L>+X98Iu)R&;HB1ABd*!+{!W6$n&JclMaW zO4XYyBT%ffQ4~i&vO~1WKj3?+Bz(l;G()cYla8;P)7bJyqxX}S5HFNnB(k!@x=7=_N! z_W(oF>^P3FnIq#aVde7^yy)ik4;1O$EIbNW89punhmqKxfT?(xF{Qj*r-R-$sGS1R zR(y8gQ-Tkyydi;JZZ5_Q04tN7q#(oPpeHxhQe)v;+|v@)ubnllEF&$(E?cb7EZq9rZsLO%JJVLKY z;VW@?nwxBVt`y~5oTS=G7m<*0KrDdeZC$A3w}-=Abp){^)yYw!Z(N3R~ZA z08$cGK<_~KJe2E0$f=`nR9@_2?l7<8VIKNl!#ogSwvl?98fID1Z7_# zn^Cw^dad-tn~?uyz+1?Q$)C5CcYur0gFka@sX6u-I#+kS3Gx`@5t*Nao0k95AG)(( ztJNd>PEn8KPEAvEZ#7|E*2Y^bT+dLwi$y}ewRBl;*qxv%cP>BUav`KcfqFt}O4CGm zPciB#@bu)w)+GjYG^NvMil)(o@FidvMFS>bIRro!p=)gBj8w%%$K-M!VkO@1U1lx(2)FA|DXD_c9;I|dXx{LbQ(fv zXTAQ!%Y5u|exv_BuF`Au;yZY6g2vTq=wZzY(qk%|6Td`HDLKwtL!24tV^444fh)r7 z9tbYpUAIEnvVus$&d#pr`5$!W`)R zzc>c(G^C9Fw?5M?f+zn6V{jK~@XU&HkJXI9m_`_bJuIlF4yrQ*f8@jy&MxR{46pM* zJyA4u(?seF!4{lom`K1rI*&$RUMKVghEGwsxrjy=iiP-WcA56P0t{zO$@@eL^_QaC z#>FuaVv4DxJ)sxawRg95LwY})>BiEbI@^)$~Nm+ z61H1Q31~W~3>_5!*eDbSVGpsnn$`s_*m@fYTEEDSWbZ!})C8wfC!EcXQ*~>Lp1t~1 z$JncgAT}+00mr80&6QAxr`NQ?rz#UeQ!=Am!E+l!#nz z%>dA&B$r#|8e)=GO0n$;Wxlo!cuNH*16YJ7@$CebUUff_QzYARHse!_59ZAr>^yQx zFiYmS7`^;KGsk%-SGd76Y!uMri2NUwSjh7bZF5V!)QsLv&M(tB!@u>c;j*$S&#as_ zel(($@_9ZBD+pkvwl0<;=i8WaWF=?EtyJf>a+fP9VX?OS^4aHsHv1fPF?!kLR!E)t zx|L)i4ev}Kfgqb^x93q96vIyq_$Z|8iVq9R@`94Tq=cyd=AC!$_f1C#)*a`iM?p=l&}$7W+aXQM9LU*wK)!HZN9f(NJ0hn^vNak^9q!l zv~NvE0o#3Z75m5D&QjF{t;6jA4Z?Es-xMBfPz1UY0;FL~IL#BN4pl)lDK0nPp^@t{ zjo{{WtI!GL4)2w}b)>~yO>BrrDA!cBi8L9$cahrLT&=+b zWwIGZXTr$QgfynV>1nPcto=V8utXZZ$Ar+?$rk_76cuwdl}5QKH1{FuU2oa z^Ka8KK93C8O=B?ixlEO6W5%nrsi4U=-3nkX(?PYWpIY@eXMYE@xMP$K_wKxq2-dBr z#oY8A_IomJu_F<%A8(@dC@0DG5c+)zk`T4LvA8#cY&6YtE>rUd0{d=#S82~L%zr~? z@fFr@L-U}*ZfG0Suclb>pSR#HJkq(4&D{{KPkZ)=C za*kk_gQ^r~X*5^0Pg6yVI+gL~zsRrZ4cti^>Sd)2Nw)M)=3+wwDT8HO>tkNn^OkHBNad36HH61zQJoQ2%UP!bL$iTug zjtJvda*eG>vCT?JXWK(5XL~6d-->l34BM=feX{M~N`TrcQXz0Z+Bzz_t59I$D@!?F z5Ku#N##Sz|4c|n7>nq81n{@|J#ZZ15Lc^Sj?n5e=Zzq;zw^i3U;#LeY4%~&W(&rZ{W)qKdUpwrVrdZ}CV#SpO{5=XqhgrA>jHR$M_0=> z)34Y7QoluO84wMoc+TX40`yn3^8>Z(mty{+t_O$dW7;+<24Bv2z6g^Py6J5&pmm$? zu_l{BI$nTbm+1!gLiYq!Suf>WK?lu58HtKE(d-ac%BF9M2|&x}GzmZtz|H=~Ufk5e zLcf8N72mV6&0YzSa%&a7DKd&~BNW-JSAk;_?NmWE0EWRmjR{3kOZ??rGnL@u^U#`R z_dQENu|G$?pfy*~D3_`0dVC*F#CyKRHOukRfy=oEBTT8Z>GnYgjzb@fW}ZTHs9FJn z^V4NicTTgV7;b>qX1@$vF?`3%RVbHRuWB6SR%cD=wpOyeREZK?uGR?XWs4srJsJd3 zD6E2X7DYadmdfra+eU8{&>C?>ZC~r(seQJ-eN-@-0QTTGPM{klOkV9kp@x$^e#hnB z%^ZXc#d-(yKPs{RO^(9eSGMZ{h@bYtSWQaN|5xvl;0b4EezL{SQGUJ4#&3?dyufQ} zZDRU^{Y{PiU$j zHwHFJYIo^o-DRcE_pqpp<`S|CVQzPH5sdx}FYs#bGu-Ivx<Z=j5=FTO>@4#jP}$oKua^p^-)tUEXpDY=QcQ_+p$qNEKw zp<4VI8vj|)kf>w_8INy-&JzmMP(!g{$2X5X5{ni|cIO4Z1bB`8rwuQE*vvurMBSjAIDMf*MNayHp*``n=dVp_o%~#0zaSfg{K1{2I51kj_I`1K>FiE8o2s zC|uY}D$IXNpk+8sigx>Eh&dNscDc&UrGA><+RjL^7K0!wFA$8r5v+oag6VzQJ2Rta}kO~yg01b*wJ3BQg?v514Zsg7LAy4 zYc@bAG@;*#0+34;nCY=K0SKZUitS)ELWfgO>pj3Hz>cDT8uA)bH%de-xB8;wt5*A= z2wNbMidnqW3Z)=ks=_y>60$%Wa)4?Tkjio^qNhg0ZsmU3XnSr!1ySK2dd5mFk)K?^ z58nZx^QHv>K~1O%A~{1iz99e=gySD^6sk=vQb8U8Md$M?24Dr{glz0o_B$Re5SwE7 z-2xYlV~-K}7|8Z#K*mIYZ@MsIcqy+x!j*v{^k))>S!j-Ga;#?PgqE8FxC?eGH;Tce z-Jqdes;oH@qa;00lcduR8yohb5Bu`vPL_0vA=H6}9lFh&iM{!B(pPStD+uJksw{kR z;lXCrk*zA&LH<~G;Q2JkmpB)0Y-t_|#WZym9`WGdg(ZUyN#`>CVO3`?Y`qfXE{dW07oz^TR--;w zM9wINi=6xe$oM6PD;qagco5d@)#jGR^Z8@n!FcvvPS3taa6IasobTe#zWZ}L-^HJO z$EwWt=&|nFnOxVU>@m=_mEwnqwJ+V;F6wjla7Z?Ed8zCJq5ClD{(TMEimH_Hpp~Ah z|FA}APk+(iFRuJNuT?5(f6@ucu`~VhHjmbk+wA76?JU<34wq>P+JFddpek)=x{kA% z77&OsWe_O))63aEoWH;cW&^L$J-9_R_q_1SFk`_!xQsPg7sG{pF<;yH_)>I%KY>jf zqUiDnFwz)h&%737sU~hLOnjw6Pw`apQfax0ZZ)cH<~-ag@InDzGeVbl`7K^rHISvp;*^fH)gtwvA8h)&cRwlteEU;6+z+HPTnzCLoWODK4=o8egaM!kXR}bOU zlTeL=Y`Rya8{2sq#JwdvnpM2sR{H%GyY+hTK#CYN;~fvUBHOxDQ1cYieFC%vgRN6; zmfI8yTbC=GrS+1V#HpKglMQ|&Jj3Q@5=amFjcSnqm!V8Y0ybTHPEvJFaI?=QA8lQ} z=gLfTG?pY}mxan0St$4Qy2Vbs5n@@ijlxYNTNkcPF>SA<^cL*!w%p9)N|~xOyA}&I zkF#7~%~{5&EWh3WrZx9XH%p4@+E-MkwC-^O$CzkXhZ}6co2{D8c5~&DA{@3j*-LK* zc5q@*%x;G^e_Kmql$-Z^)#%|H?9!X5ZHq2JlAk6-;|vP#wWr``dr5I$Xm4Pw&tvAd zqV&_zmb|_>mssvw{U!5Sw(qUBDPc6o?1q6)Vd`u?+2uN_bA2t)8b{k9Y|HT4d7*6A zQ9X?YJRqg%2OFJSrw^7OR3$vc+PvMN)msn~bGl*I^ds@whlM(iVCe#b;VCxd?fAfT zn7?G3Ik(&jH*h6U-Nm)+{kNa-UvZtLJ51p;FXQEsJT_!&$mD!PK(?t9xT1?!u}q04 z>Ig`vX-sn}QmYLJE>p!SYGUd|@r+!tNTGbYibE@dPBw_aj~lqQw;6Fg$LXLip~!u{BSfuG^QN3=_Fw4-R+`v zzvFS{({^_6oq1CJ1vcy5(Na}D`{dm@`V;5zPK0jXc^30t=Oo9E?obpXtS~d?Dm8Hj z&P-}vx4+_6e&kjBc$u#c{Szz{AleL4GZTSqw+ zu3$%;hQ~wHd(WxMvO(fJcc_{9hw%VOsQt`w`+e=0FZWT+}9wR+l#g=ZDedD=EIdnV2*op0_X+_zzINQBHgP@=-Rnb-JGX z|Iwh~gn(#paWG8dCYN@O_ycRRsm`CD(1Vx(qVrlm?t!C{H0d%fvbR18Z!rKaxJ-AR zfZ9^y*t>MET>ZOoeEbV}?CM8d{a(o9UT_m*P1{zpu)Xn;c{LlqH%5B*0()lf@P>^j zvlQL5O7_Fv$Z6vj5N4WsfzkmFDA7)y7Qx_BOlLq&u@A(35~m+SxFad-7#Z_Pjz?vT zr?{AF)2)OU^-(VDY`wtdmvoaJzrbE885lBP9ycCE40fErD~54cZDo2NMbW)DD|pbYH7xcJN>vK_Kp!GZT-PoQv7=eO)pQ6%e2_i#Lm;Ek{uMAD-e9zfovbAsqHy*rm1`|<{i zeZ!3VpD@KiB_@FUBMCU}qCb9NGlZUJ>-IMfbACWmF=BxYo{l(wIyZ^l^-s$%cF!+IGlMtp(=6amQSy#F?;x^R~U|$Q?{F)4pQL z$NeLIn}y%p(y1i61wYPrnaV&w34dj%^?9`qUFX;@AGb_ggbHLTmK^L1d)mzX(UXeq z2yk@eenB|nXEP7PL0V6jsT=7~bUV&b0_z$sWaa|{rQ0*v(+660xCTG4NokS4%GfkJ z9I9|{`88;2sgz2+`gT0WP94Z=HjAg>&Kc@rH&V9u`Hb!OBv_ommLJTQ-k!-^2U~Pg zAX58<@D&{QybC>tkIiKAC*zIH&@z{4@^r4a&sQwtV30I-Iy?T!r}`))7hYPP z#@2uOoPHgW4i7V@0_n;omABcFt@Cyz9a{OcHn z_UkmobOWVSWao;G}JXSqo_dW=;Z>8vkBdbv!E9%FwW ziD@_%Vvv@Pu^ykbvqW)qWw?_Krc&8GiEjB-9Y&($;}z_~CzBqfKhlGV>Ia_AWd3uM zQk4qTyjLyrxH;DyAT};4C!R3o?-0nQ;RB*ZeV(e6rAC$VdwW*-Su1^MCUXp>_Go}qr4;F&8g&R}ICs=pXK@dxPz=GO36CwKbUsv~V*>6r{%jCN9$R=csDBh^ zAy0|9CQ!9KC2ntG^LY9Z_f&VcoK+k};E=}%+$8q*(U^u$m2=N>%UO@|c9x02R1Z?A zRL6t>aYE+pPxT6RhG8^tqWPg{(G+Y24 zXwNvxdQ`Nt3?vC2pcpFEajyeW>bZAL4DQ|p_IE{0 z!w%qqdqJPE9>?0X`RfRoqyD==r8@3?dxR}Lwpr>sf%zXFB8d~&f88$78!4 z7+-hzY~#3R_-x_0Hhi8P&jL?OZ+c@UEZ9VObs43h%hYx(v!7_~|K}Kfq~xXjZsFsZ z?Bf${qjSd+)vl|C@3O7I(LpPPqk|U0*V8W3IY`LmYxAImE=+LJXUz9xqBL&|OFh|L zDj&@(C%Z_YW7r2LJ4@e>W0y~kmqv_ZJx{fh7L8_;PIZ#L8OsV!b(Y2*W*?ohO1w=>n2H7p3Nh@SZ1PKrZ!_)*6BED^(gk{>DPiHU>iK6WkqrGBw?N%!yZ2) zORFAb@0^(=ZQsd!&N8X#Xtw2SoOERr`~2*W(zSF}^7-s0jUUB^gr8tz13!{=`(kA5 zUn615dg;jqX^Po(t4dZz2O2+`{Ku_!SZcaWqDW0i=kueXEyo^BXS=@W(CzpmybA{4 zobV8x6Yl4D)LA+IPkv7LF30o#;4@cb|-VFa6b{#5Mqh=R*>VyJ3T;n+dKl9ju+BJ%Hf}eE?LI2v6bVbuTt60${13NB`^QH`9QWj39>=x(TdXoyIFEPX z)Az)$hduf?^q!PwAJSvzUvmIk#&6rc<5iv?Hhs=hecKLa!U@$v=d8Cvyn6NyBl7)$+TzBJm*_I zs+=PvU2hh1BSgB|i;cVyFIjst+l{$Hx5EW8V#Pr5WSMG&QC%$~`5gBcS z>E;OO%>=goW{6afz&^YgFGciXH*VUbLp@pct$1l#0(&mea-H;vOA9$>Y(O9z-|2&8y?L65t-;Q3cKd;L%GeJ3R7o zK2L9s*}eaQM@xv)rn3;Rd;ea?qp3vv7mt4G&OJKWojvh$xTQUDagPE}@^~~ep6Djm zD+J;N;;QS>e^3KR%dbv^nCB)3QV?$vm7lsIj=GX&~;^dS-d#Uq32(d}3^q&nPk?Imz= zkItdw@n~%v(M_N7>iALubv@dnvhx@n`=iFAb;N1YISADEXgLwJDR71Cev>oW9W2=pZx3c zqE(q_Y;q1@P!b()SDLCGN14A9IW>^Giz5%@?D&oB;Qf|Tc1QO0{f-t}Hxj7-cgRnW z8N04K*XAYTH@pbRMEsIhm%339Jio!T>)O`C$JfJS>fskzE%~HJcdhyJMXBX15PvsL z>V#8~*3;DT1U-<5k&Gq$6C(xoW^?`--fL(9op{{DsfDQzigbDrt->xU^cx=vHNP1n z|3Y2s0JW~KUjh5=pAspoH{15$4{2XGd()LCbqL?nL3~7vd}A%O-%qyTN3%BFCc=1A zT@g1!5&nqF{QGpMqgWJ8roBG6VXkm)GUAk_3}3QR>3q*|Tofb397nY%hIE?&4rddp z|Gg520YB5%BdSyyQEm9bTB38_Z33*Pqn{-95&v|oki;S5XJK~Fzc1!A+6Mu_d8Yx^ zwqrR%%5_6fA%cWXZP%WF!5hFi6e@7v-KfNcx*NAuhVLUdLq32r91P_;U+dmBUZP2q zRP@HcUl{B?cTAax0b_V zG|?7vnsH<4tyvg&t5!IAH4tOFe!d!`67SLBY|HNA23fh%X3JWbc?)&tC@ri(WnUo{Y)3pkz!_jwPfH>i1if5~$^=_5up zX}laY%0=b49A|ne&#~7>Y$MIeb6oHdhe$))I@;;Qjnb(Q$3DFnmUuB2yorZ%7T^dX z$YqKRCQ)+`4nh1}rp~}{9~wjDNVSaTVk*8h$kDu^*eh)YDZtVq%YWb;RQDL6@M`km z5(*XuK#7pVVO;59+Q|I#Bp6%~>w+-Q=x_WJjzNR}yx{fwju#t>!KS~LBFGy{X$;MH ziYo28(N8&!H59uhYymSmbyzk1sa<+!!zm&Gh=7L*7`k|K%E>vNt3)McLVnP8A<)sS zk=S2a7wE`pB#!J83r_qo0qr#jI!CpetBo6R{Dhwg&~cGSsxTiUxl9k5Q{Uz^x_hZ* zXzmqohU69odb=;SD$RB!PIPr{hFpabUwSDmtR4BhXju7xAo# z`+{923NBL?M=SO>^cG5a2O=D!OpfKgVmr$#E77KRsR=k?qt$xtD$-2l{8Acs%?Vd@ z3kcYBRV!4^B$Tg0v|P6z6jEHS+l>-+LAmY|tyH2X#dAub(C`sSmV1j5NFI8GV%U_8 z3R{|YsGCY}aM;{LNXEBvBpbw5()Iwy;|4KEvIjVp8N_&Lcz|Pv0ZrJ~k`y$?AO6wP zv0Bal+2Cy8g$lcFC$INRl_zO_PWD61+q@=OkK!}D2-9Wm^B4NF$O?X+ZW#aH^uL9+ z$o8JPW&hQ{0*`@FwG6d(fB3#{J&~ES_Jko3v1S1EO$V`Pt6azBAPa%x%qZe z`u>K0pXJ|&`1j>n^w0p}d4cnf;@_M2H@%bQrf=btm-%-w|9*{s|INSO;@?}F36DbH zU5;$y-yiVrYHsX9{(Xc~j`H#j{ymjbeEIiXPSNx7)4Uw+w|Bq4_=FfxiMtI~!~}o( zp2WXjZt6&ACPt^e@#_N@ZD(cs(@6-%jg1~Wlkfr$j_3$Cvfa4ybRKXVkyH|S*sqSA z&5#M*es!E_CU%e-{OY*ZObi`7~m40M`h({GJyZG1b;Cxg8ps-DC3CY zifCkTtTc&zq}@{-`%U8G0~IV|C+po>FXc4Pt@SL9dv;)BIj-%%hBR?ZYA*JZKJ;~L zX)a!}oLd4z*}a|i!&sdQ@r51!=b&Lt+YjL0#D0UfwDHOX{-2s1B!G+lgPP66n13Xa zc0n}5%lUmzQZ@ATC4C91zGK9um_7t>>szPkvy|($k_0YNSVWw46>g+zDE#RV6gu$* z2iFsuVp9p!dn5dBJ!tsnDHAg zz+&+s{aH4};F|9U2^QB|ymM*J$4ieI%&@9>R}J5^R>jw7aHa-j4R+RGkOsXqcvI7J zUW26?d`p8xDzpg8H3IhnUoYgT2Ip%qNP`FPp9|5oNP`I)tbS6ZZ`I&b4Yt={^(wV~ ziw0#4UZ$VT(6gMkcmdC=H5jSE%PTZN4W?`G=yH`_qQN&bxLJdF8hlKHJv8X8!E5;I z{dAor2>ZeZ8eyXbGc=f@!A=?s&>Y~>_`0ig(KQX8(_pCvw`p*T1`9Q~QiG2Hl6@gf zBlv2D=4-gyP^ZJk;`SPzt>JFT>U>qsjXO01yER7+XwY3hUDLB>jg~~RM#$4(rKaF-4W85t zCTNBxXmF+mvo*LsJVt8)o{1R%{7GuTKzMcLt8bz zB^sQo!KoS?r@`SG?5@F34O*ORY2Q;`%gBvuMr@^@z9I3&68VuE-UW3&+q!;=B zH80?=_-Syf1~+POxdw-8Fj9k!GX;{nY7WR5ys7 zHqwPNL?M(AT(o~qkX5nF<|?e-q{1SH3eTph5Wj@sW~=;FxI>e3wsPETE4FDAt?@Zq z?ZyAubxj|HR!EDzr?*mI*3ao z%h{otf$M5hv#zNyU4vEMtGJ-SDlPIw->LNIZ&Y|qqu zkti|HzdkvG$jwY-xFEwLfg>+rtK(HNkO^`qP`ca(sJi4a4?AurglfmHFwje1i%jf6 z!0G*3x=6t7U+H9zxawbWXZW{ul84*~>{X80oy87fu46rFEw#rOR}St{aM9&fPRep8 zdbH3PUJ>v@V#4L-BcDQR@lgn(frkRp<>n)(*(Hb6!R>U!bb+^bk6Ke61xC}hH9$zis8>gMu7`)4QMcJs=ZK0Ct%k}T>SR(yaDCeVM|z~_R$BH37m8ijtPn>zNqPKr_ElzzZ>Kxy77Xuvc#EOAp zrK5~Q#(|7Q0V12?L~m0TQ1?}Ya*b4=BRWnDa_oo`{nmHYJ2JrRB@7iE3nE2RLwS=Z zlnfOyt61YOuar1;#)%!gqSK((|z!;7S#7m~|?>!ZDOp+mF=G!uC{T1xuijtAYv7CIR(Pzh!3D+}dH-lR>DqQ1$!S{~`s9AEc9 zY^oh%Pq9N=7;p=5n1Mz<92iGNPcgtNVVLL$=`NZa&%n*#idycFXclnQt({OTHga6- zDXy3&_=!T->>0V);q&GbS5aGayt--O(a{3^Z>xS7JSr_$D2)(>kKvYzFPlzsG$t!C zPj#J5|G98@Jdu#h^cyyzYjpIqnX_;b;3!KFoAs>*nHI9H14rU7j15dqM)_V{S{g`# z&{Li9G``V9N^yIHo{_C9_UdI#-cZm&PZj+&Z*T zoG%YF-Z0*RQJk*`w5=ez|1V|?RpB-!S``mpL@v@)u@XZcv?dg#<;#~(pNhZ;#SXEz z*lgI=N3>=w%AOXUI)UrmQJXf32TLfaLsR)mK}#?}cdq)sJKu9mfy(5ROi`$Y1+8CV z$&K!_MLs)ev^b*4pw;TX?nI+a?ODg3-r_)Ul;eJH@fF?35H!;9Qlc2x+b>jIDB9{I zL6_|-6yYrN8+`O5MWHQ(YvaM#$TaG15E?CNkl`2V_&HH*-ZV+$i^k#YllUZ!bTscH zhWY1e^s}IOp*^{d)IMUA|2B$ zFOw{pJ=a(`4kZV|1>ugSuRgv5^+dbuxYAebCMAt>H0g&2G=N?EiI16%j3PI%m}EB* z@(}iy@j2pnwIAkYr(<6~v1KZ6Bb-R@AqeyEscCa|10k!YAiRVR^}ed5J`Drk&Esg@ zUkq$R?Hi}Hk484NliH}Z*DTH`6zco2hk`->x+ki5PoK8{97Vs9hSs!~*4 zXWNR7iTEGh99c=CUV_wyB(WdYSfX)Mc{E;25(oHajwR>Oh(Us28jKyYqgS%nk|$e{ zV|=nWQ1=XyEz|L8ve>gdnM%?WRYMV7UST5NT=WE_WqC-L<3X}GyaDkBr8q_o5W98v zYasY_Z!Y-#*09rO@e+@fKW7E5MIMztHPd!{HVblWdp>PF{Gt>pxU?$ zSWp8c3K(TEycIRG0It0QcX7-k_oJAi)yb;Msv&fz)>WV5^iQciHwVP(BRmU8GW`nGyRnggSh|H53UN9^N}AbJ zh6Ycoih|Mm?Xm|5?WQLS?Ybuk?ZiGpKy}k9KSY<-fl7E5JgKEHw6d!(u%esLuOwRN zwKYcQzByLtRv0IA%Iz++&x{wst}Yb9va^s?3xu%l^Mx>Rt`MNTjG&;m$}gh3&?SS`;5r0cI@&9{Dyg)Z4Ag^trt2oc?n3(du&g1)*zmABUO8PF4k z){!1R=%K3>VA889?XPz@NtlcDpxPzdg)yA+<#v^!Z6hpR(|rV8_Xa}rhl0=_pCXzm z_`;{>xc@G!AGk?v3rVW4jn~LHddYbBOl?bmPqBJceYECNA3(A%10oc{C_w703_$9v zxq$8~X}2o76_BpsdsN>UAjx*zt9JCufF#>+pW4wOp9mc|A>k91!3;@SX~J ziWTCQS0M=J@G0T4Q2Y>0sLc~71V{J?f!XbZ!0ur}pcpDN=Go%-BvtHWIeP&+s!Dt^ zR7Yy#!3NbZ4KF`Lo369qr7ZAo;Vt-&G-UZ^G>)cvXRZSOTa-9#Vb(|GOJ#&j%whzo+c*q4ahx5mtkTYZVaak!%f5GX`VwE zQs4*GbPAG0h<5b{O-8vGyDC!Igc}fqM*s;Y8M;~nQYivZ0*uk*7+IX?5QmE)v;u_<7u$ys=`8rjMiq2K<1ZCcc>*9^ZS{^P zhKnt_WHl6p_4t%dr(pxj)3If&L&KKQjXQP@7vowKV>Eq)Pv{JFqwa57&haR049PBy)hMpd6YCZ ziNYp)mZ5%=$?@|Dyus#{{j510!DoGhqf@%Lrt!H}s-d7+IO2#EO%0-FAqts}FVe;K zApu$?wbFgL1#%(q3j`W(hJY%>4+?O4;_XO(MD%NsT~C@);GU{|`6FTruhLmuVA~^N zfRvo91}xm@29ypH*Tbl?<~ljybtNiPNoD98*S+aC}M0eXp?o`C>H zky9}qI^7`yAwfM1(HTKbgD{bazo+fS{&@C#`u1)6wr|_@p1!-8Gy5G_VdlD->4tN! zFvK_AVAc@WyC`aySmPEgt}$m(^g#_nf!%t28VYO8zW5=v<`^)$)}6zm{Tmn>6QJl* zuZU%e8>TKbgZgObEqic5ow?VvDtQ$e3K+(dD2lI#M|_`#mwcaxgBjYYIzc$g_Yr9M z-iD`;K9T5)IVYnGJ_2{67QDvQ_8>FP;rYIQ%b3Z87SI>qzJ>Dmu0gT_=EA-_WX|Cs z5DrBl{LK*Abi?GmlvhwL9!1so&OjNjV0j^0ix(TwwmB9LHW)WvW`=lAey}p$MHeD_&o~YZ-&kMyRrcc5I_;i#iJHKs+h>aO+gZ2ybXH~ zA;iK5VG3pO;#WvXwAhhAiyxKH2x4Wi83pm;z>zcpAA~DV7%%o6O(XD@%m@@?A_Non z5_RBHP}%fed}l_DS5|#+97hE&DxuEfyC&62tlvciyloYijHmnX;xbf<_Y-Q%?R&-9 zqe=-vm?2i91$gmA6w?B5%mXwEAA&6?htI*o53-k!^Gpe|Wu%{Y@%Ab7H$DJAoXVhr zdDMY!;*9S%jiE{Vrg7Mr5S!3?d=_RNDjDPW+$uM3C@y10o2rurE@L~e($EV=R zsJG!b+)UV;rMzpXs{nY=Qu>|{SR92a@S?SWA5~0DAiS1TSvJ#@mV&2kHG8C z8HS!WtU~bx49!w_K zi5W3?66t=BgC%czJ_Xyh(8n75vDGkMdxwVN>){BSF2ILi*$zH~=-*nd6|WYpcW>?ZKxG*!$-bmuklfMcN5oalhb;L?grunt~^tvy5J6=u0L7a0}AHirpL%wHW0Lx-uAm<^CcO zCnMGAOwp*LbZz8W;w8K|9<|{a$Yl)uflF>N-3NyesTIq^4d-akVvCaq^EuK=#h*|e zmBg!PJ>K}4H=RgFOF`PSUpUbyS6J5VS57p1049(QbqbzGs-K5DTD|qBAmaqxZMi|3 z#e@PeC2aa~%{Epf)}q3v=o;9Otck0BXAdYB*P#&pW{_sO0o$4jTym6)`kvPcysff} zUW3FJ{_q-V#W3eCaSx!9zWjFVpX@z84CnvF-s6?P-TF5-AAA6AMA}OwYndyYu#{)v zq^sWZK?DwOqj5xMA-EIixgiDD{Nt6^!`0X5|0z_ezs9BVFP9cR4y&$vJ_ak=Js*Wr z{_}hU-hadMVYnG-1Iph(%L&KiJ3}(+C{SxhO?Z*N<}-45@ga0s>w#%hu!N%v$M)pa9_xw1Eu?FO z7Bg-}YhL7L-p0L|a`8?S#s?vz35^I|aocWGjZZ*^H@b~ufT{6*!1D|;HPT2&MLgsC zvv6*2w#l*;&|1n5Rf@wMD8r0hFo(3q;=_H|OUldPd+0Jg1%E?Y_5vK&*Q;L&`D0aM z`cn29Zbw194UZzNQ@rGP%gAFUkiTh07|uglDdSy@O@8@icuRj;%!&%3{=;z`e;(dI zs$X1W8jA;*#(K)D;31^)3^WFMUYvn+@|MG`1L=QFG&0eI)Ch6=Ak)~UR>M1rIU;yv ze{+%QD9PLB`&KyeF0Wrl!E%(MzWBWF#j%5F6y78F>O+{{n! z$?eJSY1?Bs-JO6_v=uT>-pzK^*L7dp zeck(g&S2RV|FX^gmBXY)VeRlNtDHIV{vLgUz4x5)?*`uqe9GcpruiHunF=`UW4fEe zo~FfsRi9l9Cj9#1KAi9V9({s$a=e@AK0wdn+F*j$7x(7)guCx7BpZTu^=JerhADELafkEYA*<(q980&seUHbU!2?f_j#h=2dYpl3iSlHh=LsC6$Gpmg(?E0 zM4@PHOr)5>E{GkaTK}~%otR0|4@xpa$^;J5?bl2Y$!I|+=(AwX?Bdyiuxbhj$>7Nc z?8L@O-Fqwm*6hlF6Mc$QZtenH5#Y42lszML7c1CNsXO~XGBc@7NAXb>+U5tA(Z=+I z-<_d7QbfU+Xy{KMSa;%4!gQ0^62H#Y&;JkvujBgtzZtVv)ktZNR% zY61(q4n5bz|M~CU0xa?IkJR|5d-)&a{0_qsWL%m<2rk*7V@wwv7SRn}*xL;_CffsL z`>lG}A?rP6#*XW{h+~*rm(cC~1aK(VWV@)GlgkexHz8USt{xVP4g10=X3A;zd65n9 zuQ%?ECuJgW$)1Eb7TeY#Zf8|(Mh1w9Ck;xBgM<&3uGlXK3N`c z(j9h#h3QkpV3wyJk@#cv%_0v4HWNuDk;Q+p-TLX$z16H^K(C;5E;Cwh+^vgcqXS|j zQ#G3(&`qlNh;0Zc?7iZB#A~9lN`|Q@htoWbbel4fjh+_cOot@#637Z&Lyr+rDE^P(2i_VXkyU*x0*6#_F-tO-Q{v5Uq^fz{}+OV{s2YvLhEG#@$%G$xw!V{##v%qtyWV6q5eY$$ zzuCmKnq33e>^I?W26h1;qf7sw1FtukeV(1@ z(z)AuPRP=8vu&HqcI87rZoEu`Va#_s4PB46k#f$m;ju|i#150{>vj&0Eodu^J;S#2 zD3Gpv#H5}NOFQ3UO3(FD=eJmFQmm-oX-R4?w!e0bEPgIKMi;ujtlBxd*M3PX+S#S= z50YqN`u_99t88)qI_b-;JI4>WTNF(@_YeFPSv%htyhjZF?g|NiDmxylz_{w({XAPd zWUMsrdFC3j7A<7j(Cy+});8@i$+?9&)3%GL>_WPkO&=C4?RbXWKPD>}GH)D86w%-?@cr-gcSK6FPs4G*JO{P3>*hBo8& zdim8)s+Yr0+~j3Dg)JPOZT{eqo1J)&ogCgt+VBW-4^KBw{TGJE+0YT4q{K(qlo1on zAKP!%U}W!s;YB+;HzLhE?_U^pvYr{8q*OZ_m66`>8tSEWd@a1vPP_lExyke673}Sd zZqoA=?2C*9De_4kf!NIHl2E}`WcCaC@L^InmU_ioY=36wPA_qST7PKLlc`To@Op1z zzhov!1rKwv-afGoY{tmW5jsw|MX&axCaK3Z?yMX+wv9CQJyvh|SaKX>?~Gd7^ZYSE z7;q+YPih+&cPNQiBDr0lAzkRNW(g8#JCE(0cFVVr-pevZr?8hwP1Z@7lQXAePL&m1 zmoEh&SJCa_!0_-_gqZ~ny6gaul`hA>Adr41$5Orkr>yke&RJ5{ATL)*|D0n9$sF_+ ziPOF^zaw$#PLQ~LWdX2U`^v@+09LzQ;OJUC2b)a9YTp#j!QN;Nb|iA}YBE5seMc$> zdowtAPUhg%JOJ5t#<;o%3NI^?Y^%;STB; zgsYUgT)XOzGBO$z*=VWrx~7awR4I+JUG-8KnXFP8WV`CHGBQ=AxMe%{9hx&hfnN<; zF=VSUQdUWgxprP8Fy*O~`dm9N9Vq!K#hq(c3#yDPP$^VEHM%1UQNa7odkaVz6le$+ zfwj0T0$-@&rCz+KqhJNY%=koExep6Xr8qv7)}0xdw`bm#IV*GK;ghm`W_wsupYdmE4v`(+Lxi|pe9%3QZ!Cv9{fSdEc-^xuE4U~T}LvlO=T

9A(`3C$=)m z=edqYdB}4xH!Y20#S<$f_kWoGq-1dl8$2mPG_%r4eZ_RPby8oc$Yd1M@ zuCS#Ow4$1#`TW&Zk0(?4L{^OM!!O??xuumPcYky%3s$uB(q@b^;D0?PCHU`d>Hn`w zZuYEsN!}xSn^YdZ*B%pHy45}dC2YI#dFaL(KLfH##3`lomp|C zWKt_*bDPNp<|@{3GS)Yz{AaPahmTdFWqcZE<;%$EV{G%B&ejgigqO?8)NrgLc`w+s z7@IfG5o#*AX$2!Z)aPMnT1;9-!I(6e*Dg+LK8P{v z<SOHVyZTD6-NF3kCK;CBL56tU-SGjN zS=c#}6L|Dm)ZLd0F$q3f7;42j+}jZA5BM754Wui_C@;COup48$Xk1-_>Y|wMKD*HuV%oglBnzLKIl{E8az4m5|G1ztS0SSfp*gPq_sY zhNiz9IPov_1pkX8a}-BCEmD=wZ&iY>xtD+J@#J`1W2|X67JRGrsrw-gl|dP#Mjaof z`vxO4##8Rb0$)j+#)p16p3^jbwH~=naArP0R2>$Cxg+g<{b-0Q!BUVWfDv?@G?mZc zi0d<)n{RWmIi8|!j!(I}FKDqcN`BW@MKh7cDpp(h@O8hsS-JhTS%=wyd%8+{JFzeC zIc=!?o|@X2C_4Y~V|(u%A(6ax-xnuO58!{y6V8_R4*6(l-yVllLXwpLz)_j$EwaPd!V_%ZT}QL=jyGg}OG2 z;zfmO6N^I=nOesBLsVHL^4HTO;^RKij61uc7`qh5`r7th?!TX7v+qwBKI-4>JlIUq zYscJVXCdrRr#tPQ##YadZmU}VhONCXL^M1xKT%peABBm{A)l5#^Do7CW#-MrVAWp7 zUftZX7)xfd_45;~9k`~C>CHv7`;RBV)2z^~;?(V544i2)kl4yVXyJU0htpeK z0gkD>zD~|yBOmB4W!=y2eIO;f7w2hZeQGOxR7>retWSHYW$}8jiw~qrm#4G1q8`Jl zr;{uE=<=llXybOj2)XewsSgWVC3<(us*n3_Dhc;C~}S+#1^guDO*{Pjg=G6pfJ;%(@^<3}wy*11!?NdD6R;Zi>_ej3lZj zeOr0rnaYd{W5uVKxFCf27l#Br(ZpKgWZjBm1O1AptDe1)aK{z z^OpI=^(|d0p3HtM3YT8n#NICHEhe#VN_wUXTmzqAzQukNoo|R-ewlZ^+q5D0l>7W7 za%>b0!OMM}@2M z6G;Ite#x%?R{VQ86Yt{9to$*D?OohMnqI{&E>4Xw;dDn<8fm>@#EFQkOx_|ZHSUKt zvy{?a9nRxT4rVh4%|tX&jdrkwr50;`P>yoVqU7mq=PgRUpYlVdG5cI5N?9C7Pvh#3 z(FZwh%q;&)EHUAjOe#MSbh)jX5S1OYR`0kSmlz$b{SAPp!)I`ICf&yOKvf)$F%A(JNfG|Lrnw=T{=8g1`R`oOiLI#*UIyl zpGJvg#W0_9*Hp1ZOZ!Pb2C`R|_D-Mvjv!pE8xvWbSza%pson})c8n=A}Y9hEaj1_rEZwvJ~5tke5l7@8SkV{;xMsMPCK}ut;d|o@uV8L{F;6t@I;Q6 zU+caINTkYK?ly{==XHTwaF z>v`@h>>@PR^Wb_Vx&C!28@DV}ns*-zvja6Ug4xi$>Ygz67lqy-`+~ql&_cD zgcmQB6-?---O0?cyib=U$l(n+Ok*t~R!HT!7v00^md8Xs1;XXFvVt>byo|*$YleKM z?1v94zS+$}Rz!E(NHZ@4@;zo9mmSmKF~?C#anw>vK}VSLFq^(&pfqAVd0*`t5%Vf?F z2nLV22*`kHE_;|Cnc{(06L;f-obpCb$vnmIgg=&t4)Z)v(hRflq?24doy5t;tV75d z51OJ^bm_!#&^Yp-Vwm6`Mck>Npp|1ao*i!@Zb+GkH4Gn)SW~RYa9>_|1z!iD0J`^d z-gRgV6@E*Zp%5~tF!EOm3a31rXAhe0GvAWnSnW z_u8{Gx6G=$+{e9vwn56Dm*d$#E4v5G#iv|*vDp+@-&IqsQ-Y}y3@!i!brm--882Sa z-=Pe9`ZEC8sSlwZvf$KcJrA*EGesYVt{Xkf8^4En{6_!R_+?=jFC+C$G1N^67TJV{ z!>#}0ATkPK_K;=1C=&AD!`Mky%;!v0Kps{PqjzbR{?(TL$Iv~ikV73Sjov_D7ee0s zOO$n+NP{oV&!6UTo`d&K513eed5Tr$ibm@_ERi>+CBtAhPSx(d6V-w^h5)@B?~A0l zqb5WF^)Py$2d;mNKW10GZ3{zbctxdWK44aGOrQ4B`?RSc+eqIhY_0# zXDoo^ZeD5@I>u-{czMSF+=6>6?pD1a*IuIahNjwF_J3QOzTUthY@Pq-9f8+j^D6v5 z)*$Z)dAuV$Yx_TRguG@I^xTJ9NBG%B9bqcYRC1M)y32jtGf{Opisy0yD_WCSc-ezM zISic*)bx3arU+B3&NK%58%>6DU_f##Uiay~Qq!ZN(Ps@8fc#21$%^hv%CL`0MYZ+h znleW63P?W^#5>Sr!xIm8PWgY(JMIf&;~$Rr|F(1dj!*CYy>l#p_y3d5v6D3TI>)2< z_yL0ub&h^PymQ>azwT?*Ilkq@6YepnDm44gfVSuyy)~g0ogTl#T()qe5`egD1Cb)K%a#YAM@n|t zQk#g$S}NQAj)?^+-K`jCXVP>An6J*rC1tX`qyl#(ZcK3{TX1j9#S7$852Kg_0l!_=<}po%;~{Ub9Mi*+;v z6OS-&MSSs~HYeL=xQgO#p+fRZ3J)5nx?I2-gFUj_Fq7669`mecsIDd#Wh$rUW*Z+c zET>&Vo6jhna1d$Z)E_Br{s8H@#s^X;p9sB=sD$)o8bLQvOLSTKrm$1zFp1VOvT$(a zHG;-fmvEX)Ev5k8Xf37?N#0`qT1{Sb@yCh~%Jir-0_Q_*0 zW2w1-F8Er)eqf4i9Wkpd!Q2iK)E0EBu)=BG8oGVaG%(%i$K=OTCT`3+$4))oGu8DZMbVG}A~gp~y=4Ge zrNzBZ$Up{`w3&P8TX>}EXbG2x1XjD2T9#ZHcUNt16bq|i>Od~b6$%OF^ak1p0P-FN zNPQgHCKc3^VwdG2^*U@9xOts|ug}S_V%UUKlS$Rmp2|e48|vFaO}4$Ho`m6&#~cgc zT*Z(C5%^ARJaZ5CK{50N4Z|G`YTi-LZmUtTS{1kn_=dt{)awp(b}I_SP#p(qgG6Hbf+vYx*6%89_8d9-ow`RVxF* zHlU1VpfW$Op$fOEwT$u0g7J%%=$0*+j`1Sk9^BPvmLG|<(AY{If`U0JQPq77bdPyr zd+swIbGq!tM=ei15ogUjOyiT`EpE^d2S&0zl;JSjiS*H1Qf$CWTJTdgQir-CexthF;5Xh7PGF8^v5r3{JLJ^<7~yc(D2f} zC=S%Gr0{Tgtf3R#v)Q(F$@)^P(>&%Uob2MdUZx-%MJg`ZFy~WQ$FS)2Nv4|Xu)Xq1 zG#WgbTb0Oe-)gpgf<~pzwI(zVd_sH}da%6z;vWKr4*)#os11DYnvHc(2HF7)NEzVS zqw+k*dE!udbf^sX^?qs}`RK`A>F&T_-@)+`E%$O5c8WFq7&;~Hm@ zV!Msnnf;vtdOke|PsO$xhV3?LaI*cteE`j;7n&eUR@9xMi-$2gpC(* zY_wD-0$hKWT({e{096cae}>SQdPVo^Pe@oUrX4JfQ70|pcJbMYmK^B!^$@B6^>W*6 z`vIEPa_!6An9ki6E4q?nJ(UiD+l6v?%mO3CJf3XX&J>%dw<(4vd0y^1%A=zSb3f!$ zyudLQwZ6b%{)^UfiXoRXhBAyxsHX41fYQ}nXJa=; z4G!aZJ-8RT*C@+&zT`_((91}pp`uJQAkY|K+XDok8h~xs*w3Gv{f3?2*vb0+HBuT> zE8874$adLk@ubKox=*0UZuF2njXnKjiuJ2TugYd?lS`a< z!X2}pZ<@{ly&O*qE;>~pm!R^cCP7-S!s~S)V1EEtn%an-{I%OmDu`A=q6(5Zu-asT zif$^Yv)jUfVUK{P+FpV{w%g)~2a9|FR68XzG!7S+CtgIU9L#F#ATQ85&Y{>IbLX{$ zm#78VhZstk8P}dE9mY~(Pg)@B5%q{|T28StEkm9JC`6fTc>&||bmsmAXm`Kk`1 zNHrHVr_lK5eTDI*$Ah&9R#$4TlVKF=6m?0dJY7W+)}+Ir=F9e31yDjR<~ke%)SApd zO{Q^+>wOkCLJ-VbRg0g{IO%W@(Y*(Yei+z7ISSv}%SkR(|8nTZN6>W2mg>{}^%#`S zV;%zKF21d9zw)N#yFz%jmdMP3odER{%>@#L5-#vYDJm0AVr79_bF1=NLHwmAUN(1eCS6o5j? zu-h7W_1W~aRP((n{75s+&S3@mMHugOD8;;G14bv#-#;M1MdJI&@vNpm@*e7AS&|!G0 zYy^UnN2;($0l*F`6xrMpZ_hEgEZD(;#bM({(U-Ix7bvQ%C7gvkH_}>2!9lAk!d(iD zWrVscw5%YDcnSiySb+o#R|0^oLbbn91zX4s+lzS8IE@OWY;5&)Yo8S8cgVI!fkSvR z=c8WQNmaAasz!8+3bv|%Mi$tp27s;?0bDl9f;3XwkkZPgc~QkVOn63sqiInW4uZSS zgCrBL$>2;#OQALOLJ~h;=0*l0?Xpz^>&#VT?xFIcE7pMI{-KC+KS7e37ljZI9#S;} zYb2uGR?jh-k8N%>iLuAl$di1lY$I_DNrfQmdD3PjuYsCWltKe692mkCO+-caJvuCi zi7Gba7mg(ZLvT`6&I|y|TX^!#wnLZ1xv+1gfdW%D-x=}9;FBA}r^xCKQVM8gNx zS*f6AaOz0AI$kO3%g7nU(1VkcK*sM6JjK{Z*|sw43!&CGZT!XOEqHD}a-xm?%z;wjKoO~J#!nNB;`1;ef$jw*((i7I;*bRQ<& zS*q?j87~Xz6mLgp(tLhWbRR*P-BPCLPV&TRd?X@>@R@?{#UGJM(yb?O5MHL$8CS`4?4p>1$t>0=R&9~e ziHi<=m|9r+AG`$SZW@oq@+VM_Iap;01fm|L256ye2Gl}EW~1z#XW zlPPX?-6Bp>b-w0h-%F0#d-o?6#k^l*$v_i3O=)y5q|+z<$Rb~fvVQXh#hgg?-dvkv z_B2zP=2behoGbkg$fK9-(H0oBK?CeSP(50=iupG;^-(Mj+bd*ocKa~zV@W_oXb+^TYgsen6oBwL}&R4Na5li~w)aFy- zb&QF29|_|Oe^;;_ucm~pKr59obzz=_0YS(h;bRr-?^hoUK5>a=0}SE~XqX=JeXH1$ zuSL!J_y>}(6`%_uEH)J@1cui7EF=(G?2BkDMbnxO0?vw{yDWXvdGbZ)e%|fzP$Y21 z0nSNu#9h(Iqb2y9Cpx#3vSXsql_u9 zCj`IEkv?F{RK8RUpPpj}Uk{b4&atzvN9Aokr%J1jg2zun1gu zg%yHT22%K4UaDx!K5W-b0H$td7d-YG-Uu#zoXOknmp(qrc5j<3)!JD5?f2><&*Ck= zF5oO%xxH7$_uqR%u|?pac=Ni4rUA;)&S{hvV zq8^vaazho;8DJ78ll2gIk_)`Oj7e|ykT-$uD3WsVo`z~~JM0KF(UiuG< z*byhWzhT37#H3iji&EsPYPxgAoF!U7v3jI&nWSvr0MJfM(Ov(Bt=rL0`uZE@+93zo zpa5see5u#@AyrNRpw*Om$!7O|?g zp0G}UX^+{jR1kKE?k|7kli8_yxOmk&?k9r-G4q(e2a{}1sX{JBw#*9v+SFk1vMf_s z5-1NRlEkU8`~VWya&0k)_LK>#7IV4AGYG@`vz*qY;cNEU&ZxNV&r=IdkyXavW_+_$ zz76$k`rA=)-_kD&>?xmp&24fWuu1wNRla4M&-#MOH%HZ{)AZ5ICVG7Ul^?2$U4}Lln+`#ruiX zk2`2s5B|(Ol0acU2odUBIyZO#E)f`#GL%FUA^Myu+EbOB!6ij5`LG&A6VEX(AvrD3 zDd6J%EK^x#QJ&J}s`wQveW#1GdjoU6)7?6JAsI}0g=ZRSNDV$~G|~rm)M}9RHdn3t zgR8DCj)FTqG&5A}$w-r#TCfIlE#^D4YzpZhE!+G-G1oSVvlyZ{%dz72+|oB{Uc0Z@ z=v}=dzBmm|J`Ym2Q*=SZ>o6Y}!Pf4I%DPCiq_X;W)zkDQfS-x4K3@6^-diaj!QKbN z403_TypM{e7|hg__)%?70L?U-Vi36Y%g8nh-X z>FIag(YWzR-WABChxEVeh;j8~Kg_R*x<_nm&hFuZp1{0`ep`Tk`z)p{>@llg%w@?- zLM71z@OL>Md-#?IYuFkvw*P|F?M_endI9;NONItao_NcU;W6I}JCHKG04Z`^CKZpA zvy444UCW^#LrJBo*f0q!m>OsZ!W?y;S@)!luz-ex0HVP~TG?QGo53-|G|CI+FWx~c z0MY&3aPGnP=SY*T>O8x;C%)4oaKU39ct6xu8>e2SU2?;3#%U>!tYQQA_Ax!OihIFL zOf_#VXAAbGNGav))xG_s>u1>)d&dS$K*3PVo#)tqeTjFzij5&c{1)asdizD?5 zMLqEVH~tVYI3_osPPf((ucZR4Ki0AF`(-Juh&{ExbH^{i<1s%DfdR+dg+E~UFK5U1 z_b!}u90Jq~@gfQGqqr)(X!4lT?t{;{@rigN+ban_=19tpkGCCLF~nBAA_d{)yfG?mcGZUQp=U%T%0c;S^l98;a`K zSMP?#Zhe!6V#ERqyoM4|^|5>1XwdN0^Z@A1|D1&$=qbH>FB^FvUV7smR(PPFcn^E> zKy=KLHChQQw+uMuF+WuZ8#q+mrH*ZvT@a&`kqH|nt#xIg5$mqf%kJ<`w;kzLbkDbnz0Sa<1ufa z%jeEvpR%g=BkhDD@dd%iK+1evaUyGjNF)zIXNFO%+1DoE=3GdHl-+xe<`r#cdeLcW& zpRe0FuKBt}Wv+3*lEt2@j$)^dwi_|}B*pU_{_MyQq;iyFIZ4OmgV!WE?W2Vua?;rx zhu*^Qq$_dIcZ#==)xZfR zcT%l;%OLXCa5rhYL2lr$Kvu|2c~+kiBc^*c$_FL182CC^OHN{k@~Oh3}d3p zEcH32nny#1`B^OYc)x)6Yqte1G z8zsj~Hv8nQ()gKd`^l+N;|%8iVgJ5CGh23-8#4&^b(bp~*SgEsGuXTj=X5w*2n+Z- zf-4VFGxL}qoW{QVuuJf*)A-?08|^du*9%#@Q!&Xhr#f`?@Ljh1A04$!I6CS<_m;9@gRrAY_!*``wirLcT<`c!YJD4+dvs+ZJ%2J3Ztn)K{+_VDRA z>CjZR?Q~D6&vaIIx|g)|1Lj|AlkC%&qc$>p4)o&hfn>7W479 z(x8b<|H<9$XH3SbLkNX>kNM|G%=*cM{-Z&{T(QDGSFw05*UQS}FypRSe_oEmd=hmV zG47baheJa!$6lDg+I-r*Z>LGT3P$2Y@DQB{268;9RxZEJPXw<|r2O*h{6O%>i94r# z8Y)WLC$a}W+bE5GlePQ&9qGjh?9k^gNT`p39T&8p~#% z>nf#>Wh>9cOApLoubj)55^|Y%zMr&e1{-*ONIU0fs$l$qmtCV->G^6YXbc1FJG3vyOrs`njk%YD|$>EvwpQn`Xq;S`g(#?nZxFPy<2k0 ztb6@f>0udIEZbB+Us`_)3%IabpF5rc`jv&%T=-OP#Uj9C?qOjseG{jj#F2lpS?xD- z!{%D>uDArJ%Kt2Ck2c~$WIg6-**nMlr*DY1gmgI^?f>{WmPyh?hDs)j%(p6%3zmm;Or>FkF~-KF0~u#P_r$e#N;SnOHFvVCD8bx`cR&^R`x z^D(5x?J|*uMjWen%nOI%uN71m`xEQs_Oktx}OTIi#nmmlHyc{JB9LAo%oFaWNjGeuF-)Ik9AR{*P z6kjt_jr@o5ni=_+<31yYIj$MmHH=kW87KWQlwG(IC4DlKNk67Yccrl*KiZ`ZL)j5v zFAQaW{n)*4Y#LYF)OtH`+*chE$F=H!zVScJmAVY!YHy^nb3esfyS+#;#Gg(zhai=k zbV$ZGvKR>03?aJN^8$esfw+=>l@I=dKUaqRHP#$|AP4L@Se&>o-M|9a;kO^BeWysk zuKS#T!*3`3r7kSjXX1R*uG>dMOo6vI{hsM0TmExTsdp-mW>6|S{&T#wD{*m;f{^lg zv>}z~=0Be&kkUwidxW~V4|DH@$8g50xdnVsO*SQ8~3Z%nmp=CoOWGz0(9?eSO9t}%jPyQNj9kc~p+@pA;d>$Q5A-dVfvr|k2T6&~Y*||r@{%rE- zt7pM!*L_c*wMWN^*vg~y-+D;%2eQK7T+&MeSij$w^nG$*%f9gl$9;WcDaW^Rv_h+-O7D?rgvyZPnAg##Wnf+(5U+2ersV}~!I>|MDMqwBYn#X*9 z5_|BkUebtO?AgD1Sl&(|_wW9bw#E z9BiP?zPf=KZbV8)l333hJ*>z2kwEJ|@SFwH&=$|NwUL#>6cX$-9J);#1fi*m0e`R+ z{%R|HWh=Z#b39|>&NDZjk|KAnr$dmLJ#qHYWv7~+t_PCv{CZa5=_j=v#`b&04tZ(~ zopStuvkCK~NYWWanF@QW&@?R?YJN2){)N0QbJV=_f;H<$ihHCE!`9yr|C9o{JO2>H zGHG$Q-HRo0yqNfV1x3Il+wn6`yY4%}cws#WL*q~Z3BCBUYn-#8jc7LSZ;Op_jeCO; zrzr>Vkd-?3Q_fC)VuHBDiI2OZ`c4LiyFKOqR!5tDdSt3nxkR<&f!RdozStsU3y*T4GmQdle_+Jl^CTvvU09=hI#%bj+;*uPHBXKMhWz-Cr(}tIr|v!^P_7<7IuioK z^&y?gy)~fjLCexdlsiuah)L~dETedzRgTL8%%87vwha`!O1oD%y9bK5N=lq_VW7B9 zGRHat+KI7gJstvY+ToH1aMTcv&)*|S)Dn)v4wJ{c3>dbW{V1(h(|G@-;)RjUx$VRu zx$i6`iJACAB!A+(Q}+g;Xcw~%52!wTat{Rz-C&Q9!6BkEW+fOr314+WSEE0lPB^;F zl|=V%e>pF=6C=#IP=UZvFM7vLnGW4cc4tJ8*eCT$8T7%^2KxJ@wSI`Rhz39ek2wb# zW#wr%=U88dOpJy6K&^Wxr!`0%A$=9$tPT<<3|kd})`P!thGWD$8XPk4MMQ&f9gdlB zu1Lp1kC^#V(y6QWSoduDzDpqbJ4p zRzsn|pCny=q6CtUo}d`MTZ9aIu3xm5N}p?-^%5aj($%@vD0Y@Sq0TprVz~5WsPmXn zOp%@m-C#lqu5}^#cKGWydOAG|^!_z`w|^UCICQ`8eBW4kk_Kq}Jx_=3d!Cc2NAe_3 z!t%kx+6fEVv#j9v)P_y}P5;X%itHGy+xuS)yy-Kru9*SYvvhHuHK9!1yEy+eiMNQO zog;$9D1h68#U8<56sw=z7@yBp4!U&5ik+K+#boI}WzG}9;)ss3k?qMWuklN07~wHy z+jVQpoLxGIsnJu*a14kS{1GcJ2*w8E%4>qLKv>X@Siv=<%vsVw>>r^6SEg~_g^-2w zUAk=xwGYU2FE4afcMy~Nl!8)Tqj%}%A%PGIS1s;nyKXkH;*RAt(JtLsBz4HJ>v9)5 zyLJ>iS+haGZ=?@<5+;~wXr9BLefTq)KRfVeJbw=7&!SM`{DG%mu%&!0k^6=pFx=l9?K@z787Wyd2K2*#z0KD?0dPkeY0;YYl< z@<Njzx;_X@IO|*V#xw)ndTJtmE-5mEV z!Cv9Gwgh|9lokl!UFm;?FR<1B z0Yo4HT=ZXmxJ$|HBZ;&Oq5)nmPw%YhODBD)s=f$SpVUm>W12pzOV)`Mv2dhZ{7dg0<7+X4+OWDx(H0j%DRTbtCpJMJ0;8pk& ze)*`DIFTf9kwOzOwp18G*-$u!6w0c;U}96u*B5*BN$!7|9BcrUF4C7&2c)^e%OSEN z@|SiP@MqwB-u)K;yJd!P4>})+6xZT^Yss^2SWLsdAXIBG$fn}kG(2&Qiic>hQOiH4 z!S^)yng%y$aG3@RH8??osTzz`p;h>GwQ7KSfk&@04cFjn{F_HX7^^`)4IW>i@~zU~ zP!0aRT&3^P;3^IF(cn4$t*9W(=Md9iszzv}ze<5?p9be^u#W~`UaIEjYw%VL4$@#x z4R+9A!xELhMuQtPSgOH02*SRQp%J=i@YhmR@G}j*tHEzH#oS9gp3&$P8Z6b|Tn$dp z;BXE0(O@SH3V>u^Xwd5VD-E{yGH90G)vDrU4X)GRCmI~4wT*lYU!cLSG`LKInmXad zm1+UktrWR^FJ0JJu4Y7QZQ_WAXKMm`G~8dqy@IPWx))!%O4Tz;b8M^zz4=!){*YB# zV>+i1I%tOP(G)zU!P_*0$2I(l1{*c#ugSO9;3*BpYp}ltGcgea=W1}Q28U}fUV|Mp_)m#y z;Gza=HTaeWpVZ(g4$=PZ(+K$*9IC-?8Vu54J^eTv*FFty*5D%=oUg&@8XT&@XblEx z@TY~Uo;rfm{;}{u;dUpz7f{4Ia_p+ZudKgQXgrpur3c_R(Od25%Ip`mSj3GYuXo()xduM%bvq zG7a9R!ATnIrNIyl{{DcfxL$)b8sznahv|DEgE;b{L=L_5qV_6Y7VNwpD^4-IXi)J2 zqO*Q&lAM)sqW%uQDXMxBqs4UUU7`?82rgPf*WIFug@>rHVS@@QohmHKQK8U5b)!C5 zb4OO`8#_6#$BA77Yn$-hvGL+};_m!zVuKX3b*Rd=E>(qFG`LlRRnFNx#IT-48rq<- zXQZj@;Tn8VqgM^x{aO!kkz_3ztuhpRtGW|?QH6pAE9+If>T4BNYSGEi@cQ#AeS=2d ztik$oD!t*X3Iz>T)v0)a3avO{Q5|R<7m8~YZoQU|{Wq_+SRISt)i|eRedGg=CLvsu zLwwlD3(0qi0&P|E0BIMy1uzUoH?JWidSKZ;$MKaaS*}cV%W+E0h%c5#8qpa>d zw#@cvq1yL)Co9Pn|1be0P7@QEJoxdP= zcJroB%ByFHLNy>=Uge}L8g%c4!xbSFc(O+D@{v!q(|vptf_mrLzFN4On>;awuNLO=%7xR65WUvunt}|qd}+3t0DmD+*T?!0#DYw^zspGULtbI z8;8d8xiUr!qja8EOZq z13?|ZwJ{;is-9w7XX-%F)E56;293jOVE2BJv|&gTY-QEf|+lP7b| z`hlWfaAh;jt(51?94v-715(5=vD6up0+B|LF*uX*f)vruY|8N|Pe!^)EX-M%B8ImC z|JoR{vl_f@w%jT@Pxlnf0rCb>$RCaWt<(8!ikRfwnj)H<(SyWrvDUdGRSa|H4HEst zROjSDB0hJ{aV{Msb`uMo&kcg-`Oaei&WcpgWY`FMskx%y#+^v6hOJHZ(^Z4Y9q%Y| z5GidP+HwsdldarK<>qMad3X3=+ihXS`%oaLhQpHHA z=2qwMRA{P&^KMlW=9+M^z&U$}7)cez8&V!%19!;D|)%M6>hC5HTpC z5G}F!Wdg4H6+9G$4R8(_Dn4|7l|dBx6yH@^9Do0O;?9UtyN8z+pPVcdfJQ%nQ)#6_ zQJg63hjJBPGKX71m#oEz(dTaZk5l7Qh=%4%zZMhvBq!fFcODIXF==AQbT`N}J@v_& zfWMTLm6?fjLrYpNNJ1lM8Gst!WTK@yAA|~126~uJMNX!(EKLlFAVqntaI?aMzD_E*dVz28T~j_J&U8%9Q#xsydvzV zcD^=394W#1pGJtCQkG#J;ATH*Cmcs<-;Ec9bwmSiYrG0XqAhc#XNY~J`iai@8F(K8 z_*jN`n|aP862pX1++HX{*q^~Y$0=rF{H<_CWQvhFyo^wsH%Jif$K6!sVt=$wL3kQB z)xN5wHH})Z#^YQLtzD^njoE7Xs4r7Fsf=oQoo{7|BY6yJHJ&1k2Qdg7DNbll@>{h$ z(C!&24mA?3Orw=WiO&5a#S~}lNKr3=x}y3#N%W{ZQnk*WGD)$J`%MAuG0N_`JVQ}6XC=WW^ISbyT( zljVFXTkLD?93*ru)(f5Iv=ur}@)tS})Crv%I@O2QhE|IPf59*?L@=BQXlPsHUl4=g z&2+J&pr6!1*fI^vK6J#*So#Df1PTdrB7~TM;ez?sb`1gbZL4)fetBf3Qgg`^-LmDB z%}^s7Je}49xb6icU)MNiTg0e-RY32#4b2=YIBxey2SGo_#A#20o=9}C6A#CoJ1kP8#Dm0uR z?ZHR=ZK&>Ebd(C#4cVU-HGLfo7YLfpXNLYz2M2x;h0Z_1-Uh(ZmtN6d^AM%VQgvTFJW!}s(R zhHOn125#&p^sVSG^ei19bSoSv#9n+rh%KIv)^@)TJMcasR=h_D;X2x2TjpG!gO)*o zNJJoL>ZF2Jpf}j04X?@e6F})qcSa-Wx-sX5jM>vaC?p;=3Hz4@k zp)D^EzvQF%v;}t|mxbb+Xo#(kCrpS)Xe)#j#|dErV}&p=T4={xinD#L*wb3}sUUnp zx8_K5JOWE#GE7~p3el#k>@ziL!JYhs;0cDJpn`VElwW~-Y8}a1gl_{Ri6RZI(CB%e ztMZkAxW)=^B0-E708tj9Q=J;sTtHGNQp*__NP}sC+FG#pJ&Tv#xGM_O=r!R^-54=z zGPRC^Oi@?`b)=$FvyF2qjvxI38MqF7rQ)hue*x!N!b055R}f7!=hI`fPL-;uuB=zZ zcU<7wghl^R@d%?N_)m~{SWMu}xY*3hq(CA1d#rG9dqds==L@(%A=!;O#_i?N@a`%> zW6HRMVGFmH$%i}d7%NWUy~Ld%3aCO}?k!pzG8$C;Fd$v&zp8j}guf6mCrAjvDCNIE zowuqXPcf{5EJ0cZX8fk{|E|dcQWeeyBp%VG7i^RKorA}TnSAN8(Ybt_H~_oJ9pkXw zc=S$n`=Sb?okKMpkz{xYfIovK1Lpm$s(cTSF7Y1~CmFgv1f}X!O4UX@rgl zl!Ua?~IMTuzFroGT^WZOLdWVKyybdJvxqi8j|AW!TTOHH{*D{^HlmRIO|RHQ(i zYVRYRb6=j=sdsc+{5}$Q(Hv@lusp|KXh{=%xDy}14d|5DP89IXlTgb=P_!o8Id;6* zhtK4+bE4~HkZSfvK)QOMCz2ZhMt{LLCqOU`)CsHgAsq+ZZHuHpfM=c zr(}NzQ5c1Bsz)3_=kN*m)XXbe5&}nXXT~{~P7qhMTir=DR5KR`AOl6SfBjrUVV*N~ zqS!6!NQ9cnUC|z1fy8;Db6kpOZYN{srV8;R3mmQZIk!v{O`YO3)m~|80$afh?Z+pI zo!S)5;{rD(iXl>i&e>s-*x#Cbzt3XDVl=&h*aTq9=e=q{qjR*XdwjI2Qmmf)VU)Yu zN4vEQi!$(q-|M6O4q9}7Yyg_k5?4bGw2k+1KCHiR$sleSfcQ3JECj6qjD^i;kAPML zT1_)r6==1f?P*4<1+5yi=w^j0aQ2=oM*i;${r`QT|GzKvh5vn_|GzKv|M!Ld{|_(p z$MN~@|GgLbyF1JjAC-2$eVdqT#s_QjcNL5EZNw_)g~j;fw#q4%irrNZ?#wLZq}iq7 zIFNRf;?rEt*U=fYL<~VXW{KDrVEhuE1JN*N)e_MJ(m|C3na<9kg<>>q9#z@hGnb0n z@RHCZZJOvP=vO1eq74fhhw<^!J=|DE;v4_nd#<)AOEd zo9BJE1zdKsJWO5eL!m~#T|hT*#*1rE3@>(|x<-qyFBms|<^^%qWMfw2#d)X&U(T8F zl^L|M>2C;|yQzyOP#!OyK?VHXFq(gNMJSy^CF)|I#g8B6ehEd!>-5q(umJN zW!Fpbo!cZTKQQ^bC86&x`XI4Z(}|E#!Q!V2ldxTg;d`zMM#t zP)}N7ku)#iaIhdAL$mP(*#9CIh&;UAz{c>Vk(uYqIlS^K*t3<4SP}oE;O>`bgRhL2 z6*Is0GN*>RV&gqT!+rR2)@u>TmUCdd^5E<`HjEehP!nG4M;q`3_;J#^wUh++Ze+vM zDRi+vx>mxQb?DcyT>>Ym4e|Dj8`GugtV&voS_< zA>e{{xG?dGILE%v`uH%c*}|z*8@LIDd9I}3Wfa3Jjy$)O>ufd8pCV?_Ht!jbfE~V% zZYT8L#TCm6rkw{4kza5P8-tfnJzg>3Euy~!UJ;(swKTq*vT>*@6F4PtH&JM9z~%lP zi@jrEe~&R2&}_W&w6jECb@(D2pJxS9r7-;LuorcS8@^=^sEePXdb|y19AQifUlCoo zG1s5eQ`AL%7vF5b8w;}(b+C|txB9sS@!{```4RQu3vkm>?+B$}_4nSnk3#F1=i6W} z(v!3RKls60H!UK!4rd+r+Q;ElWa;^A3Ot*U8nr>|gzw?IKlvV7Kl>h9Cw&jCQ@)3H zGC1+|)0`4^Kx{$7c;#+~P%y~_dz$C}qBGvg3e;8(aDVdx6RHf8tK1I{@fuR+%J~>= zl$#BrcD#5Ab>W$$Wlj!quj84xWlsIVZ7mOhR1G zGdqf2C(`ix1uit|iow1_X?z~8y67EhW}TUKq$6s>hTptSCn59o+#RTl$P}GbTSN4k zH>!p+(LsD=W~S_{=6Cjhx;PgV@pmJ#fE%67`Gb3%y7&SLzQXmS8d8VEhyL<98ihMA z^9-PkWn=V*SYs@d;J6Zdk5{&~?%e`&0ATSS<5gNEcG^c73q6}`G?PbRHR;n zjsJS}B%E{ItIvl|j&L35{o#xooc|ChTKopRxycInC=A~6d>9V<-i-2GM`}}o3vYWH zP_i~iniip6gdP8RduGFwA??|~!v}5-FhY)RgQtQFDd3BmbAN?Mtd@lmjF08w(_H|k z5sS%J#$k%eGN_gI;yIMYi+`g%dLNhy1xzpAhUbw^f&RnOqH*L^)XSNf#e%}L9&{Ej z9zi910WwpNPnm!lq%9{ri{Mv5rY^djV}hqyM`)&wGRf000n|Mm;x^xZ2*(hVC0I8I zA3!=~acDL0qlFimU>fO-#2cuU`Y2pGfjz?~A;0tGuKWqSh}3=v_Dm#wqD?Pcc0ZlR zC*a#iZN%Nax8W7fTc(7$r^3&1E!98 z6>LVTx4@k5#hXY^(owiL;`w^m8sYqFW34PKdr%*b^co&UO?ahxW2Sg*l+nHF`wlpW zbY9QGQPpWLPObL5cz!BvsA~x1SESAr;oOJWCiOZCa{z5&p#Y~(0Y)9;<9kb(htv%naA8#k@q}^ + + + + + + +Vulkan Memory Allocator: Choosing memory type + + + + + + + + + +

+
+ + + + + + +
+
Vulkan Memory Allocator +
+
+
+ + + + + + + + +
+
+ + +
+ +
+ + +
+
+
+
Choosing memory type
+
+
+

Physical devices in Vulkan support various combinations of memory heaps and types. Help with choosing correct and optimal memory type for your specific resource is one of the key features of this library. You can use it by filling appropriate members of VmaAllocationCreateInfo structure, as described below. You can also combine multiple methods.

+
    +
  1. If you just want to find memory type index that meets your requirements, you can use function vmaFindMemoryTypeIndex().
    2. +
  2. If you want to allocate a region of device memory without association with any specific image or buffer, you can use function vmaAllocateMemory(). Usage of this function is not recommended and usually not needed.
    3. +
  3. If you already have a buffer or an image created, you want to allocate memory for it and then you will bind it yourself, you can use function vmaAllocateMemoryForBuffer(), vmaAllocateMemoryForImage().
    4. +
  4. If you want to create a buffer or an image, allocate memory for it and bind them together, all in one call, you can use function vmaCreateBuffer(), vmaCreateImage(). This is the recommended way to use this library.
    5. +
+

When using 3. or 4., the library internally queries Vulkan for memory types supported for that buffer or image (function vkGetBufferMemoryRequirements()) and uses only one of these types.

+

If no memory type can be found that meets all the requirements, these functions return VK_ERROR_FEATURE_NOT_PRESENT.

+

You can leave VmaAllocationCreateInfo structure completely filled with zeros. It means no requirements are specified for memory type. It is valid, although not very useful.

+

+Usage

+

The easiest way to specify memory requirements is to fill member VmaAllocationCreateInfo::usage using one of the values of enum VmaMemoryUsage. It defines high level, common usage types.

+

For example, if you want to create a uniform buffer that will be filled using transfer only once or infrequently and used for rendering every frame, you can do it using following code:

+
VkBufferCreateInfo bufferInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
bufferInfo.size = 65536;
bufferInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
VmaAllocationCreateInfo allocInfo = {};
allocInfo.usage = VMA_MEMORY_USAGE_GPU_ONLY;
VkBuffer buffer;
VmaAllocation allocation;
vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr);

+Required and preferred flags

+

You can specify more detailed requirements by filling members VmaAllocationCreateInfo::requiredFlags and VmaAllocationCreateInfo::preferredFlags with a combination of bits from enum VkMemoryPropertyFlags. For example, if you want to create a buffer that will be persistently mapped on host (so it must be HOST_VISIBLE) and preferably will also be HOST_COHERENT and HOST_CACHED, use following code:

+
VmaAllocationCreateInfo allocInfo = {};
allocInfo.requiredFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
allocInfo.preferredFlags = VK_MEMORY_PROPERTY_HOST_COHERENT_BIT | VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
allocInfo.flags = VMA_ALLOCATION_CREATE_MAPPED_BIT;
VkBuffer buffer;
VmaAllocation allocation;
vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr);

A memory type is chosen that has all the required flags and as many preferred flags set as possible.

+

If you use VmaAllocationCreateInfo::usage, it is just internally converted to a set of required and preferred flags.

+

+Explicit memory types

+

If you inspected memory types available on the physical device and you have a preference for memory types that you want to use, you can fill member VmaAllocationCreateInfo::memoryTypeBits. It is a bit mask, where each bit set means that a memory type with that index is allowed to be used for the allocation. Special value 0, just like UINT32_MAX, means there are no restrictions to memory type index.

+

Please note that this member is NOT just a memory type index. Still you can use it to choose just one, specific memory type. For example, if you already determined that your buffer should be created in memory type 2, use following code:

+
uint32_t memoryTypeIndex = 2;
VmaAllocationCreateInfo allocInfo = {};
allocInfo.memoryTypeBits = 1u << memoryTypeIndex;
VkBuffer buffer;
VmaAllocation allocation;
vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr);

+Custom memory pools

+

If you allocate from custom memory pool, all the ways of specifying memory requirements described above are not applicable and the aforementioned members of VmaAllocationCreateInfo structure are ignored. Memory type is selected explicitly when creating the pool and then used to make all the allocations from that pool. For further details, see Custom memory pools.

+
+ +
+Generated by   +doxygen + 1.8.13 +
+ + diff --git a/docs/html/functions.html b/docs/html/functions.html index bb64a0b..b79b1aa 100644 --- a/docs/html/functions.html +++ b/docs/html/functions.html @@ -139,6 +139,9 @@ $(function() { : VmaAllocationInfo , VmaStats +
  • memoryTypeBits +: VmaAllocationCreateInfo +
  • memoryTypeIndex : VmaPoolCreateInfo
    • diff --git a/docs/html/functions_vars.html b/docs/html/functions_vars.html index eb1456a..fc26998 100644 --- a/docs/html/functions_vars.html +++ b/docs/html/functions_vars.html @@ -139,6 +139,9 @@ $(function() { : VmaAllocationInfo , VmaStats +
  • memoryTypeBits +: VmaAllocationCreateInfo +
  • memoryTypeIndex : VmaPoolCreateInfo
    • diff --git a/docs/html/index.html b/docs/html/index.html index fda4013..fd509d2 100644 --- a/docs/html/index.html +++ b/docs/html/index.html @@ -70,6 +70,7 @@ License: MIT

    • User guide
      • Quick start
        • +
      • Choosing memory type
      • Memory mapping
      • Custom memory pools
      • Defragmentation
        • diff --git a/docs/html/search/all_2.js b/docs/html/search/all_2.js index bf1d77b..e5f807d 100644 --- a/docs/html/search/all_2.js +++ b/docs/html/search/all_2.js @@ -1,5 +1,6 @@ var searchData= [ + ['choosing_20memory_20type',['Choosing memory type',['../choosing_memory_type.html',1,'index']]], ['configuration',['Configuration',['../configuration.html',1,'index']]], ['custom_20memory_20pools',['Custom memory pools',['../custom_memory_pools.html',1,'index']]] ]; diff --git a/docs/html/search/all_6.js b/docs/html/search/all_6.js index 0140fe2..8e80e1a 100644 --- a/docs/html/search/all_6.js +++ b/docs/html/search/all_6.js @@ -6,6 +6,7 @@ var searchData= ['memory_20mapping',['Memory mapping',['../memory_mapping.html',1,'index']]], ['memoryheap',['memoryHeap',['../struct_vma_stats.html#a0e6611508c29a187f0fd14ff1a0329c0',1,'VmaStats']]], ['memorytype',['memoryType',['../struct_vma_stats.html#a13e3caf754be79352c42408756309331',1,'VmaStats::memoryType()'],['../struct_vma_allocation_info.html#a7f6b0aa58c135e488e6b40a388dad9d5',1,'VmaAllocationInfo::memoryType()']]], + ['memorytypebits',['memoryTypeBits',['../struct_vma_allocation_create_info.html#a3bf940c0271d85d6ba32a4d820075055',1,'VmaAllocationCreateInfo']]], ['memorytypeindex',['memoryTypeIndex',['../struct_vma_pool_create_info.html#a596fa76b685d3f1f688f84a709a5b319',1,'VmaPoolCreateInfo']]], ['minblockcount',['minBlockCount',['../struct_vma_pool_create_info.html#ad8006fb803185c0a699d30f3e9a865ae',1,'VmaPoolCreateInfo']]] ]; diff --git a/docs/html/search/pages_1.js b/docs/html/search/pages_1.js index bf1d77b..e5f807d 100644 --- a/docs/html/search/pages_1.js +++ b/docs/html/search/pages_1.js @@ -1,5 +1,6 @@ var searchData= [ + ['choosing_20memory_20type',['Choosing memory type',['../choosing_memory_type.html',1,'index']]], ['configuration',['Configuration',['../configuration.html',1,'index']]], ['custom_20memory_20pools',['Custom memory pools',['../custom_memory_pools.html',1,'index']]] ]; diff --git a/docs/html/search/variables_4.js b/docs/html/search/variables_4.js index 1002ad2..96a5bce 100644 --- a/docs/html/search/variables_4.js +++ b/docs/html/search/variables_4.js @@ -5,6 +5,7 @@ var searchData= ['maxbytestomove',['maxBytesToMove',['../struct_vma_defragmentation_info.html#acb311c940a777270e67e1b81c5ab6a1d',1,'VmaDefragmentationInfo']]], ['memoryheap',['memoryHeap',['../struct_vma_stats.html#a0e6611508c29a187f0fd14ff1a0329c0',1,'VmaStats']]], ['memorytype',['memoryType',['../struct_vma_stats.html#a13e3caf754be79352c42408756309331',1,'VmaStats::memoryType()'],['../struct_vma_allocation_info.html#a7f6b0aa58c135e488e6b40a388dad9d5',1,'VmaAllocationInfo::memoryType()']]], + ['memorytypebits',['memoryTypeBits',['../struct_vma_allocation_create_info.html#a3bf940c0271d85d6ba32a4d820075055',1,'VmaAllocationCreateInfo']]], ['memorytypeindex',['memoryTypeIndex',['../struct_vma_pool_create_info.html#a596fa76b685d3f1f688f84a709a5b319',1,'VmaPoolCreateInfo']]], ['minblockcount',['minBlockCount',['../struct_vma_pool_create_info.html#ad8006fb803185c0a699d30f3e9a865ae',1,'VmaPoolCreateInfo']]] ]; diff --git a/docs/html/struct_vma_allocation_create_info-members.html b/docs/html/struct_vma_allocation_create_info-members.html index 1b63856..d9dba01 100644 --- a/docs/html/struct_vma_allocation_create_info-members.html +++ b/docs/html/struct_vma_allocation_create_info-members.html @@ -66,11 +66,12 @@ $(function() {

        This is the complete list of members for VmaAllocationCreateInfo, including all inherited members.

        - - - - - + + + + + +
        flagsVmaAllocationCreateInfo
        poolVmaAllocationCreateInfo
        preferredFlagsVmaAllocationCreateInfo
        pUserDataVmaAllocationCreateInfo
        requiredFlagsVmaAllocationCreateInfo
        usageVmaAllocationCreateInfo
        memoryTypeBitsVmaAllocationCreateInfo
        poolVmaAllocationCreateInfo
        preferredFlagsVmaAllocationCreateInfo
        pUserDataVmaAllocationCreateInfo
        requiredFlagsVmaAllocationCreateInfo
        usageVmaAllocationCreateInfo
        diff --git a/docs/html/struct_vma_allocation_create_info.html b/docs/html/struct_vma_allocation_create_info.html index 3de0ffa..43f87f0 100644 --- a/docs/html/struct_vma_allocation_create_info.html +++ b/docs/html/struct_vma_allocation_create_info.html @@ -80,14 +80,17 @@ Public Attributes  Flags that must be set in a Memory Type chosen for an allocation. More...
          VkMemoryPropertyFlags preferredFlags - Flags that preferably should be set in a Memory Type chosen for an allocation. More...
        + Flags that preferably should be set in a memory type chosen for an allocation. More...
          -void * pUserData - Custom general-purpose pointer that will be stored in VmaAllocation, can be read as VmaAllocationInfo::pUserData and changed using vmaSetAllocationUserData(). More...
        -  +uint32_t memoryTypeBits + Bitmask containing one bit set for every memory type acceptable for this allocation. More...
        +  VmaPool pool  Pool that this allocation should be created in. More...
          +void * pUserData + Custom general-purpose pointer that will be stored in VmaAllocation, can be read as VmaAllocationInfo::pUserData and changed using vmaSetAllocationUserData(). More...
        + 

        Member Data Documentation

        @@ -104,6 +107,24 @@ Public Attributes

        Use VmaAllocationCreateFlagBits enum.

        + + + +

        ◆ memoryTypeBits

        + +
        +
        + + + + +
        uint32_t VmaAllocationCreateInfo::memoryTypeBits
        +
        + +

        Bitmask containing one bit set for every memory type acceptable for this allocation.

        +

        Value 0 is equivalent to UINT32_MAX - it means any memory type is accepted if it meets other requirements specified by this structure, with no further restrictions on memory type index.
        +If pool is not null, this member is ignored.

        +
        @@ -119,7 +140,7 @@ Public Attributes

        Pool that this allocation should be created in.

        -

        Leave VK_NULL_HANDLE to allocate from general memory.

        +

        Leave VK_NULL_HANDLE to allocate from default pool. If not null, members: usage, requiredFlags, preferredFlags, memoryTypeBits are ignored.

        @@ -135,9 +156,8 @@ Public Attributes
        -

        Flags that preferably should be set in a Memory Type chosen for an allocation.

        -

        Set to 0 if no additional flags are prefered and only requiredFlags should be used.
        -If not 0, it must be a superset or equal to requiredFlags.
        +

        Flags that preferably should be set in a memory type chosen for an allocation.

        +

        Set to 0 if no additional flags are prefered.
        If pool is not null, this member is ignored.

        @@ -155,6 +175,7 @@ If pool is not null, this member is ignored.

        Custom general-purpose pointer that will be stored in VmaAllocation, can be read as VmaAllocationInfo::pUserData and changed using vmaSetAllocationUserData().

        +

        If VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT is used, it must be either null or pointer to a null-terminated string. The string will be then copied to internal buffer, so it doesn't need to be valid after allocation call.

        @@ -171,7 +192,7 @@ If pool is not null, this member is ignored.

        Flags that must be set in a Memory Type chosen for an allocation.

        -

        Leave 0 if you specify requirement via usage.
        +

        Leave 0 if you specify memory requirements in other way.
        If pool is not null, this member is ignored.

        @@ -189,7 +210,7 @@ If pool is not null, this member is ignored.

        Intended usage of memory.

        -

        Leave VMA_MEMORY_USAGE_UNKNOWN if you specify requiredFlags. You can also use both.
        +

        You can leave VMA_MEMORY_USAGE_UNKNOWN if you specify memory requirements in other way.
        If pool is not null, this member is ignored.

        diff --git a/docs/html/vk__mem__alloc_8h.html b/docs/html/vk__mem__alloc_8h.html index ef2e6af..d9324a8 100644 --- a/docs/html/vk__mem__alloc_8h.html +++ b/docs/html/vk__mem__alloc_8h.html @@ -712,15 +712,16 @@ Functions - - - - diff --git a/docs/html/vk__mem__alloc_8h_source.html b/docs/html/vk__mem__alloc_8h_source.html index 82a48d2..73752fd 100644 --- a/docs/html/vk__mem__alloc_8h_source.html +++ b/docs/html/vk__mem__alloc_8h_source.html @@ -62,154 +62,155 @@ $(function() {
        vk_mem_alloc.h
        -Go to the documentation of this file.
        1 //
        2 // Copyright (c) 2017 Advanced Micro Devices, Inc. All rights reserved.
        3 //
        4 // Permission is hereby granted, free of charge, to any person obtaining a copy
        5 // of this software and associated documentation files (the "Software"), to deal
        6 // in the Software without restriction, including without limitation the rights
        7 // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        8 // copies of the Software, and to permit persons to whom the Software is
        9 // furnished to do so, subject to the following conditions:
        10 //
        11 // The above copyright notice and this permission notice shall be included in
        12 // all copies or substantial portions of the Software.
        13 //
        14 // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        15 // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        16 // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        17 // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        18 // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        19 // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
        20 // THE SOFTWARE.
        21 //
        22 
        23 #ifndef AMD_VULKAN_MEMORY_ALLOCATOR_H
        24 #define AMD_VULKAN_MEMORY_ALLOCATOR_H
        25 
        26 #ifdef __cplusplus
        27 extern "C" {
        28 #endif
        29 
        594 #include <vulkan/vulkan.h>
        595 
        596 VK_DEFINE_HANDLE(VmaAllocator)
        597 
        598 typedef void (VKAPI_PTR *PFN_vmaAllocateDeviceMemoryFunction)(
        600  VmaAllocator allocator,
        601  uint32_t memoryType,
        602  VkDeviceMemory memory,
        603  VkDeviceSize size);
        605 typedef void (VKAPI_PTR *PFN_vmaFreeDeviceMemoryFunction)(
        606  VmaAllocator allocator,
        607  uint32_t memoryType,
        608  VkDeviceMemory memory,
        609  VkDeviceSize size);
        610 
        618 typedef struct VmaDeviceMemoryCallbacks {
        620  PFN_vmaAllocateDeviceMemoryFunction pfnAllocate;
        622  PFN_vmaFreeDeviceMemoryFunction pfnFree;
        623 } VmaDeviceMemoryCallbacks;
        624 
        626 typedef enum VmaAllocatorCreateFlagBits {
        631  VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT = 0x00000001,
        659  VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT = 0x00000002,
        660 
        661  VMA_ALLOCATOR_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
        662 } VmaAllocatorCreateFlagBits;
        663 typedef VkFlags VmaAllocatorCreateFlags;
        664 
        669 typedef struct VmaVulkanFunctions {
        670  PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties;
        671  PFN_vkGetPhysicalDeviceMemoryProperties vkGetPhysicalDeviceMemoryProperties;
        672  PFN_vkAllocateMemory vkAllocateMemory;
        673  PFN_vkFreeMemory vkFreeMemory;
        674  PFN_vkMapMemory vkMapMemory;
        675  PFN_vkUnmapMemory vkUnmapMemory;
        676  PFN_vkBindBufferMemory vkBindBufferMemory;
        677  PFN_vkBindImageMemory vkBindImageMemory;
        678  PFN_vkGetBufferMemoryRequirements vkGetBufferMemoryRequirements;
        679  PFN_vkGetImageMemoryRequirements vkGetImageMemoryRequirements;
        680  PFN_vkCreateBuffer vkCreateBuffer;
        681  PFN_vkDestroyBuffer vkDestroyBuffer;
        682  PFN_vkCreateImage vkCreateImage;
        683  PFN_vkDestroyImage vkDestroyImage;
        684  PFN_vkGetBufferMemoryRequirements2KHR vkGetBufferMemoryRequirements2KHR;
        685  PFN_vkGetImageMemoryRequirements2KHR vkGetImageMemoryRequirements2KHR;
        686 } VmaVulkanFunctions;
        687 
        689 typedef struct VmaAllocatorCreateInfo
        690 {
        692  VmaAllocatorCreateFlags flags;
        694 
        695  VkPhysicalDevice physicalDevice;
        697 
        698  VkDevice device;
        700 
        701  VkDeviceSize preferredLargeHeapBlockSize;
        703 
        704  VkDeviceSize preferredSmallHeapBlockSize;
        706 
        707  const VkAllocationCallbacks* pAllocationCallbacks;
        709 
        710  const VmaDeviceMemoryCallbacks* pDeviceMemoryCallbacks;
        724  uint32_t frameInUseCount;
        748  const VkDeviceSize* pHeapSizeLimit;
        760  const VmaVulkanFunctions* pVulkanFunctions;
        761 } VmaAllocatorCreateInfo;
        762 
        764 VkResult vmaCreateAllocator(
        765  const VmaAllocatorCreateInfo* pCreateInfo,
        766  VmaAllocator* pAllocator);
        767 
        769 void vmaDestroyAllocator(
        770  VmaAllocator allocator);
        771 
        776 void vmaGetPhysicalDeviceProperties(
        777  VmaAllocator allocator,
        778  const VkPhysicalDeviceProperties** ppPhysicalDeviceProperties);
        779 
        784 void vmaGetMemoryProperties(
        785  VmaAllocator allocator,
        786  const VkPhysicalDeviceMemoryProperties** ppPhysicalDeviceMemoryProperties);
        787 
        794 void vmaGetMemoryTypeProperties(
        795  VmaAllocator allocator,
        796  uint32_t memoryTypeIndex,
        797  VkMemoryPropertyFlags* pFlags);
        798 
        807 void vmaSetCurrentFrameIndex(
        808  VmaAllocator allocator,
        809  uint32_t frameIndex);
        810 
        813 typedef struct VmaStatInfo
        814 {
        816  uint32_t blockCount;
        818  uint32_t allocationCount;
        820  uint32_t unusedRangeCount;
        822  VkDeviceSize usedBytes;
        824  VkDeviceSize unusedBytes;
        825  VkDeviceSize allocationSizeMin, allocationSizeAvg, allocationSizeMax;
        826  VkDeviceSize unusedRangeSizeMin, unusedRangeSizeAvg, unusedRangeSizeMax;
        827 } VmaStatInfo;
        828 
        830 typedef struct VmaStats
        831 {
        832  VmaStatInfo memoryType[VK_MAX_MEMORY_TYPES];
        833  VmaStatInfo memoryHeap[VK_MAX_MEMORY_HEAPS];
        834  VmaStatInfo total;
        835 } VmaStats;
        836 
        838 void vmaCalculateStats(
        839  VmaAllocator allocator,
        840  VmaStats* pStats);
        841 
        842 #define VMA_STATS_STRING_ENABLED 1
        843 
        844 #if VMA_STATS_STRING_ENABLED
        845 
        847 
        849 void vmaBuildStatsString(
        850  VmaAllocator allocator,
        851  char** ppStatsString,
        852  VkBool32 detailedMap);
        853 
        854 void vmaFreeStatsString(
        855  VmaAllocator allocator,
        856  char* pStatsString);
        857 
        858 #endif // #if VMA_STATS_STRING_ENABLED
        859 
        860 VK_DEFINE_HANDLE(VmaPool)
        861 
        862 typedef enum VmaMemoryUsage
        863 {
        865  VMA_MEMORY_USAGE_UNKNOWN = 0,
        867  VMA_MEMORY_USAGE_GPU_ONLY = 1,
        869 
        870  VMA_MEMORY_USAGE_CPU_ONLY = 2,
        872 
        873  VMA_MEMORY_USAGE_CPU_TO_GPU = 3,
        875 
        876  VMA_MEMORY_USAGE_GPU_TO_CPU = 4,
        877  VMA_MEMORY_USAGE_MAX_ENUM = 0x7FFFFFFF
        878 } VmaMemoryUsage;
        879 
        881 typedef enum VmaAllocationCreateFlagBits {
        893  VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT = 0x00000001,
        894 
        904  VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT = 0x00000002,
        917  VMA_ALLOCATION_CREATE_MAPPED_BIT = 0x00000004,
        930  VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT = 0x00000008,
        937  VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT = 0x00000010,
        943  VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT = 0x00000020,
        944 
        945  VMA_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
        946 } VmaAllocationCreateFlagBits;
        947 typedef VkFlags VmaAllocationCreateFlags;
        948 
        949 typedef struct VmaAllocationCreateInfo
        950 {
        952  VmaAllocationCreateFlags flags;
        958  VmaMemoryUsage usage;
        963  VkMemoryPropertyFlags requiredFlags;
        969  VkMemoryPropertyFlags preferredFlags;
        971  void* pUserData;
        976  VmaPool pool;
        977 } VmaAllocationCreateInfo;
        978 
        993 VkResult vmaFindMemoryTypeIndex(
        994  VmaAllocator allocator,
        995  uint32_t memoryTypeBits,
        996  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        997  uint32_t* pMemoryTypeIndex);
        998 
        1000 typedef enum VmaPoolCreateFlagBits {
        1018  VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT = 0x00000002,
        1019 
        1020  VMA_POOL_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
        1021 } VmaPoolCreateFlagBits;
        1022 typedef VkFlags VmaPoolCreateFlags;
        1023 
        1026 typedef struct VmaPoolCreateInfo {
        1029  uint32_t memoryTypeIndex;
        1032  VmaPoolCreateFlags flags;
        1037  VkDeviceSize blockSize;
        1042  size_t minBlockCount;
        1050  size_t maxBlockCount;
        1064  uint32_t frameInUseCount;
        1065 } VmaPoolCreateInfo;
        1066 
        1069 typedef struct VmaPoolStats {
        1072  VkDeviceSize size;
        1075  VkDeviceSize unusedSize;
        1078  size_t allocationCount;
        1081  size_t unusedRangeCount;
        1088  VkDeviceSize unusedRangeSizeMax;
        1089 } VmaPoolStats;
        1090 
        1097 VkResult vmaCreatePool(
        1098  VmaAllocator allocator,
        1099  const VmaPoolCreateInfo* pCreateInfo,
        1100  VmaPool* pPool);
        1101 
        1104 void vmaDestroyPool(
        1105  VmaAllocator allocator,
        1106  VmaPool pool);
        1107 
        1114 void vmaGetPoolStats(
        1115  VmaAllocator allocator,
        1116  VmaPool pool,
        1117  VmaPoolStats* pPoolStats);
        1118 
        1125 void vmaMakePoolAllocationsLost(
        1126  VmaAllocator allocator,
        1127  VmaPool pool,
        1128  size_t* pLostAllocationCount);
        1129 
        1130 VK_DEFINE_HANDLE(VmaAllocation)
        1131 
        1132 
        1134 typedef struct VmaAllocationInfo {
        1139  uint32_t memoryType;
        1148  VkDeviceMemory deviceMemory;
        1153  VkDeviceSize offset;
        1158  VkDeviceSize size;
        1167  void* pMappedData;
        1172  void* pUserData;
        1173 } VmaAllocationInfo;
        1174 
        1185 VkResult vmaAllocateMemory(
        1186  VmaAllocator allocator,
        1187  const VkMemoryRequirements* pVkMemoryRequirements,
        1188  const VmaAllocationCreateInfo* pCreateInfo,
        1189  VmaAllocation* pAllocation,
        1190  VmaAllocationInfo* pAllocationInfo);
        1191 
        1198 VkResult vmaAllocateMemoryForBuffer(
        1199  VmaAllocator allocator,
        1200  VkBuffer buffer,
        1201  const VmaAllocationCreateInfo* pCreateInfo,
        1202  VmaAllocation* pAllocation,
        1203  VmaAllocationInfo* pAllocationInfo);
        1204 
        1206 VkResult vmaAllocateMemoryForImage(
        1207  VmaAllocator allocator,
        1208  VkImage image,
        1209  const VmaAllocationCreateInfo* pCreateInfo,
        1210  VmaAllocation* pAllocation,
        1211  VmaAllocationInfo* pAllocationInfo);
        1212 
        1214 void vmaFreeMemory(
        1215  VmaAllocator allocator,
        1216  VmaAllocation allocation);
        1217 
        1219 void vmaGetAllocationInfo(
        1220  VmaAllocator allocator,
        1221  VmaAllocation allocation,
        1222  VmaAllocationInfo* pAllocationInfo);
        1223 
        1237 void vmaSetAllocationUserData(
        1238  VmaAllocator allocator,
        1239  VmaAllocation allocation,
        1240  void* pUserData);
        1241 
        1252 void vmaCreateLostAllocation(
        1253  VmaAllocator allocator,
        1254  VmaAllocation* pAllocation);
        1255 
        1290 VkResult vmaMapMemory(
        1291  VmaAllocator allocator,
        1292  VmaAllocation allocation,
        1293  void** ppData);
        1294 
        1299 void vmaUnmapMemory(
        1300  VmaAllocator allocator,
        1301  VmaAllocation allocation);
        1302 
        1304 typedef struct VmaDefragmentationInfo {
        1309  VkDeviceSize maxBytesToMove;
        1314  uint32_t maxAllocationsToMove;
        1315 } VmaDefragmentationInfo;
        1316 
        1318 typedef struct VmaDefragmentationStats {
        1320  VkDeviceSize bytesMoved;
        1322  VkDeviceSize bytesFreed;
        1324  uint32_t allocationsMoved;
        1326  uint32_t deviceMemoryBlocksFreed;
        1327 } VmaDefragmentationStats;
        1328 
        1405 VkResult vmaDefragment(
        1406  VmaAllocator allocator,
        1407  VmaAllocation* pAllocations,
        1408  size_t allocationCount,
        1409  VkBool32* pAllocationsChanged,
        1410  const VmaDefragmentationInfo *pDefragmentationInfo,
        1411  VmaDefragmentationStats* pDefragmentationStats);
        1412 
        1439 VkResult vmaCreateBuffer(
        1440  VmaAllocator allocator,
        1441  const VkBufferCreateInfo* pBufferCreateInfo,
        1442  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        1443  VkBuffer* pBuffer,
        1444  VmaAllocation* pAllocation,
        1445  VmaAllocationInfo* pAllocationInfo);
        1446 
        1458 void vmaDestroyBuffer(
        1459  VmaAllocator allocator,
        1460  VkBuffer buffer,
        1461  VmaAllocation allocation);
        1462 
        1464 VkResult vmaCreateImage(
        1465  VmaAllocator allocator,
        1466  const VkImageCreateInfo* pImageCreateInfo,
        1467  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        1468  VkImage* pImage,
        1469  VmaAllocation* pAllocation,
        1470  VmaAllocationInfo* pAllocationInfo);
        1471 
        1483 void vmaDestroyImage(
        1484  VmaAllocator allocator,
        1485  VkImage image,
        1486  VmaAllocation allocation);
        1487 
        1488 #ifdef __cplusplus
        1489 }
        1490 #endif
        1491 
        1492 #endif // AMD_VULKAN_MEMORY_ALLOCATOR_H
        1493 
        1494 // For Visual Studio IntelliSense.
        1495 #ifdef __INTELLISENSE__
        1496 #define VMA_IMPLEMENTATION
        1497 #endif
        1498 
        1499 #ifdef VMA_IMPLEMENTATION
        1500 #undef VMA_IMPLEMENTATION
        1501 
        1502 #include <cstdint>
        1503 #include <cstdlib>
        1504 #include <cstring>
        1505 
        1506 /*******************************************************************************
        1507 CONFIGURATION SECTION
        1508 
        1509 Define some of these macros before each #include of this header or change them
        1510 here if you need other then default behavior depending on your environment.
        1511 */
        1512 
        1513 /*
        1514 Define this macro to 1 to make the library fetch pointers to Vulkan functions
        1515 internally, like:
        1516 
        1517  vulkanFunctions.vkAllocateMemory = &vkAllocateMemory;
        1518 
        1519 Define to 0 if you are going to provide you own pointers to Vulkan functions via
        1520 VmaAllocatorCreateInfo::pVulkanFunctions.
        1521 */
        1522 #ifndef VMA_STATIC_VULKAN_FUNCTIONS
        1523 #define VMA_STATIC_VULKAN_FUNCTIONS 1
        1524 #endif
        1525 
        1526 // Define this macro to 1 to make the library use STL containers instead of its own implementation.
        1527 //#define VMA_USE_STL_CONTAINERS 1
        1528 
        1529 /* Set this macro to 1 to make the library including and using STL containers:
        1530 std::pair, std::vector, std::list, std::unordered_map.
        1531 
        1532 Set it to 0 or undefined to make the library using its own implementation of
        1533 the containers.
        1534 */
        1535 #if VMA_USE_STL_CONTAINERS
        1536  #define VMA_USE_STL_VECTOR 1
        1537  #define VMA_USE_STL_UNORDERED_MAP 1
        1538  #define VMA_USE_STL_LIST 1
        1539 #endif
        1540 
        1541 #if VMA_USE_STL_VECTOR
        1542  #include <vector>
        1543 #endif
        1544 
        1545 #if VMA_USE_STL_UNORDERED_MAP
        1546  #include <unordered_map>
        1547 #endif
        1548 
        1549 #if VMA_USE_STL_LIST
        1550  #include <list>
        1551 #endif
        1552 
        1553 /*
        1554 Following headers are used in this CONFIGURATION section only, so feel free to
        1555 remove them if not needed.
        1556 */
        1557 #include <cassert> // for assert
        1558 #include <algorithm> // for min, max
        1559 #include <mutex> // for std::mutex
        1560 #include <atomic> // for std::atomic
        1561 
        1562 #if !defined(_WIN32)
        1563  #include <malloc.h> // for aligned_alloc()
        1564 #endif
        1565 
        1566 // Normal assert to check for programmer's errors, especially in Debug configuration.
        1567 #ifndef VMA_ASSERT
        1568  #ifdef _DEBUG
        1569  #define VMA_ASSERT(expr) assert(expr)
        1570  #else
        1571  #define VMA_ASSERT(expr)
        1572  #endif
        1573 #endif
        1574 
        1575 // Assert that will be called very often, like inside data structures e.g. operator[].
        1576 // Making it non-empty can make program slow.
        1577 #ifndef VMA_HEAVY_ASSERT
        1578  #ifdef _DEBUG
        1579  #define VMA_HEAVY_ASSERT(expr) //VMA_ASSERT(expr)
        1580  #else
        1581  #define VMA_HEAVY_ASSERT(expr)
        1582  #endif
        1583 #endif
        1584 
        1585 #ifndef VMA_NULL
        1586  // Value used as null pointer. Define it to e.g.: nullptr, NULL, 0, (void*)0.
        1587  #define VMA_NULL nullptr
        1588 #endif
        1589 
        1590 #ifndef VMA_ALIGN_OF
        1591  #define VMA_ALIGN_OF(type) (__alignof(type))
        1592 #endif
        1593 
        1594 #ifndef VMA_SYSTEM_ALIGNED_MALLOC
        1595  #if defined(_WIN32)
        1596  #define VMA_SYSTEM_ALIGNED_MALLOC(size, alignment) (_aligned_malloc((size), (alignment)))
        1597  #else
        1598  #define VMA_SYSTEM_ALIGNED_MALLOC(size, alignment) (aligned_alloc((alignment), (size) ))
        1599  #endif
        1600 #endif
        1601 
        1602 #ifndef VMA_SYSTEM_FREE
        1603  #if defined(_WIN32)
        1604  #define VMA_SYSTEM_FREE(ptr) _aligned_free(ptr)
        1605  #else
        1606  #define VMA_SYSTEM_FREE(ptr) free(ptr)
        1607  #endif
        1608 #endif
        1609 
        1610 #ifndef VMA_MIN
        1611  #define VMA_MIN(v1, v2) (std::min((v1), (v2)))
        1612 #endif
        1613 
        1614 #ifndef VMA_MAX
        1615  #define VMA_MAX(v1, v2) (std::max((v1), (v2)))
        1616 #endif
        1617 
        1618 #ifndef VMA_SWAP
        1619  #define VMA_SWAP(v1, v2) std::swap((v1), (v2))
        1620 #endif
        1621 
        1622 #ifndef VMA_SORT
        1623  #define VMA_SORT(beg, end, cmp) std::sort(beg, end, cmp)
        1624 #endif
        1625 
        1626 #ifndef VMA_DEBUG_LOG
        1627  #define VMA_DEBUG_LOG(format, ...)
        1628  /*
        1629  #define VMA_DEBUG_LOG(format, ...) do { \
        1630  printf(format, __VA_ARGS__); \
        1631  printf("\n"); \
        1632  } while(false)
        1633  */
        1634 #endif
        1635 
        1636 // Define this macro to 1 to enable functions: vmaBuildStatsString, vmaFreeStatsString.
        1637 #if VMA_STATS_STRING_ENABLED
        1638  static inline void VmaUint32ToStr(char* outStr, size_t strLen, uint32_t num)
        1639  {
        1640  snprintf(outStr, strLen, "%u", static_cast<unsigned int>(num));
        1641  }
        1642  static inline void VmaUint64ToStr(char* outStr, size_t strLen, uint64_t num)
        1643  {
        1644  snprintf(outStr, strLen, "%llu", static_cast<unsigned long long>(num));
        1645  }
        1646  static inline void VmaPtrToStr(char* outStr, size_t strLen, const void* ptr)
        1647  {
        1648  snprintf(outStr, strLen, "%p", ptr);
        1649  }
        1650 #endif
        1651 
        1652 #ifndef VMA_MUTEX
        1653  class VmaMutex
        1654  {
        1655  public:
        1656  VmaMutex() { }
        1657  ~VmaMutex() { }
        1658  void Lock() { m_Mutex.lock(); }
        1659  void Unlock() { m_Mutex.unlock(); }
        1660  private:
        1661  std::mutex m_Mutex;
        1662  };
        1663  #define VMA_MUTEX VmaMutex
        1664 #endif
        1665 
        1666 /*
        1667 If providing your own implementation, you need to implement a subset of std::atomic:
        1668 
        1669 - Constructor(uint32_t desired)
        1670 - uint32_t load() const
        1671 - void store(uint32_t desired)
        1672 - bool compare_exchange_weak(uint32_t& expected, uint32_t desired)
        1673 */
        1674 #ifndef VMA_ATOMIC_UINT32
        1675  #define VMA_ATOMIC_UINT32 std::atomic<uint32_t>
        1676 #endif
        1677 
        1678 #ifndef VMA_BEST_FIT
        1679 
        1691  #define VMA_BEST_FIT (1)
        1692 #endif
        1693 
        1694 #ifndef VMA_DEBUG_ALWAYS_DEDICATED_MEMORY
        1695 
        1699  #define VMA_DEBUG_ALWAYS_DEDICATED_MEMORY (0)
        1700 #endif
        1701 
        1702 #ifndef VMA_DEBUG_ALIGNMENT
        1703 
        1707  #define VMA_DEBUG_ALIGNMENT (1)
        1708 #endif
        1709 
        1710 #ifndef VMA_DEBUG_MARGIN
        1711 
        1715  #define VMA_DEBUG_MARGIN (0)
        1716 #endif
        1717 
        1718 #ifndef VMA_DEBUG_GLOBAL_MUTEX
        1719 
        1723  #define VMA_DEBUG_GLOBAL_MUTEX (0)
        1724 #endif
        1725 
        1726 #ifndef VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY
        1727 
        1731  #define VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY (1)
        1732 #endif
        1733 
        1734 #ifndef VMA_SMALL_HEAP_MAX_SIZE
        1735  #define VMA_SMALL_HEAP_MAX_SIZE (512 * 1024 * 1024)
        1737 #endif
        1738 
        1739 #ifndef VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE
        1740  #define VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE (256 * 1024 * 1024)
        1742 #endif
        1743 
        1744 #ifndef VMA_DEFAULT_SMALL_HEAP_BLOCK_SIZE
        1745  #define VMA_DEFAULT_SMALL_HEAP_BLOCK_SIZE (64 * 1024 * 1024)
        1747 #endif
        1748 
        1749 static const uint32_t VMA_FRAME_INDEX_LOST = UINT32_MAX;
        1750 
        1751 /*******************************************************************************
        1752 END OF CONFIGURATION
        1753 */
        1754 
        1755 static VkAllocationCallbacks VmaEmptyAllocationCallbacks = {
        1756  VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL };
        1757 
        1758 // Returns number of bits set to 1 in (v).
        1759 static inline uint32_t CountBitsSet(uint32_t v)
        1760 {
        1761  uint32_t c = v - ((v >> 1) & 0x55555555);
        1762  c = ((c >> 2) & 0x33333333) + (c & 0x33333333);
        1763  c = ((c >> 4) + c) & 0x0F0F0F0F;
        1764  c = ((c >> 8) + c) & 0x00FF00FF;
        1765  c = ((c >> 16) + c) & 0x0000FFFF;
        1766  return c;
        1767 }
        1768 
        1769 // Aligns given value up to nearest multiply of align value. For example: VmaAlignUp(11, 8) = 16.
        1770 // Use types like uint32_t, uint64_t as T.
        1771 template <typename T>
        1772 static inline T VmaAlignUp(T val, T align)
        1773 {
        1774  return (val + align - 1) / align * align;
        1775 }
        1776 
        1777 // Division with mathematical rounding to nearest number.
        1778 template <typename T>
        1779 inline T VmaRoundDiv(T x, T y)
        1780 {
        1781  return (x + (y / (T)2)) / y;
        1782 }
        1783 
        1784 #ifndef VMA_SORT
        1785 
        1786 template<typename Iterator, typename Compare>
        1787 Iterator VmaQuickSortPartition(Iterator beg, Iterator end, Compare cmp)
        1788 {
        1789  Iterator centerValue = end; --centerValue;
        1790  Iterator insertIndex = beg;
        1791  for(Iterator memTypeIndex = beg; memTypeIndex < centerValue; ++memTypeIndex)
        1792  {
        1793  if(cmp(*memTypeIndex, *centerValue))
        1794  {
        1795  if(insertIndex != memTypeIndex)
        1796  {
        1797  VMA_SWAP(*memTypeIndex, *insertIndex);
        1798  }
        1799  ++insertIndex;
        1800  }
        1801  }
        1802  if(insertIndex != centerValue)
        1803  {
        1804  VMA_SWAP(*insertIndex, *centerValue);
        1805  }
        1806  return insertIndex;
        1807 }
        1808 
        1809 template<typename Iterator, typename Compare>
        1810 void VmaQuickSort(Iterator beg, Iterator end, Compare cmp)
        1811 {
        1812  if(beg < end)
        1813  {
        1814  Iterator it = VmaQuickSortPartition<Iterator, Compare>(beg, end, cmp);
        1815  VmaQuickSort<Iterator, Compare>(beg, it, cmp);
        1816  VmaQuickSort<Iterator, Compare>(it + 1, end, cmp);
        1817  }
        1818 }
        1819 
        1820 #define VMA_SORT(beg, end, cmp) VmaQuickSort(beg, end, cmp)
        1821 
        1822 #endif // #ifndef VMA_SORT
        1823 
        1824 /*
        1825 Returns true if two memory blocks occupy overlapping pages.
        1826 ResourceA must be in less memory offset than ResourceB.
        1827 
        1828 Algorithm is based on "Vulkan 1.0.39 - A Specification (with all registered Vulkan extensions)"
        1829 chapter 11.6 "Resource Memory Association", paragraph "Buffer-Image Granularity".
        1830 */
        1831 static inline bool VmaBlocksOnSamePage(
        1832  VkDeviceSize resourceAOffset,
        1833  VkDeviceSize resourceASize,
        1834  VkDeviceSize resourceBOffset,
        1835  VkDeviceSize pageSize)
        1836 {
        1837  VMA_ASSERT(resourceAOffset + resourceASize <= resourceBOffset && resourceASize > 0 && pageSize > 0);
        1838  VkDeviceSize resourceAEnd = resourceAOffset + resourceASize - 1;
        1839  VkDeviceSize resourceAEndPage = resourceAEnd & ~(pageSize - 1);
        1840  VkDeviceSize resourceBStart = resourceBOffset;
        1841  VkDeviceSize resourceBStartPage = resourceBStart & ~(pageSize - 1);
        1842  return resourceAEndPage == resourceBStartPage;
        1843 }
        1844 
        1845 enum VmaSuballocationType
        1846 {
        1847  VMA_SUBALLOCATION_TYPE_FREE = 0,
        1848  VMA_SUBALLOCATION_TYPE_UNKNOWN = 1,
        1849  VMA_SUBALLOCATION_TYPE_BUFFER = 2,
        1850  VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN = 3,
        1851  VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR = 4,
        1852  VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL = 5,
        1853  VMA_SUBALLOCATION_TYPE_MAX_ENUM = 0x7FFFFFFF
        1854 };
        1855 
        1856 /*
        1857 Returns true if given suballocation types could conflict and must respect
        1858 VkPhysicalDeviceLimits::bufferImageGranularity. They conflict if one is buffer
        1859 or linear image and another one is optimal image. If type is unknown, behave
        1860 conservatively.
        1861 */
        1862 static inline bool VmaIsBufferImageGranularityConflict(
        1863  VmaSuballocationType suballocType1,
        1864  VmaSuballocationType suballocType2)
        1865 {
        1866  if(suballocType1 > suballocType2)
        1867  {
        1868  VMA_SWAP(suballocType1, suballocType2);
        1869  }
        1870 
        1871  switch(suballocType1)
        1872  {
        1873  case VMA_SUBALLOCATION_TYPE_FREE:
        1874  return false;
        1875  case VMA_SUBALLOCATION_TYPE_UNKNOWN:
        1876  return true;
        1877  case VMA_SUBALLOCATION_TYPE_BUFFER:
        1878  return
        1879  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
        1880  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
        1881  case VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN:
        1882  return
        1883  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
        1884  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR ||
        1885  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
        1886  case VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR:
        1887  return
        1888  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
        1889  case VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL:
        1890  return false;
        1891  default:
        1892  VMA_ASSERT(0);
        1893  return true;
        1894  }
        1895 }
        1896 
        1897 // Helper RAII class to lock a mutex in constructor and unlock it in destructor (at the end of scope).
        1898 struct VmaMutexLock
        1899 {
        1900 public:
        1901  VmaMutexLock(VMA_MUTEX& mutex, bool useMutex) :
        1902  m_pMutex(useMutex ? &mutex : VMA_NULL)
        1903  {
        1904  if(m_pMutex)
        1905  {
        1906  m_pMutex->Lock();
        1907  }
        1908  }
        1909 
        1910  ~VmaMutexLock()
        1911  {
        1912  if(m_pMutex)
        1913  {
        1914  m_pMutex->Unlock();
        1915  }
        1916  }
        1917 
        1918 private:
        1919  VMA_MUTEX* m_pMutex;
        1920 };
        1921 
        1922 #if VMA_DEBUG_GLOBAL_MUTEX
        1923  static VMA_MUTEX gDebugGlobalMutex;
        1924  #define VMA_DEBUG_GLOBAL_MUTEX_LOCK VmaMutexLock debugGlobalMutexLock(gDebugGlobalMutex, true);
        1925 #else
        1926  #define VMA_DEBUG_GLOBAL_MUTEX_LOCK
        1927 #endif
        1928 
        1929 // Minimum size of a free suballocation to register it in the free suballocation collection.
        1930 static const VkDeviceSize VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER = 16;
        1931 
        1932 /*
        1933 Performs binary search and returns iterator to first element that is greater or
        1934 equal to (key), according to comparison (cmp).
        1935 
        1936 Cmp should return true if first argument is less than second argument.
        1937 
        1938 Returned value is the found element, if present in the collection or place where
        1939 new element with value (key) should be inserted.
        1940 */
        1941 template <typename IterT, typename KeyT, typename CmpT>
        1942 static IterT VmaBinaryFindFirstNotLess(IterT beg, IterT end, const KeyT &key, CmpT cmp)
        1943 {
        1944  size_t down = 0, up = (end - beg);
        1945  while(down < up)
        1946  {
        1947  const size_t mid = (down + up) / 2;
        1948  if(cmp(*(beg+mid), key))
        1949  {
        1950  down = mid + 1;
        1951  }
        1952  else
        1953  {
        1954  up = mid;
        1955  }
        1956  }
        1957  return beg + down;
        1958 }
        1959 
        1961 // Memory allocation
        1962 
        1963 static void* VmaMalloc(const VkAllocationCallbacks* pAllocationCallbacks, size_t size, size_t alignment)
        1964 {
        1965  if((pAllocationCallbacks != VMA_NULL) &&
        1966  (pAllocationCallbacks->pfnAllocation != VMA_NULL))
        1967  {
        1968  return (*pAllocationCallbacks->pfnAllocation)(
        1969  pAllocationCallbacks->pUserData,
        1970  size,
        1971  alignment,
        1972  VK_SYSTEM_ALLOCATION_SCOPE_OBJECT);
        1973  }
        1974  else
        1975  {
        1976  return VMA_SYSTEM_ALIGNED_MALLOC(size, alignment);
        1977  }
        1978 }
        1979 
        1980 static void VmaFree(const VkAllocationCallbacks* pAllocationCallbacks, void* ptr)
        1981 {
        1982  if((pAllocationCallbacks != VMA_NULL) &&
        1983  (pAllocationCallbacks->pfnFree != VMA_NULL))
        1984  {
        1985  (*pAllocationCallbacks->pfnFree)(pAllocationCallbacks->pUserData, ptr);
        1986  }
        1987  else
        1988  {
        1989  VMA_SYSTEM_FREE(ptr);
        1990  }
        1991 }
        1992 
        1993 template<typename T>
        1994 static T* VmaAllocate(const VkAllocationCallbacks* pAllocationCallbacks)
        1995 {
        1996  return (T*)VmaMalloc(pAllocationCallbacks, sizeof(T), VMA_ALIGN_OF(T));
        1997 }
        1998 
        1999 template<typename T>
        2000 static T* VmaAllocateArray(const VkAllocationCallbacks* pAllocationCallbacks, size_t count)
        2001 {
        2002  return (T*)VmaMalloc(pAllocationCallbacks, sizeof(T) * count, VMA_ALIGN_OF(T));
        2003 }
        2004 
        2005 #define vma_new(allocator, type) new(VmaAllocate<type>(allocator))(type)
        2006 
        2007 #define vma_new_array(allocator, type, count) new(VmaAllocateArray<type>((allocator), (count)))(type)
        2008 
        2009 template<typename T>
        2010 static void vma_delete(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr)
        2011 {
        2012  ptr->~T();
        2013  VmaFree(pAllocationCallbacks, ptr);
        2014 }
        2015 
        2016 template<typename T>
        2017 static void vma_delete_array(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr, size_t count)
        2018 {
        2019  if(ptr != VMA_NULL)
        2020  {
        2021  for(size_t i = count; i--; )
        2022  {
        2023  ptr[i].~T();
        2024  }
        2025  VmaFree(pAllocationCallbacks, ptr);
        2026  }
        2027 }
        2028 
        2029 // STL-compatible allocator.
        2030 template<typename T>
        2031 class VmaStlAllocator
        2032 {
        2033 public:
        2034  const VkAllocationCallbacks* const m_pCallbacks;
        2035  typedef T value_type;
        2036 
        2037  VmaStlAllocator(const VkAllocationCallbacks* pCallbacks) : m_pCallbacks(pCallbacks) { }
        2038  template<typename U> VmaStlAllocator(const VmaStlAllocator<U>& src) : m_pCallbacks(src.m_pCallbacks) { }
        2039 
        2040  T* allocate(size_t n) { return VmaAllocateArray<T>(m_pCallbacks, n); }
        2041  void deallocate(T* p, size_t n) { VmaFree(m_pCallbacks, p); }
        2042 
        2043  template<typename U>
        2044  bool operator==(const VmaStlAllocator<U>& rhs) const
        2045  {
        2046  return m_pCallbacks == rhs.m_pCallbacks;
        2047  }
        2048  template<typename U>
        2049  bool operator!=(const VmaStlAllocator<U>& rhs) const
        2050  {
        2051  return m_pCallbacks != rhs.m_pCallbacks;
        2052  }
        2053 
        2054  VmaStlAllocator& operator=(const VmaStlAllocator& x) = delete;
        2055 };
        2056 
        2057 #if VMA_USE_STL_VECTOR
        2058 
        2059 #define VmaVector std::vector
        2060 
        2061 template<typename T, typename allocatorT>
        2062 static void VmaVectorInsert(std::vector<T, allocatorT>& vec, size_t index, const T& item)
        2063 {
        2064  vec.insert(vec.begin() + index, item);
        2065 }
        2066 
        2067 template<typename T, typename allocatorT>
        2068 static void VmaVectorRemove(std::vector<T, allocatorT>& vec, size_t index)
        2069 {
        2070  vec.erase(vec.begin() + index);
        2071 }
        2072 
        2073 #else // #if VMA_USE_STL_VECTOR
        2074 
        2075 /* Class with interface compatible with subset of std::vector.
        2076 T must be POD because constructors and destructors are not called and memcpy is
        2077 used for these objects. */
        2078 template<typename T, typename AllocatorT>
        2079 class VmaVector
        2080 {
        2081 public:
        2082  typedef T value_type;
        2083 
        2084  VmaVector(const AllocatorT& allocator) :
        2085  m_Allocator(allocator),
        2086  m_pArray(VMA_NULL),
        2087  m_Count(0),
        2088  m_Capacity(0)
        2089  {
        2090  }
        2091 
        2092  VmaVector(size_t count, const AllocatorT& allocator) :
        2093  m_Allocator(allocator),
        2094  m_pArray(count ? (T*)VmaAllocateArray<T>(allocator.m_pCallbacks, count) : VMA_NULL),
        2095  m_Count(count),
        2096  m_Capacity(count)
        2097  {
        2098  }
        2099 
        2100  VmaVector(const VmaVector<T, AllocatorT>& src) :
        2101  m_Allocator(src.m_Allocator),
        2102  m_pArray(src.m_Count ? (T*)VmaAllocateArray<T>(src.m_Allocator.m_pCallbacks, src.m_Count) : VMA_NULL),
        2103  m_Count(src.m_Count),
        2104  m_Capacity(src.m_Count)
        2105  {
        2106  if(m_Count != 0)
        2107  {
        2108  memcpy(m_pArray, src.m_pArray, m_Count * sizeof(T));
        2109  }
        2110  }
        2111 
        2112  ~VmaVector()
        2113  {
        2114  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
        2115  }
        2116 
        2117  VmaVector& operator=(const VmaVector<T, AllocatorT>& rhs)
        2118  {
        2119  if(&rhs != this)
        2120  {
        2121  resize(rhs.m_Count);
        2122  if(m_Count != 0)
        2123  {
        2124  memcpy(m_pArray, rhs.m_pArray, m_Count * sizeof(T));
        2125  }
        2126  }
        2127  return *this;
        2128  }
        2129 
        2130  bool empty() const { return m_Count == 0; }
        2131  size_t size() const { return m_Count; }
        2132  T* data() { return m_pArray; }
        2133  const T* data() const { return m_pArray; }
        2134 
        2135  T& operator[](size_t index)
        2136  {
        2137  VMA_HEAVY_ASSERT(index < m_Count);
        2138  return m_pArray[index];
        2139  }
        2140  const T& operator[](size_t index) const
        2141  {
        2142  VMA_HEAVY_ASSERT(index < m_Count);
        2143  return m_pArray[index];
        2144  }
        2145 
        2146  T& front()
        2147  {
        2148  VMA_HEAVY_ASSERT(m_Count > 0);
        2149  return m_pArray[0];
        2150  }
        2151  const T& front() const
        2152  {
        2153  VMA_HEAVY_ASSERT(m_Count > 0);
        2154  return m_pArray[0];
        2155  }
        2156  T& back()
        2157  {
        2158  VMA_HEAVY_ASSERT(m_Count > 0);
        2159  return m_pArray[m_Count - 1];
        2160  }
        2161  const T& back() const
        2162  {
        2163  VMA_HEAVY_ASSERT(m_Count > 0);
        2164  return m_pArray[m_Count - 1];
        2165  }
        2166 
        2167  void reserve(size_t newCapacity, bool freeMemory = false)
        2168  {
        2169  newCapacity = VMA_MAX(newCapacity, m_Count);
        2170 
        2171  if((newCapacity < m_Capacity) && !freeMemory)
        2172  {
        2173  newCapacity = m_Capacity;
        2174  }
        2175 
        2176  if(newCapacity != m_Capacity)
        2177  {
        2178  T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator, newCapacity) : VMA_NULL;
        2179  if(m_Count != 0)
        2180  {
        2181  memcpy(newArray, m_pArray, m_Count * sizeof(T));
        2182  }
        2183  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
        2184  m_Capacity = newCapacity;
        2185  m_pArray = newArray;
        2186  }
        2187  }
        2188 
        2189  void resize(size_t newCount, bool freeMemory = false)
        2190  {
        2191  size_t newCapacity = m_Capacity;
        2192  if(newCount > m_Capacity)
        2193  {
        2194  newCapacity = VMA_MAX(newCount, VMA_MAX(m_Capacity * 3 / 2, (size_t)8));
        2195  }
        2196  else if(freeMemory)
        2197  {
        2198  newCapacity = newCount;
        2199  }
        2200 
        2201  if(newCapacity != m_Capacity)
        2202  {
        2203  T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator.m_pCallbacks, newCapacity) : VMA_NULL;
        2204  const size_t elementsToCopy = VMA_MIN(m_Count, newCount);
        2205  if(elementsToCopy != 0)
        2206  {
        2207  memcpy(newArray, m_pArray, elementsToCopy * sizeof(T));
        2208  }
        2209  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
        2210  m_Capacity = newCapacity;
        2211  m_pArray = newArray;
        2212  }
        2213 
        2214  m_Count = newCount;
        2215  }
        2216 
        2217  void clear(bool freeMemory = false)
        2218  {
        2219  resize(0, freeMemory);
        2220  }
        2221 
        2222  void insert(size_t index, const T& src)
        2223  {
        2224  VMA_HEAVY_ASSERT(index <= m_Count);
        2225  const size_t oldCount = size();
        2226  resize(oldCount + 1);
        2227  if(index < oldCount)
        2228  {
        2229  memmove(m_pArray + (index + 1), m_pArray + index, (oldCount - index) * sizeof(T));
        2230  }
        2231  m_pArray[index] = src;
        2232  }
        2233 
        2234  void remove(size_t index)
        2235  {
        2236  VMA_HEAVY_ASSERT(index < m_Count);
        2237  const size_t oldCount = size();
        2238  if(index < oldCount - 1)
        2239  {
        2240  memmove(m_pArray + index, m_pArray + (index + 1), (oldCount - index - 1) * sizeof(T));
        2241  }
        2242  resize(oldCount - 1);
        2243  }
        2244 
        2245  void push_back(const T& src)
        2246  {
        2247  const size_t newIndex = size();
        2248  resize(newIndex + 1);
        2249  m_pArray[newIndex] = src;
        2250  }
        2251 
        2252  void pop_back()
        2253  {
        2254  VMA_HEAVY_ASSERT(m_Count > 0);
        2255  resize(size() - 1);
        2256  }
        2257 
        2258  void push_front(const T& src)
        2259  {
        2260  insert(0, src);
        2261  }
        2262 
        2263  void pop_front()
        2264  {
        2265  VMA_HEAVY_ASSERT(m_Count > 0);
        2266  remove(0);
        2267  }
        2268 
        2269  typedef T* iterator;
        2270 
        2271  iterator begin() { return m_pArray; }
        2272  iterator end() { return m_pArray + m_Count; }
        2273 
        2274 private:
        2275  AllocatorT m_Allocator;
        2276  T* m_pArray;
        2277  size_t m_Count;
        2278  size_t m_Capacity;
        2279 };
        2280 
        2281 template<typename T, typename allocatorT>
        2282 static void VmaVectorInsert(VmaVector<T, allocatorT>& vec, size_t index, const T& item)
        2283 {
        2284  vec.insert(index, item);
        2285 }
        2286 
        2287 template<typename T, typename allocatorT>
        2288 static void VmaVectorRemove(VmaVector<T, allocatorT>& vec, size_t index)
        2289 {
        2290  vec.remove(index);
        2291 }
        2292 
        2293 #endif // #if VMA_USE_STL_VECTOR
        2294 
        2295 template<typename CmpLess, typename VectorT>
        2296 size_t VmaVectorInsertSorted(VectorT& vector, const typename VectorT::value_type& value)
        2297 {
        2298  const size_t indexToInsert = VmaBinaryFindFirstNotLess(
        2299  vector.data(),
        2300  vector.data() + vector.size(),
        2301  value,
        2302  CmpLess()) - vector.data();
        2303  VmaVectorInsert(vector, indexToInsert, value);
        2304  return indexToInsert;
        2305 }
        2306 
        2307 template<typename CmpLess, typename VectorT>
        2308 bool VmaVectorRemoveSorted(VectorT& vector, const typename VectorT::value_type& value)
        2309 {
        2310  CmpLess comparator;
        2311  typename VectorT::iterator it = VmaBinaryFindFirstNotLess(
        2312  vector.begin(),
        2313  vector.end(),
        2314  value,
        2315  comparator);
        2316  if((it != vector.end()) && !comparator(*it, value) && !comparator(value, *it))
        2317  {
        2318  size_t indexToRemove = it - vector.begin();
        2319  VmaVectorRemove(vector, indexToRemove);
        2320  return true;
        2321  }
        2322  return false;
        2323 }
        2324 
        2325 template<typename CmpLess, typename VectorT>
        2326 size_t VmaVectorFindSorted(const VectorT& vector, const typename VectorT::value_type& value)
        2327 {
        2328  CmpLess comparator;
        2329  typename VectorT::iterator it = VmaBinaryFindFirstNotLess(
        2330  vector.data(),
        2331  vector.data() + vector.size(),
        2332  value,
        2333  comparator);
        2334  if(it != vector.size() && !comparator(*it, value) && !comparator(value, *it))
        2335  {
        2336  return it - vector.begin();
        2337  }
        2338  else
        2339  {
        2340  return vector.size();
        2341  }
        2342 }
        2343 
        2345 // class VmaPoolAllocator
        2346 
        2347 /*
        2348 Allocator for objects of type T using a list of arrays (pools) to speed up
        2349 allocation. Number of elements that can be allocated is not bounded because
        2350 allocator can create multiple blocks.
        2351 */
        2352 template<typename T>
        2353 class VmaPoolAllocator
        2354 {
        2355 public:
        2356  VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, size_t itemsPerBlock);
        2357  ~VmaPoolAllocator();
        2358  void Clear();
        2359  T* Alloc();
        2360  void Free(T* ptr);
        2361 
        2362 private:
        2363  union Item
        2364  {
        2365  uint32_t NextFreeIndex;
        2366  T Value;
        2367  };
        2368 
        2369  struct ItemBlock
        2370  {
        2371  Item* pItems;
        2372  uint32_t FirstFreeIndex;
        2373  };
        2374 
        2375  const VkAllocationCallbacks* m_pAllocationCallbacks;
        2376  size_t m_ItemsPerBlock;
        2377  VmaVector< ItemBlock, VmaStlAllocator<ItemBlock> > m_ItemBlocks;
        2378 
        2379  ItemBlock& CreateNewBlock();
        2380 };
        2381 
        2382 template<typename T>
        2383 VmaPoolAllocator<T>::VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, size_t itemsPerBlock) :
        2384  m_pAllocationCallbacks(pAllocationCallbacks),
        2385  m_ItemsPerBlock(itemsPerBlock),
        2386  m_ItemBlocks(VmaStlAllocator<ItemBlock>(pAllocationCallbacks))
        2387 {
        2388  VMA_ASSERT(itemsPerBlock > 0);
        2389 }
        2390 
        2391 template<typename T>
        2392 VmaPoolAllocator<T>::~VmaPoolAllocator()
        2393 {
        2394  Clear();
        2395 }
        2396 
        2397 template<typename T>
        2398 void VmaPoolAllocator<T>::Clear()
        2399 {
        2400  for(size_t i = m_ItemBlocks.size(); i--; )
        2401  vma_delete_array(m_pAllocationCallbacks, m_ItemBlocks[i].pItems, m_ItemsPerBlock);
        2402  m_ItemBlocks.clear();
        2403 }
        2404 
        2405 template<typename T>
        2406 T* VmaPoolAllocator<T>::Alloc()
        2407 {
        2408  for(size_t i = m_ItemBlocks.size(); i--; )
        2409  {
        2410  ItemBlock& block = m_ItemBlocks[i];
        2411  // This block has some free items: Use first one.
        2412  if(block.FirstFreeIndex != UINT32_MAX)
        2413  {
        2414  Item* const pItem = &block.pItems[block.FirstFreeIndex];
        2415  block.FirstFreeIndex = pItem->NextFreeIndex;
        2416  return &pItem->Value;
        2417  }
        2418  }
        2419 
        2420  // No block has free item: Create new one and use it.
        2421  ItemBlock& newBlock = CreateNewBlock();
        2422  Item* const pItem = &newBlock.pItems[0];
        2423  newBlock.FirstFreeIndex = pItem->NextFreeIndex;
        2424  return &pItem->Value;
        2425 }
        2426 
        2427 template<typename T>
        2428 void VmaPoolAllocator<T>::Free(T* ptr)
        2429 {
        2430  // Search all memory blocks to find ptr.
        2431  for(size_t i = 0; i < m_ItemBlocks.size(); ++i)
        2432  {
        2433  ItemBlock& block = m_ItemBlocks[i];
        2434 
        2435  // Casting to union.
        2436  Item* pItemPtr;
        2437  memcpy(&pItemPtr, &ptr, sizeof(pItemPtr));
        2438 
        2439  // Check if pItemPtr is in address range of this block.
        2440  if((pItemPtr >= block.pItems) && (pItemPtr < block.pItems + m_ItemsPerBlock))
        2441  {
        2442  const uint32_t index = static_cast<uint32_t>(pItemPtr - block.pItems);
        2443  pItemPtr->NextFreeIndex = block.FirstFreeIndex;
        2444  block.FirstFreeIndex = index;
        2445  return;
        2446  }
        2447  }
        2448  VMA_ASSERT(0 && "Pointer doesn't belong to this memory pool.");
        2449 }
        2450 
        2451 template<typename T>
        2452 typename VmaPoolAllocator<T>::ItemBlock& VmaPoolAllocator<T>::CreateNewBlock()
        2453 {
        2454  ItemBlock newBlock = {
        2455  vma_new_array(m_pAllocationCallbacks, Item, m_ItemsPerBlock), 0 };
        2456 
        2457  m_ItemBlocks.push_back(newBlock);
        2458 
        2459  // Setup singly-linked list of all free items in this block.
        2460  for(uint32_t i = 0; i < m_ItemsPerBlock - 1; ++i)
        2461  newBlock.pItems[i].NextFreeIndex = i + 1;
        2462  newBlock.pItems[m_ItemsPerBlock - 1].NextFreeIndex = UINT32_MAX;
        2463  return m_ItemBlocks.back();
        2464 }
        2465 
        2467 // class VmaRawList, VmaList
        2468 
        2469 #if VMA_USE_STL_LIST
        2470 
        2471 #define VmaList std::list
        2472 
        2473 #else // #if VMA_USE_STL_LIST
        2474 
        2475 template<typename T>
        2476 struct VmaListItem
        2477 {
        2478  VmaListItem* pPrev;
        2479  VmaListItem* pNext;
        2480  T Value;
        2481 };
        2482 
        2483 // Doubly linked list.
        2484 template<typename T>
        2485 class VmaRawList
        2486 {
        2487 public:
        2488  typedef VmaListItem<T> ItemType;
        2489 
        2490  VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks);
        2491  ~VmaRawList();
        2492  void Clear();
        2493 
        2494  size_t GetCount() const { return m_Count; }
        2495  bool IsEmpty() const { return m_Count == 0; }
        2496 
        2497  ItemType* Front() { return m_pFront; }
        2498  const ItemType* Front() const { return m_pFront; }
        2499  ItemType* Back() { return m_pBack; }
        2500  const ItemType* Back() const { return m_pBack; }
        2501 
        2502  ItemType* PushBack();
        2503  ItemType* PushFront();
        2504  ItemType* PushBack(const T& value);
        2505  ItemType* PushFront(const T& value);
        2506  void PopBack();
        2507  void PopFront();
        2508 
        2509  // Item can be null - it means PushBack.
        2510  ItemType* InsertBefore(ItemType* pItem);
        2511  // Item can be null - it means PushFront.
        2512  ItemType* InsertAfter(ItemType* pItem);
        2513 
        2514  ItemType* InsertBefore(ItemType* pItem, const T& value);
        2515  ItemType* InsertAfter(ItemType* pItem, const T& value);
        2516 
        2517  void Remove(ItemType* pItem);
        2518 
        2519 private:
        2520  const VkAllocationCallbacks* const m_pAllocationCallbacks;
        2521  VmaPoolAllocator<ItemType> m_ItemAllocator;
        2522  ItemType* m_pFront;
        2523  ItemType* m_pBack;
        2524  size_t m_Count;
        2525 
        2526  // Declared not defined, to block copy constructor and assignment operator.
        2527  VmaRawList(const VmaRawList<T>& src);
        2528  VmaRawList<T>& operator=(const VmaRawList<T>& rhs);
        2529 };
        2530 
        2531 template<typename T>
        2532 VmaRawList<T>::VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks) :
        2533  m_pAllocationCallbacks(pAllocationCallbacks),
        2534  m_ItemAllocator(pAllocationCallbacks, 128),
        2535  m_pFront(VMA_NULL),
        2536  m_pBack(VMA_NULL),
        2537  m_Count(0)
        2538 {
        2539 }
        2540 
        2541 template<typename T>
        2542 VmaRawList<T>::~VmaRawList()
        2543 {
        2544  // Intentionally not calling Clear, because that would be unnecessary
        2545  // computations to return all items to m_ItemAllocator as free.
        2546 }
        2547 
        2548 template<typename T>
        2549 void VmaRawList<T>::Clear()
        2550 {
        2551  if(IsEmpty() == false)
        2552  {
        2553  ItemType* pItem = m_pBack;
        2554  while(pItem != VMA_NULL)
        2555  {
        2556  ItemType* const pPrevItem = pItem->pPrev;
        2557  m_ItemAllocator.Free(pItem);
        2558  pItem = pPrevItem;
        2559  }
        2560  m_pFront = VMA_NULL;
        2561  m_pBack = VMA_NULL;
        2562  m_Count = 0;
        2563  }
        2564 }
        2565 
        2566 template<typename T>
        2567 VmaListItem<T>* VmaRawList<T>::PushBack()
        2568 {
        2569  ItemType* const pNewItem = m_ItemAllocator.Alloc();
        2570  pNewItem->pNext = VMA_NULL;
        2571  if(IsEmpty())
        2572  {
        2573  pNewItem->pPrev = VMA_NULL;
        2574  m_pFront = pNewItem;
        2575  m_pBack = pNewItem;
        2576  m_Count = 1;
        2577  }
        2578  else
        2579  {
        2580  pNewItem->pPrev = m_pBack;
        2581  m_pBack->pNext = pNewItem;
        2582  m_pBack = pNewItem;
        2583  ++m_Count;
        2584  }
        2585  return pNewItem;
        2586 }
        2587 
        2588 template<typename T>
        2589 VmaListItem<T>* VmaRawList<T>::PushFront()
        2590 {
        2591  ItemType* const pNewItem = m_ItemAllocator.Alloc();
        2592  pNewItem->pPrev = VMA_NULL;
        2593  if(IsEmpty())
        2594  {
        2595  pNewItem->pNext = VMA_NULL;
        2596  m_pFront = pNewItem;
        2597  m_pBack = pNewItem;
        2598  m_Count = 1;
        2599  }
        2600  else
        2601  {
        2602  pNewItem->pNext = m_pFront;
        2603  m_pFront->pPrev = pNewItem;
        2604  m_pFront = pNewItem;
        2605  ++m_Count;
        2606  }
        2607  return pNewItem;
        2608 }
        2609 
        2610 template<typename T>
        2611 VmaListItem<T>* VmaRawList<T>::PushBack(const T& value)
        2612 {
        2613  ItemType* const pNewItem = PushBack();
        2614  pNewItem->Value = value;
        2615  return pNewItem;
        2616 }
        2617 
        2618 template<typename T>
        2619 VmaListItem<T>* VmaRawList<T>::PushFront(const T& value)
        2620 {
        2621  ItemType* const pNewItem = PushFront();
        2622  pNewItem->Value = value;
        2623  return pNewItem;
        2624 }
        2625 
        2626 template<typename T>
        2627 void VmaRawList<T>::PopBack()
        2628 {
        2629  VMA_HEAVY_ASSERT(m_Count > 0);
        2630  ItemType* const pBackItem = m_pBack;
        2631  ItemType* const pPrevItem = pBackItem->pPrev;
        2632  if(pPrevItem != VMA_NULL)
        2633  {
        2634  pPrevItem->pNext = VMA_NULL;
        2635  }
        2636  m_pBack = pPrevItem;
        2637  m_ItemAllocator.Free(pBackItem);
        2638  --m_Count;
        2639 }
        2640 
        2641 template<typename T>
        2642 void VmaRawList<T>::PopFront()
        2643 {
        2644  VMA_HEAVY_ASSERT(m_Count > 0);
        2645  ItemType* const pFrontItem = m_pFront;
        2646  ItemType* const pNextItem = pFrontItem->pNext;
        2647  if(pNextItem != VMA_NULL)
        2648  {
        2649  pNextItem->pPrev = VMA_NULL;
        2650  }
        2651  m_pFront = pNextItem;
        2652  m_ItemAllocator.Free(pFrontItem);
        2653  --m_Count;
        2654 }
        2655 
        2656 template<typename T>
        2657 void VmaRawList<T>::Remove(ItemType* pItem)
        2658 {
        2659  VMA_HEAVY_ASSERT(pItem != VMA_NULL);
        2660  VMA_HEAVY_ASSERT(m_Count > 0);
        2661 
        2662  if(pItem->pPrev != VMA_NULL)
        2663  {
        2664  pItem->pPrev->pNext = pItem->pNext;
        2665  }
        2666  else
        2667  {
        2668  VMA_HEAVY_ASSERT(m_pFront == pItem);
        2669  m_pFront = pItem->pNext;
        2670  }
        2671 
        2672  if(pItem->pNext != VMA_NULL)
        2673  {
        2674  pItem->pNext->pPrev = pItem->pPrev;
        2675  }
        2676  else
        2677  {
        2678  VMA_HEAVY_ASSERT(m_pBack == pItem);
        2679  m_pBack = pItem->pPrev;
        2680  }
        2681 
        2682  m_ItemAllocator.Free(pItem);
        2683  --m_Count;
        2684 }
        2685 
        2686 template<typename T>
        2687 VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem)
        2688 {
        2689  if(pItem != VMA_NULL)
        2690  {
        2691  ItemType* const prevItem = pItem->pPrev;
        2692  ItemType* const newItem = m_ItemAllocator.Alloc();
        2693  newItem->pPrev = prevItem;
        2694  newItem->pNext = pItem;
        2695  pItem->pPrev = newItem;
        2696  if(prevItem != VMA_NULL)
        2697  {
        2698  prevItem->pNext = newItem;
        2699  }
        2700  else
        2701  {
        2702  VMA_HEAVY_ASSERT(m_pFront == pItem);
        2703  m_pFront = newItem;
        2704  }
        2705  ++m_Count;
        2706  return newItem;
        2707  }
        2708  else
        2709  return PushBack();
        2710 }
        2711 
        2712 template<typename T>
        2713 VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem)
        2714 {
        2715  if(pItem != VMA_NULL)
        2716  {
        2717  ItemType* const nextItem = pItem->pNext;
        2718  ItemType* const newItem = m_ItemAllocator.Alloc();
        2719  newItem->pNext = nextItem;
        2720  newItem->pPrev = pItem;
        2721  pItem->pNext = newItem;
        2722  if(nextItem != VMA_NULL)
        2723  {
        2724  nextItem->pPrev = newItem;
        2725  }
        2726  else
        2727  {
        2728  VMA_HEAVY_ASSERT(m_pBack == pItem);
        2729  m_pBack = newItem;
        2730  }
        2731  ++m_Count;
        2732  return newItem;
        2733  }
        2734  else
        2735  return PushFront();
        2736 }
        2737 
        2738 template<typename T>
        2739 VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem, const T& value)
        2740 {
        2741  ItemType* const newItem = InsertBefore(pItem);
        2742  newItem->Value = value;
        2743  return newItem;
        2744 }
        2745 
        2746 template<typename T>
        2747 VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem, const T& value)
        2748 {
        2749  ItemType* const newItem = InsertAfter(pItem);
        2750  newItem->Value = value;
        2751  return newItem;
        2752 }
        2753 
        2754 template<typename T, typename AllocatorT>
        2755 class VmaList
        2756 {
        2757 public:
        2758  class iterator
        2759  {
        2760  public:
        2761  iterator() :
        2762  m_pList(VMA_NULL),
        2763  m_pItem(VMA_NULL)
        2764  {
        2765  }
        2766 
        2767  T& operator*() const
        2768  {
        2769  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2770  return m_pItem->Value;
        2771  }
        2772  T* operator->() const
        2773  {
        2774  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2775  return &m_pItem->Value;
        2776  }
        2777 
        2778  iterator& operator++()
        2779  {
        2780  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2781  m_pItem = m_pItem->pNext;
        2782  return *this;
        2783  }
        2784  iterator& operator--()
        2785  {
        2786  if(m_pItem != VMA_NULL)
        2787  {
        2788  m_pItem = m_pItem->pPrev;
        2789  }
        2790  else
        2791  {
        2792  VMA_HEAVY_ASSERT(!m_pList.IsEmpty());
        2793  m_pItem = m_pList->Back();
        2794  }
        2795  return *this;
        2796  }
        2797 
        2798  iterator operator++(int)
        2799  {
        2800  iterator result = *this;
        2801  ++*this;
        2802  return result;
        2803  }
        2804  iterator operator--(int)
        2805  {
        2806  iterator result = *this;
        2807  --*this;
        2808  return result;
        2809  }
        2810 
        2811  bool operator==(const iterator& rhs) const
        2812  {
        2813  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
        2814  return m_pItem == rhs.m_pItem;
        2815  }
        2816  bool operator!=(const iterator& rhs) const
        2817  {
        2818  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
        2819  return m_pItem != rhs.m_pItem;
        2820  }
        2821 
        2822  private:
        2823  VmaRawList<T>* m_pList;
        2824  VmaListItem<T>* m_pItem;
        2825 
        2826  iterator(VmaRawList<T>* pList, VmaListItem<T>* pItem) :
        2827  m_pList(pList),
        2828  m_pItem(pItem)
        2829  {
        2830  }
        2831 
        2832  friend class VmaList<T, AllocatorT>;
        2833  };
        2834 
        2835  class const_iterator
        2836  {
        2837  public:
        2838  const_iterator() :
        2839  m_pList(VMA_NULL),
        2840  m_pItem(VMA_NULL)
        2841  {
        2842  }
        2843 
        2844  const_iterator(const iterator& src) :
        2845  m_pList(src.m_pList),
        2846  m_pItem(src.m_pItem)
        2847  {
        2848  }
        2849 
        2850  const T& operator*() const
        2851  {
        2852  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2853  return m_pItem->Value;
        2854  }
        2855  const T* operator->() const
        2856  {
        2857  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2858  return &m_pItem->Value;
        2859  }
        2860 
        2861  const_iterator& operator++()
        2862  {
        2863  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2864  m_pItem = m_pItem->pNext;
        2865  return *this;
        2866  }
        2867  const_iterator& operator--()
        2868  {
        2869  if(m_pItem != VMA_NULL)
        2870  {
        2871  m_pItem = m_pItem->pPrev;
        2872  }
        2873  else
        2874  {
        2875  VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
        2876  m_pItem = m_pList->Back();
        2877  }
        2878  return *this;
        2879  }
        2880 
        2881  const_iterator operator++(int)
        2882  {
        2883  const_iterator result = *this;
        2884  ++*this;
        2885  return result;
        2886  }
        2887  const_iterator operator--(int)
        2888  {
        2889  const_iterator result = *this;
        2890  --*this;
        2891  return result;
        2892  }
        2893 
        2894  bool operator==(const const_iterator& rhs) const
        2895  {
        2896  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
        2897  return m_pItem == rhs.m_pItem;
        2898  }
        2899  bool operator!=(const const_iterator& rhs) const
        2900  {
        2901  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
        2902  return m_pItem != rhs.m_pItem;
        2903  }
        2904 
        2905  private:
        2906  const_iterator(const VmaRawList<T>* pList, const VmaListItem<T>* pItem) :
        2907  m_pList(pList),
        2908  m_pItem(pItem)
        2909  {
        2910  }
        2911 
        2912  const VmaRawList<T>* m_pList;
        2913  const VmaListItem<T>* m_pItem;
        2914 
        2915  friend class VmaList<T, AllocatorT>;
        2916  };
        2917 
        2918  VmaList(const AllocatorT& allocator) : m_RawList(allocator.m_pCallbacks) { }
        2919 
        2920  bool empty() const { return m_RawList.IsEmpty(); }
        2921  size_t size() const { return m_RawList.GetCount(); }
        2922 
        2923  iterator begin() { return iterator(&m_RawList, m_RawList.Front()); }
        2924  iterator end() { return iterator(&m_RawList, VMA_NULL); }
        2925 
        2926  const_iterator cbegin() const { return const_iterator(&m_RawList, m_RawList.Front()); }
        2927  const_iterator cend() const { return const_iterator(&m_RawList, VMA_NULL); }
        2928 
        2929  void clear() { m_RawList.Clear(); }
        2930  void push_back(const T& value) { m_RawList.PushBack(value); }
        2931  void erase(iterator it) { m_RawList.Remove(it.m_pItem); }
        2932  iterator insert(iterator it, const T& value) { return iterator(&m_RawList, m_RawList.InsertBefore(it.m_pItem, value)); }
        2933 
        2934 private:
        2935  VmaRawList<T> m_RawList;
        2936 };
        2937 
        2938 #endif // #if VMA_USE_STL_LIST
        2939 
        2941 // class VmaMap
        2942 
        2943 // Unused in this version.
        2944 #if 0
        2945 
        2946 #if VMA_USE_STL_UNORDERED_MAP
        2947 
        2948 #define VmaPair std::pair
        2949 
        2950 #define VMA_MAP_TYPE(KeyT, ValueT) \
        2951  std::unordered_map< KeyT, ValueT, std::hash<KeyT>, std::equal_to<KeyT>, VmaStlAllocator< std::pair<KeyT, ValueT> > >
        2952 
        2953 #else // #if VMA_USE_STL_UNORDERED_MAP
        2954 
        2955 template<typename T1, typename T2>
        2956 struct VmaPair
        2957 {
        2958  T1 first;
        2959  T2 second;
        2960 
        2961  VmaPair() : first(), second() { }
        2962  VmaPair(const T1& firstSrc, const T2& secondSrc) : first(firstSrc), second(secondSrc) { }
        2963 };
        2964 
        2965 /* Class compatible with subset of interface of std::unordered_map.
        2966 KeyT, ValueT must be POD because they will be stored in VmaVector.
        2967 */
        2968 template<typename KeyT, typename ValueT>
        2969 class VmaMap
        2970 {
        2971 public:
        2972  typedef VmaPair<KeyT, ValueT> PairType;
        2973  typedef PairType* iterator;
        2974 
        2975  VmaMap(const VmaStlAllocator<PairType>& allocator) : m_Vector(allocator) { }
        2976 
        2977  iterator begin() { return m_Vector.begin(); }
        2978  iterator end() { return m_Vector.end(); }
        2979 
        2980  void insert(const PairType& pair);
        2981  iterator find(const KeyT& key);
        2982  void erase(iterator it);
        2983 
        2984 private:
        2985  VmaVector< PairType, VmaStlAllocator<PairType> > m_Vector;
        2986 };
        2987 
        2988 #define VMA_MAP_TYPE(KeyT, ValueT) VmaMap<KeyT, ValueT>
        2989 
        2990 template<typename FirstT, typename SecondT>
        2991 struct VmaPairFirstLess
        2992 {
        2993  bool operator()(const VmaPair<FirstT, SecondT>& lhs, const VmaPair<FirstT, SecondT>& rhs) const
        2994  {
        2995  return lhs.first < rhs.first;
        2996  }
        2997  bool operator()(const VmaPair<FirstT, SecondT>& lhs, const FirstT& rhsFirst) const
        2998  {
        2999  return lhs.first < rhsFirst;
        3000  }
        3001 };
        3002 
        3003 template<typename KeyT, typename ValueT>
        3004 void VmaMap<KeyT, ValueT>::insert(const PairType& pair)
        3005 {
        3006  const size_t indexToInsert = VmaBinaryFindFirstNotLess(
        3007  m_Vector.data(),
        3008  m_Vector.data() + m_Vector.size(),
        3009  pair,
        3010  VmaPairFirstLess<KeyT, ValueT>()) - m_Vector.data();
        3011  VmaVectorInsert(m_Vector, indexToInsert, pair);
        3012 }
        3013 
        3014 template<typename KeyT, typename ValueT>
        3015 VmaPair<KeyT, ValueT>* VmaMap<KeyT, ValueT>::find(const KeyT& key)
        3016 {
        3017  PairType* it = VmaBinaryFindFirstNotLess(
        3018  m_Vector.data(),
        3019  m_Vector.data() + m_Vector.size(),
        3020  key,
        3021  VmaPairFirstLess<KeyT, ValueT>());
        3022  if((it != m_Vector.end()) && (it->first == key))
        3023  {
        3024  return it;
        3025  }
        3026  else
        3027  {
        3028  return m_Vector.end();
        3029  }
        3030 }
        3031 
        3032 template<typename KeyT, typename ValueT>
        3033 void VmaMap<KeyT, ValueT>::erase(iterator it)
        3034 {
        3035  VmaVectorRemove(m_Vector, it - m_Vector.begin());
        3036 }
        3037 
        3038 #endif // #if VMA_USE_STL_UNORDERED_MAP
        3039 
        3040 #endif // #if 0
        3041 
        3043 
        3044 class VmaDeviceMemoryBlock;
        3045 
        3046 struct VmaAllocation_T
        3047 {
        3048 private:
        3049  static const uint8_t MAP_COUNT_FLAG_PERSISTENT_MAP = 0x80;
        3050 
        3051  enum FLAGS
        3052  {
        3053  FLAG_USER_DATA_STRING = 0x01,
        3054  };
        3055 
        3056 public:
        3057  enum ALLOCATION_TYPE
        3058  {
        3059  ALLOCATION_TYPE_NONE,
        3060  ALLOCATION_TYPE_BLOCK,
        3061  ALLOCATION_TYPE_DEDICATED,
        3062  };
        3063 
        3064  VmaAllocation_T(uint32_t currentFrameIndex, bool userDataString) :
        3065  m_Alignment(1),
        3066  m_Size(0),
        3067  m_pUserData(VMA_NULL),
        3068  m_LastUseFrameIndex(currentFrameIndex),
        3069  m_Type((uint8_t)ALLOCATION_TYPE_NONE),
        3070  m_SuballocationType((uint8_t)VMA_SUBALLOCATION_TYPE_UNKNOWN),
        3071  m_MapCount(0),
        3072  m_Flags(userDataString ? (uint8_t)FLAG_USER_DATA_STRING : 0)
        3073  {
        3074  }
        3075 
        3076  ~VmaAllocation_T()
        3077  {
        3078  VMA_ASSERT((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) == 0 && "Allocation was not unmapped before destruction.");
        3079 
        3080  // Check if owned string was freed.
        3081  VMA_ASSERT(m_pUserData == VMA_NULL);
        3082  }
        3083 
        3084  void InitBlockAllocation(
        3085  VmaPool hPool,
        3086  VmaDeviceMemoryBlock* block,
        3087  VkDeviceSize offset,
        3088  VkDeviceSize alignment,
        3089  VkDeviceSize size,
        3090  VmaSuballocationType suballocationType,
        3091  bool mapped,
        3092  bool canBecomeLost)
        3093  {
        3094  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
        3095  VMA_ASSERT(block != VMA_NULL);
        3096  m_Type = (uint8_t)ALLOCATION_TYPE_BLOCK;
        3097  m_Alignment = alignment;
        3098  m_Size = size;
        3099  m_MapCount = mapped ? MAP_COUNT_FLAG_PERSISTENT_MAP : 0;
        3100  m_SuballocationType = (uint8_t)suballocationType;
        3101  m_BlockAllocation.m_hPool = hPool;
        3102  m_BlockAllocation.m_Block = block;
        3103  m_BlockAllocation.m_Offset = offset;
        3104  m_BlockAllocation.m_CanBecomeLost = canBecomeLost;
        3105  }
        3106 
        3107  void InitLost()
        3108  {
        3109  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
        3110  VMA_ASSERT(m_LastUseFrameIndex.load() == VMA_FRAME_INDEX_LOST);
        3111  m_Type = (uint8_t)ALLOCATION_TYPE_BLOCK;
        3112  m_BlockAllocation.m_hPool = VK_NULL_HANDLE;
        3113  m_BlockAllocation.m_Block = VMA_NULL;
        3114  m_BlockAllocation.m_Offset = 0;
        3115  m_BlockAllocation.m_CanBecomeLost = true;
        3116  }
        3117 
        3118  void ChangeBlockAllocation(
        3119  VmaDeviceMemoryBlock* block,
        3120  VkDeviceSize offset)
        3121  {
        3122  VMA_ASSERT(block != VMA_NULL);
        3123  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
        3124  m_BlockAllocation.m_Block = block;
        3125  m_BlockAllocation.m_Offset = offset;
        3126  }
        3127 
        3128  // pMappedData not null means allocation is created with MAPPED flag.
        3129  void InitDedicatedAllocation(
        3130  uint32_t memoryTypeIndex,
        3131  VkDeviceMemory hMemory,
        3132  VmaSuballocationType suballocationType,
        3133  void* pMappedData,
        3134  VkDeviceSize size)
        3135  {
        3136  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
        3137  VMA_ASSERT(hMemory != VK_NULL_HANDLE);
        3138  m_Type = (uint8_t)ALLOCATION_TYPE_DEDICATED;
        3139  m_Alignment = 0;
        3140  m_Size = size;
        3141  m_SuballocationType = (uint8_t)suballocationType;
        3142  m_MapCount = (pMappedData != VMA_NULL) ? MAP_COUNT_FLAG_PERSISTENT_MAP : 0;
        3143  m_DedicatedAllocation.m_MemoryTypeIndex = memoryTypeIndex;
        3144  m_DedicatedAllocation.m_hMemory = hMemory;
        3145  m_DedicatedAllocation.m_pMappedData = pMappedData;
        3146  }
        3147 
        3148  ALLOCATION_TYPE GetType() const { return (ALLOCATION_TYPE)m_Type; }
        3149  VkDeviceSize GetAlignment() const { return m_Alignment; }
        3150  VkDeviceSize GetSize() const { return m_Size; }
        3151  bool IsUserDataString() const { return (m_Flags & FLAG_USER_DATA_STRING) != 0; }
        3152  void* GetUserData() const { return m_pUserData; }
        3153  void SetUserData(VmaAllocator hAllocator, void* pUserData);
        3154  VmaSuballocationType GetSuballocationType() const { return (VmaSuballocationType)m_SuballocationType; }
        3155 
        3156  VmaDeviceMemoryBlock* GetBlock() const
        3157  {
        3158  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
        3159  return m_BlockAllocation.m_Block;
        3160  }
        3161  VkDeviceSize GetOffset() const;
        3162  VkDeviceMemory GetMemory() const;
        3163  uint32_t GetMemoryTypeIndex() const;
        3164  bool IsPersistentMap() const { return (m_MapCount & MAP_COUNT_FLAG_PERSISTENT_MAP) != 0; }
        3165  void* GetMappedData() const;
        3166  bool CanBecomeLost() const;
        3167  VmaPool GetPool() const;
        3168 
        3169  uint32_t GetLastUseFrameIndex() const
        3170  {
        3171  return m_LastUseFrameIndex.load();
        3172  }
        3173  bool CompareExchangeLastUseFrameIndex(uint32_t& expected, uint32_t desired)
        3174  {
        3175  return m_LastUseFrameIndex.compare_exchange_weak(expected, desired);
        3176  }
        3177  /*
        3178  - If hAllocation.LastUseFrameIndex + frameInUseCount < allocator.CurrentFrameIndex,
        3179  makes it lost by setting LastUseFrameIndex = VMA_FRAME_INDEX_LOST and returns true.
        3180  - Else, returns false.
        3181 
        3182  If hAllocation is already lost, assert - you should not call it then.
        3183  If hAllocation was not created with CAN_BECOME_LOST_BIT, assert.
        3184  */
        3185  bool MakeLost(uint32_t currentFrameIndex, uint32_t frameInUseCount);
        3186 
        3187  void DedicatedAllocCalcStatsInfo(VmaStatInfo& outInfo)
        3188  {
        3189  VMA_ASSERT(m_Type == ALLOCATION_TYPE_DEDICATED);
        3190  outInfo.blockCount = 1;
        3191  outInfo.allocationCount = 1;
        3192  outInfo.unusedRangeCount = 0;
        3193  outInfo.usedBytes = m_Size;
        3194  outInfo.unusedBytes = 0;
        3195  outInfo.allocationSizeMin = outInfo.allocationSizeMax = m_Size;
        3196  outInfo.unusedRangeSizeMin = UINT64_MAX;
        3197  outInfo.unusedRangeSizeMax = 0;
        3198  }
        3199 
        3200  void BlockAllocMap();
        3201  void BlockAllocUnmap();
        3202  VkResult DedicatedAllocMap(VmaAllocator hAllocator, void** ppData);
        3203  void DedicatedAllocUnmap(VmaAllocator hAllocator);
        3204 
        3205 private:
        3206  VkDeviceSize m_Alignment;
        3207  VkDeviceSize m_Size;
        3208  void* m_pUserData;
        3209  VMA_ATOMIC_UINT32 m_LastUseFrameIndex;
        3210  uint8_t m_Type; // ALLOCATION_TYPE
        3211  uint8_t m_SuballocationType; // VmaSuballocationType
        3212  // Bit 0x80 is set when allocation was created with VMA_ALLOCATION_CREATE_MAPPED_BIT.
        3213  // Bits with mask 0x7F, used only when ALLOCATION_TYPE_DEDICATED, are reference counter for vmaMapMemory()/vmaUnmapMemory().
        3214  uint8_t m_MapCount;
        3215  uint8_t m_Flags; // enum FLAGS
        3216 
        3217  // Allocation out of VmaDeviceMemoryBlock.
        3218  struct BlockAllocation
        3219  {
        3220  VmaPool m_hPool; // Null if belongs to general memory.
        3221  VmaDeviceMemoryBlock* m_Block;
        3222  VkDeviceSize m_Offset;
        3223  bool m_CanBecomeLost;
        3224  };
        3225 
        3226  // Allocation for an object that has its own private VkDeviceMemory.
        3227  struct DedicatedAllocation
        3228  {
        3229  uint32_t m_MemoryTypeIndex;
        3230  VkDeviceMemory m_hMemory;
        3231  void* m_pMappedData; // Not null means memory is mapped.
        3232  };
        3233 
        3234  union
        3235  {
        3236  // Allocation out of VmaDeviceMemoryBlock.
        3237  BlockAllocation m_BlockAllocation;
        3238  // Allocation for an object that has its own private VkDeviceMemory.
        3239  DedicatedAllocation m_DedicatedAllocation;
        3240  };
        3241 
        3242  void FreeUserDataString(VmaAllocator hAllocator);
        3243 };
        3244 
        3245 /*
        3246 Represents a region of VmaDeviceMemoryBlock that is either assigned and returned as
        3247 allocated memory block or free.
        3248 */
        3249 struct VmaSuballocation
        3250 {
        3251  VkDeviceSize offset;
        3252  VkDeviceSize size;
        3253  VmaAllocation hAllocation;
        3254  VmaSuballocationType type;
        3255 };
        3256 
        3257 typedef VmaList< VmaSuballocation, VmaStlAllocator<VmaSuballocation> > VmaSuballocationList;
        3258 
        3259 // Cost of one additional allocation lost, as equivalent in bytes.
        3260 static const VkDeviceSize VMA_LOST_ALLOCATION_COST = 1048576;
        3261 
        3262 /*
        3263 Parameters of planned allocation inside a VmaDeviceMemoryBlock.
        3264 
        3265 If canMakeOtherLost was false:
        3266 - item points to a FREE suballocation.
        3267 - itemsToMakeLostCount is 0.
        3268 
        3269 If canMakeOtherLost was true:
        3270 - item points to first of sequence of suballocations, which are either FREE,
        3271  or point to VmaAllocations that can become lost.
        3272 - itemsToMakeLostCount is the number of VmaAllocations that need to be made lost for
        3273  the requested allocation to succeed.
        3274 */
        3275 struct VmaAllocationRequest
        3276 {
        3277  VkDeviceSize offset;
        3278  VkDeviceSize sumFreeSize; // Sum size of free items that overlap with proposed allocation.
        3279  VkDeviceSize sumItemSize; // Sum size of items to make lost that overlap with proposed allocation.
        3280  VmaSuballocationList::iterator item;
        3281  size_t itemsToMakeLostCount;
        3282 
        3283  VkDeviceSize CalcCost() const
        3284  {
        3285  return sumItemSize + itemsToMakeLostCount * VMA_LOST_ALLOCATION_COST;
        3286  }
        3287 };
        3288 
        3289 /*
        3290 Data structure used for bookkeeping of allocations and unused ranges of memory
        3291 in a single VkDeviceMemory block.
        3292 */
        3293 class VmaBlockMetadata
        3294 {
        3295 public:
        3296  VmaBlockMetadata(VmaAllocator hAllocator);
        3297  ~VmaBlockMetadata();
        3298  void Init(VkDeviceSize size);
        3299 
        3300  // Validates all data structures inside this object. If not valid, returns false.
        3301  bool Validate() const;
        3302  VkDeviceSize GetSize() const { return m_Size; }
        3303  size_t GetAllocationCount() const { return m_Suballocations.size() - m_FreeCount; }
        3304  VkDeviceSize GetSumFreeSize() const { return m_SumFreeSize; }
        3305  VkDeviceSize GetUnusedRangeSizeMax() const;
        3306  // Returns true if this block is empty - contains only single free suballocation.
        3307  bool IsEmpty() const;
        3308 
        3309  void CalcAllocationStatInfo(VmaStatInfo& outInfo) const;
        3310  void AddPoolStats(VmaPoolStats& inoutStats) const;
        3311 
        3312 #if VMA_STATS_STRING_ENABLED
        3313  void PrintDetailedMap(class VmaJsonWriter& json) const;
        3314 #endif
        3315 
        3316  // Creates trivial request for case when block is empty.
        3317  void CreateFirstAllocationRequest(VmaAllocationRequest* pAllocationRequest);
        3318 
        3319  // Tries to find a place for suballocation with given parameters inside this block.
        3320  // If succeeded, fills pAllocationRequest and returns true.
        3321  // If failed, returns false.
        3322  bool CreateAllocationRequest(
        3323  uint32_t currentFrameIndex,
        3324  uint32_t frameInUseCount,
        3325  VkDeviceSize bufferImageGranularity,
        3326  VkDeviceSize allocSize,
        3327  VkDeviceSize allocAlignment,
        3328  VmaSuballocationType allocType,
        3329  bool canMakeOtherLost,
        3330  VmaAllocationRequest* pAllocationRequest);
        3331 
        3332  bool MakeRequestedAllocationsLost(
        3333  uint32_t currentFrameIndex,
        3334  uint32_t frameInUseCount,
        3335  VmaAllocationRequest* pAllocationRequest);
        3336 
        3337  uint32_t MakeAllocationsLost(uint32_t currentFrameIndex, uint32_t frameInUseCount);
        3338 
        3339  // Makes actual allocation based on request. Request must already be checked and valid.
        3340  void Alloc(
        3341  const VmaAllocationRequest& request,
        3342  VmaSuballocationType type,
        3343  VkDeviceSize allocSize,
        3344  VmaAllocation hAllocation);
        3345 
        3346  // Frees suballocation assigned to given memory region.
        3347  void Free(const VmaAllocation allocation);
        3348 
        3349 private:
        3350  VkDeviceSize m_Size;
        3351  uint32_t m_FreeCount;
        3352  VkDeviceSize m_SumFreeSize;
        3353  VmaSuballocationList m_Suballocations;
        3354  // Suballocations that are free and have size greater than certain threshold.
        3355  // Sorted by size, ascending.
        3356  VmaVector< VmaSuballocationList::iterator, VmaStlAllocator< VmaSuballocationList::iterator > > m_FreeSuballocationsBySize;
        3357 
        3358  bool ValidateFreeSuballocationList() const;
        3359 
        3360  // Checks if requested suballocation with given parameters can be placed in given pFreeSuballocItem.
        3361  // If yes, fills pOffset and returns true. If no, returns false.
        3362  bool CheckAllocation(
        3363  uint32_t currentFrameIndex,
        3364  uint32_t frameInUseCount,
        3365  VkDeviceSize bufferImageGranularity,
        3366  VkDeviceSize allocSize,
        3367  VkDeviceSize allocAlignment,
        3368  VmaSuballocationType allocType,
        3369  VmaSuballocationList::const_iterator suballocItem,
        3370  bool canMakeOtherLost,
        3371  VkDeviceSize* pOffset,
        3372  size_t* itemsToMakeLostCount,
        3373  VkDeviceSize* pSumFreeSize,
        3374  VkDeviceSize* pSumItemSize) const;
        3375  // Given free suballocation, it merges it with following one, which must also be free.
        3376  void MergeFreeWithNext(VmaSuballocationList::iterator item);
        3377  // Releases given suballocation, making it free.
        3378  // Merges it with adjacent free suballocations if applicable.
        3379  // Returns iterator to new free suballocation at this place.
        3380  VmaSuballocationList::iterator FreeSuballocation(VmaSuballocationList::iterator suballocItem);
        3381  // Given free suballocation, it inserts it into sorted list of
        3382  // m_FreeSuballocationsBySize if it's suitable.
        3383  void RegisterFreeSuballocation(VmaSuballocationList::iterator item);
        3384  // Given free suballocation, it removes it from sorted list of
        3385  // m_FreeSuballocationsBySize if it's suitable.
        3386  void UnregisterFreeSuballocation(VmaSuballocationList::iterator item);
        3387 };
        3388 
        3389 // Helper class that represents mapped memory. Synchronized internally.
        3390 class VmaDeviceMemoryMapping
        3391 {
        3392 public:
        3393  VmaDeviceMemoryMapping();
        3394  ~VmaDeviceMemoryMapping();
        3395 
        3396  void* GetMappedData() const { return m_pMappedData; }
        3397 
        3398  // ppData can be null.
        3399  VkResult Map(VmaAllocator hAllocator, VkDeviceMemory hMemory, void **ppData);
        3400  void Unmap(VmaAllocator hAllocator, VkDeviceMemory hMemory);
        3401 
        3402 private:
        3403  VMA_MUTEX m_Mutex;
        3404  uint32_t m_MapCount;
        3405  void* m_pMappedData;
        3406 };
        3407 
        3408 /*
        3409 Represents a single block of device memory (`VkDeviceMemory`) with all the
        3410 data about its regions (aka suballocations, `VmaAllocation`), assigned and free.
        3411 
        3412 Thread-safety: This class must be externally synchronized.
        3413 */
        3414 class VmaDeviceMemoryBlock
        3415 {
        3416 public:
        3417  uint32_t m_MemoryTypeIndex;
        3418  VkDeviceMemory m_hMemory;
        3419  VmaDeviceMemoryMapping m_Mapping;
        3420  VmaBlockMetadata m_Metadata;
        3421 
        3422  VmaDeviceMemoryBlock(VmaAllocator hAllocator);
        3423 
        3424  ~VmaDeviceMemoryBlock()
        3425  {
        3426  VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
        3427  }
        3428 
        3429  // Always call after construction.
        3430  void Init(
        3431  uint32_t newMemoryTypeIndex,
        3432  VkDeviceMemory newMemory,
        3433  VkDeviceSize newSize);
        3434  // Always call before destruction.
        3435  void Destroy(VmaAllocator allocator);
        3436 
        3437  // Validates all data structures inside this object. If not valid, returns false.
        3438  bool Validate() const;
        3439 
        3440  // ppData can be null.
        3441  VkResult Map(VmaAllocator hAllocator, void** ppData);
        3442  void Unmap(VmaAllocator hAllocator);
        3443 };
        3444 
        3445 struct VmaPointerLess
        3446 {
        3447  bool operator()(const void* lhs, const void* rhs) const
        3448  {
        3449  return lhs < rhs;
        3450  }
        3451 };
        3452 
        3453 class VmaDefragmentator;
        3454 
        3455 /*
        3456 Sequence of VmaDeviceMemoryBlock. Represents memory blocks allocated for a specific
        3457 Vulkan memory type.
        3458 
        3459 Synchronized internally with a mutex.
        3460 */
        3461 struct VmaBlockVector
        3462 {
        3463  VmaBlockVector(
        3464  VmaAllocator hAllocator,
        3465  uint32_t memoryTypeIndex,
        3466  VkDeviceSize preferredBlockSize,
        3467  size_t minBlockCount,
        3468  size_t maxBlockCount,
        3469  VkDeviceSize bufferImageGranularity,
        3470  uint32_t frameInUseCount,
        3471  bool isCustomPool);
        3472  ~VmaBlockVector();
        3473 
        3474  VkResult CreateMinBlocks();
        3475 
        3476  uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
        3477  VkDeviceSize GetPreferredBlockSize() const { return m_PreferredBlockSize; }
        3478  VkDeviceSize GetBufferImageGranularity() const { return m_BufferImageGranularity; }
        3479  uint32_t GetFrameInUseCount() const { return m_FrameInUseCount; }
        3480 
        3481  void GetPoolStats(VmaPoolStats* pStats);
        3482 
        3483  bool IsEmpty() const { return m_Blocks.empty(); }
        3484 
        3485  VkResult Allocate(
        3486  VmaPool hCurrentPool,
        3487  uint32_t currentFrameIndex,
        3488  const VkMemoryRequirements& vkMemReq,
        3489  const VmaAllocationCreateInfo& createInfo,
        3490  VmaSuballocationType suballocType,
        3491  VmaAllocation* pAllocation);
        3492 
        3493  void Free(
        3494  VmaAllocation hAllocation);
        3495 
        3496  // Adds statistics of this BlockVector to pStats.
        3497  void AddStats(VmaStats* pStats);
        3498 
        3499 #if VMA_STATS_STRING_ENABLED
        3500  void PrintDetailedMap(class VmaJsonWriter& json);
        3501 #endif
        3502 
        3503  void MakePoolAllocationsLost(
        3504  uint32_t currentFrameIndex,
        3505  size_t* pLostAllocationCount);
        3506 
        3507  VmaDefragmentator* EnsureDefragmentator(
        3508  VmaAllocator hAllocator,
        3509  uint32_t currentFrameIndex);
        3510 
        3511  VkResult Defragment(
        3512  VmaDefragmentationStats* pDefragmentationStats,
        3513  VkDeviceSize& maxBytesToMove,
        3514  uint32_t& maxAllocationsToMove);
        3515 
        3516  void DestroyDefragmentator();
        3517 
        3518 private:
        3519  friend class VmaDefragmentator;
        3520 
        3521  const VmaAllocator m_hAllocator;
        3522  const uint32_t m_MemoryTypeIndex;
        3523  const VkDeviceSize m_PreferredBlockSize;
        3524  const size_t m_MinBlockCount;
        3525  const size_t m_MaxBlockCount;
        3526  const VkDeviceSize m_BufferImageGranularity;
        3527  const uint32_t m_FrameInUseCount;
        3528  const bool m_IsCustomPool;
        3529  VMA_MUTEX m_Mutex;
        3530  // Incrementally sorted by sumFreeSize, ascending.
        3531  VmaVector< VmaDeviceMemoryBlock*, VmaStlAllocator<VmaDeviceMemoryBlock*> > m_Blocks;
        3532  /* There can be at most one allocation that is completely empty - a
        3533  hysteresis to avoid pessimistic case of alternating creation and destruction
        3534  of a VkDeviceMemory. */
        3535  bool m_HasEmptyBlock;
        3536  VmaDefragmentator* m_pDefragmentator;
        3537 
        3538  // Finds and removes given block from vector.
        3539  void Remove(VmaDeviceMemoryBlock* pBlock);
        3540 
        3541  // Performs single step in sorting m_Blocks. They may not be fully sorted
        3542  // after this call.
        3543  void IncrementallySortBlocks();
        3544 
        3545  VkResult CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex);
        3546 };
        3547 
        3548 struct VmaPool_T
        3549 {
        3550 public:
        3551  VmaBlockVector m_BlockVector;
        3552 
        3553  // Takes ownership.
        3554  VmaPool_T(
        3555  VmaAllocator hAllocator,
        3556  const VmaPoolCreateInfo& createInfo);
        3557  ~VmaPool_T();
        3558 
        3559  VmaBlockVector& GetBlockVector() { return m_BlockVector; }
        3560 
        3561 #if VMA_STATS_STRING_ENABLED
        3562  //void PrintDetailedMap(class VmaStringBuilder& sb);
        3563 #endif
        3564 };
        3565 
        3566 class VmaDefragmentator
        3567 {
        3568  const VmaAllocator m_hAllocator;
        3569  VmaBlockVector* const m_pBlockVector;
        3570  uint32_t m_CurrentFrameIndex;
        3571  VkDeviceSize m_BytesMoved;
        3572  uint32_t m_AllocationsMoved;
        3573 
        3574  struct AllocationInfo
        3575  {
        3576  VmaAllocation m_hAllocation;
        3577  VkBool32* m_pChanged;
        3578 
        3579  AllocationInfo() :
        3580  m_hAllocation(VK_NULL_HANDLE),
        3581  m_pChanged(VMA_NULL)
        3582  {
        3583  }
        3584  };
        3585 
        3586  struct AllocationInfoSizeGreater
        3587  {
        3588  bool operator()(const AllocationInfo& lhs, const AllocationInfo& rhs) const
        3589  {
        3590  return lhs.m_hAllocation->GetSize() > rhs.m_hAllocation->GetSize();
        3591  }
        3592  };
        3593 
        3594  // Used between AddAllocation and Defragment.
        3595  VmaVector< AllocationInfo, VmaStlAllocator<AllocationInfo> > m_Allocations;
        3596 
        3597  struct BlockInfo
        3598  {
        3599  VmaDeviceMemoryBlock* m_pBlock;
        3600  bool m_HasNonMovableAllocations;
        3601  VmaVector< AllocationInfo, VmaStlAllocator<AllocationInfo> > m_Allocations;
        3602 
        3603  BlockInfo(const VkAllocationCallbacks* pAllocationCallbacks) :
        3604  m_pBlock(VMA_NULL),
        3605  m_HasNonMovableAllocations(true),
        3606  m_Allocations(pAllocationCallbacks),
        3607  m_pMappedDataForDefragmentation(VMA_NULL)
        3608  {
        3609  }
        3610 
        3611  void CalcHasNonMovableAllocations()
        3612  {
        3613  const size_t blockAllocCount = m_pBlock->m_Metadata.GetAllocationCount();
        3614  const size_t defragmentAllocCount = m_Allocations.size();
        3615  m_HasNonMovableAllocations = blockAllocCount != defragmentAllocCount;
        3616  }
        3617 
        3618  void SortAllocationsBySizeDescecnding()
        3619  {
        3620  VMA_SORT(m_Allocations.begin(), m_Allocations.end(), AllocationInfoSizeGreater());
        3621  }
        3622 
        3623  VkResult EnsureMapping(VmaAllocator hAllocator, void** ppMappedData);
        3624  void Unmap(VmaAllocator hAllocator);
        3625 
        3626  private:
        3627  // Not null if mapped for defragmentation only, not originally mapped.
        3628  void* m_pMappedDataForDefragmentation;
        3629  };
        3630 
        3631  struct BlockPointerLess
        3632  {
        3633  bool operator()(const BlockInfo* pLhsBlockInfo, const VmaDeviceMemoryBlock* pRhsBlock) const
        3634  {
        3635  return pLhsBlockInfo->m_pBlock < pRhsBlock;
        3636  }
        3637  bool operator()(const BlockInfo* pLhsBlockInfo, const BlockInfo* pRhsBlockInfo) const
        3638  {
        3639  return pLhsBlockInfo->m_pBlock < pRhsBlockInfo->m_pBlock;
        3640  }
        3641  };
        3642 
        3643  // 1. Blocks with some non-movable allocations go first.
        3644  // 2. Blocks with smaller sumFreeSize go first.
        3645  struct BlockInfoCompareMoveDestination
        3646  {
        3647  bool operator()(const BlockInfo* pLhsBlockInfo, const BlockInfo* pRhsBlockInfo) const
        3648  {
        3649  if(pLhsBlockInfo->m_HasNonMovableAllocations && !pRhsBlockInfo->m_HasNonMovableAllocations)
        3650  {
        3651  return true;
        3652  }
        3653  if(!pLhsBlockInfo->m_HasNonMovableAllocations && pRhsBlockInfo->m_HasNonMovableAllocations)
        3654  {
        3655  return false;
        3656  }
        3657  if(pLhsBlockInfo->m_pBlock->m_Metadata.GetSumFreeSize() < pRhsBlockInfo->m_pBlock->m_Metadata.GetSumFreeSize())
        3658  {
        3659  return true;
        3660  }
        3661  return false;
        3662  }
        3663  };
        3664 
        3665  typedef VmaVector< BlockInfo*, VmaStlAllocator<BlockInfo*> > BlockInfoVector;
        3666  BlockInfoVector m_Blocks;
        3667 
        3668  VkResult DefragmentRound(
        3669  VkDeviceSize maxBytesToMove,
        3670  uint32_t maxAllocationsToMove);
        3671 
        3672  static bool MoveMakesSense(
        3673  size_t dstBlockIndex, VkDeviceSize dstOffset,
        3674  size_t srcBlockIndex, VkDeviceSize srcOffset);
        3675 
        3676 public:
        3677  VmaDefragmentator(
        3678  VmaAllocator hAllocator,
        3679  VmaBlockVector* pBlockVector,
        3680  uint32_t currentFrameIndex);
        3681 
        3682  ~VmaDefragmentator();
        3683 
        3684  VkDeviceSize GetBytesMoved() const { return m_BytesMoved; }
        3685  uint32_t GetAllocationsMoved() const { return m_AllocationsMoved; }
        3686 
        3687  void AddAllocation(VmaAllocation hAlloc, VkBool32* pChanged);
        3688 
        3689  VkResult Defragment(
        3690  VkDeviceSize maxBytesToMove,
        3691  uint32_t maxAllocationsToMove);
        3692 };
        3693 
        3694 // Main allocator object.
        3695 struct VmaAllocator_T
        3696 {
        3697  bool m_UseMutex;
        3698  bool m_UseKhrDedicatedAllocation;
        3699  VkDevice m_hDevice;
        3700  bool m_AllocationCallbacksSpecified;
        3701  VkAllocationCallbacks m_AllocationCallbacks;
        3702  VmaDeviceMemoryCallbacks m_DeviceMemoryCallbacks;
        3703 
        3704  // Number of bytes free out of limit, or VK_WHOLE_SIZE if not limit for that heap.
        3705  VkDeviceSize m_HeapSizeLimit[VK_MAX_MEMORY_HEAPS];
        3706  VMA_MUTEX m_HeapSizeLimitMutex;
        3707 
        3708  VkPhysicalDeviceProperties m_PhysicalDeviceProperties;
        3709  VkPhysicalDeviceMemoryProperties m_MemProps;
        3710 
        3711  // Default pools.
        3712  VmaBlockVector* m_pBlockVectors[VK_MAX_MEMORY_TYPES];
        3713 
        3714  // Each vector is sorted by memory (handle value).
        3715  typedef VmaVector< VmaAllocation, VmaStlAllocator<VmaAllocation> > AllocationVectorType;
        3716  AllocationVectorType* m_pDedicatedAllocations[VK_MAX_MEMORY_TYPES];
        3717  VMA_MUTEX m_DedicatedAllocationsMutex[VK_MAX_MEMORY_TYPES];
        3718 
        3719  VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo);
        3720  ~VmaAllocator_T();
        3721 
        3722  const VkAllocationCallbacks* GetAllocationCallbacks() const
        3723  {
        3724  return m_AllocationCallbacksSpecified ? &m_AllocationCallbacks : 0;
        3725  }
        3726  const VmaVulkanFunctions& GetVulkanFunctions() const
        3727  {
        3728  return m_VulkanFunctions;
        3729  }
        3730 
        3731  VkDeviceSize GetBufferImageGranularity() const
        3732  {
        3733  return VMA_MAX(
        3734  static_cast<VkDeviceSize>(VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY),
        3735  m_PhysicalDeviceProperties.limits.bufferImageGranularity);
        3736  }
        3737 
        3738  uint32_t GetMemoryHeapCount() const { return m_MemProps.memoryHeapCount; }
        3739  uint32_t GetMemoryTypeCount() const { return m_MemProps.memoryTypeCount; }
        3740 
        3741  uint32_t MemoryTypeIndexToHeapIndex(uint32_t memTypeIndex) const
        3742  {
        3743  VMA_ASSERT(memTypeIndex < m_MemProps.memoryTypeCount);
        3744  return m_MemProps.memoryTypes[memTypeIndex].heapIndex;
        3745  }
        3746 
        3747  void GetBufferMemoryRequirements(
        3748  VkBuffer hBuffer,
        3749  VkMemoryRequirements& memReq,
        3750  bool& requiresDedicatedAllocation,
        3751  bool& prefersDedicatedAllocation) const;
        3752  void GetImageMemoryRequirements(
        3753  VkImage hImage,
        3754  VkMemoryRequirements& memReq,
        3755  bool& requiresDedicatedAllocation,
        3756  bool& prefersDedicatedAllocation) const;
        3757 
        3758  // Main allocation function.
        3759  VkResult AllocateMemory(
        3760  const VkMemoryRequirements& vkMemReq,
        3761  bool requiresDedicatedAllocation,
        3762  bool prefersDedicatedAllocation,
        3763  VkBuffer dedicatedBuffer,
        3764  VkImage dedicatedImage,
        3765  const VmaAllocationCreateInfo& createInfo,
        3766  VmaSuballocationType suballocType,
        3767  VmaAllocation* pAllocation);
        3768 
        3769  // Main deallocation function.
        3770  void FreeMemory(const VmaAllocation allocation);
        3771 
        3772  void CalculateStats(VmaStats* pStats);
        3773 
        3774 #if VMA_STATS_STRING_ENABLED
        3775  void PrintDetailedMap(class VmaJsonWriter& json);
        3776 #endif
        3777 
        3778  VkResult Defragment(
        3779  VmaAllocation* pAllocations,
        3780  size_t allocationCount,
        3781  VkBool32* pAllocationsChanged,
        3782  const VmaDefragmentationInfo* pDefragmentationInfo,
        3783  VmaDefragmentationStats* pDefragmentationStats);
        3784 
        3785  void GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo);
        3786 
        3787  VkResult CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool);
        3788  void DestroyPool(VmaPool pool);
        3789  void GetPoolStats(VmaPool pool, VmaPoolStats* pPoolStats);
        3790 
        3791  void SetCurrentFrameIndex(uint32_t frameIndex);
        3792 
        3793  void MakePoolAllocationsLost(
        3794  VmaPool hPool,
        3795  size_t* pLostAllocationCount);
        3796 
        3797  void CreateLostAllocation(VmaAllocation* pAllocation);
        3798 
        3799  VkResult AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory);
        3800  void FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory);
        3801 
        3802  VkResult Map(VmaAllocation hAllocation, void** ppData);
        3803  void Unmap(VmaAllocation hAllocation);
        3804 
        3805 private:
        3806  VkDeviceSize m_PreferredLargeHeapBlockSize;
        3807  VkDeviceSize m_PreferredSmallHeapBlockSize;
        3808 
        3809  VkPhysicalDevice m_PhysicalDevice;
        3810  VMA_ATOMIC_UINT32 m_CurrentFrameIndex;
        3811 
        3812  VMA_MUTEX m_PoolsMutex;
        3813  // Protected by m_PoolsMutex. Sorted by pointer value.
        3814  VmaVector<VmaPool, VmaStlAllocator<VmaPool> > m_Pools;
        3815 
        3816  VmaVulkanFunctions m_VulkanFunctions;
        3817 
        3818  void ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions);
        3819 
        3820  VkDeviceSize CalcPreferredBlockSize(uint32_t memTypeIndex);
        3821 
        3822  VkResult AllocateMemoryOfType(
        3823  const VkMemoryRequirements& vkMemReq,
        3824  bool dedicatedAllocation,
        3825  VkBuffer dedicatedBuffer,
        3826  VkImage dedicatedImage,
        3827  const VmaAllocationCreateInfo& createInfo,
        3828  uint32_t memTypeIndex,
        3829  VmaSuballocationType suballocType,
        3830  VmaAllocation* pAllocation);
        3831 
        3832  // Allocates and registers new VkDeviceMemory specifically for single allocation.
        3833  VkResult AllocateDedicatedMemory(
        3834  VkDeviceSize size,
        3835  VmaSuballocationType suballocType,
        3836  uint32_t memTypeIndex,
        3837  bool map,
        3838  bool isUserDataString,
        3839  void* pUserData,
        3840  VkBuffer dedicatedBuffer,
        3841  VkImage dedicatedImage,
        3842  VmaAllocation* pAllocation);
        3843 
        3844  // Tries to free pMemory as Dedicated Memory. Returns true if found and freed.
        3845  void FreeDedicatedMemory(VmaAllocation allocation);
        3846 };
        3847 
        3849 // Memory allocation #2 after VmaAllocator_T definition
        3850 
        3851 static void* VmaMalloc(VmaAllocator hAllocator, size_t size, size_t alignment)
        3852 {
        3853  return VmaMalloc(&hAllocator->m_AllocationCallbacks, size, alignment);
        3854 }
        3855 
        3856 static void VmaFree(VmaAllocator hAllocator, void* ptr)
        3857 {
        3858  VmaFree(&hAllocator->m_AllocationCallbacks, ptr);
        3859 }
        3860 
        3861 template<typename T>
        3862 static T* VmaAllocate(VmaAllocator hAllocator)
        3863 {
        3864  return (T*)VmaMalloc(hAllocator, sizeof(T), VMA_ALIGN_OF(T));
        3865 }
        3866 
        3867 template<typename T>
        3868 static T* VmaAllocateArray(VmaAllocator hAllocator, size_t count)
        3869 {
        3870  return (T*)VmaMalloc(hAllocator, sizeof(T) * count, VMA_ALIGN_OF(T));
        3871 }
        3872 
        3873 template<typename T>
        3874 static void vma_delete(VmaAllocator hAllocator, T* ptr)
        3875 {
        3876  if(ptr != VMA_NULL)
        3877  {
        3878  ptr->~T();
        3879  VmaFree(hAllocator, ptr);
        3880  }
        3881 }
        3882 
        3883 template<typename T>
        3884 static void vma_delete_array(VmaAllocator hAllocator, T* ptr, size_t count)
        3885 {
        3886  if(ptr != VMA_NULL)
        3887  {
        3888  for(size_t i = count; i--; )
        3889  ptr[i].~T();
        3890  VmaFree(hAllocator, ptr);
        3891  }
        3892 }
        3893 
        3895 // VmaStringBuilder
        3896 
        3897 #if VMA_STATS_STRING_ENABLED
        3898 
        3899 class VmaStringBuilder
        3900 {
        3901 public:
        3902  VmaStringBuilder(VmaAllocator alloc) : m_Data(VmaStlAllocator<char>(alloc->GetAllocationCallbacks())) { }
        3903  size_t GetLength() const { return m_Data.size(); }
        3904  const char* GetData() const { return m_Data.data(); }
        3905 
        3906  void Add(char ch) { m_Data.push_back(ch); }
        3907  void Add(const char* pStr);
        3908  void AddNewLine() { Add('\n'); }
        3909  void AddNumber(uint32_t num);
        3910  void AddNumber(uint64_t num);
        3911  void AddPointer(const void* ptr);
        3912 
        3913 private:
        3914  VmaVector< char, VmaStlAllocator<char> > m_Data;
        3915 };
        3916 
        3917 void VmaStringBuilder::Add(const char* pStr)
        3918 {
        3919  const size_t strLen = strlen(pStr);
        3920  if(strLen > 0)
        3921  {
        3922  const size_t oldCount = m_Data.size();
        3923  m_Data.resize(oldCount + strLen);
        3924  memcpy(m_Data.data() + oldCount, pStr, strLen);
        3925  }
        3926 }
        3927 
        3928 void VmaStringBuilder::AddNumber(uint32_t num)
        3929 {
        3930  char buf[11];
        3931  VmaUint32ToStr(buf, sizeof(buf), num);
        3932  Add(buf);
        3933 }
        3934 
        3935 void VmaStringBuilder::AddNumber(uint64_t num)
        3936 {
        3937  char buf[21];
        3938  VmaUint64ToStr(buf, sizeof(buf), num);
        3939  Add(buf);
        3940 }
        3941 
        3942 void VmaStringBuilder::AddPointer(const void* ptr)
        3943 {
        3944  char buf[21];
        3945  VmaPtrToStr(buf, sizeof(buf), ptr);
        3946  Add(buf);
        3947 }
        3948 
        3949 #endif // #if VMA_STATS_STRING_ENABLED
        3950 
        3952 // VmaJsonWriter
        3953 
        3954 #if VMA_STATS_STRING_ENABLED
        3955 
        3956 class VmaJsonWriter
        3957 {
        3958 public:
        3959  VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb);
        3960  ~VmaJsonWriter();
        3961 
        3962  void BeginObject(bool singleLine = false);
        3963  void EndObject();
        3964 
        3965  void BeginArray(bool singleLine = false);
        3966  void EndArray();
        3967 
        3968  void WriteString(const char* pStr);
        3969  void BeginString(const char* pStr = VMA_NULL);
        3970  void ContinueString(const char* pStr);
        3971  void ContinueString(uint32_t n);
        3972  void ContinueString(uint64_t n);
        3973  void ContinueString_Pointer(const void* ptr);
        3974  void EndString(const char* pStr = VMA_NULL);
        3975 
        3976  void WriteNumber(uint32_t n);
        3977  void WriteNumber(uint64_t n);
        3978  void WriteBool(bool b);
        3979  void WriteNull();
        3980 
        3981 private:
        3982  static const char* const INDENT;
        3983 
        3984  enum COLLECTION_TYPE
        3985  {
        3986  COLLECTION_TYPE_OBJECT,
        3987  COLLECTION_TYPE_ARRAY,
        3988  };
        3989  struct StackItem
        3990  {
        3991  COLLECTION_TYPE type;
        3992  uint32_t valueCount;
        3993  bool singleLineMode;
        3994  };
        3995 
        3996  VmaStringBuilder& m_SB;
        3997  VmaVector< StackItem, VmaStlAllocator<StackItem> > m_Stack;
        3998  bool m_InsideString;
        3999 
        4000  void BeginValue(bool isString);
        4001  void WriteIndent(bool oneLess = false);
        4002 };
        4003 
        4004 const char* const VmaJsonWriter::INDENT = " ";
        4005 
        4006 VmaJsonWriter::VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb) :
        4007  m_SB(sb),
        4008  m_Stack(VmaStlAllocator<StackItem>(pAllocationCallbacks)),
        4009  m_InsideString(false)
        4010 {
        4011 }
        4012 
        4013 VmaJsonWriter::~VmaJsonWriter()
        4014 {
        4015  VMA_ASSERT(!m_InsideString);
        4016  VMA_ASSERT(m_Stack.empty());
        4017 }
        4018 
        4019 void VmaJsonWriter::BeginObject(bool singleLine)
        4020 {
        4021  VMA_ASSERT(!m_InsideString);
        4022 
        4023  BeginValue(false);
        4024  m_SB.Add('{');
        4025 
        4026  StackItem item;
        4027  item.type = COLLECTION_TYPE_OBJECT;
        4028  item.valueCount = 0;
        4029  item.singleLineMode = singleLine;
        4030  m_Stack.push_back(item);
        4031 }
        4032 
        4033 void VmaJsonWriter::EndObject()
        4034 {
        4035  VMA_ASSERT(!m_InsideString);
        4036 
        4037  WriteIndent(true);
        4038  m_SB.Add('}');
        4039 
        4040  VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_OBJECT);
        4041  m_Stack.pop_back();
        4042 }
        4043 
        4044 void VmaJsonWriter::BeginArray(bool singleLine)
        4045 {
        4046  VMA_ASSERT(!m_InsideString);
        4047 
        4048  BeginValue(false);
        4049  m_SB.Add('[');
        4050 
        4051  StackItem item;
        4052  item.type = COLLECTION_TYPE_ARRAY;
        4053  item.valueCount = 0;
        4054  item.singleLineMode = singleLine;
        4055  m_Stack.push_back(item);
        4056 }
        4057 
        4058 void VmaJsonWriter::EndArray()
        4059 {
        4060  VMA_ASSERT(!m_InsideString);
        4061 
        4062  WriteIndent(true);
        4063  m_SB.Add(']');
        4064 
        4065  VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_ARRAY);
        4066  m_Stack.pop_back();
        4067 }
        4068 
        4069 void VmaJsonWriter::WriteString(const char* pStr)
        4070 {
        4071  BeginString(pStr);
        4072  EndString();
        4073 }
        4074 
        4075 void VmaJsonWriter::BeginString(const char* pStr)
        4076 {
        4077  VMA_ASSERT(!m_InsideString);
        4078 
        4079  BeginValue(true);
        4080  m_SB.Add('"');
        4081  m_InsideString = true;
        4082  if(pStr != VMA_NULL && pStr[0] != '\0')
        4083  {
        4084  ContinueString(pStr);
        4085  }
        4086 }
        4087 
        4088 void VmaJsonWriter::ContinueString(const char* pStr)
        4089 {
        4090  VMA_ASSERT(m_InsideString);
        4091 
        4092  const size_t strLen = strlen(pStr);
        4093  for(size_t i = 0; i < strLen; ++i)
        4094  {
        4095  char ch = pStr[i];
        4096  if(ch == '\'')
        4097  {
        4098  m_SB.Add("\\\\");
        4099  }
        4100  else if(ch == '"')
        4101  {
        4102  m_SB.Add("\\\"");
        4103  }
        4104  else if(ch >= 32)
        4105  {
        4106  m_SB.Add(ch);
        4107  }
        4108  else switch(ch)
        4109  {
        4110  case '\b':
        4111  m_SB.Add("\\b");
        4112  break;
        4113  case '\f':
        4114  m_SB.Add("\\f");
        4115  break;
        4116  case '\n':
        4117  m_SB.Add("\\n");
        4118  break;
        4119  case '\r':
        4120  m_SB.Add("\\r");
        4121  break;
        4122  case '\t':
        4123  m_SB.Add("\\t");
        4124  break;
        4125  default:
        4126  VMA_ASSERT(0 && "Character not currently supported.");
        4127  break;
        4128  }
        4129  }
        4130 }
        4131 
        4132 void VmaJsonWriter::ContinueString(uint32_t n)
        4133 {
        4134  VMA_ASSERT(m_InsideString);
        4135  m_SB.AddNumber(n);
        4136 }
        4137 
        4138 void VmaJsonWriter::ContinueString(uint64_t n)
        4139 {
        4140  VMA_ASSERT(m_InsideString);
        4141  m_SB.AddNumber(n);
        4142 }
        4143 
        4144 void VmaJsonWriter::ContinueString_Pointer(const void* ptr)
        4145 {
        4146  VMA_ASSERT(m_InsideString);
        4147  m_SB.AddPointer(ptr);
        4148 }
        4149 
        4150 void VmaJsonWriter::EndString(const char* pStr)
        4151 {
        4152  VMA_ASSERT(m_InsideString);
        4153  if(pStr != VMA_NULL && pStr[0] != '\0')
        4154  {
        4155  ContinueString(pStr);
        4156  }
        4157  m_SB.Add('"');
        4158  m_InsideString = false;
        4159 }
        4160 
        4161 void VmaJsonWriter::WriteNumber(uint32_t n)
        4162 {
        4163  VMA_ASSERT(!m_InsideString);
        4164  BeginValue(false);
        4165  m_SB.AddNumber(n);
        4166 }
        4167 
        4168 void VmaJsonWriter::WriteNumber(uint64_t n)
        4169 {
        4170  VMA_ASSERT(!m_InsideString);
        4171  BeginValue(false);
        4172  m_SB.AddNumber(n);
        4173 }
        4174 
        4175 void VmaJsonWriter::WriteBool(bool b)
        4176 {
        4177  VMA_ASSERT(!m_InsideString);
        4178  BeginValue(false);
        4179  m_SB.Add(b ? "true" : "false");
        4180 }
        4181 
        4182 void VmaJsonWriter::WriteNull()
        4183 {
        4184  VMA_ASSERT(!m_InsideString);
        4185  BeginValue(false);
        4186  m_SB.Add("null");
        4187 }
        4188 
        4189 void VmaJsonWriter::BeginValue(bool isString)
        4190 {
        4191  if(!m_Stack.empty())
        4192  {
        4193  StackItem& currItem = m_Stack.back();
        4194  if(currItem.type == COLLECTION_TYPE_OBJECT &&
        4195  currItem.valueCount % 2 == 0)
        4196  {
        4197  VMA_ASSERT(isString);
        4198  }
        4199 
        4200  if(currItem.type == COLLECTION_TYPE_OBJECT &&
        4201  currItem.valueCount % 2 != 0)
        4202  {
        4203  m_SB.Add(": ");
        4204  }
        4205  else if(currItem.valueCount > 0)
        4206  {
        4207  m_SB.Add(", ");
        4208  WriteIndent();
        4209  }
        4210  else
        4211  {
        4212  WriteIndent();
        4213  }
        4214  ++currItem.valueCount;
        4215  }
        4216 }
        4217 
        4218 void VmaJsonWriter::WriteIndent(bool oneLess)
        4219 {
        4220  if(!m_Stack.empty() && !m_Stack.back().singleLineMode)
        4221  {
        4222  m_SB.AddNewLine();
        4223 
        4224  size_t count = m_Stack.size();
        4225  if(count > 0 && oneLess)
        4226  {
        4227  --count;
        4228  }
        4229  for(size_t i = 0; i < count; ++i)
        4230  {
        4231  m_SB.Add(INDENT);
        4232  }
        4233  }
        4234 }
        4235 
        4236 #endif // #if VMA_STATS_STRING_ENABLED
        4237 
        4239 
        4240 void VmaAllocation_T::SetUserData(VmaAllocator hAllocator, void* pUserData)
        4241 {
        4242  if(IsUserDataString())
        4243  {
        4244  VMA_ASSERT(pUserData == VMA_NULL || pUserData != m_pUserData);
        4245 
        4246  FreeUserDataString(hAllocator);
        4247 
        4248  if(pUserData != VMA_NULL)
        4249  {
        4250  const char* const newStrSrc = (char*)pUserData;
        4251  const size_t newStrLen = strlen(newStrSrc);
        4252  char* const newStrDst = vma_new_array(hAllocator, char, newStrLen + 1);
        4253  memcpy(newStrDst, newStrSrc, newStrLen + 1);
        4254  m_pUserData = newStrDst;
        4255  }
        4256  }
        4257  else
        4258  {
        4259  m_pUserData = pUserData;
        4260  }
        4261 }
        4262 
        4263 VkDeviceSize VmaAllocation_T::GetOffset() const
        4264 {
        4265  switch(m_Type)
        4266  {
        4267  case ALLOCATION_TYPE_BLOCK:
        4268  return m_BlockAllocation.m_Offset;
        4269  case ALLOCATION_TYPE_DEDICATED:
        4270  return 0;
        4271  default:
        4272  VMA_ASSERT(0);
        4273  return 0;
        4274  }
        4275 }
        4276 
        4277 VkDeviceMemory VmaAllocation_T::GetMemory() const
        4278 {
        4279  switch(m_Type)
        4280  {
        4281  case ALLOCATION_TYPE_BLOCK:
        4282  return m_BlockAllocation.m_Block->m_hMemory;
        4283  case ALLOCATION_TYPE_DEDICATED:
        4284  return m_DedicatedAllocation.m_hMemory;
        4285  default:
        4286  VMA_ASSERT(0);
        4287  return VK_NULL_HANDLE;
        4288  }
        4289 }
        4290 
        4291 uint32_t VmaAllocation_T::GetMemoryTypeIndex() const
        4292 {
        4293  switch(m_Type)
        4294  {
        4295  case ALLOCATION_TYPE_BLOCK:
        4296  return m_BlockAllocation.m_Block->m_MemoryTypeIndex;
        4297  case ALLOCATION_TYPE_DEDICATED:
        4298  return m_DedicatedAllocation.m_MemoryTypeIndex;
        4299  default:
        4300  VMA_ASSERT(0);
        4301  return UINT32_MAX;
        4302  }
        4303 }
        4304 
        4305 void* VmaAllocation_T::GetMappedData() const
        4306 {
        4307  switch(m_Type)
        4308  {
        4309  case ALLOCATION_TYPE_BLOCK:
        4310  if(m_MapCount != 0)
        4311  {
        4312  void* pBlockData = m_BlockAllocation.m_Block->m_Mapping.GetMappedData();
        4313  VMA_ASSERT(pBlockData != VMA_NULL);
        4314  return (char*)pBlockData + m_BlockAllocation.m_Offset;
        4315  }
        4316  else
        4317  {
        4318  return VMA_NULL;
        4319  }
        4320  break;
        4321  case ALLOCATION_TYPE_DEDICATED:
        4322  VMA_ASSERT((m_DedicatedAllocation.m_pMappedData != VMA_NULL) == (m_MapCount != 0));
        4323  return m_DedicatedAllocation.m_pMappedData;
        4324  default:
        4325  VMA_ASSERT(0);
        4326  return VMA_NULL;
        4327  }
        4328 }
        4329 
        4330 bool VmaAllocation_T::CanBecomeLost() const
        4331 {
        4332  switch(m_Type)
        4333  {
        4334  case ALLOCATION_TYPE_BLOCK:
        4335  return m_BlockAllocation.m_CanBecomeLost;
        4336  case ALLOCATION_TYPE_DEDICATED:
        4337  return false;
        4338  default:
        4339  VMA_ASSERT(0);
        4340  return false;
        4341  }
        4342 }
        4343 
        4344 VmaPool VmaAllocation_T::GetPool() const
        4345 {
        4346  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
        4347  return m_BlockAllocation.m_hPool;
        4348 }
        4349 
        4350 bool VmaAllocation_T::MakeLost(uint32_t currentFrameIndex, uint32_t frameInUseCount)
        4351 {
        4352  VMA_ASSERT(CanBecomeLost());
        4353 
        4354  /*
        4355  Warning: This is a carefully designed algorithm.
        4356  Do not modify unless you really know what you're doing :)
        4357  */
        4358  uint32_t localLastUseFrameIndex = GetLastUseFrameIndex();
        4359  for(;;)
        4360  {
        4361  if(localLastUseFrameIndex == VMA_FRAME_INDEX_LOST)
        4362  {
        4363  VMA_ASSERT(0);
        4364  return false;
        4365  }
        4366  else if(localLastUseFrameIndex + frameInUseCount >= currentFrameIndex)
        4367  {
        4368  return false;
        4369  }
        4370  else // Last use time earlier than current time.
        4371  {
        4372  if(CompareExchangeLastUseFrameIndex(localLastUseFrameIndex, VMA_FRAME_INDEX_LOST))
        4373  {
        4374  // Setting hAllocation.LastUseFrameIndex atomic to VMA_FRAME_INDEX_LOST is enough to mark it as LOST.
        4375  // Calling code just needs to unregister this allocation in owning VmaDeviceMemoryBlock.
        4376  return true;
        4377  }
        4378  }
        4379  }
        4380 }
        4381 
        4382 void VmaAllocation_T::FreeUserDataString(VmaAllocator hAllocator)
        4383 {
        4384  VMA_ASSERT(IsUserDataString());
        4385  if(m_pUserData != VMA_NULL)
        4386  {
        4387  char* const oldStr = (char*)m_pUserData;
        4388  const size_t oldStrLen = strlen(oldStr);
        4389  vma_delete_array(hAllocator, oldStr, oldStrLen + 1);
        4390  m_pUserData = VMA_NULL;
        4391  }
        4392 }
        4393 
        4394 void VmaAllocation_T::BlockAllocMap()
        4395 {
        4396  VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
        4397 
        4398  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) < 0x7F)
        4399  {
        4400  ++m_MapCount;
        4401  }
        4402  else
        4403  {
        4404  VMA_ASSERT(0 && "Allocation mapped too many times simultaneously.");
        4405  }
        4406 }
        4407 
        4408 void VmaAllocation_T::BlockAllocUnmap()
        4409 {
        4410  VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
        4411 
        4412  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) != 0)
        4413  {
        4414  --m_MapCount;
        4415  }
        4416  else
        4417  {
        4418  VMA_ASSERT(0 && "Unmapping allocation not previously mapped.");
        4419  }
        4420 }
        4421 
        4422 VkResult VmaAllocation_T::DedicatedAllocMap(VmaAllocator hAllocator, void** ppData)
        4423 {
        4424  VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
        4425 
        4426  if(m_MapCount != 0)
        4427  {
        4428  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) < 0x7F)
        4429  {
        4430  VMA_ASSERT(m_DedicatedAllocation.m_pMappedData != VMA_NULL);
        4431  *ppData = m_DedicatedAllocation.m_pMappedData;
        4432  ++m_MapCount;
        4433  return VK_SUCCESS;
        4434  }
        4435  else
        4436  {
        4437  VMA_ASSERT(0 && "Dedicated allocation mapped too many times simultaneously.");
        4438  return VK_ERROR_MEMORY_MAP_FAILED;
        4439  }
        4440  }
        4441  else
        4442  {
        4443  VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
        4444  hAllocator->m_hDevice,
        4445  m_DedicatedAllocation.m_hMemory,
        4446  0, // offset
        4447  VK_WHOLE_SIZE,
        4448  0, // flags
        4449  ppData);
        4450  if(result == VK_SUCCESS)
        4451  {
        4452  m_DedicatedAllocation.m_pMappedData = *ppData;
        4453  m_MapCount = 1;
        4454  }
        4455  return result;
        4456  }
        4457 }
        4458 
        4459 void VmaAllocation_T::DedicatedAllocUnmap(VmaAllocator hAllocator)
        4460 {
        4461  VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
        4462 
        4463  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) != 0)
        4464  {
        4465  --m_MapCount;
        4466  if(m_MapCount == 0)
        4467  {
        4468  m_DedicatedAllocation.m_pMappedData = VMA_NULL;
        4469  (*hAllocator->GetVulkanFunctions().vkUnmapMemory)(
        4470  hAllocator->m_hDevice,
        4471  m_DedicatedAllocation.m_hMemory);
        4472  }
        4473  }
        4474  else
        4475  {
        4476  VMA_ASSERT(0 && "Unmapping dedicated allocation not previously mapped.");
        4477  }
        4478 }
        4479 
        4480 #if VMA_STATS_STRING_ENABLED
        4481 
        4482 // Correspond to values of enum VmaSuballocationType.
        4483 static const char* VMA_SUBALLOCATION_TYPE_NAMES[] = {
        4484  "FREE",
        4485  "UNKNOWN",
        4486  "BUFFER",
        4487  "IMAGE_UNKNOWN",
        4488  "IMAGE_LINEAR",
        4489  "IMAGE_OPTIMAL",
        4490 };
        4491 
        4492 static void VmaPrintStatInfo(VmaJsonWriter& json, const VmaStatInfo& stat)
        4493 {
        4494  json.BeginObject();
        4495 
        4496  json.WriteString("Blocks");
        4497  json.WriteNumber(stat.blockCount);
        4498 
        4499  json.WriteString("Allocations");
        4500  json.WriteNumber(stat.allocationCount);
        4501 
        4502  json.WriteString("UnusedRanges");
        4503  json.WriteNumber(stat.unusedRangeCount);
        4504 
        4505  json.WriteString("UsedBytes");
        4506  json.WriteNumber(stat.usedBytes);
        4507 
        4508  json.WriteString("UnusedBytes");
        4509  json.WriteNumber(stat.unusedBytes);
        4510 
        4511  if(stat.allocationCount > 1)
        4512  {
        4513  json.WriteString("AllocationSize");
        4514  json.BeginObject(true);
        4515  json.WriteString("Min");
        4516  json.WriteNumber(stat.allocationSizeMin);
        4517  json.WriteString("Avg");
        4518  json.WriteNumber(stat.allocationSizeAvg);
        4519  json.WriteString("Max");
        4520  json.WriteNumber(stat.allocationSizeMax);
        4521  json.EndObject();
        4522  }
        4523 
        4524  if(stat.unusedRangeCount > 1)
        4525  {
        4526  json.WriteString("UnusedRangeSize");
        4527  json.BeginObject(true);
        4528  json.WriteString("Min");
        4529  json.WriteNumber(stat.unusedRangeSizeMin);
        4530  json.WriteString("Avg");
        4531  json.WriteNumber(stat.unusedRangeSizeAvg);
        4532  json.WriteString("Max");
        4533  json.WriteNumber(stat.unusedRangeSizeMax);
        4534  json.EndObject();
        4535  }
        4536 
        4537  json.EndObject();
        4538 }
        4539 
        4540 #endif // #if VMA_STATS_STRING_ENABLED
        4541 
        4542 struct VmaSuballocationItemSizeLess
        4543 {
        4544  bool operator()(
        4545  const VmaSuballocationList::iterator lhs,
        4546  const VmaSuballocationList::iterator rhs) const
        4547  {
        4548  return lhs->size < rhs->size;
        4549  }
        4550  bool operator()(
        4551  const VmaSuballocationList::iterator lhs,
        4552  VkDeviceSize rhsSize) const
        4553  {
        4554  return lhs->size < rhsSize;
        4555  }
        4556 };
        4557 
        4559 // class VmaBlockMetadata
        4560 
        4561 VmaBlockMetadata::VmaBlockMetadata(VmaAllocator hAllocator) :
        4562  m_Size(0),
        4563  m_FreeCount(0),
        4564  m_SumFreeSize(0),
        4565  m_Suballocations(VmaStlAllocator<VmaSuballocation>(hAllocator->GetAllocationCallbacks())),
        4566  m_FreeSuballocationsBySize(VmaStlAllocator<VmaSuballocationList::iterator>(hAllocator->GetAllocationCallbacks()))
        4567 {
        4568 }
        4569 
        4570 VmaBlockMetadata::~VmaBlockMetadata()
        4571 {
        4572 }
        4573 
        4574 void VmaBlockMetadata::Init(VkDeviceSize size)
        4575 {
        4576  m_Size = size;
        4577  m_FreeCount = 1;
        4578  m_SumFreeSize = size;
        4579 
        4580  VmaSuballocation suballoc = {};
        4581  suballoc.offset = 0;
        4582  suballoc.size = size;
        4583  suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
        4584  suballoc.hAllocation = VK_NULL_HANDLE;
        4585 
        4586  m_Suballocations.push_back(suballoc);
        4587  VmaSuballocationList::iterator suballocItem = m_Suballocations.end();
        4588  --suballocItem;
        4589  m_FreeSuballocationsBySize.push_back(suballocItem);
        4590 }
        4591 
        4592 bool VmaBlockMetadata::Validate() const
        4593 {
        4594  if(m_Suballocations.empty())
        4595  {
        4596  return false;
        4597  }
        4598 
        4599  // Expected offset of new suballocation as calculates from previous ones.
        4600  VkDeviceSize calculatedOffset = 0;
        4601  // Expected number of free suballocations as calculated from traversing their list.
        4602  uint32_t calculatedFreeCount = 0;
        4603  // Expected sum size of free suballocations as calculated from traversing their list.
        4604  VkDeviceSize calculatedSumFreeSize = 0;
        4605  // Expected number of free suballocations that should be registered in
        4606  // m_FreeSuballocationsBySize calculated from traversing their list.
        4607  size_t freeSuballocationsToRegister = 0;
        4608  // True if previous visisted suballocation was free.
        4609  bool prevFree = false;
        4610 
        4611  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
        4612  suballocItem != m_Suballocations.cend();
        4613  ++suballocItem)
        4614  {
        4615  const VmaSuballocation& subAlloc = *suballocItem;
        4616 
        4617  // Actual offset of this suballocation doesn't match expected one.
        4618  if(subAlloc.offset != calculatedOffset)
        4619  {
        4620  return false;
        4621  }
        4622 
        4623  const bool currFree = (subAlloc.type == VMA_SUBALLOCATION_TYPE_FREE);
        4624  // Two adjacent free suballocations are invalid. They should be merged.
        4625  if(prevFree && currFree)
        4626  {
        4627  return false;
        4628  }
        4629  prevFree = currFree;
        4630 
        4631  if(currFree != (subAlloc.hAllocation == VK_NULL_HANDLE))
        4632  {
        4633  return false;
        4634  }
        4635 
        4636  if(currFree)
        4637  {
        4638  calculatedSumFreeSize += subAlloc.size;
        4639  ++calculatedFreeCount;
        4640  if(subAlloc.size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
        4641  {
        4642  ++freeSuballocationsToRegister;
        4643  }
        4644  }
        4645 
        4646  calculatedOffset += subAlloc.size;
        4647  }
        4648 
        4649  // Number of free suballocations registered in m_FreeSuballocationsBySize doesn't
        4650  // match expected one.
        4651  if(m_FreeSuballocationsBySize.size() != freeSuballocationsToRegister)
        4652  {
        4653  return false;
        4654  }
        4655 
        4656  VkDeviceSize lastSize = 0;
        4657  for(size_t i = 0; i < m_FreeSuballocationsBySize.size(); ++i)
        4658  {
        4659  VmaSuballocationList::iterator suballocItem = m_FreeSuballocationsBySize[i];
        4660 
        4661  // Only free suballocations can be registered in m_FreeSuballocationsBySize.
        4662  if(suballocItem->type != VMA_SUBALLOCATION_TYPE_FREE)
        4663  {
        4664  return false;
        4665  }
        4666  // They must be sorted by size ascending.
        4667  if(suballocItem->size < lastSize)
        4668  {
        4669  return false;
        4670  }
        4671 
        4672  lastSize = suballocItem->size;
        4673  }
        4674 
        4675  // Check if totals match calculacted values.
        4676  return
        4677  ValidateFreeSuballocationList() &&
        4678  (calculatedOffset == m_Size) &&
        4679  (calculatedSumFreeSize == m_SumFreeSize) &&
        4680  (calculatedFreeCount == m_FreeCount);
        4681 }
        4682 
        4683 VkDeviceSize VmaBlockMetadata::GetUnusedRangeSizeMax() const
        4684 {
        4685  if(!m_FreeSuballocationsBySize.empty())
        4686  {
        4687  return m_FreeSuballocationsBySize.back()->size;
        4688  }
        4689  else
        4690  {
        4691  return 0;
        4692  }
        4693 }
        4694 
        4695 bool VmaBlockMetadata::IsEmpty() const
        4696 {
        4697  return (m_Suballocations.size() == 1) && (m_FreeCount == 1);
        4698 }
        4699 
        4700 void VmaBlockMetadata::CalcAllocationStatInfo(VmaStatInfo& outInfo) const
        4701 {
        4702  outInfo.blockCount = 1;
        4703 
        4704  const uint32_t rangeCount = (uint32_t)m_Suballocations.size();
        4705  outInfo.allocationCount = rangeCount - m_FreeCount;
        4706  outInfo.unusedRangeCount = m_FreeCount;
        4707 
        4708  outInfo.unusedBytes = m_SumFreeSize;
        4709  outInfo.usedBytes = m_Size - outInfo.unusedBytes;
        4710 
        4711  outInfo.allocationSizeMin = UINT64_MAX;
        4712  outInfo.allocationSizeMax = 0;
        4713  outInfo.unusedRangeSizeMin = UINT64_MAX;
        4714  outInfo.unusedRangeSizeMax = 0;
        4715 
        4716  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
        4717  suballocItem != m_Suballocations.cend();
        4718  ++suballocItem)
        4719  {
        4720  const VmaSuballocation& suballoc = *suballocItem;
        4721  if(suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
        4722  {
        4723  outInfo.allocationSizeMin = VMA_MIN(outInfo.allocationSizeMin, suballoc.size);
        4724  outInfo.allocationSizeMax = VMA_MAX(outInfo.allocationSizeMax, suballoc.size);
        4725  }
        4726  else
        4727  {
        4728  outInfo.unusedRangeSizeMin = VMA_MIN(outInfo.unusedRangeSizeMin, suballoc.size);
        4729  outInfo.unusedRangeSizeMax = VMA_MAX(outInfo.unusedRangeSizeMax, suballoc.size);
        4730  }
        4731  }
        4732 }
        4733 
        4734 void VmaBlockMetadata::AddPoolStats(VmaPoolStats& inoutStats) const
        4735 {
        4736  const uint32_t rangeCount = (uint32_t)m_Suballocations.size();
        4737 
        4738  inoutStats.size += m_Size;
        4739  inoutStats.unusedSize += m_SumFreeSize;
        4740  inoutStats.allocationCount += rangeCount - m_FreeCount;
        4741  inoutStats.unusedRangeCount += m_FreeCount;
        4742  inoutStats.unusedRangeSizeMax = VMA_MAX(inoutStats.unusedRangeSizeMax, GetUnusedRangeSizeMax());
        4743 }
        4744 
        4745 #if VMA_STATS_STRING_ENABLED
        4746 
        4747 void VmaBlockMetadata::PrintDetailedMap(class VmaJsonWriter& json) const
        4748 {
        4749  json.BeginObject();
        4750 
        4751  json.WriteString("TotalBytes");
        4752  json.WriteNumber(m_Size);
        4753 
        4754  json.WriteString("UnusedBytes");
        4755  json.WriteNumber(m_SumFreeSize);
        4756 
        4757  json.WriteString("Allocations");
        4758  json.WriteNumber(m_Suballocations.size() - m_FreeCount);
        4759 
        4760  json.WriteString("UnusedRanges");
        4761  json.WriteNumber(m_FreeCount);
        4762 
        4763  json.WriteString("Suballocations");
        4764  json.BeginArray();
        4765  size_t i = 0;
        4766  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
        4767  suballocItem != m_Suballocations.cend();
        4768  ++suballocItem, ++i)
        4769  {
        4770  json.BeginObject(true);
        4771 
        4772  json.WriteString("Type");
        4773  json.WriteString(VMA_SUBALLOCATION_TYPE_NAMES[suballocItem->type]);
        4774 
        4775  json.WriteString("Size");
        4776  json.WriteNumber(suballocItem->size);
        4777 
        4778  json.WriteString("Offset");
        4779  json.WriteNumber(suballocItem->offset);
        4780 
        4781  if(suballocItem->type != VMA_SUBALLOCATION_TYPE_FREE)
        4782  {
        4783  const void* pUserData = suballocItem->hAllocation->GetUserData();
        4784  if(pUserData != VMA_NULL)
        4785  {
        4786  json.WriteString("UserData");
        4787  if(suballocItem->hAllocation->IsUserDataString())
        4788  {
        4789  json.WriteString((const char*)pUserData);
        4790  }
        4791  else
        4792  {
        4793  json.BeginString();
        4794  json.ContinueString_Pointer(pUserData);
        4795  json.EndString();
        4796  }
        4797  }
        4798  }
        4799 
        4800  json.EndObject();
        4801  }
        4802  json.EndArray();
        4803 
        4804  json.EndObject();
        4805 }
        4806 
        4807 #endif // #if VMA_STATS_STRING_ENABLED
        4808 
        4809 /*
        4810 How many suitable free suballocations to analyze before choosing best one.
        4811 - Set to 1 to use First-Fit algorithm - first suitable free suballocation will
        4812  be chosen.
        4813 - Set to UINT32_MAX to use Best-Fit/Worst-Fit algorithm - all suitable free
        4814  suballocations will be analized and best one will be chosen.
        4815 - Any other value is also acceptable.
        4816 */
        4817 //static const uint32_t MAX_SUITABLE_SUBALLOCATIONS_TO_CHECK = 8;
        4818 
        4819 void VmaBlockMetadata::CreateFirstAllocationRequest(VmaAllocationRequest* pAllocationRequest)
        4820 {
        4821  VMA_ASSERT(IsEmpty());
        4822  pAllocationRequest->offset = 0;
        4823  pAllocationRequest->sumFreeSize = m_SumFreeSize;
        4824  pAllocationRequest->sumItemSize = 0;
        4825  pAllocationRequest->item = m_Suballocations.begin();
        4826  pAllocationRequest->itemsToMakeLostCount = 0;
        4827 }
        4828 
        4829 bool VmaBlockMetadata::CreateAllocationRequest(
        4830  uint32_t currentFrameIndex,
        4831  uint32_t frameInUseCount,
        4832  VkDeviceSize bufferImageGranularity,
        4833  VkDeviceSize allocSize,
        4834  VkDeviceSize allocAlignment,
        4835  VmaSuballocationType allocType,
        4836  bool canMakeOtherLost,
        4837  VmaAllocationRequest* pAllocationRequest)
        4838 {
        4839  VMA_ASSERT(allocSize > 0);
        4840  VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
        4841  VMA_ASSERT(pAllocationRequest != VMA_NULL);
        4842  VMA_HEAVY_ASSERT(Validate());
        4843 
        4844  // There is not enough total free space in this block to fullfill the request: Early return.
        4845  if(canMakeOtherLost == false && m_SumFreeSize < allocSize)
        4846  {
        4847  return false;
        4848  }
        4849 
        4850  // New algorithm, efficiently searching freeSuballocationsBySize.
        4851  const size_t freeSuballocCount = m_FreeSuballocationsBySize.size();
        4852  if(freeSuballocCount > 0)
        4853  {
        4854  if(VMA_BEST_FIT)
        4855  {
        4856  // Find first free suballocation with size not less than allocSize.
        4857  VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
        4858  m_FreeSuballocationsBySize.data(),
        4859  m_FreeSuballocationsBySize.data() + freeSuballocCount,
        4860  allocSize,
        4861  VmaSuballocationItemSizeLess());
        4862  size_t index = it - m_FreeSuballocationsBySize.data();
        4863  for(; index < freeSuballocCount; ++index)
        4864  {
        4865  if(CheckAllocation(
        4866  currentFrameIndex,
        4867  frameInUseCount,
        4868  bufferImageGranularity,
        4869  allocSize,
        4870  allocAlignment,
        4871  allocType,
        4872  m_FreeSuballocationsBySize[index],
        4873  false, // canMakeOtherLost
        4874  &pAllocationRequest->offset,
        4875  &pAllocationRequest->itemsToMakeLostCount,
        4876  &pAllocationRequest->sumFreeSize,
        4877  &pAllocationRequest->sumItemSize))
        4878  {
        4879  pAllocationRequest->item = m_FreeSuballocationsBySize[index];
        4880  return true;
        4881  }
        4882  }
        4883  }
        4884  else
        4885  {
        4886  // Search staring from biggest suballocations.
        4887  for(size_t index = freeSuballocCount; index--; )
        4888  {
        4889  if(CheckAllocation(
        4890  currentFrameIndex,
        4891  frameInUseCount,
        4892  bufferImageGranularity,
        4893  allocSize,
        4894  allocAlignment,
        4895  allocType,
        4896  m_FreeSuballocationsBySize[index],
        4897  false, // canMakeOtherLost
        4898  &pAllocationRequest->offset,
        4899  &pAllocationRequest->itemsToMakeLostCount,
        4900  &pAllocationRequest->sumFreeSize,
        4901  &pAllocationRequest->sumItemSize))
        4902  {
        4903  pAllocationRequest->item = m_FreeSuballocationsBySize[index];
        4904  return true;
        4905  }
        4906  }
        4907  }
        4908  }
        4909 
        4910  if(canMakeOtherLost)
        4911  {
        4912  // Brute-force algorithm. TODO: Come up with something better.
        4913 
        4914  pAllocationRequest->sumFreeSize = VK_WHOLE_SIZE;
        4915  pAllocationRequest->sumItemSize = VK_WHOLE_SIZE;
        4916 
        4917  VmaAllocationRequest tmpAllocRequest = {};
        4918  for(VmaSuballocationList::iterator suballocIt = m_Suballocations.begin();
        4919  suballocIt != m_Suballocations.end();
        4920  ++suballocIt)
        4921  {
        4922  if(suballocIt->type == VMA_SUBALLOCATION_TYPE_FREE ||
        4923  suballocIt->hAllocation->CanBecomeLost())
        4924  {
        4925  if(CheckAllocation(
        4926  currentFrameIndex,
        4927  frameInUseCount,
        4928  bufferImageGranularity,
        4929  allocSize,
        4930  allocAlignment,
        4931  allocType,
        4932  suballocIt,
        4933  canMakeOtherLost,
        4934  &tmpAllocRequest.offset,
        4935  &tmpAllocRequest.itemsToMakeLostCount,
        4936  &tmpAllocRequest.sumFreeSize,
        4937  &tmpAllocRequest.sumItemSize))
        4938  {
        4939  tmpAllocRequest.item = suballocIt;
        4940 
        4941  if(tmpAllocRequest.CalcCost() < pAllocationRequest->CalcCost())
        4942  {
        4943  *pAllocationRequest = tmpAllocRequest;
        4944  }
        4945  }
        4946  }
        4947  }
        4948 
        4949  if(pAllocationRequest->sumItemSize != VK_WHOLE_SIZE)
        4950  {
        4951  return true;
        4952  }
        4953  }
        4954 
        4955  return false;
        4956 }
        4957 
        4958 bool VmaBlockMetadata::MakeRequestedAllocationsLost(
        4959  uint32_t currentFrameIndex,
        4960  uint32_t frameInUseCount,
        4961  VmaAllocationRequest* pAllocationRequest)
        4962 {
        4963  while(pAllocationRequest->itemsToMakeLostCount > 0)
        4964  {
        4965  if(pAllocationRequest->item->type == VMA_SUBALLOCATION_TYPE_FREE)
        4966  {
        4967  ++pAllocationRequest->item;
        4968  }
        4969  VMA_ASSERT(pAllocationRequest->item != m_Suballocations.end());
        4970  VMA_ASSERT(pAllocationRequest->item->hAllocation != VK_NULL_HANDLE);
        4971  VMA_ASSERT(pAllocationRequest->item->hAllocation->CanBecomeLost());
        4972  if(pAllocationRequest->item->hAllocation->MakeLost(currentFrameIndex, frameInUseCount))
        4973  {
        4974  pAllocationRequest->item = FreeSuballocation(pAllocationRequest->item);
        4975  --pAllocationRequest->itemsToMakeLostCount;
        4976  }
        4977  else
        4978  {
        4979  return false;
        4980  }
        4981  }
        4982 
        4983  VMA_HEAVY_ASSERT(Validate());
        4984  VMA_ASSERT(pAllocationRequest->item != m_Suballocations.end());
        4985  VMA_ASSERT(pAllocationRequest->item->type == VMA_SUBALLOCATION_TYPE_FREE);
        4986 
        4987  return true;
        4988 }
        4989 
        4990 uint32_t VmaBlockMetadata::MakeAllocationsLost(uint32_t currentFrameIndex, uint32_t frameInUseCount)
        4991 {
        4992  uint32_t lostAllocationCount = 0;
        4993  for(VmaSuballocationList::iterator it = m_Suballocations.begin();
        4994  it != m_Suballocations.end();
        4995  ++it)
        4996  {
        4997  if(it->type != VMA_SUBALLOCATION_TYPE_FREE &&
        4998  it->hAllocation->CanBecomeLost() &&
        4999  it->hAllocation->MakeLost(currentFrameIndex, frameInUseCount))
        5000  {
        5001  it = FreeSuballocation(it);
        5002  ++lostAllocationCount;
        5003  }
        5004  }
        5005  return lostAllocationCount;
        5006 }
        5007 
        5008 void VmaBlockMetadata::Alloc(
        5009  const VmaAllocationRequest& request,
        5010  VmaSuballocationType type,
        5011  VkDeviceSize allocSize,
        5012  VmaAllocation hAllocation)
        5013 {
        5014  VMA_ASSERT(request.item != m_Suballocations.end());
        5015  VmaSuballocation& suballoc = *request.item;
        5016  // Given suballocation is a free block.
        5017  VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
        5018  // Given offset is inside this suballocation.
        5019  VMA_ASSERT(request.offset >= suballoc.offset);
        5020  const VkDeviceSize paddingBegin = request.offset - suballoc.offset;
        5021  VMA_ASSERT(suballoc.size >= paddingBegin + allocSize);
        5022  const VkDeviceSize paddingEnd = suballoc.size - paddingBegin - allocSize;
        5023 
        5024  // Unregister this free suballocation from m_FreeSuballocationsBySize and update
        5025  // it to become used.
        5026  UnregisterFreeSuballocation(request.item);
        5027 
        5028  suballoc.offset = request.offset;
        5029  suballoc.size = allocSize;
        5030  suballoc.type = type;
        5031  suballoc.hAllocation = hAllocation;
        5032 
        5033  // If there are any free bytes remaining at the end, insert new free suballocation after current one.
        5034  if(paddingEnd)
        5035  {
        5036  VmaSuballocation paddingSuballoc = {};
        5037  paddingSuballoc.offset = request.offset + allocSize;
        5038  paddingSuballoc.size = paddingEnd;
        5039  paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
        5040  VmaSuballocationList::iterator next = request.item;
        5041  ++next;
        5042  const VmaSuballocationList::iterator paddingEndItem =
        5043  m_Suballocations.insert(next, paddingSuballoc);
        5044  RegisterFreeSuballocation(paddingEndItem);
        5045  }
        5046 
        5047  // If there are any free bytes remaining at the beginning, insert new free suballocation before current one.
        5048  if(paddingBegin)
        5049  {
        5050  VmaSuballocation paddingSuballoc = {};
        5051  paddingSuballoc.offset = request.offset - paddingBegin;
        5052  paddingSuballoc.size = paddingBegin;
        5053  paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
        5054  const VmaSuballocationList::iterator paddingBeginItem =
        5055  m_Suballocations.insert(request.item, paddingSuballoc);
        5056  RegisterFreeSuballocation(paddingBeginItem);
        5057  }
        5058 
        5059  // Update totals.
        5060  m_FreeCount = m_FreeCount - 1;
        5061  if(paddingBegin > 0)
        5062  {
        5063  ++m_FreeCount;
        5064  }
        5065  if(paddingEnd > 0)
        5066  {
        5067  ++m_FreeCount;
        5068  }
        5069  m_SumFreeSize -= allocSize;
        5070 }
        5071 
        5072 void VmaBlockMetadata::Free(const VmaAllocation allocation)
        5073 {
        5074  for(VmaSuballocationList::iterator suballocItem = m_Suballocations.begin();
        5075  suballocItem != m_Suballocations.end();
        5076  ++suballocItem)
        5077  {
        5078  VmaSuballocation& suballoc = *suballocItem;
        5079  if(suballoc.hAllocation == allocation)
        5080  {
        5081  FreeSuballocation(suballocItem);
        5082  VMA_HEAVY_ASSERT(Validate());
        5083  return;
        5084  }
        5085  }
        5086  VMA_ASSERT(0 && "Not found!");
        5087 }
        5088 
        5089 bool VmaBlockMetadata::ValidateFreeSuballocationList() const
        5090 {
        5091  VkDeviceSize lastSize = 0;
        5092  for(size_t i = 0, count = m_FreeSuballocationsBySize.size(); i < count; ++i)
        5093  {
        5094  const VmaSuballocationList::iterator it = m_FreeSuballocationsBySize[i];
        5095 
        5096  if(it->type != VMA_SUBALLOCATION_TYPE_FREE)
        5097  {
        5098  VMA_ASSERT(0);
        5099  return false;
        5100  }
        5101  if(it->size < VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
        5102  {
        5103  VMA_ASSERT(0);
        5104  return false;
        5105  }
        5106  if(it->size < lastSize)
        5107  {
        5108  VMA_ASSERT(0);
        5109  return false;
        5110  }
        5111 
        5112  lastSize = it->size;
        5113  }
        5114  return true;
        5115 }
        5116 
        5117 bool VmaBlockMetadata::CheckAllocation(
        5118  uint32_t currentFrameIndex,
        5119  uint32_t frameInUseCount,
        5120  VkDeviceSize bufferImageGranularity,
        5121  VkDeviceSize allocSize,
        5122  VkDeviceSize allocAlignment,
        5123  VmaSuballocationType allocType,
        5124  VmaSuballocationList::const_iterator suballocItem,
        5125  bool canMakeOtherLost,
        5126  VkDeviceSize* pOffset,
        5127  size_t* itemsToMakeLostCount,
        5128  VkDeviceSize* pSumFreeSize,
        5129  VkDeviceSize* pSumItemSize) const
        5130 {
        5131  VMA_ASSERT(allocSize > 0);
        5132  VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
        5133  VMA_ASSERT(suballocItem != m_Suballocations.cend());
        5134  VMA_ASSERT(pOffset != VMA_NULL);
        5135 
        5136  *itemsToMakeLostCount = 0;
        5137  *pSumFreeSize = 0;
        5138  *pSumItemSize = 0;
        5139 
        5140  if(canMakeOtherLost)
        5141  {
        5142  if(suballocItem->type == VMA_SUBALLOCATION_TYPE_FREE)
        5143  {
        5144  *pSumFreeSize = suballocItem->size;
        5145  }
        5146  else
        5147  {
        5148  if(suballocItem->hAllocation->CanBecomeLost() &&
        5149  suballocItem->hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
        5150  {
        5151  ++*itemsToMakeLostCount;
        5152  *pSumItemSize = suballocItem->size;
        5153  }
        5154  else
        5155  {
        5156  return false;
        5157  }
        5158  }
        5159 
        5160  // Remaining size is too small for this request: Early return.
        5161  if(m_Size - suballocItem->offset < allocSize)
        5162  {
        5163  return false;
        5164  }
        5165 
        5166  // Start from offset equal to beginning of this suballocation.
        5167  *pOffset = suballocItem->offset;
        5168 
        5169  // Apply VMA_DEBUG_MARGIN at the beginning.
        5170  if((VMA_DEBUG_MARGIN > 0) && suballocItem != m_Suballocations.cbegin())
        5171  {
        5172  *pOffset += VMA_DEBUG_MARGIN;
        5173  }
        5174 
        5175  // Apply alignment.
        5176  const VkDeviceSize alignment = VMA_MAX(allocAlignment, static_cast<VkDeviceSize>(VMA_DEBUG_ALIGNMENT));
        5177  *pOffset = VmaAlignUp(*pOffset, alignment);
        5178 
        5179  // Check previous suballocations for BufferImageGranularity conflicts.
        5180  // Make bigger alignment if necessary.
        5181  if(bufferImageGranularity > 1)
        5182  {
        5183  bool bufferImageGranularityConflict = false;
        5184  VmaSuballocationList::const_iterator prevSuballocItem = suballocItem;
        5185  while(prevSuballocItem != m_Suballocations.cbegin())
        5186  {
        5187  --prevSuballocItem;
        5188  const VmaSuballocation& prevSuballoc = *prevSuballocItem;
        5189  if(VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, *pOffset, bufferImageGranularity))
        5190  {
        5191  if(VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
        5192  {
        5193  bufferImageGranularityConflict = true;
        5194  break;
        5195  }
        5196  }
        5197  else
        5198  // Already on previous page.
        5199  break;
        5200  }
        5201  if(bufferImageGranularityConflict)
        5202  {
        5203  *pOffset = VmaAlignUp(*pOffset, bufferImageGranularity);
        5204  }
        5205  }
        5206 
        5207  // Now that we have final *pOffset, check if we are past suballocItem.
        5208  // If yes, return false - this function should be called for another suballocItem as starting point.
        5209  if(*pOffset >= suballocItem->offset + suballocItem->size)
        5210  {
        5211  return false;
        5212  }
        5213 
        5214  // Calculate padding at the beginning based on current offset.
        5215  const VkDeviceSize paddingBegin = *pOffset - suballocItem->offset;
        5216 
        5217  // Calculate required margin at the end if this is not last suballocation.
        5218  VmaSuballocationList::const_iterator next = suballocItem;
        5219  ++next;
        5220  const VkDeviceSize requiredEndMargin =
        5221  (next != m_Suballocations.cend()) ? VMA_DEBUG_MARGIN : 0;
        5222 
        5223  const VkDeviceSize totalSize = paddingBegin + allocSize + requiredEndMargin;
        5224  // Another early return check.
        5225  if(suballocItem->offset + totalSize > m_Size)
        5226  {
        5227  return false;
        5228  }
        5229 
        5230  // Advance lastSuballocItem until desired size is reached.
        5231  // Update itemsToMakeLostCount.
        5232  VmaSuballocationList::const_iterator lastSuballocItem = suballocItem;
        5233  if(totalSize > suballocItem->size)
        5234  {
        5235  VkDeviceSize remainingSize = totalSize - suballocItem->size;
        5236  while(remainingSize > 0)
        5237  {
        5238  ++lastSuballocItem;
        5239  if(lastSuballocItem == m_Suballocations.cend())
        5240  {
        5241  return false;
        5242  }
        5243  if(lastSuballocItem->type == VMA_SUBALLOCATION_TYPE_FREE)
        5244  {
        5245  *pSumFreeSize += lastSuballocItem->size;
        5246  }
        5247  else
        5248  {
        5249  VMA_ASSERT(lastSuballocItem->hAllocation != VK_NULL_HANDLE);
        5250  if(lastSuballocItem->hAllocation->CanBecomeLost() &&
        5251  lastSuballocItem->hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
        5252  {
        5253  ++*itemsToMakeLostCount;
        5254  *pSumItemSize += lastSuballocItem->size;
        5255  }
        5256  else
        5257  {
        5258  return false;
        5259  }
        5260  }
        5261  remainingSize = (lastSuballocItem->size < remainingSize) ?
        5262  remainingSize - lastSuballocItem->size : 0;
        5263  }
        5264  }
        5265 
        5266  // Check next suballocations for BufferImageGranularity conflicts.
        5267  // If conflict exists, we must mark more allocations lost or fail.
        5268  if(bufferImageGranularity > 1)
        5269  {
        5270  VmaSuballocationList::const_iterator nextSuballocItem = lastSuballocItem;
        5271  ++nextSuballocItem;
        5272  while(nextSuballocItem != m_Suballocations.cend())
        5273  {
        5274  const VmaSuballocation& nextSuballoc = *nextSuballocItem;
        5275  if(VmaBlocksOnSamePage(*pOffset, allocSize, nextSuballoc.offset, bufferImageGranularity))
        5276  {
        5277  if(VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
        5278  {
        5279  VMA_ASSERT(nextSuballoc.hAllocation != VK_NULL_HANDLE);
        5280  if(nextSuballoc.hAllocation->CanBecomeLost() &&
        5281  nextSuballoc.hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
        5282  {
        5283  ++*itemsToMakeLostCount;
        5284  }
        5285  else
        5286  {
        5287  return false;
        5288  }
        5289  }
        5290  }
        5291  else
        5292  {
        5293  // Already on next page.
        5294  break;
        5295  }
        5296  ++nextSuballocItem;
        5297  }
        5298  }
        5299  }
        5300  else
        5301  {
        5302  const VmaSuballocation& suballoc = *suballocItem;
        5303  VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
        5304 
        5305  *pSumFreeSize = suballoc.size;
        5306 
        5307  // Size of this suballocation is too small for this request: Early return.
        5308  if(suballoc.size < allocSize)
        5309  {
        5310  return false;
        5311  }
        5312 
        5313  // Start from offset equal to beginning of this suballocation.
        5314  *pOffset = suballoc.offset;
        5315 
        5316  // Apply VMA_DEBUG_MARGIN at the beginning.
        5317  if((VMA_DEBUG_MARGIN > 0) && suballocItem != m_Suballocations.cbegin())
        5318  {
        5319  *pOffset += VMA_DEBUG_MARGIN;
        5320  }
        5321 
        5322  // Apply alignment.
        5323  const VkDeviceSize alignment = VMA_MAX(allocAlignment, static_cast<VkDeviceSize>(VMA_DEBUG_ALIGNMENT));
        5324  *pOffset = VmaAlignUp(*pOffset, alignment);
        5325 
        5326  // Check previous suballocations for BufferImageGranularity conflicts.
        5327  // Make bigger alignment if necessary.
        5328  if(bufferImageGranularity > 1)
        5329  {
        5330  bool bufferImageGranularityConflict = false;
        5331  VmaSuballocationList::const_iterator prevSuballocItem = suballocItem;
        5332  while(prevSuballocItem != m_Suballocations.cbegin())
        5333  {
        5334  --prevSuballocItem;
        5335  const VmaSuballocation& prevSuballoc = *prevSuballocItem;
        5336  if(VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, *pOffset, bufferImageGranularity))
        5337  {
        5338  if(VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
        5339  {
        5340  bufferImageGranularityConflict = true;
        5341  break;
        5342  }
        5343  }
        5344  else
        5345  // Already on previous page.
        5346  break;
        5347  }
        5348  if(bufferImageGranularityConflict)
        5349  {
        5350  *pOffset = VmaAlignUp(*pOffset, bufferImageGranularity);
        5351  }
        5352  }
        5353 
        5354  // Calculate padding at the beginning based on current offset.
        5355  const VkDeviceSize paddingBegin = *pOffset - suballoc.offset;
        5356 
        5357  // Calculate required margin at the end if this is not last suballocation.
        5358  VmaSuballocationList::const_iterator next = suballocItem;
        5359  ++next;
        5360  const VkDeviceSize requiredEndMargin =
        5361  (next != m_Suballocations.cend()) ? VMA_DEBUG_MARGIN : 0;
        5362 
        5363  // Fail if requested size plus margin before and after is bigger than size of this suballocation.
        5364  if(paddingBegin + allocSize + requiredEndMargin > suballoc.size)
        5365  {
        5366  return false;
        5367  }
        5368 
        5369  // Check next suballocations for BufferImageGranularity conflicts.
        5370  // If conflict exists, allocation cannot be made here.
        5371  if(bufferImageGranularity > 1)
        5372  {
        5373  VmaSuballocationList::const_iterator nextSuballocItem = suballocItem;
        5374  ++nextSuballocItem;
        5375  while(nextSuballocItem != m_Suballocations.cend())
        5376  {
        5377  const VmaSuballocation& nextSuballoc = *nextSuballocItem;
        5378  if(VmaBlocksOnSamePage(*pOffset, allocSize, nextSuballoc.offset, bufferImageGranularity))
        5379  {
        5380  if(VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
        5381  {
        5382  return false;
        5383  }
        5384  }
        5385  else
        5386  {
        5387  // Already on next page.
        5388  break;
        5389  }
        5390  ++nextSuballocItem;
        5391  }
        5392  }
        5393  }
        5394 
        5395  // All tests passed: Success. pOffset is already filled.
        5396  return true;
        5397 }
        5398 
        5399 void VmaBlockMetadata::MergeFreeWithNext(VmaSuballocationList::iterator item)
        5400 {
        5401  VMA_ASSERT(item != m_Suballocations.end());
        5402  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
        5403 
        5404  VmaSuballocationList::iterator nextItem = item;
        5405  ++nextItem;
        5406  VMA_ASSERT(nextItem != m_Suballocations.end());
        5407  VMA_ASSERT(nextItem->type == VMA_SUBALLOCATION_TYPE_FREE);
        5408 
        5409  item->size += nextItem->size;
        5410  --m_FreeCount;
        5411  m_Suballocations.erase(nextItem);
        5412 }
        5413 
        5414 VmaSuballocationList::iterator VmaBlockMetadata::FreeSuballocation(VmaSuballocationList::iterator suballocItem)
        5415 {
        5416  // Change this suballocation to be marked as free.
        5417  VmaSuballocation& suballoc = *suballocItem;
        5418  suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
        5419  suballoc.hAllocation = VK_NULL_HANDLE;
        5420 
        5421  // Update totals.
        5422  ++m_FreeCount;
        5423  m_SumFreeSize += suballoc.size;
        5424 
        5425  // Merge with previous and/or next suballocation if it's also free.
        5426  bool mergeWithNext = false;
        5427  bool mergeWithPrev = false;
        5428 
        5429  VmaSuballocationList::iterator nextItem = suballocItem;
        5430  ++nextItem;
        5431  if((nextItem != m_Suballocations.end()) && (nextItem->type == VMA_SUBALLOCATION_TYPE_FREE))
        5432  {
        5433  mergeWithNext = true;
        5434  }
        5435 
        5436  VmaSuballocationList::iterator prevItem = suballocItem;
        5437  if(suballocItem != m_Suballocations.begin())
        5438  {
        5439  --prevItem;
        5440  if(prevItem->type == VMA_SUBALLOCATION_TYPE_FREE)
        5441  {
        5442  mergeWithPrev = true;
        5443  }
        5444  }
        5445 
        5446  if(mergeWithNext)
        5447  {
        5448  UnregisterFreeSuballocation(nextItem);
        5449  MergeFreeWithNext(suballocItem);
        5450  }
        5451 
        5452  if(mergeWithPrev)
        5453  {
        5454  UnregisterFreeSuballocation(prevItem);
        5455  MergeFreeWithNext(prevItem);
        5456  RegisterFreeSuballocation(prevItem);
        5457  return prevItem;
        5458  }
        5459  else
        5460  {
        5461  RegisterFreeSuballocation(suballocItem);
        5462  return suballocItem;
        5463  }
        5464 }
        5465 
        5466 void VmaBlockMetadata::RegisterFreeSuballocation(VmaSuballocationList::iterator item)
        5467 {
        5468  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
        5469  VMA_ASSERT(item->size > 0);
        5470 
        5471  // You may want to enable this validation at the beginning or at the end of
        5472  // this function, depending on what do you want to check.
        5473  VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
        5474 
        5475  if(item->size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
        5476  {
        5477  if(m_FreeSuballocationsBySize.empty())
        5478  {
        5479  m_FreeSuballocationsBySize.push_back(item);
        5480  }
        5481  else
        5482  {
        5483  VmaVectorInsertSorted<VmaSuballocationItemSizeLess>(m_FreeSuballocationsBySize, item);
        5484  }
        5485  }
        5486 
        5487  //VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
        5488 }
        5489 
        5490 
        5491 void VmaBlockMetadata::UnregisterFreeSuballocation(VmaSuballocationList::iterator item)
        5492 {
        5493  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
        5494  VMA_ASSERT(item->size > 0);
        5495 
        5496  // You may want to enable this validation at the beginning or at the end of
        5497  // this function, depending on what do you want to check.
        5498  VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
        5499 
        5500  if(item->size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
        5501  {
        5502  VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
        5503  m_FreeSuballocationsBySize.data(),
        5504  m_FreeSuballocationsBySize.data() + m_FreeSuballocationsBySize.size(),
        5505  item,
        5506  VmaSuballocationItemSizeLess());
        5507  for(size_t index = it - m_FreeSuballocationsBySize.data();
        5508  index < m_FreeSuballocationsBySize.size();
        5509  ++index)
        5510  {
        5511  if(m_FreeSuballocationsBySize[index] == item)
        5512  {
        5513  VmaVectorRemove(m_FreeSuballocationsBySize, index);
        5514  return;
        5515  }
        5516  VMA_ASSERT((m_FreeSuballocationsBySize[index]->size == item->size) && "Not found.");
        5517  }
        5518  VMA_ASSERT(0 && "Not found.");
        5519  }
        5520 
        5521  //VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
        5522 }
        5523 
        5525 // class VmaDeviceMemoryMapping
        5526 
        5527 VmaDeviceMemoryMapping::VmaDeviceMemoryMapping() :
        5528  m_MapCount(0),
        5529  m_pMappedData(VMA_NULL)
        5530 {
        5531 }
        5532 
        5533 VmaDeviceMemoryMapping::~VmaDeviceMemoryMapping()
        5534 {
        5535  VMA_ASSERT(m_MapCount == 0 && "VkDeviceMemory block is being destroyed while it is still mapped.");
        5536 }
        5537 
        5538 VkResult VmaDeviceMemoryMapping::Map(VmaAllocator hAllocator, VkDeviceMemory hMemory, void **ppData)
        5539 {
        5540  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
        5541  if(m_MapCount != 0)
        5542  {
        5543  ++m_MapCount;
        5544  VMA_ASSERT(m_pMappedData != VMA_NULL);
        5545  if(ppData != VMA_NULL)
        5546  {
        5547  *ppData = m_pMappedData;
        5548  }
        5549  return VK_SUCCESS;
        5550  }
        5551  else
        5552  {
        5553  VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
        5554  hAllocator->m_hDevice,
        5555  hMemory,
        5556  0, // offset
        5557  VK_WHOLE_SIZE,
        5558  0, // flags
        5559  &m_pMappedData);
        5560  if(result == VK_SUCCESS)
        5561  {
        5562  if(ppData != VMA_NULL)
        5563  {
        5564  *ppData = m_pMappedData;
        5565  }
        5566  m_MapCount = 1;
        5567  }
        5568  return result;
        5569  }
        5570 }
        5571 
        5572 void VmaDeviceMemoryMapping::Unmap(VmaAllocator hAllocator, VkDeviceMemory hMemory)
        5573 {
        5574  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
        5575  if(m_MapCount != 0)
        5576  {
        5577  if(--m_MapCount == 0)
        5578  {
        5579  m_pMappedData = VMA_NULL;
        5580  (*hAllocator->GetVulkanFunctions().vkUnmapMemory)(hAllocator->m_hDevice, hMemory);
        5581  }
        5582  }
        5583  else
        5584  {
        5585  VMA_ASSERT(0 && "VkDeviceMemory block is being unmapped while it was not previously mapped.");
        5586  }
        5587 }
        5588 
        5590 // class VmaDeviceMemoryBlock
        5591 
        5592 VmaDeviceMemoryBlock::VmaDeviceMemoryBlock(VmaAllocator hAllocator) :
        5593  m_MemoryTypeIndex(UINT32_MAX),
        5594  m_hMemory(VK_NULL_HANDLE),
        5595  m_Metadata(hAllocator)
        5596 {
        5597 }
        5598 
        5599 void VmaDeviceMemoryBlock::Init(
        5600  uint32_t newMemoryTypeIndex,
        5601  VkDeviceMemory newMemory,
        5602  VkDeviceSize newSize)
        5603 {
        5604  VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
        5605 
        5606  m_MemoryTypeIndex = newMemoryTypeIndex;
        5607  m_hMemory = newMemory;
        5608 
        5609  m_Metadata.Init(newSize);
        5610 }
        5611 
        5612 void VmaDeviceMemoryBlock::Destroy(VmaAllocator allocator)
        5613 {
        5614  // This is the most important assert in the entire library.
        5615  // Hitting it means you have some memory leak - unreleased VmaAllocation objects.
        5616  VMA_ASSERT(m_Metadata.IsEmpty() && "Some allocations were not freed before destruction of this memory block!");
        5617 
        5618  VMA_ASSERT(m_hMemory != VK_NULL_HANDLE);
        5619  allocator->FreeVulkanMemory(m_MemoryTypeIndex, m_Metadata.GetSize(), m_hMemory);
        5620  m_hMemory = VK_NULL_HANDLE;
        5621 }
        5622 
        5623 bool VmaDeviceMemoryBlock::Validate() const
        5624 {
        5625  if((m_hMemory == VK_NULL_HANDLE) ||
        5626  (m_Metadata.GetSize() == 0))
        5627  {
        5628  return false;
        5629  }
        5630 
        5631  return m_Metadata.Validate();
        5632 }
        5633 
        5634 VkResult VmaDeviceMemoryBlock::Map(VmaAllocator hAllocator, void** ppData)
        5635 {
        5636  return m_Mapping.Map(hAllocator, m_hMemory, ppData);
        5637 }
        5638 
        5639 void VmaDeviceMemoryBlock::Unmap(VmaAllocator hAllocator)
        5640 {
        5641  m_Mapping.Unmap(hAllocator, m_hMemory);
        5642 }
        5643 
        5644 static void InitStatInfo(VmaStatInfo& outInfo)
        5645 {
        5646  memset(&outInfo, 0, sizeof(outInfo));
        5647  outInfo.allocationSizeMin = UINT64_MAX;
        5648  outInfo.unusedRangeSizeMin = UINT64_MAX;
        5649 }
        5650 
        5651 // Adds statistics srcInfo into inoutInfo, like: inoutInfo += srcInfo.
        5652 static void VmaAddStatInfo(VmaStatInfo& inoutInfo, const VmaStatInfo& srcInfo)
        5653 {
        5654  inoutInfo.blockCount += srcInfo.blockCount;
        5655  inoutInfo.allocationCount += srcInfo.allocationCount;
        5656  inoutInfo.unusedRangeCount += srcInfo.unusedRangeCount;
        5657  inoutInfo.usedBytes += srcInfo.usedBytes;
        5658  inoutInfo.unusedBytes += srcInfo.unusedBytes;
        5659  inoutInfo.allocationSizeMin = VMA_MIN(inoutInfo.allocationSizeMin, srcInfo.allocationSizeMin);
        5660  inoutInfo.allocationSizeMax = VMA_MAX(inoutInfo.allocationSizeMax, srcInfo.allocationSizeMax);
        5661  inoutInfo.unusedRangeSizeMin = VMA_MIN(inoutInfo.unusedRangeSizeMin, srcInfo.unusedRangeSizeMin);
        5662  inoutInfo.unusedRangeSizeMax = VMA_MAX(inoutInfo.unusedRangeSizeMax, srcInfo.unusedRangeSizeMax);
        5663 }
        5664 
        5665 static void VmaPostprocessCalcStatInfo(VmaStatInfo& inoutInfo)
        5666 {
        5667  inoutInfo.allocationSizeAvg = (inoutInfo.allocationCount > 0) ?
        5668  VmaRoundDiv<VkDeviceSize>(inoutInfo.usedBytes, inoutInfo.allocationCount) : 0;
        5669  inoutInfo.unusedRangeSizeAvg = (inoutInfo.unusedRangeCount > 0) ?
        5670  VmaRoundDiv<VkDeviceSize>(inoutInfo.unusedBytes, inoutInfo.unusedRangeCount) : 0;
        5671 }
        5672 
        5673 VmaPool_T::VmaPool_T(
        5674  VmaAllocator hAllocator,
        5675  const VmaPoolCreateInfo& createInfo) :
        5676  m_BlockVector(
        5677  hAllocator,
        5678  createInfo.memoryTypeIndex,
        5679  createInfo.blockSize,
        5680  createInfo.minBlockCount,
        5681  createInfo.maxBlockCount,
        5682  (createInfo.flags & VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT) != 0 ? 1 : hAllocator->GetBufferImageGranularity(),
        5683  createInfo.frameInUseCount,
        5684  true) // isCustomPool
        5685 {
        5686 }
        5687 
        5688 VmaPool_T::~VmaPool_T()
        5689 {
        5690 }
        5691 
        5692 #if VMA_STATS_STRING_ENABLED
        5693 
        5694 #endif // #if VMA_STATS_STRING_ENABLED
        5695 
        5696 VmaBlockVector::VmaBlockVector(
        5697  VmaAllocator hAllocator,
        5698  uint32_t memoryTypeIndex,
        5699  VkDeviceSize preferredBlockSize,
        5700  size_t minBlockCount,
        5701  size_t maxBlockCount,
        5702  VkDeviceSize bufferImageGranularity,
        5703  uint32_t frameInUseCount,
        5704  bool isCustomPool) :
        5705  m_hAllocator(hAllocator),
        5706  m_MemoryTypeIndex(memoryTypeIndex),
        5707  m_PreferredBlockSize(preferredBlockSize),
        5708  m_MinBlockCount(minBlockCount),
        5709  m_MaxBlockCount(maxBlockCount),
        5710  m_BufferImageGranularity(bufferImageGranularity),
        5711  m_FrameInUseCount(frameInUseCount),
        5712  m_IsCustomPool(isCustomPool),
        5713  m_Blocks(VmaStlAllocator<VmaDeviceMemoryBlock*>(hAllocator->GetAllocationCallbacks())),
        5714  m_HasEmptyBlock(false),
        5715  m_pDefragmentator(VMA_NULL)
        5716 {
        5717 }
        5718 
        5719 VmaBlockVector::~VmaBlockVector()
        5720 {
        5721  VMA_ASSERT(m_pDefragmentator == VMA_NULL);
        5722 
        5723  for(size_t i = m_Blocks.size(); i--; )
        5724  {
        5725  m_Blocks[i]->Destroy(m_hAllocator);
        5726  vma_delete(m_hAllocator, m_Blocks[i]);
        5727  }
        5728 }
        5729 
        5730 VkResult VmaBlockVector::CreateMinBlocks()
        5731 {
        5732  for(size_t i = 0; i < m_MinBlockCount; ++i)
        5733  {
        5734  VkResult res = CreateBlock(m_PreferredBlockSize, VMA_NULL);
        5735  if(res != VK_SUCCESS)
        5736  {
        5737  return res;
        5738  }
        5739  }
        5740  return VK_SUCCESS;
        5741 }
        5742 
        5743 void VmaBlockVector::GetPoolStats(VmaPoolStats* pStats)
        5744 {
        5745  pStats->size = 0;
        5746  pStats->unusedSize = 0;
        5747  pStats->allocationCount = 0;
        5748  pStats->unusedRangeCount = 0;
        5749  pStats->unusedRangeSizeMax = 0;
        5750 
        5751  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        5752 
        5753  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
        5754  {
        5755  const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
        5756  VMA_ASSERT(pBlock);
        5757  VMA_HEAVY_ASSERT(pBlock->Validate());
        5758  pBlock->m_Metadata.AddPoolStats(*pStats);
        5759  }
        5760 }
        5761 
        5762 static const uint32_t VMA_ALLOCATION_TRY_COUNT = 32;
        5763 
        5764 VkResult VmaBlockVector::Allocate(
        5765  VmaPool hCurrentPool,
        5766  uint32_t currentFrameIndex,
        5767  const VkMemoryRequirements& vkMemReq,
        5768  const VmaAllocationCreateInfo& createInfo,
        5769  VmaSuballocationType suballocType,
        5770  VmaAllocation* pAllocation)
        5771 {
        5772  const bool mapped = (createInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0;
        5773  const bool isUserDataString = (createInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0;
        5774 
        5775  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        5776 
        5777  // 1. Search existing allocations. Try to allocate without making other allocations lost.
        5778  // Forward order in m_Blocks - prefer blocks with smallest amount of free space.
        5779  for(size_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex )
        5780  {
        5781  VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
        5782  VMA_ASSERT(pCurrBlock);
        5783  VmaAllocationRequest currRequest = {};
        5784  if(pCurrBlock->m_Metadata.CreateAllocationRequest(
        5785  currentFrameIndex,
        5786  m_FrameInUseCount,
        5787  m_BufferImageGranularity,
        5788  vkMemReq.size,
        5789  vkMemReq.alignment,
        5790  suballocType,
        5791  false, // canMakeOtherLost
        5792  &currRequest))
        5793  {
        5794  // Allocate from pCurrBlock.
        5795  VMA_ASSERT(currRequest.itemsToMakeLostCount == 0);
        5796 
        5797  if(mapped)
        5798  {
        5799  VkResult res = pCurrBlock->Map(m_hAllocator, nullptr);
        5800  if(res != VK_SUCCESS)
        5801  {
        5802  return res;
        5803  }
        5804  }
        5805 
        5806  // We no longer have an empty Allocation.
        5807  if(pCurrBlock->m_Metadata.IsEmpty())
        5808  {
        5809  m_HasEmptyBlock = false;
        5810  }
        5811 
        5812  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
        5813  pCurrBlock->m_Metadata.Alloc(currRequest, suballocType, vkMemReq.size, *pAllocation);
        5814  (*pAllocation)->InitBlockAllocation(
        5815  hCurrentPool,
        5816  pCurrBlock,
        5817  currRequest.offset,
        5818  vkMemReq.alignment,
        5819  vkMemReq.size,
        5820  suballocType,
        5821  mapped,
        5822  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
        5823  VMA_HEAVY_ASSERT(pCurrBlock->Validate());
        5824  VMA_DEBUG_LOG(" Returned from existing allocation #%u", (uint32_t)blockIndex);
        5825  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
        5826  return VK_SUCCESS;
        5827  }
        5828  }
        5829 
        5830  const bool canCreateNewBlock =
        5831  ((createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == 0) &&
        5832  (m_Blocks.size() < m_MaxBlockCount);
        5833 
        5834  // 2. Try to create new block.
        5835  if(canCreateNewBlock)
        5836  {
        5837  // 2.1. Start with full preferredBlockSize.
        5838  VkDeviceSize blockSize = m_PreferredBlockSize;
        5839  size_t newBlockIndex = 0;
        5840  VkResult res = CreateBlock(blockSize, &newBlockIndex);
        5841  // Allocating blocks of other sizes is allowed only in default pools.
        5842  // In custom pools block size is fixed.
        5843  if(res < 0 && m_IsCustomPool == false)
        5844  {
        5845  // 2.2. Try half the size.
        5846  blockSize /= 2;
        5847  if(blockSize >= vkMemReq.size)
        5848  {
        5849  res = CreateBlock(blockSize, &newBlockIndex);
        5850  if(res < 0)
        5851  {
        5852  // 2.3. Try quarter the size.
        5853  blockSize /= 2;
        5854  if(blockSize >= vkMemReq.size)
        5855  {
        5856  res = CreateBlock(blockSize, &newBlockIndex);
        5857  }
        5858  }
        5859  }
        5860  }
        5861  if(res == VK_SUCCESS)
        5862  {
        5863  VmaDeviceMemoryBlock* const pBlock = m_Blocks[newBlockIndex];
        5864  VMA_ASSERT(pBlock->m_Metadata.GetSize() >= vkMemReq.size);
        5865 
        5866  if(mapped)
        5867  {
        5868  res = pBlock->Map(m_hAllocator, nullptr);
        5869  if(res != VK_SUCCESS)
        5870  {
        5871  return res;
        5872  }
        5873  }
        5874 
        5875  // Allocate from pBlock. Because it is empty, dstAllocRequest can be trivially filled.
        5876  VmaAllocationRequest allocRequest;
        5877  pBlock->m_Metadata.CreateFirstAllocationRequest(&allocRequest);
        5878  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
        5879  pBlock->m_Metadata.Alloc(allocRequest, suballocType, vkMemReq.size, *pAllocation);
        5880  (*pAllocation)->InitBlockAllocation(
        5881  hCurrentPool,
        5882  pBlock,
        5883  allocRequest.offset,
        5884  vkMemReq.alignment,
        5885  vkMemReq.size,
        5886  suballocType,
        5887  mapped,
        5888  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
        5889  VMA_HEAVY_ASSERT(pBlock->Validate());
        5890  VMA_DEBUG_LOG(" Created new allocation Size=%llu", allocInfo.allocationSize);
        5891  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
        5892  return VK_SUCCESS;
        5893  }
        5894  }
        5895 
        5896  const bool canMakeOtherLost = (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT) != 0;
        5897 
        5898  // 3. Try to allocate from existing blocks with making other allocations lost.
        5899  if(canMakeOtherLost)
        5900  {
        5901  uint32_t tryIndex = 0;
        5902  for(; tryIndex < VMA_ALLOCATION_TRY_COUNT; ++tryIndex)
        5903  {
        5904  VmaDeviceMemoryBlock* pBestRequestBlock = VMA_NULL;
        5905  VmaAllocationRequest bestRequest = {};
        5906  VkDeviceSize bestRequestCost = VK_WHOLE_SIZE;
        5907 
        5908  // 1. Search existing allocations.
        5909  // Forward order in m_Blocks - prefer blocks with smallest amount of free space.
        5910  for(size_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex )
        5911  {
        5912  VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
        5913  VMA_ASSERT(pCurrBlock);
        5914  VmaAllocationRequest currRequest = {};
        5915  if(pCurrBlock->m_Metadata.CreateAllocationRequest(
        5916  currentFrameIndex,
        5917  m_FrameInUseCount,
        5918  m_BufferImageGranularity,
        5919  vkMemReq.size,
        5920  vkMemReq.alignment,
        5921  suballocType,
        5922  canMakeOtherLost,
        5923  &currRequest))
        5924  {
        5925  const VkDeviceSize currRequestCost = currRequest.CalcCost();
        5926  if(pBestRequestBlock == VMA_NULL ||
        5927  currRequestCost < bestRequestCost)
        5928  {
        5929  pBestRequestBlock = pCurrBlock;
        5930  bestRequest = currRequest;
        5931  bestRequestCost = currRequestCost;
        5932 
        5933  if(bestRequestCost == 0)
        5934  {
        5935  break;
        5936  }
        5937  }
        5938  }
        5939  }
        5940 
        5941  if(pBestRequestBlock != VMA_NULL)
        5942  {
        5943  if(mapped)
        5944  {
        5945  VkResult res = pBestRequestBlock->Map(m_hAllocator, nullptr);
        5946  if(res != VK_SUCCESS)
        5947  {
        5948  return res;
        5949  }
        5950  }
        5951 
        5952  if(pBestRequestBlock->m_Metadata.MakeRequestedAllocationsLost(
        5953  currentFrameIndex,
        5954  m_FrameInUseCount,
        5955  &bestRequest))
        5956  {
        5957  // We no longer have an empty Allocation.
        5958  if(pBestRequestBlock->m_Metadata.IsEmpty())
        5959  {
        5960  m_HasEmptyBlock = false;
        5961  }
        5962  // Allocate from this pBlock.
        5963  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
        5964  pBestRequestBlock->m_Metadata.Alloc(bestRequest, suballocType, vkMemReq.size, *pAllocation);
        5965  (*pAllocation)->InitBlockAllocation(
        5966  hCurrentPool,
        5967  pBestRequestBlock,
        5968  bestRequest.offset,
        5969  vkMemReq.alignment,
        5970  vkMemReq.size,
        5971  suballocType,
        5972  mapped,
        5973  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
        5974  VMA_HEAVY_ASSERT(pBlock->Validate());
        5975  VMA_DEBUG_LOG(" Returned from existing allocation #%u", (uint32_t)blockIndex);
        5976  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
        5977  return VK_SUCCESS;
        5978  }
        5979  // else: Some allocations must have been touched while we are here. Next try.
        5980  }
        5981  else
        5982  {
        5983  // Could not find place in any of the blocks - break outer loop.
        5984  break;
        5985  }
        5986  }
        5987  /* Maximum number of tries exceeded - a very unlike event when many other
        5988  threads are simultaneously touching allocations making it impossible to make
        5989  lost at the same time as we try to allocate. */
        5990  if(tryIndex == VMA_ALLOCATION_TRY_COUNT)
        5991  {
        5992  return VK_ERROR_TOO_MANY_OBJECTS;
        5993  }
        5994  }
        5995 
        5996  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        5997 }
        5998 
        5999 void VmaBlockVector::Free(
        6000  VmaAllocation hAllocation)
        6001 {
        6002  VmaDeviceMemoryBlock* pBlockToDelete = VMA_NULL;
        6003 
        6004  // Scope for lock.
        6005  {
        6006  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        6007 
        6008  VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
        6009 
        6010  if(hAllocation->IsPersistentMap())
        6011  {
        6012  pBlock->m_Mapping.Unmap(m_hAllocator, pBlock->m_hMemory);
        6013  }
        6014 
        6015  pBlock->m_Metadata.Free(hAllocation);
        6016  VMA_HEAVY_ASSERT(pBlock->Validate());
        6017 
        6018  VMA_DEBUG_LOG(" Freed from MemoryTypeIndex=%u", memTypeIndex);
        6019 
        6020  // pBlock became empty after this deallocation.
        6021  if(pBlock->m_Metadata.IsEmpty())
        6022  {
        6023  // Already has empty Allocation. We don't want to have two, so delete this one.
        6024  if(m_HasEmptyBlock && m_Blocks.size() > m_MinBlockCount)
        6025  {
        6026  pBlockToDelete = pBlock;
        6027  Remove(pBlock);
        6028  }
        6029  // We now have first empty Allocation.
        6030  else
        6031  {
        6032  m_HasEmptyBlock = true;
        6033  }
        6034  }
        6035  // pBlock didn't become empty, but we have another empty block - find and free that one.
        6036  // (This is optional, heuristics.)
        6037  else if(m_HasEmptyBlock)
        6038  {
        6039  VmaDeviceMemoryBlock* pLastBlock = m_Blocks.back();
        6040  if(pLastBlock->m_Metadata.IsEmpty() && m_Blocks.size() > m_MinBlockCount)
        6041  {
        6042  pBlockToDelete = pLastBlock;
        6043  m_Blocks.pop_back();
        6044  m_HasEmptyBlock = false;
        6045  }
        6046  }
        6047 
        6048  IncrementallySortBlocks();
        6049  }
        6050 
        6051  // Destruction of a free Allocation. Deferred until this point, outside of mutex
        6052  // lock, for performance reason.
        6053  if(pBlockToDelete != VMA_NULL)
        6054  {
        6055  VMA_DEBUG_LOG(" Deleted empty allocation");
        6056  pBlockToDelete->Destroy(m_hAllocator);
        6057  vma_delete(m_hAllocator, pBlockToDelete);
        6058  }
        6059 }
        6060 
        6061 void VmaBlockVector::Remove(VmaDeviceMemoryBlock* pBlock)
        6062 {
        6063  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
        6064  {
        6065  if(m_Blocks[blockIndex] == pBlock)
        6066  {
        6067  VmaVectorRemove(m_Blocks, blockIndex);
        6068  return;
        6069  }
        6070  }
        6071  VMA_ASSERT(0);
        6072 }
        6073 
        6074 void VmaBlockVector::IncrementallySortBlocks()
        6075 {
        6076  // Bubble sort only until first swap.
        6077  for(size_t i = 1; i < m_Blocks.size(); ++i)
        6078  {
        6079  if(m_Blocks[i - 1]->m_Metadata.GetSumFreeSize() > m_Blocks[i]->m_Metadata.GetSumFreeSize())
        6080  {
        6081  VMA_SWAP(m_Blocks[i - 1], m_Blocks[i]);
        6082  return;
        6083  }
        6084  }
        6085 }
        6086 
        6087 VkResult VmaBlockVector::CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex)
        6088 {
        6089  VkMemoryAllocateInfo allocInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
        6090  allocInfo.memoryTypeIndex = m_MemoryTypeIndex;
        6091  allocInfo.allocationSize = blockSize;
        6092  VkDeviceMemory mem = VK_NULL_HANDLE;
        6093  VkResult res = m_hAllocator->AllocateVulkanMemory(&allocInfo, &mem);
        6094  if(res < 0)
        6095  {
        6096  return res;
        6097  }
        6098 
        6099  // New VkDeviceMemory successfully created.
        6100 
        6101  // Create new Allocation for it.
        6102  VmaDeviceMemoryBlock* const pBlock = vma_new(m_hAllocator, VmaDeviceMemoryBlock)(m_hAllocator);
        6103  pBlock->Init(
        6104  m_MemoryTypeIndex,
        6105  mem,
        6106  allocInfo.allocationSize);
        6107 
        6108  m_Blocks.push_back(pBlock);
        6109  if(pNewBlockIndex != VMA_NULL)
        6110  {
        6111  *pNewBlockIndex = m_Blocks.size() - 1;
        6112  }
        6113 
        6114  return VK_SUCCESS;
        6115 }
        6116 
        6117 #if VMA_STATS_STRING_ENABLED
        6118 
        6119 void VmaBlockVector::PrintDetailedMap(class VmaJsonWriter& json)
        6120 {
        6121  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        6122 
        6123  json.BeginObject();
        6124 
        6125  if(m_IsCustomPool)
        6126  {
        6127  json.WriteString("MemoryTypeIndex");
        6128  json.WriteNumber(m_MemoryTypeIndex);
        6129 
        6130  json.WriteString("BlockSize");
        6131  json.WriteNumber(m_PreferredBlockSize);
        6132 
        6133  json.WriteString("BlockCount");
        6134  json.BeginObject(true);
        6135  if(m_MinBlockCount > 0)
        6136  {
        6137  json.WriteString("Min");
        6138  json.WriteNumber(m_MinBlockCount);
        6139  }
        6140  if(m_MaxBlockCount < SIZE_MAX)
        6141  {
        6142  json.WriteString("Max");
        6143  json.WriteNumber(m_MaxBlockCount);
        6144  }
        6145  json.WriteString("Cur");
        6146  json.WriteNumber(m_Blocks.size());
        6147  json.EndObject();
        6148 
        6149  if(m_FrameInUseCount > 0)
        6150  {
        6151  json.WriteString("FrameInUseCount");
        6152  json.WriteNumber(m_FrameInUseCount);
        6153  }
        6154  }
        6155  else
        6156  {
        6157  json.WriteString("PreferredBlockSize");
        6158  json.WriteNumber(m_PreferredBlockSize);
        6159  }
        6160 
        6161  json.WriteString("Blocks");
        6162  json.BeginArray();
        6163  for(size_t i = 0; i < m_Blocks.size(); ++i)
        6164  {
        6165  m_Blocks[i]->m_Metadata.PrintDetailedMap(json);
        6166  }
        6167  json.EndArray();
        6168 
        6169  json.EndObject();
        6170 }
        6171 
        6172 #endif // #if VMA_STATS_STRING_ENABLED
        6173 
        6174 VmaDefragmentator* VmaBlockVector::EnsureDefragmentator(
        6175  VmaAllocator hAllocator,
        6176  uint32_t currentFrameIndex)
        6177 {
        6178  if(m_pDefragmentator == VMA_NULL)
        6179  {
        6180  m_pDefragmentator = vma_new(m_hAllocator, VmaDefragmentator)(
        6181  hAllocator,
        6182  this,
        6183  currentFrameIndex);
        6184  }
        6185 
        6186  return m_pDefragmentator;
        6187 }
        6188 
        6189 VkResult VmaBlockVector::Defragment(
        6190  VmaDefragmentationStats* pDefragmentationStats,
        6191  VkDeviceSize& maxBytesToMove,
        6192  uint32_t& maxAllocationsToMove)
        6193 {
        6194  if(m_pDefragmentator == VMA_NULL)
        6195  {
        6196  return VK_SUCCESS;
        6197  }
        6198 
        6199  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        6200 
        6201  // Defragment.
        6202  VkResult result = m_pDefragmentator->Defragment(maxBytesToMove, maxAllocationsToMove);
        6203 
        6204  // Accumulate statistics.
        6205  if(pDefragmentationStats != VMA_NULL)
        6206  {
        6207  const VkDeviceSize bytesMoved = m_pDefragmentator->GetBytesMoved();
        6208  const uint32_t allocationsMoved = m_pDefragmentator->GetAllocationsMoved();
        6209  pDefragmentationStats->bytesMoved += bytesMoved;
        6210  pDefragmentationStats->allocationsMoved += allocationsMoved;
        6211  VMA_ASSERT(bytesMoved <= maxBytesToMove);
        6212  VMA_ASSERT(allocationsMoved <= maxAllocationsToMove);
        6213  maxBytesToMove -= bytesMoved;
        6214  maxAllocationsToMove -= allocationsMoved;
        6215  }
        6216 
        6217  // Free empty blocks.
        6218  m_HasEmptyBlock = false;
        6219  for(size_t blockIndex = m_Blocks.size(); blockIndex--; )
        6220  {
        6221  VmaDeviceMemoryBlock* pBlock = m_Blocks[blockIndex];
        6222  if(pBlock->m_Metadata.IsEmpty())
        6223  {
        6224  if(m_Blocks.size() > m_MinBlockCount)
        6225  {
        6226  if(pDefragmentationStats != VMA_NULL)
        6227  {
        6228  ++pDefragmentationStats->deviceMemoryBlocksFreed;
        6229  pDefragmentationStats->bytesFreed += pBlock->m_Metadata.GetSize();
        6230  }
        6231 
        6232  VmaVectorRemove(m_Blocks, blockIndex);
        6233  pBlock->Destroy(m_hAllocator);
        6234  vma_delete(m_hAllocator, pBlock);
        6235  }
        6236  else
        6237  {
        6238  m_HasEmptyBlock = true;
        6239  }
        6240  }
        6241  }
        6242 
        6243  return result;
        6244 }
        6245 
        6246 void VmaBlockVector::DestroyDefragmentator()
        6247 {
        6248  if(m_pDefragmentator != VMA_NULL)
        6249  {
        6250  vma_delete(m_hAllocator, m_pDefragmentator);
        6251  m_pDefragmentator = VMA_NULL;
        6252  }
        6253 }
        6254 
        6255 void VmaBlockVector::MakePoolAllocationsLost(
        6256  uint32_t currentFrameIndex,
        6257  size_t* pLostAllocationCount)
        6258 {
        6259  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        6260 
        6261  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
        6262  {
        6263  VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
        6264  VMA_ASSERT(pBlock);
        6265  pBlock->m_Metadata.MakeAllocationsLost(currentFrameIndex, m_FrameInUseCount);
        6266  }
        6267 }
        6268 
        6269 void VmaBlockVector::AddStats(VmaStats* pStats)
        6270 {
        6271  const uint32_t memTypeIndex = m_MemoryTypeIndex;
        6272  const uint32_t memHeapIndex = m_hAllocator->MemoryTypeIndexToHeapIndex(memTypeIndex);
        6273 
        6274  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        6275 
        6276  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
        6277  {
        6278  const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
        6279  VMA_ASSERT(pBlock);
        6280  VMA_HEAVY_ASSERT(pBlock->Validate());
        6281  VmaStatInfo allocationStatInfo;
        6282  pBlock->m_Metadata.CalcAllocationStatInfo(allocationStatInfo);
        6283  VmaAddStatInfo(pStats->total, allocationStatInfo);
        6284  VmaAddStatInfo(pStats->memoryType[memTypeIndex], allocationStatInfo);
        6285  VmaAddStatInfo(pStats->memoryHeap[memHeapIndex], allocationStatInfo);
        6286  }
        6287 }
        6288 
        6290 // VmaDefragmentator members definition
        6291 
        6292 VmaDefragmentator::VmaDefragmentator(
        6293  VmaAllocator hAllocator,
        6294  VmaBlockVector* pBlockVector,
        6295  uint32_t currentFrameIndex) :
        6296  m_hAllocator(hAllocator),
        6297  m_pBlockVector(pBlockVector),
        6298  m_CurrentFrameIndex(currentFrameIndex),
        6299  m_BytesMoved(0),
        6300  m_AllocationsMoved(0),
        6301  m_Allocations(VmaStlAllocator<AllocationInfo>(hAllocator->GetAllocationCallbacks())),
        6302  m_Blocks(VmaStlAllocator<BlockInfo*>(hAllocator->GetAllocationCallbacks()))
        6303 {
        6304 }
        6305 
        6306 VmaDefragmentator::~VmaDefragmentator()
        6307 {
        6308  for(size_t i = m_Blocks.size(); i--; )
        6309  {
        6310  vma_delete(m_hAllocator, m_Blocks[i]);
        6311  }
        6312 }
        6313 
        6314 void VmaDefragmentator::AddAllocation(VmaAllocation hAlloc, VkBool32* pChanged)
        6315 {
        6316  AllocationInfo allocInfo;
        6317  allocInfo.m_hAllocation = hAlloc;
        6318  allocInfo.m_pChanged = pChanged;
        6319  m_Allocations.push_back(allocInfo);
        6320 }
        6321 
        6322 VkResult VmaDefragmentator::BlockInfo::EnsureMapping(VmaAllocator hAllocator, void** ppMappedData)
        6323 {
        6324  // It has already been mapped for defragmentation.
        6325  if(m_pMappedDataForDefragmentation)
        6326  {
        6327  *ppMappedData = m_pMappedDataForDefragmentation;
        6328  return VK_SUCCESS;
        6329  }
        6330 
        6331  // It is originally mapped.
        6332  if(m_pBlock->m_Mapping.GetMappedData())
        6333  {
        6334  *ppMappedData = m_pBlock->m_Mapping.GetMappedData();
        6335  return VK_SUCCESS;
        6336  }
        6337 
        6338  // Map on first usage.
        6339  VkResult res = m_pBlock->Map(hAllocator, &m_pMappedDataForDefragmentation);
        6340  *ppMappedData = m_pMappedDataForDefragmentation;
        6341  return res;
        6342 }
        6343 
        6344 void VmaDefragmentator::BlockInfo::Unmap(VmaAllocator hAllocator)
        6345 {
        6346  if(m_pMappedDataForDefragmentation != VMA_NULL)
        6347  {
        6348  m_pBlock->Unmap(hAllocator);
        6349  }
        6350 }
        6351 
        6352 VkResult VmaDefragmentator::DefragmentRound(
        6353  VkDeviceSize maxBytesToMove,
        6354  uint32_t maxAllocationsToMove)
        6355 {
        6356  if(m_Blocks.empty())
        6357  {
        6358  return VK_SUCCESS;
        6359  }
        6360 
        6361  size_t srcBlockIndex = m_Blocks.size() - 1;
        6362  size_t srcAllocIndex = SIZE_MAX;
        6363  for(;;)
        6364  {
        6365  // 1. Find next allocation to move.
        6366  // 1.1. Start from last to first m_Blocks - they are sorted from most "destination" to most "source".
        6367  // 1.2. Then start from last to first m_Allocations - they are sorted from largest to smallest.
        6368  while(srcAllocIndex >= m_Blocks[srcBlockIndex]->m_Allocations.size())
        6369  {
        6370  if(m_Blocks[srcBlockIndex]->m_Allocations.empty())
        6371  {
        6372  // Finished: no more allocations to process.
        6373  if(srcBlockIndex == 0)
        6374  {
        6375  return VK_SUCCESS;
        6376  }
        6377  else
        6378  {
        6379  --srcBlockIndex;
        6380  srcAllocIndex = SIZE_MAX;
        6381  }
        6382  }
        6383  else
        6384  {
        6385  srcAllocIndex = m_Blocks[srcBlockIndex]->m_Allocations.size() - 1;
        6386  }
        6387  }
        6388 
        6389  BlockInfo* pSrcBlockInfo = m_Blocks[srcBlockIndex];
        6390  AllocationInfo& allocInfo = pSrcBlockInfo->m_Allocations[srcAllocIndex];
        6391 
        6392  const VkDeviceSize size = allocInfo.m_hAllocation->GetSize();
        6393  const VkDeviceSize srcOffset = allocInfo.m_hAllocation->GetOffset();
        6394  const VkDeviceSize alignment = allocInfo.m_hAllocation->GetAlignment();
        6395  const VmaSuballocationType suballocType = allocInfo.m_hAllocation->GetSuballocationType();
        6396 
        6397  // 2. Try to find new place for this allocation in preceding or current block.
        6398  for(size_t dstBlockIndex = 0; dstBlockIndex <= srcBlockIndex; ++dstBlockIndex)
        6399  {
        6400  BlockInfo* pDstBlockInfo = m_Blocks[dstBlockIndex];
        6401  VmaAllocationRequest dstAllocRequest;
        6402  if(pDstBlockInfo->m_pBlock->m_Metadata.CreateAllocationRequest(
        6403  m_CurrentFrameIndex,
        6404  m_pBlockVector->GetFrameInUseCount(),
        6405  m_pBlockVector->GetBufferImageGranularity(),
        6406  size,
        6407  alignment,
        6408  suballocType,
        6409  false, // canMakeOtherLost
        6410  &dstAllocRequest) &&
        6411  MoveMakesSense(
        6412  dstBlockIndex, dstAllocRequest.offset, srcBlockIndex, srcOffset))
        6413  {
        6414  VMA_ASSERT(dstAllocRequest.itemsToMakeLostCount == 0);
        6415 
        6416  // Reached limit on number of allocations or bytes to move.
        6417  if((m_AllocationsMoved + 1 > maxAllocationsToMove) ||
        6418  (m_BytesMoved + size > maxBytesToMove))
        6419  {
        6420  return VK_INCOMPLETE;
        6421  }
        6422 
        6423  void* pDstMappedData = VMA_NULL;
        6424  VkResult res = pDstBlockInfo->EnsureMapping(m_hAllocator, &pDstMappedData);
        6425  if(res != VK_SUCCESS)
        6426  {
        6427  return res;
        6428  }
        6429 
        6430  void* pSrcMappedData = VMA_NULL;
        6431  res = pSrcBlockInfo->EnsureMapping(m_hAllocator, &pSrcMappedData);
        6432  if(res != VK_SUCCESS)
        6433  {
        6434  return res;
        6435  }
        6436 
        6437  // THE PLACE WHERE ACTUAL DATA COPY HAPPENS.
        6438  memcpy(
        6439  reinterpret_cast<char*>(pDstMappedData) + dstAllocRequest.offset,
        6440  reinterpret_cast<char*>(pSrcMappedData) + srcOffset,
        6441  static_cast<size_t>(size));
        6442 
        6443  pDstBlockInfo->m_pBlock->m_Metadata.Alloc(dstAllocRequest, suballocType, size, allocInfo.m_hAllocation);
        6444  pSrcBlockInfo->m_pBlock->m_Metadata.Free(allocInfo.m_hAllocation);
        6445 
        6446  allocInfo.m_hAllocation->ChangeBlockAllocation(pDstBlockInfo->m_pBlock, dstAllocRequest.offset);
        6447 
        6448  if(allocInfo.m_pChanged != VMA_NULL)
        6449  {
        6450  *allocInfo.m_pChanged = VK_TRUE;
        6451  }
        6452 
        6453  ++m_AllocationsMoved;
        6454  m_BytesMoved += size;
        6455 
        6456  VmaVectorRemove(pSrcBlockInfo->m_Allocations, srcAllocIndex);
        6457 
        6458  break;
        6459  }
        6460  }
        6461 
        6462  // If not processed, this allocInfo remains in pBlockInfo->m_Allocations for next round.
        6463 
        6464  if(srcAllocIndex > 0)
        6465  {
        6466  --srcAllocIndex;
        6467  }
        6468  else
        6469  {
        6470  if(srcBlockIndex > 0)
        6471  {
        6472  --srcBlockIndex;
        6473  srcAllocIndex = SIZE_MAX;
        6474  }
        6475  else
        6476  {
        6477  return VK_SUCCESS;
        6478  }
        6479  }
        6480  }
        6481 }
        6482 
        6483 VkResult VmaDefragmentator::Defragment(
        6484  VkDeviceSize maxBytesToMove,
        6485  uint32_t maxAllocationsToMove)
        6486 {
        6487  if(m_Allocations.empty())
        6488  {
        6489  return VK_SUCCESS;
        6490  }
        6491 
        6492  // Create block info for each block.
        6493  const size_t blockCount = m_pBlockVector->m_Blocks.size();
        6494  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
        6495  {
        6496  BlockInfo* pBlockInfo = vma_new(m_hAllocator, BlockInfo)(m_hAllocator->GetAllocationCallbacks());
        6497  pBlockInfo->m_pBlock = m_pBlockVector->m_Blocks[blockIndex];
        6498  m_Blocks.push_back(pBlockInfo);
        6499  }
        6500 
        6501  // Sort them by m_pBlock pointer value.
        6502  VMA_SORT(m_Blocks.begin(), m_Blocks.end(), BlockPointerLess());
        6503 
        6504  // Move allocation infos from m_Allocations to appropriate m_Blocks[memTypeIndex].m_Allocations.
        6505  for(size_t blockIndex = 0, allocCount = m_Allocations.size(); blockIndex < allocCount; ++blockIndex)
        6506  {
        6507  AllocationInfo& allocInfo = m_Allocations[blockIndex];
        6508  // Now as we are inside VmaBlockVector::m_Mutex, we can make final check if this allocation was not lost.
        6509  if(allocInfo.m_hAllocation->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST)
        6510  {
        6511  VmaDeviceMemoryBlock* pBlock = allocInfo.m_hAllocation->GetBlock();
        6512  BlockInfoVector::iterator it = VmaBinaryFindFirstNotLess(m_Blocks.begin(), m_Blocks.end(), pBlock, BlockPointerLess());
        6513  if(it != m_Blocks.end() && (*it)->m_pBlock == pBlock)
        6514  {
        6515  (*it)->m_Allocations.push_back(allocInfo);
        6516  }
        6517  else
        6518  {
        6519  VMA_ASSERT(0);
        6520  }
        6521  }
        6522  }
        6523  m_Allocations.clear();
        6524 
        6525  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
        6526  {
        6527  BlockInfo* pBlockInfo = m_Blocks[blockIndex];
        6528  pBlockInfo->CalcHasNonMovableAllocations();
        6529  pBlockInfo->SortAllocationsBySizeDescecnding();
        6530  }
        6531 
        6532  // Sort m_Blocks this time by the main criterium, from most "destination" to most "source" blocks.
        6533  VMA_SORT(m_Blocks.begin(), m_Blocks.end(), BlockInfoCompareMoveDestination());
        6534 
        6535  // Execute defragmentation rounds (the main part).
        6536  VkResult result = VK_SUCCESS;
        6537  for(size_t round = 0; (round < 2) && (result == VK_SUCCESS); ++round)
        6538  {
        6539  result = DefragmentRound(maxBytesToMove, maxAllocationsToMove);
        6540  }
        6541 
        6542  // Unmap blocks that were mapped for defragmentation.
        6543  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
        6544  {
        6545  m_Blocks[blockIndex]->Unmap(m_hAllocator);
        6546  }
        6547 
        6548  return result;
        6549 }
        6550 
        6551 bool VmaDefragmentator::MoveMakesSense(
        6552  size_t dstBlockIndex, VkDeviceSize dstOffset,
        6553  size_t srcBlockIndex, VkDeviceSize srcOffset)
        6554 {
        6555  if(dstBlockIndex < srcBlockIndex)
        6556  {
        6557  return true;
        6558  }
        6559  if(dstBlockIndex > srcBlockIndex)
        6560  {
        6561  return false;
        6562  }
        6563  if(dstOffset < srcOffset)
        6564  {
        6565  return true;
        6566  }
        6567  return false;
        6568 }
        6569 
        6571 // VmaAllocator_T
        6572 
        6573 VmaAllocator_T::VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo) :
        6574  m_UseMutex((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT) == 0),
        6575  m_UseKhrDedicatedAllocation((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT) != 0),
        6576  m_PhysicalDevice(pCreateInfo->physicalDevice),
        6577  m_hDevice(pCreateInfo->device),
        6578  m_AllocationCallbacksSpecified(pCreateInfo->pAllocationCallbacks != VMA_NULL),
        6579  m_AllocationCallbacks(pCreateInfo->pAllocationCallbacks ?
        6580  *pCreateInfo->pAllocationCallbacks : VmaEmptyAllocationCallbacks),
        6581  m_PreferredLargeHeapBlockSize(0),
        6582  m_PreferredSmallHeapBlockSize(0),
        6583  m_CurrentFrameIndex(0),
        6584  m_Pools(VmaStlAllocator<VmaPool>(GetAllocationCallbacks()))
        6585 {
        6586  VMA_ASSERT(pCreateInfo->physicalDevice && pCreateInfo->device);
        6587 
        6588  memset(&m_DeviceMemoryCallbacks, 0 ,sizeof(m_DeviceMemoryCallbacks));
        6589  memset(&m_MemProps, 0, sizeof(m_MemProps));
        6590  memset(&m_PhysicalDeviceProperties, 0, sizeof(m_PhysicalDeviceProperties));
        6591 
        6592  memset(&m_pBlockVectors, 0, sizeof(m_pBlockVectors));
        6593  memset(&m_pDedicatedAllocations, 0, sizeof(m_pDedicatedAllocations));
        6594 
        6595  for(uint32_t i = 0; i < VK_MAX_MEMORY_HEAPS; ++i)
        6596  {
        6597  m_HeapSizeLimit[i] = VK_WHOLE_SIZE;
        6598  }
        6599 
        6600  if(pCreateInfo->pDeviceMemoryCallbacks != VMA_NULL)
        6601  {
        6602  m_DeviceMemoryCallbacks.pfnAllocate = pCreateInfo->pDeviceMemoryCallbacks->pfnAllocate;
        6603  m_DeviceMemoryCallbacks.pfnFree = pCreateInfo->pDeviceMemoryCallbacks->pfnFree;
        6604  }
        6605 
        6606  ImportVulkanFunctions(pCreateInfo->pVulkanFunctions);
        6607 
        6608  (*m_VulkanFunctions.vkGetPhysicalDeviceProperties)(m_PhysicalDevice, &m_PhysicalDeviceProperties);
        6609  (*m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties)(m_PhysicalDevice, &m_MemProps);
        6610 
        6611  m_PreferredLargeHeapBlockSize = (pCreateInfo->preferredLargeHeapBlockSize != 0) ?
        6612  pCreateInfo->preferredLargeHeapBlockSize : static_cast<VkDeviceSize>(VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE);
        6613  m_PreferredSmallHeapBlockSize = (pCreateInfo->preferredSmallHeapBlockSize != 0) ?
        6614  pCreateInfo->preferredSmallHeapBlockSize : static_cast<VkDeviceSize>(VMA_DEFAULT_SMALL_HEAP_BLOCK_SIZE);
        6615 
        6616  if(pCreateInfo->pHeapSizeLimit != VMA_NULL)
        6617  {
        6618  for(uint32_t heapIndex = 0; heapIndex < GetMemoryHeapCount(); ++heapIndex)
        6619  {
        6620  const VkDeviceSize limit = pCreateInfo->pHeapSizeLimit[heapIndex];
        6621  if(limit != VK_WHOLE_SIZE)
        6622  {
        6623  m_HeapSizeLimit[heapIndex] = limit;
        6624  if(limit < m_MemProps.memoryHeaps[heapIndex].size)
        6625  {
        6626  m_MemProps.memoryHeaps[heapIndex].size = limit;
        6627  }
        6628  }
        6629  }
        6630  }
        6631 
        6632  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
        6633  {
        6634  const VkDeviceSize preferredBlockSize = CalcPreferredBlockSize(memTypeIndex);
        6635 
        6636  m_pBlockVectors[memTypeIndex] = vma_new(this, VmaBlockVector)(
        6637  this,
        6638  memTypeIndex,
        6639  preferredBlockSize,
        6640  0,
        6641  SIZE_MAX,
        6642  GetBufferImageGranularity(),
        6643  pCreateInfo->frameInUseCount,
        6644  false); // isCustomPool
        6645  // No need to call m_pBlockVectors[memTypeIndex][blockVectorTypeIndex]->CreateMinBlocks here,
        6646  // becase minBlockCount is 0.
        6647  m_pDedicatedAllocations[memTypeIndex] = vma_new(this, AllocationVectorType)(VmaStlAllocator<VmaAllocation>(GetAllocationCallbacks()));
        6648  }
        6649 }
        6650 
        6651 VmaAllocator_T::~VmaAllocator_T()
        6652 {
        6653  VMA_ASSERT(m_Pools.empty());
        6654 
        6655  for(size_t i = GetMemoryTypeCount(); i--; )
        6656  {
        6657  vma_delete(this, m_pDedicatedAllocations[i]);
        6658  vma_delete(this, m_pBlockVectors[i]);
        6659  }
        6660 }
        6661 
        6662 void VmaAllocator_T::ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions)
        6663 {
        6664 #if VMA_STATIC_VULKAN_FUNCTIONS == 1
        6665  m_VulkanFunctions.vkGetPhysicalDeviceProperties = &vkGetPhysicalDeviceProperties;
        6666  m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties = &vkGetPhysicalDeviceMemoryProperties;
        6667  m_VulkanFunctions.vkAllocateMemory = &vkAllocateMemory;
        6668  m_VulkanFunctions.vkFreeMemory = &vkFreeMemory;
        6669  m_VulkanFunctions.vkMapMemory = &vkMapMemory;
        6670  m_VulkanFunctions.vkUnmapMemory = &vkUnmapMemory;
        6671  m_VulkanFunctions.vkBindBufferMemory = &vkBindBufferMemory;
        6672  m_VulkanFunctions.vkBindImageMemory = &vkBindImageMemory;
        6673  m_VulkanFunctions.vkGetBufferMemoryRequirements = &vkGetBufferMemoryRequirements;
        6674  m_VulkanFunctions.vkGetImageMemoryRequirements = &vkGetImageMemoryRequirements;
        6675  m_VulkanFunctions.vkCreateBuffer = &vkCreateBuffer;
        6676  m_VulkanFunctions.vkDestroyBuffer = &vkDestroyBuffer;
        6677  m_VulkanFunctions.vkCreateImage = &vkCreateImage;
        6678  m_VulkanFunctions.vkDestroyImage = &vkDestroyImage;
        6679  // Ignoring vkGetBufferMemoryRequirements2KHR.
        6680  // Ignoring vkGetImageMemoryRequirements2KHR.
        6681 #endif // #if VMA_STATIC_VULKAN_FUNCTIONS == 1
        6682 
        6683 #define VMA_COPY_IF_NOT_NULL(funcName) \
        6684  if(pVulkanFunctions->funcName != VMA_NULL) m_VulkanFunctions.funcName = pVulkanFunctions->funcName;
        6685 
        6686  if(pVulkanFunctions != VMA_NULL)
        6687  {
        6688  VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceProperties);
        6689  VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceMemoryProperties);
        6690  VMA_COPY_IF_NOT_NULL(vkAllocateMemory);
        6691  VMA_COPY_IF_NOT_NULL(vkFreeMemory);
        6692  VMA_COPY_IF_NOT_NULL(vkMapMemory);
        6693  VMA_COPY_IF_NOT_NULL(vkUnmapMemory);
        6694  VMA_COPY_IF_NOT_NULL(vkBindBufferMemory);
        6695  VMA_COPY_IF_NOT_NULL(vkBindImageMemory);
        6696  VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements);
        6697  VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements);
        6698  VMA_COPY_IF_NOT_NULL(vkCreateBuffer);
        6699  VMA_COPY_IF_NOT_NULL(vkDestroyBuffer);
        6700  VMA_COPY_IF_NOT_NULL(vkCreateImage);
        6701  VMA_COPY_IF_NOT_NULL(vkDestroyImage);
        6702  VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements2KHR);
        6703  VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements2KHR);
        6704  }
        6705 
        6706 #undef VMA_COPY_IF_NOT_NULL
        6707 
        6708  // If these asserts are hit, you must either #define VMA_STATIC_VULKAN_FUNCTIONS 1
        6709  // or pass valid pointers as VmaAllocatorCreateInfo::pVulkanFunctions.
        6710  VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceProperties != VMA_NULL);
        6711  VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties != VMA_NULL);
        6712  VMA_ASSERT(m_VulkanFunctions.vkAllocateMemory != VMA_NULL);
        6713  VMA_ASSERT(m_VulkanFunctions.vkFreeMemory != VMA_NULL);
        6714  VMA_ASSERT(m_VulkanFunctions.vkMapMemory != VMA_NULL);
        6715  VMA_ASSERT(m_VulkanFunctions.vkUnmapMemory != VMA_NULL);
        6716  VMA_ASSERT(m_VulkanFunctions.vkBindBufferMemory != VMA_NULL);
        6717  VMA_ASSERT(m_VulkanFunctions.vkBindImageMemory != VMA_NULL);
        6718  VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements != VMA_NULL);
        6719  VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements != VMA_NULL);
        6720  VMA_ASSERT(m_VulkanFunctions.vkCreateBuffer != VMA_NULL);
        6721  VMA_ASSERT(m_VulkanFunctions.vkDestroyBuffer != VMA_NULL);
        6722  VMA_ASSERT(m_VulkanFunctions.vkCreateImage != VMA_NULL);
        6723  VMA_ASSERT(m_VulkanFunctions.vkDestroyImage != VMA_NULL);
        6724  if(m_UseKhrDedicatedAllocation)
        6725  {
        6726  VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR != VMA_NULL);
        6727  VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements2KHR != VMA_NULL);
        6728  }
        6729 }
        6730 
        6731 VkDeviceSize VmaAllocator_T::CalcPreferredBlockSize(uint32_t memTypeIndex)
        6732 {
        6733  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
        6734  const VkDeviceSize heapSize = m_MemProps.memoryHeaps[heapIndex].size;
        6735  return (heapSize <= VMA_SMALL_HEAP_MAX_SIZE) ?
        6736  m_PreferredSmallHeapBlockSize : m_PreferredLargeHeapBlockSize;
        6737 }
        6738 
        6739 VkResult VmaAllocator_T::AllocateMemoryOfType(
        6740  const VkMemoryRequirements& vkMemReq,
        6741  bool dedicatedAllocation,
        6742  VkBuffer dedicatedBuffer,
        6743  VkImage dedicatedImage,
        6744  const VmaAllocationCreateInfo& createInfo,
        6745  uint32_t memTypeIndex,
        6746  VmaSuballocationType suballocType,
        6747  VmaAllocation* pAllocation)
        6748 {
        6749  VMA_ASSERT(pAllocation != VMA_NULL);
        6750  VMA_DEBUG_LOG(" AllocateMemory: MemoryTypeIndex=%u, Size=%llu", memTypeIndex, vkMemReq.size);
        6751 
        6752  VmaAllocationCreateInfo finalCreateInfo = createInfo;
        6753 
        6754  // If memory type is not HOST_VISIBLE, disable MAPPED.
        6755  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0 &&
        6756  (m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) == 0)
        6757  {
        6758  finalCreateInfo.flags &= ~VMA_ALLOCATION_CREATE_MAPPED_BIT;
        6759  }
        6760 
        6761  VmaBlockVector* const blockVector = m_pBlockVectors[memTypeIndex];
        6762  VMA_ASSERT(blockVector);
        6763 
        6764  const VkDeviceSize preferredBlockSize = blockVector->GetPreferredBlockSize();
        6765  bool preferDedicatedMemory =
        6766  VMA_DEBUG_ALWAYS_DEDICATED_MEMORY ||
        6767  dedicatedAllocation ||
        6768  // Heuristics: Allocate dedicated memory if requested size if greater than half of preferred block size.
        6769  vkMemReq.size > preferredBlockSize / 2;
        6770 
        6771  if(preferDedicatedMemory &&
        6772  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == 0 &&
        6773  finalCreateInfo.pool == VK_NULL_HANDLE)
        6774  {
        6775  finalCreateInfo.flags |= VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
        6776  }
        6777 
        6778  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0)
        6779  {
        6780  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
        6781  {
        6782  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        6783  }
        6784  else
        6785  {
        6786  return AllocateDedicatedMemory(
        6787  vkMemReq.size,
        6788  suballocType,
        6789  memTypeIndex,
        6790  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0,
        6791  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0,
        6792  finalCreateInfo.pUserData,
        6793  dedicatedBuffer,
        6794  dedicatedImage,
        6795  pAllocation);
        6796  }
        6797  }
        6798  else
        6799  {
        6800  VkResult res = blockVector->Allocate(
        6801  VK_NULL_HANDLE, // hCurrentPool
        6802  m_CurrentFrameIndex.load(),
        6803  vkMemReq,
        6804  finalCreateInfo,
        6805  suballocType,
        6806  pAllocation);
        6807  if(res == VK_SUCCESS)
        6808  {
        6809  return res;
        6810  }
        6811 
        6812  // 5. Try dedicated memory.
        6813  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
        6814  {
        6815  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        6816  }
        6817  else
        6818  {
        6819  res = AllocateDedicatedMemory(
        6820  vkMemReq.size,
        6821  suballocType,
        6822  memTypeIndex,
        6823  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0,
        6824  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0,
        6825  finalCreateInfo.pUserData,
        6826  dedicatedBuffer,
        6827  dedicatedImage,
        6828  pAllocation);
        6829  if(res == VK_SUCCESS)
        6830  {
        6831  // Succeeded: AllocateDedicatedMemory function already filld pMemory, nothing more to do here.
        6832  VMA_DEBUG_LOG(" Allocated as DedicatedMemory");
        6833  return VK_SUCCESS;
        6834  }
        6835  else
        6836  {
        6837  // Everything failed: Return error code.
        6838  VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
        6839  return res;
        6840  }
        6841  }
        6842  }
        6843 }
        6844 
        6845 VkResult VmaAllocator_T::AllocateDedicatedMemory(
        6846  VkDeviceSize size,
        6847  VmaSuballocationType suballocType,
        6848  uint32_t memTypeIndex,
        6849  bool map,
        6850  bool isUserDataString,
        6851  void* pUserData,
        6852  VkBuffer dedicatedBuffer,
        6853  VkImage dedicatedImage,
        6854  VmaAllocation* pAllocation)
        6855 {
        6856  VMA_ASSERT(pAllocation);
        6857 
        6858  VkMemoryAllocateInfo allocInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
        6859  allocInfo.memoryTypeIndex = memTypeIndex;
        6860  allocInfo.allocationSize = size;
        6861 
        6862  VkMemoryDedicatedAllocateInfoKHR dedicatedAllocInfo = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO_KHR };
        6863  if(m_UseKhrDedicatedAllocation)
        6864  {
        6865  if(dedicatedBuffer != VK_NULL_HANDLE)
        6866  {
        6867  VMA_ASSERT(dedicatedImage == VK_NULL_HANDLE);
        6868  dedicatedAllocInfo.buffer = dedicatedBuffer;
        6869  allocInfo.pNext = &dedicatedAllocInfo;
        6870  }
        6871  else if(dedicatedImage != VK_NULL_HANDLE)
        6872  {
        6873  dedicatedAllocInfo.image = dedicatedImage;
        6874  allocInfo.pNext = &dedicatedAllocInfo;
        6875  }
        6876  }
        6877 
        6878  // Allocate VkDeviceMemory.
        6879  VkDeviceMemory hMemory = VK_NULL_HANDLE;
        6880  VkResult res = AllocateVulkanMemory(&allocInfo, &hMemory);
        6881  if(res < 0)
        6882  {
        6883  VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
        6884  return res;
        6885  }
        6886 
        6887  void* pMappedData = nullptr;
        6888  if(map)
        6889  {
        6890  res = (*m_VulkanFunctions.vkMapMemory)(
        6891  m_hDevice,
        6892  hMemory,
        6893  0,
        6894  VK_WHOLE_SIZE,
        6895  0,
        6896  &pMappedData);
        6897  if(res < 0)
        6898  {
        6899  VMA_DEBUG_LOG(" vkMapMemory FAILED");
        6900  FreeVulkanMemory(memTypeIndex, size, hMemory);
        6901  return res;
        6902  }
        6903  }
        6904 
        6905  *pAllocation = vma_new(this, VmaAllocation_T)(m_CurrentFrameIndex.load(), isUserDataString);
        6906  (*pAllocation)->InitDedicatedAllocation(memTypeIndex, hMemory, suballocType, pMappedData, size);
        6907  (*pAllocation)->SetUserData(this, pUserData);
        6908 
        6909  // Register it in m_pDedicatedAllocations.
        6910  {
        6911  VmaMutexLock lock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
        6912  AllocationVectorType* pDedicatedAllocations = m_pDedicatedAllocations[memTypeIndex];
        6913  VMA_ASSERT(pDedicatedAllocations);
        6914  VmaVectorInsertSorted<VmaPointerLess>(*pDedicatedAllocations, *pAllocation);
        6915  }
        6916 
        6917  VMA_DEBUG_LOG(" Allocated DedicatedMemory MemoryTypeIndex=#%u", memTypeIndex);
        6918 
        6919  return VK_SUCCESS;
        6920 }
        6921 
        6922 void VmaAllocator_T::GetBufferMemoryRequirements(
        6923  VkBuffer hBuffer,
        6924  VkMemoryRequirements& memReq,
        6925  bool& requiresDedicatedAllocation,
        6926  bool& prefersDedicatedAllocation) const
        6927 {
        6928  if(m_UseKhrDedicatedAllocation)
        6929  {
        6930  VkBufferMemoryRequirementsInfo2KHR memReqInfo = { VK_STRUCTURE_TYPE_BUFFER_MEMORY_REQUIREMENTS_INFO_2_KHR };
        6931  memReqInfo.buffer = hBuffer;
        6932 
        6933  VkMemoryDedicatedRequirementsKHR memDedicatedReq = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
        6934 
        6935  VkMemoryRequirements2KHR memReq2 = { VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
        6936  memReq2.pNext = &memDedicatedReq;
        6937 
        6938  (*m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
        6939 
        6940  memReq = memReq2.memoryRequirements;
        6941  requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
        6942  prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
        6943  }
        6944  else
        6945  {
        6946  (*m_VulkanFunctions.vkGetBufferMemoryRequirements)(m_hDevice, hBuffer, &memReq);
        6947  requiresDedicatedAllocation = false;
        6948  prefersDedicatedAllocation = false;
        6949  }
        6950 }
        6951 
        6952 void VmaAllocator_T::GetImageMemoryRequirements(
        6953  VkImage hImage,
        6954  VkMemoryRequirements& memReq,
        6955  bool& requiresDedicatedAllocation,
        6956  bool& prefersDedicatedAllocation) const
        6957 {
        6958  if(m_UseKhrDedicatedAllocation)
        6959  {
        6960  VkImageMemoryRequirementsInfo2KHR memReqInfo = { VK_STRUCTURE_TYPE_IMAGE_MEMORY_REQUIREMENTS_INFO_2_KHR };
        6961  memReqInfo.image = hImage;
        6962 
        6963  VkMemoryDedicatedRequirementsKHR memDedicatedReq = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
        6964 
        6965  VkMemoryRequirements2KHR memReq2 = { VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
        6966  memReq2.pNext = &memDedicatedReq;
        6967 
        6968  (*m_VulkanFunctions.vkGetImageMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
        6969 
        6970  memReq = memReq2.memoryRequirements;
        6971  requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
        6972  prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
        6973  }
        6974  else
        6975  {
        6976  (*m_VulkanFunctions.vkGetImageMemoryRequirements)(m_hDevice, hImage, &memReq);
        6977  requiresDedicatedAllocation = false;
        6978  prefersDedicatedAllocation = false;
        6979  }
        6980 }
        6981 
        6982 VkResult VmaAllocator_T::AllocateMemory(
        6983  const VkMemoryRequirements& vkMemReq,
        6984  bool requiresDedicatedAllocation,
        6985  bool prefersDedicatedAllocation,
        6986  VkBuffer dedicatedBuffer,
        6987  VkImage dedicatedImage,
        6988  const VmaAllocationCreateInfo& createInfo,
        6989  VmaSuballocationType suballocType,
        6990  VmaAllocation* pAllocation)
        6991 {
        6992  if((createInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0 &&
        6993  (createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
        6994  {
        6995  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT together with VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT makes no sense.");
        6996  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        6997  }
        6998  if((createInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0 &&
        6999  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0)
        7000  {
        7001  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_MAPPED_BIT together with VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT is invalid.");
        7002  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7003  }
        7004  if(requiresDedicatedAllocation)
        7005  {
        7006  if((createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
        7007  {
        7008  VMA_ASSERT(0 && "VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT specified while dedicated allocation is required.");
        7009  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7010  }
        7011  if(createInfo.pool != VK_NULL_HANDLE)
        7012  {
        7013  VMA_ASSERT(0 && "Pool specified while dedicated allocation is required.");
        7014  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7015  }
        7016  }
        7017  if((createInfo.pool != VK_NULL_HANDLE) &&
        7018  ((createInfo.flags & (VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT)) != 0))
        7019  {
        7020  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT when pool != null is invalid.");
        7021  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7022  }
        7023 
        7024  if(createInfo.pool != VK_NULL_HANDLE)
        7025  {
        7026  return createInfo.pool->m_BlockVector.Allocate(
        7027  createInfo.pool,
        7028  m_CurrentFrameIndex.load(),
        7029  vkMemReq,
        7030  createInfo,
        7031  suballocType,
        7032  pAllocation);
        7033  }
        7034  else
        7035  {
        7036  // Bit mask of memory Vulkan types acceptable for this allocation.
        7037  uint32_t memoryTypeBits = vkMemReq.memoryTypeBits;
        7038  uint32_t memTypeIndex = UINT32_MAX;
        7039  VkResult res = vmaFindMemoryTypeIndex(this, memoryTypeBits, &createInfo, &memTypeIndex);
        7040  if(res == VK_SUCCESS)
        7041  {
        7042  res = AllocateMemoryOfType(
        7043  vkMemReq,
        7044  requiresDedicatedAllocation || prefersDedicatedAllocation,
        7045  dedicatedBuffer,
        7046  dedicatedImage,
        7047  createInfo,
        7048  memTypeIndex,
        7049  suballocType,
        7050  pAllocation);
        7051  // Succeeded on first try.
        7052  if(res == VK_SUCCESS)
        7053  {
        7054  return res;
        7055  }
        7056  // Allocation from this memory type failed. Try other compatible memory types.
        7057  else
        7058  {
        7059  for(;;)
        7060  {
        7061  // Remove old memTypeIndex from list of possibilities.
        7062  memoryTypeBits &= ~(1u << memTypeIndex);
        7063  // Find alternative memTypeIndex.
        7064  res = vmaFindMemoryTypeIndex(this, memoryTypeBits, &createInfo, &memTypeIndex);
        7065  if(res == VK_SUCCESS)
        7066  {
        7067  res = AllocateMemoryOfType(
        7068  vkMemReq,
        7069  requiresDedicatedAllocation || prefersDedicatedAllocation,
        7070  dedicatedBuffer,
        7071  dedicatedImage,
        7072  createInfo,
        7073  memTypeIndex,
        7074  suballocType,
        7075  pAllocation);
        7076  // Allocation from this alternative memory type succeeded.
        7077  if(res == VK_SUCCESS)
        7078  {
        7079  return res;
        7080  }
        7081  // else: Allocation from this memory type failed. Try next one - next loop iteration.
        7082  }
        7083  // No other matching memory type index could be found.
        7084  else
        7085  {
        7086  // Not returning res, which is VK_ERROR_FEATURE_NOT_PRESENT, because we already failed to allocate once.
        7087  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7088  }
        7089  }
        7090  }
        7091  }
        7092  // Can't find any single memory type maching requirements. res is VK_ERROR_FEATURE_NOT_PRESENT.
        7093  else
        7094  return res;
        7095  }
        7096 }
        7097 
        7098 void VmaAllocator_T::FreeMemory(const VmaAllocation allocation)
        7099 {
        7100  VMA_ASSERT(allocation);
        7101 
        7102  if(allocation->CanBecomeLost() == false ||
        7103  allocation->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST)
        7104  {
        7105  switch(allocation->GetType())
        7106  {
        7107  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
        7108  {
        7109  VmaBlockVector* pBlockVector = VMA_NULL;
        7110  VmaPool hPool = allocation->GetPool();
        7111  if(hPool != VK_NULL_HANDLE)
        7112  {
        7113  pBlockVector = &hPool->m_BlockVector;
        7114  }
        7115  else
        7116  {
        7117  const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
        7118  pBlockVector = m_pBlockVectors[memTypeIndex];
        7119  }
        7120  pBlockVector->Free(allocation);
        7121  }
        7122  break;
        7123  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
        7124  FreeDedicatedMemory(allocation);
        7125  break;
        7126  default:
        7127  VMA_ASSERT(0);
        7128  }
        7129  }
        7130 
        7131  allocation->SetUserData(this, VMA_NULL);
        7132  vma_delete(this, allocation);
        7133 }
        7134 
        7135 void VmaAllocator_T::CalculateStats(VmaStats* pStats)
        7136 {
        7137  // Initialize.
        7138  InitStatInfo(pStats->total);
        7139  for(size_t i = 0; i < VK_MAX_MEMORY_TYPES; ++i)
        7140  InitStatInfo(pStats->memoryType[i]);
        7141  for(size_t i = 0; i < VK_MAX_MEMORY_HEAPS; ++i)
        7142  InitStatInfo(pStats->memoryHeap[i]);
        7143 
        7144  // Process default pools.
        7145  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
        7146  {
        7147  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
        7148  VmaBlockVector* const pBlockVector = m_pBlockVectors[memTypeIndex];
        7149  VMA_ASSERT(pBlockVector);
        7150  pBlockVector->AddStats(pStats);
        7151  }
        7152 
        7153  // Process custom pools.
        7154  {
        7155  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
        7156  for(size_t poolIndex = 0, poolCount = m_Pools.size(); poolIndex < poolCount; ++poolIndex)
        7157  {
        7158  m_Pools[poolIndex]->GetBlockVector().AddStats(pStats);
        7159  }
        7160  }
        7161 
        7162  // Process dedicated allocations.
        7163  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
        7164  {
        7165  const uint32_t memHeapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
        7166  VmaMutexLock dedicatedAllocationsLock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
        7167  AllocationVectorType* const pDedicatedAllocVector = m_pDedicatedAllocations[memTypeIndex];
        7168  VMA_ASSERT(pDedicatedAllocVector);
        7169  for(size_t allocIndex = 0, allocCount = pDedicatedAllocVector->size(); allocIndex < allocCount; ++allocIndex)
        7170  {
        7171  VmaStatInfo allocationStatInfo;
        7172  (*pDedicatedAllocVector)[allocIndex]->DedicatedAllocCalcStatsInfo(allocationStatInfo);
        7173  VmaAddStatInfo(pStats->total, allocationStatInfo);
        7174  VmaAddStatInfo(pStats->memoryType[memTypeIndex], allocationStatInfo);
        7175  VmaAddStatInfo(pStats->memoryHeap[memHeapIndex], allocationStatInfo);
        7176  }
        7177  }
        7178 
        7179  // Postprocess.
        7180  VmaPostprocessCalcStatInfo(pStats->total);
        7181  for(size_t i = 0; i < GetMemoryTypeCount(); ++i)
        7182  VmaPostprocessCalcStatInfo(pStats->memoryType[i]);
        7183  for(size_t i = 0; i < GetMemoryHeapCount(); ++i)
        7184  VmaPostprocessCalcStatInfo(pStats->memoryHeap[i]);
        7185 }
        7186 
        7187 static const uint32_t VMA_VENDOR_ID_AMD = 4098;
        7188 
        7189 VkResult VmaAllocator_T::Defragment(
        7190  VmaAllocation* pAllocations,
        7191  size_t allocationCount,
        7192  VkBool32* pAllocationsChanged,
        7193  const VmaDefragmentationInfo* pDefragmentationInfo,
        7194  VmaDefragmentationStats* pDefragmentationStats)
        7195 {
        7196  if(pAllocationsChanged != VMA_NULL)
        7197  {
        7198  memset(pAllocationsChanged, 0, sizeof(*pAllocationsChanged));
        7199  }
        7200  if(pDefragmentationStats != VMA_NULL)
        7201  {
        7202  memset(pDefragmentationStats, 0, sizeof(*pDefragmentationStats));
        7203  }
        7204 
        7205  const uint32_t currentFrameIndex = m_CurrentFrameIndex.load();
        7206 
        7207  VmaMutexLock poolsLock(m_PoolsMutex, m_UseMutex);
        7208 
        7209  const size_t poolCount = m_Pools.size();
        7210 
        7211  // Dispatch pAllocations among defragmentators. Create them in BlockVectors when necessary.
        7212  for(size_t allocIndex = 0; allocIndex < allocationCount; ++allocIndex)
        7213  {
        7214  VmaAllocation hAlloc = pAllocations[allocIndex];
        7215  VMA_ASSERT(hAlloc);
        7216  const uint32_t memTypeIndex = hAlloc->GetMemoryTypeIndex();
        7217  // DedicatedAlloc cannot be defragmented.
        7218  if((hAlloc->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK) &&
        7219  // Only HOST_VISIBLE memory types can be defragmented.
        7220  ((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0) &&
        7221  // Lost allocation cannot be defragmented.
        7222  (hAlloc->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST))
        7223  {
        7224  VmaBlockVector* pAllocBlockVector = nullptr;
        7225 
        7226  const VmaPool hAllocPool = hAlloc->GetPool();
        7227  // This allocation belongs to custom pool.
        7228  if(hAllocPool != VK_NULL_HANDLE)
        7229  {
        7230  pAllocBlockVector = &hAllocPool->GetBlockVector();
        7231  }
        7232  // This allocation belongs to general pool.
        7233  else
        7234  {
        7235  pAllocBlockVector = m_pBlockVectors[memTypeIndex];
        7236  }
        7237 
        7238  VmaDefragmentator* const pDefragmentator = pAllocBlockVector->EnsureDefragmentator(this, currentFrameIndex);
        7239 
        7240  VkBool32* const pChanged = (pAllocationsChanged != VMA_NULL) ?
        7241  &pAllocationsChanged[allocIndex] : VMA_NULL;
        7242  pDefragmentator->AddAllocation(hAlloc, pChanged);
        7243  }
        7244  }
        7245 
        7246  VkResult result = VK_SUCCESS;
        7247 
        7248  // ======== Main processing.
        7249 
        7250  VkDeviceSize maxBytesToMove = SIZE_MAX;
        7251  uint32_t maxAllocationsToMove = UINT32_MAX;
        7252  if(pDefragmentationInfo != VMA_NULL)
        7253  {
        7254  maxBytesToMove = pDefragmentationInfo->maxBytesToMove;
        7255  maxAllocationsToMove = pDefragmentationInfo->maxAllocationsToMove;
        7256  }
        7257 
        7258  // Process standard memory.
        7259  for(uint32_t memTypeIndex = 0;
        7260  (memTypeIndex < GetMemoryTypeCount()) && (result == VK_SUCCESS);
        7261  ++memTypeIndex)
        7262  {
        7263  // Only HOST_VISIBLE memory types can be defragmented.
        7264  if((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
        7265  {
        7266  result = m_pBlockVectors[memTypeIndex]->Defragment(
        7267  pDefragmentationStats,
        7268  maxBytesToMove,
        7269  maxAllocationsToMove);
        7270  }
        7271  }
        7272 
        7273  // Process custom pools.
        7274  for(size_t poolIndex = 0; (poolIndex < poolCount) && (result == VK_SUCCESS); ++poolIndex)
        7275  {
        7276  result = m_Pools[poolIndex]->GetBlockVector().Defragment(
        7277  pDefragmentationStats,
        7278  maxBytesToMove,
        7279  maxAllocationsToMove);
        7280  }
        7281 
        7282  // ======== Destroy defragmentators.
        7283 
        7284  // Process custom pools.
        7285  for(size_t poolIndex = poolCount; poolIndex--; )
        7286  {
        7287  m_Pools[poolIndex]->GetBlockVector().DestroyDefragmentator();
        7288  }
        7289 
        7290  // Process standard memory.
        7291  for(uint32_t memTypeIndex = GetMemoryTypeCount(); memTypeIndex--; )
        7292  {
        7293  if((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
        7294  {
        7295  m_pBlockVectors[memTypeIndex]->DestroyDefragmentator();
        7296  }
        7297  }
        7298 
        7299  return result;
        7300 }
        7301 
        7302 void VmaAllocator_T::GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo)
        7303 {
        7304  if(hAllocation->CanBecomeLost())
        7305  {
        7306  /*
        7307  Warning: This is a carefully designed algorithm.
        7308  Do not modify unless you really know what you're doing :)
        7309  */
        7310  uint32_t localCurrFrameIndex = m_CurrentFrameIndex.load();
        7311  uint32_t localLastUseFrameIndex = hAllocation->GetLastUseFrameIndex();
        7312  for(;;)
        7313  {
        7314  if(localLastUseFrameIndex == VMA_FRAME_INDEX_LOST)
        7315  {
        7316  pAllocationInfo->memoryType = UINT32_MAX;
        7317  pAllocationInfo->deviceMemory = VK_NULL_HANDLE;
        7318  pAllocationInfo->offset = 0;
        7319  pAllocationInfo->size = hAllocation->GetSize();
        7320  pAllocationInfo->pMappedData = VMA_NULL;
        7321  pAllocationInfo->pUserData = hAllocation->GetUserData();
        7322  return;
        7323  }
        7324  else if(localLastUseFrameIndex == localCurrFrameIndex)
        7325  {
        7326  pAllocationInfo->memoryType = hAllocation->GetMemoryTypeIndex();
        7327  pAllocationInfo->deviceMemory = hAllocation->GetMemory();
        7328  pAllocationInfo->offset = hAllocation->GetOffset();
        7329  pAllocationInfo->size = hAllocation->GetSize();
        7330  pAllocationInfo->pMappedData = VMA_NULL;
        7331  pAllocationInfo->pUserData = hAllocation->GetUserData();
        7332  return;
        7333  }
        7334  else // Last use time earlier than current time.
        7335  {
        7336  if(hAllocation->CompareExchangeLastUseFrameIndex(localLastUseFrameIndex, localCurrFrameIndex))
        7337  {
        7338  localLastUseFrameIndex = localCurrFrameIndex;
        7339  }
        7340  }
        7341  }
        7342  }
        7343  else
        7344  {
        7345  pAllocationInfo->memoryType = hAllocation->GetMemoryTypeIndex();
        7346  pAllocationInfo->deviceMemory = hAllocation->GetMemory();
        7347  pAllocationInfo->offset = hAllocation->GetOffset();
        7348  pAllocationInfo->size = hAllocation->GetSize();
        7349  pAllocationInfo->pMappedData = hAllocation->GetMappedData();
        7350  pAllocationInfo->pUserData = hAllocation->GetUserData();
        7351  }
        7352 }
        7353 
        7354 VkResult VmaAllocator_T::CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool)
        7355 {
        7356  VMA_DEBUG_LOG(" CreatePool: MemoryTypeIndex=%u", pCreateInfo->memoryTypeIndex);
        7357 
        7358  VmaPoolCreateInfo newCreateInfo = *pCreateInfo;
        7359 
        7360  if(newCreateInfo.maxBlockCount == 0)
        7361  {
        7362  newCreateInfo.maxBlockCount = SIZE_MAX;
        7363  }
        7364  if(newCreateInfo.blockSize == 0)
        7365  {
        7366  newCreateInfo.blockSize = CalcPreferredBlockSize(newCreateInfo.memoryTypeIndex);
        7367  }
        7368 
        7369  *pPool = vma_new(this, VmaPool_T)(this, newCreateInfo);
        7370 
        7371  VkResult res = (*pPool)->m_BlockVector.CreateMinBlocks();
        7372  if(res != VK_SUCCESS)
        7373  {
        7374  vma_delete(this, *pPool);
        7375  *pPool = VMA_NULL;
        7376  return res;
        7377  }
        7378 
        7379  // Add to m_Pools.
        7380  {
        7381  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
        7382  VmaVectorInsertSorted<VmaPointerLess>(m_Pools, *pPool);
        7383  }
        7384 
        7385  return VK_SUCCESS;
        7386 }
        7387 
        7388 void VmaAllocator_T::DestroyPool(VmaPool pool)
        7389 {
        7390  // Remove from m_Pools.
        7391  {
        7392  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
        7393  bool success = VmaVectorRemoveSorted<VmaPointerLess>(m_Pools, pool);
        7394  VMA_ASSERT(success && "Pool not found in Allocator.");
        7395  }
        7396 
        7397  vma_delete(this, pool);
        7398 }
        7399 
        7400 void VmaAllocator_T::GetPoolStats(VmaPool pool, VmaPoolStats* pPoolStats)
        7401 {
        7402  pool->m_BlockVector.GetPoolStats(pPoolStats);
        7403 }
        7404 
        7405 void VmaAllocator_T::SetCurrentFrameIndex(uint32_t frameIndex)
        7406 {
        7407  m_CurrentFrameIndex.store(frameIndex);
        7408 }
        7409 
        7410 void VmaAllocator_T::MakePoolAllocationsLost(
        7411  VmaPool hPool,
        7412  size_t* pLostAllocationCount)
        7413 {
        7414  hPool->m_BlockVector.MakePoolAllocationsLost(
        7415  m_CurrentFrameIndex.load(),
        7416  pLostAllocationCount);
        7417 }
        7418 
        7419 void VmaAllocator_T::CreateLostAllocation(VmaAllocation* pAllocation)
        7420 {
        7421  *pAllocation = vma_new(this, VmaAllocation_T)(VMA_FRAME_INDEX_LOST, false);
        7422  (*pAllocation)->InitLost();
        7423 }
        7424 
        7425 VkResult VmaAllocator_T::AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory)
        7426 {
        7427  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(pAllocateInfo->memoryTypeIndex);
        7428 
        7429  VkResult res;
        7430  if(m_HeapSizeLimit[heapIndex] != VK_WHOLE_SIZE)
        7431  {
        7432  VmaMutexLock lock(m_HeapSizeLimitMutex, m_UseMutex);
        7433  if(m_HeapSizeLimit[heapIndex] >= pAllocateInfo->allocationSize)
        7434  {
        7435  res = (*m_VulkanFunctions.vkAllocateMemory)(m_hDevice, pAllocateInfo, GetAllocationCallbacks(), pMemory);
        7436  if(res == VK_SUCCESS)
        7437  {
        7438  m_HeapSizeLimit[heapIndex] -= pAllocateInfo->allocationSize;
        7439  }
        7440  }
        7441  else
        7442  {
        7443  res = VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7444  }
        7445  }
        7446  else
        7447  {
        7448  res = (*m_VulkanFunctions.vkAllocateMemory)(m_hDevice, pAllocateInfo, GetAllocationCallbacks(), pMemory);
        7449  }
        7450 
        7451  if(res == VK_SUCCESS && m_DeviceMemoryCallbacks.pfnAllocate != VMA_NULL)
        7452  {
        7453  (*m_DeviceMemoryCallbacks.pfnAllocate)(this, pAllocateInfo->memoryTypeIndex, *pMemory, pAllocateInfo->allocationSize);
        7454  }
        7455 
        7456  return res;
        7457 }
        7458 
        7459 void VmaAllocator_T::FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory)
        7460 {
        7461  if(m_DeviceMemoryCallbacks.pfnFree != VMA_NULL)
        7462  {
        7463  (*m_DeviceMemoryCallbacks.pfnFree)(this, memoryType, hMemory, size);
        7464  }
        7465 
        7466  (*m_VulkanFunctions.vkFreeMemory)(m_hDevice, hMemory, GetAllocationCallbacks());
        7467 
        7468  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memoryType);
        7469  if(m_HeapSizeLimit[heapIndex] != VK_WHOLE_SIZE)
        7470  {
        7471  VmaMutexLock lock(m_HeapSizeLimitMutex, m_UseMutex);
        7472  m_HeapSizeLimit[heapIndex] += size;
        7473  }
        7474 }
        7475 
        7476 VkResult VmaAllocator_T::Map(VmaAllocation hAllocation, void** ppData)
        7477 {
        7478  if(hAllocation->CanBecomeLost())
        7479  {
        7480  return VK_ERROR_MEMORY_MAP_FAILED;
        7481  }
        7482 
        7483  switch(hAllocation->GetType())
        7484  {
        7485  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
        7486  {
        7487  VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
        7488  char *pBytes = nullptr;
        7489  VkResult res = pBlock->Map(this, (void**)&pBytes);
        7490  if(res == VK_SUCCESS)
        7491  {
        7492  *ppData = pBytes + (ptrdiff_t)hAllocation->GetOffset();
        7493  hAllocation->BlockAllocMap();
        7494  }
        7495  return res;
        7496  }
        7497  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
        7498  return hAllocation->DedicatedAllocMap(this, ppData);
        7499  default:
        7500  VMA_ASSERT(0);
        7501  return VK_ERROR_MEMORY_MAP_FAILED;
        7502  }
        7503 }
        7504 
        7505 void VmaAllocator_T::Unmap(VmaAllocation hAllocation)
        7506 {
        7507  switch(hAllocation->GetType())
        7508  {
        7509  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
        7510  {
        7511  VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
        7512  hAllocation->BlockAllocUnmap();
        7513  pBlock->Unmap(this);
        7514  }
        7515  break;
        7516  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
        7517  hAllocation->DedicatedAllocUnmap(this);
        7518  break;
        7519  default:
        7520  VMA_ASSERT(0);
        7521  }
        7522 }
        7523 
        7524 void VmaAllocator_T::FreeDedicatedMemory(VmaAllocation allocation)
        7525 {
        7526  VMA_ASSERT(allocation && allocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
        7527 
        7528  const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
        7529  {
        7530  VmaMutexLock lock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
        7531  AllocationVectorType* const pDedicatedAllocations = m_pDedicatedAllocations[memTypeIndex];
        7532  VMA_ASSERT(pDedicatedAllocations);
        7533  bool success = VmaVectorRemoveSorted<VmaPointerLess>(*pDedicatedAllocations, allocation);
        7534  VMA_ASSERT(success);
        7535  }
        7536 
        7537  VkDeviceMemory hMemory = allocation->GetMemory();
        7538 
        7539  if(allocation->GetMappedData() != VMA_NULL)
        7540  {
        7541  (*m_VulkanFunctions.vkUnmapMemory)(m_hDevice, hMemory);
        7542  }
        7543 
        7544  FreeVulkanMemory(memTypeIndex, allocation->GetSize(), hMemory);
        7545 
        7546  VMA_DEBUG_LOG(" Freed DedicatedMemory MemoryTypeIndex=%u", memTypeIndex);
        7547 }
        7548 
        7549 #if VMA_STATS_STRING_ENABLED
        7550 
        7551 void VmaAllocator_T::PrintDetailedMap(VmaJsonWriter& json)
        7552 {
        7553  bool dedicatedAllocationsStarted = false;
        7554  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
        7555  {
        7556  VmaMutexLock dedicatedAllocationsLock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
        7557  AllocationVectorType* const pDedicatedAllocVector = m_pDedicatedAllocations[memTypeIndex];
        7558  VMA_ASSERT(pDedicatedAllocVector);
        7559  if(pDedicatedAllocVector->empty() == false)
        7560  {
        7561  if(dedicatedAllocationsStarted == false)
        7562  {
        7563  dedicatedAllocationsStarted = true;
        7564  json.WriteString("DedicatedAllocations");
        7565  json.BeginObject();
        7566  }
        7567 
        7568  json.BeginString("Type ");
        7569  json.ContinueString(memTypeIndex);
        7570  json.EndString();
        7571 
        7572  json.BeginArray();
        7573 
        7574  for(size_t i = 0; i < pDedicatedAllocVector->size(); ++i)
        7575  {
        7576  const VmaAllocation hAlloc = (*pDedicatedAllocVector)[i];
        7577  json.BeginObject(true);
        7578 
        7579  json.WriteString("Type");
        7580  json.WriteString(VMA_SUBALLOCATION_TYPE_NAMES[hAlloc->GetSuballocationType()]);
        7581 
        7582  json.WriteString("Size");
        7583  json.WriteNumber(hAlloc->GetSize());
        7584 
        7585  const void* pUserData = hAlloc->GetUserData();
        7586  if(pUserData != VMA_NULL)
        7587  {
        7588  json.WriteString("UserData");
        7589  if(hAlloc->IsUserDataString())
        7590  {
        7591  json.WriteString((const char*)pUserData);
        7592  }
        7593  else
        7594  {
        7595  json.BeginString();
        7596  json.ContinueString_Pointer(pUserData);
        7597  json.EndString();
        7598  }
        7599  }
        7600 
        7601  json.EndObject();
        7602  }
        7603 
        7604  json.EndArray();
        7605  }
        7606  }
        7607  if(dedicatedAllocationsStarted)
        7608  {
        7609  json.EndObject();
        7610  }
        7611 
        7612  {
        7613  bool allocationsStarted = false;
        7614  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
        7615  {
        7616  if(m_pBlockVectors[memTypeIndex]->IsEmpty() == false)
        7617  {
        7618  if(allocationsStarted == false)
        7619  {
        7620  allocationsStarted = true;
        7621  json.WriteString("DefaultPools");
        7622  json.BeginObject();
        7623  }
        7624 
        7625  json.BeginString("Type ");
        7626  json.ContinueString(memTypeIndex);
        7627  json.EndString();
        7628 
        7629  m_pBlockVectors[memTypeIndex]->PrintDetailedMap(json);
        7630  }
        7631  }
        7632  if(allocationsStarted)
        7633  {
        7634  json.EndObject();
        7635  }
        7636  }
        7637 
        7638  {
        7639  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
        7640  const size_t poolCount = m_Pools.size();
        7641  if(poolCount > 0)
        7642  {
        7643  json.WriteString("Pools");
        7644  json.BeginArray();
        7645  for(size_t poolIndex = 0; poolIndex < poolCount; ++poolIndex)
        7646  {
        7647  m_Pools[poolIndex]->m_BlockVector.PrintDetailedMap(json);
        7648  }
        7649  json.EndArray();
        7650  }
        7651  }
        7652 }
        7653 
        7654 #endif // #if VMA_STATS_STRING_ENABLED
        7655 
        7656 static VkResult AllocateMemoryForImage(
        7657  VmaAllocator allocator,
        7658  VkImage image,
        7659  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        7660  VmaSuballocationType suballocType,
        7661  VmaAllocation* pAllocation)
        7662 {
        7663  VMA_ASSERT(allocator && (image != VK_NULL_HANDLE) && pAllocationCreateInfo && pAllocation);
        7664 
        7665  VkMemoryRequirements vkMemReq = {};
        7666  bool requiresDedicatedAllocation = false;
        7667  bool prefersDedicatedAllocation = false;
        7668  allocator->GetImageMemoryRequirements(image, vkMemReq,
        7669  requiresDedicatedAllocation, prefersDedicatedAllocation);
        7670 
        7671  return allocator->AllocateMemory(
        7672  vkMemReq,
        7673  requiresDedicatedAllocation,
        7674  prefersDedicatedAllocation,
        7675  VK_NULL_HANDLE, // dedicatedBuffer
        7676  image, // dedicatedImage
        7677  *pAllocationCreateInfo,
        7678  suballocType,
        7679  pAllocation);
        7680 }
        7681 
        7683 // Public interface
        7684 
        7685 VkResult vmaCreateAllocator(
        7686  const VmaAllocatorCreateInfo* pCreateInfo,
        7687  VmaAllocator* pAllocator)
        7688 {
        7689  VMA_ASSERT(pCreateInfo && pAllocator);
        7690  VMA_DEBUG_LOG("vmaCreateAllocator");
        7691  *pAllocator = vma_new(pCreateInfo->pAllocationCallbacks, VmaAllocator_T)(pCreateInfo);
        7692  return VK_SUCCESS;
        7693 }
        7694 
        7695 void vmaDestroyAllocator(
        7696  VmaAllocator allocator)
        7697 {
        7698  if(allocator != VK_NULL_HANDLE)
        7699  {
        7700  VMA_DEBUG_LOG("vmaDestroyAllocator");
        7701  VkAllocationCallbacks allocationCallbacks = allocator->m_AllocationCallbacks;
        7702  vma_delete(&allocationCallbacks, allocator);
        7703  }
        7704 }
        7705 
        7706 void vmaGetPhysicalDeviceProperties(
        7707  VmaAllocator allocator,
        7708  const VkPhysicalDeviceProperties **ppPhysicalDeviceProperties)
        7709 {
        7710  VMA_ASSERT(allocator && ppPhysicalDeviceProperties);
        7711  *ppPhysicalDeviceProperties = &allocator->m_PhysicalDeviceProperties;
        7712 }
        7713 
        7714 void vmaGetMemoryProperties(
        7715  VmaAllocator allocator,
        7716  const VkPhysicalDeviceMemoryProperties** ppPhysicalDeviceMemoryProperties)
        7717 {
        7718  VMA_ASSERT(allocator && ppPhysicalDeviceMemoryProperties);
        7719  *ppPhysicalDeviceMemoryProperties = &allocator->m_MemProps;
        7720 }
        7721 
        7722 void vmaGetMemoryTypeProperties(
        7723  VmaAllocator allocator,
        7724  uint32_t memoryTypeIndex,
        7725  VkMemoryPropertyFlags* pFlags)
        7726 {
        7727  VMA_ASSERT(allocator && pFlags);
        7728  VMA_ASSERT(memoryTypeIndex < allocator->GetMemoryTypeCount());
        7729  *pFlags = allocator->m_MemProps.memoryTypes[memoryTypeIndex].propertyFlags;
        7730 }
        7731 
        7732 void vmaSetCurrentFrameIndex(
        7733  VmaAllocator allocator,
        7734  uint32_t frameIndex)
        7735 {
        7736  VMA_ASSERT(allocator);
        7737  VMA_ASSERT(frameIndex != VMA_FRAME_INDEX_LOST);
        7738 
        7739  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        7740 
        7741  allocator->SetCurrentFrameIndex(frameIndex);
        7742 }
        7743 
        7744 void vmaCalculateStats(
        7745  VmaAllocator allocator,
        7746  VmaStats* pStats)
        7747 {
        7748  VMA_ASSERT(allocator && pStats);
        7749  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        7750  allocator->CalculateStats(pStats);
        7751 }
        7752 
        7753 #if VMA_STATS_STRING_ENABLED
        7754 
        7755 void vmaBuildStatsString(
        7756  VmaAllocator allocator,
        7757  char** ppStatsString,
        7758  VkBool32 detailedMap)
        7759 {
        7760  VMA_ASSERT(allocator && ppStatsString);
        7761  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        7762 
        7763  VmaStringBuilder sb(allocator);
        7764  {
        7765  VmaJsonWriter json(allocator->GetAllocationCallbacks(), sb);
        7766  json.BeginObject();
        7767 
        7768  VmaStats stats;
        7769  allocator->CalculateStats(&stats);
        7770 
        7771  json.WriteString("Total");
        7772  VmaPrintStatInfo(json, stats.total);
        7773 
        7774  for(uint32_t heapIndex = 0; heapIndex < allocator->GetMemoryHeapCount(); ++heapIndex)
        7775  {
        7776  json.BeginString("Heap ");
        7777  json.ContinueString(heapIndex);
        7778  json.EndString();
        7779  json.BeginObject();
        7780 
        7781  json.WriteString("Size");
        7782  json.WriteNumber(allocator->m_MemProps.memoryHeaps[heapIndex].size);
        7783 
        7784  json.WriteString("Flags");
        7785  json.BeginArray(true);
        7786  if((allocator->m_MemProps.memoryHeaps[heapIndex].flags & VK_MEMORY_HEAP_DEVICE_LOCAL_BIT) != 0)
        7787  {
        7788  json.WriteString("DEVICE_LOCAL");
        7789  }
        7790  json.EndArray();
        7791 
        7792  if(stats.memoryHeap[heapIndex].blockCount > 0)
        7793  {
        7794  json.WriteString("Stats");
        7795  VmaPrintStatInfo(json, stats.memoryHeap[heapIndex]);
        7796  }
        7797 
        7798  for(uint32_t typeIndex = 0; typeIndex < allocator->GetMemoryTypeCount(); ++typeIndex)
        7799  {
        7800  if(allocator->MemoryTypeIndexToHeapIndex(typeIndex) == heapIndex)
        7801  {
        7802  json.BeginString("Type ");
        7803  json.ContinueString(typeIndex);
        7804  json.EndString();
        7805 
        7806  json.BeginObject();
        7807 
        7808  json.WriteString("Flags");
        7809  json.BeginArray(true);
        7810  VkMemoryPropertyFlags flags = allocator->m_MemProps.memoryTypes[typeIndex].propertyFlags;
        7811  if((flags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT) != 0)
        7812  {
        7813  json.WriteString("DEVICE_LOCAL");
        7814  }
        7815  if((flags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
        7816  {
        7817  json.WriteString("HOST_VISIBLE");
        7818  }
        7819  if((flags & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT) != 0)
        7820  {
        7821  json.WriteString("HOST_COHERENT");
        7822  }
        7823  if((flags & VK_MEMORY_PROPERTY_HOST_CACHED_BIT) != 0)
        7824  {
        7825  json.WriteString("HOST_CACHED");
        7826  }
        7827  if((flags & VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT) != 0)
        7828  {
        7829  json.WriteString("LAZILY_ALLOCATED");
        7830  }
        7831  json.EndArray();
        7832 
        7833  if(stats.memoryType[typeIndex].blockCount > 0)
        7834  {
        7835  json.WriteString("Stats");
        7836  VmaPrintStatInfo(json, stats.memoryType[typeIndex]);
        7837  }
        7838 
        7839  json.EndObject();
        7840  }
        7841  }
        7842 
        7843  json.EndObject();
        7844  }
        7845  if(detailedMap == VK_TRUE)
        7846  {
        7847  allocator->PrintDetailedMap(json);
        7848  }
        7849 
        7850  json.EndObject();
        7851  }
        7852 
        7853  const size_t len = sb.GetLength();
        7854  char* const pChars = vma_new_array(allocator, char, len + 1);
        7855  if(len > 0)
        7856  {
        7857  memcpy(pChars, sb.GetData(), len);
        7858  }
        7859  pChars[len] = '\0';
        7860  *ppStatsString = pChars;
        7861 }
        7862 
        7863 void vmaFreeStatsString(
        7864  VmaAllocator allocator,
        7865  char* pStatsString)
        7866 {
        7867  if(pStatsString != VMA_NULL)
        7868  {
        7869  VMA_ASSERT(allocator);
        7870  size_t len = strlen(pStatsString);
        7871  vma_delete_array(allocator, pStatsString, len + 1);
        7872  }
        7873 }
        7874 
        7875 #endif // #if VMA_STATS_STRING_ENABLED
        7876 
        7879 VkResult vmaFindMemoryTypeIndex(
        7880  VmaAllocator allocator,
        7881  uint32_t memoryTypeBits,
        7882  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        7883  uint32_t* pMemoryTypeIndex)
        7884 {
        7885  VMA_ASSERT(allocator != VK_NULL_HANDLE);
        7886  VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
        7887  VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
        7888 
        7889  uint32_t requiredFlags = pAllocationCreateInfo->requiredFlags;
        7890  uint32_t preferredFlags = pAllocationCreateInfo->preferredFlags;
        7891  if(preferredFlags == 0)
        7892  {
        7893  preferredFlags = requiredFlags;
        7894  }
        7895  // preferredFlags, if not 0, must be a superset of requiredFlags.
        7896  VMA_ASSERT((requiredFlags & ~preferredFlags) == 0);
        7897 
        7898  // Convert usage to requiredFlags and preferredFlags.
        7899  switch(pAllocationCreateInfo->usage)
        7900  {
        7901  case VMA_MEMORY_USAGE_UNKNOWN:
        7902  break;
        7903  case VMA_MEMORY_USAGE_GPU_ONLY:
        7904  preferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
        7905  break;
        7906  case VMA_MEMORY_USAGE_CPU_ONLY:
        7907  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
        7908  break;
        7909  case VMA_MEMORY_USAGE_CPU_TO_GPU:
        7910  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
        7911  preferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
        7912  break;
        7913  case VMA_MEMORY_USAGE_GPU_TO_CPU:
        7914  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
        7915  preferredFlags |= VK_MEMORY_PROPERTY_HOST_COHERENT_BIT | VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
        7916  break;
        7917  default:
        7918  break;
        7919  }
        7920 
        7921  *pMemoryTypeIndex = UINT32_MAX;
        7922  uint32_t minCost = UINT32_MAX;
        7923  for(uint32_t memTypeIndex = 0, memTypeBit = 1;
        7924  memTypeIndex < allocator->GetMemoryTypeCount();
        7925  ++memTypeIndex, memTypeBit <<= 1)
        7926  {
        7927  // This memory type is acceptable according to memoryTypeBits bitmask.
        7928  if((memTypeBit & memoryTypeBits) != 0)
        7929  {
        7930  const VkMemoryPropertyFlags currFlags =
        7931  allocator->m_MemProps.memoryTypes[memTypeIndex].propertyFlags;
        7932  // This memory type contains requiredFlags.
        7933  if((requiredFlags & ~currFlags) == 0)
        7934  {
        7935  // Calculate cost as number of bits from preferredFlags not present in this memory type.
        7936  uint32_t currCost = CountBitsSet(preferredFlags & ~currFlags);
        7937  // Remember memory type with lowest cost.
        7938  if(currCost < minCost)
        7939  {
        7940  *pMemoryTypeIndex = memTypeIndex;
        7941  if(currCost == 0)
        7942  {
        7943  return VK_SUCCESS;
        7944  }
        7945  minCost = currCost;
        7946  }
        7947  }
        7948  }
        7949  }
        7950  return (*pMemoryTypeIndex != UINT32_MAX) ? VK_SUCCESS : VK_ERROR_FEATURE_NOT_PRESENT;
        7951 }
        7952 
        7953 VkResult vmaCreatePool(
        7954  VmaAllocator allocator,
        7955  const VmaPoolCreateInfo* pCreateInfo,
        7956  VmaPool* pPool)
        7957 {
        7958  VMA_ASSERT(allocator && pCreateInfo && pPool);
        7959 
        7960  VMA_DEBUG_LOG("vmaCreatePool");
        7961 
        7962  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        7963 
        7964  return allocator->CreatePool(pCreateInfo, pPool);
        7965 }
        7966 
        7967 void vmaDestroyPool(
        7968  VmaAllocator allocator,
        7969  VmaPool pool)
        7970 {
        7971  VMA_ASSERT(allocator);
        7972 
        7973  if(pool == VK_NULL_HANDLE)
        7974  {
        7975  return;
        7976  }
        7977 
        7978  VMA_DEBUG_LOG("vmaDestroyPool");
        7979 
        7980  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        7981 
        7982  allocator->DestroyPool(pool);
        7983 }
        7984 
        7985 void vmaGetPoolStats(
        7986  VmaAllocator allocator,
        7987  VmaPool pool,
        7988  VmaPoolStats* pPoolStats)
        7989 {
        7990  VMA_ASSERT(allocator && pool && pPoolStats);
        7991 
        7992  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        7993 
        7994  allocator->GetPoolStats(pool, pPoolStats);
        7995 }
        7996 
        7997 void vmaMakePoolAllocationsLost(
        7998  VmaAllocator allocator,
        7999  VmaPool pool,
        8000  size_t* pLostAllocationCount)
        8001 {
        8002  VMA_ASSERT(allocator && pool);
        8003 
        8004  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8005 
        8006  allocator->MakePoolAllocationsLost(pool, pLostAllocationCount);
        8007 }
        8008 
        8009 VkResult vmaAllocateMemory(
        8010  VmaAllocator allocator,
        8011  const VkMemoryRequirements* pVkMemoryRequirements,
        8012  const VmaAllocationCreateInfo* pCreateInfo,
        8013  VmaAllocation* pAllocation,
        8014  VmaAllocationInfo* pAllocationInfo)
        8015 {
        8016  VMA_ASSERT(allocator && pVkMemoryRequirements && pCreateInfo && pAllocation);
        8017 
        8018  VMA_DEBUG_LOG("vmaAllocateMemory");
        8019 
        8020  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8021 
        8022  VkResult result = allocator->AllocateMemory(
        8023  *pVkMemoryRequirements,
        8024  false, // requiresDedicatedAllocation
        8025  false, // prefersDedicatedAllocation
        8026  VK_NULL_HANDLE, // dedicatedBuffer
        8027  VK_NULL_HANDLE, // dedicatedImage
        8028  *pCreateInfo,
        8029  VMA_SUBALLOCATION_TYPE_UNKNOWN,
        8030  pAllocation);
        8031 
        8032  if(pAllocationInfo && result == VK_SUCCESS)
        8033  {
        8034  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
        8035  }
        8036 
        8037  return result;
        8038 }
        8039 
        8040 VkResult vmaAllocateMemoryForBuffer(
        8041  VmaAllocator allocator,
        8042  VkBuffer buffer,
        8043  const VmaAllocationCreateInfo* pCreateInfo,
        8044  VmaAllocation* pAllocation,
        8045  VmaAllocationInfo* pAllocationInfo)
        8046 {
        8047  VMA_ASSERT(allocator && buffer != VK_NULL_HANDLE && pCreateInfo && pAllocation);
        8048 
        8049  VMA_DEBUG_LOG("vmaAllocateMemoryForBuffer");
        8050 
        8051  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8052 
        8053  VkMemoryRequirements vkMemReq = {};
        8054  bool requiresDedicatedAllocation = false;
        8055  bool prefersDedicatedAllocation = false;
        8056  allocator->GetBufferMemoryRequirements(buffer, vkMemReq,
        8057  requiresDedicatedAllocation,
        8058  prefersDedicatedAllocation);
        8059 
        8060  VkResult result = allocator->AllocateMemory(
        8061  vkMemReq,
        8062  requiresDedicatedAllocation,
        8063  prefersDedicatedAllocation,
        8064  buffer, // dedicatedBuffer
        8065  VK_NULL_HANDLE, // dedicatedImage
        8066  *pCreateInfo,
        8067  VMA_SUBALLOCATION_TYPE_BUFFER,
        8068  pAllocation);
        8069 
        8070  if(pAllocationInfo && result == VK_SUCCESS)
        8071  {
        8072  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
        8073  }
        8074 
        8075  return result;
        8076 }
        8077 
        8078 VkResult vmaAllocateMemoryForImage(
        8079  VmaAllocator allocator,
        8080  VkImage image,
        8081  const VmaAllocationCreateInfo* pCreateInfo,
        8082  VmaAllocation* pAllocation,
        8083  VmaAllocationInfo* pAllocationInfo)
        8084 {
        8085  VMA_ASSERT(allocator && image != VK_NULL_HANDLE && pCreateInfo && pAllocation);
        8086 
        8087  VMA_DEBUG_LOG("vmaAllocateMemoryForImage");
        8088 
        8089  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8090 
        8091  VkResult result = AllocateMemoryForImage(
        8092  allocator,
        8093  image,
        8094  pCreateInfo,
        8095  VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN,
        8096  pAllocation);
        8097 
        8098  if(pAllocationInfo && result == VK_SUCCESS)
        8099  {
        8100  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
        8101  }
        8102 
        8103  return result;
        8104 }
        8105 
        8106 void vmaFreeMemory(
        8107  VmaAllocator allocator,
        8108  VmaAllocation allocation)
        8109 {
        8110  VMA_ASSERT(allocator && allocation);
        8111 
        8112  VMA_DEBUG_LOG("vmaFreeMemory");
        8113 
        8114  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8115 
        8116  allocator->FreeMemory(allocation);
        8117 }
        8118 
        8119 void vmaGetAllocationInfo(
        8120  VmaAllocator allocator,
        8121  VmaAllocation allocation,
        8122  VmaAllocationInfo* pAllocationInfo)
        8123 {
        8124  VMA_ASSERT(allocator && allocation && pAllocationInfo);
        8125 
        8126  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8127 
        8128  allocator->GetAllocationInfo(allocation, pAllocationInfo);
        8129 }
        8130 
        8131 void vmaSetAllocationUserData(
        8132  VmaAllocator allocator,
        8133  VmaAllocation allocation,
        8134  void* pUserData)
        8135 {
        8136  VMA_ASSERT(allocator && allocation);
        8137 
        8138  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8139 
        8140  allocation->SetUserData(allocator, pUserData);
        8141 }
        8142 
        8143 void vmaCreateLostAllocation(
        8144  VmaAllocator allocator,
        8145  VmaAllocation* pAllocation)
        8146 {
        8147  VMA_ASSERT(allocator && pAllocation);
        8148 
        8149  VMA_DEBUG_GLOBAL_MUTEX_LOCK;
        8150 
        8151  allocator->CreateLostAllocation(pAllocation);
        8152 }
        8153 
        8154 VkResult vmaMapMemory(
        8155  VmaAllocator allocator,
        8156  VmaAllocation allocation,
        8157  void** ppData)
        8158 {
        8159  VMA_ASSERT(allocator && allocation && ppData);
        8160 
        8161  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8162 
        8163  return allocator->Map(allocation, ppData);
        8164 }
        8165 
        8166 void vmaUnmapMemory(
        8167  VmaAllocator allocator,
        8168  VmaAllocation allocation)
        8169 {
        8170  VMA_ASSERT(allocator && allocation);
        8171 
        8172  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8173 
        8174  allocator->Unmap(allocation);
        8175 }
        8176 
        8177 VkResult vmaDefragment(
        8178  VmaAllocator allocator,
        8179  VmaAllocation* pAllocations,
        8180  size_t allocationCount,
        8181  VkBool32* pAllocationsChanged,
        8182  const VmaDefragmentationInfo *pDefragmentationInfo,
        8183  VmaDefragmentationStats* pDefragmentationStats)
        8184 {
        8185  VMA_ASSERT(allocator && pAllocations);
        8186 
        8187  VMA_DEBUG_LOG("vmaDefragment");
        8188 
        8189  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8190 
        8191  return allocator->Defragment(pAllocations, allocationCount, pAllocationsChanged, pDefragmentationInfo, pDefragmentationStats);
        8192 }
        8193 
        8194 VkResult vmaCreateBuffer(
        8195  VmaAllocator allocator,
        8196  const VkBufferCreateInfo* pBufferCreateInfo,
        8197  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        8198  VkBuffer* pBuffer,
        8199  VmaAllocation* pAllocation,
        8200  VmaAllocationInfo* pAllocationInfo)
        8201 {
        8202  VMA_ASSERT(allocator && pBufferCreateInfo && pAllocationCreateInfo && pBuffer && pAllocation);
        8203 
        8204  VMA_DEBUG_LOG("vmaCreateBuffer");
        8205 
        8206  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8207 
        8208  *pBuffer = VK_NULL_HANDLE;
        8209  *pAllocation = VK_NULL_HANDLE;
        8210 
        8211  // 1. Create VkBuffer.
        8212  VkResult res = (*allocator->GetVulkanFunctions().vkCreateBuffer)(
        8213  allocator->m_hDevice,
        8214  pBufferCreateInfo,
        8215  allocator->GetAllocationCallbacks(),
        8216  pBuffer);
        8217  if(res >= 0)
        8218  {
        8219  // 2. vkGetBufferMemoryRequirements.
        8220  VkMemoryRequirements vkMemReq = {};
        8221  bool requiresDedicatedAllocation = false;
        8222  bool prefersDedicatedAllocation = false;
        8223  allocator->GetBufferMemoryRequirements(*pBuffer, vkMemReq,
        8224  requiresDedicatedAllocation, prefersDedicatedAllocation);
        8225 
        8226  // 3. Allocate memory using allocator.
        8227  res = allocator->AllocateMemory(
        8228  vkMemReq,
        8229  requiresDedicatedAllocation,
        8230  prefersDedicatedAllocation,
        8231  *pBuffer, // dedicatedBuffer
        8232  VK_NULL_HANDLE, // dedicatedImage
        8233  *pAllocationCreateInfo,
        8234  VMA_SUBALLOCATION_TYPE_BUFFER,
        8235  pAllocation);
        8236  if(res >= 0)
        8237  {
        8238  // 3. Bind buffer with memory.
        8239  res = (*allocator->GetVulkanFunctions().vkBindBufferMemory)(
        8240  allocator->m_hDevice,
        8241  *pBuffer,
        8242  (*pAllocation)->GetMemory(),
        8243  (*pAllocation)->GetOffset());
        8244  if(res >= 0)
        8245  {
        8246  // All steps succeeded.
        8247  if(pAllocationInfo != VMA_NULL)
        8248  {
        8249  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
        8250  }
        8251  return VK_SUCCESS;
        8252  }
        8253  allocator->FreeMemory(*pAllocation);
        8254  *pAllocation = VK_NULL_HANDLE;
        8255  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
        8256  *pBuffer = VK_NULL_HANDLE;
        8257  return res;
        8258  }
        8259  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
        8260  *pBuffer = VK_NULL_HANDLE;
        8261  return res;
        8262  }
        8263  return res;
        8264 }
        8265 
        8266 void vmaDestroyBuffer(
        8267  VmaAllocator allocator,
        8268  VkBuffer buffer,
        8269  VmaAllocation allocation)
        8270 {
        8271  if(buffer != VK_NULL_HANDLE)
        8272  {
        8273  VMA_ASSERT(allocator);
        8274 
        8275  VMA_DEBUG_LOG("vmaDestroyBuffer");
        8276 
        8277  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8278 
        8279  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, buffer, allocator->GetAllocationCallbacks());
        8280 
        8281  allocator->FreeMemory(allocation);
        8282  }
        8283 }
        8284 
        8285 VkResult vmaCreateImage(
        8286  VmaAllocator allocator,
        8287  const VkImageCreateInfo* pImageCreateInfo,
        8288  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        8289  VkImage* pImage,
        8290  VmaAllocation* pAllocation,
        8291  VmaAllocationInfo* pAllocationInfo)
        8292 {
        8293  VMA_ASSERT(allocator && pImageCreateInfo && pAllocationCreateInfo && pImage && pAllocation);
        8294 
        8295  VMA_DEBUG_LOG("vmaCreateImage");
        8296 
        8297  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8298 
        8299  *pImage = VK_NULL_HANDLE;
        8300  *pAllocation = VK_NULL_HANDLE;
        8301 
        8302  // 1. Create VkImage.
        8303  VkResult res = (*allocator->GetVulkanFunctions().vkCreateImage)(
        8304  allocator->m_hDevice,
        8305  pImageCreateInfo,
        8306  allocator->GetAllocationCallbacks(),
        8307  pImage);
        8308  if(res >= 0)
        8309  {
        8310  VmaSuballocationType suballocType = pImageCreateInfo->tiling == VK_IMAGE_TILING_OPTIMAL ?
        8311  VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL :
        8312  VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR;
        8313 
        8314  // 2. Allocate memory using allocator.
        8315  res = AllocateMemoryForImage(allocator, *pImage, pAllocationCreateInfo, suballocType, pAllocation);
        8316  if(res >= 0)
        8317  {
        8318  // 3. Bind image with memory.
        8319  res = (*allocator->GetVulkanFunctions().vkBindImageMemory)(
        8320  allocator->m_hDevice,
        8321  *pImage,
        8322  (*pAllocation)->GetMemory(),
        8323  (*pAllocation)->GetOffset());
        8324  if(res >= 0)
        8325  {
        8326  // All steps succeeded.
        8327  if(pAllocationInfo != VMA_NULL)
        8328  {
        8329  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
        8330  }
        8331  return VK_SUCCESS;
        8332  }
        8333  allocator->FreeMemory(*pAllocation);
        8334  *pAllocation = VK_NULL_HANDLE;
        8335  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, *pImage, allocator->GetAllocationCallbacks());
        8336  *pImage = VK_NULL_HANDLE;
        8337  return res;
        8338  }
        8339  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, *pImage, allocator->GetAllocationCallbacks());
        8340  *pImage = VK_NULL_HANDLE;
        8341  return res;
        8342  }
        8343  return res;
        8344 }
        8345 
        8346 void vmaDestroyImage(
        8347  VmaAllocator allocator,
        8348  VkImage image,
        8349  VmaAllocation allocation)
        8350 {
        8351  if(image != VK_NULL_HANDLE)
        8352  {
        8353  VMA_ASSERT(allocator);
        8354 
        8355  VMA_DEBUG_LOG("vmaDestroyImage");
        8356 
        8357  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8358 
        8359  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, image, allocator->GetAllocationCallbacks());
        8360 
        8361  allocator->FreeMemory(allocation);
        8362  }
        8363 }
        8364 
        8365 #endif // #ifdef VMA_IMPLEMENTATION
        VmaVulkanFunctions::vkGetPhysicalDeviceProperties
        PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties
        Definition: vk_mem_alloc.h:670
        -
        VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT
        Set this flag if the allocation should have its own memory block.
        Definition: vk_mem_alloc.h:893
        +Go to the documentation of this file.
        1 //
        2 // Copyright (c) 2017 Advanced Micro Devices, Inc. All rights reserved.
        3 //
        4 // Permission is hereby granted, free of charge, to any person obtaining a copy
        5 // of this software and associated documentation files (the "Software"), to deal
        6 // in the Software without restriction, including without limitation the rights
        7 // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        8 // copies of the Software, and to permit persons to whom the Software is
        9 // furnished to do so, subject to the following conditions:
        10 //
        11 // The above copyright notice and this permission notice shall be included in
        12 // all copies or substantial portions of the Software.
        13 //
        14 // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        15 // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        16 // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        17 // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        18 // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        19 // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
        20 // THE SOFTWARE.
        21 //
        22 
        23 #ifndef AMD_VULKAN_MEMORY_ALLOCATOR_H
        24 #define AMD_VULKAN_MEMORY_ALLOCATOR_H
        25 
        26 #ifdef __cplusplus
        27 extern "C" {
        28 #endif
        29 
        688 #include <vulkan/vulkan.h>
        689 
        690 VK_DEFINE_HANDLE(VmaAllocator)
        691 
        692 typedef void (VKAPI_PTR *PFN_vmaAllocateDeviceMemoryFunction)(
        694  VmaAllocator allocator,
        695  uint32_t memoryType,
        696  VkDeviceMemory memory,
        697  VkDeviceSize size);
        699 typedef void (VKAPI_PTR *PFN_vmaFreeDeviceMemoryFunction)(
        700  VmaAllocator allocator,
        701  uint32_t memoryType,
        702  VkDeviceMemory memory,
        703  VkDeviceSize size);
        704 
        712 typedef struct VmaDeviceMemoryCallbacks {
        714  PFN_vmaAllocateDeviceMemoryFunction pfnAllocate;
        716  PFN_vmaFreeDeviceMemoryFunction pfnFree;
        717 } VmaDeviceMemoryCallbacks;
        718 
        720 typedef enum VmaAllocatorCreateFlagBits {
        725  VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT = 0x00000001,
        753  VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT = 0x00000002,
        754 
        755  VMA_ALLOCATOR_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
        756 } VmaAllocatorCreateFlagBits;
        757 typedef VkFlags VmaAllocatorCreateFlags;
        758 
        763 typedef struct VmaVulkanFunctions {
        764  PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties;
        765  PFN_vkGetPhysicalDeviceMemoryProperties vkGetPhysicalDeviceMemoryProperties;
        766  PFN_vkAllocateMemory vkAllocateMemory;
        767  PFN_vkFreeMemory vkFreeMemory;
        768  PFN_vkMapMemory vkMapMemory;
        769  PFN_vkUnmapMemory vkUnmapMemory;
        770  PFN_vkBindBufferMemory vkBindBufferMemory;
        771  PFN_vkBindImageMemory vkBindImageMemory;
        772  PFN_vkGetBufferMemoryRequirements vkGetBufferMemoryRequirements;
        773  PFN_vkGetImageMemoryRequirements vkGetImageMemoryRequirements;
        774  PFN_vkCreateBuffer vkCreateBuffer;
        775  PFN_vkDestroyBuffer vkDestroyBuffer;
        776  PFN_vkCreateImage vkCreateImage;
        777  PFN_vkDestroyImage vkDestroyImage;
        778  PFN_vkGetBufferMemoryRequirements2KHR vkGetBufferMemoryRequirements2KHR;
        779  PFN_vkGetImageMemoryRequirements2KHR vkGetImageMemoryRequirements2KHR;
        780 } VmaVulkanFunctions;
        781 
        783 typedef struct VmaAllocatorCreateInfo
        784 {
        786  VmaAllocatorCreateFlags flags;
        788 
        789  VkPhysicalDevice physicalDevice;
        791 
        792  VkDevice device;
        794 
        795  VkDeviceSize preferredLargeHeapBlockSize;
        797 
        798  VkDeviceSize preferredSmallHeapBlockSize;
        800 
        801  const VkAllocationCallbacks* pAllocationCallbacks;
        803 
        804  const VmaDeviceMemoryCallbacks* pDeviceMemoryCallbacks;
        818  uint32_t frameInUseCount;
        842  const VkDeviceSize* pHeapSizeLimit;
        854  const VmaVulkanFunctions* pVulkanFunctions;
        855 } VmaAllocatorCreateInfo;
        856 
        858 VkResult vmaCreateAllocator(
        859  const VmaAllocatorCreateInfo* pCreateInfo,
        860  VmaAllocator* pAllocator);
        861 
        863 void vmaDestroyAllocator(
        864  VmaAllocator allocator);
        865 
        870 void vmaGetPhysicalDeviceProperties(
        871  VmaAllocator allocator,
        872  const VkPhysicalDeviceProperties** ppPhysicalDeviceProperties);
        873 
        878 void vmaGetMemoryProperties(
        879  VmaAllocator allocator,
        880  const VkPhysicalDeviceMemoryProperties** ppPhysicalDeviceMemoryProperties);
        881 
        888 void vmaGetMemoryTypeProperties(
        889  VmaAllocator allocator,
        890  uint32_t memoryTypeIndex,
        891  VkMemoryPropertyFlags* pFlags);
        892 
        901 void vmaSetCurrentFrameIndex(
        902  VmaAllocator allocator,
        903  uint32_t frameIndex);
        904 
        907 typedef struct VmaStatInfo
        908 {
        910  uint32_t blockCount;
        912  uint32_t allocationCount;
        914  uint32_t unusedRangeCount;
        916  VkDeviceSize usedBytes;
        918  VkDeviceSize unusedBytes;
        919  VkDeviceSize allocationSizeMin, allocationSizeAvg, allocationSizeMax;
        920  VkDeviceSize unusedRangeSizeMin, unusedRangeSizeAvg, unusedRangeSizeMax;
        921 } VmaStatInfo;
        922 
        924 typedef struct VmaStats
        925 {
        926  VmaStatInfo memoryType[VK_MAX_MEMORY_TYPES];
        927  VmaStatInfo memoryHeap[VK_MAX_MEMORY_HEAPS];
        928  VmaStatInfo total;
        929 } VmaStats;
        930 
        932 void vmaCalculateStats(
        933  VmaAllocator allocator,
        934  VmaStats* pStats);
        935 
        936 #define VMA_STATS_STRING_ENABLED 1
        937 
        938 #if VMA_STATS_STRING_ENABLED
        939 
        941 
        943 void vmaBuildStatsString(
        944  VmaAllocator allocator,
        945  char** ppStatsString,
        946  VkBool32 detailedMap);
        947 
        948 void vmaFreeStatsString(
        949  VmaAllocator allocator,
        950  char* pStatsString);
        951 
        952 #endif // #if VMA_STATS_STRING_ENABLED
        953 
        954 VK_DEFINE_HANDLE(VmaPool)
        955 
        956 typedef enum VmaMemoryUsage
        957 {
        961  VMA_MEMORY_USAGE_UNKNOWN = 0,
        974  VMA_MEMORY_USAGE_GPU_ONLY = 1,
        983  VMA_MEMORY_USAGE_CPU_ONLY = 2,
        989  VMA_MEMORY_USAGE_CPU_TO_GPU = 3,
        994  VMA_MEMORY_USAGE_GPU_TO_CPU = 4,
        995  VMA_MEMORY_USAGE_MAX_ENUM = 0x7FFFFFFF
        996 } VmaMemoryUsage;
        997 
        999 typedef enum VmaAllocationCreateFlagBits {
        1011  VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT = 0x00000001,
        1012 
        1022  VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT = 0x00000002,
        1035  VMA_ALLOCATION_CREATE_MAPPED_BIT = 0x00000004,
        1048  VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT = 0x00000008,
        1055  VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT = 0x00000010,
        1061  VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT = 0x00000020,
        1062 
        1063  VMA_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
        1064 } VmaAllocationCreateFlagBits;
        1065 typedef VkFlags VmaAllocationCreateFlags;
        1066 
        1067 typedef struct VmaAllocationCreateInfo
        1068 {
        1070  VmaAllocationCreateFlags flags;
        1076  VmaMemoryUsage usage;
        1081  VkMemoryPropertyFlags requiredFlags;
        1086  VkMemoryPropertyFlags preferredFlags;
        1094  uint32_t memoryTypeBits;
        1100  VmaPool pool;
        1107  void* pUserData;
        1108 } VmaAllocationCreateInfo;
        1109 
        1124 VkResult vmaFindMemoryTypeIndex(
        1125  VmaAllocator allocator,
        1126  uint32_t memoryTypeBits,
        1127  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        1128  uint32_t* pMemoryTypeIndex);
        1129 
        1131 typedef enum VmaPoolCreateFlagBits {
        1149  VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT = 0x00000002,
        1150 
        1151  VMA_POOL_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
        1152 } VmaPoolCreateFlagBits;
        1153 typedef VkFlags VmaPoolCreateFlags;
        1154 
        1157 typedef struct VmaPoolCreateInfo {
        1160  uint32_t memoryTypeIndex;
        1163  VmaPoolCreateFlags flags;
        1168  VkDeviceSize blockSize;
        1173  size_t minBlockCount;
        1181  size_t maxBlockCount;
        1195  uint32_t frameInUseCount;
        1196 } VmaPoolCreateInfo;
        1197 
        1200 typedef struct VmaPoolStats {
        1203  VkDeviceSize size;
        1206  VkDeviceSize unusedSize;
        1209  size_t allocationCount;
        1212  size_t unusedRangeCount;
        1219  VkDeviceSize unusedRangeSizeMax;
        1220 } VmaPoolStats;
        1221 
        1228 VkResult vmaCreatePool(
        1229  VmaAllocator allocator,
        1230  const VmaPoolCreateInfo* pCreateInfo,
        1231  VmaPool* pPool);
        1232 
        1235 void vmaDestroyPool(
        1236  VmaAllocator allocator,
        1237  VmaPool pool);
        1238 
        1245 void vmaGetPoolStats(
        1246  VmaAllocator allocator,
        1247  VmaPool pool,
        1248  VmaPoolStats* pPoolStats);
        1249 
        1256 void vmaMakePoolAllocationsLost(
        1257  VmaAllocator allocator,
        1258  VmaPool pool,
        1259  size_t* pLostAllocationCount);
        1260 
        1261 VK_DEFINE_HANDLE(VmaAllocation)
        1262 
        1263 
        1265 typedef struct VmaAllocationInfo {
        1270  uint32_t memoryType;
        1279  VkDeviceMemory deviceMemory;
        1284  VkDeviceSize offset;
        1289  VkDeviceSize size;
        1298  void* pMappedData;
        1303  void* pUserData;
        1304 } VmaAllocationInfo;
        1305 
        1316 VkResult vmaAllocateMemory(
        1317  VmaAllocator allocator,
        1318  const VkMemoryRequirements* pVkMemoryRequirements,
        1319  const VmaAllocationCreateInfo* pCreateInfo,
        1320  VmaAllocation* pAllocation,
        1321  VmaAllocationInfo* pAllocationInfo);
        1322 
        1329 VkResult vmaAllocateMemoryForBuffer(
        1330  VmaAllocator allocator,
        1331  VkBuffer buffer,
        1332  const VmaAllocationCreateInfo* pCreateInfo,
        1333  VmaAllocation* pAllocation,
        1334  VmaAllocationInfo* pAllocationInfo);
        1335 
        1337 VkResult vmaAllocateMemoryForImage(
        1338  VmaAllocator allocator,
        1339  VkImage image,
        1340  const VmaAllocationCreateInfo* pCreateInfo,
        1341  VmaAllocation* pAllocation,
        1342  VmaAllocationInfo* pAllocationInfo);
        1343 
        1345 void vmaFreeMemory(
        1346  VmaAllocator allocator,
        1347  VmaAllocation allocation);
        1348 
        1350 void vmaGetAllocationInfo(
        1351  VmaAllocator allocator,
        1352  VmaAllocation allocation,
        1353  VmaAllocationInfo* pAllocationInfo);
        1354 
        1368 void vmaSetAllocationUserData(
        1369  VmaAllocator allocator,
        1370  VmaAllocation allocation,
        1371  void* pUserData);
        1372 
        1383 void vmaCreateLostAllocation(
        1384  VmaAllocator allocator,
        1385  VmaAllocation* pAllocation);
        1386 
        1421 VkResult vmaMapMemory(
        1422  VmaAllocator allocator,
        1423  VmaAllocation allocation,
        1424  void** ppData);
        1425 
        1430 void vmaUnmapMemory(
        1431  VmaAllocator allocator,
        1432  VmaAllocation allocation);
        1433 
        1435 typedef struct VmaDefragmentationInfo {
        1440  VkDeviceSize maxBytesToMove;
        1445  uint32_t maxAllocationsToMove;
        1446 } VmaDefragmentationInfo;
        1447 
        1449 typedef struct VmaDefragmentationStats {
        1451  VkDeviceSize bytesMoved;
        1453  VkDeviceSize bytesFreed;
        1455  uint32_t allocationsMoved;
        1457  uint32_t deviceMemoryBlocksFreed;
        1458 } VmaDefragmentationStats;
        1459 
        1536 VkResult vmaDefragment(
        1537  VmaAllocator allocator,
        1538  VmaAllocation* pAllocations,
        1539  size_t allocationCount,
        1540  VkBool32* pAllocationsChanged,
        1541  const VmaDefragmentationInfo *pDefragmentationInfo,
        1542  VmaDefragmentationStats* pDefragmentationStats);
        1543 
        1570 VkResult vmaCreateBuffer(
        1571  VmaAllocator allocator,
        1572  const VkBufferCreateInfo* pBufferCreateInfo,
        1573  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        1574  VkBuffer* pBuffer,
        1575  VmaAllocation* pAllocation,
        1576  VmaAllocationInfo* pAllocationInfo);
        1577 
        1589 void vmaDestroyBuffer(
        1590  VmaAllocator allocator,
        1591  VkBuffer buffer,
        1592  VmaAllocation allocation);
        1593 
        1595 VkResult vmaCreateImage(
        1596  VmaAllocator allocator,
        1597  const VkImageCreateInfo* pImageCreateInfo,
        1598  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        1599  VkImage* pImage,
        1600  VmaAllocation* pAllocation,
        1601  VmaAllocationInfo* pAllocationInfo);
        1602 
        1614 void vmaDestroyImage(
        1615  VmaAllocator allocator,
        1616  VkImage image,
        1617  VmaAllocation allocation);
        1618 
        1619 #ifdef __cplusplus
        1620 }
        1621 #endif
        1622 
        1623 #endif // AMD_VULKAN_MEMORY_ALLOCATOR_H
        1624 
        1625 // For Visual Studio IntelliSense.
        1626 #ifdef __INTELLISENSE__
        1627 #define VMA_IMPLEMENTATION
        1628 #endif
        1629 
        1630 #ifdef VMA_IMPLEMENTATION
        1631 #undef VMA_IMPLEMENTATION
        1632 
        1633 #include <cstdint>
        1634 #include <cstdlib>
        1635 #include <cstring>
        1636 
        1637 /*******************************************************************************
        1638 CONFIGURATION SECTION
        1639 
        1640 Define some of these macros before each #include of this header or change them
        1641 here if you need other then default behavior depending on your environment.
        1642 */
        1643 
        1644 /*
        1645 Define this macro to 1 to make the library fetch pointers to Vulkan functions
        1646 internally, like:
        1647 
        1648  vulkanFunctions.vkAllocateMemory = &vkAllocateMemory;
        1649 
        1650 Define to 0 if you are going to provide you own pointers to Vulkan functions via
        1651 VmaAllocatorCreateInfo::pVulkanFunctions.
        1652 */
        1653 #ifndef VMA_STATIC_VULKAN_FUNCTIONS
        1654 #define VMA_STATIC_VULKAN_FUNCTIONS 1
        1655 #endif
        1656 
        1657 // Define this macro to 1 to make the library use STL containers instead of its own implementation.
        1658 //#define VMA_USE_STL_CONTAINERS 1
        1659 
        1660 /* Set this macro to 1 to make the library including and using STL containers:
        1661 std::pair, std::vector, std::list, std::unordered_map.
        1662 
        1663 Set it to 0 or undefined to make the library using its own implementation of
        1664 the containers.
        1665 */
        1666 #if VMA_USE_STL_CONTAINERS
        1667  #define VMA_USE_STL_VECTOR 1
        1668  #define VMA_USE_STL_UNORDERED_MAP 1
        1669  #define VMA_USE_STL_LIST 1
        1670 #endif
        1671 
        1672 #if VMA_USE_STL_VECTOR
        1673  #include <vector>
        1674 #endif
        1675 
        1676 #if VMA_USE_STL_UNORDERED_MAP
        1677  #include <unordered_map>
        1678 #endif
        1679 
        1680 #if VMA_USE_STL_LIST
        1681  #include <list>
        1682 #endif
        1683 
        1684 /*
        1685 Following headers are used in this CONFIGURATION section only, so feel free to
        1686 remove them if not needed.
        1687 */
        1688 #include <cassert> // for assert
        1689 #include <algorithm> // for min, max
        1690 #include <mutex> // for std::mutex
        1691 #include <atomic> // for std::atomic
        1692 
        1693 #if !defined(_WIN32)
        1694  #include <malloc.h> // for aligned_alloc()
        1695 #endif
        1696 
        1697 // Normal assert to check for programmer's errors, especially in Debug configuration.
        1698 #ifndef VMA_ASSERT
        1699  #ifdef _DEBUG
        1700  #define VMA_ASSERT(expr) assert(expr)
        1701  #else
        1702  #define VMA_ASSERT(expr)
        1703  #endif
        1704 #endif
        1705 
        1706 // Assert that will be called very often, like inside data structures e.g. operator[].
        1707 // Making it non-empty can make program slow.
        1708 #ifndef VMA_HEAVY_ASSERT
        1709  #ifdef _DEBUG
        1710  #define VMA_HEAVY_ASSERT(expr) //VMA_ASSERT(expr)
        1711  #else
        1712  #define VMA_HEAVY_ASSERT(expr)
        1713  #endif
        1714 #endif
        1715 
        1716 #ifndef VMA_NULL
        1717  // Value used as null pointer. Define it to e.g.: nullptr, NULL, 0, (void*)0.
        1718  #define VMA_NULL nullptr
        1719 #endif
        1720 
        1721 #ifndef VMA_ALIGN_OF
        1722  #define VMA_ALIGN_OF(type) (__alignof(type))
        1723 #endif
        1724 
        1725 #ifndef VMA_SYSTEM_ALIGNED_MALLOC
        1726  #if defined(_WIN32)
        1727  #define VMA_SYSTEM_ALIGNED_MALLOC(size, alignment) (_aligned_malloc((size), (alignment)))
        1728  #else
        1729  #define VMA_SYSTEM_ALIGNED_MALLOC(size, alignment) (aligned_alloc((alignment), (size) ))
        1730  #endif
        1731 #endif
        1732 
        1733 #ifndef VMA_SYSTEM_FREE
        1734  #if defined(_WIN32)
        1735  #define VMA_SYSTEM_FREE(ptr) _aligned_free(ptr)
        1736  #else
        1737  #define VMA_SYSTEM_FREE(ptr) free(ptr)
        1738  #endif
        1739 #endif
        1740 
        1741 #ifndef VMA_MIN
        1742  #define VMA_MIN(v1, v2) (std::min((v1), (v2)))
        1743 #endif
        1744 
        1745 #ifndef VMA_MAX
        1746  #define VMA_MAX(v1, v2) (std::max((v1), (v2)))
        1747 #endif
        1748 
        1749 #ifndef VMA_SWAP
        1750  #define VMA_SWAP(v1, v2) std::swap((v1), (v2))
        1751 #endif
        1752 
        1753 #ifndef VMA_SORT
        1754  #define VMA_SORT(beg, end, cmp) std::sort(beg, end, cmp)
        1755 #endif
        1756 
        1757 #ifndef VMA_DEBUG_LOG
        1758  #define VMA_DEBUG_LOG(format, ...)
        1759  /*
        1760  #define VMA_DEBUG_LOG(format, ...) do { \
        1761  printf(format, __VA_ARGS__); \
        1762  printf("\n"); \
        1763  } while(false)
        1764  */
        1765 #endif
        1766 
        1767 // Define this macro to 1 to enable functions: vmaBuildStatsString, vmaFreeStatsString.
        1768 #if VMA_STATS_STRING_ENABLED
        1769  static inline void VmaUint32ToStr(char* outStr, size_t strLen, uint32_t num)
        1770  {
        1771  snprintf(outStr, strLen, "%u", static_cast<unsigned int>(num));
        1772  }
        1773  static inline void VmaUint64ToStr(char* outStr, size_t strLen, uint64_t num)
        1774  {
        1775  snprintf(outStr, strLen, "%llu", static_cast<unsigned long long>(num));
        1776  }
        1777  static inline void VmaPtrToStr(char* outStr, size_t strLen, const void* ptr)
        1778  {
        1779  snprintf(outStr, strLen, "%p", ptr);
        1780  }
        1781 #endif
        1782 
        1783 #ifndef VMA_MUTEX
        1784  class VmaMutex
        1785  {
        1786  public:
        1787  VmaMutex() { }
        1788  ~VmaMutex() { }
        1789  void Lock() { m_Mutex.lock(); }
        1790  void Unlock() { m_Mutex.unlock(); }
        1791  private:
        1792  std::mutex m_Mutex;
        1793  };
        1794  #define VMA_MUTEX VmaMutex
        1795 #endif
        1796 
        1797 /*
        1798 If providing your own implementation, you need to implement a subset of std::atomic:
        1799 
        1800 - Constructor(uint32_t desired)
        1801 - uint32_t load() const
        1802 - void store(uint32_t desired)
        1803 - bool compare_exchange_weak(uint32_t& expected, uint32_t desired)
        1804 */
        1805 #ifndef VMA_ATOMIC_UINT32
        1806  #define VMA_ATOMIC_UINT32 std::atomic<uint32_t>
        1807 #endif
        1808 
        1809 #ifndef VMA_BEST_FIT
        1810 
        1822  #define VMA_BEST_FIT (1)
        1823 #endif
        1824 
        1825 #ifndef VMA_DEBUG_ALWAYS_DEDICATED_MEMORY
        1826 
        1830  #define VMA_DEBUG_ALWAYS_DEDICATED_MEMORY (0)
        1831 #endif
        1832 
        1833 #ifndef VMA_DEBUG_ALIGNMENT
        1834 
        1838  #define VMA_DEBUG_ALIGNMENT (1)
        1839 #endif
        1840 
        1841 #ifndef VMA_DEBUG_MARGIN
        1842 
        1846  #define VMA_DEBUG_MARGIN (0)
        1847 #endif
        1848 
        1849 #ifndef VMA_DEBUG_GLOBAL_MUTEX
        1850 
        1854  #define VMA_DEBUG_GLOBAL_MUTEX (0)
        1855 #endif
        1856 
        1857 #ifndef VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY
        1858 
        1862  #define VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY (1)
        1863 #endif
        1864 
        1865 #ifndef VMA_SMALL_HEAP_MAX_SIZE
        1866  #define VMA_SMALL_HEAP_MAX_SIZE (512 * 1024 * 1024)
        1868 #endif
        1869 
        1870 #ifndef VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE
        1871  #define VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE (256 * 1024 * 1024)
        1873 #endif
        1874 
        1875 #ifndef VMA_DEFAULT_SMALL_HEAP_BLOCK_SIZE
        1876  #define VMA_DEFAULT_SMALL_HEAP_BLOCK_SIZE (64 * 1024 * 1024)
        1878 #endif
        1879 
        1880 static const uint32_t VMA_FRAME_INDEX_LOST = UINT32_MAX;
        1881 
        1882 /*******************************************************************************
        1883 END OF CONFIGURATION
        1884 */
        1885 
        1886 static VkAllocationCallbacks VmaEmptyAllocationCallbacks = {
        1887  VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL };
        1888 
        1889 // Returns number of bits set to 1 in (v).
        1890 static inline uint32_t VmaCountBitsSet(uint32_t v)
        1891 {
        1892  uint32_t c = v - ((v >> 1) & 0x55555555);
        1893  c = ((c >> 2) & 0x33333333) + (c & 0x33333333);
        1894  c = ((c >> 4) + c) & 0x0F0F0F0F;
        1895  c = ((c >> 8) + c) & 0x00FF00FF;
        1896  c = ((c >> 16) + c) & 0x0000FFFF;
        1897  return c;
        1898 }
        1899 
        1900 // Aligns given value up to nearest multiply of align value. For example: VmaAlignUp(11, 8) = 16.
        1901 // Use types like uint32_t, uint64_t as T.
        1902 template <typename T>
        1903 static inline T VmaAlignUp(T val, T align)
        1904 {
        1905  return (val + align - 1) / align * align;
        1906 }
        1907 
        1908 // Division with mathematical rounding to nearest number.
        1909 template <typename T>
        1910 inline T VmaRoundDiv(T x, T y)
        1911 {
        1912  return (x + (y / (T)2)) / y;
        1913 }
        1914 
        1915 #ifndef VMA_SORT
        1916 
        1917 template<typename Iterator, typename Compare>
        1918 Iterator VmaQuickSortPartition(Iterator beg, Iterator end, Compare cmp)
        1919 {
        1920  Iterator centerValue = end; --centerValue;
        1921  Iterator insertIndex = beg;
        1922  for(Iterator memTypeIndex = beg; memTypeIndex < centerValue; ++memTypeIndex)
        1923  {
        1924  if(cmp(*memTypeIndex, *centerValue))
        1925  {
        1926  if(insertIndex != memTypeIndex)
        1927  {
        1928  VMA_SWAP(*memTypeIndex, *insertIndex);
        1929  }
        1930  ++insertIndex;
        1931  }
        1932  }
        1933  if(insertIndex != centerValue)
        1934  {
        1935  VMA_SWAP(*insertIndex, *centerValue);
        1936  }
        1937  return insertIndex;
        1938 }
        1939 
        1940 template<typename Iterator, typename Compare>
        1941 void VmaQuickSort(Iterator beg, Iterator end, Compare cmp)
        1942 {
        1943  if(beg < end)
        1944  {
        1945  Iterator it = VmaQuickSortPartition<Iterator, Compare>(beg, end, cmp);
        1946  VmaQuickSort<Iterator, Compare>(beg, it, cmp);
        1947  VmaQuickSort<Iterator, Compare>(it + 1, end, cmp);
        1948  }
        1949 }
        1950 
        1951 #define VMA_SORT(beg, end, cmp) VmaQuickSort(beg, end, cmp)
        1952 
        1953 #endif // #ifndef VMA_SORT
        1954 
        1955 /*
        1956 Returns true if two memory blocks occupy overlapping pages.
        1957 ResourceA must be in less memory offset than ResourceB.
        1958 
        1959 Algorithm is based on "Vulkan 1.0.39 - A Specification (with all registered Vulkan extensions)"
        1960 chapter 11.6 "Resource Memory Association", paragraph "Buffer-Image Granularity".
        1961 */
        1962 static inline bool VmaBlocksOnSamePage(
        1963  VkDeviceSize resourceAOffset,
        1964  VkDeviceSize resourceASize,
        1965  VkDeviceSize resourceBOffset,
        1966  VkDeviceSize pageSize)
        1967 {
        1968  VMA_ASSERT(resourceAOffset + resourceASize <= resourceBOffset && resourceASize > 0 && pageSize > 0);
        1969  VkDeviceSize resourceAEnd = resourceAOffset + resourceASize - 1;
        1970  VkDeviceSize resourceAEndPage = resourceAEnd & ~(pageSize - 1);
        1971  VkDeviceSize resourceBStart = resourceBOffset;
        1972  VkDeviceSize resourceBStartPage = resourceBStart & ~(pageSize - 1);
        1973  return resourceAEndPage == resourceBStartPage;
        1974 }
        1975 
        1976 enum VmaSuballocationType
        1977 {
        1978  VMA_SUBALLOCATION_TYPE_FREE = 0,
        1979  VMA_SUBALLOCATION_TYPE_UNKNOWN = 1,
        1980  VMA_SUBALLOCATION_TYPE_BUFFER = 2,
        1981  VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN = 3,
        1982  VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR = 4,
        1983  VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL = 5,
        1984  VMA_SUBALLOCATION_TYPE_MAX_ENUM = 0x7FFFFFFF
        1985 };
        1986 
        1987 /*
        1988 Returns true if given suballocation types could conflict and must respect
        1989 VkPhysicalDeviceLimits::bufferImageGranularity. They conflict if one is buffer
        1990 or linear image and another one is optimal image. If type is unknown, behave
        1991 conservatively.
        1992 */
        1993 static inline bool VmaIsBufferImageGranularityConflict(
        1994  VmaSuballocationType suballocType1,
        1995  VmaSuballocationType suballocType2)
        1996 {
        1997  if(suballocType1 > suballocType2)
        1998  {
        1999  VMA_SWAP(suballocType1, suballocType2);
        2000  }
        2001 
        2002  switch(suballocType1)
        2003  {
        2004  case VMA_SUBALLOCATION_TYPE_FREE:
        2005  return false;
        2006  case VMA_SUBALLOCATION_TYPE_UNKNOWN:
        2007  return true;
        2008  case VMA_SUBALLOCATION_TYPE_BUFFER:
        2009  return
        2010  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
        2011  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
        2012  case VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN:
        2013  return
        2014  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
        2015  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR ||
        2016  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
        2017  case VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR:
        2018  return
        2019  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
        2020  case VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL:
        2021  return false;
        2022  default:
        2023  VMA_ASSERT(0);
        2024  return true;
        2025  }
        2026 }
        2027 
        2028 // Helper RAII class to lock a mutex in constructor and unlock it in destructor (at the end of scope).
        2029 struct VmaMutexLock
        2030 {
        2031 public:
        2032  VmaMutexLock(VMA_MUTEX& mutex, bool useMutex) :
        2033  m_pMutex(useMutex ? &mutex : VMA_NULL)
        2034  {
        2035  if(m_pMutex)
        2036  {
        2037  m_pMutex->Lock();
        2038  }
        2039  }
        2040 
        2041  ~VmaMutexLock()
        2042  {
        2043  if(m_pMutex)
        2044  {
        2045  m_pMutex->Unlock();
        2046  }
        2047  }
        2048 
        2049 private:
        2050  VMA_MUTEX* m_pMutex;
        2051 };
        2052 
        2053 #if VMA_DEBUG_GLOBAL_MUTEX
        2054  static VMA_MUTEX gDebugGlobalMutex;
        2055  #define VMA_DEBUG_GLOBAL_MUTEX_LOCK VmaMutexLock debugGlobalMutexLock(gDebugGlobalMutex, true);
        2056 #else
        2057  #define VMA_DEBUG_GLOBAL_MUTEX_LOCK
        2058 #endif
        2059 
        2060 // Minimum size of a free suballocation to register it in the free suballocation collection.
        2061 static const VkDeviceSize VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER = 16;
        2062 
        2063 /*
        2064 Performs binary search and returns iterator to first element that is greater or
        2065 equal to (key), according to comparison (cmp).
        2066 
        2067 Cmp should return true if first argument is less than second argument.
        2068 
        2069 Returned value is the found element, if present in the collection or place where
        2070 new element with value (key) should be inserted.
        2071 */
        2072 template <typename IterT, typename KeyT, typename CmpT>
        2073 static IterT VmaBinaryFindFirstNotLess(IterT beg, IterT end, const KeyT &key, CmpT cmp)
        2074 {
        2075  size_t down = 0, up = (end - beg);
        2076  while(down < up)
        2077  {
        2078  const size_t mid = (down + up) / 2;
        2079  if(cmp(*(beg+mid), key))
        2080  {
        2081  down = mid + 1;
        2082  }
        2083  else
        2084  {
        2085  up = mid;
        2086  }
        2087  }
        2088  return beg + down;
        2089 }
        2090 
        2092 // Memory allocation
        2093 
        2094 static void* VmaMalloc(const VkAllocationCallbacks* pAllocationCallbacks, size_t size, size_t alignment)
        2095 {
        2096  if((pAllocationCallbacks != VMA_NULL) &&
        2097  (pAllocationCallbacks->pfnAllocation != VMA_NULL))
        2098  {
        2099  return (*pAllocationCallbacks->pfnAllocation)(
        2100  pAllocationCallbacks->pUserData,
        2101  size,
        2102  alignment,
        2103  VK_SYSTEM_ALLOCATION_SCOPE_OBJECT);
        2104  }
        2105  else
        2106  {
        2107  return VMA_SYSTEM_ALIGNED_MALLOC(size, alignment);
        2108  }
        2109 }
        2110 
        2111 static void VmaFree(const VkAllocationCallbacks* pAllocationCallbacks, void* ptr)
        2112 {
        2113  if((pAllocationCallbacks != VMA_NULL) &&
        2114  (pAllocationCallbacks->pfnFree != VMA_NULL))
        2115  {
        2116  (*pAllocationCallbacks->pfnFree)(pAllocationCallbacks->pUserData, ptr);
        2117  }
        2118  else
        2119  {
        2120  VMA_SYSTEM_FREE(ptr);
        2121  }
        2122 }
        2123 
        2124 template<typename T>
        2125 static T* VmaAllocate(const VkAllocationCallbacks* pAllocationCallbacks)
        2126 {
        2127  return (T*)VmaMalloc(pAllocationCallbacks, sizeof(T), VMA_ALIGN_OF(T));
        2128 }
        2129 
        2130 template<typename T>
        2131 static T* VmaAllocateArray(const VkAllocationCallbacks* pAllocationCallbacks, size_t count)
        2132 {
        2133  return (T*)VmaMalloc(pAllocationCallbacks, sizeof(T) * count, VMA_ALIGN_OF(T));
        2134 }
        2135 
        2136 #define vma_new(allocator, type) new(VmaAllocate<type>(allocator))(type)
        2137 
        2138 #define vma_new_array(allocator, type, count) new(VmaAllocateArray<type>((allocator), (count)))(type)
        2139 
        2140 template<typename T>
        2141 static void vma_delete(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr)
        2142 {
        2143  ptr->~T();
        2144  VmaFree(pAllocationCallbacks, ptr);
        2145 }
        2146 
        2147 template<typename T>
        2148 static void vma_delete_array(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr, size_t count)
        2149 {
        2150  if(ptr != VMA_NULL)
        2151  {
        2152  for(size_t i = count; i--; )
        2153  {
        2154  ptr[i].~T();
        2155  }
        2156  VmaFree(pAllocationCallbacks, ptr);
        2157  }
        2158 }
        2159 
        2160 // STL-compatible allocator.
        2161 template<typename T>
        2162 class VmaStlAllocator
        2163 {
        2164 public:
        2165  const VkAllocationCallbacks* const m_pCallbacks;
        2166  typedef T value_type;
        2167 
        2168  VmaStlAllocator(const VkAllocationCallbacks* pCallbacks) : m_pCallbacks(pCallbacks) { }
        2169  template<typename U> VmaStlAllocator(const VmaStlAllocator<U>& src) : m_pCallbacks(src.m_pCallbacks) { }
        2170 
        2171  T* allocate(size_t n) { return VmaAllocateArray<T>(m_pCallbacks, n); }
        2172  void deallocate(T* p, size_t n) { VmaFree(m_pCallbacks, p); }
        2173 
        2174  template<typename U>
        2175  bool operator==(const VmaStlAllocator<U>& rhs) const
        2176  {
        2177  return m_pCallbacks == rhs.m_pCallbacks;
        2178  }
        2179  template<typename U>
        2180  bool operator!=(const VmaStlAllocator<U>& rhs) const
        2181  {
        2182  return m_pCallbacks != rhs.m_pCallbacks;
        2183  }
        2184 
        2185  VmaStlAllocator& operator=(const VmaStlAllocator& x) = delete;
        2186 };
        2187 
        2188 #if VMA_USE_STL_VECTOR
        2189 
        2190 #define VmaVector std::vector
        2191 
        2192 template<typename T, typename allocatorT>
        2193 static void VmaVectorInsert(std::vector<T, allocatorT>& vec, size_t index, const T& item)
        2194 {
        2195  vec.insert(vec.begin() + index, item);
        2196 }
        2197 
        2198 template<typename T, typename allocatorT>
        2199 static void VmaVectorRemove(std::vector<T, allocatorT>& vec, size_t index)
        2200 {
        2201  vec.erase(vec.begin() + index);
        2202 }
        2203 
        2204 #else // #if VMA_USE_STL_VECTOR
        2205 
        2206 /* Class with interface compatible with subset of std::vector.
        2207 T must be POD because constructors and destructors are not called and memcpy is
        2208 used for these objects. */
        2209 template<typename T, typename AllocatorT>
        2210 class VmaVector
        2211 {
        2212 public:
        2213  typedef T value_type;
        2214 
        2215  VmaVector(const AllocatorT& allocator) :
        2216  m_Allocator(allocator),
        2217  m_pArray(VMA_NULL),
        2218  m_Count(0),
        2219  m_Capacity(0)
        2220  {
        2221  }
        2222 
        2223  VmaVector(size_t count, const AllocatorT& allocator) :
        2224  m_Allocator(allocator),
        2225  m_pArray(count ? (T*)VmaAllocateArray<T>(allocator.m_pCallbacks, count) : VMA_NULL),
        2226  m_Count(count),
        2227  m_Capacity(count)
        2228  {
        2229  }
        2230 
        2231  VmaVector(const VmaVector<T, AllocatorT>& src) :
        2232  m_Allocator(src.m_Allocator),
        2233  m_pArray(src.m_Count ? (T*)VmaAllocateArray<T>(src.m_Allocator.m_pCallbacks, src.m_Count) : VMA_NULL),
        2234  m_Count(src.m_Count),
        2235  m_Capacity(src.m_Count)
        2236  {
        2237  if(m_Count != 0)
        2238  {
        2239  memcpy(m_pArray, src.m_pArray, m_Count * sizeof(T));
        2240  }
        2241  }
        2242 
        2243  ~VmaVector()
        2244  {
        2245  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
        2246  }
        2247 
        2248  VmaVector& operator=(const VmaVector<T, AllocatorT>& rhs)
        2249  {
        2250  if(&rhs != this)
        2251  {
        2252  resize(rhs.m_Count);
        2253  if(m_Count != 0)
        2254  {
        2255  memcpy(m_pArray, rhs.m_pArray, m_Count * sizeof(T));
        2256  }
        2257  }
        2258  return *this;
        2259  }
        2260 
        2261  bool empty() const { return m_Count == 0; }
        2262  size_t size() const { return m_Count; }
        2263  T* data() { return m_pArray; }
        2264  const T* data() const { return m_pArray; }
        2265 
        2266  T& operator[](size_t index)
        2267  {
        2268  VMA_HEAVY_ASSERT(index < m_Count);
        2269  return m_pArray[index];
        2270  }
        2271  const T& operator[](size_t index) const
        2272  {
        2273  VMA_HEAVY_ASSERT(index < m_Count);
        2274  return m_pArray[index];
        2275  }
        2276 
        2277  T& front()
        2278  {
        2279  VMA_HEAVY_ASSERT(m_Count > 0);
        2280  return m_pArray[0];
        2281  }
        2282  const T& front() const
        2283  {
        2284  VMA_HEAVY_ASSERT(m_Count > 0);
        2285  return m_pArray[0];
        2286  }
        2287  T& back()
        2288  {
        2289  VMA_HEAVY_ASSERT(m_Count > 0);
        2290  return m_pArray[m_Count - 1];
        2291  }
        2292  const T& back() const
        2293  {
        2294  VMA_HEAVY_ASSERT(m_Count > 0);
        2295  return m_pArray[m_Count - 1];
        2296  }
        2297 
        2298  void reserve(size_t newCapacity, bool freeMemory = false)
        2299  {
        2300  newCapacity = VMA_MAX(newCapacity, m_Count);
        2301 
        2302  if((newCapacity < m_Capacity) && !freeMemory)
        2303  {
        2304  newCapacity = m_Capacity;
        2305  }
        2306 
        2307  if(newCapacity != m_Capacity)
        2308  {
        2309  T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator, newCapacity) : VMA_NULL;
        2310  if(m_Count != 0)
        2311  {
        2312  memcpy(newArray, m_pArray, m_Count * sizeof(T));
        2313  }
        2314  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
        2315  m_Capacity = newCapacity;
        2316  m_pArray = newArray;
        2317  }
        2318  }
        2319 
        2320  void resize(size_t newCount, bool freeMemory = false)
        2321  {
        2322  size_t newCapacity = m_Capacity;
        2323  if(newCount > m_Capacity)
        2324  {
        2325  newCapacity = VMA_MAX(newCount, VMA_MAX(m_Capacity * 3 / 2, (size_t)8));
        2326  }
        2327  else if(freeMemory)
        2328  {
        2329  newCapacity = newCount;
        2330  }
        2331 
        2332  if(newCapacity != m_Capacity)
        2333  {
        2334  T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator.m_pCallbacks, newCapacity) : VMA_NULL;
        2335  const size_t elementsToCopy = VMA_MIN(m_Count, newCount);
        2336  if(elementsToCopy != 0)
        2337  {
        2338  memcpy(newArray, m_pArray, elementsToCopy * sizeof(T));
        2339  }
        2340  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
        2341  m_Capacity = newCapacity;
        2342  m_pArray = newArray;
        2343  }
        2344 
        2345  m_Count = newCount;
        2346  }
        2347 
        2348  void clear(bool freeMemory = false)
        2349  {
        2350  resize(0, freeMemory);
        2351  }
        2352 
        2353  void insert(size_t index, const T& src)
        2354  {
        2355  VMA_HEAVY_ASSERT(index <= m_Count);
        2356  const size_t oldCount = size();
        2357  resize(oldCount + 1);
        2358  if(index < oldCount)
        2359  {
        2360  memmove(m_pArray + (index + 1), m_pArray + index, (oldCount - index) * sizeof(T));
        2361  }
        2362  m_pArray[index] = src;
        2363  }
        2364 
        2365  void remove(size_t index)
        2366  {
        2367  VMA_HEAVY_ASSERT(index < m_Count);
        2368  const size_t oldCount = size();
        2369  if(index < oldCount - 1)
        2370  {
        2371  memmove(m_pArray + index, m_pArray + (index + 1), (oldCount - index - 1) * sizeof(T));
        2372  }
        2373  resize(oldCount - 1);
        2374  }
        2375 
        2376  void push_back(const T& src)
        2377  {
        2378  const size_t newIndex = size();
        2379  resize(newIndex + 1);
        2380  m_pArray[newIndex] = src;
        2381  }
        2382 
        2383  void pop_back()
        2384  {
        2385  VMA_HEAVY_ASSERT(m_Count > 0);
        2386  resize(size() - 1);
        2387  }
        2388 
        2389  void push_front(const T& src)
        2390  {
        2391  insert(0, src);
        2392  }
        2393 
        2394  void pop_front()
        2395  {
        2396  VMA_HEAVY_ASSERT(m_Count > 0);
        2397  remove(0);
        2398  }
        2399 
        2400  typedef T* iterator;
        2401 
        2402  iterator begin() { return m_pArray; }
        2403  iterator end() { return m_pArray + m_Count; }
        2404 
        2405 private:
        2406  AllocatorT m_Allocator;
        2407  T* m_pArray;
        2408  size_t m_Count;
        2409  size_t m_Capacity;
        2410 };
        2411 
        2412 template<typename T, typename allocatorT>
        2413 static void VmaVectorInsert(VmaVector<T, allocatorT>& vec, size_t index, const T& item)
        2414 {
        2415  vec.insert(index, item);
        2416 }
        2417 
        2418 template<typename T, typename allocatorT>
        2419 static void VmaVectorRemove(VmaVector<T, allocatorT>& vec, size_t index)
        2420 {
        2421  vec.remove(index);
        2422 }
        2423 
        2424 #endif // #if VMA_USE_STL_VECTOR
        2425 
        2426 template<typename CmpLess, typename VectorT>
        2427 size_t VmaVectorInsertSorted(VectorT& vector, const typename VectorT::value_type& value)
        2428 {
        2429  const size_t indexToInsert = VmaBinaryFindFirstNotLess(
        2430  vector.data(),
        2431  vector.data() + vector.size(),
        2432  value,
        2433  CmpLess()) - vector.data();
        2434  VmaVectorInsert(vector, indexToInsert, value);
        2435  return indexToInsert;
        2436 }
        2437 
        2438 template<typename CmpLess, typename VectorT>
        2439 bool VmaVectorRemoveSorted(VectorT& vector, const typename VectorT::value_type& value)
        2440 {
        2441  CmpLess comparator;
        2442  typename VectorT::iterator it = VmaBinaryFindFirstNotLess(
        2443  vector.begin(),
        2444  vector.end(),
        2445  value,
        2446  comparator);
        2447  if((it != vector.end()) && !comparator(*it, value) && !comparator(value, *it))
        2448  {
        2449  size_t indexToRemove = it - vector.begin();
        2450  VmaVectorRemove(vector, indexToRemove);
        2451  return true;
        2452  }
        2453  return false;
        2454 }
        2455 
        2456 template<typename CmpLess, typename VectorT>
        2457 size_t VmaVectorFindSorted(const VectorT& vector, const typename VectorT::value_type& value)
        2458 {
        2459  CmpLess comparator;
        2460  typename VectorT::iterator it = VmaBinaryFindFirstNotLess(
        2461  vector.data(),
        2462  vector.data() + vector.size(),
        2463  value,
        2464  comparator);
        2465  if(it != vector.size() && !comparator(*it, value) && !comparator(value, *it))
        2466  {
        2467  return it - vector.begin();
        2468  }
        2469  else
        2470  {
        2471  return vector.size();
        2472  }
        2473 }
        2474 
        2476 // class VmaPoolAllocator
        2477 
        2478 /*
        2479 Allocator for objects of type T using a list of arrays (pools) to speed up
        2480 allocation. Number of elements that can be allocated is not bounded because
        2481 allocator can create multiple blocks.
        2482 */
        2483 template<typename T>
        2484 class VmaPoolAllocator
        2485 {
        2486 public:
        2487  VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, size_t itemsPerBlock);
        2488  ~VmaPoolAllocator();
        2489  void Clear();
        2490  T* Alloc();
        2491  void Free(T* ptr);
        2492 
        2493 private:
        2494  union Item
        2495  {
        2496  uint32_t NextFreeIndex;
        2497  T Value;
        2498  };
        2499 
        2500  struct ItemBlock
        2501  {
        2502  Item* pItems;
        2503  uint32_t FirstFreeIndex;
        2504  };
        2505 
        2506  const VkAllocationCallbacks* m_pAllocationCallbacks;
        2507  size_t m_ItemsPerBlock;
        2508  VmaVector< ItemBlock, VmaStlAllocator<ItemBlock> > m_ItemBlocks;
        2509 
        2510  ItemBlock& CreateNewBlock();
        2511 };
        2512 
        2513 template<typename T>
        2514 VmaPoolAllocator<T>::VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, size_t itemsPerBlock) :
        2515  m_pAllocationCallbacks(pAllocationCallbacks),
        2516  m_ItemsPerBlock(itemsPerBlock),
        2517  m_ItemBlocks(VmaStlAllocator<ItemBlock>(pAllocationCallbacks))
        2518 {
        2519  VMA_ASSERT(itemsPerBlock > 0);
        2520 }
        2521 
        2522 template<typename T>
        2523 VmaPoolAllocator<T>::~VmaPoolAllocator()
        2524 {
        2525  Clear();
        2526 }
        2527 
        2528 template<typename T>
        2529 void VmaPoolAllocator<T>::Clear()
        2530 {
        2531  for(size_t i = m_ItemBlocks.size(); i--; )
        2532  vma_delete_array(m_pAllocationCallbacks, m_ItemBlocks[i].pItems, m_ItemsPerBlock);
        2533  m_ItemBlocks.clear();
        2534 }
        2535 
        2536 template<typename T>
        2537 T* VmaPoolAllocator<T>::Alloc()
        2538 {
        2539  for(size_t i = m_ItemBlocks.size(); i--; )
        2540  {
        2541  ItemBlock& block = m_ItemBlocks[i];
        2542  // This block has some free items: Use first one.
        2543  if(block.FirstFreeIndex != UINT32_MAX)
        2544  {
        2545  Item* const pItem = &block.pItems[block.FirstFreeIndex];
        2546  block.FirstFreeIndex = pItem->NextFreeIndex;
        2547  return &pItem->Value;
        2548  }
        2549  }
        2550 
        2551  // No block has free item: Create new one and use it.
        2552  ItemBlock& newBlock = CreateNewBlock();
        2553  Item* const pItem = &newBlock.pItems[0];
        2554  newBlock.FirstFreeIndex = pItem->NextFreeIndex;
        2555  return &pItem->Value;
        2556 }
        2557 
        2558 template<typename T>
        2559 void VmaPoolAllocator<T>::Free(T* ptr)
        2560 {
        2561  // Search all memory blocks to find ptr.
        2562  for(size_t i = 0; i < m_ItemBlocks.size(); ++i)
        2563  {
        2564  ItemBlock& block = m_ItemBlocks[i];
        2565 
        2566  // Casting to union.
        2567  Item* pItemPtr;
        2568  memcpy(&pItemPtr, &ptr, sizeof(pItemPtr));
        2569 
        2570  // Check if pItemPtr is in address range of this block.
        2571  if((pItemPtr >= block.pItems) && (pItemPtr < block.pItems + m_ItemsPerBlock))
        2572  {
        2573  const uint32_t index = static_cast<uint32_t>(pItemPtr - block.pItems);
        2574  pItemPtr->NextFreeIndex = block.FirstFreeIndex;
        2575  block.FirstFreeIndex = index;
        2576  return;
        2577  }
        2578  }
        2579  VMA_ASSERT(0 && "Pointer doesn't belong to this memory pool.");
        2580 }
        2581 
        2582 template<typename T>
        2583 typename VmaPoolAllocator<T>::ItemBlock& VmaPoolAllocator<T>::CreateNewBlock()
        2584 {
        2585  ItemBlock newBlock = {
        2586  vma_new_array(m_pAllocationCallbacks, Item, m_ItemsPerBlock), 0 };
        2587 
        2588  m_ItemBlocks.push_back(newBlock);
        2589 
        2590  // Setup singly-linked list of all free items in this block.
        2591  for(uint32_t i = 0; i < m_ItemsPerBlock - 1; ++i)
        2592  newBlock.pItems[i].NextFreeIndex = i + 1;
        2593  newBlock.pItems[m_ItemsPerBlock - 1].NextFreeIndex = UINT32_MAX;
        2594  return m_ItemBlocks.back();
        2595 }
        2596 
        2598 // class VmaRawList, VmaList
        2599 
        2600 #if VMA_USE_STL_LIST
        2601 
        2602 #define VmaList std::list
        2603 
        2604 #else // #if VMA_USE_STL_LIST
        2605 
        2606 template<typename T>
        2607 struct VmaListItem
        2608 {
        2609  VmaListItem* pPrev;
        2610  VmaListItem* pNext;
        2611  T Value;
        2612 };
        2613 
        2614 // Doubly linked list.
        2615 template<typename T>
        2616 class VmaRawList
        2617 {
        2618 public:
        2619  typedef VmaListItem<T> ItemType;
        2620 
        2621  VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks);
        2622  ~VmaRawList();
        2623  void Clear();
        2624 
        2625  size_t GetCount() const { return m_Count; }
        2626  bool IsEmpty() const { return m_Count == 0; }
        2627 
        2628  ItemType* Front() { return m_pFront; }
        2629  const ItemType* Front() const { return m_pFront; }
        2630  ItemType* Back() { return m_pBack; }
        2631  const ItemType* Back() const { return m_pBack; }
        2632 
        2633  ItemType* PushBack();
        2634  ItemType* PushFront();
        2635  ItemType* PushBack(const T& value);
        2636  ItemType* PushFront(const T& value);
        2637  void PopBack();
        2638  void PopFront();
        2639 
        2640  // Item can be null - it means PushBack.
        2641  ItemType* InsertBefore(ItemType* pItem);
        2642  // Item can be null - it means PushFront.
        2643  ItemType* InsertAfter(ItemType* pItem);
        2644 
        2645  ItemType* InsertBefore(ItemType* pItem, const T& value);
        2646  ItemType* InsertAfter(ItemType* pItem, const T& value);
        2647 
        2648  void Remove(ItemType* pItem);
        2649 
        2650 private:
        2651  const VkAllocationCallbacks* const m_pAllocationCallbacks;
        2652  VmaPoolAllocator<ItemType> m_ItemAllocator;
        2653  ItemType* m_pFront;
        2654  ItemType* m_pBack;
        2655  size_t m_Count;
        2656 
        2657  // Declared not defined, to block copy constructor and assignment operator.
        2658  VmaRawList(const VmaRawList<T>& src);
        2659  VmaRawList<T>& operator=(const VmaRawList<T>& rhs);
        2660 };
        2661 
        2662 template<typename T>
        2663 VmaRawList<T>::VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks) :
        2664  m_pAllocationCallbacks(pAllocationCallbacks),
        2665  m_ItemAllocator(pAllocationCallbacks, 128),
        2666  m_pFront(VMA_NULL),
        2667  m_pBack(VMA_NULL),
        2668  m_Count(0)
        2669 {
        2670 }
        2671 
        2672 template<typename T>
        2673 VmaRawList<T>::~VmaRawList()
        2674 {
        2675  // Intentionally not calling Clear, because that would be unnecessary
        2676  // computations to return all items to m_ItemAllocator as free.
        2677 }
        2678 
        2679 template<typename T>
        2680 void VmaRawList<T>::Clear()
        2681 {
        2682  if(IsEmpty() == false)
        2683  {
        2684  ItemType* pItem = m_pBack;
        2685  while(pItem != VMA_NULL)
        2686  {
        2687  ItemType* const pPrevItem = pItem->pPrev;
        2688  m_ItemAllocator.Free(pItem);
        2689  pItem = pPrevItem;
        2690  }
        2691  m_pFront = VMA_NULL;
        2692  m_pBack = VMA_NULL;
        2693  m_Count = 0;
        2694  }
        2695 }
        2696 
        2697 template<typename T>
        2698 VmaListItem<T>* VmaRawList<T>::PushBack()
        2699 {
        2700  ItemType* const pNewItem = m_ItemAllocator.Alloc();
        2701  pNewItem->pNext = VMA_NULL;
        2702  if(IsEmpty())
        2703  {
        2704  pNewItem->pPrev = VMA_NULL;
        2705  m_pFront = pNewItem;
        2706  m_pBack = pNewItem;
        2707  m_Count = 1;
        2708  }
        2709  else
        2710  {
        2711  pNewItem->pPrev = m_pBack;
        2712  m_pBack->pNext = pNewItem;
        2713  m_pBack = pNewItem;
        2714  ++m_Count;
        2715  }
        2716  return pNewItem;
        2717 }
        2718 
        2719 template<typename T>
        2720 VmaListItem<T>* VmaRawList<T>::PushFront()
        2721 {
        2722  ItemType* const pNewItem = m_ItemAllocator.Alloc();
        2723  pNewItem->pPrev = VMA_NULL;
        2724  if(IsEmpty())
        2725  {
        2726  pNewItem->pNext = VMA_NULL;
        2727  m_pFront = pNewItem;
        2728  m_pBack = pNewItem;
        2729  m_Count = 1;
        2730  }
        2731  else
        2732  {
        2733  pNewItem->pNext = m_pFront;
        2734  m_pFront->pPrev = pNewItem;
        2735  m_pFront = pNewItem;
        2736  ++m_Count;
        2737  }
        2738  return pNewItem;
        2739 }
        2740 
        2741 template<typename T>
        2742 VmaListItem<T>* VmaRawList<T>::PushBack(const T& value)
        2743 {
        2744  ItemType* const pNewItem = PushBack();
        2745  pNewItem->Value = value;
        2746  return pNewItem;
        2747 }
        2748 
        2749 template<typename T>
        2750 VmaListItem<T>* VmaRawList<T>::PushFront(const T& value)
        2751 {
        2752  ItemType* const pNewItem = PushFront();
        2753  pNewItem->Value = value;
        2754  return pNewItem;
        2755 }
        2756 
        2757 template<typename T>
        2758 void VmaRawList<T>::PopBack()
        2759 {
        2760  VMA_HEAVY_ASSERT(m_Count > 0);
        2761  ItemType* const pBackItem = m_pBack;
        2762  ItemType* const pPrevItem = pBackItem->pPrev;
        2763  if(pPrevItem != VMA_NULL)
        2764  {
        2765  pPrevItem->pNext = VMA_NULL;
        2766  }
        2767  m_pBack = pPrevItem;
        2768  m_ItemAllocator.Free(pBackItem);
        2769  --m_Count;
        2770 }
        2771 
        2772 template<typename T>
        2773 void VmaRawList<T>::PopFront()
        2774 {
        2775  VMA_HEAVY_ASSERT(m_Count > 0);
        2776  ItemType* const pFrontItem = m_pFront;
        2777  ItemType* const pNextItem = pFrontItem->pNext;
        2778  if(pNextItem != VMA_NULL)
        2779  {
        2780  pNextItem->pPrev = VMA_NULL;
        2781  }
        2782  m_pFront = pNextItem;
        2783  m_ItemAllocator.Free(pFrontItem);
        2784  --m_Count;
        2785 }
        2786 
        2787 template<typename T>
        2788 void VmaRawList<T>::Remove(ItemType* pItem)
        2789 {
        2790  VMA_HEAVY_ASSERT(pItem != VMA_NULL);
        2791  VMA_HEAVY_ASSERT(m_Count > 0);
        2792 
        2793  if(pItem->pPrev != VMA_NULL)
        2794  {
        2795  pItem->pPrev->pNext = pItem->pNext;
        2796  }
        2797  else
        2798  {
        2799  VMA_HEAVY_ASSERT(m_pFront == pItem);
        2800  m_pFront = pItem->pNext;
        2801  }
        2802 
        2803  if(pItem->pNext != VMA_NULL)
        2804  {
        2805  pItem->pNext->pPrev = pItem->pPrev;
        2806  }
        2807  else
        2808  {
        2809  VMA_HEAVY_ASSERT(m_pBack == pItem);
        2810  m_pBack = pItem->pPrev;
        2811  }
        2812 
        2813  m_ItemAllocator.Free(pItem);
        2814  --m_Count;
        2815 }
        2816 
        2817 template<typename T>
        2818 VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem)
        2819 {
        2820  if(pItem != VMA_NULL)
        2821  {
        2822  ItemType* const prevItem = pItem->pPrev;
        2823  ItemType* const newItem = m_ItemAllocator.Alloc();
        2824  newItem->pPrev = prevItem;
        2825  newItem->pNext = pItem;
        2826  pItem->pPrev = newItem;
        2827  if(prevItem != VMA_NULL)
        2828  {
        2829  prevItem->pNext = newItem;
        2830  }
        2831  else
        2832  {
        2833  VMA_HEAVY_ASSERT(m_pFront == pItem);
        2834  m_pFront = newItem;
        2835  }
        2836  ++m_Count;
        2837  return newItem;
        2838  }
        2839  else
        2840  return PushBack();
        2841 }
        2842 
        2843 template<typename T>
        2844 VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem)
        2845 {
        2846  if(pItem != VMA_NULL)
        2847  {
        2848  ItemType* const nextItem = pItem->pNext;
        2849  ItemType* const newItem = m_ItemAllocator.Alloc();
        2850  newItem->pNext = nextItem;
        2851  newItem->pPrev = pItem;
        2852  pItem->pNext = newItem;
        2853  if(nextItem != VMA_NULL)
        2854  {
        2855  nextItem->pPrev = newItem;
        2856  }
        2857  else
        2858  {
        2859  VMA_HEAVY_ASSERT(m_pBack == pItem);
        2860  m_pBack = newItem;
        2861  }
        2862  ++m_Count;
        2863  return newItem;
        2864  }
        2865  else
        2866  return PushFront();
        2867 }
        2868 
        2869 template<typename T>
        2870 VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem, const T& value)
        2871 {
        2872  ItemType* const newItem = InsertBefore(pItem);
        2873  newItem->Value = value;
        2874  return newItem;
        2875 }
        2876 
        2877 template<typename T>
        2878 VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem, const T& value)
        2879 {
        2880  ItemType* const newItem = InsertAfter(pItem);
        2881  newItem->Value = value;
        2882  return newItem;
        2883 }
        2884 
        2885 template<typename T, typename AllocatorT>
        2886 class VmaList
        2887 {
        2888 public:
        2889  class iterator
        2890  {
        2891  public:
        2892  iterator() :
        2893  m_pList(VMA_NULL),
        2894  m_pItem(VMA_NULL)
        2895  {
        2896  }
        2897 
        2898  T& operator*() const
        2899  {
        2900  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2901  return m_pItem->Value;
        2902  }
        2903  T* operator->() const
        2904  {
        2905  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2906  return &m_pItem->Value;
        2907  }
        2908 
        2909  iterator& operator++()
        2910  {
        2911  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2912  m_pItem = m_pItem->pNext;
        2913  return *this;
        2914  }
        2915  iterator& operator--()
        2916  {
        2917  if(m_pItem != VMA_NULL)
        2918  {
        2919  m_pItem = m_pItem->pPrev;
        2920  }
        2921  else
        2922  {
        2923  VMA_HEAVY_ASSERT(!m_pList.IsEmpty());
        2924  m_pItem = m_pList->Back();
        2925  }
        2926  return *this;
        2927  }
        2928 
        2929  iterator operator++(int)
        2930  {
        2931  iterator result = *this;
        2932  ++*this;
        2933  return result;
        2934  }
        2935  iterator operator--(int)
        2936  {
        2937  iterator result = *this;
        2938  --*this;
        2939  return result;
        2940  }
        2941 
        2942  bool operator==(const iterator& rhs) const
        2943  {
        2944  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
        2945  return m_pItem == rhs.m_pItem;
        2946  }
        2947  bool operator!=(const iterator& rhs) const
        2948  {
        2949  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
        2950  return m_pItem != rhs.m_pItem;
        2951  }
        2952 
        2953  private:
        2954  VmaRawList<T>* m_pList;
        2955  VmaListItem<T>* m_pItem;
        2956 
        2957  iterator(VmaRawList<T>* pList, VmaListItem<T>* pItem) :
        2958  m_pList(pList),
        2959  m_pItem(pItem)
        2960  {
        2961  }
        2962 
        2963  friend class VmaList<T, AllocatorT>;
        2964  };
        2965 
        2966  class const_iterator
        2967  {
        2968  public:
        2969  const_iterator() :
        2970  m_pList(VMA_NULL),
        2971  m_pItem(VMA_NULL)
        2972  {
        2973  }
        2974 
        2975  const_iterator(const iterator& src) :
        2976  m_pList(src.m_pList),
        2977  m_pItem(src.m_pItem)
        2978  {
        2979  }
        2980 
        2981  const T& operator*() const
        2982  {
        2983  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2984  return m_pItem->Value;
        2985  }
        2986  const T* operator->() const
        2987  {
        2988  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2989  return &m_pItem->Value;
        2990  }
        2991 
        2992  const_iterator& operator++()
        2993  {
        2994  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
        2995  m_pItem = m_pItem->pNext;
        2996  return *this;
        2997  }
        2998  const_iterator& operator--()
        2999  {
        3000  if(m_pItem != VMA_NULL)
        3001  {
        3002  m_pItem = m_pItem->pPrev;
        3003  }
        3004  else
        3005  {
        3006  VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
        3007  m_pItem = m_pList->Back();
        3008  }
        3009  return *this;
        3010  }
        3011 
        3012  const_iterator operator++(int)
        3013  {
        3014  const_iterator result = *this;
        3015  ++*this;
        3016  return result;
        3017  }
        3018  const_iterator operator--(int)
        3019  {
        3020  const_iterator result = *this;
        3021  --*this;
        3022  return result;
        3023  }
        3024 
        3025  bool operator==(const const_iterator& rhs) const
        3026  {
        3027  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
        3028  return m_pItem == rhs.m_pItem;
        3029  }
        3030  bool operator!=(const const_iterator& rhs) const
        3031  {
        3032  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
        3033  return m_pItem != rhs.m_pItem;
        3034  }
        3035 
        3036  private:
        3037  const_iterator(const VmaRawList<T>* pList, const VmaListItem<T>* pItem) :
        3038  m_pList(pList),
        3039  m_pItem(pItem)
        3040  {
        3041  }
        3042 
        3043  const VmaRawList<T>* m_pList;
        3044  const VmaListItem<T>* m_pItem;
        3045 
        3046  friend class VmaList<T, AllocatorT>;
        3047  };
        3048 
        3049  VmaList(const AllocatorT& allocator) : m_RawList(allocator.m_pCallbacks) { }
        3050 
        3051  bool empty() const { return m_RawList.IsEmpty(); }
        3052  size_t size() const { return m_RawList.GetCount(); }
        3053 
        3054  iterator begin() { return iterator(&m_RawList, m_RawList.Front()); }
        3055  iterator end() { return iterator(&m_RawList, VMA_NULL); }
        3056 
        3057  const_iterator cbegin() const { return const_iterator(&m_RawList, m_RawList.Front()); }
        3058  const_iterator cend() const { return const_iterator(&m_RawList, VMA_NULL); }
        3059 
        3060  void clear() { m_RawList.Clear(); }
        3061  void push_back(const T& value) { m_RawList.PushBack(value); }
        3062  void erase(iterator it) { m_RawList.Remove(it.m_pItem); }
        3063  iterator insert(iterator it, const T& value) { return iterator(&m_RawList, m_RawList.InsertBefore(it.m_pItem, value)); }
        3064 
        3065 private:
        3066  VmaRawList<T> m_RawList;
        3067 };
        3068 
        3069 #endif // #if VMA_USE_STL_LIST
        3070 
        3072 // class VmaMap
        3073 
        3074 // Unused in this version.
        3075 #if 0
        3076 
        3077 #if VMA_USE_STL_UNORDERED_MAP
        3078 
        3079 #define VmaPair std::pair
        3080 
        3081 #define VMA_MAP_TYPE(KeyT, ValueT) \
        3082  std::unordered_map< KeyT, ValueT, std::hash<KeyT>, std::equal_to<KeyT>, VmaStlAllocator< std::pair<KeyT, ValueT> > >
        3083 
        3084 #else // #if VMA_USE_STL_UNORDERED_MAP
        3085 
        3086 template<typename T1, typename T2>
        3087 struct VmaPair
        3088 {
        3089  T1 first;
        3090  T2 second;
        3091 
        3092  VmaPair() : first(), second() { }
        3093  VmaPair(const T1& firstSrc, const T2& secondSrc) : first(firstSrc), second(secondSrc) { }
        3094 };
        3095 
        3096 /* Class compatible with subset of interface of std::unordered_map.
        3097 KeyT, ValueT must be POD because they will be stored in VmaVector.
        3098 */
        3099 template<typename KeyT, typename ValueT>
        3100 class VmaMap
        3101 {
        3102 public:
        3103  typedef VmaPair<KeyT, ValueT> PairType;
        3104  typedef PairType* iterator;
        3105 
        3106  VmaMap(const VmaStlAllocator<PairType>& allocator) : m_Vector(allocator) { }
        3107 
        3108  iterator begin() { return m_Vector.begin(); }
        3109  iterator end() { return m_Vector.end(); }
        3110 
        3111  void insert(const PairType& pair);
        3112  iterator find(const KeyT& key);
        3113  void erase(iterator it);
        3114 
        3115 private:
        3116  VmaVector< PairType, VmaStlAllocator<PairType> > m_Vector;
        3117 };
        3118 
        3119 #define VMA_MAP_TYPE(KeyT, ValueT) VmaMap<KeyT, ValueT>
        3120 
        3121 template<typename FirstT, typename SecondT>
        3122 struct VmaPairFirstLess
        3123 {
        3124  bool operator()(const VmaPair<FirstT, SecondT>& lhs, const VmaPair<FirstT, SecondT>& rhs) const
        3125  {
        3126  return lhs.first < rhs.first;
        3127  }
        3128  bool operator()(const VmaPair<FirstT, SecondT>& lhs, const FirstT& rhsFirst) const
        3129  {
        3130  return lhs.first < rhsFirst;
        3131  }
        3132 };
        3133 
        3134 template<typename KeyT, typename ValueT>
        3135 void VmaMap<KeyT, ValueT>::insert(const PairType& pair)
        3136 {
        3137  const size_t indexToInsert = VmaBinaryFindFirstNotLess(
        3138  m_Vector.data(),
        3139  m_Vector.data() + m_Vector.size(),
        3140  pair,
        3141  VmaPairFirstLess<KeyT, ValueT>()) - m_Vector.data();
        3142  VmaVectorInsert(m_Vector, indexToInsert, pair);
        3143 }
        3144 
        3145 template<typename KeyT, typename ValueT>
        3146 VmaPair<KeyT, ValueT>* VmaMap<KeyT, ValueT>::find(const KeyT& key)
        3147 {
        3148  PairType* it = VmaBinaryFindFirstNotLess(
        3149  m_Vector.data(),
        3150  m_Vector.data() + m_Vector.size(),
        3151  key,
        3152  VmaPairFirstLess<KeyT, ValueT>());
        3153  if((it != m_Vector.end()) && (it->first == key))
        3154  {
        3155  return it;
        3156  }
        3157  else
        3158  {
        3159  return m_Vector.end();
        3160  }
        3161 }
        3162 
        3163 template<typename KeyT, typename ValueT>
        3164 void VmaMap<KeyT, ValueT>::erase(iterator it)
        3165 {
        3166  VmaVectorRemove(m_Vector, it - m_Vector.begin());
        3167 }
        3168 
        3169 #endif // #if VMA_USE_STL_UNORDERED_MAP
        3170 
        3171 #endif // #if 0
        3172 
        3174 
        3175 class VmaDeviceMemoryBlock;
        3176 
        3177 struct VmaAllocation_T
        3178 {
        3179 private:
        3180  static const uint8_t MAP_COUNT_FLAG_PERSISTENT_MAP = 0x80;
        3181 
        3182  enum FLAGS
        3183  {
        3184  FLAG_USER_DATA_STRING = 0x01,
        3185  };
        3186 
        3187 public:
        3188  enum ALLOCATION_TYPE
        3189  {
        3190  ALLOCATION_TYPE_NONE,
        3191  ALLOCATION_TYPE_BLOCK,
        3192  ALLOCATION_TYPE_DEDICATED,
        3193  };
        3194 
        3195  VmaAllocation_T(uint32_t currentFrameIndex, bool userDataString) :
        3196  m_Alignment(1),
        3197  m_Size(0),
        3198  m_pUserData(VMA_NULL),
        3199  m_LastUseFrameIndex(currentFrameIndex),
        3200  m_Type((uint8_t)ALLOCATION_TYPE_NONE),
        3201  m_SuballocationType((uint8_t)VMA_SUBALLOCATION_TYPE_UNKNOWN),
        3202  m_MapCount(0),
        3203  m_Flags(userDataString ? (uint8_t)FLAG_USER_DATA_STRING : 0)
        3204  {
        3205  }
        3206 
        3207  ~VmaAllocation_T()
        3208  {
        3209  VMA_ASSERT((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) == 0 && "Allocation was not unmapped before destruction.");
        3210 
        3211  // Check if owned string was freed.
        3212  VMA_ASSERT(m_pUserData == VMA_NULL);
        3213  }
        3214 
        3215  void InitBlockAllocation(
        3216  VmaPool hPool,
        3217  VmaDeviceMemoryBlock* block,
        3218  VkDeviceSize offset,
        3219  VkDeviceSize alignment,
        3220  VkDeviceSize size,
        3221  VmaSuballocationType suballocationType,
        3222  bool mapped,
        3223  bool canBecomeLost)
        3224  {
        3225  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
        3226  VMA_ASSERT(block != VMA_NULL);
        3227  m_Type = (uint8_t)ALLOCATION_TYPE_BLOCK;
        3228  m_Alignment = alignment;
        3229  m_Size = size;
        3230  m_MapCount = mapped ? MAP_COUNT_FLAG_PERSISTENT_MAP : 0;
        3231  m_SuballocationType = (uint8_t)suballocationType;
        3232  m_BlockAllocation.m_hPool = hPool;
        3233  m_BlockAllocation.m_Block = block;
        3234  m_BlockAllocation.m_Offset = offset;
        3235  m_BlockAllocation.m_CanBecomeLost = canBecomeLost;
        3236  }
        3237 
        3238  void InitLost()
        3239  {
        3240  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
        3241  VMA_ASSERT(m_LastUseFrameIndex.load() == VMA_FRAME_INDEX_LOST);
        3242  m_Type = (uint8_t)ALLOCATION_TYPE_BLOCK;
        3243  m_BlockAllocation.m_hPool = VK_NULL_HANDLE;
        3244  m_BlockAllocation.m_Block = VMA_NULL;
        3245  m_BlockAllocation.m_Offset = 0;
        3246  m_BlockAllocation.m_CanBecomeLost = true;
        3247  }
        3248 
        3249  void ChangeBlockAllocation(
        3250  VmaDeviceMemoryBlock* block,
        3251  VkDeviceSize offset)
        3252  {
        3253  VMA_ASSERT(block != VMA_NULL);
        3254  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
        3255  m_BlockAllocation.m_Block = block;
        3256  m_BlockAllocation.m_Offset = offset;
        3257  }
        3258 
        3259  // pMappedData not null means allocation is created with MAPPED flag.
        3260  void InitDedicatedAllocation(
        3261  uint32_t memoryTypeIndex,
        3262  VkDeviceMemory hMemory,
        3263  VmaSuballocationType suballocationType,
        3264  void* pMappedData,
        3265  VkDeviceSize size)
        3266  {
        3267  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
        3268  VMA_ASSERT(hMemory != VK_NULL_HANDLE);
        3269  m_Type = (uint8_t)ALLOCATION_TYPE_DEDICATED;
        3270  m_Alignment = 0;
        3271  m_Size = size;
        3272  m_SuballocationType = (uint8_t)suballocationType;
        3273  m_MapCount = (pMappedData != VMA_NULL) ? MAP_COUNT_FLAG_PERSISTENT_MAP : 0;
        3274  m_DedicatedAllocation.m_MemoryTypeIndex = memoryTypeIndex;
        3275  m_DedicatedAllocation.m_hMemory = hMemory;
        3276  m_DedicatedAllocation.m_pMappedData = pMappedData;
        3277  }
        3278 
        3279  ALLOCATION_TYPE GetType() const { return (ALLOCATION_TYPE)m_Type; }
        3280  VkDeviceSize GetAlignment() const { return m_Alignment; }
        3281  VkDeviceSize GetSize() const { return m_Size; }
        3282  bool IsUserDataString() const { return (m_Flags & FLAG_USER_DATA_STRING) != 0; }
        3283  void* GetUserData() const { return m_pUserData; }
        3284  void SetUserData(VmaAllocator hAllocator, void* pUserData);
        3285  VmaSuballocationType GetSuballocationType() const { return (VmaSuballocationType)m_SuballocationType; }
        3286 
        3287  VmaDeviceMemoryBlock* GetBlock() const
        3288  {
        3289  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
        3290  return m_BlockAllocation.m_Block;
        3291  }
        3292  VkDeviceSize GetOffset() const;
        3293  VkDeviceMemory GetMemory() const;
        3294  uint32_t GetMemoryTypeIndex() const;
        3295  bool IsPersistentMap() const { return (m_MapCount & MAP_COUNT_FLAG_PERSISTENT_MAP) != 0; }
        3296  void* GetMappedData() const;
        3297  bool CanBecomeLost() const;
        3298  VmaPool GetPool() const;
        3299 
        3300  uint32_t GetLastUseFrameIndex() const
        3301  {
        3302  return m_LastUseFrameIndex.load();
        3303  }
        3304  bool CompareExchangeLastUseFrameIndex(uint32_t& expected, uint32_t desired)
        3305  {
        3306  return m_LastUseFrameIndex.compare_exchange_weak(expected, desired);
        3307  }
        3308  /*
        3309  - If hAllocation.LastUseFrameIndex + frameInUseCount < allocator.CurrentFrameIndex,
        3310  makes it lost by setting LastUseFrameIndex = VMA_FRAME_INDEX_LOST and returns true.
        3311  - Else, returns false.
        3312 
        3313  If hAllocation is already lost, assert - you should not call it then.
        3314  If hAllocation was not created with CAN_BECOME_LOST_BIT, assert.
        3315  */
        3316  bool MakeLost(uint32_t currentFrameIndex, uint32_t frameInUseCount);
        3317 
        3318  void DedicatedAllocCalcStatsInfo(VmaStatInfo& outInfo)
        3319  {
        3320  VMA_ASSERT(m_Type == ALLOCATION_TYPE_DEDICATED);
        3321  outInfo.blockCount = 1;
        3322  outInfo.allocationCount = 1;
        3323  outInfo.unusedRangeCount = 0;
        3324  outInfo.usedBytes = m_Size;
        3325  outInfo.unusedBytes = 0;
        3326  outInfo.allocationSizeMin = outInfo.allocationSizeMax = m_Size;
        3327  outInfo.unusedRangeSizeMin = UINT64_MAX;
        3328  outInfo.unusedRangeSizeMax = 0;
        3329  }
        3330 
        3331  void BlockAllocMap();
        3332  void BlockAllocUnmap();
        3333  VkResult DedicatedAllocMap(VmaAllocator hAllocator, void** ppData);
        3334  void DedicatedAllocUnmap(VmaAllocator hAllocator);
        3335 
        3336 private:
        3337  VkDeviceSize m_Alignment;
        3338  VkDeviceSize m_Size;
        3339  void* m_pUserData;
        3340  VMA_ATOMIC_UINT32 m_LastUseFrameIndex;
        3341  uint8_t m_Type; // ALLOCATION_TYPE
        3342  uint8_t m_SuballocationType; // VmaSuballocationType
        3343  // Bit 0x80 is set when allocation was created with VMA_ALLOCATION_CREATE_MAPPED_BIT.
        3344  // Bits with mask 0x7F, used only when ALLOCATION_TYPE_DEDICATED, are reference counter for vmaMapMemory()/vmaUnmapMemory().
        3345  uint8_t m_MapCount;
        3346  uint8_t m_Flags; // enum FLAGS
        3347 
        3348  // Allocation out of VmaDeviceMemoryBlock.
        3349  struct BlockAllocation
        3350  {
        3351  VmaPool m_hPool; // Null if belongs to general memory.
        3352  VmaDeviceMemoryBlock* m_Block;
        3353  VkDeviceSize m_Offset;
        3354  bool m_CanBecomeLost;
        3355  };
        3356 
        3357  // Allocation for an object that has its own private VkDeviceMemory.
        3358  struct DedicatedAllocation
        3359  {
        3360  uint32_t m_MemoryTypeIndex;
        3361  VkDeviceMemory m_hMemory;
        3362  void* m_pMappedData; // Not null means memory is mapped.
        3363  };
        3364 
        3365  union
        3366  {
        3367  // Allocation out of VmaDeviceMemoryBlock.
        3368  BlockAllocation m_BlockAllocation;
        3369  // Allocation for an object that has its own private VkDeviceMemory.
        3370  DedicatedAllocation m_DedicatedAllocation;
        3371  };
        3372 
        3373  void FreeUserDataString(VmaAllocator hAllocator);
        3374 };
        3375 
        3376 /*
        3377 Represents a region of VmaDeviceMemoryBlock that is either assigned and returned as
        3378 allocated memory block or free.
        3379 */
        3380 struct VmaSuballocation
        3381 {
        3382  VkDeviceSize offset;
        3383  VkDeviceSize size;
        3384  VmaAllocation hAllocation;
        3385  VmaSuballocationType type;
        3386 };
        3387 
        3388 typedef VmaList< VmaSuballocation, VmaStlAllocator<VmaSuballocation> > VmaSuballocationList;
        3389 
        3390 // Cost of one additional allocation lost, as equivalent in bytes.
        3391 static const VkDeviceSize VMA_LOST_ALLOCATION_COST = 1048576;
        3392 
        3393 /*
        3394 Parameters of planned allocation inside a VmaDeviceMemoryBlock.
        3395 
        3396 If canMakeOtherLost was false:
        3397 - item points to a FREE suballocation.
        3398 - itemsToMakeLostCount is 0.
        3399 
        3400 If canMakeOtherLost was true:
        3401 - item points to first of sequence of suballocations, which are either FREE,
        3402  or point to VmaAllocations that can become lost.
        3403 - itemsToMakeLostCount is the number of VmaAllocations that need to be made lost for
        3404  the requested allocation to succeed.
        3405 */
        3406 struct VmaAllocationRequest
        3407 {
        3408  VkDeviceSize offset;
        3409  VkDeviceSize sumFreeSize; // Sum size of free items that overlap with proposed allocation.
        3410  VkDeviceSize sumItemSize; // Sum size of items to make lost that overlap with proposed allocation.
        3411  VmaSuballocationList::iterator item;
        3412  size_t itemsToMakeLostCount;
        3413 
        3414  VkDeviceSize CalcCost() const
        3415  {
        3416  return sumItemSize + itemsToMakeLostCount * VMA_LOST_ALLOCATION_COST;
        3417  }
        3418 };
        3419 
        3420 /*
        3421 Data structure used for bookkeeping of allocations and unused ranges of memory
        3422 in a single VkDeviceMemory block.
        3423 */
        3424 class VmaBlockMetadata
        3425 {
        3426 public:
        3427  VmaBlockMetadata(VmaAllocator hAllocator);
        3428  ~VmaBlockMetadata();
        3429  void Init(VkDeviceSize size);
        3430 
        3431  // Validates all data structures inside this object. If not valid, returns false.
        3432  bool Validate() const;
        3433  VkDeviceSize GetSize() const { return m_Size; }
        3434  size_t GetAllocationCount() const { return m_Suballocations.size() - m_FreeCount; }
        3435  VkDeviceSize GetSumFreeSize() const { return m_SumFreeSize; }
        3436  VkDeviceSize GetUnusedRangeSizeMax() const;
        3437  // Returns true if this block is empty - contains only single free suballocation.
        3438  bool IsEmpty() const;
        3439 
        3440  void CalcAllocationStatInfo(VmaStatInfo& outInfo) const;
        3441  void AddPoolStats(VmaPoolStats& inoutStats) const;
        3442 
        3443 #if VMA_STATS_STRING_ENABLED
        3444  void PrintDetailedMap(class VmaJsonWriter& json) const;
        3445 #endif
        3446 
        3447  // Creates trivial request for case when block is empty.
        3448  void CreateFirstAllocationRequest(VmaAllocationRequest* pAllocationRequest);
        3449 
        3450  // Tries to find a place for suballocation with given parameters inside this block.
        3451  // If succeeded, fills pAllocationRequest and returns true.
        3452  // If failed, returns false.
        3453  bool CreateAllocationRequest(
        3454  uint32_t currentFrameIndex,
        3455  uint32_t frameInUseCount,
        3456  VkDeviceSize bufferImageGranularity,
        3457  VkDeviceSize allocSize,
        3458  VkDeviceSize allocAlignment,
        3459  VmaSuballocationType allocType,
        3460  bool canMakeOtherLost,
        3461  VmaAllocationRequest* pAllocationRequest);
        3462 
        3463  bool MakeRequestedAllocationsLost(
        3464  uint32_t currentFrameIndex,
        3465  uint32_t frameInUseCount,
        3466  VmaAllocationRequest* pAllocationRequest);
        3467 
        3468  uint32_t MakeAllocationsLost(uint32_t currentFrameIndex, uint32_t frameInUseCount);
        3469 
        3470  // Makes actual allocation based on request. Request must already be checked and valid.
        3471  void Alloc(
        3472  const VmaAllocationRequest& request,
        3473  VmaSuballocationType type,
        3474  VkDeviceSize allocSize,
        3475  VmaAllocation hAllocation);
        3476 
        3477  // Frees suballocation assigned to given memory region.
        3478  void Free(const VmaAllocation allocation);
        3479 
        3480 private:
        3481  VkDeviceSize m_Size;
        3482  uint32_t m_FreeCount;
        3483  VkDeviceSize m_SumFreeSize;
        3484  VmaSuballocationList m_Suballocations;
        3485  // Suballocations that are free and have size greater than certain threshold.
        3486  // Sorted by size, ascending.
        3487  VmaVector< VmaSuballocationList::iterator, VmaStlAllocator< VmaSuballocationList::iterator > > m_FreeSuballocationsBySize;
        3488 
        3489  bool ValidateFreeSuballocationList() const;
        3490 
        3491  // Checks if requested suballocation with given parameters can be placed in given pFreeSuballocItem.
        3492  // If yes, fills pOffset and returns true. If no, returns false.
        3493  bool CheckAllocation(
        3494  uint32_t currentFrameIndex,
        3495  uint32_t frameInUseCount,
        3496  VkDeviceSize bufferImageGranularity,
        3497  VkDeviceSize allocSize,
        3498  VkDeviceSize allocAlignment,
        3499  VmaSuballocationType allocType,
        3500  VmaSuballocationList::const_iterator suballocItem,
        3501  bool canMakeOtherLost,
        3502  VkDeviceSize* pOffset,
        3503  size_t* itemsToMakeLostCount,
        3504  VkDeviceSize* pSumFreeSize,
        3505  VkDeviceSize* pSumItemSize) const;
        3506  // Given free suballocation, it merges it with following one, which must also be free.
        3507  void MergeFreeWithNext(VmaSuballocationList::iterator item);
        3508  // Releases given suballocation, making it free.
        3509  // Merges it with adjacent free suballocations if applicable.
        3510  // Returns iterator to new free suballocation at this place.
        3511  VmaSuballocationList::iterator FreeSuballocation(VmaSuballocationList::iterator suballocItem);
        3512  // Given free suballocation, it inserts it into sorted list of
        3513  // m_FreeSuballocationsBySize if it's suitable.
        3514  void RegisterFreeSuballocation(VmaSuballocationList::iterator item);
        3515  // Given free suballocation, it removes it from sorted list of
        3516  // m_FreeSuballocationsBySize if it's suitable.
        3517  void UnregisterFreeSuballocation(VmaSuballocationList::iterator item);
        3518 };
        3519 
        3520 // Helper class that represents mapped memory. Synchronized internally.
        3521 class VmaDeviceMemoryMapping
        3522 {
        3523 public:
        3524  VmaDeviceMemoryMapping();
        3525  ~VmaDeviceMemoryMapping();
        3526 
        3527  void* GetMappedData() const { return m_pMappedData; }
        3528 
        3529  // ppData can be null.
        3530  VkResult Map(VmaAllocator hAllocator, VkDeviceMemory hMemory, void **ppData);
        3531  void Unmap(VmaAllocator hAllocator, VkDeviceMemory hMemory);
        3532 
        3533 private:
        3534  VMA_MUTEX m_Mutex;
        3535  uint32_t m_MapCount;
        3536  void* m_pMappedData;
        3537 };
        3538 
        3539 /*
        3540 Represents a single block of device memory (`VkDeviceMemory`) with all the
        3541 data about its regions (aka suballocations, `VmaAllocation`), assigned and free.
        3542 
        3543 Thread-safety: This class must be externally synchronized.
        3544 */
        3545 class VmaDeviceMemoryBlock
        3546 {
        3547 public:
        3548  uint32_t m_MemoryTypeIndex;
        3549  VkDeviceMemory m_hMemory;
        3550  VmaDeviceMemoryMapping m_Mapping;
        3551  VmaBlockMetadata m_Metadata;
        3552 
        3553  VmaDeviceMemoryBlock(VmaAllocator hAllocator);
        3554 
        3555  ~VmaDeviceMemoryBlock()
        3556  {
        3557  VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
        3558  }
        3559 
        3560  // Always call after construction.
        3561  void Init(
        3562  uint32_t newMemoryTypeIndex,
        3563  VkDeviceMemory newMemory,
        3564  VkDeviceSize newSize);
        3565  // Always call before destruction.
        3566  void Destroy(VmaAllocator allocator);
        3567 
        3568  // Validates all data structures inside this object. If not valid, returns false.
        3569  bool Validate() const;
        3570 
        3571  // ppData can be null.
        3572  VkResult Map(VmaAllocator hAllocator, void** ppData);
        3573  void Unmap(VmaAllocator hAllocator);
        3574 };
        3575 
        3576 struct VmaPointerLess
        3577 {
        3578  bool operator()(const void* lhs, const void* rhs) const
        3579  {
        3580  return lhs < rhs;
        3581  }
        3582 };
        3583 
        3584 class VmaDefragmentator;
        3585 
        3586 /*
        3587 Sequence of VmaDeviceMemoryBlock. Represents memory blocks allocated for a specific
        3588 Vulkan memory type.
        3589 
        3590 Synchronized internally with a mutex.
        3591 */
        3592 struct VmaBlockVector
        3593 {
        3594  VmaBlockVector(
        3595  VmaAllocator hAllocator,
        3596  uint32_t memoryTypeIndex,
        3597  VkDeviceSize preferredBlockSize,
        3598  size_t minBlockCount,
        3599  size_t maxBlockCount,
        3600  VkDeviceSize bufferImageGranularity,
        3601  uint32_t frameInUseCount,
        3602  bool isCustomPool);
        3603  ~VmaBlockVector();
        3604 
        3605  VkResult CreateMinBlocks();
        3606 
        3607  uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
        3608  VkDeviceSize GetPreferredBlockSize() const { return m_PreferredBlockSize; }
        3609  VkDeviceSize GetBufferImageGranularity() const { return m_BufferImageGranularity; }
        3610  uint32_t GetFrameInUseCount() const { return m_FrameInUseCount; }
        3611 
        3612  void GetPoolStats(VmaPoolStats* pStats);
        3613 
        3614  bool IsEmpty() const { return m_Blocks.empty(); }
        3615 
        3616  VkResult Allocate(
        3617  VmaPool hCurrentPool,
        3618  uint32_t currentFrameIndex,
        3619  const VkMemoryRequirements& vkMemReq,
        3620  const VmaAllocationCreateInfo& createInfo,
        3621  VmaSuballocationType suballocType,
        3622  VmaAllocation* pAllocation);
        3623 
        3624  void Free(
        3625  VmaAllocation hAllocation);
        3626 
        3627  // Adds statistics of this BlockVector to pStats.
        3628  void AddStats(VmaStats* pStats);
        3629 
        3630 #if VMA_STATS_STRING_ENABLED
        3631  void PrintDetailedMap(class VmaJsonWriter& json);
        3632 #endif
        3633 
        3634  void MakePoolAllocationsLost(
        3635  uint32_t currentFrameIndex,
        3636  size_t* pLostAllocationCount);
        3637 
        3638  VmaDefragmentator* EnsureDefragmentator(
        3639  VmaAllocator hAllocator,
        3640  uint32_t currentFrameIndex);
        3641 
        3642  VkResult Defragment(
        3643  VmaDefragmentationStats* pDefragmentationStats,
        3644  VkDeviceSize& maxBytesToMove,
        3645  uint32_t& maxAllocationsToMove);
        3646 
        3647  void DestroyDefragmentator();
        3648 
        3649 private:
        3650  friend class VmaDefragmentator;
        3651 
        3652  const VmaAllocator m_hAllocator;
        3653  const uint32_t m_MemoryTypeIndex;
        3654  const VkDeviceSize m_PreferredBlockSize;
        3655  const size_t m_MinBlockCount;
        3656  const size_t m_MaxBlockCount;
        3657  const VkDeviceSize m_BufferImageGranularity;
        3658  const uint32_t m_FrameInUseCount;
        3659  const bool m_IsCustomPool;
        3660  VMA_MUTEX m_Mutex;
        3661  // Incrementally sorted by sumFreeSize, ascending.
        3662  VmaVector< VmaDeviceMemoryBlock*, VmaStlAllocator<VmaDeviceMemoryBlock*> > m_Blocks;
        3663  /* There can be at most one allocation that is completely empty - a
        3664  hysteresis to avoid pessimistic case of alternating creation and destruction
        3665  of a VkDeviceMemory. */
        3666  bool m_HasEmptyBlock;
        3667  VmaDefragmentator* m_pDefragmentator;
        3668 
        3669  // Finds and removes given block from vector.
        3670  void Remove(VmaDeviceMemoryBlock* pBlock);
        3671 
        3672  // Performs single step in sorting m_Blocks. They may not be fully sorted
        3673  // after this call.
        3674  void IncrementallySortBlocks();
        3675 
        3676  VkResult CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex);
        3677 };
        3678 
        3679 struct VmaPool_T
        3680 {
        3681 public:
        3682  VmaBlockVector m_BlockVector;
        3683 
        3684  // Takes ownership.
        3685  VmaPool_T(
        3686  VmaAllocator hAllocator,
        3687  const VmaPoolCreateInfo& createInfo);
        3688  ~VmaPool_T();
        3689 
        3690  VmaBlockVector& GetBlockVector() { return m_BlockVector; }
        3691 
        3692 #if VMA_STATS_STRING_ENABLED
        3693  //void PrintDetailedMap(class VmaStringBuilder& sb);
        3694 #endif
        3695 };
        3696 
        3697 class VmaDefragmentator
        3698 {
        3699  const VmaAllocator m_hAllocator;
        3700  VmaBlockVector* const m_pBlockVector;
        3701  uint32_t m_CurrentFrameIndex;
        3702  VkDeviceSize m_BytesMoved;
        3703  uint32_t m_AllocationsMoved;
        3704 
        3705  struct AllocationInfo
        3706  {
        3707  VmaAllocation m_hAllocation;
        3708  VkBool32* m_pChanged;
        3709 
        3710  AllocationInfo() :
        3711  m_hAllocation(VK_NULL_HANDLE),
        3712  m_pChanged(VMA_NULL)
        3713  {
        3714  }
        3715  };
        3716 
        3717  struct AllocationInfoSizeGreater
        3718  {
        3719  bool operator()(const AllocationInfo& lhs, const AllocationInfo& rhs) const
        3720  {
        3721  return lhs.m_hAllocation->GetSize() > rhs.m_hAllocation->GetSize();
        3722  }
        3723  };
        3724 
        3725  // Used between AddAllocation and Defragment.
        3726  VmaVector< AllocationInfo, VmaStlAllocator<AllocationInfo> > m_Allocations;
        3727 
        3728  struct BlockInfo
        3729  {
        3730  VmaDeviceMemoryBlock* m_pBlock;
        3731  bool m_HasNonMovableAllocations;
        3732  VmaVector< AllocationInfo, VmaStlAllocator<AllocationInfo> > m_Allocations;
        3733 
        3734  BlockInfo(const VkAllocationCallbacks* pAllocationCallbacks) :
        3735  m_pBlock(VMA_NULL),
        3736  m_HasNonMovableAllocations(true),
        3737  m_Allocations(pAllocationCallbacks),
        3738  m_pMappedDataForDefragmentation(VMA_NULL)
        3739  {
        3740  }
        3741 
        3742  void CalcHasNonMovableAllocations()
        3743  {
        3744  const size_t blockAllocCount = m_pBlock->m_Metadata.GetAllocationCount();
        3745  const size_t defragmentAllocCount = m_Allocations.size();
        3746  m_HasNonMovableAllocations = blockAllocCount != defragmentAllocCount;
        3747  }
        3748 
        3749  void SortAllocationsBySizeDescecnding()
        3750  {
        3751  VMA_SORT(m_Allocations.begin(), m_Allocations.end(), AllocationInfoSizeGreater());
        3752  }
        3753 
        3754  VkResult EnsureMapping(VmaAllocator hAllocator, void** ppMappedData);
        3755  void Unmap(VmaAllocator hAllocator);
        3756 
        3757  private:
        3758  // Not null if mapped for defragmentation only, not originally mapped.
        3759  void* m_pMappedDataForDefragmentation;
        3760  };
        3761 
        3762  struct BlockPointerLess
        3763  {
        3764  bool operator()(const BlockInfo* pLhsBlockInfo, const VmaDeviceMemoryBlock* pRhsBlock) const
        3765  {
        3766  return pLhsBlockInfo->m_pBlock < pRhsBlock;
        3767  }
        3768  bool operator()(const BlockInfo* pLhsBlockInfo, const BlockInfo* pRhsBlockInfo) const
        3769  {
        3770  return pLhsBlockInfo->m_pBlock < pRhsBlockInfo->m_pBlock;
        3771  }
        3772  };
        3773 
        3774  // 1. Blocks with some non-movable allocations go first.
        3775  // 2. Blocks with smaller sumFreeSize go first.
        3776  struct BlockInfoCompareMoveDestination
        3777  {
        3778  bool operator()(const BlockInfo* pLhsBlockInfo, const BlockInfo* pRhsBlockInfo) const
        3779  {
        3780  if(pLhsBlockInfo->m_HasNonMovableAllocations && !pRhsBlockInfo->m_HasNonMovableAllocations)
        3781  {
        3782  return true;
        3783  }
        3784  if(!pLhsBlockInfo->m_HasNonMovableAllocations && pRhsBlockInfo->m_HasNonMovableAllocations)
        3785  {
        3786  return false;
        3787  }
        3788  if(pLhsBlockInfo->m_pBlock->m_Metadata.GetSumFreeSize() < pRhsBlockInfo->m_pBlock->m_Metadata.GetSumFreeSize())
        3789  {
        3790  return true;
        3791  }
        3792  return false;
        3793  }
        3794  };
        3795 
        3796  typedef VmaVector< BlockInfo*, VmaStlAllocator<BlockInfo*> > BlockInfoVector;
        3797  BlockInfoVector m_Blocks;
        3798 
        3799  VkResult DefragmentRound(
        3800  VkDeviceSize maxBytesToMove,
        3801  uint32_t maxAllocationsToMove);
        3802 
        3803  static bool MoveMakesSense(
        3804  size_t dstBlockIndex, VkDeviceSize dstOffset,
        3805  size_t srcBlockIndex, VkDeviceSize srcOffset);
        3806 
        3807 public:
        3808  VmaDefragmentator(
        3809  VmaAllocator hAllocator,
        3810  VmaBlockVector* pBlockVector,
        3811  uint32_t currentFrameIndex);
        3812 
        3813  ~VmaDefragmentator();
        3814 
        3815  VkDeviceSize GetBytesMoved() const { return m_BytesMoved; }
        3816  uint32_t GetAllocationsMoved() const { return m_AllocationsMoved; }
        3817 
        3818  void AddAllocation(VmaAllocation hAlloc, VkBool32* pChanged);
        3819 
        3820  VkResult Defragment(
        3821  VkDeviceSize maxBytesToMove,
        3822  uint32_t maxAllocationsToMove);
        3823 };
        3824 
        3825 // Main allocator object.
        3826 struct VmaAllocator_T
        3827 {
        3828  bool m_UseMutex;
        3829  bool m_UseKhrDedicatedAllocation;
        3830  VkDevice m_hDevice;
        3831  bool m_AllocationCallbacksSpecified;
        3832  VkAllocationCallbacks m_AllocationCallbacks;
        3833  VmaDeviceMemoryCallbacks m_DeviceMemoryCallbacks;
        3834 
        3835  // Number of bytes free out of limit, or VK_WHOLE_SIZE if not limit for that heap.
        3836  VkDeviceSize m_HeapSizeLimit[VK_MAX_MEMORY_HEAPS];
        3837  VMA_MUTEX m_HeapSizeLimitMutex;
        3838 
        3839  VkPhysicalDeviceProperties m_PhysicalDeviceProperties;
        3840  VkPhysicalDeviceMemoryProperties m_MemProps;
        3841 
        3842  // Default pools.
        3843  VmaBlockVector* m_pBlockVectors[VK_MAX_MEMORY_TYPES];
        3844 
        3845  // Each vector is sorted by memory (handle value).
        3846  typedef VmaVector< VmaAllocation, VmaStlAllocator<VmaAllocation> > AllocationVectorType;
        3847  AllocationVectorType* m_pDedicatedAllocations[VK_MAX_MEMORY_TYPES];
        3848  VMA_MUTEX m_DedicatedAllocationsMutex[VK_MAX_MEMORY_TYPES];
        3849 
        3850  VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo);
        3851  ~VmaAllocator_T();
        3852 
        3853  const VkAllocationCallbacks* GetAllocationCallbacks() const
        3854  {
        3855  return m_AllocationCallbacksSpecified ? &m_AllocationCallbacks : 0;
        3856  }
        3857  const VmaVulkanFunctions& GetVulkanFunctions() const
        3858  {
        3859  return m_VulkanFunctions;
        3860  }
        3861 
        3862  VkDeviceSize GetBufferImageGranularity() const
        3863  {
        3864  return VMA_MAX(
        3865  static_cast<VkDeviceSize>(VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY),
        3866  m_PhysicalDeviceProperties.limits.bufferImageGranularity);
        3867  }
        3868 
        3869  uint32_t GetMemoryHeapCount() const { return m_MemProps.memoryHeapCount; }
        3870  uint32_t GetMemoryTypeCount() const { return m_MemProps.memoryTypeCount; }
        3871 
        3872  uint32_t MemoryTypeIndexToHeapIndex(uint32_t memTypeIndex) const
        3873  {
        3874  VMA_ASSERT(memTypeIndex < m_MemProps.memoryTypeCount);
        3875  return m_MemProps.memoryTypes[memTypeIndex].heapIndex;
        3876  }
        3877 
        3878  void GetBufferMemoryRequirements(
        3879  VkBuffer hBuffer,
        3880  VkMemoryRequirements& memReq,
        3881  bool& requiresDedicatedAllocation,
        3882  bool& prefersDedicatedAllocation) const;
        3883  void GetImageMemoryRequirements(
        3884  VkImage hImage,
        3885  VkMemoryRequirements& memReq,
        3886  bool& requiresDedicatedAllocation,
        3887  bool& prefersDedicatedAllocation) const;
        3888 
        3889  // Main allocation function.
        3890  VkResult AllocateMemory(
        3891  const VkMemoryRequirements& vkMemReq,
        3892  bool requiresDedicatedAllocation,
        3893  bool prefersDedicatedAllocation,
        3894  VkBuffer dedicatedBuffer,
        3895  VkImage dedicatedImage,
        3896  const VmaAllocationCreateInfo& createInfo,
        3897  VmaSuballocationType suballocType,
        3898  VmaAllocation* pAllocation);
        3899 
        3900  // Main deallocation function.
        3901  void FreeMemory(const VmaAllocation allocation);
        3902 
        3903  void CalculateStats(VmaStats* pStats);
        3904 
        3905 #if VMA_STATS_STRING_ENABLED
        3906  void PrintDetailedMap(class VmaJsonWriter& json);
        3907 #endif
        3908 
        3909  VkResult Defragment(
        3910  VmaAllocation* pAllocations,
        3911  size_t allocationCount,
        3912  VkBool32* pAllocationsChanged,
        3913  const VmaDefragmentationInfo* pDefragmentationInfo,
        3914  VmaDefragmentationStats* pDefragmentationStats);
        3915 
        3916  void GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo);
        3917 
        3918  VkResult CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool);
        3919  void DestroyPool(VmaPool pool);
        3920  void GetPoolStats(VmaPool pool, VmaPoolStats* pPoolStats);
        3921 
        3922  void SetCurrentFrameIndex(uint32_t frameIndex);
        3923 
        3924  void MakePoolAllocationsLost(
        3925  VmaPool hPool,
        3926  size_t* pLostAllocationCount);
        3927 
        3928  void CreateLostAllocation(VmaAllocation* pAllocation);
        3929 
        3930  VkResult AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory);
        3931  void FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory);
        3932 
        3933  VkResult Map(VmaAllocation hAllocation, void** ppData);
        3934  void Unmap(VmaAllocation hAllocation);
        3935 
        3936 private:
        3937  VkDeviceSize m_PreferredLargeHeapBlockSize;
        3938  VkDeviceSize m_PreferredSmallHeapBlockSize;
        3939 
        3940  VkPhysicalDevice m_PhysicalDevice;
        3941  VMA_ATOMIC_UINT32 m_CurrentFrameIndex;
        3942 
        3943  VMA_MUTEX m_PoolsMutex;
        3944  // Protected by m_PoolsMutex. Sorted by pointer value.
        3945  VmaVector<VmaPool, VmaStlAllocator<VmaPool> > m_Pools;
        3946 
        3947  VmaVulkanFunctions m_VulkanFunctions;
        3948 
        3949  void ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions);
        3950 
        3951  VkDeviceSize CalcPreferredBlockSize(uint32_t memTypeIndex);
        3952 
        3953  VkResult AllocateMemoryOfType(
        3954  const VkMemoryRequirements& vkMemReq,
        3955  bool dedicatedAllocation,
        3956  VkBuffer dedicatedBuffer,
        3957  VkImage dedicatedImage,
        3958  const VmaAllocationCreateInfo& createInfo,
        3959  uint32_t memTypeIndex,
        3960  VmaSuballocationType suballocType,
        3961  VmaAllocation* pAllocation);
        3962 
        3963  // Allocates and registers new VkDeviceMemory specifically for single allocation.
        3964  VkResult AllocateDedicatedMemory(
        3965  VkDeviceSize size,
        3966  VmaSuballocationType suballocType,
        3967  uint32_t memTypeIndex,
        3968  bool map,
        3969  bool isUserDataString,
        3970  void* pUserData,
        3971  VkBuffer dedicatedBuffer,
        3972  VkImage dedicatedImage,
        3973  VmaAllocation* pAllocation);
        3974 
        3975  // Tries to free pMemory as Dedicated Memory. Returns true if found and freed.
        3976  void FreeDedicatedMemory(VmaAllocation allocation);
        3977 };
        3978 
        3980 // Memory allocation #2 after VmaAllocator_T definition
        3981 
        3982 static void* VmaMalloc(VmaAllocator hAllocator, size_t size, size_t alignment)
        3983 {
        3984  return VmaMalloc(&hAllocator->m_AllocationCallbacks, size, alignment);
        3985 }
        3986 
        3987 static void VmaFree(VmaAllocator hAllocator, void* ptr)
        3988 {
        3989  VmaFree(&hAllocator->m_AllocationCallbacks, ptr);
        3990 }
        3991 
        3992 template<typename T>
        3993 static T* VmaAllocate(VmaAllocator hAllocator)
        3994 {
        3995  return (T*)VmaMalloc(hAllocator, sizeof(T), VMA_ALIGN_OF(T));
        3996 }
        3997 
        3998 template<typename T>
        3999 static T* VmaAllocateArray(VmaAllocator hAllocator, size_t count)
        4000 {
        4001  return (T*)VmaMalloc(hAllocator, sizeof(T) * count, VMA_ALIGN_OF(T));
        4002 }
        4003 
        4004 template<typename T>
        4005 static void vma_delete(VmaAllocator hAllocator, T* ptr)
        4006 {
        4007  if(ptr != VMA_NULL)
        4008  {
        4009  ptr->~T();
        4010  VmaFree(hAllocator, ptr);
        4011  }
        4012 }
        4013 
        4014 template<typename T>
        4015 static void vma_delete_array(VmaAllocator hAllocator, T* ptr, size_t count)
        4016 {
        4017  if(ptr != VMA_NULL)
        4018  {
        4019  for(size_t i = count; i--; )
        4020  ptr[i].~T();
        4021  VmaFree(hAllocator, ptr);
        4022  }
        4023 }
        4024 
        4026 // VmaStringBuilder
        4027 
        4028 #if VMA_STATS_STRING_ENABLED
        4029 
        4030 class VmaStringBuilder
        4031 {
        4032 public:
        4033  VmaStringBuilder(VmaAllocator alloc) : m_Data(VmaStlAllocator<char>(alloc->GetAllocationCallbacks())) { }
        4034  size_t GetLength() const { return m_Data.size(); }
        4035  const char* GetData() const { return m_Data.data(); }
        4036 
        4037  void Add(char ch) { m_Data.push_back(ch); }
        4038  void Add(const char* pStr);
        4039  void AddNewLine() { Add('\n'); }
        4040  void AddNumber(uint32_t num);
        4041  void AddNumber(uint64_t num);
        4042  void AddPointer(const void* ptr);
        4043 
        4044 private:
        4045  VmaVector< char, VmaStlAllocator<char> > m_Data;
        4046 };
        4047 
        4048 void VmaStringBuilder::Add(const char* pStr)
        4049 {
        4050  const size_t strLen = strlen(pStr);
        4051  if(strLen > 0)
        4052  {
        4053  const size_t oldCount = m_Data.size();
        4054  m_Data.resize(oldCount + strLen);
        4055  memcpy(m_Data.data() + oldCount, pStr, strLen);
        4056  }
        4057 }
        4058 
        4059 void VmaStringBuilder::AddNumber(uint32_t num)
        4060 {
        4061  char buf[11];
        4062  VmaUint32ToStr(buf, sizeof(buf), num);
        4063  Add(buf);
        4064 }
        4065 
        4066 void VmaStringBuilder::AddNumber(uint64_t num)
        4067 {
        4068  char buf[21];
        4069  VmaUint64ToStr(buf, sizeof(buf), num);
        4070  Add(buf);
        4071 }
        4072 
        4073 void VmaStringBuilder::AddPointer(const void* ptr)
        4074 {
        4075  char buf[21];
        4076  VmaPtrToStr(buf, sizeof(buf), ptr);
        4077  Add(buf);
        4078 }
        4079 
        4080 #endif // #if VMA_STATS_STRING_ENABLED
        4081 
        4083 // VmaJsonWriter
        4084 
        4085 #if VMA_STATS_STRING_ENABLED
        4086 
        4087 class VmaJsonWriter
        4088 {
        4089 public:
        4090  VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb);
        4091  ~VmaJsonWriter();
        4092 
        4093  void BeginObject(bool singleLine = false);
        4094  void EndObject();
        4095 
        4096  void BeginArray(bool singleLine = false);
        4097  void EndArray();
        4098 
        4099  void WriteString(const char* pStr);
        4100  void BeginString(const char* pStr = VMA_NULL);
        4101  void ContinueString(const char* pStr);
        4102  void ContinueString(uint32_t n);
        4103  void ContinueString(uint64_t n);
        4104  void ContinueString_Pointer(const void* ptr);
        4105  void EndString(const char* pStr = VMA_NULL);
        4106 
        4107  void WriteNumber(uint32_t n);
        4108  void WriteNumber(uint64_t n);
        4109  void WriteBool(bool b);
        4110  void WriteNull();
        4111 
        4112 private:
        4113  static const char* const INDENT;
        4114 
        4115  enum COLLECTION_TYPE
        4116  {
        4117  COLLECTION_TYPE_OBJECT,
        4118  COLLECTION_TYPE_ARRAY,
        4119  };
        4120  struct StackItem
        4121  {
        4122  COLLECTION_TYPE type;
        4123  uint32_t valueCount;
        4124  bool singleLineMode;
        4125  };
        4126 
        4127  VmaStringBuilder& m_SB;
        4128  VmaVector< StackItem, VmaStlAllocator<StackItem> > m_Stack;
        4129  bool m_InsideString;
        4130 
        4131  void BeginValue(bool isString);
        4132  void WriteIndent(bool oneLess = false);
        4133 };
        4134 
        4135 const char* const VmaJsonWriter::INDENT = " ";
        4136 
        4137 VmaJsonWriter::VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb) :
        4138  m_SB(sb),
        4139  m_Stack(VmaStlAllocator<StackItem>(pAllocationCallbacks)),
        4140  m_InsideString(false)
        4141 {
        4142 }
        4143 
        4144 VmaJsonWriter::~VmaJsonWriter()
        4145 {
        4146  VMA_ASSERT(!m_InsideString);
        4147  VMA_ASSERT(m_Stack.empty());
        4148 }
        4149 
        4150 void VmaJsonWriter::BeginObject(bool singleLine)
        4151 {
        4152  VMA_ASSERT(!m_InsideString);
        4153 
        4154  BeginValue(false);
        4155  m_SB.Add('{');
        4156 
        4157  StackItem item;
        4158  item.type = COLLECTION_TYPE_OBJECT;
        4159  item.valueCount = 0;
        4160  item.singleLineMode = singleLine;
        4161  m_Stack.push_back(item);
        4162 }
        4163 
        4164 void VmaJsonWriter::EndObject()
        4165 {
        4166  VMA_ASSERT(!m_InsideString);
        4167 
        4168  WriteIndent(true);
        4169  m_SB.Add('}');
        4170 
        4171  VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_OBJECT);
        4172  m_Stack.pop_back();
        4173 }
        4174 
        4175 void VmaJsonWriter::BeginArray(bool singleLine)
        4176 {
        4177  VMA_ASSERT(!m_InsideString);
        4178 
        4179  BeginValue(false);
        4180  m_SB.Add('[');
        4181 
        4182  StackItem item;
        4183  item.type = COLLECTION_TYPE_ARRAY;
        4184  item.valueCount = 0;
        4185  item.singleLineMode = singleLine;
        4186  m_Stack.push_back(item);
        4187 }
        4188 
        4189 void VmaJsonWriter::EndArray()
        4190 {
        4191  VMA_ASSERT(!m_InsideString);
        4192 
        4193  WriteIndent(true);
        4194  m_SB.Add(']');
        4195 
        4196  VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_ARRAY);
        4197  m_Stack.pop_back();
        4198 }
        4199 
        4200 void VmaJsonWriter::WriteString(const char* pStr)
        4201 {
        4202  BeginString(pStr);
        4203  EndString();
        4204 }
        4205 
        4206 void VmaJsonWriter::BeginString(const char* pStr)
        4207 {
        4208  VMA_ASSERT(!m_InsideString);
        4209 
        4210  BeginValue(true);
        4211  m_SB.Add('"');
        4212  m_InsideString = true;
        4213  if(pStr != VMA_NULL && pStr[0] != '\0')
        4214  {
        4215  ContinueString(pStr);
        4216  }
        4217 }
        4218 
        4219 void VmaJsonWriter::ContinueString(const char* pStr)
        4220 {
        4221  VMA_ASSERT(m_InsideString);
        4222 
        4223  const size_t strLen = strlen(pStr);
        4224  for(size_t i = 0; i < strLen; ++i)
        4225  {
        4226  char ch = pStr[i];
        4227  if(ch == '\'')
        4228  {
        4229  m_SB.Add("\\\\");
        4230  }
        4231  else if(ch == '"')
        4232  {
        4233  m_SB.Add("\\\"");
        4234  }
        4235  else if(ch >= 32)
        4236  {
        4237  m_SB.Add(ch);
        4238  }
        4239  else switch(ch)
        4240  {
        4241  case '\b':
        4242  m_SB.Add("\\b");
        4243  break;
        4244  case '\f':
        4245  m_SB.Add("\\f");
        4246  break;
        4247  case '\n':
        4248  m_SB.Add("\\n");
        4249  break;
        4250  case '\r':
        4251  m_SB.Add("\\r");
        4252  break;
        4253  case '\t':
        4254  m_SB.Add("\\t");
        4255  break;
        4256  default:
        4257  VMA_ASSERT(0 && "Character not currently supported.");
        4258  break;
        4259  }
        4260  }
        4261 }
        4262 
        4263 void VmaJsonWriter::ContinueString(uint32_t n)
        4264 {
        4265  VMA_ASSERT(m_InsideString);
        4266  m_SB.AddNumber(n);
        4267 }
        4268 
        4269 void VmaJsonWriter::ContinueString(uint64_t n)
        4270 {
        4271  VMA_ASSERT(m_InsideString);
        4272  m_SB.AddNumber(n);
        4273 }
        4274 
        4275 void VmaJsonWriter::ContinueString_Pointer(const void* ptr)
        4276 {
        4277  VMA_ASSERT(m_InsideString);
        4278  m_SB.AddPointer(ptr);
        4279 }
        4280 
        4281 void VmaJsonWriter::EndString(const char* pStr)
        4282 {
        4283  VMA_ASSERT(m_InsideString);
        4284  if(pStr != VMA_NULL && pStr[0] != '\0')
        4285  {
        4286  ContinueString(pStr);
        4287  }
        4288  m_SB.Add('"');
        4289  m_InsideString = false;
        4290 }
        4291 
        4292 void VmaJsonWriter::WriteNumber(uint32_t n)
        4293 {
        4294  VMA_ASSERT(!m_InsideString);
        4295  BeginValue(false);
        4296  m_SB.AddNumber(n);
        4297 }
        4298 
        4299 void VmaJsonWriter::WriteNumber(uint64_t n)
        4300 {
        4301  VMA_ASSERT(!m_InsideString);
        4302  BeginValue(false);
        4303  m_SB.AddNumber(n);
        4304 }
        4305 
        4306 void VmaJsonWriter::WriteBool(bool b)
        4307 {
        4308  VMA_ASSERT(!m_InsideString);
        4309  BeginValue(false);
        4310  m_SB.Add(b ? "true" : "false");
        4311 }
        4312 
        4313 void VmaJsonWriter::WriteNull()
        4314 {
        4315  VMA_ASSERT(!m_InsideString);
        4316  BeginValue(false);
        4317  m_SB.Add("null");
        4318 }
        4319 
        4320 void VmaJsonWriter::BeginValue(bool isString)
        4321 {
        4322  if(!m_Stack.empty())
        4323  {
        4324  StackItem& currItem = m_Stack.back();
        4325  if(currItem.type == COLLECTION_TYPE_OBJECT &&
        4326  currItem.valueCount % 2 == 0)
        4327  {
        4328  VMA_ASSERT(isString);
        4329  }
        4330 
        4331  if(currItem.type == COLLECTION_TYPE_OBJECT &&
        4332  currItem.valueCount % 2 != 0)
        4333  {
        4334  m_SB.Add(": ");
        4335  }
        4336  else if(currItem.valueCount > 0)
        4337  {
        4338  m_SB.Add(", ");
        4339  WriteIndent();
        4340  }
        4341  else
        4342  {
        4343  WriteIndent();
        4344  }
        4345  ++currItem.valueCount;
        4346  }
        4347 }
        4348 
        4349 void VmaJsonWriter::WriteIndent(bool oneLess)
        4350 {
        4351  if(!m_Stack.empty() && !m_Stack.back().singleLineMode)
        4352  {
        4353  m_SB.AddNewLine();
        4354 
        4355  size_t count = m_Stack.size();
        4356  if(count > 0 && oneLess)
        4357  {
        4358  --count;
        4359  }
        4360  for(size_t i = 0; i < count; ++i)
        4361  {
        4362  m_SB.Add(INDENT);
        4363  }
        4364  }
        4365 }
        4366 
        4367 #endif // #if VMA_STATS_STRING_ENABLED
        4368 
        4370 
        4371 void VmaAllocation_T::SetUserData(VmaAllocator hAllocator, void* pUserData)
        4372 {
        4373  if(IsUserDataString())
        4374  {
        4375  VMA_ASSERT(pUserData == VMA_NULL || pUserData != m_pUserData);
        4376 
        4377  FreeUserDataString(hAllocator);
        4378 
        4379  if(pUserData != VMA_NULL)
        4380  {
        4381  const char* const newStrSrc = (char*)pUserData;
        4382  const size_t newStrLen = strlen(newStrSrc);
        4383  char* const newStrDst = vma_new_array(hAllocator, char, newStrLen + 1);
        4384  memcpy(newStrDst, newStrSrc, newStrLen + 1);
        4385  m_pUserData = newStrDst;
        4386  }
        4387  }
        4388  else
        4389  {
        4390  m_pUserData = pUserData;
        4391  }
        4392 }
        4393 
        4394 VkDeviceSize VmaAllocation_T::GetOffset() const
        4395 {
        4396  switch(m_Type)
        4397  {
        4398  case ALLOCATION_TYPE_BLOCK:
        4399  return m_BlockAllocation.m_Offset;
        4400  case ALLOCATION_TYPE_DEDICATED:
        4401  return 0;
        4402  default:
        4403  VMA_ASSERT(0);
        4404  return 0;
        4405  }
        4406 }
        4407 
        4408 VkDeviceMemory VmaAllocation_T::GetMemory() const
        4409 {
        4410  switch(m_Type)
        4411  {
        4412  case ALLOCATION_TYPE_BLOCK:
        4413  return m_BlockAllocation.m_Block->m_hMemory;
        4414  case ALLOCATION_TYPE_DEDICATED:
        4415  return m_DedicatedAllocation.m_hMemory;
        4416  default:
        4417  VMA_ASSERT(0);
        4418  return VK_NULL_HANDLE;
        4419  }
        4420 }
        4421 
        4422 uint32_t VmaAllocation_T::GetMemoryTypeIndex() const
        4423 {
        4424  switch(m_Type)
        4425  {
        4426  case ALLOCATION_TYPE_BLOCK:
        4427  return m_BlockAllocation.m_Block->m_MemoryTypeIndex;
        4428  case ALLOCATION_TYPE_DEDICATED:
        4429  return m_DedicatedAllocation.m_MemoryTypeIndex;
        4430  default:
        4431  VMA_ASSERT(0);
        4432  return UINT32_MAX;
        4433  }
        4434 }
        4435 
        4436 void* VmaAllocation_T::GetMappedData() const
        4437 {
        4438  switch(m_Type)
        4439  {
        4440  case ALLOCATION_TYPE_BLOCK:
        4441  if(m_MapCount != 0)
        4442  {
        4443  void* pBlockData = m_BlockAllocation.m_Block->m_Mapping.GetMappedData();
        4444  VMA_ASSERT(pBlockData != VMA_NULL);
        4445  return (char*)pBlockData + m_BlockAllocation.m_Offset;
        4446  }
        4447  else
        4448  {
        4449  return VMA_NULL;
        4450  }
        4451  break;
        4452  case ALLOCATION_TYPE_DEDICATED:
        4453  VMA_ASSERT((m_DedicatedAllocation.m_pMappedData != VMA_NULL) == (m_MapCount != 0));
        4454  return m_DedicatedAllocation.m_pMappedData;
        4455  default:
        4456  VMA_ASSERT(0);
        4457  return VMA_NULL;
        4458  }
        4459 }
        4460 
        4461 bool VmaAllocation_T::CanBecomeLost() const
        4462 {
        4463  switch(m_Type)
        4464  {
        4465  case ALLOCATION_TYPE_BLOCK:
        4466  return m_BlockAllocation.m_CanBecomeLost;
        4467  case ALLOCATION_TYPE_DEDICATED:
        4468  return false;
        4469  default:
        4470  VMA_ASSERT(0);
        4471  return false;
        4472  }
        4473 }
        4474 
        4475 VmaPool VmaAllocation_T::GetPool() const
        4476 {
        4477  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
        4478  return m_BlockAllocation.m_hPool;
        4479 }
        4480 
        4481 bool VmaAllocation_T::MakeLost(uint32_t currentFrameIndex, uint32_t frameInUseCount)
        4482 {
        4483  VMA_ASSERT(CanBecomeLost());
        4484 
        4485  /*
        4486  Warning: This is a carefully designed algorithm.
        4487  Do not modify unless you really know what you're doing :)
        4488  */
        4489  uint32_t localLastUseFrameIndex = GetLastUseFrameIndex();
        4490  for(;;)
        4491  {
        4492  if(localLastUseFrameIndex == VMA_FRAME_INDEX_LOST)
        4493  {
        4494  VMA_ASSERT(0);
        4495  return false;
        4496  }
        4497  else if(localLastUseFrameIndex + frameInUseCount >= currentFrameIndex)
        4498  {
        4499  return false;
        4500  }
        4501  else // Last use time earlier than current time.
        4502  {
        4503  if(CompareExchangeLastUseFrameIndex(localLastUseFrameIndex, VMA_FRAME_INDEX_LOST))
        4504  {
        4505  // Setting hAllocation.LastUseFrameIndex atomic to VMA_FRAME_INDEX_LOST is enough to mark it as LOST.
        4506  // Calling code just needs to unregister this allocation in owning VmaDeviceMemoryBlock.
        4507  return true;
        4508  }
        4509  }
        4510  }
        4511 }
        4512 
        4513 void VmaAllocation_T::FreeUserDataString(VmaAllocator hAllocator)
        4514 {
        4515  VMA_ASSERT(IsUserDataString());
        4516  if(m_pUserData != VMA_NULL)
        4517  {
        4518  char* const oldStr = (char*)m_pUserData;
        4519  const size_t oldStrLen = strlen(oldStr);
        4520  vma_delete_array(hAllocator, oldStr, oldStrLen + 1);
        4521  m_pUserData = VMA_NULL;
        4522  }
        4523 }
        4524 
        4525 void VmaAllocation_T::BlockAllocMap()
        4526 {
        4527  VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
        4528 
        4529  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) < 0x7F)
        4530  {
        4531  ++m_MapCount;
        4532  }
        4533  else
        4534  {
        4535  VMA_ASSERT(0 && "Allocation mapped too many times simultaneously.");
        4536  }
        4537 }
        4538 
        4539 void VmaAllocation_T::BlockAllocUnmap()
        4540 {
        4541  VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
        4542 
        4543  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) != 0)
        4544  {
        4545  --m_MapCount;
        4546  }
        4547  else
        4548  {
        4549  VMA_ASSERT(0 && "Unmapping allocation not previously mapped.");
        4550  }
        4551 }
        4552 
        4553 VkResult VmaAllocation_T::DedicatedAllocMap(VmaAllocator hAllocator, void** ppData)
        4554 {
        4555  VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
        4556 
        4557  if(m_MapCount != 0)
        4558  {
        4559  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) < 0x7F)
        4560  {
        4561  VMA_ASSERT(m_DedicatedAllocation.m_pMappedData != VMA_NULL);
        4562  *ppData = m_DedicatedAllocation.m_pMappedData;
        4563  ++m_MapCount;
        4564  return VK_SUCCESS;
        4565  }
        4566  else
        4567  {
        4568  VMA_ASSERT(0 && "Dedicated allocation mapped too many times simultaneously.");
        4569  return VK_ERROR_MEMORY_MAP_FAILED;
        4570  }
        4571  }
        4572  else
        4573  {
        4574  VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
        4575  hAllocator->m_hDevice,
        4576  m_DedicatedAllocation.m_hMemory,
        4577  0, // offset
        4578  VK_WHOLE_SIZE,
        4579  0, // flags
        4580  ppData);
        4581  if(result == VK_SUCCESS)
        4582  {
        4583  m_DedicatedAllocation.m_pMappedData = *ppData;
        4584  m_MapCount = 1;
        4585  }
        4586  return result;
        4587  }
        4588 }
        4589 
        4590 void VmaAllocation_T::DedicatedAllocUnmap(VmaAllocator hAllocator)
        4591 {
        4592  VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
        4593 
        4594  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) != 0)
        4595  {
        4596  --m_MapCount;
        4597  if(m_MapCount == 0)
        4598  {
        4599  m_DedicatedAllocation.m_pMappedData = VMA_NULL;
        4600  (*hAllocator->GetVulkanFunctions().vkUnmapMemory)(
        4601  hAllocator->m_hDevice,
        4602  m_DedicatedAllocation.m_hMemory);
        4603  }
        4604  }
        4605  else
        4606  {
        4607  VMA_ASSERT(0 && "Unmapping dedicated allocation not previously mapped.");
        4608  }
        4609 }
        4610 
        4611 #if VMA_STATS_STRING_ENABLED
        4612 
        4613 // Correspond to values of enum VmaSuballocationType.
        4614 static const char* VMA_SUBALLOCATION_TYPE_NAMES[] = {
        4615  "FREE",
        4616  "UNKNOWN",
        4617  "BUFFER",
        4618  "IMAGE_UNKNOWN",
        4619  "IMAGE_LINEAR",
        4620  "IMAGE_OPTIMAL",
        4621 };
        4622 
        4623 static void VmaPrintStatInfo(VmaJsonWriter& json, const VmaStatInfo& stat)
        4624 {
        4625  json.BeginObject();
        4626 
        4627  json.WriteString("Blocks");
        4628  json.WriteNumber(stat.blockCount);
        4629 
        4630  json.WriteString("Allocations");
        4631  json.WriteNumber(stat.allocationCount);
        4632 
        4633  json.WriteString("UnusedRanges");
        4634  json.WriteNumber(stat.unusedRangeCount);
        4635 
        4636  json.WriteString("UsedBytes");
        4637  json.WriteNumber(stat.usedBytes);
        4638 
        4639  json.WriteString("UnusedBytes");
        4640  json.WriteNumber(stat.unusedBytes);
        4641 
        4642  if(stat.allocationCount > 1)
        4643  {
        4644  json.WriteString("AllocationSize");
        4645  json.BeginObject(true);
        4646  json.WriteString("Min");
        4647  json.WriteNumber(stat.allocationSizeMin);
        4648  json.WriteString("Avg");
        4649  json.WriteNumber(stat.allocationSizeAvg);
        4650  json.WriteString("Max");
        4651  json.WriteNumber(stat.allocationSizeMax);
        4652  json.EndObject();
        4653  }
        4654 
        4655  if(stat.unusedRangeCount > 1)
        4656  {
        4657  json.WriteString("UnusedRangeSize");
        4658  json.BeginObject(true);
        4659  json.WriteString("Min");
        4660  json.WriteNumber(stat.unusedRangeSizeMin);
        4661  json.WriteString("Avg");
        4662  json.WriteNumber(stat.unusedRangeSizeAvg);
        4663  json.WriteString("Max");
        4664  json.WriteNumber(stat.unusedRangeSizeMax);
        4665  json.EndObject();
        4666  }
        4667 
        4668  json.EndObject();
        4669 }
        4670 
        4671 #endif // #if VMA_STATS_STRING_ENABLED
        4672 
        4673 struct VmaSuballocationItemSizeLess
        4674 {
        4675  bool operator()(
        4676  const VmaSuballocationList::iterator lhs,
        4677  const VmaSuballocationList::iterator rhs) const
        4678  {
        4679  return lhs->size < rhs->size;
        4680  }
        4681  bool operator()(
        4682  const VmaSuballocationList::iterator lhs,
        4683  VkDeviceSize rhsSize) const
        4684  {
        4685  return lhs->size < rhsSize;
        4686  }
        4687 };
        4688 
        4690 // class VmaBlockMetadata
        4691 
        4692 VmaBlockMetadata::VmaBlockMetadata(VmaAllocator hAllocator) :
        4693  m_Size(0),
        4694  m_FreeCount(0),
        4695  m_SumFreeSize(0),
        4696  m_Suballocations(VmaStlAllocator<VmaSuballocation>(hAllocator->GetAllocationCallbacks())),
        4697  m_FreeSuballocationsBySize(VmaStlAllocator<VmaSuballocationList::iterator>(hAllocator->GetAllocationCallbacks()))
        4698 {
        4699 }
        4700 
        4701 VmaBlockMetadata::~VmaBlockMetadata()
        4702 {
        4703 }
        4704 
        4705 void VmaBlockMetadata::Init(VkDeviceSize size)
        4706 {
        4707  m_Size = size;
        4708  m_FreeCount = 1;
        4709  m_SumFreeSize = size;
        4710 
        4711  VmaSuballocation suballoc = {};
        4712  suballoc.offset = 0;
        4713  suballoc.size = size;
        4714  suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
        4715  suballoc.hAllocation = VK_NULL_HANDLE;
        4716 
        4717  m_Suballocations.push_back(suballoc);
        4718  VmaSuballocationList::iterator suballocItem = m_Suballocations.end();
        4719  --suballocItem;
        4720  m_FreeSuballocationsBySize.push_back(suballocItem);
        4721 }
        4722 
        4723 bool VmaBlockMetadata::Validate() const
        4724 {
        4725  if(m_Suballocations.empty())
        4726  {
        4727  return false;
        4728  }
        4729 
        4730  // Expected offset of new suballocation as calculates from previous ones.
        4731  VkDeviceSize calculatedOffset = 0;
        4732  // Expected number of free suballocations as calculated from traversing their list.
        4733  uint32_t calculatedFreeCount = 0;
        4734  // Expected sum size of free suballocations as calculated from traversing their list.
        4735  VkDeviceSize calculatedSumFreeSize = 0;
        4736  // Expected number of free suballocations that should be registered in
        4737  // m_FreeSuballocationsBySize calculated from traversing their list.
        4738  size_t freeSuballocationsToRegister = 0;
        4739  // True if previous visisted suballocation was free.
        4740  bool prevFree = false;
        4741 
        4742  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
        4743  suballocItem != m_Suballocations.cend();
        4744  ++suballocItem)
        4745  {
        4746  const VmaSuballocation& subAlloc = *suballocItem;
        4747 
        4748  // Actual offset of this suballocation doesn't match expected one.
        4749  if(subAlloc.offset != calculatedOffset)
        4750  {
        4751  return false;
        4752  }
        4753 
        4754  const bool currFree = (subAlloc.type == VMA_SUBALLOCATION_TYPE_FREE);
        4755  // Two adjacent free suballocations are invalid. They should be merged.
        4756  if(prevFree && currFree)
        4757  {
        4758  return false;
        4759  }
        4760  prevFree = currFree;
        4761 
        4762  if(currFree != (subAlloc.hAllocation == VK_NULL_HANDLE))
        4763  {
        4764  return false;
        4765  }
        4766 
        4767  if(currFree)
        4768  {
        4769  calculatedSumFreeSize += subAlloc.size;
        4770  ++calculatedFreeCount;
        4771  if(subAlloc.size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
        4772  {
        4773  ++freeSuballocationsToRegister;
        4774  }
        4775  }
        4776 
        4777  calculatedOffset += subAlloc.size;
        4778  }
        4779 
        4780  // Number of free suballocations registered in m_FreeSuballocationsBySize doesn't
        4781  // match expected one.
        4782  if(m_FreeSuballocationsBySize.size() != freeSuballocationsToRegister)
        4783  {
        4784  return false;
        4785  }
        4786 
        4787  VkDeviceSize lastSize = 0;
        4788  for(size_t i = 0; i < m_FreeSuballocationsBySize.size(); ++i)
        4789  {
        4790  VmaSuballocationList::iterator suballocItem = m_FreeSuballocationsBySize[i];
        4791 
        4792  // Only free suballocations can be registered in m_FreeSuballocationsBySize.
        4793  if(suballocItem->type != VMA_SUBALLOCATION_TYPE_FREE)
        4794  {
        4795  return false;
        4796  }
        4797  // They must be sorted by size ascending.
        4798  if(suballocItem->size < lastSize)
        4799  {
        4800  return false;
        4801  }
        4802 
        4803  lastSize = suballocItem->size;
        4804  }
        4805 
        4806  // Check if totals match calculacted values.
        4807  return
        4808  ValidateFreeSuballocationList() &&
        4809  (calculatedOffset == m_Size) &&
        4810  (calculatedSumFreeSize == m_SumFreeSize) &&
        4811  (calculatedFreeCount == m_FreeCount);
        4812 }
        4813 
        4814 VkDeviceSize VmaBlockMetadata::GetUnusedRangeSizeMax() const
        4815 {
        4816  if(!m_FreeSuballocationsBySize.empty())
        4817  {
        4818  return m_FreeSuballocationsBySize.back()->size;
        4819  }
        4820  else
        4821  {
        4822  return 0;
        4823  }
        4824 }
        4825 
        4826 bool VmaBlockMetadata::IsEmpty() const
        4827 {
        4828  return (m_Suballocations.size() == 1) && (m_FreeCount == 1);
        4829 }
        4830 
        4831 void VmaBlockMetadata::CalcAllocationStatInfo(VmaStatInfo& outInfo) const
        4832 {
        4833  outInfo.blockCount = 1;
        4834 
        4835  const uint32_t rangeCount = (uint32_t)m_Suballocations.size();
        4836  outInfo.allocationCount = rangeCount - m_FreeCount;
        4837  outInfo.unusedRangeCount = m_FreeCount;
        4838 
        4839  outInfo.unusedBytes = m_SumFreeSize;
        4840  outInfo.usedBytes = m_Size - outInfo.unusedBytes;
        4841 
        4842  outInfo.allocationSizeMin = UINT64_MAX;
        4843  outInfo.allocationSizeMax = 0;
        4844  outInfo.unusedRangeSizeMin = UINT64_MAX;
        4845  outInfo.unusedRangeSizeMax = 0;
        4846 
        4847  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
        4848  suballocItem != m_Suballocations.cend();
        4849  ++suballocItem)
        4850  {
        4851  const VmaSuballocation& suballoc = *suballocItem;
        4852  if(suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
        4853  {
        4854  outInfo.allocationSizeMin = VMA_MIN(outInfo.allocationSizeMin, suballoc.size);
        4855  outInfo.allocationSizeMax = VMA_MAX(outInfo.allocationSizeMax, suballoc.size);
        4856  }
        4857  else
        4858  {
        4859  outInfo.unusedRangeSizeMin = VMA_MIN(outInfo.unusedRangeSizeMin, suballoc.size);
        4860  outInfo.unusedRangeSizeMax = VMA_MAX(outInfo.unusedRangeSizeMax, suballoc.size);
        4861  }
        4862  }
        4863 }
        4864 
        4865 void VmaBlockMetadata::AddPoolStats(VmaPoolStats& inoutStats) const
        4866 {
        4867  const uint32_t rangeCount = (uint32_t)m_Suballocations.size();
        4868 
        4869  inoutStats.size += m_Size;
        4870  inoutStats.unusedSize += m_SumFreeSize;
        4871  inoutStats.allocationCount += rangeCount - m_FreeCount;
        4872  inoutStats.unusedRangeCount += m_FreeCount;
        4873  inoutStats.unusedRangeSizeMax = VMA_MAX(inoutStats.unusedRangeSizeMax, GetUnusedRangeSizeMax());
        4874 }
        4875 
        4876 #if VMA_STATS_STRING_ENABLED
        4877 
        4878 void VmaBlockMetadata::PrintDetailedMap(class VmaJsonWriter& json) const
        4879 {
        4880  json.BeginObject();
        4881 
        4882  json.WriteString("TotalBytes");
        4883  json.WriteNumber(m_Size);
        4884 
        4885  json.WriteString("UnusedBytes");
        4886  json.WriteNumber(m_SumFreeSize);
        4887 
        4888  json.WriteString("Allocations");
        4889  json.WriteNumber(m_Suballocations.size() - m_FreeCount);
        4890 
        4891  json.WriteString("UnusedRanges");
        4892  json.WriteNumber(m_FreeCount);
        4893 
        4894  json.WriteString("Suballocations");
        4895  json.BeginArray();
        4896  size_t i = 0;
        4897  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
        4898  suballocItem != m_Suballocations.cend();
        4899  ++suballocItem, ++i)
        4900  {
        4901  json.BeginObject(true);
        4902 
        4903  json.WriteString("Type");
        4904  json.WriteString(VMA_SUBALLOCATION_TYPE_NAMES[suballocItem->type]);
        4905 
        4906  json.WriteString("Size");
        4907  json.WriteNumber(suballocItem->size);
        4908 
        4909  json.WriteString("Offset");
        4910  json.WriteNumber(suballocItem->offset);
        4911 
        4912  if(suballocItem->type != VMA_SUBALLOCATION_TYPE_FREE)
        4913  {
        4914  const void* pUserData = suballocItem->hAllocation->GetUserData();
        4915  if(pUserData != VMA_NULL)
        4916  {
        4917  json.WriteString("UserData");
        4918  if(suballocItem->hAllocation->IsUserDataString())
        4919  {
        4920  json.WriteString((const char*)pUserData);
        4921  }
        4922  else
        4923  {
        4924  json.BeginString();
        4925  json.ContinueString_Pointer(pUserData);
        4926  json.EndString();
        4927  }
        4928  }
        4929  }
        4930 
        4931  json.EndObject();
        4932  }
        4933  json.EndArray();
        4934 
        4935  json.EndObject();
        4936 }
        4937 
        4938 #endif // #if VMA_STATS_STRING_ENABLED
        4939 
        4940 /*
        4941 How many suitable free suballocations to analyze before choosing best one.
        4942 - Set to 1 to use First-Fit algorithm - first suitable free suballocation will
        4943  be chosen.
        4944 - Set to UINT32_MAX to use Best-Fit/Worst-Fit algorithm - all suitable free
        4945  suballocations will be analized and best one will be chosen.
        4946 - Any other value is also acceptable.
        4947 */
        4948 //static const uint32_t MAX_SUITABLE_SUBALLOCATIONS_TO_CHECK = 8;
        4949 
        4950 void VmaBlockMetadata::CreateFirstAllocationRequest(VmaAllocationRequest* pAllocationRequest)
        4951 {
        4952  VMA_ASSERT(IsEmpty());
        4953  pAllocationRequest->offset = 0;
        4954  pAllocationRequest->sumFreeSize = m_SumFreeSize;
        4955  pAllocationRequest->sumItemSize = 0;
        4956  pAllocationRequest->item = m_Suballocations.begin();
        4957  pAllocationRequest->itemsToMakeLostCount = 0;
        4958 }
        4959 
        4960 bool VmaBlockMetadata::CreateAllocationRequest(
        4961  uint32_t currentFrameIndex,
        4962  uint32_t frameInUseCount,
        4963  VkDeviceSize bufferImageGranularity,
        4964  VkDeviceSize allocSize,
        4965  VkDeviceSize allocAlignment,
        4966  VmaSuballocationType allocType,
        4967  bool canMakeOtherLost,
        4968  VmaAllocationRequest* pAllocationRequest)
        4969 {
        4970  VMA_ASSERT(allocSize > 0);
        4971  VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
        4972  VMA_ASSERT(pAllocationRequest != VMA_NULL);
        4973  VMA_HEAVY_ASSERT(Validate());
        4974 
        4975  // There is not enough total free space in this block to fullfill the request: Early return.
        4976  if(canMakeOtherLost == false && m_SumFreeSize < allocSize)
        4977  {
        4978  return false;
        4979  }
        4980 
        4981  // New algorithm, efficiently searching freeSuballocationsBySize.
        4982  const size_t freeSuballocCount = m_FreeSuballocationsBySize.size();
        4983  if(freeSuballocCount > 0)
        4984  {
        4985  if(VMA_BEST_FIT)
        4986  {
        4987  // Find first free suballocation with size not less than allocSize.
        4988  VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
        4989  m_FreeSuballocationsBySize.data(),
        4990  m_FreeSuballocationsBySize.data() + freeSuballocCount,
        4991  allocSize,
        4992  VmaSuballocationItemSizeLess());
        4993  size_t index = it - m_FreeSuballocationsBySize.data();
        4994  for(; index < freeSuballocCount; ++index)
        4995  {
        4996  if(CheckAllocation(
        4997  currentFrameIndex,
        4998  frameInUseCount,
        4999  bufferImageGranularity,
        5000  allocSize,
        5001  allocAlignment,
        5002  allocType,
        5003  m_FreeSuballocationsBySize[index],
        5004  false, // canMakeOtherLost
        5005  &pAllocationRequest->offset,
        5006  &pAllocationRequest->itemsToMakeLostCount,
        5007  &pAllocationRequest->sumFreeSize,
        5008  &pAllocationRequest->sumItemSize))
        5009  {
        5010  pAllocationRequest->item = m_FreeSuballocationsBySize[index];
        5011  return true;
        5012  }
        5013  }
        5014  }
        5015  else
        5016  {
        5017  // Search staring from biggest suballocations.
        5018  for(size_t index = freeSuballocCount; index--; )
        5019  {
        5020  if(CheckAllocation(
        5021  currentFrameIndex,
        5022  frameInUseCount,
        5023  bufferImageGranularity,
        5024  allocSize,
        5025  allocAlignment,
        5026  allocType,
        5027  m_FreeSuballocationsBySize[index],
        5028  false, // canMakeOtherLost
        5029  &pAllocationRequest->offset,
        5030  &pAllocationRequest->itemsToMakeLostCount,
        5031  &pAllocationRequest->sumFreeSize,
        5032  &pAllocationRequest->sumItemSize))
        5033  {
        5034  pAllocationRequest->item = m_FreeSuballocationsBySize[index];
        5035  return true;
        5036  }
        5037  }
        5038  }
        5039  }
        5040 
        5041  if(canMakeOtherLost)
        5042  {
        5043  // Brute-force algorithm. TODO: Come up with something better.
        5044 
        5045  pAllocationRequest->sumFreeSize = VK_WHOLE_SIZE;
        5046  pAllocationRequest->sumItemSize = VK_WHOLE_SIZE;
        5047 
        5048  VmaAllocationRequest tmpAllocRequest = {};
        5049  for(VmaSuballocationList::iterator suballocIt = m_Suballocations.begin();
        5050  suballocIt != m_Suballocations.end();
        5051  ++suballocIt)
        5052  {
        5053  if(suballocIt->type == VMA_SUBALLOCATION_TYPE_FREE ||
        5054  suballocIt->hAllocation->CanBecomeLost())
        5055  {
        5056  if(CheckAllocation(
        5057  currentFrameIndex,
        5058  frameInUseCount,
        5059  bufferImageGranularity,
        5060  allocSize,
        5061  allocAlignment,
        5062  allocType,
        5063  suballocIt,
        5064  canMakeOtherLost,
        5065  &tmpAllocRequest.offset,
        5066  &tmpAllocRequest.itemsToMakeLostCount,
        5067  &tmpAllocRequest.sumFreeSize,
        5068  &tmpAllocRequest.sumItemSize))
        5069  {
        5070  tmpAllocRequest.item = suballocIt;
        5071 
        5072  if(tmpAllocRequest.CalcCost() < pAllocationRequest->CalcCost())
        5073  {
        5074  *pAllocationRequest = tmpAllocRequest;
        5075  }
        5076  }
        5077  }
        5078  }
        5079 
        5080  if(pAllocationRequest->sumItemSize != VK_WHOLE_SIZE)
        5081  {
        5082  return true;
        5083  }
        5084  }
        5085 
        5086  return false;
        5087 }
        5088 
        5089 bool VmaBlockMetadata::MakeRequestedAllocationsLost(
        5090  uint32_t currentFrameIndex,
        5091  uint32_t frameInUseCount,
        5092  VmaAllocationRequest* pAllocationRequest)
        5093 {
        5094  while(pAllocationRequest->itemsToMakeLostCount > 0)
        5095  {
        5096  if(pAllocationRequest->item->type == VMA_SUBALLOCATION_TYPE_FREE)
        5097  {
        5098  ++pAllocationRequest->item;
        5099  }
        5100  VMA_ASSERT(pAllocationRequest->item != m_Suballocations.end());
        5101  VMA_ASSERT(pAllocationRequest->item->hAllocation != VK_NULL_HANDLE);
        5102  VMA_ASSERT(pAllocationRequest->item->hAllocation->CanBecomeLost());
        5103  if(pAllocationRequest->item->hAllocation->MakeLost(currentFrameIndex, frameInUseCount))
        5104  {
        5105  pAllocationRequest->item = FreeSuballocation(pAllocationRequest->item);
        5106  --pAllocationRequest->itemsToMakeLostCount;
        5107  }
        5108  else
        5109  {
        5110  return false;
        5111  }
        5112  }
        5113 
        5114  VMA_HEAVY_ASSERT(Validate());
        5115  VMA_ASSERT(pAllocationRequest->item != m_Suballocations.end());
        5116  VMA_ASSERT(pAllocationRequest->item->type == VMA_SUBALLOCATION_TYPE_FREE);
        5117 
        5118  return true;
        5119 }
        5120 
        5121 uint32_t VmaBlockMetadata::MakeAllocationsLost(uint32_t currentFrameIndex, uint32_t frameInUseCount)
        5122 {
        5123  uint32_t lostAllocationCount = 0;
        5124  for(VmaSuballocationList::iterator it = m_Suballocations.begin();
        5125  it != m_Suballocations.end();
        5126  ++it)
        5127  {
        5128  if(it->type != VMA_SUBALLOCATION_TYPE_FREE &&
        5129  it->hAllocation->CanBecomeLost() &&
        5130  it->hAllocation->MakeLost(currentFrameIndex, frameInUseCount))
        5131  {
        5132  it = FreeSuballocation(it);
        5133  ++lostAllocationCount;
        5134  }
        5135  }
        5136  return lostAllocationCount;
        5137 }
        5138 
        5139 void VmaBlockMetadata::Alloc(
        5140  const VmaAllocationRequest& request,
        5141  VmaSuballocationType type,
        5142  VkDeviceSize allocSize,
        5143  VmaAllocation hAllocation)
        5144 {
        5145  VMA_ASSERT(request.item != m_Suballocations.end());
        5146  VmaSuballocation& suballoc = *request.item;
        5147  // Given suballocation is a free block.
        5148  VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
        5149  // Given offset is inside this suballocation.
        5150  VMA_ASSERT(request.offset >= suballoc.offset);
        5151  const VkDeviceSize paddingBegin = request.offset - suballoc.offset;
        5152  VMA_ASSERT(suballoc.size >= paddingBegin + allocSize);
        5153  const VkDeviceSize paddingEnd = suballoc.size - paddingBegin - allocSize;
        5154 
        5155  // Unregister this free suballocation from m_FreeSuballocationsBySize and update
        5156  // it to become used.
        5157  UnregisterFreeSuballocation(request.item);
        5158 
        5159  suballoc.offset = request.offset;
        5160  suballoc.size = allocSize;
        5161  suballoc.type = type;
        5162  suballoc.hAllocation = hAllocation;
        5163 
        5164  // If there are any free bytes remaining at the end, insert new free suballocation after current one.
        5165  if(paddingEnd)
        5166  {
        5167  VmaSuballocation paddingSuballoc = {};
        5168  paddingSuballoc.offset = request.offset + allocSize;
        5169  paddingSuballoc.size = paddingEnd;
        5170  paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
        5171  VmaSuballocationList::iterator next = request.item;
        5172  ++next;
        5173  const VmaSuballocationList::iterator paddingEndItem =
        5174  m_Suballocations.insert(next, paddingSuballoc);
        5175  RegisterFreeSuballocation(paddingEndItem);
        5176  }
        5177 
        5178  // If there are any free bytes remaining at the beginning, insert new free suballocation before current one.
        5179  if(paddingBegin)
        5180  {
        5181  VmaSuballocation paddingSuballoc = {};
        5182  paddingSuballoc.offset = request.offset - paddingBegin;
        5183  paddingSuballoc.size = paddingBegin;
        5184  paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
        5185  const VmaSuballocationList::iterator paddingBeginItem =
        5186  m_Suballocations.insert(request.item, paddingSuballoc);
        5187  RegisterFreeSuballocation(paddingBeginItem);
        5188  }
        5189 
        5190  // Update totals.
        5191  m_FreeCount = m_FreeCount - 1;
        5192  if(paddingBegin > 0)
        5193  {
        5194  ++m_FreeCount;
        5195  }
        5196  if(paddingEnd > 0)
        5197  {
        5198  ++m_FreeCount;
        5199  }
        5200  m_SumFreeSize -= allocSize;
        5201 }
        5202 
        5203 void VmaBlockMetadata::Free(const VmaAllocation allocation)
        5204 {
        5205  for(VmaSuballocationList::iterator suballocItem = m_Suballocations.begin();
        5206  suballocItem != m_Suballocations.end();
        5207  ++suballocItem)
        5208  {
        5209  VmaSuballocation& suballoc = *suballocItem;
        5210  if(suballoc.hAllocation == allocation)
        5211  {
        5212  FreeSuballocation(suballocItem);
        5213  VMA_HEAVY_ASSERT(Validate());
        5214  return;
        5215  }
        5216  }
        5217  VMA_ASSERT(0 && "Not found!");
        5218 }
        5219 
        5220 bool VmaBlockMetadata::ValidateFreeSuballocationList() const
        5221 {
        5222  VkDeviceSize lastSize = 0;
        5223  for(size_t i = 0, count = m_FreeSuballocationsBySize.size(); i < count; ++i)
        5224  {
        5225  const VmaSuballocationList::iterator it = m_FreeSuballocationsBySize[i];
        5226 
        5227  if(it->type != VMA_SUBALLOCATION_TYPE_FREE)
        5228  {
        5229  VMA_ASSERT(0);
        5230  return false;
        5231  }
        5232  if(it->size < VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
        5233  {
        5234  VMA_ASSERT(0);
        5235  return false;
        5236  }
        5237  if(it->size < lastSize)
        5238  {
        5239  VMA_ASSERT(0);
        5240  return false;
        5241  }
        5242 
        5243  lastSize = it->size;
        5244  }
        5245  return true;
        5246 }
        5247 
        5248 bool VmaBlockMetadata::CheckAllocation(
        5249  uint32_t currentFrameIndex,
        5250  uint32_t frameInUseCount,
        5251  VkDeviceSize bufferImageGranularity,
        5252  VkDeviceSize allocSize,
        5253  VkDeviceSize allocAlignment,
        5254  VmaSuballocationType allocType,
        5255  VmaSuballocationList::const_iterator suballocItem,
        5256  bool canMakeOtherLost,
        5257  VkDeviceSize* pOffset,
        5258  size_t* itemsToMakeLostCount,
        5259  VkDeviceSize* pSumFreeSize,
        5260  VkDeviceSize* pSumItemSize) const
        5261 {
        5262  VMA_ASSERT(allocSize > 0);
        5263  VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
        5264  VMA_ASSERT(suballocItem != m_Suballocations.cend());
        5265  VMA_ASSERT(pOffset != VMA_NULL);
        5266 
        5267  *itemsToMakeLostCount = 0;
        5268  *pSumFreeSize = 0;
        5269  *pSumItemSize = 0;
        5270 
        5271  if(canMakeOtherLost)
        5272  {
        5273  if(suballocItem->type == VMA_SUBALLOCATION_TYPE_FREE)
        5274  {
        5275  *pSumFreeSize = suballocItem->size;
        5276  }
        5277  else
        5278  {
        5279  if(suballocItem->hAllocation->CanBecomeLost() &&
        5280  suballocItem->hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
        5281  {
        5282  ++*itemsToMakeLostCount;
        5283  *pSumItemSize = suballocItem->size;
        5284  }
        5285  else
        5286  {
        5287  return false;
        5288  }
        5289  }
        5290 
        5291  // Remaining size is too small for this request: Early return.
        5292  if(m_Size - suballocItem->offset < allocSize)
        5293  {
        5294  return false;
        5295  }
        5296 
        5297  // Start from offset equal to beginning of this suballocation.
        5298  *pOffset = suballocItem->offset;
        5299 
        5300  // Apply VMA_DEBUG_MARGIN at the beginning.
        5301  if((VMA_DEBUG_MARGIN > 0) && suballocItem != m_Suballocations.cbegin())
        5302  {
        5303  *pOffset += VMA_DEBUG_MARGIN;
        5304  }
        5305 
        5306  // Apply alignment.
        5307  const VkDeviceSize alignment = VMA_MAX(allocAlignment, static_cast<VkDeviceSize>(VMA_DEBUG_ALIGNMENT));
        5308  *pOffset = VmaAlignUp(*pOffset, alignment);
        5309 
        5310  // Check previous suballocations for BufferImageGranularity conflicts.
        5311  // Make bigger alignment if necessary.
        5312  if(bufferImageGranularity > 1)
        5313  {
        5314  bool bufferImageGranularityConflict = false;
        5315  VmaSuballocationList::const_iterator prevSuballocItem = suballocItem;
        5316  while(prevSuballocItem != m_Suballocations.cbegin())
        5317  {
        5318  --prevSuballocItem;
        5319  const VmaSuballocation& prevSuballoc = *prevSuballocItem;
        5320  if(VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, *pOffset, bufferImageGranularity))
        5321  {
        5322  if(VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
        5323  {
        5324  bufferImageGranularityConflict = true;
        5325  break;
        5326  }
        5327  }
        5328  else
        5329  // Already on previous page.
        5330  break;
        5331  }
        5332  if(bufferImageGranularityConflict)
        5333  {
        5334  *pOffset = VmaAlignUp(*pOffset, bufferImageGranularity);
        5335  }
        5336  }
        5337 
        5338  // Now that we have final *pOffset, check if we are past suballocItem.
        5339  // If yes, return false - this function should be called for another suballocItem as starting point.
        5340  if(*pOffset >= suballocItem->offset + suballocItem->size)
        5341  {
        5342  return false;
        5343  }
        5344 
        5345  // Calculate padding at the beginning based on current offset.
        5346  const VkDeviceSize paddingBegin = *pOffset - suballocItem->offset;
        5347 
        5348  // Calculate required margin at the end if this is not last suballocation.
        5349  VmaSuballocationList::const_iterator next = suballocItem;
        5350  ++next;
        5351  const VkDeviceSize requiredEndMargin =
        5352  (next != m_Suballocations.cend()) ? VMA_DEBUG_MARGIN : 0;
        5353 
        5354  const VkDeviceSize totalSize = paddingBegin + allocSize + requiredEndMargin;
        5355  // Another early return check.
        5356  if(suballocItem->offset + totalSize > m_Size)
        5357  {
        5358  return false;
        5359  }
        5360 
        5361  // Advance lastSuballocItem until desired size is reached.
        5362  // Update itemsToMakeLostCount.
        5363  VmaSuballocationList::const_iterator lastSuballocItem = suballocItem;
        5364  if(totalSize > suballocItem->size)
        5365  {
        5366  VkDeviceSize remainingSize = totalSize - suballocItem->size;
        5367  while(remainingSize > 0)
        5368  {
        5369  ++lastSuballocItem;
        5370  if(lastSuballocItem == m_Suballocations.cend())
        5371  {
        5372  return false;
        5373  }
        5374  if(lastSuballocItem->type == VMA_SUBALLOCATION_TYPE_FREE)
        5375  {
        5376  *pSumFreeSize += lastSuballocItem->size;
        5377  }
        5378  else
        5379  {
        5380  VMA_ASSERT(lastSuballocItem->hAllocation != VK_NULL_HANDLE);
        5381  if(lastSuballocItem->hAllocation->CanBecomeLost() &&
        5382  lastSuballocItem->hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
        5383  {
        5384  ++*itemsToMakeLostCount;
        5385  *pSumItemSize += lastSuballocItem->size;
        5386  }
        5387  else
        5388  {
        5389  return false;
        5390  }
        5391  }
        5392  remainingSize = (lastSuballocItem->size < remainingSize) ?
        5393  remainingSize - lastSuballocItem->size : 0;
        5394  }
        5395  }
        5396 
        5397  // Check next suballocations for BufferImageGranularity conflicts.
        5398  // If conflict exists, we must mark more allocations lost or fail.
        5399  if(bufferImageGranularity > 1)
        5400  {
        5401  VmaSuballocationList::const_iterator nextSuballocItem = lastSuballocItem;
        5402  ++nextSuballocItem;
        5403  while(nextSuballocItem != m_Suballocations.cend())
        5404  {
        5405  const VmaSuballocation& nextSuballoc = *nextSuballocItem;
        5406  if(VmaBlocksOnSamePage(*pOffset, allocSize, nextSuballoc.offset, bufferImageGranularity))
        5407  {
        5408  if(VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
        5409  {
        5410  VMA_ASSERT(nextSuballoc.hAllocation != VK_NULL_HANDLE);
        5411  if(nextSuballoc.hAllocation->CanBecomeLost() &&
        5412  nextSuballoc.hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
        5413  {
        5414  ++*itemsToMakeLostCount;
        5415  }
        5416  else
        5417  {
        5418  return false;
        5419  }
        5420  }
        5421  }
        5422  else
        5423  {
        5424  // Already on next page.
        5425  break;
        5426  }
        5427  ++nextSuballocItem;
        5428  }
        5429  }
        5430  }
        5431  else
        5432  {
        5433  const VmaSuballocation& suballoc = *suballocItem;
        5434  VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
        5435 
        5436  *pSumFreeSize = suballoc.size;
        5437 
        5438  // Size of this suballocation is too small for this request: Early return.
        5439  if(suballoc.size < allocSize)
        5440  {
        5441  return false;
        5442  }
        5443 
        5444  // Start from offset equal to beginning of this suballocation.
        5445  *pOffset = suballoc.offset;
        5446 
        5447  // Apply VMA_DEBUG_MARGIN at the beginning.
        5448  if((VMA_DEBUG_MARGIN > 0) && suballocItem != m_Suballocations.cbegin())
        5449  {
        5450  *pOffset += VMA_DEBUG_MARGIN;
        5451  }
        5452 
        5453  // Apply alignment.
        5454  const VkDeviceSize alignment = VMA_MAX(allocAlignment, static_cast<VkDeviceSize>(VMA_DEBUG_ALIGNMENT));
        5455  *pOffset = VmaAlignUp(*pOffset, alignment);
        5456 
        5457  // Check previous suballocations for BufferImageGranularity conflicts.
        5458  // Make bigger alignment if necessary.
        5459  if(bufferImageGranularity > 1)
        5460  {
        5461  bool bufferImageGranularityConflict = false;
        5462  VmaSuballocationList::const_iterator prevSuballocItem = suballocItem;
        5463  while(prevSuballocItem != m_Suballocations.cbegin())
        5464  {
        5465  --prevSuballocItem;
        5466  const VmaSuballocation& prevSuballoc = *prevSuballocItem;
        5467  if(VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, *pOffset, bufferImageGranularity))
        5468  {
        5469  if(VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
        5470  {
        5471  bufferImageGranularityConflict = true;
        5472  break;
        5473  }
        5474  }
        5475  else
        5476  // Already on previous page.
        5477  break;
        5478  }
        5479  if(bufferImageGranularityConflict)
        5480  {
        5481  *pOffset = VmaAlignUp(*pOffset, bufferImageGranularity);
        5482  }
        5483  }
        5484 
        5485  // Calculate padding at the beginning based on current offset.
        5486  const VkDeviceSize paddingBegin = *pOffset - suballoc.offset;
        5487 
        5488  // Calculate required margin at the end if this is not last suballocation.
        5489  VmaSuballocationList::const_iterator next = suballocItem;
        5490  ++next;
        5491  const VkDeviceSize requiredEndMargin =
        5492  (next != m_Suballocations.cend()) ? VMA_DEBUG_MARGIN : 0;
        5493 
        5494  // Fail if requested size plus margin before and after is bigger than size of this suballocation.
        5495  if(paddingBegin + allocSize + requiredEndMargin > suballoc.size)
        5496  {
        5497  return false;
        5498  }
        5499 
        5500  // Check next suballocations for BufferImageGranularity conflicts.
        5501  // If conflict exists, allocation cannot be made here.
        5502  if(bufferImageGranularity > 1)
        5503  {
        5504  VmaSuballocationList::const_iterator nextSuballocItem = suballocItem;
        5505  ++nextSuballocItem;
        5506  while(nextSuballocItem != m_Suballocations.cend())
        5507  {
        5508  const VmaSuballocation& nextSuballoc = *nextSuballocItem;
        5509  if(VmaBlocksOnSamePage(*pOffset, allocSize, nextSuballoc.offset, bufferImageGranularity))
        5510  {
        5511  if(VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
        5512  {
        5513  return false;
        5514  }
        5515  }
        5516  else
        5517  {
        5518  // Already on next page.
        5519  break;
        5520  }
        5521  ++nextSuballocItem;
        5522  }
        5523  }
        5524  }
        5525 
        5526  // All tests passed: Success. pOffset is already filled.
        5527  return true;
        5528 }
        5529 
        5530 void VmaBlockMetadata::MergeFreeWithNext(VmaSuballocationList::iterator item)
        5531 {
        5532  VMA_ASSERT(item != m_Suballocations.end());
        5533  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
        5534 
        5535  VmaSuballocationList::iterator nextItem = item;
        5536  ++nextItem;
        5537  VMA_ASSERT(nextItem != m_Suballocations.end());
        5538  VMA_ASSERT(nextItem->type == VMA_SUBALLOCATION_TYPE_FREE);
        5539 
        5540  item->size += nextItem->size;
        5541  --m_FreeCount;
        5542  m_Suballocations.erase(nextItem);
        5543 }
        5544 
        5545 VmaSuballocationList::iterator VmaBlockMetadata::FreeSuballocation(VmaSuballocationList::iterator suballocItem)
        5546 {
        5547  // Change this suballocation to be marked as free.
        5548  VmaSuballocation& suballoc = *suballocItem;
        5549  suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
        5550  suballoc.hAllocation = VK_NULL_HANDLE;
        5551 
        5552  // Update totals.
        5553  ++m_FreeCount;
        5554  m_SumFreeSize += suballoc.size;
        5555 
        5556  // Merge with previous and/or next suballocation if it's also free.
        5557  bool mergeWithNext = false;
        5558  bool mergeWithPrev = false;
        5559 
        5560  VmaSuballocationList::iterator nextItem = suballocItem;
        5561  ++nextItem;
        5562  if((nextItem != m_Suballocations.end()) && (nextItem->type == VMA_SUBALLOCATION_TYPE_FREE))
        5563  {
        5564  mergeWithNext = true;
        5565  }
        5566 
        5567  VmaSuballocationList::iterator prevItem = suballocItem;
        5568  if(suballocItem != m_Suballocations.begin())
        5569  {
        5570  --prevItem;
        5571  if(prevItem->type == VMA_SUBALLOCATION_TYPE_FREE)
        5572  {
        5573  mergeWithPrev = true;
        5574  }
        5575  }
        5576 
        5577  if(mergeWithNext)
        5578  {
        5579  UnregisterFreeSuballocation(nextItem);
        5580  MergeFreeWithNext(suballocItem);
        5581  }
        5582 
        5583  if(mergeWithPrev)
        5584  {
        5585  UnregisterFreeSuballocation(prevItem);
        5586  MergeFreeWithNext(prevItem);
        5587  RegisterFreeSuballocation(prevItem);
        5588  return prevItem;
        5589  }
        5590  else
        5591  {
        5592  RegisterFreeSuballocation(suballocItem);
        5593  return suballocItem;
        5594  }
        5595 }
        5596 
        5597 void VmaBlockMetadata::RegisterFreeSuballocation(VmaSuballocationList::iterator item)
        5598 {
        5599  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
        5600  VMA_ASSERT(item->size > 0);
        5601 
        5602  // You may want to enable this validation at the beginning or at the end of
        5603  // this function, depending on what do you want to check.
        5604  VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
        5605 
        5606  if(item->size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
        5607  {
        5608  if(m_FreeSuballocationsBySize.empty())
        5609  {
        5610  m_FreeSuballocationsBySize.push_back(item);
        5611  }
        5612  else
        5613  {
        5614  VmaVectorInsertSorted<VmaSuballocationItemSizeLess>(m_FreeSuballocationsBySize, item);
        5615  }
        5616  }
        5617 
        5618  //VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
        5619 }
        5620 
        5621 
        5622 void VmaBlockMetadata::UnregisterFreeSuballocation(VmaSuballocationList::iterator item)
        5623 {
        5624  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
        5625  VMA_ASSERT(item->size > 0);
        5626 
        5627  // You may want to enable this validation at the beginning or at the end of
        5628  // this function, depending on what do you want to check.
        5629  VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
        5630 
        5631  if(item->size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
        5632  {
        5633  VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
        5634  m_FreeSuballocationsBySize.data(),
        5635  m_FreeSuballocationsBySize.data() + m_FreeSuballocationsBySize.size(),
        5636  item,
        5637  VmaSuballocationItemSizeLess());
        5638  for(size_t index = it - m_FreeSuballocationsBySize.data();
        5639  index < m_FreeSuballocationsBySize.size();
        5640  ++index)
        5641  {
        5642  if(m_FreeSuballocationsBySize[index] == item)
        5643  {
        5644  VmaVectorRemove(m_FreeSuballocationsBySize, index);
        5645  return;
        5646  }
        5647  VMA_ASSERT((m_FreeSuballocationsBySize[index]->size == item->size) && "Not found.");
        5648  }
        5649  VMA_ASSERT(0 && "Not found.");
        5650  }
        5651 
        5652  //VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
        5653 }
        5654 
        5656 // class VmaDeviceMemoryMapping
        5657 
        5658 VmaDeviceMemoryMapping::VmaDeviceMemoryMapping() :
        5659  m_MapCount(0),
        5660  m_pMappedData(VMA_NULL)
        5661 {
        5662 }
        5663 
        5664 VmaDeviceMemoryMapping::~VmaDeviceMemoryMapping()
        5665 {
        5666  VMA_ASSERT(m_MapCount == 0 && "VkDeviceMemory block is being destroyed while it is still mapped.");
        5667 }
        5668 
        5669 VkResult VmaDeviceMemoryMapping::Map(VmaAllocator hAllocator, VkDeviceMemory hMemory, void **ppData)
        5670 {
        5671  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
        5672  if(m_MapCount != 0)
        5673  {
        5674  ++m_MapCount;
        5675  VMA_ASSERT(m_pMappedData != VMA_NULL);
        5676  if(ppData != VMA_NULL)
        5677  {
        5678  *ppData = m_pMappedData;
        5679  }
        5680  return VK_SUCCESS;
        5681  }
        5682  else
        5683  {
        5684  VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
        5685  hAllocator->m_hDevice,
        5686  hMemory,
        5687  0, // offset
        5688  VK_WHOLE_SIZE,
        5689  0, // flags
        5690  &m_pMappedData);
        5691  if(result == VK_SUCCESS)
        5692  {
        5693  if(ppData != VMA_NULL)
        5694  {
        5695  *ppData = m_pMappedData;
        5696  }
        5697  m_MapCount = 1;
        5698  }
        5699  return result;
        5700  }
        5701 }
        5702 
        5703 void VmaDeviceMemoryMapping::Unmap(VmaAllocator hAllocator, VkDeviceMemory hMemory)
        5704 {
        5705  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
        5706  if(m_MapCount != 0)
        5707  {
        5708  if(--m_MapCount == 0)
        5709  {
        5710  m_pMappedData = VMA_NULL;
        5711  (*hAllocator->GetVulkanFunctions().vkUnmapMemory)(hAllocator->m_hDevice, hMemory);
        5712  }
        5713  }
        5714  else
        5715  {
        5716  VMA_ASSERT(0 && "VkDeviceMemory block is being unmapped while it was not previously mapped.");
        5717  }
        5718 }
        5719 
        5721 // class VmaDeviceMemoryBlock
        5722 
        5723 VmaDeviceMemoryBlock::VmaDeviceMemoryBlock(VmaAllocator hAllocator) :
        5724  m_MemoryTypeIndex(UINT32_MAX),
        5725  m_hMemory(VK_NULL_HANDLE),
        5726  m_Metadata(hAllocator)
        5727 {
        5728 }
        5729 
        5730 void VmaDeviceMemoryBlock::Init(
        5731  uint32_t newMemoryTypeIndex,
        5732  VkDeviceMemory newMemory,
        5733  VkDeviceSize newSize)
        5734 {
        5735  VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
        5736 
        5737  m_MemoryTypeIndex = newMemoryTypeIndex;
        5738  m_hMemory = newMemory;
        5739 
        5740  m_Metadata.Init(newSize);
        5741 }
        5742 
        5743 void VmaDeviceMemoryBlock::Destroy(VmaAllocator allocator)
        5744 {
        5745  // This is the most important assert in the entire library.
        5746  // Hitting it means you have some memory leak - unreleased VmaAllocation objects.
        5747  VMA_ASSERT(m_Metadata.IsEmpty() && "Some allocations were not freed before destruction of this memory block!");
        5748 
        5749  VMA_ASSERT(m_hMemory != VK_NULL_HANDLE);
        5750  allocator->FreeVulkanMemory(m_MemoryTypeIndex, m_Metadata.GetSize(), m_hMemory);
        5751  m_hMemory = VK_NULL_HANDLE;
        5752 }
        5753 
        5754 bool VmaDeviceMemoryBlock::Validate() const
        5755 {
        5756  if((m_hMemory == VK_NULL_HANDLE) ||
        5757  (m_Metadata.GetSize() == 0))
        5758  {
        5759  return false;
        5760  }
        5761 
        5762  return m_Metadata.Validate();
        5763 }
        5764 
        5765 VkResult VmaDeviceMemoryBlock::Map(VmaAllocator hAllocator, void** ppData)
        5766 {
        5767  return m_Mapping.Map(hAllocator, m_hMemory, ppData);
        5768 }
        5769 
        5770 void VmaDeviceMemoryBlock::Unmap(VmaAllocator hAllocator)
        5771 {
        5772  m_Mapping.Unmap(hAllocator, m_hMemory);
        5773 }
        5774 
        5775 static void InitStatInfo(VmaStatInfo& outInfo)
        5776 {
        5777  memset(&outInfo, 0, sizeof(outInfo));
        5778  outInfo.allocationSizeMin = UINT64_MAX;
        5779  outInfo.unusedRangeSizeMin = UINT64_MAX;
        5780 }
        5781 
        5782 // Adds statistics srcInfo into inoutInfo, like: inoutInfo += srcInfo.
        5783 static void VmaAddStatInfo(VmaStatInfo& inoutInfo, const VmaStatInfo& srcInfo)
        5784 {
        5785  inoutInfo.blockCount += srcInfo.blockCount;
        5786  inoutInfo.allocationCount += srcInfo.allocationCount;
        5787  inoutInfo.unusedRangeCount += srcInfo.unusedRangeCount;
        5788  inoutInfo.usedBytes += srcInfo.usedBytes;
        5789  inoutInfo.unusedBytes += srcInfo.unusedBytes;
        5790  inoutInfo.allocationSizeMin = VMA_MIN(inoutInfo.allocationSizeMin, srcInfo.allocationSizeMin);
        5791  inoutInfo.allocationSizeMax = VMA_MAX(inoutInfo.allocationSizeMax, srcInfo.allocationSizeMax);
        5792  inoutInfo.unusedRangeSizeMin = VMA_MIN(inoutInfo.unusedRangeSizeMin, srcInfo.unusedRangeSizeMin);
        5793  inoutInfo.unusedRangeSizeMax = VMA_MAX(inoutInfo.unusedRangeSizeMax, srcInfo.unusedRangeSizeMax);
        5794 }
        5795 
        5796 static void VmaPostprocessCalcStatInfo(VmaStatInfo& inoutInfo)
        5797 {
        5798  inoutInfo.allocationSizeAvg = (inoutInfo.allocationCount > 0) ?
        5799  VmaRoundDiv<VkDeviceSize>(inoutInfo.usedBytes, inoutInfo.allocationCount) : 0;
        5800  inoutInfo.unusedRangeSizeAvg = (inoutInfo.unusedRangeCount > 0) ?
        5801  VmaRoundDiv<VkDeviceSize>(inoutInfo.unusedBytes, inoutInfo.unusedRangeCount) : 0;
        5802 }
        5803 
        5804 VmaPool_T::VmaPool_T(
        5805  VmaAllocator hAllocator,
        5806  const VmaPoolCreateInfo& createInfo) :
        5807  m_BlockVector(
        5808  hAllocator,
        5809  createInfo.memoryTypeIndex,
        5810  createInfo.blockSize,
        5811  createInfo.minBlockCount,
        5812  createInfo.maxBlockCount,
        5813  (createInfo.flags & VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT) != 0 ? 1 : hAllocator->GetBufferImageGranularity(),
        5814  createInfo.frameInUseCount,
        5815  true) // isCustomPool
        5816 {
        5817 }
        5818 
        5819 VmaPool_T::~VmaPool_T()
        5820 {
        5821 }
        5822 
        5823 #if VMA_STATS_STRING_ENABLED
        5824 
        5825 #endif // #if VMA_STATS_STRING_ENABLED
        5826 
        5827 VmaBlockVector::VmaBlockVector(
        5828  VmaAllocator hAllocator,
        5829  uint32_t memoryTypeIndex,
        5830  VkDeviceSize preferredBlockSize,
        5831  size_t minBlockCount,
        5832  size_t maxBlockCount,
        5833  VkDeviceSize bufferImageGranularity,
        5834  uint32_t frameInUseCount,
        5835  bool isCustomPool) :
        5836  m_hAllocator(hAllocator),
        5837  m_MemoryTypeIndex(memoryTypeIndex),
        5838  m_PreferredBlockSize(preferredBlockSize),
        5839  m_MinBlockCount(minBlockCount),
        5840  m_MaxBlockCount(maxBlockCount),
        5841  m_BufferImageGranularity(bufferImageGranularity),
        5842  m_FrameInUseCount(frameInUseCount),
        5843  m_IsCustomPool(isCustomPool),
        5844  m_Blocks(VmaStlAllocator<VmaDeviceMemoryBlock*>(hAllocator->GetAllocationCallbacks())),
        5845  m_HasEmptyBlock(false),
        5846  m_pDefragmentator(VMA_NULL)
        5847 {
        5848 }
        5849 
        5850 VmaBlockVector::~VmaBlockVector()
        5851 {
        5852  VMA_ASSERT(m_pDefragmentator == VMA_NULL);
        5853 
        5854  for(size_t i = m_Blocks.size(); i--; )
        5855  {
        5856  m_Blocks[i]->Destroy(m_hAllocator);
        5857  vma_delete(m_hAllocator, m_Blocks[i]);
        5858  }
        5859 }
        5860 
        5861 VkResult VmaBlockVector::CreateMinBlocks()
        5862 {
        5863  for(size_t i = 0; i < m_MinBlockCount; ++i)
        5864  {
        5865  VkResult res = CreateBlock(m_PreferredBlockSize, VMA_NULL);
        5866  if(res != VK_SUCCESS)
        5867  {
        5868  return res;
        5869  }
        5870  }
        5871  return VK_SUCCESS;
        5872 }
        5873 
        5874 void VmaBlockVector::GetPoolStats(VmaPoolStats* pStats)
        5875 {
        5876  pStats->size = 0;
        5877  pStats->unusedSize = 0;
        5878  pStats->allocationCount = 0;
        5879  pStats->unusedRangeCount = 0;
        5880  pStats->unusedRangeSizeMax = 0;
        5881 
        5882  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        5883 
        5884  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
        5885  {
        5886  const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
        5887  VMA_ASSERT(pBlock);
        5888  VMA_HEAVY_ASSERT(pBlock->Validate());
        5889  pBlock->m_Metadata.AddPoolStats(*pStats);
        5890  }
        5891 }
        5892 
        5893 static const uint32_t VMA_ALLOCATION_TRY_COUNT = 32;
        5894 
        5895 VkResult VmaBlockVector::Allocate(
        5896  VmaPool hCurrentPool,
        5897  uint32_t currentFrameIndex,
        5898  const VkMemoryRequirements& vkMemReq,
        5899  const VmaAllocationCreateInfo& createInfo,
        5900  VmaSuballocationType suballocType,
        5901  VmaAllocation* pAllocation)
        5902 {
        5903  const bool mapped = (createInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0;
        5904  const bool isUserDataString = (createInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0;
        5905 
        5906  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        5907 
        5908  // 1. Search existing allocations. Try to allocate without making other allocations lost.
        5909  // Forward order in m_Blocks - prefer blocks with smallest amount of free space.
        5910  for(size_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex )
        5911  {
        5912  VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
        5913  VMA_ASSERT(pCurrBlock);
        5914  VmaAllocationRequest currRequest = {};
        5915  if(pCurrBlock->m_Metadata.CreateAllocationRequest(
        5916  currentFrameIndex,
        5917  m_FrameInUseCount,
        5918  m_BufferImageGranularity,
        5919  vkMemReq.size,
        5920  vkMemReq.alignment,
        5921  suballocType,
        5922  false, // canMakeOtherLost
        5923  &currRequest))
        5924  {
        5925  // Allocate from pCurrBlock.
        5926  VMA_ASSERT(currRequest.itemsToMakeLostCount == 0);
        5927 
        5928  if(mapped)
        5929  {
        5930  VkResult res = pCurrBlock->Map(m_hAllocator, nullptr);
        5931  if(res != VK_SUCCESS)
        5932  {
        5933  return res;
        5934  }
        5935  }
        5936 
        5937  // We no longer have an empty Allocation.
        5938  if(pCurrBlock->m_Metadata.IsEmpty())
        5939  {
        5940  m_HasEmptyBlock = false;
        5941  }
        5942 
        5943  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
        5944  pCurrBlock->m_Metadata.Alloc(currRequest, suballocType, vkMemReq.size, *pAllocation);
        5945  (*pAllocation)->InitBlockAllocation(
        5946  hCurrentPool,
        5947  pCurrBlock,
        5948  currRequest.offset,
        5949  vkMemReq.alignment,
        5950  vkMemReq.size,
        5951  suballocType,
        5952  mapped,
        5953  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
        5954  VMA_HEAVY_ASSERT(pCurrBlock->Validate());
        5955  VMA_DEBUG_LOG(" Returned from existing allocation #%u", (uint32_t)blockIndex);
        5956  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
        5957  return VK_SUCCESS;
        5958  }
        5959  }
        5960 
        5961  const bool canCreateNewBlock =
        5962  ((createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == 0) &&
        5963  (m_Blocks.size() < m_MaxBlockCount);
        5964 
        5965  // 2. Try to create new block.
        5966  if(canCreateNewBlock)
        5967  {
        5968  // 2.1. Start with full preferredBlockSize.
        5969  VkDeviceSize blockSize = m_PreferredBlockSize;
        5970  size_t newBlockIndex = 0;
        5971  VkResult res = CreateBlock(blockSize, &newBlockIndex);
        5972  // Allocating blocks of other sizes is allowed only in default pools.
        5973  // In custom pools block size is fixed.
        5974  if(res < 0 && m_IsCustomPool == false)
        5975  {
        5976  // 2.2. Try half the size.
        5977  blockSize /= 2;
        5978  if(blockSize >= vkMemReq.size)
        5979  {
        5980  res = CreateBlock(blockSize, &newBlockIndex);
        5981  if(res < 0)
        5982  {
        5983  // 2.3. Try quarter the size.
        5984  blockSize /= 2;
        5985  if(blockSize >= vkMemReq.size)
        5986  {
        5987  res = CreateBlock(blockSize, &newBlockIndex);
        5988  }
        5989  }
        5990  }
        5991  }
        5992  if(res == VK_SUCCESS)
        5993  {
        5994  VmaDeviceMemoryBlock* const pBlock = m_Blocks[newBlockIndex];
        5995  VMA_ASSERT(pBlock->m_Metadata.GetSize() >= vkMemReq.size);
        5996 
        5997  if(mapped)
        5998  {
        5999  res = pBlock->Map(m_hAllocator, nullptr);
        6000  if(res != VK_SUCCESS)
        6001  {
        6002  return res;
        6003  }
        6004  }
        6005 
        6006  // Allocate from pBlock. Because it is empty, dstAllocRequest can be trivially filled.
        6007  VmaAllocationRequest allocRequest;
        6008  pBlock->m_Metadata.CreateFirstAllocationRequest(&allocRequest);
        6009  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
        6010  pBlock->m_Metadata.Alloc(allocRequest, suballocType, vkMemReq.size, *pAllocation);
        6011  (*pAllocation)->InitBlockAllocation(
        6012  hCurrentPool,
        6013  pBlock,
        6014  allocRequest.offset,
        6015  vkMemReq.alignment,
        6016  vkMemReq.size,
        6017  suballocType,
        6018  mapped,
        6019  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
        6020  VMA_HEAVY_ASSERT(pBlock->Validate());
        6021  VMA_DEBUG_LOG(" Created new allocation Size=%llu", allocInfo.allocationSize);
        6022  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
        6023  return VK_SUCCESS;
        6024  }
        6025  }
        6026 
        6027  const bool canMakeOtherLost = (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT) != 0;
        6028 
        6029  // 3. Try to allocate from existing blocks with making other allocations lost.
        6030  if(canMakeOtherLost)
        6031  {
        6032  uint32_t tryIndex = 0;
        6033  for(; tryIndex < VMA_ALLOCATION_TRY_COUNT; ++tryIndex)
        6034  {
        6035  VmaDeviceMemoryBlock* pBestRequestBlock = VMA_NULL;
        6036  VmaAllocationRequest bestRequest = {};
        6037  VkDeviceSize bestRequestCost = VK_WHOLE_SIZE;
        6038 
        6039  // 1. Search existing allocations.
        6040  // Forward order in m_Blocks - prefer blocks with smallest amount of free space.
        6041  for(size_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex )
        6042  {
        6043  VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
        6044  VMA_ASSERT(pCurrBlock);
        6045  VmaAllocationRequest currRequest = {};
        6046  if(pCurrBlock->m_Metadata.CreateAllocationRequest(
        6047  currentFrameIndex,
        6048  m_FrameInUseCount,
        6049  m_BufferImageGranularity,
        6050  vkMemReq.size,
        6051  vkMemReq.alignment,
        6052  suballocType,
        6053  canMakeOtherLost,
        6054  &currRequest))
        6055  {
        6056  const VkDeviceSize currRequestCost = currRequest.CalcCost();
        6057  if(pBestRequestBlock == VMA_NULL ||
        6058  currRequestCost < bestRequestCost)
        6059  {
        6060  pBestRequestBlock = pCurrBlock;
        6061  bestRequest = currRequest;
        6062  bestRequestCost = currRequestCost;
        6063 
        6064  if(bestRequestCost == 0)
        6065  {
        6066  break;
        6067  }
        6068  }
        6069  }
        6070  }
        6071 
        6072  if(pBestRequestBlock != VMA_NULL)
        6073  {
        6074  if(mapped)
        6075  {
        6076  VkResult res = pBestRequestBlock->Map(m_hAllocator, nullptr);
        6077  if(res != VK_SUCCESS)
        6078  {
        6079  return res;
        6080  }
        6081  }
        6082 
        6083  if(pBestRequestBlock->m_Metadata.MakeRequestedAllocationsLost(
        6084  currentFrameIndex,
        6085  m_FrameInUseCount,
        6086  &bestRequest))
        6087  {
        6088  // We no longer have an empty Allocation.
        6089  if(pBestRequestBlock->m_Metadata.IsEmpty())
        6090  {
        6091  m_HasEmptyBlock = false;
        6092  }
        6093  // Allocate from this pBlock.
        6094  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
        6095  pBestRequestBlock->m_Metadata.Alloc(bestRequest, suballocType, vkMemReq.size, *pAllocation);
        6096  (*pAllocation)->InitBlockAllocation(
        6097  hCurrentPool,
        6098  pBestRequestBlock,
        6099  bestRequest.offset,
        6100  vkMemReq.alignment,
        6101  vkMemReq.size,
        6102  suballocType,
        6103  mapped,
        6104  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
        6105  VMA_HEAVY_ASSERT(pBlock->Validate());
        6106  VMA_DEBUG_LOG(" Returned from existing allocation #%u", (uint32_t)blockIndex);
        6107  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
        6108  return VK_SUCCESS;
        6109  }
        6110  // else: Some allocations must have been touched while we are here. Next try.
        6111  }
        6112  else
        6113  {
        6114  // Could not find place in any of the blocks - break outer loop.
        6115  break;
        6116  }
        6117  }
        6118  /* Maximum number of tries exceeded - a very unlike event when many other
        6119  threads are simultaneously touching allocations making it impossible to make
        6120  lost at the same time as we try to allocate. */
        6121  if(tryIndex == VMA_ALLOCATION_TRY_COUNT)
        6122  {
        6123  return VK_ERROR_TOO_MANY_OBJECTS;
        6124  }
        6125  }
        6126 
        6127  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        6128 }
        6129 
        6130 void VmaBlockVector::Free(
        6131  VmaAllocation hAllocation)
        6132 {
        6133  VmaDeviceMemoryBlock* pBlockToDelete = VMA_NULL;
        6134 
        6135  // Scope for lock.
        6136  {
        6137  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        6138 
        6139  VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
        6140 
        6141  if(hAllocation->IsPersistentMap())
        6142  {
        6143  pBlock->m_Mapping.Unmap(m_hAllocator, pBlock->m_hMemory);
        6144  }
        6145 
        6146  pBlock->m_Metadata.Free(hAllocation);
        6147  VMA_HEAVY_ASSERT(pBlock->Validate());
        6148 
        6149  VMA_DEBUG_LOG(" Freed from MemoryTypeIndex=%u", memTypeIndex);
        6150 
        6151  // pBlock became empty after this deallocation.
        6152  if(pBlock->m_Metadata.IsEmpty())
        6153  {
        6154  // Already has empty Allocation. We don't want to have two, so delete this one.
        6155  if(m_HasEmptyBlock && m_Blocks.size() > m_MinBlockCount)
        6156  {
        6157  pBlockToDelete = pBlock;
        6158  Remove(pBlock);
        6159  }
        6160  // We now have first empty Allocation.
        6161  else
        6162  {
        6163  m_HasEmptyBlock = true;
        6164  }
        6165  }
        6166  // pBlock didn't become empty, but we have another empty block - find and free that one.
        6167  // (This is optional, heuristics.)
        6168  else if(m_HasEmptyBlock)
        6169  {
        6170  VmaDeviceMemoryBlock* pLastBlock = m_Blocks.back();
        6171  if(pLastBlock->m_Metadata.IsEmpty() && m_Blocks.size() > m_MinBlockCount)
        6172  {
        6173  pBlockToDelete = pLastBlock;
        6174  m_Blocks.pop_back();
        6175  m_HasEmptyBlock = false;
        6176  }
        6177  }
        6178 
        6179  IncrementallySortBlocks();
        6180  }
        6181 
        6182  // Destruction of a free Allocation. Deferred until this point, outside of mutex
        6183  // lock, for performance reason.
        6184  if(pBlockToDelete != VMA_NULL)
        6185  {
        6186  VMA_DEBUG_LOG(" Deleted empty allocation");
        6187  pBlockToDelete->Destroy(m_hAllocator);
        6188  vma_delete(m_hAllocator, pBlockToDelete);
        6189  }
        6190 }
        6191 
        6192 void VmaBlockVector::Remove(VmaDeviceMemoryBlock* pBlock)
        6193 {
        6194  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
        6195  {
        6196  if(m_Blocks[blockIndex] == pBlock)
        6197  {
        6198  VmaVectorRemove(m_Blocks, blockIndex);
        6199  return;
        6200  }
        6201  }
        6202  VMA_ASSERT(0);
        6203 }
        6204 
        6205 void VmaBlockVector::IncrementallySortBlocks()
        6206 {
        6207  // Bubble sort only until first swap.
        6208  for(size_t i = 1; i < m_Blocks.size(); ++i)
        6209  {
        6210  if(m_Blocks[i - 1]->m_Metadata.GetSumFreeSize() > m_Blocks[i]->m_Metadata.GetSumFreeSize())
        6211  {
        6212  VMA_SWAP(m_Blocks[i - 1], m_Blocks[i]);
        6213  return;
        6214  }
        6215  }
        6216 }
        6217 
        6218 VkResult VmaBlockVector::CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex)
        6219 {
        6220  VkMemoryAllocateInfo allocInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
        6221  allocInfo.memoryTypeIndex = m_MemoryTypeIndex;
        6222  allocInfo.allocationSize = blockSize;
        6223  VkDeviceMemory mem = VK_NULL_HANDLE;
        6224  VkResult res = m_hAllocator->AllocateVulkanMemory(&allocInfo, &mem);
        6225  if(res < 0)
        6226  {
        6227  return res;
        6228  }
        6229 
        6230  // New VkDeviceMemory successfully created.
        6231 
        6232  // Create new Allocation for it.
        6233  VmaDeviceMemoryBlock* const pBlock = vma_new(m_hAllocator, VmaDeviceMemoryBlock)(m_hAllocator);
        6234  pBlock->Init(
        6235  m_MemoryTypeIndex,
        6236  mem,
        6237  allocInfo.allocationSize);
        6238 
        6239  m_Blocks.push_back(pBlock);
        6240  if(pNewBlockIndex != VMA_NULL)
        6241  {
        6242  *pNewBlockIndex = m_Blocks.size() - 1;
        6243  }
        6244 
        6245  return VK_SUCCESS;
        6246 }
        6247 
        6248 #if VMA_STATS_STRING_ENABLED
        6249 
        6250 void VmaBlockVector::PrintDetailedMap(class VmaJsonWriter& json)
        6251 {
        6252  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        6253 
        6254  json.BeginObject();
        6255 
        6256  if(m_IsCustomPool)
        6257  {
        6258  json.WriteString("MemoryTypeIndex");
        6259  json.WriteNumber(m_MemoryTypeIndex);
        6260 
        6261  json.WriteString("BlockSize");
        6262  json.WriteNumber(m_PreferredBlockSize);
        6263 
        6264  json.WriteString("BlockCount");
        6265  json.BeginObject(true);
        6266  if(m_MinBlockCount > 0)
        6267  {
        6268  json.WriteString("Min");
        6269  json.WriteNumber(m_MinBlockCount);
        6270  }
        6271  if(m_MaxBlockCount < SIZE_MAX)
        6272  {
        6273  json.WriteString("Max");
        6274  json.WriteNumber(m_MaxBlockCount);
        6275  }
        6276  json.WriteString("Cur");
        6277  json.WriteNumber(m_Blocks.size());
        6278  json.EndObject();
        6279 
        6280  if(m_FrameInUseCount > 0)
        6281  {
        6282  json.WriteString("FrameInUseCount");
        6283  json.WriteNumber(m_FrameInUseCount);
        6284  }
        6285  }
        6286  else
        6287  {
        6288  json.WriteString("PreferredBlockSize");
        6289  json.WriteNumber(m_PreferredBlockSize);
        6290  }
        6291 
        6292  json.WriteString("Blocks");
        6293  json.BeginArray();
        6294  for(size_t i = 0; i < m_Blocks.size(); ++i)
        6295  {
        6296  m_Blocks[i]->m_Metadata.PrintDetailedMap(json);
        6297  }
        6298  json.EndArray();
        6299 
        6300  json.EndObject();
        6301 }
        6302 
        6303 #endif // #if VMA_STATS_STRING_ENABLED
        6304 
        6305 VmaDefragmentator* VmaBlockVector::EnsureDefragmentator(
        6306  VmaAllocator hAllocator,
        6307  uint32_t currentFrameIndex)
        6308 {
        6309  if(m_pDefragmentator == VMA_NULL)
        6310  {
        6311  m_pDefragmentator = vma_new(m_hAllocator, VmaDefragmentator)(
        6312  hAllocator,
        6313  this,
        6314  currentFrameIndex);
        6315  }
        6316 
        6317  return m_pDefragmentator;
        6318 }
        6319 
        6320 VkResult VmaBlockVector::Defragment(
        6321  VmaDefragmentationStats* pDefragmentationStats,
        6322  VkDeviceSize& maxBytesToMove,
        6323  uint32_t& maxAllocationsToMove)
        6324 {
        6325  if(m_pDefragmentator == VMA_NULL)
        6326  {
        6327  return VK_SUCCESS;
        6328  }
        6329 
        6330  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        6331 
        6332  // Defragment.
        6333  VkResult result = m_pDefragmentator->Defragment(maxBytesToMove, maxAllocationsToMove);
        6334 
        6335  // Accumulate statistics.
        6336  if(pDefragmentationStats != VMA_NULL)
        6337  {
        6338  const VkDeviceSize bytesMoved = m_pDefragmentator->GetBytesMoved();
        6339  const uint32_t allocationsMoved = m_pDefragmentator->GetAllocationsMoved();
        6340  pDefragmentationStats->bytesMoved += bytesMoved;
        6341  pDefragmentationStats->allocationsMoved += allocationsMoved;
        6342  VMA_ASSERT(bytesMoved <= maxBytesToMove);
        6343  VMA_ASSERT(allocationsMoved <= maxAllocationsToMove);
        6344  maxBytesToMove -= bytesMoved;
        6345  maxAllocationsToMove -= allocationsMoved;
        6346  }
        6347 
        6348  // Free empty blocks.
        6349  m_HasEmptyBlock = false;
        6350  for(size_t blockIndex = m_Blocks.size(); blockIndex--; )
        6351  {
        6352  VmaDeviceMemoryBlock* pBlock = m_Blocks[blockIndex];
        6353  if(pBlock->m_Metadata.IsEmpty())
        6354  {
        6355  if(m_Blocks.size() > m_MinBlockCount)
        6356  {
        6357  if(pDefragmentationStats != VMA_NULL)
        6358  {
        6359  ++pDefragmentationStats->deviceMemoryBlocksFreed;
        6360  pDefragmentationStats->bytesFreed += pBlock->m_Metadata.GetSize();
        6361  }
        6362 
        6363  VmaVectorRemove(m_Blocks, blockIndex);
        6364  pBlock->Destroy(m_hAllocator);
        6365  vma_delete(m_hAllocator, pBlock);
        6366  }
        6367  else
        6368  {
        6369  m_HasEmptyBlock = true;
        6370  }
        6371  }
        6372  }
        6373 
        6374  return result;
        6375 }
        6376 
        6377 void VmaBlockVector::DestroyDefragmentator()
        6378 {
        6379  if(m_pDefragmentator != VMA_NULL)
        6380  {
        6381  vma_delete(m_hAllocator, m_pDefragmentator);
        6382  m_pDefragmentator = VMA_NULL;
        6383  }
        6384 }
        6385 
        6386 void VmaBlockVector::MakePoolAllocationsLost(
        6387  uint32_t currentFrameIndex,
        6388  size_t* pLostAllocationCount)
        6389 {
        6390  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        6391 
        6392  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
        6393  {
        6394  VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
        6395  VMA_ASSERT(pBlock);
        6396  pBlock->m_Metadata.MakeAllocationsLost(currentFrameIndex, m_FrameInUseCount);
        6397  }
        6398 }
        6399 
        6400 void VmaBlockVector::AddStats(VmaStats* pStats)
        6401 {
        6402  const uint32_t memTypeIndex = m_MemoryTypeIndex;
        6403  const uint32_t memHeapIndex = m_hAllocator->MemoryTypeIndexToHeapIndex(memTypeIndex);
        6404 
        6405  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
        6406 
        6407  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
        6408  {
        6409  const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
        6410  VMA_ASSERT(pBlock);
        6411  VMA_HEAVY_ASSERT(pBlock->Validate());
        6412  VmaStatInfo allocationStatInfo;
        6413  pBlock->m_Metadata.CalcAllocationStatInfo(allocationStatInfo);
        6414  VmaAddStatInfo(pStats->total, allocationStatInfo);
        6415  VmaAddStatInfo(pStats->memoryType[memTypeIndex], allocationStatInfo);
        6416  VmaAddStatInfo(pStats->memoryHeap[memHeapIndex], allocationStatInfo);
        6417  }
        6418 }
        6419 
        6421 // VmaDefragmentator members definition
        6422 
        6423 VmaDefragmentator::VmaDefragmentator(
        6424  VmaAllocator hAllocator,
        6425  VmaBlockVector* pBlockVector,
        6426  uint32_t currentFrameIndex) :
        6427  m_hAllocator(hAllocator),
        6428  m_pBlockVector(pBlockVector),
        6429  m_CurrentFrameIndex(currentFrameIndex),
        6430  m_BytesMoved(0),
        6431  m_AllocationsMoved(0),
        6432  m_Allocations(VmaStlAllocator<AllocationInfo>(hAllocator->GetAllocationCallbacks())),
        6433  m_Blocks(VmaStlAllocator<BlockInfo*>(hAllocator->GetAllocationCallbacks()))
        6434 {
        6435 }
        6436 
        6437 VmaDefragmentator::~VmaDefragmentator()
        6438 {
        6439  for(size_t i = m_Blocks.size(); i--; )
        6440  {
        6441  vma_delete(m_hAllocator, m_Blocks[i]);
        6442  }
        6443 }
        6444 
        6445 void VmaDefragmentator::AddAllocation(VmaAllocation hAlloc, VkBool32* pChanged)
        6446 {
        6447  AllocationInfo allocInfo;
        6448  allocInfo.m_hAllocation = hAlloc;
        6449  allocInfo.m_pChanged = pChanged;
        6450  m_Allocations.push_back(allocInfo);
        6451 }
        6452 
        6453 VkResult VmaDefragmentator::BlockInfo::EnsureMapping(VmaAllocator hAllocator, void** ppMappedData)
        6454 {
        6455  // It has already been mapped for defragmentation.
        6456  if(m_pMappedDataForDefragmentation)
        6457  {
        6458  *ppMappedData = m_pMappedDataForDefragmentation;
        6459  return VK_SUCCESS;
        6460  }
        6461 
        6462  // It is originally mapped.
        6463  if(m_pBlock->m_Mapping.GetMappedData())
        6464  {
        6465  *ppMappedData = m_pBlock->m_Mapping.GetMappedData();
        6466  return VK_SUCCESS;
        6467  }
        6468 
        6469  // Map on first usage.
        6470  VkResult res = m_pBlock->Map(hAllocator, &m_pMappedDataForDefragmentation);
        6471  *ppMappedData = m_pMappedDataForDefragmentation;
        6472  return res;
        6473 }
        6474 
        6475 void VmaDefragmentator::BlockInfo::Unmap(VmaAllocator hAllocator)
        6476 {
        6477  if(m_pMappedDataForDefragmentation != VMA_NULL)
        6478  {
        6479  m_pBlock->Unmap(hAllocator);
        6480  }
        6481 }
        6482 
        6483 VkResult VmaDefragmentator::DefragmentRound(
        6484  VkDeviceSize maxBytesToMove,
        6485  uint32_t maxAllocationsToMove)
        6486 {
        6487  if(m_Blocks.empty())
        6488  {
        6489  return VK_SUCCESS;
        6490  }
        6491 
        6492  size_t srcBlockIndex = m_Blocks.size() - 1;
        6493  size_t srcAllocIndex = SIZE_MAX;
        6494  for(;;)
        6495  {
        6496  // 1. Find next allocation to move.
        6497  // 1.1. Start from last to first m_Blocks - they are sorted from most "destination" to most "source".
        6498  // 1.2. Then start from last to first m_Allocations - they are sorted from largest to smallest.
        6499  while(srcAllocIndex >= m_Blocks[srcBlockIndex]->m_Allocations.size())
        6500  {
        6501  if(m_Blocks[srcBlockIndex]->m_Allocations.empty())
        6502  {
        6503  // Finished: no more allocations to process.
        6504  if(srcBlockIndex == 0)
        6505  {
        6506  return VK_SUCCESS;
        6507  }
        6508  else
        6509  {
        6510  --srcBlockIndex;
        6511  srcAllocIndex = SIZE_MAX;
        6512  }
        6513  }
        6514  else
        6515  {
        6516  srcAllocIndex = m_Blocks[srcBlockIndex]->m_Allocations.size() - 1;
        6517  }
        6518  }
        6519 
        6520  BlockInfo* pSrcBlockInfo = m_Blocks[srcBlockIndex];
        6521  AllocationInfo& allocInfo = pSrcBlockInfo->m_Allocations[srcAllocIndex];
        6522 
        6523  const VkDeviceSize size = allocInfo.m_hAllocation->GetSize();
        6524  const VkDeviceSize srcOffset = allocInfo.m_hAllocation->GetOffset();
        6525  const VkDeviceSize alignment = allocInfo.m_hAllocation->GetAlignment();
        6526  const VmaSuballocationType suballocType = allocInfo.m_hAllocation->GetSuballocationType();
        6527 
        6528  // 2. Try to find new place for this allocation in preceding or current block.
        6529  for(size_t dstBlockIndex = 0; dstBlockIndex <= srcBlockIndex; ++dstBlockIndex)
        6530  {
        6531  BlockInfo* pDstBlockInfo = m_Blocks[dstBlockIndex];
        6532  VmaAllocationRequest dstAllocRequest;
        6533  if(pDstBlockInfo->m_pBlock->m_Metadata.CreateAllocationRequest(
        6534  m_CurrentFrameIndex,
        6535  m_pBlockVector->GetFrameInUseCount(),
        6536  m_pBlockVector->GetBufferImageGranularity(),
        6537  size,
        6538  alignment,
        6539  suballocType,
        6540  false, // canMakeOtherLost
        6541  &dstAllocRequest) &&
        6542  MoveMakesSense(
        6543  dstBlockIndex, dstAllocRequest.offset, srcBlockIndex, srcOffset))
        6544  {
        6545  VMA_ASSERT(dstAllocRequest.itemsToMakeLostCount == 0);
        6546 
        6547  // Reached limit on number of allocations or bytes to move.
        6548  if((m_AllocationsMoved + 1 > maxAllocationsToMove) ||
        6549  (m_BytesMoved + size > maxBytesToMove))
        6550  {
        6551  return VK_INCOMPLETE;
        6552  }
        6553 
        6554  void* pDstMappedData = VMA_NULL;
        6555  VkResult res = pDstBlockInfo->EnsureMapping(m_hAllocator, &pDstMappedData);
        6556  if(res != VK_SUCCESS)
        6557  {
        6558  return res;
        6559  }
        6560 
        6561  void* pSrcMappedData = VMA_NULL;
        6562  res = pSrcBlockInfo->EnsureMapping(m_hAllocator, &pSrcMappedData);
        6563  if(res != VK_SUCCESS)
        6564  {
        6565  return res;
        6566  }
        6567 
        6568  // THE PLACE WHERE ACTUAL DATA COPY HAPPENS.
        6569  memcpy(
        6570  reinterpret_cast<char*>(pDstMappedData) + dstAllocRequest.offset,
        6571  reinterpret_cast<char*>(pSrcMappedData) + srcOffset,
        6572  static_cast<size_t>(size));
        6573 
        6574  pDstBlockInfo->m_pBlock->m_Metadata.Alloc(dstAllocRequest, suballocType, size, allocInfo.m_hAllocation);
        6575  pSrcBlockInfo->m_pBlock->m_Metadata.Free(allocInfo.m_hAllocation);
        6576 
        6577  allocInfo.m_hAllocation->ChangeBlockAllocation(pDstBlockInfo->m_pBlock, dstAllocRequest.offset);
        6578 
        6579  if(allocInfo.m_pChanged != VMA_NULL)
        6580  {
        6581  *allocInfo.m_pChanged = VK_TRUE;
        6582  }
        6583 
        6584  ++m_AllocationsMoved;
        6585  m_BytesMoved += size;
        6586 
        6587  VmaVectorRemove(pSrcBlockInfo->m_Allocations, srcAllocIndex);
        6588 
        6589  break;
        6590  }
        6591  }
        6592 
        6593  // If not processed, this allocInfo remains in pBlockInfo->m_Allocations for next round.
        6594 
        6595  if(srcAllocIndex > 0)
        6596  {
        6597  --srcAllocIndex;
        6598  }
        6599  else
        6600  {
        6601  if(srcBlockIndex > 0)
        6602  {
        6603  --srcBlockIndex;
        6604  srcAllocIndex = SIZE_MAX;
        6605  }
        6606  else
        6607  {
        6608  return VK_SUCCESS;
        6609  }
        6610  }
        6611  }
        6612 }
        6613 
        6614 VkResult VmaDefragmentator::Defragment(
        6615  VkDeviceSize maxBytesToMove,
        6616  uint32_t maxAllocationsToMove)
        6617 {
        6618  if(m_Allocations.empty())
        6619  {
        6620  return VK_SUCCESS;
        6621  }
        6622 
        6623  // Create block info for each block.
        6624  const size_t blockCount = m_pBlockVector->m_Blocks.size();
        6625  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
        6626  {
        6627  BlockInfo* pBlockInfo = vma_new(m_hAllocator, BlockInfo)(m_hAllocator->GetAllocationCallbacks());
        6628  pBlockInfo->m_pBlock = m_pBlockVector->m_Blocks[blockIndex];
        6629  m_Blocks.push_back(pBlockInfo);
        6630  }
        6631 
        6632  // Sort them by m_pBlock pointer value.
        6633  VMA_SORT(m_Blocks.begin(), m_Blocks.end(), BlockPointerLess());
        6634 
        6635  // Move allocation infos from m_Allocations to appropriate m_Blocks[memTypeIndex].m_Allocations.
        6636  for(size_t blockIndex = 0, allocCount = m_Allocations.size(); blockIndex < allocCount; ++blockIndex)
        6637  {
        6638  AllocationInfo& allocInfo = m_Allocations[blockIndex];
        6639  // Now as we are inside VmaBlockVector::m_Mutex, we can make final check if this allocation was not lost.
        6640  if(allocInfo.m_hAllocation->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST)
        6641  {
        6642  VmaDeviceMemoryBlock* pBlock = allocInfo.m_hAllocation->GetBlock();
        6643  BlockInfoVector::iterator it = VmaBinaryFindFirstNotLess(m_Blocks.begin(), m_Blocks.end(), pBlock, BlockPointerLess());
        6644  if(it != m_Blocks.end() && (*it)->m_pBlock == pBlock)
        6645  {
        6646  (*it)->m_Allocations.push_back(allocInfo);
        6647  }
        6648  else
        6649  {
        6650  VMA_ASSERT(0);
        6651  }
        6652  }
        6653  }
        6654  m_Allocations.clear();
        6655 
        6656  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
        6657  {
        6658  BlockInfo* pBlockInfo = m_Blocks[blockIndex];
        6659  pBlockInfo->CalcHasNonMovableAllocations();
        6660  pBlockInfo->SortAllocationsBySizeDescecnding();
        6661  }
        6662 
        6663  // Sort m_Blocks this time by the main criterium, from most "destination" to most "source" blocks.
        6664  VMA_SORT(m_Blocks.begin(), m_Blocks.end(), BlockInfoCompareMoveDestination());
        6665 
        6666  // Execute defragmentation rounds (the main part).
        6667  VkResult result = VK_SUCCESS;
        6668  for(size_t round = 0; (round < 2) && (result == VK_SUCCESS); ++round)
        6669  {
        6670  result = DefragmentRound(maxBytesToMove, maxAllocationsToMove);
        6671  }
        6672 
        6673  // Unmap blocks that were mapped for defragmentation.
        6674  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
        6675  {
        6676  m_Blocks[blockIndex]->Unmap(m_hAllocator);
        6677  }
        6678 
        6679  return result;
        6680 }
        6681 
        6682 bool VmaDefragmentator::MoveMakesSense(
        6683  size_t dstBlockIndex, VkDeviceSize dstOffset,
        6684  size_t srcBlockIndex, VkDeviceSize srcOffset)
        6685 {
        6686  if(dstBlockIndex < srcBlockIndex)
        6687  {
        6688  return true;
        6689  }
        6690  if(dstBlockIndex > srcBlockIndex)
        6691  {
        6692  return false;
        6693  }
        6694  if(dstOffset < srcOffset)
        6695  {
        6696  return true;
        6697  }
        6698  return false;
        6699 }
        6700 
        6702 // VmaAllocator_T
        6703 
        6704 VmaAllocator_T::VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo) :
        6705  m_UseMutex((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT) == 0),
        6706  m_UseKhrDedicatedAllocation((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT) != 0),
        6707  m_PhysicalDevice(pCreateInfo->physicalDevice),
        6708  m_hDevice(pCreateInfo->device),
        6709  m_AllocationCallbacksSpecified(pCreateInfo->pAllocationCallbacks != VMA_NULL),
        6710  m_AllocationCallbacks(pCreateInfo->pAllocationCallbacks ?
        6711  *pCreateInfo->pAllocationCallbacks : VmaEmptyAllocationCallbacks),
        6712  m_PreferredLargeHeapBlockSize(0),
        6713  m_PreferredSmallHeapBlockSize(0),
        6714  m_CurrentFrameIndex(0),
        6715  m_Pools(VmaStlAllocator<VmaPool>(GetAllocationCallbacks()))
        6716 {
        6717  VMA_ASSERT(pCreateInfo->physicalDevice && pCreateInfo->device);
        6718 
        6719  memset(&m_DeviceMemoryCallbacks, 0 ,sizeof(m_DeviceMemoryCallbacks));
        6720  memset(&m_MemProps, 0, sizeof(m_MemProps));
        6721  memset(&m_PhysicalDeviceProperties, 0, sizeof(m_PhysicalDeviceProperties));
        6722 
        6723  memset(&m_pBlockVectors, 0, sizeof(m_pBlockVectors));
        6724  memset(&m_pDedicatedAllocations, 0, sizeof(m_pDedicatedAllocations));
        6725 
        6726  for(uint32_t i = 0; i < VK_MAX_MEMORY_HEAPS; ++i)
        6727  {
        6728  m_HeapSizeLimit[i] = VK_WHOLE_SIZE;
        6729  }
        6730 
        6731  if(pCreateInfo->pDeviceMemoryCallbacks != VMA_NULL)
        6732  {
        6733  m_DeviceMemoryCallbacks.pfnAllocate = pCreateInfo->pDeviceMemoryCallbacks->pfnAllocate;
        6734  m_DeviceMemoryCallbacks.pfnFree = pCreateInfo->pDeviceMemoryCallbacks->pfnFree;
        6735  }
        6736 
        6737  ImportVulkanFunctions(pCreateInfo->pVulkanFunctions);
        6738 
        6739  (*m_VulkanFunctions.vkGetPhysicalDeviceProperties)(m_PhysicalDevice, &m_PhysicalDeviceProperties);
        6740  (*m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties)(m_PhysicalDevice, &m_MemProps);
        6741 
        6742  m_PreferredLargeHeapBlockSize = (pCreateInfo->preferredLargeHeapBlockSize != 0) ?
        6743  pCreateInfo->preferredLargeHeapBlockSize : static_cast<VkDeviceSize>(VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE);
        6744  m_PreferredSmallHeapBlockSize = (pCreateInfo->preferredSmallHeapBlockSize != 0) ?
        6745  pCreateInfo->preferredSmallHeapBlockSize : static_cast<VkDeviceSize>(VMA_DEFAULT_SMALL_HEAP_BLOCK_SIZE);
        6746 
        6747  if(pCreateInfo->pHeapSizeLimit != VMA_NULL)
        6748  {
        6749  for(uint32_t heapIndex = 0; heapIndex < GetMemoryHeapCount(); ++heapIndex)
        6750  {
        6751  const VkDeviceSize limit = pCreateInfo->pHeapSizeLimit[heapIndex];
        6752  if(limit != VK_WHOLE_SIZE)
        6753  {
        6754  m_HeapSizeLimit[heapIndex] = limit;
        6755  if(limit < m_MemProps.memoryHeaps[heapIndex].size)
        6756  {
        6757  m_MemProps.memoryHeaps[heapIndex].size = limit;
        6758  }
        6759  }
        6760  }
        6761  }
        6762 
        6763  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
        6764  {
        6765  const VkDeviceSize preferredBlockSize = CalcPreferredBlockSize(memTypeIndex);
        6766 
        6767  m_pBlockVectors[memTypeIndex] = vma_new(this, VmaBlockVector)(
        6768  this,
        6769  memTypeIndex,
        6770  preferredBlockSize,
        6771  0,
        6772  SIZE_MAX,
        6773  GetBufferImageGranularity(),
        6774  pCreateInfo->frameInUseCount,
        6775  false); // isCustomPool
        6776  // No need to call m_pBlockVectors[memTypeIndex][blockVectorTypeIndex]->CreateMinBlocks here,
        6777  // becase minBlockCount is 0.
        6778  m_pDedicatedAllocations[memTypeIndex] = vma_new(this, AllocationVectorType)(VmaStlAllocator<VmaAllocation>(GetAllocationCallbacks()));
        6779  }
        6780 }
        6781 
        6782 VmaAllocator_T::~VmaAllocator_T()
        6783 {
        6784  VMA_ASSERT(m_Pools.empty());
        6785 
        6786  for(size_t i = GetMemoryTypeCount(); i--; )
        6787  {
        6788  vma_delete(this, m_pDedicatedAllocations[i]);
        6789  vma_delete(this, m_pBlockVectors[i]);
        6790  }
        6791 }
        6792 
        6793 void VmaAllocator_T::ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions)
        6794 {
        6795 #if VMA_STATIC_VULKAN_FUNCTIONS == 1
        6796  m_VulkanFunctions.vkGetPhysicalDeviceProperties = &vkGetPhysicalDeviceProperties;
        6797  m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties = &vkGetPhysicalDeviceMemoryProperties;
        6798  m_VulkanFunctions.vkAllocateMemory = &vkAllocateMemory;
        6799  m_VulkanFunctions.vkFreeMemory = &vkFreeMemory;
        6800  m_VulkanFunctions.vkMapMemory = &vkMapMemory;
        6801  m_VulkanFunctions.vkUnmapMemory = &vkUnmapMemory;
        6802  m_VulkanFunctions.vkBindBufferMemory = &vkBindBufferMemory;
        6803  m_VulkanFunctions.vkBindImageMemory = &vkBindImageMemory;
        6804  m_VulkanFunctions.vkGetBufferMemoryRequirements = &vkGetBufferMemoryRequirements;
        6805  m_VulkanFunctions.vkGetImageMemoryRequirements = &vkGetImageMemoryRequirements;
        6806  m_VulkanFunctions.vkCreateBuffer = &vkCreateBuffer;
        6807  m_VulkanFunctions.vkDestroyBuffer = &vkDestroyBuffer;
        6808  m_VulkanFunctions.vkCreateImage = &vkCreateImage;
        6809  m_VulkanFunctions.vkDestroyImage = &vkDestroyImage;
        6810  if(m_UseKhrDedicatedAllocation)
        6811  {
        6812  m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR =
        6813  (PFN_vkGetBufferMemoryRequirements2KHR)vkGetDeviceProcAddr(m_hDevice, "vkGetBufferMemoryRequirements2KHR");
        6814  m_VulkanFunctions.vkGetImageMemoryRequirements2KHR =
        6815  (PFN_vkGetImageMemoryRequirements2KHR)vkGetDeviceProcAddr(m_hDevice, "vkGetImageMemoryRequirements2KHR");
        6816  }
        6817 #endif // #if VMA_STATIC_VULKAN_FUNCTIONS == 1
        6818 
        6819 #define VMA_COPY_IF_NOT_NULL(funcName) \
        6820  if(pVulkanFunctions->funcName != VMA_NULL) m_VulkanFunctions.funcName = pVulkanFunctions->funcName;
        6821 
        6822  if(pVulkanFunctions != VMA_NULL)
        6823  {
        6824  VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceProperties);
        6825  VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceMemoryProperties);
        6826  VMA_COPY_IF_NOT_NULL(vkAllocateMemory);
        6827  VMA_COPY_IF_NOT_NULL(vkFreeMemory);
        6828  VMA_COPY_IF_NOT_NULL(vkMapMemory);
        6829  VMA_COPY_IF_NOT_NULL(vkUnmapMemory);
        6830  VMA_COPY_IF_NOT_NULL(vkBindBufferMemory);
        6831  VMA_COPY_IF_NOT_NULL(vkBindImageMemory);
        6832  VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements);
        6833  VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements);
        6834  VMA_COPY_IF_NOT_NULL(vkCreateBuffer);
        6835  VMA_COPY_IF_NOT_NULL(vkDestroyBuffer);
        6836  VMA_COPY_IF_NOT_NULL(vkCreateImage);
        6837  VMA_COPY_IF_NOT_NULL(vkDestroyImage);
        6838  VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements2KHR);
        6839  VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements2KHR);
        6840  }
        6841 
        6842 #undef VMA_COPY_IF_NOT_NULL
        6843 
        6844  // If these asserts are hit, you must either #define VMA_STATIC_VULKAN_FUNCTIONS 1
        6845  // or pass valid pointers as VmaAllocatorCreateInfo::pVulkanFunctions.
        6846  VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceProperties != VMA_NULL);
        6847  VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties != VMA_NULL);
        6848  VMA_ASSERT(m_VulkanFunctions.vkAllocateMemory != VMA_NULL);
        6849  VMA_ASSERT(m_VulkanFunctions.vkFreeMemory != VMA_NULL);
        6850  VMA_ASSERT(m_VulkanFunctions.vkMapMemory != VMA_NULL);
        6851  VMA_ASSERT(m_VulkanFunctions.vkUnmapMemory != VMA_NULL);
        6852  VMA_ASSERT(m_VulkanFunctions.vkBindBufferMemory != VMA_NULL);
        6853  VMA_ASSERT(m_VulkanFunctions.vkBindImageMemory != VMA_NULL);
        6854  VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements != VMA_NULL);
        6855  VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements != VMA_NULL);
        6856  VMA_ASSERT(m_VulkanFunctions.vkCreateBuffer != VMA_NULL);
        6857  VMA_ASSERT(m_VulkanFunctions.vkDestroyBuffer != VMA_NULL);
        6858  VMA_ASSERT(m_VulkanFunctions.vkCreateImage != VMA_NULL);
        6859  VMA_ASSERT(m_VulkanFunctions.vkDestroyImage != VMA_NULL);
        6860  if(m_UseKhrDedicatedAllocation)
        6861  {
        6862  VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR != VMA_NULL);
        6863  VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements2KHR != VMA_NULL);
        6864  }
        6865 }
        6866 
        6867 VkDeviceSize VmaAllocator_T::CalcPreferredBlockSize(uint32_t memTypeIndex)
        6868 {
        6869  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
        6870  const VkDeviceSize heapSize = m_MemProps.memoryHeaps[heapIndex].size;
        6871  return (heapSize <= VMA_SMALL_HEAP_MAX_SIZE) ?
        6872  m_PreferredSmallHeapBlockSize : m_PreferredLargeHeapBlockSize;
        6873 }
        6874 
        6875 VkResult VmaAllocator_T::AllocateMemoryOfType(
        6876  const VkMemoryRequirements& vkMemReq,
        6877  bool dedicatedAllocation,
        6878  VkBuffer dedicatedBuffer,
        6879  VkImage dedicatedImage,
        6880  const VmaAllocationCreateInfo& createInfo,
        6881  uint32_t memTypeIndex,
        6882  VmaSuballocationType suballocType,
        6883  VmaAllocation* pAllocation)
        6884 {
        6885  VMA_ASSERT(pAllocation != VMA_NULL);
        6886  VMA_DEBUG_LOG(" AllocateMemory: MemoryTypeIndex=%u, Size=%llu", memTypeIndex, vkMemReq.size);
        6887 
        6888  VmaAllocationCreateInfo finalCreateInfo = createInfo;
        6889 
        6890  // If memory type is not HOST_VISIBLE, disable MAPPED.
        6891  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0 &&
        6892  (m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) == 0)
        6893  {
        6894  finalCreateInfo.flags &= ~VMA_ALLOCATION_CREATE_MAPPED_BIT;
        6895  }
        6896 
        6897  VmaBlockVector* const blockVector = m_pBlockVectors[memTypeIndex];
        6898  VMA_ASSERT(blockVector);
        6899 
        6900  const VkDeviceSize preferredBlockSize = blockVector->GetPreferredBlockSize();
        6901  bool preferDedicatedMemory =
        6902  VMA_DEBUG_ALWAYS_DEDICATED_MEMORY ||
        6903  dedicatedAllocation ||
        6904  // Heuristics: Allocate dedicated memory if requested size if greater than half of preferred block size.
        6905  vkMemReq.size > preferredBlockSize / 2;
        6906 
        6907  if(preferDedicatedMemory &&
        6908  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == 0 &&
        6909  finalCreateInfo.pool == VK_NULL_HANDLE)
        6910  {
        6911  finalCreateInfo.flags |= VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
        6912  }
        6913 
        6914  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0)
        6915  {
        6916  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
        6917  {
        6918  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        6919  }
        6920  else
        6921  {
        6922  return AllocateDedicatedMemory(
        6923  vkMemReq.size,
        6924  suballocType,
        6925  memTypeIndex,
        6926  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0,
        6927  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0,
        6928  finalCreateInfo.pUserData,
        6929  dedicatedBuffer,
        6930  dedicatedImage,
        6931  pAllocation);
        6932  }
        6933  }
        6934  else
        6935  {
        6936  VkResult res = blockVector->Allocate(
        6937  VK_NULL_HANDLE, // hCurrentPool
        6938  m_CurrentFrameIndex.load(),
        6939  vkMemReq,
        6940  finalCreateInfo,
        6941  suballocType,
        6942  pAllocation);
        6943  if(res == VK_SUCCESS)
        6944  {
        6945  return res;
        6946  }
        6947 
        6948  // 5. Try dedicated memory.
        6949  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
        6950  {
        6951  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        6952  }
        6953  else
        6954  {
        6955  res = AllocateDedicatedMemory(
        6956  vkMemReq.size,
        6957  suballocType,
        6958  memTypeIndex,
        6959  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0,
        6960  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0,
        6961  finalCreateInfo.pUserData,
        6962  dedicatedBuffer,
        6963  dedicatedImage,
        6964  pAllocation);
        6965  if(res == VK_SUCCESS)
        6966  {
        6967  // Succeeded: AllocateDedicatedMemory function already filld pMemory, nothing more to do here.
        6968  VMA_DEBUG_LOG(" Allocated as DedicatedMemory");
        6969  return VK_SUCCESS;
        6970  }
        6971  else
        6972  {
        6973  // Everything failed: Return error code.
        6974  VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
        6975  return res;
        6976  }
        6977  }
        6978  }
        6979 }
        6980 
        6981 VkResult VmaAllocator_T::AllocateDedicatedMemory(
        6982  VkDeviceSize size,
        6983  VmaSuballocationType suballocType,
        6984  uint32_t memTypeIndex,
        6985  bool map,
        6986  bool isUserDataString,
        6987  void* pUserData,
        6988  VkBuffer dedicatedBuffer,
        6989  VkImage dedicatedImage,
        6990  VmaAllocation* pAllocation)
        6991 {
        6992  VMA_ASSERT(pAllocation);
        6993 
        6994  VkMemoryAllocateInfo allocInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
        6995  allocInfo.memoryTypeIndex = memTypeIndex;
        6996  allocInfo.allocationSize = size;
        6997 
        6998  VkMemoryDedicatedAllocateInfoKHR dedicatedAllocInfo = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO_KHR };
        6999  if(m_UseKhrDedicatedAllocation)
        7000  {
        7001  if(dedicatedBuffer != VK_NULL_HANDLE)
        7002  {
        7003  VMA_ASSERT(dedicatedImage == VK_NULL_HANDLE);
        7004  dedicatedAllocInfo.buffer = dedicatedBuffer;
        7005  allocInfo.pNext = &dedicatedAllocInfo;
        7006  }
        7007  else if(dedicatedImage != VK_NULL_HANDLE)
        7008  {
        7009  dedicatedAllocInfo.image = dedicatedImage;
        7010  allocInfo.pNext = &dedicatedAllocInfo;
        7011  }
        7012  }
        7013 
        7014  // Allocate VkDeviceMemory.
        7015  VkDeviceMemory hMemory = VK_NULL_HANDLE;
        7016  VkResult res = AllocateVulkanMemory(&allocInfo, &hMemory);
        7017  if(res < 0)
        7018  {
        7019  VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
        7020  return res;
        7021  }
        7022 
        7023  void* pMappedData = nullptr;
        7024  if(map)
        7025  {
        7026  res = (*m_VulkanFunctions.vkMapMemory)(
        7027  m_hDevice,
        7028  hMemory,
        7029  0,
        7030  VK_WHOLE_SIZE,
        7031  0,
        7032  &pMappedData);
        7033  if(res < 0)
        7034  {
        7035  VMA_DEBUG_LOG(" vkMapMemory FAILED");
        7036  FreeVulkanMemory(memTypeIndex, size, hMemory);
        7037  return res;
        7038  }
        7039  }
        7040 
        7041  *pAllocation = vma_new(this, VmaAllocation_T)(m_CurrentFrameIndex.load(), isUserDataString);
        7042  (*pAllocation)->InitDedicatedAllocation(memTypeIndex, hMemory, suballocType, pMappedData, size);
        7043  (*pAllocation)->SetUserData(this, pUserData);
        7044 
        7045  // Register it in m_pDedicatedAllocations.
        7046  {
        7047  VmaMutexLock lock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
        7048  AllocationVectorType* pDedicatedAllocations = m_pDedicatedAllocations[memTypeIndex];
        7049  VMA_ASSERT(pDedicatedAllocations);
        7050  VmaVectorInsertSorted<VmaPointerLess>(*pDedicatedAllocations, *pAllocation);
        7051  }
        7052 
        7053  VMA_DEBUG_LOG(" Allocated DedicatedMemory MemoryTypeIndex=#%u", memTypeIndex);
        7054 
        7055  return VK_SUCCESS;
        7056 }
        7057 
        7058 void VmaAllocator_T::GetBufferMemoryRequirements(
        7059  VkBuffer hBuffer,
        7060  VkMemoryRequirements& memReq,
        7061  bool& requiresDedicatedAllocation,
        7062  bool& prefersDedicatedAllocation) const
        7063 {
        7064  if(m_UseKhrDedicatedAllocation)
        7065  {
        7066  VkBufferMemoryRequirementsInfo2KHR memReqInfo = { VK_STRUCTURE_TYPE_BUFFER_MEMORY_REQUIREMENTS_INFO_2_KHR };
        7067  memReqInfo.buffer = hBuffer;
        7068 
        7069  VkMemoryDedicatedRequirementsKHR memDedicatedReq = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
        7070 
        7071  VkMemoryRequirements2KHR memReq2 = { VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
        7072  memReq2.pNext = &memDedicatedReq;
        7073 
        7074  (*m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
        7075 
        7076  memReq = memReq2.memoryRequirements;
        7077  requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
        7078  prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
        7079  }
        7080  else
        7081  {
        7082  (*m_VulkanFunctions.vkGetBufferMemoryRequirements)(m_hDevice, hBuffer, &memReq);
        7083  requiresDedicatedAllocation = false;
        7084  prefersDedicatedAllocation = false;
        7085  }
        7086 }
        7087 
        7088 void VmaAllocator_T::GetImageMemoryRequirements(
        7089  VkImage hImage,
        7090  VkMemoryRequirements& memReq,
        7091  bool& requiresDedicatedAllocation,
        7092  bool& prefersDedicatedAllocation) const
        7093 {
        7094  if(m_UseKhrDedicatedAllocation)
        7095  {
        7096  VkImageMemoryRequirementsInfo2KHR memReqInfo = { VK_STRUCTURE_TYPE_IMAGE_MEMORY_REQUIREMENTS_INFO_2_KHR };
        7097  memReqInfo.image = hImage;
        7098 
        7099  VkMemoryDedicatedRequirementsKHR memDedicatedReq = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
        7100 
        7101  VkMemoryRequirements2KHR memReq2 = { VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
        7102  memReq2.pNext = &memDedicatedReq;
        7103 
        7104  (*m_VulkanFunctions.vkGetImageMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
        7105 
        7106  memReq = memReq2.memoryRequirements;
        7107  requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
        7108  prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
        7109  }
        7110  else
        7111  {
        7112  (*m_VulkanFunctions.vkGetImageMemoryRequirements)(m_hDevice, hImage, &memReq);
        7113  requiresDedicatedAllocation = false;
        7114  prefersDedicatedAllocation = false;
        7115  }
        7116 }
        7117 
        7118 VkResult VmaAllocator_T::AllocateMemory(
        7119  const VkMemoryRequirements& vkMemReq,
        7120  bool requiresDedicatedAllocation,
        7121  bool prefersDedicatedAllocation,
        7122  VkBuffer dedicatedBuffer,
        7123  VkImage dedicatedImage,
        7124  const VmaAllocationCreateInfo& createInfo,
        7125  VmaSuballocationType suballocType,
        7126  VmaAllocation* pAllocation)
        7127 {
        7128  if((createInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0 &&
        7129  (createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
        7130  {
        7131  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT together with VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT makes no sense.");
        7132  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7133  }
        7134  if((createInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0 &&
        7135  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0)
        7136  {
        7137  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_MAPPED_BIT together with VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT is invalid.");
        7138  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7139  }
        7140  if(requiresDedicatedAllocation)
        7141  {
        7142  if((createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
        7143  {
        7144  VMA_ASSERT(0 && "VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT specified while dedicated allocation is required.");
        7145  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7146  }
        7147  if(createInfo.pool != VK_NULL_HANDLE)
        7148  {
        7149  VMA_ASSERT(0 && "Pool specified while dedicated allocation is required.");
        7150  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7151  }
        7152  }
        7153  if((createInfo.pool != VK_NULL_HANDLE) &&
        7154  ((createInfo.flags & (VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT)) != 0))
        7155  {
        7156  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT when pool != null is invalid.");
        7157  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7158  }
        7159 
        7160  if(createInfo.pool != VK_NULL_HANDLE)
        7161  {
        7162  return createInfo.pool->m_BlockVector.Allocate(
        7163  createInfo.pool,
        7164  m_CurrentFrameIndex.load(),
        7165  vkMemReq,
        7166  createInfo,
        7167  suballocType,
        7168  pAllocation);
        7169  }
        7170  else
        7171  {
        7172  // Bit mask of memory Vulkan types acceptable for this allocation.
        7173  uint32_t memoryTypeBits = vkMemReq.memoryTypeBits;
        7174  uint32_t memTypeIndex = UINT32_MAX;
        7175  VkResult res = vmaFindMemoryTypeIndex(this, memoryTypeBits, &createInfo, &memTypeIndex);
        7176  if(res == VK_SUCCESS)
        7177  {
        7178  res = AllocateMemoryOfType(
        7179  vkMemReq,
        7180  requiresDedicatedAllocation || prefersDedicatedAllocation,
        7181  dedicatedBuffer,
        7182  dedicatedImage,
        7183  createInfo,
        7184  memTypeIndex,
        7185  suballocType,
        7186  pAllocation);
        7187  // Succeeded on first try.
        7188  if(res == VK_SUCCESS)
        7189  {
        7190  return res;
        7191  }
        7192  // Allocation from this memory type failed. Try other compatible memory types.
        7193  else
        7194  {
        7195  for(;;)
        7196  {
        7197  // Remove old memTypeIndex from list of possibilities.
        7198  memoryTypeBits &= ~(1u << memTypeIndex);
        7199  // Find alternative memTypeIndex.
        7200  res = vmaFindMemoryTypeIndex(this, memoryTypeBits, &createInfo, &memTypeIndex);
        7201  if(res == VK_SUCCESS)
        7202  {
        7203  res = AllocateMemoryOfType(
        7204  vkMemReq,
        7205  requiresDedicatedAllocation || prefersDedicatedAllocation,
        7206  dedicatedBuffer,
        7207  dedicatedImage,
        7208  createInfo,
        7209  memTypeIndex,
        7210  suballocType,
        7211  pAllocation);
        7212  // Allocation from this alternative memory type succeeded.
        7213  if(res == VK_SUCCESS)
        7214  {
        7215  return res;
        7216  }
        7217  // else: Allocation from this memory type failed. Try next one - next loop iteration.
        7218  }
        7219  // No other matching memory type index could be found.
        7220  else
        7221  {
        7222  // Not returning res, which is VK_ERROR_FEATURE_NOT_PRESENT, because we already failed to allocate once.
        7223  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7224  }
        7225  }
        7226  }
        7227  }
        7228  // Can't find any single memory type maching requirements. res is VK_ERROR_FEATURE_NOT_PRESENT.
        7229  else
        7230  return res;
        7231  }
        7232 }
        7233 
        7234 void VmaAllocator_T::FreeMemory(const VmaAllocation allocation)
        7235 {
        7236  VMA_ASSERT(allocation);
        7237 
        7238  if(allocation->CanBecomeLost() == false ||
        7239  allocation->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST)
        7240  {
        7241  switch(allocation->GetType())
        7242  {
        7243  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
        7244  {
        7245  VmaBlockVector* pBlockVector = VMA_NULL;
        7246  VmaPool hPool = allocation->GetPool();
        7247  if(hPool != VK_NULL_HANDLE)
        7248  {
        7249  pBlockVector = &hPool->m_BlockVector;
        7250  }
        7251  else
        7252  {
        7253  const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
        7254  pBlockVector = m_pBlockVectors[memTypeIndex];
        7255  }
        7256  pBlockVector->Free(allocation);
        7257  }
        7258  break;
        7259  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
        7260  FreeDedicatedMemory(allocation);
        7261  break;
        7262  default:
        7263  VMA_ASSERT(0);
        7264  }
        7265  }
        7266 
        7267  allocation->SetUserData(this, VMA_NULL);
        7268  vma_delete(this, allocation);
        7269 }
        7270 
        7271 void VmaAllocator_T::CalculateStats(VmaStats* pStats)
        7272 {
        7273  // Initialize.
        7274  InitStatInfo(pStats->total);
        7275  for(size_t i = 0; i < VK_MAX_MEMORY_TYPES; ++i)
        7276  InitStatInfo(pStats->memoryType[i]);
        7277  for(size_t i = 0; i < VK_MAX_MEMORY_HEAPS; ++i)
        7278  InitStatInfo(pStats->memoryHeap[i]);
        7279 
        7280  // Process default pools.
        7281  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
        7282  {
        7283  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
        7284  VmaBlockVector* const pBlockVector = m_pBlockVectors[memTypeIndex];
        7285  VMA_ASSERT(pBlockVector);
        7286  pBlockVector->AddStats(pStats);
        7287  }
        7288 
        7289  // Process custom pools.
        7290  {
        7291  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
        7292  for(size_t poolIndex = 0, poolCount = m_Pools.size(); poolIndex < poolCount; ++poolIndex)
        7293  {
        7294  m_Pools[poolIndex]->GetBlockVector().AddStats(pStats);
        7295  }
        7296  }
        7297 
        7298  // Process dedicated allocations.
        7299  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
        7300  {
        7301  const uint32_t memHeapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
        7302  VmaMutexLock dedicatedAllocationsLock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
        7303  AllocationVectorType* const pDedicatedAllocVector = m_pDedicatedAllocations[memTypeIndex];
        7304  VMA_ASSERT(pDedicatedAllocVector);
        7305  for(size_t allocIndex = 0, allocCount = pDedicatedAllocVector->size(); allocIndex < allocCount; ++allocIndex)
        7306  {
        7307  VmaStatInfo allocationStatInfo;
        7308  (*pDedicatedAllocVector)[allocIndex]->DedicatedAllocCalcStatsInfo(allocationStatInfo);
        7309  VmaAddStatInfo(pStats->total, allocationStatInfo);
        7310  VmaAddStatInfo(pStats->memoryType[memTypeIndex], allocationStatInfo);
        7311  VmaAddStatInfo(pStats->memoryHeap[memHeapIndex], allocationStatInfo);
        7312  }
        7313  }
        7314 
        7315  // Postprocess.
        7316  VmaPostprocessCalcStatInfo(pStats->total);
        7317  for(size_t i = 0; i < GetMemoryTypeCount(); ++i)
        7318  VmaPostprocessCalcStatInfo(pStats->memoryType[i]);
        7319  for(size_t i = 0; i < GetMemoryHeapCount(); ++i)
        7320  VmaPostprocessCalcStatInfo(pStats->memoryHeap[i]);
        7321 }
        7322 
        7323 static const uint32_t VMA_VENDOR_ID_AMD = 4098;
        7324 
        7325 VkResult VmaAllocator_T::Defragment(
        7326  VmaAllocation* pAllocations,
        7327  size_t allocationCount,
        7328  VkBool32* pAllocationsChanged,
        7329  const VmaDefragmentationInfo* pDefragmentationInfo,
        7330  VmaDefragmentationStats* pDefragmentationStats)
        7331 {
        7332  if(pAllocationsChanged != VMA_NULL)
        7333  {
        7334  memset(pAllocationsChanged, 0, sizeof(*pAllocationsChanged));
        7335  }
        7336  if(pDefragmentationStats != VMA_NULL)
        7337  {
        7338  memset(pDefragmentationStats, 0, sizeof(*pDefragmentationStats));
        7339  }
        7340 
        7341  const uint32_t currentFrameIndex = m_CurrentFrameIndex.load();
        7342 
        7343  VmaMutexLock poolsLock(m_PoolsMutex, m_UseMutex);
        7344 
        7345  const size_t poolCount = m_Pools.size();
        7346 
        7347  // Dispatch pAllocations among defragmentators. Create them in BlockVectors when necessary.
        7348  for(size_t allocIndex = 0; allocIndex < allocationCount; ++allocIndex)
        7349  {
        7350  VmaAllocation hAlloc = pAllocations[allocIndex];
        7351  VMA_ASSERT(hAlloc);
        7352  const uint32_t memTypeIndex = hAlloc->GetMemoryTypeIndex();
        7353  // DedicatedAlloc cannot be defragmented.
        7354  if((hAlloc->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK) &&
        7355  // Only HOST_VISIBLE memory types can be defragmented.
        7356  ((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0) &&
        7357  // Lost allocation cannot be defragmented.
        7358  (hAlloc->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST))
        7359  {
        7360  VmaBlockVector* pAllocBlockVector = nullptr;
        7361 
        7362  const VmaPool hAllocPool = hAlloc->GetPool();
        7363  // This allocation belongs to custom pool.
        7364  if(hAllocPool != VK_NULL_HANDLE)
        7365  {
        7366  pAllocBlockVector = &hAllocPool->GetBlockVector();
        7367  }
        7368  // This allocation belongs to general pool.
        7369  else
        7370  {
        7371  pAllocBlockVector = m_pBlockVectors[memTypeIndex];
        7372  }
        7373 
        7374  VmaDefragmentator* const pDefragmentator = pAllocBlockVector->EnsureDefragmentator(this, currentFrameIndex);
        7375 
        7376  VkBool32* const pChanged = (pAllocationsChanged != VMA_NULL) ?
        7377  &pAllocationsChanged[allocIndex] : VMA_NULL;
        7378  pDefragmentator->AddAllocation(hAlloc, pChanged);
        7379  }
        7380  }
        7381 
        7382  VkResult result = VK_SUCCESS;
        7383 
        7384  // ======== Main processing.
        7385 
        7386  VkDeviceSize maxBytesToMove = SIZE_MAX;
        7387  uint32_t maxAllocationsToMove = UINT32_MAX;
        7388  if(pDefragmentationInfo != VMA_NULL)
        7389  {
        7390  maxBytesToMove = pDefragmentationInfo->maxBytesToMove;
        7391  maxAllocationsToMove = pDefragmentationInfo->maxAllocationsToMove;
        7392  }
        7393 
        7394  // Process standard memory.
        7395  for(uint32_t memTypeIndex = 0;
        7396  (memTypeIndex < GetMemoryTypeCount()) && (result == VK_SUCCESS);
        7397  ++memTypeIndex)
        7398  {
        7399  // Only HOST_VISIBLE memory types can be defragmented.
        7400  if((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
        7401  {
        7402  result = m_pBlockVectors[memTypeIndex]->Defragment(
        7403  pDefragmentationStats,
        7404  maxBytesToMove,
        7405  maxAllocationsToMove);
        7406  }
        7407  }
        7408 
        7409  // Process custom pools.
        7410  for(size_t poolIndex = 0; (poolIndex < poolCount) && (result == VK_SUCCESS); ++poolIndex)
        7411  {
        7412  result = m_Pools[poolIndex]->GetBlockVector().Defragment(
        7413  pDefragmentationStats,
        7414  maxBytesToMove,
        7415  maxAllocationsToMove);
        7416  }
        7417 
        7418  // ======== Destroy defragmentators.
        7419 
        7420  // Process custom pools.
        7421  for(size_t poolIndex = poolCount; poolIndex--; )
        7422  {
        7423  m_Pools[poolIndex]->GetBlockVector().DestroyDefragmentator();
        7424  }
        7425 
        7426  // Process standard memory.
        7427  for(uint32_t memTypeIndex = GetMemoryTypeCount(); memTypeIndex--; )
        7428  {
        7429  if((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
        7430  {
        7431  m_pBlockVectors[memTypeIndex]->DestroyDefragmentator();
        7432  }
        7433  }
        7434 
        7435  return result;
        7436 }
        7437 
        7438 void VmaAllocator_T::GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo)
        7439 {
        7440  if(hAllocation->CanBecomeLost())
        7441  {
        7442  /*
        7443  Warning: This is a carefully designed algorithm.
        7444  Do not modify unless you really know what you're doing :)
        7445  */
        7446  uint32_t localCurrFrameIndex = m_CurrentFrameIndex.load();
        7447  uint32_t localLastUseFrameIndex = hAllocation->GetLastUseFrameIndex();
        7448  for(;;)
        7449  {
        7450  if(localLastUseFrameIndex == VMA_FRAME_INDEX_LOST)
        7451  {
        7452  pAllocationInfo->memoryType = UINT32_MAX;
        7453  pAllocationInfo->deviceMemory = VK_NULL_HANDLE;
        7454  pAllocationInfo->offset = 0;
        7455  pAllocationInfo->size = hAllocation->GetSize();
        7456  pAllocationInfo->pMappedData = VMA_NULL;
        7457  pAllocationInfo->pUserData = hAllocation->GetUserData();
        7458  return;
        7459  }
        7460  else if(localLastUseFrameIndex == localCurrFrameIndex)
        7461  {
        7462  pAllocationInfo->memoryType = hAllocation->GetMemoryTypeIndex();
        7463  pAllocationInfo->deviceMemory = hAllocation->GetMemory();
        7464  pAllocationInfo->offset = hAllocation->GetOffset();
        7465  pAllocationInfo->size = hAllocation->GetSize();
        7466  pAllocationInfo->pMappedData = VMA_NULL;
        7467  pAllocationInfo->pUserData = hAllocation->GetUserData();
        7468  return;
        7469  }
        7470  else // Last use time earlier than current time.
        7471  {
        7472  if(hAllocation->CompareExchangeLastUseFrameIndex(localLastUseFrameIndex, localCurrFrameIndex))
        7473  {
        7474  localLastUseFrameIndex = localCurrFrameIndex;
        7475  }
        7476  }
        7477  }
        7478  }
        7479  else
        7480  {
        7481  pAllocationInfo->memoryType = hAllocation->GetMemoryTypeIndex();
        7482  pAllocationInfo->deviceMemory = hAllocation->GetMemory();
        7483  pAllocationInfo->offset = hAllocation->GetOffset();
        7484  pAllocationInfo->size = hAllocation->GetSize();
        7485  pAllocationInfo->pMappedData = hAllocation->GetMappedData();
        7486  pAllocationInfo->pUserData = hAllocation->GetUserData();
        7487  }
        7488 }
        7489 
        7490 VkResult VmaAllocator_T::CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool)
        7491 {
        7492  VMA_DEBUG_LOG(" CreatePool: MemoryTypeIndex=%u", pCreateInfo->memoryTypeIndex);
        7493 
        7494  VmaPoolCreateInfo newCreateInfo = *pCreateInfo;
        7495 
        7496  if(newCreateInfo.maxBlockCount == 0)
        7497  {
        7498  newCreateInfo.maxBlockCount = SIZE_MAX;
        7499  }
        7500  if(newCreateInfo.blockSize == 0)
        7501  {
        7502  newCreateInfo.blockSize = CalcPreferredBlockSize(newCreateInfo.memoryTypeIndex);
        7503  }
        7504 
        7505  *pPool = vma_new(this, VmaPool_T)(this, newCreateInfo);
        7506 
        7507  VkResult res = (*pPool)->m_BlockVector.CreateMinBlocks();
        7508  if(res != VK_SUCCESS)
        7509  {
        7510  vma_delete(this, *pPool);
        7511  *pPool = VMA_NULL;
        7512  return res;
        7513  }
        7514 
        7515  // Add to m_Pools.
        7516  {
        7517  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
        7518  VmaVectorInsertSorted<VmaPointerLess>(m_Pools, *pPool);
        7519  }
        7520 
        7521  return VK_SUCCESS;
        7522 }
        7523 
        7524 void VmaAllocator_T::DestroyPool(VmaPool pool)
        7525 {
        7526  // Remove from m_Pools.
        7527  {
        7528  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
        7529  bool success = VmaVectorRemoveSorted<VmaPointerLess>(m_Pools, pool);
        7530  VMA_ASSERT(success && "Pool not found in Allocator.");
        7531  }
        7532 
        7533  vma_delete(this, pool);
        7534 }
        7535 
        7536 void VmaAllocator_T::GetPoolStats(VmaPool pool, VmaPoolStats* pPoolStats)
        7537 {
        7538  pool->m_BlockVector.GetPoolStats(pPoolStats);
        7539 }
        7540 
        7541 void VmaAllocator_T::SetCurrentFrameIndex(uint32_t frameIndex)
        7542 {
        7543  m_CurrentFrameIndex.store(frameIndex);
        7544 }
        7545 
        7546 void VmaAllocator_T::MakePoolAllocationsLost(
        7547  VmaPool hPool,
        7548  size_t* pLostAllocationCount)
        7549 {
        7550  hPool->m_BlockVector.MakePoolAllocationsLost(
        7551  m_CurrentFrameIndex.load(),
        7552  pLostAllocationCount);
        7553 }
        7554 
        7555 void VmaAllocator_T::CreateLostAllocation(VmaAllocation* pAllocation)
        7556 {
        7557  *pAllocation = vma_new(this, VmaAllocation_T)(VMA_FRAME_INDEX_LOST, false);
        7558  (*pAllocation)->InitLost();
        7559 }
        7560 
        7561 VkResult VmaAllocator_T::AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory)
        7562 {
        7563  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(pAllocateInfo->memoryTypeIndex);
        7564 
        7565  VkResult res;
        7566  if(m_HeapSizeLimit[heapIndex] != VK_WHOLE_SIZE)
        7567  {
        7568  VmaMutexLock lock(m_HeapSizeLimitMutex, m_UseMutex);
        7569  if(m_HeapSizeLimit[heapIndex] >= pAllocateInfo->allocationSize)
        7570  {
        7571  res = (*m_VulkanFunctions.vkAllocateMemory)(m_hDevice, pAllocateInfo, GetAllocationCallbacks(), pMemory);
        7572  if(res == VK_SUCCESS)
        7573  {
        7574  m_HeapSizeLimit[heapIndex] -= pAllocateInfo->allocationSize;
        7575  }
        7576  }
        7577  else
        7578  {
        7579  res = VK_ERROR_OUT_OF_DEVICE_MEMORY;
        7580  }
        7581  }
        7582  else
        7583  {
        7584  res = (*m_VulkanFunctions.vkAllocateMemory)(m_hDevice, pAllocateInfo, GetAllocationCallbacks(), pMemory);
        7585  }
        7586 
        7587  if(res == VK_SUCCESS && m_DeviceMemoryCallbacks.pfnAllocate != VMA_NULL)
        7588  {
        7589  (*m_DeviceMemoryCallbacks.pfnAllocate)(this, pAllocateInfo->memoryTypeIndex, *pMemory, pAllocateInfo->allocationSize);
        7590  }
        7591 
        7592  return res;
        7593 }
        7594 
        7595 void VmaAllocator_T::FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory)
        7596 {
        7597  if(m_DeviceMemoryCallbacks.pfnFree != VMA_NULL)
        7598  {
        7599  (*m_DeviceMemoryCallbacks.pfnFree)(this, memoryType, hMemory, size);
        7600  }
        7601 
        7602  (*m_VulkanFunctions.vkFreeMemory)(m_hDevice, hMemory, GetAllocationCallbacks());
        7603 
        7604  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memoryType);
        7605  if(m_HeapSizeLimit[heapIndex] != VK_WHOLE_SIZE)
        7606  {
        7607  VmaMutexLock lock(m_HeapSizeLimitMutex, m_UseMutex);
        7608  m_HeapSizeLimit[heapIndex] += size;
        7609  }
        7610 }
        7611 
        7612 VkResult VmaAllocator_T::Map(VmaAllocation hAllocation, void** ppData)
        7613 {
        7614  if(hAllocation->CanBecomeLost())
        7615  {
        7616  return VK_ERROR_MEMORY_MAP_FAILED;
        7617  }
        7618 
        7619  switch(hAllocation->GetType())
        7620  {
        7621  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
        7622  {
        7623  VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
        7624  char *pBytes = nullptr;
        7625  VkResult res = pBlock->Map(this, (void**)&pBytes);
        7626  if(res == VK_SUCCESS)
        7627  {
        7628  *ppData = pBytes + (ptrdiff_t)hAllocation->GetOffset();
        7629  hAllocation->BlockAllocMap();
        7630  }
        7631  return res;
        7632  }
        7633  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
        7634  return hAllocation->DedicatedAllocMap(this, ppData);
        7635  default:
        7636  VMA_ASSERT(0);
        7637  return VK_ERROR_MEMORY_MAP_FAILED;
        7638  }
        7639 }
        7640 
        7641 void VmaAllocator_T::Unmap(VmaAllocation hAllocation)
        7642 {
        7643  switch(hAllocation->GetType())
        7644  {
        7645  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
        7646  {
        7647  VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
        7648  hAllocation->BlockAllocUnmap();
        7649  pBlock->Unmap(this);
        7650  }
        7651  break;
        7652  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
        7653  hAllocation->DedicatedAllocUnmap(this);
        7654  break;
        7655  default:
        7656  VMA_ASSERT(0);
        7657  }
        7658 }
        7659 
        7660 void VmaAllocator_T::FreeDedicatedMemory(VmaAllocation allocation)
        7661 {
        7662  VMA_ASSERT(allocation && allocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
        7663 
        7664  const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
        7665  {
        7666  VmaMutexLock lock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
        7667  AllocationVectorType* const pDedicatedAllocations = m_pDedicatedAllocations[memTypeIndex];
        7668  VMA_ASSERT(pDedicatedAllocations);
        7669  bool success = VmaVectorRemoveSorted<VmaPointerLess>(*pDedicatedAllocations, allocation);
        7670  VMA_ASSERT(success);
        7671  }
        7672 
        7673  VkDeviceMemory hMemory = allocation->GetMemory();
        7674 
        7675  if(allocation->GetMappedData() != VMA_NULL)
        7676  {
        7677  (*m_VulkanFunctions.vkUnmapMemory)(m_hDevice, hMemory);
        7678  }
        7679 
        7680  FreeVulkanMemory(memTypeIndex, allocation->GetSize(), hMemory);
        7681 
        7682  VMA_DEBUG_LOG(" Freed DedicatedMemory MemoryTypeIndex=%u", memTypeIndex);
        7683 }
        7684 
        7685 #if VMA_STATS_STRING_ENABLED
        7686 
        7687 void VmaAllocator_T::PrintDetailedMap(VmaJsonWriter& json)
        7688 {
        7689  bool dedicatedAllocationsStarted = false;
        7690  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
        7691  {
        7692  VmaMutexLock dedicatedAllocationsLock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
        7693  AllocationVectorType* const pDedicatedAllocVector = m_pDedicatedAllocations[memTypeIndex];
        7694  VMA_ASSERT(pDedicatedAllocVector);
        7695  if(pDedicatedAllocVector->empty() == false)
        7696  {
        7697  if(dedicatedAllocationsStarted == false)
        7698  {
        7699  dedicatedAllocationsStarted = true;
        7700  json.WriteString("DedicatedAllocations");
        7701  json.BeginObject();
        7702  }
        7703 
        7704  json.BeginString("Type ");
        7705  json.ContinueString(memTypeIndex);
        7706  json.EndString();
        7707 
        7708  json.BeginArray();
        7709 
        7710  for(size_t i = 0; i < pDedicatedAllocVector->size(); ++i)
        7711  {
        7712  const VmaAllocation hAlloc = (*pDedicatedAllocVector)[i];
        7713  json.BeginObject(true);
        7714 
        7715  json.WriteString("Type");
        7716  json.WriteString(VMA_SUBALLOCATION_TYPE_NAMES[hAlloc->GetSuballocationType()]);
        7717 
        7718  json.WriteString("Size");
        7719  json.WriteNumber(hAlloc->GetSize());
        7720 
        7721  const void* pUserData = hAlloc->GetUserData();
        7722  if(pUserData != VMA_NULL)
        7723  {
        7724  json.WriteString("UserData");
        7725  if(hAlloc->IsUserDataString())
        7726  {
        7727  json.WriteString((const char*)pUserData);
        7728  }
        7729  else
        7730  {
        7731  json.BeginString();
        7732  json.ContinueString_Pointer(pUserData);
        7733  json.EndString();
        7734  }
        7735  }
        7736 
        7737  json.EndObject();
        7738  }
        7739 
        7740  json.EndArray();
        7741  }
        7742  }
        7743  if(dedicatedAllocationsStarted)
        7744  {
        7745  json.EndObject();
        7746  }
        7747 
        7748  {
        7749  bool allocationsStarted = false;
        7750  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
        7751  {
        7752  if(m_pBlockVectors[memTypeIndex]->IsEmpty() == false)
        7753  {
        7754  if(allocationsStarted == false)
        7755  {
        7756  allocationsStarted = true;
        7757  json.WriteString("DefaultPools");
        7758  json.BeginObject();
        7759  }
        7760 
        7761  json.BeginString("Type ");
        7762  json.ContinueString(memTypeIndex);
        7763  json.EndString();
        7764 
        7765  m_pBlockVectors[memTypeIndex]->PrintDetailedMap(json);
        7766  }
        7767  }
        7768  if(allocationsStarted)
        7769  {
        7770  json.EndObject();
        7771  }
        7772  }
        7773 
        7774  {
        7775  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
        7776  const size_t poolCount = m_Pools.size();
        7777  if(poolCount > 0)
        7778  {
        7779  json.WriteString("Pools");
        7780  json.BeginArray();
        7781  for(size_t poolIndex = 0; poolIndex < poolCount; ++poolIndex)
        7782  {
        7783  m_Pools[poolIndex]->m_BlockVector.PrintDetailedMap(json);
        7784  }
        7785  json.EndArray();
        7786  }
        7787  }
        7788 }
        7789 
        7790 #endif // #if VMA_STATS_STRING_ENABLED
        7791 
        7792 static VkResult AllocateMemoryForImage(
        7793  VmaAllocator allocator,
        7794  VkImage image,
        7795  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        7796  VmaSuballocationType suballocType,
        7797  VmaAllocation* pAllocation)
        7798 {
        7799  VMA_ASSERT(allocator && (image != VK_NULL_HANDLE) && pAllocationCreateInfo && pAllocation);
        7800 
        7801  VkMemoryRequirements vkMemReq = {};
        7802  bool requiresDedicatedAllocation = false;
        7803  bool prefersDedicatedAllocation = false;
        7804  allocator->GetImageMemoryRequirements(image, vkMemReq,
        7805  requiresDedicatedAllocation, prefersDedicatedAllocation);
        7806 
        7807  return allocator->AllocateMemory(
        7808  vkMemReq,
        7809  requiresDedicatedAllocation,
        7810  prefersDedicatedAllocation,
        7811  VK_NULL_HANDLE, // dedicatedBuffer
        7812  image, // dedicatedImage
        7813  *pAllocationCreateInfo,
        7814  suballocType,
        7815  pAllocation);
        7816 }
        7817 
        7819 // Public interface
        7820 
        7821 VkResult vmaCreateAllocator(
        7822  const VmaAllocatorCreateInfo* pCreateInfo,
        7823  VmaAllocator* pAllocator)
        7824 {
        7825  VMA_ASSERT(pCreateInfo && pAllocator);
        7826  VMA_DEBUG_LOG("vmaCreateAllocator");
        7827  *pAllocator = vma_new(pCreateInfo->pAllocationCallbacks, VmaAllocator_T)(pCreateInfo);
        7828  return VK_SUCCESS;
        7829 }
        7830 
        7831 void vmaDestroyAllocator(
        7832  VmaAllocator allocator)
        7833 {
        7834  if(allocator != VK_NULL_HANDLE)
        7835  {
        7836  VMA_DEBUG_LOG("vmaDestroyAllocator");
        7837  VkAllocationCallbacks allocationCallbacks = allocator->m_AllocationCallbacks;
        7838  vma_delete(&allocationCallbacks, allocator);
        7839  }
        7840 }
        7841 
        7842 void vmaGetPhysicalDeviceProperties(
        7843  VmaAllocator allocator,
        7844  const VkPhysicalDeviceProperties **ppPhysicalDeviceProperties)
        7845 {
        7846  VMA_ASSERT(allocator && ppPhysicalDeviceProperties);
        7847  *ppPhysicalDeviceProperties = &allocator->m_PhysicalDeviceProperties;
        7848 }
        7849 
        7850 void vmaGetMemoryProperties(
        7851  VmaAllocator allocator,
        7852  const VkPhysicalDeviceMemoryProperties** ppPhysicalDeviceMemoryProperties)
        7853 {
        7854  VMA_ASSERT(allocator && ppPhysicalDeviceMemoryProperties);
        7855  *ppPhysicalDeviceMemoryProperties = &allocator->m_MemProps;
        7856 }
        7857 
        7858 void vmaGetMemoryTypeProperties(
        7859  VmaAllocator allocator,
        7860  uint32_t memoryTypeIndex,
        7861  VkMemoryPropertyFlags* pFlags)
        7862 {
        7863  VMA_ASSERT(allocator && pFlags);
        7864  VMA_ASSERT(memoryTypeIndex < allocator->GetMemoryTypeCount());
        7865  *pFlags = allocator->m_MemProps.memoryTypes[memoryTypeIndex].propertyFlags;
        7866 }
        7867 
        7868 void vmaSetCurrentFrameIndex(
        7869  VmaAllocator allocator,
        7870  uint32_t frameIndex)
        7871 {
        7872  VMA_ASSERT(allocator);
        7873  VMA_ASSERT(frameIndex != VMA_FRAME_INDEX_LOST);
        7874 
        7875  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        7876 
        7877  allocator->SetCurrentFrameIndex(frameIndex);
        7878 }
        7879 
        7880 void vmaCalculateStats(
        7881  VmaAllocator allocator,
        7882  VmaStats* pStats)
        7883 {
        7884  VMA_ASSERT(allocator && pStats);
        7885  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        7886  allocator->CalculateStats(pStats);
        7887 }
        7888 
        7889 #if VMA_STATS_STRING_ENABLED
        7890 
        7891 void vmaBuildStatsString(
        7892  VmaAllocator allocator,
        7893  char** ppStatsString,
        7894  VkBool32 detailedMap)
        7895 {
        7896  VMA_ASSERT(allocator && ppStatsString);
        7897  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        7898 
        7899  VmaStringBuilder sb(allocator);
        7900  {
        7901  VmaJsonWriter json(allocator->GetAllocationCallbacks(), sb);
        7902  json.BeginObject();
        7903 
        7904  VmaStats stats;
        7905  allocator->CalculateStats(&stats);
        7906 
        7907  json.WriteString("Total");
        7908  VmaPrintStatInfo(json, stats.total);
        7909 
        7910  for(uint32_t heapIndex = 0; heapIndex < allocator->GetMemoryHeapCount(); ++heapIndex)
        7911  {
        7912  json.BeginString("Heap ");
        7913  json.ContinueString(heapIndex);
        7914  json.EndString();
        7915  json.BeginObject();
        7916 
        7917  json.WriteString("Size");
        7918  json.WriteNumber(allocator->m_MemProps.memoryHeaps[heapIndex].size);
        7919 
        7920  json.WriteString("Flags");
        7921  json.BeginArray(true);
        7922  if((allocator->m_MemProps.memoryHeaps[heapIndex].flags & VK_MEMORY_HEAP_DEVICE_LOCAL_BIT) != 0)
        7923  {
        7924  json.WriteString("DEVICE_LOCAL");
        7925  }
        7926  json.EndArray();
        7927 
        7928  if(stats.memoryHeap[heapIndex].blockCount > 0)
        7929  {
        7930  json.WriteString("Stats");
        7931  VmaPrintStatInfo(json, stats.memoryHeap[heapIndex]);
        7932  }
        7933 
        7934  for(uint32_t typeIndex = 0; typeIndex < allocator->GetMemoryTypeCount(); ++typeIndex)
        7935  {
        7936  if(allocator->MemoryTypeIndexToHeapIndex(typeIndex) == heapIndex)
        7937  {
        7938  json.BeginString("Type ");
        7939  json.ContinueString(typeIndex);
        7940  json.EndString();
        7941 
        7942  json.BeginObject();
        7943 
        7944  json.WriteString("Flags");
        7945  json.BeginArray(true);
        7946  VkMemoryPropertyFlags flags = allocator->m_MemProps.memoryTypes[typeIndex].propertyFlags;
        7947  if((flags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT) != 0)
        7948  {
        7949  json.WriteString("DEVICE_LOCAL");
        7950  }
        7951  if((flags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
        7952  {
        7953  json.WriteString("HOST_VISIBLE");
        7954  }
        7955  if((flags & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT) != 0)
        7956  {
        7957  json.WriteString("HOST_COHERENT");
        7958  }
        7959  if((flags & VK_MEMORY_PROPERTY_HOST_CACHED_BIT) != 0)
        7960  {
        7961  json.WriteString("HOST_CACHED");
        7962  }
        7963  if((flags & VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT) != 0)
        7964  {
        7965  json.WriteString("LAZILY_ALLOCATED");
        7966  }
        7967  json.EndArray();
        7968 
        7969  if(stats.memoryType[typeIndex].blockCount > 0)
        7970  {
        7971  json.WriteString("Stats");
        7972  VmaPrintStatInfo(json, stats.memoryType[typeIndex]);
        7973  }
        7974 
        7975  json.EndObject();
        7976  }
        7977  }
        7978 
        7979  json.EndObject();
        7980  }
        7981  if(detailedMap == VK_TRUE)
        7982  {
        7983  allocator->PrintDetailedMap(json);
        7984  }
        7985 
        7986  json.EndObject();
        7987  }
        7988 
        7989  const size_t len = sb.GetLength();
        7990  char* const pChars = vma_new_array(allocator, char, len + 1);
        7991  if(len > 0)
        7992  {
        7993  memcpy(pChars, sb.GetData(), len);
        7994  }
        7995  pChars[len] = '\0';
        7996  *ppStatsString = pChars;
        7997 }
        7998 
        7999 void vmaFreeStatsString(
        8000  VmaAllocator allocator,
        8001  char* pStatsString)
        8002 {
        8003  if(pStatsString != VMA_NULL)
        8004  {
        8005  VMA_ASSERT(allocator);
        8006  size_t len = strlen(pStatsString);
        8007  vma_delete_array(allocator, pStatsString, len + 1);
        8008  }
        8009 }
        8010 
        8011 #endif // #if VMA_STATS_STRING_ENABLED
        8012 
        8013 /*
        8014 This function is not protected by any mutex because it just reads immutable data.
        8015 */
        8016 VkResult vmaFindMemoryTypeIndex(
        8017  VmaAllocator allocator,
        8018  uint32_t memoryTypeBits,
        8019  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        8020  uint32_t* pMemoryTypeIndex)
        8021 {
        8022  VMA_ASSERT(allocator != VK_NULL_HANDLE);
        8023  VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
        8024  VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
        8025 
        8026  if(pAllocationCreateInfo->memoryTypeBits != 0)
        8027  {
        8028  memoryTypeBits &= pAllocationCreateInfo->memoryTypeBits;
        8029  }
        8030 
        8031  uint32_t requiredFlags = pAllocationCreateInfo->requiredFlags;
        8032  uint32_t preferredFlags = pAllocationCreateInfo->preferredFlags;
        8033 
        8034  // Convert usage to requiredFlags and preferredFlags.
        8035  switch(pAllocationCreateInfo->usage)
        8036  {
        8037  case VMA_MEMORY_USAGE_UNKNOWN:
        8038  break;
        8039  case VMA_MEMORY_USAGE_GPU_ONLY:
        8040  preferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
        8041  break;
        8042  case VMA_MEMORY_USAGE_CPU_ONLY:
        8043  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
        8044  break;
        8045  case VMA_MEMORY_USAGE_CPU_TO_GPU:
        8046  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
        8047  preferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
        8048  break;
        8049  case VMA_MEMORY_USAGE_GPU_TO_CPU:
        8050  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
        8051  preferredFlags |= VK_MEMORY_PROPERTY_HOST_COHERENT_BIT | VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
        8052  break;
        8053  default:
        8054  break;
        8055  }
        8056 
        8057  *pMemoryTypeIndex = UINT32_MAX;
        8058  uint32_t minCost = UINT32_MAX;
        8059  for(uint32_t memTypeIndex = 0, memTypeBit = 1;
        8060  memTypeIndex < allocator->GetMemoryTypeCount();
        8061  ++memTypeIndex, memTypeBit <<= 1)
        8062  {
        8063  // This memory type is acceptable according to memoryTypeBits bitmask.
        8064  if((memTypeBit & memoryTypeBits) != 0)
        8065  {
        8066  const VkMemoryPropertyFlags currFlags =
        8067  allocator->m_MemProps.memoryTypes[memTypeIndex].propertyFlags;
        8068  // This memory type contains requiredFlags.
        8069  if((requiredFlags & ~currFlags) == 0)
        8070  {
        8071  // Calculate cost as number of bits from preferredFlags not present in this memory type.
        8072  uint32_t currCost = VmaCountBitsSet(preferredFlags & ~currFlags);
        8073  // Remember memory type with lowest cost.
        8074  if(currCost < minCost)
        8075  {
        8076  *pMemoryTypeIndex = memTypeIndex;
        8077  if(currCost == 0)
        8078  {
        8079  return VK_SUCCESS;
        8080  }
        8081  minCost = currCost;
        8082  }
        8083  }
        8084  }
        8085  }
        8086  return (*pMemoryTypeIndex != UINT32_MAX) ? VK_SUCCESS : VK_ERROR_FEATURE_NOT_PRESENT;
        8087 }
        8088 
        8089 VkResult vmaCreatePool(
        8090  VmaAllocator allocator,
        8091  const VmaPoolCreateInfo* pCreateInfo,
        8092  VmaPool* pPool)
        8093 {
        8094  VMA_ASSERT(allocator && pCreateInfo && pPool);
        8095 
        8096  VMA_DEBUG_LOG("vmaCreatePool");
        8097 
        8098  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8099 
        8100  return allocator->CreatePool(pCreateInfo, pPool);
        8101 }
        8102 
        8103 void vmaDestroyPool(
        8104  VmaAllocator allocator,
        8105  VmaPool pool)
        8106 {
        8107  VMA_ASSERT(allocator);
        8108 
        8109  if(pool == VK_NULL_HANDLE)
        8110  {
        8111  return;
        8112  }
        8113 
        8114  VMA_DEBUG_LOG("vmaDestroyPool");
        8115 
        8116  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8117 
        8118  allocator->DestroyPool(pool);
        8119 }
        8120 
        8121 void vmaGetPoolStats(
        8122  VmaAllocator allocator,
        8123  VmaPool pool,
        8124  VmaPoolStats* pPoolStats)
        8125 {
        8126  VMA_ASSERT(allocator && pool && pPoolStats);
        8127 
        8128  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8129 
        8130  allocator->GetPoolStats(pool, pPoolStats);
        8131 }
        8132 
        8133 void vmaMakePoolAllocationsLost(
        8134  VmaAllocator allocator,
        8135  VmaPool pool,
        8136  size_t* pLostAllocationCount)
        8137 {
        8138  VMA_ASSERT(allocator && pool);
        8139 
        8140  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8141 
        8142  allocator->MakePoolAllocationsLost(pool, pLostAllocationCount);
        8143 }
        8144 
        8145 VkResult vmaAllocateMemory(
        8146  VmaAllocator allocator,
        8147  const VkMemoryRequirements* pVkMemoryRequirements,
        8148  const VmaAllocationCreateInfo* pCreateInfo,
        8149  VmaAllocation* pAllocation,
        8150  VmaAllocationInfo* pAllocationInfo)
        8151 {
        8152  VMA_ASSERT(allocator && pVkMemoryRequirements && pCreateInfo && pAllocation);
        8153 
        8154  VMA_DEBUG_LOG("vmaAllocateMemory");
        8155 
        8156  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8157 
        8158  VkResult result = allocator->AllocateMemory(
        8159  *pVkMemoryRequirements,
        8160  false, // requiresDedicatedAllocation
        8161  false, // prefersDedicatedAllocation
        8162  VK_NULL_HANDLE, // dedicatedBuffer
        8163  VK_NULL_HANDLE, // dedicatedImage
        8164  *pCreateInfo,
        8165  VMA_SUBALLOCATION_TYPE_UNKNOWN,
        8166  pAllocation);
        8167 
        8168  if(pAllocationInfo && result == VK_SUCCESS)
        8169  {
        8170  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
        8171  }
        8172 
        8173  return result;
        8174 }
        8175 
        8176 VkResult vmaAllocateMemoryForBuffer(
        8177  VmaAllocator allocator,
        8178  VkBuffer buffer,
        8179  const VmaAllocationCreateInfo* pCreateInfo,
        8180  VmaAllocation* pAllocation,
        8181  VmaAllocationInfo* pAllocationInfo)
        8182 {
        8183  VMA_ASSERT(allocator && buffer != VK_NULL_HANDLE && pCreateInfo && pAllocation);
        8184 
        8185  VMA_DEBUG_LOG("vmaAllocateMemoryForBuffer");
        8186 
        8187  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8188 
        8189  VkMemoryRequirements vkMemReq = {};
        8190  bool requiresDedicatedAllocation = false;
        8191  bool prefersDedicatedAllocation = false;
        8192  allocator->GetBufferMemoryRequirements(buffer, vkMemReq,
        8193  requiresDedicatedAllocation,
        8194  prefersDedicatedAllocation);
        8195 
        8196  VkResult result = allocator->AllocateMemory(
        8197  vkMemReq,
        8198  requiresDedicatedAllocation,
        8199  prefersDedicatedAllocation,
        8200  buffer, // dedicatedBuffer
        8201  VK_NULL_HANDLE, // dedicatedImage
        8202  *pCreateInfo,
        8203  VMA_SUBALLOCATION_TYPE_BUFFER,
        8204  pAllocation);
        8205 
        8206  if(pAllocationInfo && result == VK_SUCCESS)
        8207  {
        8208  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
        8209  }
        8210 
        8211  return result;
        8212 }
        8213 
        8214 VkResult vmaAllocateMemoryForImage(
        8215  VmaAllocator allocator,
        8216  VkImage image,
        8217  const VmaAllocationCreateInfo* pCreateInfo,
        8218  VmaAllocation* pAllocation,
        8219  VmaAllocationInfo* pAllocationInfo)
        8220 {
        8221  VMA_ASSERT(allocator && image != VK_NULL_HANDLE && pCreateInfo && pAllocation);
        8222 
        8223  VMA_DEBUG_LOG("vmaAllocateMemoryForImage");
        8224 
        8225  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8226 
        8227  VkResult result = AllocateMemoryForImage(
        8228  allocator,
        8229  image,
        8230  pCreateInfo,
        8231  VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN,
        8232  pAllocation);
        8233 
        8234  if(pAllocationInfo && result == VK_SUCCESS)
        8235  {
        8236  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
        8237  }
        8238 
        8239  return result;
        8240 }
        8241 
        8242 void vmaFreeMemory(
        8243  VmaAllocator allocator,
        8244  VmaAllocation allocation)
        8245 {
        8246  VMA_ASSERT(allocator && allocation);
        8247 
        8248  VMA_DEBUG_LOG("vmaFreeMemory");
        8249 
        8250  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8251 
        8252  allocator->FreeMemory(allocation);
        8253 }
        8254 
        8255 void vmaGetAllocationInfo(
        8256  VmaAllocator allocator,
        8257  VmaAllocation allocation,
        8258  VmaAllocationInfo* pAllocationInfo)
        8259 {
        8260  VMA_ASSERT(allocator && allocation && pAllocationInfo);
        8261 
        8262  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8263 
        8264  allocator->GetAllocationInfo(allocation, pAllocationInfo);
        8265 }
        8266 
        8267 void vmaSetAllocationUserData(
        8268  VmaAllocator allocator,
        8269  VmaAllocation allocation,
        8270  void* pUserData)
        8271 {
        8272  VMA_ASSERT(allocator && allocation);
        8273 
        8274  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8275 
        8276  allocation->SetUserData(allocator, pUserData);
        8277 }
        8278 
        8279 void vmaCreateLostAllocation(
        8280  VmaAllocator allocator,
        8281  VmaAllocation* pAllocation)
        8282 {
        8283  VMA_ASSERT(allocator && pAllocation);
        8284 
        8285  VMA_DEBUG_GLOBAL_MUTEX_LOCK;
        8286 
        8287  allocator->CreateLostAllocation(pAllocation);
        8288 }
        8289 
        8290 VkResult vmaMapMemory(
        8291  VmaAllocator allocator,
        8292  VmaAllocation allocation,
        8293  void** ppData)
        8294 {
        8295  VMA_ASSERT(allocator && allocation && ppData);
        8296 
        8297  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8298 
        8299  return allocator->Map(allocation, ppData);
        8300 }
        8301 
        8302 void vmaUnmapMemory(
        8303  VmaAllocator allocator,
        8304  VmaAllocation allocation)
        8305 {
        8306  VMA_ASSERT(allocator && allocation);
        8307 
        8308  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8309 
        8310  allocator->Unmap(allocation);
        8311 }
        8312 
        8313 VkResult vmaDefragment(
        8314  VmaAllocator allocator,
        8315  VmaAllocation* pAllocations,
        8316  size_t allocationCount,
        8317  VkBool32* pAllocationsChanged,
        8318  const VmaDefragmentationInfo *pDefragmentationInfo,
        8319  VmaDefragmentationStats* pDefragmentationStats)
        8320 {
        8321  VMA_ASSERT(allocator && pAllocations);
        8322 
        8323  VMA_DEBUG_LOG("vmaDefragment");
        8324 
        8325  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8326 
        8327  return allocator->Defragment(pAllocations, allocationCount, pAllocationsChanged, pDefragmentationInfo, pDefragmentationStats);
        8328 }
        8329 
        8330 VkResult vmaCreateBuffer(
        8331  VmaAllocator allocator,
        8332  const VkBufferCreateInfo* pBufferCreateInfo,
        8333  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        8334  VkBuffer* pBuffer,
        8335  VmaAllocation* pAllocation,
        8336  VmaAllocationInfo* pAllocationInfo)
        8337 {
        8338  VMA_ASSERT(allocator && pBufferCreateInfo && pAllocationCreateInfo && pBuffer && pAllocation);
        8339 
        8340  VMA_DEBUG_LOG("vmaCreateBuffer");
        8341 
        8342  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8343 
        8344  *pBuffer = VK_NULL_HANDLE;
        8345  *pAllocation = VK_NULL_HANDLE;
        8346 
        8347  // 1. Create VkBuffer.
        8348  VkResult res = (*allocator->GetVulkanFunctions().vkCreateBuffer)(
        8349  allocator->m_hDevice,
        8350  pBufferCreateInfo,
        8351  allocator->GetAllocationCallbacks(),
        8352  pBuffer);
        8353  if(res >= 0)
        8354  {
        8355  // 2. vkGetBufferMemoryRequirements.
        8356  VkMemoryRequirements vkMemReq = {};
        8357  bool requiresDedicatedAllocation = false;
        8358  bool prefersDedicatedAllocation = false;
        8359  allocator->GetBufferMemoryRequirements(*pBuffer, vkMemReq,
        8360  requiresDedicatedAllocation, prefersDedicatedAllocation);
        8361 
        8362  // 3. Allocate memory using allocator.
        8363  res = allocator->AllocateMemory(
        8364  vkMemReq,
        8365  requiresDedicatedAllocation,
        8366  prefersDedicatedAllocation,
        8367  *pBuffer, // dedicatedBuffer
        8368  VK_NULL_HANDLE, // dedicatedImage
        8369  *pAllocationCreateInfo,
        8370  VMA_SUBALLOCATION_TYPE_BUFFER,
        8371  pAllocation);
        8372  if(res >= 0)
        8373  {
        8374  // 3. Bind buffer with memory.
        8375  res = (*allocator->GetVulkanFunctions().vkBindBufferMemory)(
        8376  allocator->m_hDevice,
        8377  *pBuffer,
        8378  (*pAllocation)->GetMemory(),
        8379  (*pAllocation)->GetOffset());
        8380  if(res >= 0)
        8381  {
        8382  // All steps succeeded.
        8383  if(pAllocationInfo != VMA_NULL)
        8384  {
        8385  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
        8386  }
        8387  return VK_SUCCESS;
        8388  }
        8389  allocator->FreeMemory(*pAllocation);
        8390  *pAllocation = VK_NULL_HANDLE;
        8391  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
        8392  *pBuffer = VK_NULL_HANDLE;
        8393  return res;
        8394  }
        8395  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
        8396  *pBuffer = VK_NULL_HANDLE;
        8397  return res;
        8398  }
        8399  return res;
        8400 }
        8401 
        8402 void vmaDestroyBuffer(
        8403  VmaAllocator allocator,
        8404  VkBuffer buffer,
        8405  VmaAllocation allocation)
        8406 {
        8407  if(buffer != VK_NULL_HANDLE)
        8408  {
        8409  VMA_ASSERT(allocator);
        8410 
        8411  VMA_DEBUG_LOG("vmaDestroyBuffer");
        8412 
        8413  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8414 
        8415  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, buffer, allocator->GetAllocationCallbacks());
        8416 
        8417  allocator->FreeMemory(allocation);
        8418  }
        8419 }
        8420 
        8421 VkResult vmaCreateImage(
        8422  VmaAllocator allocator,
        8423  const VkImageCreateInfo* pImageCreateInfo,
        8424  const VmaAllocationCreateInfo* pAllocationCreateInfo,
        8425  VkImage* pImage,
        8426  VmaAllocation* pAllocation,
        8427  VmaAllocationInfo* pAllocationInfo)
        8428 {
        8429  VMA_ASSERT(allocator && pImageCreateInfo && pAllocationCreateInfo && pImage && pAllocation);
        8430 
        8431  VMA_DEBUG_LOG("vmaCreateImage");
        8432 
        8433  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8434 
        8435  *pImage = VK_NULL_HANDLE;
        8436  *pAllocation = VK_NULL_HANDLE;
        8437 
        8438  // 1. Create VkImage.
        8439  VkResult res = (*allocator->GetVulkanFunctions().vkCreateImage)(
        8440  allocator->m_hDevice,
        8441  pImageCreateInfo,
        8442  allocator->GetAllocationCallbacks(),
        8443  pImage);
        8444  if(res >= 0)
        8445  {
        8446  VmaSuballocationType suballocType = pImageCreateInfo->tiling == VK_IMAGE_TILING_OPTIMAL ?
        8447  VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL :
        8448  VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR;
        8449 
        8450  // 2. Allocate memory using allocator.
        8451  res = AllocateMemoryForImage(allocator, *pImage, pAllocationCreateInfo, suballocType, pAllocation);
        8452  if(res >= 0)
        8453  {
        8454  // 3. Bind image with memory.
        8455  res = (*allocator->GetVulkanFunctions().vkBindImageMemory)(
        8456  allocator->m_hDevice,
        8457  *pImage,
        8458  (*pAllocation)->GetMemory(),
        8459  (*pAllocation)->GetOffset());
        8460  if(res >= 0)
        8461  {
        8462  // All steps succeeded.
        8463  if(pAllocationInfo != VMA_NULL)
        8464  {
        8465  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
        8466  }
        8467  return VK_SUCCESS;
        8468  }
        8469  allocator->FreeMemory(*pAllocation);
        8470  *pAllocation = VK_NULL_HANDLE;
        8471  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, *pImage, allocator->GetAllocationCallbacks());
        8472  *pImage = VK_NULL_HANDLE;
        8473  return res;
        8474  }
        8475  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, *pImage, allocator->GetAllocationCallbacks());
        8476  *pImage = VK_NULL_HANDLE;
        8477  return res;
        8478  }
        8479  return res;
        8480 }
        8481 
        8482 void vmaDestroyImage(
        8483  VmaAllocator allocator,
        8484  VkImage image,
        8485  VmaAllocation allocation)
        8486 {
        8487  if(image != VK_NULL_HANDLE)
        8488  {
        8489  VMA_ASSERT(allocator);
        8490 
        8491  VMA_DEBUG_LOG("vmaDestroyImage");
        8492 
        8493  VMA_DEBUG_GLOBAL_MUTEX_LOCK
        8494 
        8495  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, image, allocator->GetAllocationCallbacks());
        8496 
        8497  allocator->FreeMemory(allocation);
        8498  }
        8499 }
        8500 
        8501 #endif // #ifdef VMA_IMPLEMENTATION
        VmaVulkanFunctions::vkGetPhysicalDeviceProperties
        PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties
        Definition: vk_mem_alloc.h:764
        +
        VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT
        Set this flag if the allocation should have its own memory block.
        Definition: vk_mem_alloc.h:1011
        vmaUnmapMemory
        void vmaUnmapMemory(VmaAllocator allocator, VmaAllocation allocation)
        Unmaps memory represented by given allocation, mapped previously using vmaMapMemory().
        -
        VmaAllocatorCreateInfo::physicalDevice
        VkPhysicalDevice physicalDevice
        Vulkan physical device.
        Definition: vk_mem_alloc.h:695
        +
        VmaAllocatorCreateInfo::physicalDevice
        VkPhysicalDevice physicalDevice
        Vulkan physical device.
        Definition: vk_mem_alloc.h:789
        vmaDefragment
        VkResult vmaDefragment(VmaAllocator allocator, VmaAllocation *pAllocations, size_t allocationCount, VkBool32 *pAllocationsChanged, const VmaDefragmentationInfo *pDefragmentationInfo, VmaDefragmentationStats *pDefragmentationStats)
        Compacts memory by moving allocations.
        -
        VmaVulkanFunctions::vkCreateBuffer
        PFN_vkCreateBuffer vkCreateBuffer
        Definition: vk_mem_alloc.h:680
        +
        VmaVulkanFunctions::vkCreateBuffer
        PFN_vkCreateBuffer vkCreateBuffer
        Definition: vk_mem_alloc.h:774
        vmaFreeStatsString
        void vmaFreeStatsString(VmaAllocator allocator, char *pStatsString)
        VmaStats
        struct VmaStats VmaStats
        General statistics from current state of Allocator.
        -
        VMA_MEMORY_USAGE_GPU_ONLY
        Memory will be used on device only, so faster access from the device is preferred. No need to be mappable on host.
        Definition: vk_mem_alloc.h:867
        -
        VmaVulkanFunctions::vkMapMemory
        PFN_vkMapMemory vkMapMemory
        Definition: vk_mem_alloc.h:674
        -
        VmaAllocationInfo::deviceMemory
        VkDeviceMemory deviceMemory
        Handle to Vulkan memory object.
        Definition: vk_mem_alloc.h:1148
        -
        VmaAllocatorCreateInfo::flags
        VmaAllocatorCreateFlags flags
        Flags for created allocator. Use VmaAllocatorCreateFlagBits enum.
        Definition: vk_mem_alloc.h:692
        -
        VmaDefragmentationInfo::maxAllocationsToMove
        uint32_t maxAllocationsToMove
        Maximum number of allocations that can be moved to different place.
        Definition: vk_mem_alloc.h:1314
        -
        VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT
        Use this flag if you always allocate only buffers and linear images or only optimal images out of thi...
        Definition: vk_mem_alloc.h:1018
        +
        VMA_MEMORY_USAGE_GPU_ONLY
        Definition: vk_mem_alloc.h:974
        +
        VmaVulkanFunctions::vkMapMemory
        PFN_vkMapMemory vkMapMemory
        Definition: vk_mem_alloc.h:768
        +
        VmaAllocationInfo::deviceMemory
        VkDeviceMemory deviceMemory
        Handle to Vulkan memory object.
        Definition: vk_mem_alloc.h:1279
        +
        VmaAllocatorCreateInfo::flags
        VmaAllocatorCreateFlags flags
        Flags for created allocator. Use VmaAllocatorCreateFlagBits enum.
        Definition: vk_mem_alloc.h:786
        +
        VmaDefragmentationInfo::maxAllocationsToMove
        uint32_t maxAllocationsToMove
        Maximum number of allocations that can be moved to different place.
        Definition: vk_mem_alloc.h:1445
        +
        VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT
        Use this flag if you always allocate only buffers and linear images or only optimal images out of thi...
        Definition: vk_mem_alloc.h:1149
        vmaMakePoolAllocationsLost
        void vmaMakePoolAllocationsLost(VmaAllocator allocator, VmaPool pool, size_t *pLostAllocationCount)
        Marks all allocations in given pool as lost if they are not used in current frame or VmaPoolCreateInf...
        -
        VmaPoolStats::size
        VkDeviceSize size
        Total amount of VkDeviceMemory allocated from Vulkan for this pool, in bytes.
        Definition: vk_mem_alloc.h:1072
        -
        VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT
        Definition: vk_mem_alloc.h:930
        -
        VmaAllocatorCreateFlags
        VkFlags VmaAllocatorCreateFlags
        Definition: vk_mem_alloc.h:663
        -
        VmaAllocationCreateInfo::preferredFlags
        VkMemoryPropertyFlags preferredFlags
        Flags that preferably should be set in a Memory Type chosen for an allocation.
        Definition: vk_mem_alloc.h:969
        -
        VMA_MEMORY_USAGE_MAX_ENUM
        Definition: vk_mem_alloc.h:877
        -
        VmaAllocatorCreateInfo::pAllocationCallbacks
        const VkAllocationCallbacks * pAllocationCallbacks
        Custom CPU memory allocation callbacks.
        Definition: vk_mem_alloc.h:707
        +
        VmaPoolStats::size
        VkDeviceSize size
        Total amount of VkDeviceMemory allocated from Vulkan for this pool, in bytes.
        Definition: vk_mem_alloc.h:1203
        +
        VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT
        Definition: vk_mem_alloc.h:1048
        +
        VmaAllocatorCreateFlags
        VkFlags VmaAllocatorCreateFlags
        Definition: vk_mem_alloc.h:757
        +
        VmaAllocationCreateInfo::preferredFlags
        VkMemoryPropertyFlags preferredFlags
        Flags that preferably should be set in a memory type chosen for an allocation.
        Definition: vk_mem_alloc.h:1086
        +
        VMA_MEMORY_USAGE_MAX_ENUM
        Definition: vk_mem_alloc.h:995
        +
        VmaAllocatorCreateInfo::pAllocationCallbacks
        const VkAllocationCallbacks * pAllocationCallbacks
        Custom CPU memory allocation callbacks.
        Definition: vk_mem_alloc.h:801
        vmaCalculateStats
        void vmaCalculateStats(VmaAllocator allocator, VmaStats *pStats)
        Retrieves statistics from current state of the Allocator.
        -
        VmaAllocatorCreateInfo::pVulkanFunctions
        const VmaVulkanFunctions * pVulkanFunctions
        Pointers to Vulkan functions. Can be null if you leave define VMA_STATIC_VULKAN_FUNCTIONS 1...
        Definition: vk_mem_alloc.h:760
        -
        VmaAllocatorCreateInfo
        Description of a Allocator to be created.
        Definition: vk_mem_alloc.h:689
        -
        VmaAllocatorCreateInfo::preferredSmallHeapBlockSize
        VkDeviceSize preferredSmallHeapBlockSize
        Preferred size of a single VkDeviceMemory block to be allocated from small heaps <= 512 MB...
        Definition: vk_mem_alloc.h:704
        +
        VmaAllocatorCreateInfo::pVulkanFunctions
        const VmaVulkanFunctions * pVulkanFunctions
        Pointers to Vulkan functions. Can be null if you leave define VMA_STATIC_VULKAN_FUNCTIONS 1...
        Definition: vk_mem_alloc.h:854
        +
        VmaAllocatorCreateInfo
        Description of a Allocator to be created.
        Definition: vk_mem_alloc.h:783
        +
        VmaAllocatorCreateInfo::preferredSmallHeapBlockSize
        VkDeviceSize preferredSmallHeapBlockSize
        Preferred size of a single VkDeviceMemory block to be allocated from small heaps <= 512 MB...
        Definition: vk_mem_alloc.h:798
        vmaDestroyAllocator
        void vmaDestroyAllocator(VmaAllocator allocator)
        Destroys allocator object.
        -
        VmaAllocationCreateFlagBits
        VmaAllocationCreateFlagBits
        Flags to be passed as VmaAllocationCreateInfo::flags.
        Definition: vk_mem_alloc.h:881
        +
        VmaAllocationCreateFlagBits
        VmaAllocationCreateFlagBits
        Flags to be passed as VmaAllocationCreateInfo::flags.
        Definition: vk_mem_alloc.h:999
        vmaGetAllocationInfo
        void vmaGetAllocationInfo(VmaAllocator allocator, VmaAllocation allocation, VmaAllocationInfo *pAllocationInfo)
        Returns current information about specified allocation.
        -
        VmaStatInfo::allocationSizeMax
        VkDeviceSize allocationSizeMax
        Definition: vk_mem_alloc.h:825
        -
        VmaVulkanFunctions::vkBindImageMemory
        PFN_vkBindImageMemory vkBindImageMemory
        Definition: vk_mem_alloc.h:677
        -
        VmaStatInfo::unusedBytes
        VkDeviceSize unusedBytes
        Total number of bytes occupied by unused ranges.
        Definition: vk_mem_alloc.h:824
        -
        VmaVulkanFunctions::vkGetImageMemoryRequirements2KHR
        PFN_vkGetImageMemoryRequirements2KHR vkGetImageMemoryRequirements2KHR
        Definition: vk_mem_alloc.h:685
        -
        VmaDefragmentationStats
        Statistics returned by function vmaDefragment().
        Definition: vk_mem_alloc.h:1318
        +
        VmaStatInfo::allocationSizeMax
        VkDeviceSize allocationSizeMax
        Definition: vk_mem_alloc.h:919
        +
        VmaVulkanFunctions::vkBindImageMemory
        PFN_vkBindImageMemory vkBindImageMemory
        Definition: vk_mem_alloc.h:771
        +
        VmaStatInfo::unusedBytes
        VkDeviceSize unusedBytes
        Total number of bytes occupied by unused ranges.
        Definition: vk_mem_alloc.h:918
        +
        VmaVulkanFunctions::vkGetImageMemoryRequirements2KHR
        PFN_vkGetImageMemoryRequirements2KHR vkGetImageMemoryRequirements2KHR
        Definition: vk_mem_alloc.h:779
        +
        VmaDefragmentationStats
        Statistics returned by function vmaDefragment().
        Definition: vk_mem_alloc.h:1449
        vmaFreeMemory
        void vmaFreeMemory(VmaAllocator allocator, VmaAllocation allocation)
        Frees memory previously allocated using vmaAllocateMemory(), vmaAllocateMemoryForBuffer(), or vmaAllocateMemoryForImage().
        -
        VmaAllocatorCreateInfo::frameInUseCount
        uint32_t frameInUseCount
        Maximum number of additional frames that are in use at the same time as current frame.
        Definition: vk_mem_alloc.h:724
        -
        VmaStats::total
        VmaStatInfo total
        Definition: vk_mem_alloc.h:834
        -
        VmaDefragmentationStats::deviceMemoryBlocksFreed
        uint32_t deviceMemoryBlocksFreed
        Number of empty VkDeviceMemory objects that have been released to the system.
        Definition: vk_mem_alloc.h:1326
        -
        VmaAllocationCreateInfo::flags
        VmaAllocationCreateFlags flags
        Use VmaAllocationCreateFlagBits enum.
        Definition: vk_mem_alloc.h:952
        -
        VmaDefragmentationInfo::maxBytesToMove
        VkDeviceSize maxBytesToMove
        Maximum total numbers of bytes that can be copied while moving allocations to different places...
        Definition: vk_mem_alloc.h:1309
        -
        VmaVulkanFunctions::vkGetBufferMemoryRequirements
        PFN_vkGetBufferMemoryRequirements vkGetBufferMemoryRequirements
        Definition: vk_mem_alloc.h:678
        -
        PFN_vmaAllocateDeviceMemoryFunction
        void(VKAPI_PTR * PFN_vmaAllocateDeviceMemoryFunction)(VmaAllocator allocator, uint32_t memoryType, VkDeviceMemory memory, VkDeviceSize size)
        Callback function called after successful vkAllocateMemory.
        Definition: vk_mem_alloc.h:599
        -
        VmaAllocatorCreateInfo::device
        VkDevice device
        Vulkan device.
        Definition: vk_mem_alloc.h:698
        -
        VmaPoolCreateInfo
        Describes parameter of created VmaPool.
        Definition: vk_mem_alloc.h:1026
        -
        VMA_POOL_CREATE_FLAG_BITS_MAX_ENUM
        Definition: vk_mem_alloc.h:1020
        -
        VmaAllocationInfo::size
        VkDeviceSize size
        Size of this allocation, in bytes.
        Definition: vk_mem_alloc.h:1158
        +
        VmaAllocatorCreateInfo::frameInUseCount
        uint32_t frameInUseCount
        Maximum number of additional frames that are in use at the same time as current frame.
        Definition: vk_mem_alloc.h:818
        +
        VmaStats::total
        VmaStatInfo total
        Definition: vk_mem_alloc.h:928
        +
        VmaDefragmentationStats::deviceMemoryBlocksFreed
        uint32_t deviceMemoryBlocksFreed
        Number of empty VkDeviceMemory objects that have been released to the system.
        Definition: vk_mem_alloc.h:1457
        +
        VmaAllocationCreateInfo::flags
        VmaAllocationCreateFlags flags
        Use VmaAllocationCreateFlagBits enum.
        Definition: vk_mem_alloc.h:1070
        +
        VmaDefragmentationInfo::maxBytesToMove
        VkDeviceSize maxBytesToMove
        Maximum total numbers of bytes that can be copied while moving allocations to different places...
        Definition: vk_mem_alloc.h:1440
        +
        VmaVulkanFunctions::vkGetBufferMemoryRequirements
        PFN_vkGetBufferMemoryRequirements vkGetBufferMemoryRequirements
        Definition: vk_mem_alloc.h:772
        +
        PFN_vmaAllocateDeviceMemoryFunction
        void(VKAPI_PTR * PFN_vmaAllocateDeviceMemoryFunction)(VmaAllocator allocator, uint32_t memoryType, VkDeviceMemory memory, VkDeviceSize size)
        Callback function called after successful vkAllocateMemory.
        Definition: vk_mem_alloc.h:693
        +
        VmaAllocatorCreateInfo::device
        VkDevice device
        Vulkan device.
        Definition: vk_mem_alloc.h:792
        +
        VmaPoolCreateInfo
        Describes parameter of created VmaPool.
        Definition: vk_mem_alloc.h:1157
        +
        VMA_POOL_CREATE_FLAG_BITS_MAX_ENUM
        Definition: vk_mem_alloc.h:1151
        +
        VmaAllocationInfo::size
        VkDeviceSize size
        Size of this allocation, in bytes.
        Definition: vk_mem_alloc.h:1289
        vmaGetMemoryTypeProperties
        void vmaGetMemoryTypeProperties(VmaAllocator allocator, uint32_t memoryTypeIndex, VkMemoryPropertyFlags *pFlags)
        Given Memory Type Index, returns Property Flags of this memory type.
        -
        VmaVulkanFunctions::vkUnmapMemory
        PFN_vkUnmapMemory vkUnmapMemory
        Definition: vk_mem_alloc.h:675
        -
        VmaAllocationCreateInfo::pUserData
        void * pUserData
        Custom general-purpose pointer that will be stored in VmaAllocation, can be read as VmaAllocationInfo...
        Definition: vk_mem_alloc.h:971
        -
        VmaPoolCreateInfo::minBlockCount
        size_t minBlockCount
        Minimum number of blocks to be always allocated in this pool, even if they stay empty.
        Definition: vk_mem_alloc.h:1042
        -
        VmaPoolStats::allocationCount
        size_t allocationCount
        Number of VmaAllocation objects created from this pool that were not destroyed or lost...
        Definition: vk_mem_alloc.h:1078
        +
        VmaVulkanFunctions::vkUnmapMemory
        PFN_vkUnmapMemory vkUnmapMemory
        Definition: vk_mem_alloc.h:769
        +
        VmaAllocationCreateInfo::pUserData
        void * pUserData
        Custom general-purpose pointer that will be stored in VmaAllocation, can be read as VmaAllocationInfo...
        Definition: vk_mem_alloc.h:1107
        +
        VmaPoolCreateInfo::minBlockCount
        size_t minBlockCount
        Minimum number of blocks to be always allocated in this pool, even if they stay empty.
        Definition: vk_mem_alloc.h:1173
        +
        VmaPoolStats::allocationCount
        size_t allocationCount
        Number of VmaAllocation objects created from this pool that were not destroyed or lost...
        Definition: vk_mem_alloc.h:1209
        VmaVulkanFunctions
        struct VmaVulkanFunctions VmaVulkanFunctions
        Pointers to some Vulkan functions - a subset used by the library.
        -
        VMA_ALLOCATOR_CREATE_FLAG_BITS_MAX_ENUM
        Definition: vk_mem_alloc.h:661
        -
        VmaPoolCreateInfo::memoryTypeIndex
        uint32_t memoryTypeIndex
        Vulkan memory type index to allocate this pool from.
        Definition: vk_mem_alloc.h:1029
        +
        VMA_ALLOCATOR_CREATE_FLAG_BITS_MAX_ENUM
        Definition: vk_mem_alloc.h:755
        +
        VmaPoolCreateInfo::memoryTypeIndex
        uint32_t memoryTypeIndex
        Vulkan memory type index to allocate this pool from.
        Definition: vk_mem_alloc.h:1160
        vmaFindMemoryTypeIndex
        VkResult vmaFindMemoryTypeIndex(VmaAllocator allocator, uint32_t memoryTypeBits, const VmaAllocationCreateInfo *pAllocationCreateInfo, uint32_t *pMemoryTypeIndex)
        -
        VmaMemoryUsage
        VmaMemoryUsage
        Definition: vk_mem_alloc.h:862
        +
        VmaMemoryUsage
        VmaMemoryUsage
        Definition: vk_mem_alloc.h:956
        VmaAllocationInfo
        struct VmaAllocationInfo VmaAllocationInfo
        Parameters of VmaAllocation objects, that can be retrieved using function vmaGetAllocationInfo().
        -
        VmaDefragmentationInfo
        Optional configuration parameters to be passed to function vmaDefragment().
        Definition: vk_mem_alloc.h:1304
        +
        VmaDefragmentationInfo
        Optional configuration parameters to be passed to function vmaDefragment().
        Definition: vk_mem_alloc.h:1435
        VmaPoolCreateInfo
        struct VmaPoolCreateInfo VmaPoolCreateInfo
        Describes parameter of created VmaPool.
        vmaDestroyPool
        void vmaDestroyPool(VmaAllocator allocator, VmaPool pool)
        Destroys VmaPool object and frees Vulkan device memory.
        -
        VmaDefragmentationStats::bytesFreed
        VkDeviceSize bytesFreed
        Total number of bytes that have been released to the system by freeing empty VkDeviceMemory objects...
        Definition: vk_mem_alloc.h:1322
        -
        VMA_MEMORY_USAGE_CPU_TO_GPU
        Memory will be used for frequent (dynamic) updates from host and reads on device (upload).
        Definition: vk_mem_alloc.h:873
        -
        VmaVulkanFunctions::vkBindBufferMemory
        PFN_vkBindBufferMemory vkBindBufferMemory
        Definition: vk_mem_alloc.h:676
        +
        VmaDefragmentationStats::bytesFreed
        VkDeviceSize bytesFreed
        Total number of bytes that have been released to the system by freeing empty VkDeviceMemory objects...
        Definition: vk_mem_alloc.h:1453
        +
        VMA_MEMORY_USAGE_CPU_TO_GPU
        Definition: vk_mem_alloc.h:989
        +
        VmaAllocationCreateInfo::memoryTypeBits
        uint32_t memoryTypeBits
        Bitmask containing one bit set for every memory type acceptable for this allocation.
        Definition: vk_mem_alloc.h:1094
        +
        VmaVulkanFunctions::vkBindBufferMemory
        PFN_vkBindBufferMemory vkBindBufferMemory
        Definition: vk_mem_alloc.h:770
        vmaGetPoolStats
        void vmaGetPoolStats(VmaAllocator allocator, VmaPool pool, VmaPoolStats *pPoolStats)
        Retrieves statistics of existing VmaPool object.
        VmaDefragmentationInfo
        struct VmaDefragmentationInfo VmaDefragmentationInfo
        Optional configuration parameters to be passed to function vmaDefragment().
        -
        VmaStats
        General statistics from current state of Allocator.
        Definition: vk_mem_alloc.h:830
        -
        PFN_vmaFreeDeviceMemoryFunction
        void(VKAPI_PTR * PFN_vmaFreeDeviceMemoryFunction)(VmaAllocator allocator, uint32_t memoryType, VkDeviceMemory memory, VkDeviceSize size)
        Callback function called before vkFreeMemory.
        Definition: vk_mem_alloc.h:605
        +
        VmaStats
        General statistics from current state of Allocator.
        Definition: vk_mem_alloc.h:924
        +
        PFN_vmaFreeDeviceMemoryFunction
        void(VKAPI_PTR * PFN_vmaFreeDeviceMemoryFunction)(VmaAllocator allocator, uint32_t memoryType, VkDeviceMemory memory, VkDeviceSize size)
        Callback function called before vkFreeMemory.
        Definition: vk_mem_alloc.h:699
        vmaSetAllocationUserData
        void vmaSetAllocationUserData(VmaAllocator allocator, VmaAllocation allocation, void *pUserData)
        Sets pUserData in given allocation to new value.
        vmaCreatePool
        VkResult vmaCreatePool(VmaAllocator allocator, const VmaPoolCreateInfo *pCreateInfo, VmaPool *pPool)
        Allocates Vulkan device memory and creates VmaPool object.
        -
        VmaAllocatorCreateFlagBits
        VmaAllocatorCreateFlagBits
        Flags for created VmaAllocator.
        Definition: vk_mem_alloc.h:626
        +
        VmaAllocatorCreateFlagBits
        VmaAllocatorCreateFlagBits
        Flags for created VmaAllocator.
        Definition: vk_mem_alloc.h:720
        VmaStatInfo
        struct VmaStatInfo VmaStatInfo
        Calculated statistics of memory usage in entire allocator.
        -
        VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT
        Allocator and all objects created from it will not be synchronized internally, so you must guarantee ...
        Definition: vk_mem_alloc.h:631
        -
        VmaDefragmentationStats::allocationsMoved
        uint32_t allocationsMoved
        Number of allocations that have been moved to different places.
        Definition: vk_mem_alloc.h:1324
        +
        VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT
        Allocator and all objects created from it will not be synchronized internally, so you must guarantee ...
        Definition: vk_mem_alloc.h:725
        +
        VmaDefragmentationStats::allocationsMoved
        uint32_t allocationsMoved
        Number of allocations that have been moved to different places.
        Definition: vk_mem_alloc.h:1455
        vmaCreateLostAllocation
        void vmaCreateLostAllocation(VmaAllocator allocator, VmaAllocation *pAllocation)
        Creates new allocation that is in lost state from the beginning.
        -
        VmaAllocationCreateInfo::requiredFlags
        VkMemoryPropertyFlags requiredFlags
        Flags that must be set in a Memory Type chosen for an allocation.
        Definition: vk_mem_alloc.h:963
        -
        VmaPoolStats::unusedRangeSizeMax
        VkDeviceSize unusedRangeSizeMax
        Size of the largest continuous free memory region.
        Definition: vk_mem_alloc.h:1088
        +
        VmaAllocationCreateInfo::requiredFlags
        VkMemoryPropertyFlags requiredFlags
        Flags that must be set in a Memory Type chosen for an allocation.
        Definition: vk_mem_alloc.h:1081
        +
        VmaPoolStats::unusedRangeSizeMax
        VkDeviceSize unusedRangeSizeMax
        Size of the largest continuous free memory region.
        Definition: vk_mem_alloc.h:1219
        vmaBuildStatsString
        void vmaBuildStatsString(VmaAllocator allocator, char **ppStatsString, VkBool32 detailedMap)
        Builds and returns statistics as string in JSON format.
        -
        VmaVulkanFunctions::vkGetPhysicalDeviceMemoryProperties
        PFN_vkGetPhysicalDeviceMemoryProperties vkGetPhysicalDeviceMemoryProperties
        Definition: vk_mem_alloc.h:671
        -
        VmaStatInfo
        Calculated statistics of memory usage in entire allocator.
        Definition: vk_mem_alloc.h:813
        -
        VmaPoolCreateInfo::blockSize
        VkDeviceSize blockSize
        Size of a single VkDeviceMemory block to be allocated as part of this pool, in bytes.
        Definition: vk_mem_alloc.h:1037
        -
        VmaDeviceMemoryCallbacks
        Set of callbacks that the library will call for vkAllocateMemory and vkFreeMemory.
        Definition: vk_mem_alloc.h:618
        +
        VmaVulkanFunctions::vkGetPhysicalDeviceMemoryProperties
        PFN_vkGetPhysicalDeviceMemoryProperties vkGetPhysicalDeviceMemoryProperties
        Definition: vk_mem_alloc.h:765
        +
        VmaStatInfo
        Calculated statistics of memory usage in entire allocator.
        Definition: vk_mem_alloc.h:907
        +
        VmaPoolCreateInfo::blockSize
        VkDeviceSize blockSize
        Size of a single VkDeviceMemory block to be allocated as part of this pool, in bytes.
        Definition: vk_mem_alloc.h:1168
        +
        VmaDeviceMemoryCallbacks
        Set of callbacks that the library will call for vkAllocateMemory and vkFreeMemory.
        Definition: vk_mem_alloc.h:712
        vmaCreateBuffer
        VkResult vmaCreateBuffer(VmaAllocator allocator, const VkBufferCreateInfo *pBufferCreateInfo, const VmaAllocationCreateInfo *pAllocationCreateInfo, VkBuffer *pBuffer, VmaAllocation *pAllocation, VmaAllocationInfo *pAllocationInfo)
        -
        VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT
        Definition: vk_mem_alloc.h:937
        -
        VmaStatInfo::unusedRangeSizeMin
        VkDeviceSize unusedRangeSizeMin
        Definition: vk_mem_alloc.h:826
        -
        VmaDeviceMemoryCallbacks::pfnFree
        PFN_vmaFreeDeviceMemoryFunction pfnFree
        Optional, can be null.
        Definition: vk_mem_alloc.h:622
        -
        VmaPoolCreateInfo::flags
        VmaPoolCreateFlags flags
        Use combination of VmaPoolCreateFlagBits.
        Definition: vk_mem_alloc.h:1032
        -
        VMA_MEMORY_USAGE_GPU_TO_CPU
        Memory will be used for frequent writing on device and readback on host (download).
        Definition: vk_mem_alloc.h:876
        +
        VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT
        Definition: vk_mem_alloc.h:1055
        +
        VmaStatInfo::unusedRangeSizeMin
        VkDeviceSize unusedRangeSizeMin
        Definition: vk_mem_alloc.h:920
        +
        VmaDeviceMemoryCallbacks::pfnFree
        PFN_vmaFreeDeviceMemoryFunction pfnFree
        Optional, can be null.
        Definition: vk_mem_alloc.h:716
        +
        VmaPoolCreateInfo::flags
        VmaPoolCreateFlags flags
        Use combination of VmaPoolCreateFlagBits.
        Definition: vk_mem_alloc.h:1163
        +
        VMA_MEMORY_USAGE_GPU_TO_CPU
        Definition: vk_mem_alloc.h:994
        VmaPoolStats
        struct VmaPoolStats VmaPoolStats
        Describes parameter of existing VmaPool.
        vmaCreateImage
        VkResult vmaCreateImage(VmaAllocator allocator, const VkImageCreateInfo *pImageCreateInfo, const VmaAllocationCreateInfo *pAllocationCreateInfo, VkImage *pImage, VmaAllocation *pAllocation, VmaAllocationInfo *pAllocationInfo)
        Function similar to vmaCreateBuffer().
        -
        VmaAllocationCreateInfo::usage
        VmaMemoryUsage usage
        Intended usage of memory.
        Definition: vk_mem_alloc.h:958
        -
        VmaAllocationCreateInfo
        Definition: vk_mem_alloc.h:949
        -
        VmaStatInfo::blockCount
        uint32_t blockCount
        Number of VkDeviceMemory Vulkan memory blocks allocated.
        Definition: vk_mem_alloc.h:816
        -
        VmaVulkanFunctions::vkFreeMemory
        PFN_vkFreeMemory vkFreeMemory
        Definition: vk_mem_alloc.h:673
        -
        VmaPoolCreateInfo::maxBlockCount
        size_t maxBlockCount
        Maximum number of blocks that can be allocated in this pool.
        Definition: vk_mem_alloc.h:1050
        -
        VmaAllocatorCreateInfo::pDeviceMemoryCallbacks
        const VmaDeviceMemoryCallbacks * pDeviceMemoryCallbacks
        Informative callbacks for vkAllocateMemory, vkFreeMemory.
        Definition: vk_mem_alloc.h:710
        -
        VmaPoolStats::unusedRangeCount
        size_t unusedRangeCount
        Number of continuous memory ranges in the pool not used by any VmaAllocation.
        Definition: vk_mem_alloc.h:1081
        -
        VmaAllocationCreateFlags
        VkFlags VmaAllocationCreateFlags
        Definition: vk_mem_alloc.h:947
        -
        VmaAllocationCreateInfo::pool
        VmaPool pool
        Pool that this allocation should be created in.
        Definition: vk_mem_alloc.h:976
        +
        VmaAllocationCreateInfo::usage
        VmaMemoryUsage usage
        Intended usage of memory.
        Definition: vk_mem_alloc.h:1076
        +
        VmaAllocationCreateInfo
        Definition: vk_mem_alloc.h:1067
        +
        VmaStatInfo::blockCount
        uint32_t blockCount
        Number of VkDeviceMemory Vulkan memory blocks allocated.
        Definition: vk_mem_alloc.h:910
        +
        VmaVulkanFunctions::vkFreeMemory
        PFN_vkFreeMemory vkFreeMemory
        Definition: vk_mem_alloc.h:767
        +
        VmaPoolCreateInfo::maxBlockCount
        size_t maxBlockCount
        Maximum number of blocks that can be allocated in this pool.
        Definition: vk_mem_alloc.h:1181
        +
        VmaAllocatorCreateInfo::pDeviceMemoryCallbacks
        const VmaDeviceMemoryCallbacks * pDeviceMemoryCallbacks
        Informative callbacks for vkAllocateMemory, vkFreeMemory.
        Definition: vk_mem_alloc.h:804
        +
        VmaPoolStats::unusedRangeCount
        size_t unusedRangeCount
        Number of continuous memory ranges in the pool not used by any VmaAllocation.
        Definition: vk_mem_alloc.h:1212
        +
        VmaAllocationCreateFlags
        VkFlags VmaAllocationCreateFlags
        Definition: vk_mem_alloc.h:1065
        +
        VmaAllocationCreateInfo::pool
        VmaPool pool
        Pool that this allocation should be created in.
        Definition: vk_mem_alloc.h:1100
        vmaGetMemoryProperties
        void vmaGetMemoryProperties(VmaAllocator allocator, const VkPhysicalDeviceMemoryProperties **ppPhysicalDeviceMemoryProperties)
        -
        VmaAllocatorCreateInfo::pHeapSizeLimit
        const VkDeviceSize * pHeapSizeLimit
        Either NULL or a pointer to an array of limits on maximum number of bytes that can be allocated out o...
        Definition: vk_mem_alloc.h:748
        -
        VmaStats::memoryType
        VmaStatInfo memoryType[VK_MAX_MEMORY_TYPES]
        Definition: vk_mem_alloc.h:832
        -
        VMA_ALLOCATION_CREATE_MAPPED_BIT
        Set this flag to use a memory that will be persistently mapped and retrieve pointer to it...
        Definition: vk_mem_alloc.h:917
        -
        VmaStatInfo::allocationSizeMin
        VkDeviceSize allocationSizeMin
        Definition: vk_mem_alloc.h:825
        -
        VmaVulkanFunctions::vkCreateImage
        PFN_vkCreateImage vkCreateImage
        Definition: vk_mem_alloc.h:682
        -
        VmaDeviceMemoryCallbacks::pfnAllocate
        PFN_vmaAllocateDeviceMemoryFunction pfnAllocate
        Optional, can be null.
        Definition: vk_mem_alloc.h:620
        -
        VmaVulkanFunctions::vkDestroyBuffer
        PFN_vkDestroyBuffer vkDestroyBuffer
        Definition: vk_mem_alloc.h:681
        +
        VmaAllocatorCreateInfo::pHeapSizeLimit
        const VkDeviceSize * pHeapSizeLimit
        Either NULL or a pointer to an array of limits on maximum number of bytes that can be allocated out o...
        Definition: vk_mem_alloc.h:842
        +
        VmaStats::memoryType
        VmaStatInfo memoryType[VK_MAX_MEMORY_TYPES]
        Definition: vk_mem_alloc.h:926
        +
        VMA_ALLOCATION_CREATE_MAPPED_BIT
        Set this flag to use a memory that will be persistently mapped and retrieve pointer to it...
        Definition: vk_mem_alloc.h:1035
        +
        VmaStatInfo::allocationSizeMin
        VkDeviceSize allocationSizeMin
        Definition: vk_mem_alloc.h:919
        +
        VmaVulkanFunctions::vkCreateImage
        PFN_vkCreateImage vkCreateImage
        Definition: vk_mem_alloc.h:776
        +
        VmaDeviceMemoryCallbacks::pfnAllocate
        PFN_vmaAllocateDeviceMemoryFunction pfnAllocate
        Optional, can be null.
        Definition: vk_mem_alloc.h:714
        +
        VmaVulkanFunctions::vkDestroyBuffer
        PFN_vkDestroyBuffer vkDestroyBuffer
        Definition: vk_mem_alloc.h:775
        vmaMapMemory
        VkResult vmaMapMemory(VmaAllocator allocator, VmaAllocation allocation, void **ppData)
        Maps memory represented by given allocation and returns pointer to it.
        -
        VmaPoolCreateInfo::frameInUseCount
        uint32_t frameInUseCount
        Maximum number of additional frames that are in use at the same time as current frame.
        Definition: vk_mem_alloc.h:1064
        +
        VmaPoolCreateInfo::frameInUseCount
        uint32_t frameInUseCount
        Maximum number of additional frames that are in use at the same time as current frame.
        Definition: vk_mem_alloc.h:1195
        vmaAllocateMemoryForImage
        VkResult vmaAllocateMemoryForImage(VmaAllocator allocator, VkImage image, const VmaAllocationCreateInfo *pCreateInfo, VmaAllocation *pAllocation, VmaAllocationInfo *pAllocationInfo)
        Function similar to vmaAllocateMemoryForBuffer().
        VmaAllocatorCreateInfo
        struct VmaAllocatorCreateInfo VmaAllocatorCreateInfo
        Description of a Allocator to be created.
        -
        VmaAllocationInfo::pUserData
        void * pUserData
        Custom general-purpose pointer that was passed as VmaAllocationCreateInfo::pUserData or set using vma...
        Definition: vk_mem_alloc.h:1172
        -
        VmaAllocatorCreateInfo::preferredLargeHeapBlockSize
        VkDeviceSize preferredLargeHeapBlockSize
        Preferred size of a single VkDeviceMemory block to be allocated from large heaps. ...
        Definition: vk_mem_alloc.h:701
        -
        VmaStatInfo::allocationSizeAvg
        VkDeviceSize allocationSizeAvg
        Definition: vk_mem_alloc.h:825
        -
        VmaStatInfo::usedBytes
        VkDeviceSize usedBytes
        Total number of bytes occupied by all allocations.
        Definition: vk_mem_alloc.h:822
        +
        VmaAllocationInfo::pUserData
        void * pUserData
        Custom general-purpose pointer that was passed as VmaAllocationCreateInfo::pUserData or set using vma...
        Definition: vk_mem_alloc.h:1303
        +
        VmaAllocatorCreateInfo::preferredLargeHeapBlockSize
        VkDeviceSize preferredLargeHeapBlockSize
        Preferred size of a single VkDeviceMemory block to be allocated from large heaps. ...
        Definition: vk_mem_alloc.h:795
        +
        VmaStatInfo::allocationSizeAvg
        VkDeviceSize allocationSizeAvg
        Definition: vk_mem_alloc.h:919
        +
        VmaStatInfo::usedBytes
        VkDeviceSize usedBytes
        Total number of bytes occupied by all allocations.
        Definition: vk_mem_alloc.h:916
        VmaDeviceMemoryCallbacks
        struct VmaDeviceMemoryCallbacks VmaDeviceMemoryCallbacks
        Set of callbacks that the library will call for vkAllocateMemory and vkFreeMemory.
        -
        VmaPoolStats
        Describes parameter of existing VmaPool.
        Definition: vk_mem_alloc.h:1069
        -
        VmaAllocationInfo::offset
        VkDeviceSize offset
        Offset into deviceMemory object to the beginning of this allocation, in bytes. (deviceMemory, offset) pair is unique to this allocation.
        Definition: vk_mem_alloc.h:1153
        -
        VMA_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM
        Definition: vk_mem_alloc.h:945
        -
        VmaDefragmentationStats::bytesMoved
        VkDeviceSize bytesMoved
        Total number of bytes that have been copied while moving allocations to different places...
        Definition: vk_mem_alloc.h:1320
        -
        VmaVulkanFunctions
        Pointers to some Vulkan functions - a subset used by the library.
        Definition: vk_mem_alloc.h:669
        +
        VmaPoolStats
        Describes parameter of existing VmaPool.
        Definition: vk_mem_alloc.h:1200
        +
        VmaAllocationInfo::offset
        VkDeviceSize offset
        Offset into deviceMemory object to the beginning of this allocation, in bytes. (deviceMemory, offset) pair is unique to this allocation.
        Definition: vk_mem_alloc.h:1284
        +
        VMA_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM
        Definition: vk_mem_alloc.h:1063
        +
        VmaDefragmentationStats::bytesMoved
        VkDeviceSize bytesMoved
        Total number of bytes that have been copied while moving allocations to different places...
        Definition: vk_mem_alloc.h:1451
        +
        VmaVulkanFunctions
        Pointers to some Vulkan functions - a subset used by the library.
        Definition: vk_mem_alloc.h:763
        vmaCreateAllocator
        VkResult vmaCreateAllocator(const VmaAllocatorCreateInfo *pCreateInfo, VmaAllocator *pAllocator)
        Creates Allocator object.
        -
        VmaVulkanFunctions::vkGetBufferMemoryRequirements2KHR
        PFN_vkGetBufferMemoryRequirements2KHR vkGetBufferMemoryRequirements2KHR
        Definition: vk_mem_alloc.h:684
        -
        VmaStatInfo::unusedRangeCount
        uint32_t unusedRangeCount
        Number of free ranges of memory between allocations.
        Definition: vk_mem_alloc.h:820
        -
        VMA_MEMORY_USAGE_UNKNOWN
        No intended memory usage specified. Use other members of VmaAllocationCreateInfo to specify your requ...
        Definition: vk_mem_alloc.h:865
        -
        VmaPoolCreateFlags
        VkFlags VmaPoolCreateFlags
        Definition: vk_mem_alloc.h:1022
        +
        VmaVulkanFunctions::vkGetBufferMemoryRequirements2KHR
        PFN_vkGetBufferMemoryRequirements2KHR vkGetBufferMemoryRequirements2KHR
        Definition: vk_mem_alloc.h:778
        +
        VmaStatInfo::unusedRangeCount
        uint32_t unusedRangeCount
        Number of free ranges of memory between allocations.
        Definition: vk_mem_alloc.h:914
        +
        VMA_MEMORY_USAGE_UNKNOWN
        Definition: vk_mem_alloc.h:961
        +
        VmaPoolCreateFlags
        VkFlags VmaPoolCreateFlags
        Definition: vk_mem_alloc.h:1153
        vmaGetPhysicalDeviceProperties
        void vmaGetPhysicalDeviceProperties(VmaAllocator allocator, const VkPhysicalDeviceProperties **ppPhysicalDeviceProperties)
        -
        VmaStatInfo::allocationCount
        uint32_t allocationCount
        Number of VmaAllocation allocation objects allocated.
        Definition: vk_mem_alloc.h:818
        -
        VmaVulkanFunctions::vkGetImageMemoryRequirements
        PFN_vkGetImageMemoryRequirements vkGetImageMemoryRequirements
        Definition: vk_mem_alloc.h:679
        -
        VmaVulkanFunctions::vkDestroyImage
        PFN_vkDestroyImage vkDestroyImage
        Definition: vk_mem_alloc.h:683
        -
        VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT
        Set this flag to only try to allocate from existing VkDeviceMemory blocks and never create new such b...
        Definition: vk_mem_alloc.h:904
        -
        VMA_MEMORY_USAGE_CPU_ONLY
        Memory will be mapped on host. Could be used for transfer to/from device.
        Definition: vk_mem_alloc.h:870
        -
        VmaAllocationInfo::pMappedData
        void * pMappedData
        Pointer to the beginning of this allocation as mapped data.
        Definition: vk_mem_alloc.h:1167
        +
        VmaStatInfo::allocationCount
        uint32_t allocationCount
        Number of VmaAllocation allocation objects allocated.
        Definition: vk_mem_alloc.h:912
        +
        VmaVulkanFunctions::vkGetImageMemoryRequirements
        PFN_vkGetImageMemoryRequirements vkGetImageMemoryRequirements
        Definition: vk_mem_alloc.h:773
        +
        VmaVulkanFunctions::vkDestroyImage
        PFN_vkDestroyImage vkDestroyImage
        Definition: vk_mem_alloc.h:777
        +
        VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT
        Set this flag to only try to allocate from existing VkDeviceMemory blocks and never create new such b...
        Definition: vk_mem_alloc.h:1022
        +
        VMA_MEMORY_USAGE_CPU_ONLY
        Definition: vk_mem_alloc.h:983
        +
        VmaAllocationInfo::pMappedData
        void * pMappedData
        Pointer to the beginning of this allocation as mapped data.
        Definition: vk_mem_alloc.h:1298
        vmaDestroyImage
        void vmaDestroyImage(VmaAllocator allocator, VkImage image, VmaAllocation allocation)
        Destroys Vulkan image and frees allocated memory.
        -
        VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT
        Enables usage of VK_KHR_dedicated_allocation extension.
        Definition: vk_mem_alloc.h:659
        +
        VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT
        Enables usage of VK_KHR_dedicated_allocation extension.
        Definition: vk_mem_alloc.h:753
        VmaDefragmentationStats
        struct VmaDefragmentationStats VmaDefragmentationStats
        Statistics returned by function vmaDefragment().
        -
        VmaVulkanFunctions::vkAllocateMemory
        PFN_vkAllocateMemory vkAllocateMemory
        Definition: vk_mem_alloc.h:672
        -
        VmaAllocationInfo
        Parameters of VmaAllocation objects, that can be retrieved using function vmaGetAllocationInfo().
        Definition: vk_mem_alloc.h:1134
        +
        VmaVulkanFunctions::vkAllocateMemory
        PFN_vkAllocateMemory vkAllocateMemory
        Definition: vk_mem_alloc.h:766
        +
        VmaAllocationInfo
        Parameters of VmaAllocation objects, that can be retrieved using function vmaGetAllocationInfo().
        Definition: vk_mem_alloc.h:1265
        vmaAllocateMemory
        VkResult vmaAllocateMemory(VmaAllocator allocator, const VkMemoryRequirements *pVkMemoryRequirements, const VmaAllocationCreateInfo *pCreateInfo, VmaAllocation *pAllocation, VmaAllocationInfo *pAllocationInfo)
        General purpose memory allocation.
        vmaSetCurrentFrameIndex
        void vmaSetCurrentFrameIndex(VmaAllocator allocator, uint32_t frameIndex)
        Sets index of the current frame.
        VmaAllocationCreateInfo
        struct VmaAllocationCreateInfo VmaAllocationCreateInfo
        vmaAllocateMemoryForBuffer
        VkResult vmaAllocateMemoryForBuffer(VmaAllocator allocator, VkBuffer buffer, const VmaAllocationCreateInfo *pCreateInfo, VmaAllocation *pAllocation, VmaAllocationInfo *pAllocationInfo)
        -
        VmaPoolCreateFlagBits
        VmaPoolCreateFlagBits
        Flags to be passed as VmaPoolCreateInfo::flags.
        Definition: vk_mem_alloc.h:1000
        -
        VmaStatInfo::unusedRangeSizeAvg
        VkDeviceSize unusedRangeSizeAvg
        Definition: vk_mem_alloc.h:826
        -
        VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT
        Definition: vk_mem_alloc.h:943
        -
        VmaStats::memoryHeap
        VmaStatInfo memoryHeap[VK_MAX_MEMORY_HEAPS]
        Definition: vk_mem_alloc.h:833
        +
        VmaPoolCreateFlagBits
        VmaPoolCreateFlagBits
        Flags to be passed as VmaPoolCreateInfo::flags.
        Definition: vk_mem_alloc.h:1131
        +
        VmaStatInfo::unusedRangeSizeAvg
        VkDeviceSize unusedRangeSizeAvg
        Definition: vk_mem_alloc.h:920
        +
        VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT
        Definition: vk_mem_alloc.h:1061
        +
        VmaStats::memoryHeap
        VmaStatInfo memoryHeap[VK_MAX_MEMORY_HEAPS]
        Definition: vk_mem_alloc.h:927
        vmaDestroyBuffer
        void vmaDestroyBuffer(VmaAllocator allocator, VkBuffer buffer, VmaAllocation allocation)
        Destroys Vulkan buffer and frees allocated memory.
        -
        VmaPoolStats::unusedSize
        VkDeviceSize unusedSize
        Total number of bytes in the pool not used by any VmaAllocation.
        Definition: vk_mem_alloc.h:1075
        -
        VmaStatInfo::unusedRangeSizeMax
        VkDeviceSize unusedRangeSizeMax
        Definition: vk_mem_alloc.h:826
        -
        VmaAllocationInfo::memoryType
        uint32_t memoryType
        Memory type index that this allocation was allocated from.
        Definition: vk_mem_alloc.h:1139
        +
        VmaPoolStats::unusedSize
        VkDeviceSize unusedSize
        Total number of bytes in the pool not used by any VmaAllocation.
        Definition: vk_mem_alloc.h:1206
        +
        VmaStatInfo::unusedRangeSizeMax
        VkDeviceSize unusedRangeSizeMax
        Definition: vk_mem_alloc.h:920
        +
        VmaAllocationInfo::memoryType
        uint32_t memoryType
        Memory type index that this allocation was allocated from.
        Definition: vk_mem_alloc.h:1270
        diff --git a/docs/html/vk_khr_dedicated_allocation.html b/docs/html/vk_khr_dedicated_allocation.html index fded3c2..6b0002f 100644 --- a/docs/html/vk_khr_dedicated_allocation.html +++ b/docs/html/vk_khr_dedicated_allocation.html @@ -74,13 +74,7 @@ $(function() {
      • VK_KHR_dedicated_allocation

        • If you enabled these extensions:

        -

        2 . Query device for pointers to following 2 extension functions, using vkGetDeviceProcAddr(). Pass them in structure VmaVulkanFunctions while creating your VmaAllocator.

        -
          -
        • vkGetBufferMemoryRequirements2KHR
          • -
        • vkGetImageMemoryRequirements2KHR
          • -
        -

        Other members of this structure can be null as long as you leave VMA_STATIC_VULKAN_FUNCTIONS defined to 1, which is the default.

        -
        VmaVulkanFunctions vulkanFunctions = {};
        vulkanFunctions.vkGetBufferMemoryRequirements2KHR =
        (PFN_vkGetBufferMemoryRequirements2KHR)vkGetDeviceProcAddr(device, "vkGetBufferMemoryRequirements2KHR");
        vulkanFunctions.vkGetImageMemoryRequirements2KHR =
        (PFN_vkGetImageMemoryRequirements2KHR)vkGetDeviceProcAddr(device, "vkGetImageMemoryRequirements2KHR");
        VmaAllocatorCreateInfo allocatorInfo = {};
        allocatorInfo.pVulkanFunctions = &vulkanFunctions;
        // Fill other members of allocatorInfo...

        3 . Use VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT flag when creating your VmaAllocator to inform the library that you enabled required extensions and you want the library to use them.

        +

        2 . Use VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT flag when creating your VmaAllocator to inform the library that you enabled required extensions and you want the library to use them.

        allocatorInfo.flags |= VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT;
        vmaCreateAllocator(&allocatorInfo, &allocator);

        That's all. The extension will be automatically used whenever you create a buffer using vmaCreateBuffer() or image using vmaCreateImage().

        When using the extension together with Vulkan Validation Layer, you will receive warnings like this: 

        vkBindBufferMemory(): Binding memory to buffer 0x33 but vkGetBufferMemoryRequirements() has not been called on that buffer.

        It is OK, you should just ignore it. It happens because you use function vkGetBufferMemoryRequirements2KHR() instead of standard vkGetBufferMemoryRequirements(), while the validation layer seems to be unaware of it.

        diff --git a/src/vk_mem_alloc.h b/src/vk_mem_alloc.h index cb5680f..8b3fd76 100644 --- a/src/vk_mem_alloc.h +++ b/src/vk_mem_alloc.h @@ -40,6 +40,7 @@ Table of contents: - User guide - \subpage quick_start + - \subpage choosing_memory_type - \subpage memory_mapping - \subpage custom_memory_pools - \subpage defragmentation @@ -118,6 +119,121 @@ vmaDestroyBuffer(allocator, buffer, allocation); vmaDestroyAllocator(allocator); \endcode + +\page choosing_memory_type Choosing memory type + +Physical devices in Vulkan support various combinations of memory heaps and +types. Help with choosing correct and optimal memory type for your specific +resource is one of the key features of this library. You can use it by filling +appropriate members of VmaAllocationCreateInfo structure, as described below. +You can also combine multiple methods. + +-# If you just want to find memory type index that meets your requirements, you + can use function vmaFindMemoryTypeIndex(). +-# If you want to allocate a region of device memory without association with any + specific image or buffer, you can use function vmaAllocateMemory(). Usage of + this function is not recommended and usually not needed. +-# If you already have a buffer or an image created, you want to allocate memory + for it and then you will bind it yourself, you can use function + vmaAllocateMemoryForBuffer(), vmaAllocateMemoryForImage(). +-# If you want to create a buffer or an image, allocate memory for it and bind + them together, all in one call, you can use function vmaCreateBuffer(), + vmaCreateImage(). This is the recommended way to use this library. + +When using 3. or 4., the library internally queries Vulkan for memory types +supported for that buffer or image (function `vkGetBufferMemoryRequirements()`) +and uses only one of these types. + +If no memory type can be found that meets all the requirements, these functions +return `VK_ERROR_FEATURE_NOT_PRESENT`. + +You can leave VmaAllocationCreateInfo structure completely filled with zeros. +It means no requirements are specified for memory type. +It is valid, although not very useful. + +\section choosing_memory_type_usage Usage + +The easiest way to specify memory requirements is to fill member +VmaAllocationCreateInfo::usage using one of the values of enum `VmaMemoryUsage`. +It defines high level, common usage types. + +For example, if you want to create a uniform buffer that will be filled using +transfer only once or infrequently and used for rendering every frame, you can +do it using following code: + +\code +VkBufferCreateInfo bufferInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO }; +bufferInfo.size = 65536; +bufferInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT; + +VmaAllocationCreateInfo allocInfo = {}; +allocInfo.usage = VMA_MEMORY_USAGE_GPU_ONLY; + +VkBuffer buffer; +VmaAllocation allocation; +vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr); +\endcode + +\section choosing_memory_type_required_preferred_flags Required and preferred flags + +You can specify more detailed requirements by filling members +VmaAllocationCreateInfo::requiredFlags and VmaAllocationCreateInfo::preferredFlags +with a combination of bits from enum `VkMemoryPropertyFlags`. For example, +if you want to create a buffer that will be persistently mapped on host (so it +must be `HOST_VISIBLE`) and preferably will also be `HOST_COHERENT` and `HOST_CACHED`, +use following code: + +\code +VmaAllocationCreateInfo allocInfo = {}; +allocInfo.requiredFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT; +allocInfo.preferredFlags = VK_MEMORY_PROPERTY_HOST_COHERENT_BIT | VK_MEMORY_PROPERTY_HOST_CACHED_BIT; +allocInfo.flags = VMA_ALLOCATION_CREATE_MAPPED_BIT; + +VkBuffer buffer; +VmaAllocation allocation; +vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr); +\endcode + +A memory type is chosen that has all the required flags and as many preferred +flags set as possible. + +If you use VmaAllocationCreateInfo::usage, it is just internally converted to +a set of required and preferred flags. + +\section choosing_memory_type_explicit_memory_types Explicit memory types + +If you inspected memory types available on the physical device and you have +a preference for memory types that you want to use, you can fill member +VmaAllocationCreateInfo::memoryTypeBits. It is a bit mask, where each bit set +means that a memory type with that index is allowed to be used for the +allocation. Special value 0, just like UINT32_MAX, means there are no +restrictions to memory type index. + +Please note that this member is NOT just a memory type index. +Still you can use it to choose just one, specific memory type. +For example, if you already determined that your buffer should be created in +memory type 2, use following code: + +\code +uint32_t memoryTypeIndex = 2; + +VmaAllocationCreateInfo allocInfo = {}; +allocInfo.memoryTypeBits = 1u << memoryTypeIndex; + +VkBuffer buffer; +VmaAllocation allocation; +vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr); +\endcode + +\section choosing_memory_type_custom_memory_pools Custom memory pools + +If you allocate from custom memory pool, all the ways of specifying memory +requirements described above are not applicable and the aforementioned members +of VmaAllocationCreateInfo structure are ignored. Memory type is selected +explicitly when creating the pool and then used to make all the allocations from +that pool. For further details, see \ref custom_memory_pools. + + \page memory_mapping Memory mapping \section persistently_mapped_memory Persistently mapped memory @@ -839,18 +955,42 @@ VK_DEFINE_HANDLE(VmaPool) typedef enum VmaMemoryUsage { - /// No intended memory usage specified. Use other members of VmaAllocationCreateInfo to specify your requirements. + /** No intended memory usage specified. + Use other members of VmaAllocationCreateInfo to specify your requirements. + */ VMA_MEMORY_USAGE_UNKNOWN = 0, - /// Memory will be used on device only, so faster access from the device is preferred. No need to be mappable on host. + /** Memory will be used on device only, so faster access from the device is preferred. + It usually means device-local GPU memory. + No need to be mappable on host. + Good e.g. for images to be used as attachments, images containing textures to be sampled, + buffers used as vertex buffer, index buffer, uniform buffer and majority of + other types of resources used by device. + You can still do transfers from/to such resource to/from host memory. + + The allocation may still end up in `HOST_VISIBLE` memory on some implementations. + In such case, you are free to map it. + You can also use `VMA_ALLOCATION_CREATE_MAPPED_BIT` with this usage type. + */ VMA_MEMORY_USAGE_GPU_ONLY = 1, - /// Memory will be mapped on host. Could be used for transfer to/from device. - /** Guarantees to be `HOST_VISIBLE` and `HOST_COHERENT`. */ + /** Memory will be mapped and used on host. + It usually means CPU system memory. + Could be used for transfer to/from device. + Good e.g. for "staging" copy of buffers and images, used as transfer source or destination. + Resources created in this pool may still be accessible to the device, but access to them can be slower. + + Guarantees to be `HOST_VISIBLE` and `HOST_COHERENT`. + */ VMA_MEMORY_USAGE_CPU_ONLY = 2, - /// Memory will be used for frequent (dynamic) updates from host and reads on device (upload). - /** Guarantees to be `HOST_VISIBLE`. */ + /** Memory will be used for frequent (dynamic) updates from host and reads on device (upload). + Good e.g. for vertex buffers or uniform buffers updated every frame. + + Guarantees to be `HOST_VISIBLE`. + */ VMA_MEMORY_USAGE_CPU_TO_GPU = 3, - /// Memory will be used for frequent writing on device and readback on host (download). - /** Guarantees to be `HOST_VISIBLE`. */ + /** Memory will be used for frequent writing on device and readback on host (download). + + Guarantees to be `HOST_VISIBLE`. + */ VMA_MEMORY_USAGE_GPU_TO_CPU = 4, VMA_MEMORY_USAGE_MAX_ENUM = 0x7FFFFFFF } VmaMemoryUsage; @@ -930,28 +1070,41 @@ typedef struct VmaAllocationCreateInfo VmaAllocationCreateFlags flags; /** \brief Intended usage of memory. - Leave `VMA_MEMORY_USAGE_UNKNOWN` if you specify `requiredFlags`. You can also use both. \n + You can leave `VMA_MEMORY_USAGE_UNKNOWN` if you specify memory requirements in other way. \n If `pool` is not null, this member is ignored. */ VmaMemoryUsage usage; /** \brief Flags that must be set in a Memory Type chosen for an allocation. - Leave 0 if you specify requirement via usage. \n + Leave 0 if you specify memory requirements in other way. \n If `pool` is not null, this member is ignored.*/ VkMemoryPropertyFlags requiredFlags; - /** \brief Flags that preferably should be set in a Memory Type chosen for an allocation. + /** \brief Flags that preferably should be set in a memory type chosen for an allocation. - Set to 0 if no additional flags are prefered and only `requiredFlags` should be used. \n - If not 0, it must be a superset or equal to `requiredFlags`. \n + Set to 0 if no additional flags are prefered. \n If `pool` is not null, this member is ignored. */ VkMemoryPropertyFlags preferredFlags; - /** \brief Custom general-purpose pointer that will be stored in VmaAllocation, can be read as VmaAllocationInfo::pUserData and changed using vmaSetAllocationUserData(). */ - void* pUserData; + /** \brief Bitmask containing one bit set for every memory type acceptable for this allocation. + + Value 0 is equivalent to `UINT32_MAX` - it means any memory type is accepted if + it meets other requirements specified by this structure, with no further + restrictions on memory type index. \n + If `pool` is not null, this member is ignored. + */ + uint32_t memoryTypeBits; /** \brief Pool that this allocation should be created in. - Leave `VK_NULL_HANDLE` to allocate from general memory. + Leave `VK_NULL_HANDLE` to allocate from default pool. If not null, members: + `usage`, `requiredFlags`, `preferredFlags`, `memoryTypeBits` are ignored. */ VmaPool pool; + /** \brief Custom general-purpose pointer that will be stored in VmaAllocation, can be read as VmaAllocationInfo::pUserData and changed using vmaSetAllocationUserData(). + + If `VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT` is used, it must be either + null or pointer to a null-terminated string. The string will be then copied to + internal buffer, so it doesn't need to be valid after allocation call. + */ + void* pUserData; } VmaAllocationCreateInfo; /** @@ -1734,7 +1887,7 @@ static VkAllocationCallbacks VmaEmptyAllocationCallbacks = { VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL }; // Returns number of bits set to 1 in (v). -static inline uint32_t CountBitsSet(uint32_t v) +static inline uint32_t VmaCountBitsSet(uint32_t v) { uint32_t c = v - ((v >> 1) & 0x55555555); c = ((c >> 2) & 0x33333333) + (c & 0x33333333); @@ -7857,7 +8010,8 @@ void vmaFreeStatsString( #endif // #if VMA_STATS_STRING_ENABLED -/** This function is not protected by any mutex because it just reads immutable data. +/* +This function is not protected by any mutex because it just reads immutable data. */ VkResult vmaFindMemoryTypeIndex( VmaAllocator allocator, @@ -7868,15 +8022,14 @@ VkResult vmaFindMemoryTypeIndex( VMA_ASSERT(allocator != VK_NULL_HANDLE); VMA_ASSERT(pAllocationCreateInfo != VMA_NULL); VMA_ASSERT(pMemoryTypeIndex != VMA_NULL); + + if(pAllocationCreateInfo->memoryTypeBits != 0) + { + memoryTypeBits &= pAllocationCreateInfo->memoryTypeBits; + } uint32_t requiredFlags = pAllocationCreateInfo->requiredFlags; uint32_t preferredFlags = pAllocationCreateInfo->preferredFlags; - if(preferredFlags == 0) - { - preferredFlags = requiredFlags; - } - // preferredFlags, if not 0, must be a superset of requiredFlags. - VMA_ASSERT((requiredFlags & ~preferredFlags) == 0); // Convert usage to requiredFlags and preferredFlags. switch(pAllocationCreateInfo->usage) @@ -7916,7 +8069,7 @@ VkResult vmaFindMemoryTypeIndex( if((requiredFlags & ~currFlags) == 0) { // Calculate cost as number of bits from preferredFlags not present in this memory type. - uint32_t currCost = CountBitsSet(preferredFlags & ~currFlags); + uint32_t currCost = VmaCountBitsSet(preferredFlags & ~currFlags); // Remember memory type with lowest cost. if(currCost < minCost) {
        Enumerator
        VMA_MEMORY_USAGE_UNKNOWN 

        No intended memory usage specified. Use other members of VmaAllocationCreateInfo to specify your requirements.

        VMA_MEMORY_USAGE_GPU_ONLY 

        Memory will be used on device only, so faster access from the device is preferred. No need to be mappable on host.

        +
        VMA_MEMORY_USAGE_GPU_ONLY 

        Memory will be used on device only, so faster access from the device is preferred. It usually means device-local GPU memory. No need to be mappable on host. Good e.g. for images to be used as attachments, images containing textures to be sampled, buffers used as vertex buffer, index buffer, uniform buffer and majority of other types of resources used by device. You can still do transfers from/to such resource to/from host memory.

        +

        The allocation may still end up in HOST_VISIBLE memory on some implementations. In such case, you are free to map it. You can also use VMA_ALLOCATION_CREATE_MAPPED_BIT with this usage type.

        VMA_MEMORY_USAGE_CPU_ONLY 

        Memory will be mapped on host. Could be used for transfer to/from device.

        +
        VMA_MEMORY_USAGE_CPU_ONLY 

        Memory will be mapped and used on host. It usually means CPU system memory. Could be used for transfer to/from device. Good e.g. for "staging" copy of buffers and images, used as transfer source or destination. Resources created in this pool may still be accessible to the device, but access to them can be slower.

        Guarantees to be HOST_VISIBLE and HOST_COHERENT.

        VMA_MEMORY_USAGE_CPU_TO_GPU 

        Memory will be used for frequent (dynamic) updates from host and reads on device (upload).

        +
        VMA_MEMORY_USAGE_CPU_TO_GPU 

        Memory will be used for frequent (dynamic) updates from host and reads on device (upload). Good e.g. for vertex buffers or uniform buffers updated every frame.

        Guarantees to be HOST_VISIBLE.

        VMA_MEMORY_USAGE_GPU_TO_CPU 

        Memory will be used for frequent writing on device and readback on host (download).

        +
        VMA_MEMORY_USAGE_GPU_TO_CPU 

        Memory will be used for frequent writing on device and readback on host (download).

        Guarantees to be HOST_VISIBLE.

        VMA_MEMORY_USAGE_MAX_ENUM