diff --git a/docs/html/functions.html b/docs/html/functions.html index 1ee5475..24ff073 100644 --- a/docs/html/functions.html +++ b/docs/html/functions.html @@ -279,15 +279,9 @@ $(function() {
  • vkGetBufferMemoryRequirements : VmaVulkanFunctions
    • -
  • vkGetBufferMemoryRequirements2KHR -: VmaVulkanFunctions -
  • vkGetImageMemoryRequirements : VmaVulkanFunctions
    • -
  • vkGetImageMemoryRequirements2KHR -: VmaVulkanFunctions -
  • vkGetPhysicalDeviceMemoryProperties : VmaVulkanFunctions
    • diff --git a/docs/html/functions_vars.html b/docs/html/functions_vars.html index 9a0ee19..c7b0790 100644 --- a/docs/html/functions_vars.html +++ b/docs/html/functions_vars.html @@ -279,15 +279,9 @@ $(function() {
  • vkGetBufferMemoryRequirements : VmaVulkanFunctions
    • -
  • vkGetBufferMemoryRequirements2KHR -: VmaVulkanFunctions -
  • vkGetImageMemoryRequirements : VmaVulkanFunctions
    • -
  • vkGetImageMemoryRequirements2KHR -: VmaVulkanFunctions -
  • vkGetPhysicalDeviceMemoryProperties : VmaVulkanFunctions
    • diff --git a/docs/html/general_considerations.html b/docs/html/general_considerations.html index 139d6e6..5d4362d 100644 --- a/docs/html/general_considerations.html +++ b/docs/html/general_considerations.html @@ -91,6 +91,7 @@ Features not supported

    Features deliberately excluded from the scope of this library:

    diff --git a/docs/html/globals.html b/docs/html/globals.html index e94b352..d583338 100644 --- a/docs/html/globals.html +++ b/docs/html/globals.html @@ -101,6 +101,9 @@ $(function() {
  • VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT : vk_mem_alloc.h
    • +
  • VMA_DEDICATED_ALLOCATION +: vk_mem_alloc.h +
  • VMA_MEMORY_USAGE_CPU_ONLY : vk_mem_alloc.h
    • diff --git a/docs/html/globals_defs.html b/docs/html/globals_defs.html index 95a5846..aa8115b 100644 --- a/docs/html/globals_defs.html +++ b/docs/html/globals_defs.html @@ -59,6 +59,9 @@ $(function() {
     
    - -

    ◆ vkGetBufferMemoryRequirements2KHR

    - -
    -
    - - - - -
    PFN_vkGetBufferMemoryRequirements2KHR VmaVulkanFunctions::vkGetBufferMemoryRequirements2KHR
    -
    -
    @@ -262,20 +244,6 @@ Public Attributes
    -
    - - -

    ◆ vkGetImageMemoryRequirements2KHR

    - -
    -
    - - - - -
    PFN_vkGetImageMemoryRequirements2KHR VmaVulkanFunctions::vkGetImageMemoryRequirements2KHR
    -
    -
    diff --git a/docs/html/usage_patterns.html b/docs/html/usage_patterns.html index 314a272..0495f56 100644 --- a/docs/html/usage_patterns.html +++ b/docs/html/usage_patterns.html @@ -115,7 +115,7 @@ Direct access versus transfer
  • Create just single copy using VMA_MEMORY_USAGE_GPU_TO_CPU, write to it directly on GPU, map it and read it on CPU.

    • You should take some measurements to decide which option is faster in case of your specific resource.

    -

    If you don't want to specialize your code for specific types of GPUs, yon can still make an simple optimization for cases when your resource ends up in mappable memory to use it directly in this case instead of creating CPU-side staging copy. For details see Finding out if memory is mappable.

    +

    If you don't want to specialize your code for specific types of GPUs, you can still make an simple optimization for cases when your resource ends up in mappable memory to use it directly in this case instead of creating CPU-side staging copy. For details see Finding out if memory is mappable.

    diff --git a/docs/html/vk__mem__alloc_8h.html b/docs/html/vk__mem__alloc_8h.html index 6fa4720..90dd3a3 100644 --- a/docs/html/vk__mem__alloc_8h.html +++ b/docs/html/vk__mem__alloc_8h.html @@ -109,6 +109,8 @@ Classes + +

    Macros

    #define VMA_DEDICATED_ALLOCATION   0
     
    #define VMA_STATS_STRING_ENABLED   1
     
    @@ -306,6 +308,20 @@ Functions
     

    Macro Definition Documentation

    + +

    ◆ VMA_DEDICATED_ALLOCATION

    + +
    +
    + + + + +
    #define VMA_DEDICATED_ALLOCATION   0
    +
    + +
    +

    ◆ VMA_STATS_STRING_ENABLED

    @@ -731,14 +747,14 @@ Functions

    Usage:

    Allocation may still end up in HOST_VISIBLE memory on some implementations. In such case, you are free to map it. You can use VMA_ALLOCATION_CREATE_MAPPED_BIT with this usage type.

    -VMA_MEMORY_USAGE_CPU_ONLY 

    Memory will be mappable on host. It usually means CPU (system) memory. Resources created in this pool may still be accessible to the device, but access to them can be slower. Guarantees to be HOST_VISIBLE and HOST_COHERENT. CPU read may be uncached. It is roughly equivalent of D3D12_HEAP_TYPE_UPLOAD.

    +VMA_MEMORY_USAGE_CPU_ONLY 

    Memory will be mappable on host. It usually means CPU (system) memory. Guarantees to be HOST_VISIBLE and HOST_COHERENT. CPU access is typically uncached. Writes may be write-combined. Resources created in this pool may still be accessible to the device, but access to them can be slow. It is roughly equivalent of D3D12_HEAP_TYPE_UPLOAD.

    Usage: Staging copy of resources used as transfer source.

    -VMA_MEMORY_USAGE_CPU_TO_GPU 

    Memory that is both mappable on host (guarantees to be HOST_VISIBLE) and preferably fast to access by GPU. CPU reads may be uncached and very slow.

    +VMA_MEMORY_USAGE_CPU_TO_GPU 

    Memory that is both mappable on host (guarantees to be HOST_VISIBLE) and preferably fast to access by GPU. CPU access is typically uncached. Writes may be write-combined.

    Usage: Resources written frequently by host (dynamic), read by device. E.g. textures, vertex buffers, uniform buffers updated every frame or every draw call.

    VMA_MEMORY_USAGE_GPU_TO_CPU 

    Memory mappable on host (guarantees to be HOST_VISIBLE) and cached. It is roughly equivalent of D3D12_HEAP_TYPE_READBACK.

    diff --git a/docs/html/vk__mem__alloc_8h_source.html b/docs/html/vk__mem__alloc_8h_source.html index f66dc35..0a0e08e 100644 --- a/docs/html/vk__mem__alloc_8h_source.html +++ b/docs/html/vk__mem__alloc_8h_source.html @@ -62,162 +62,160 @@ $(function() {
    vk_mem_alloc.h
    -Go to the documentation of this file.
    1 //
    2 // Copyright (c) 2017-2018 Advanced Micro Devices, Inc. All rights reserved.
    3 //
    4 // Permission is hereby granted, free of charge, to any person obtaining a copy
    5 // of this software and associated documentation files (the "Software"), to deal
    6 // in the Software without restriction, including without limitation the rights
    7 // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
    8 // copies of the Software, and to permit persons to whom the Software is
    9 // furnished to do so, subject to the following conditions:
    10 //
    11 // The above copyright notice and this permission notice shall be included in
    12 // all copies or substantial portions of the Software.
    13 //
    14 // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    15 // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    16 // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
    17 // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    18 // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
    19 // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
    20 // THE SOFTWARE.
    21 //
    22 
    23 #ifndef AMD_VULKAN_MEMORY_ALLOCATOR_H
    24 #define AMD_VULKAN_MEMORY_ALLOCATOR_H
    25 
    26 #ifdef __cplusplus
    27 extern "C" {
    28 #endif
    29 
    1078 #include <vulkan/vulkan.h>
    1079 
    1089 VK_DEFINE_HANDLE(VmaAllocator)
    1090 
    1091 typedef void (VKAPI_PTR *PFN_vmaAllocateDeviceMemoryFunction)(
    1093  VmaAllocator allocator,
    1094  uint32_t memoryType,
    1095  VkDeviceMemory memory,
    1096  VkDeviceSize size);
    1098 typedef void (VKAPI_PTR *PFN_vmaFreeDeviceMemoryFunction)(
    1099  VmaAllocator allocator,
    1100  uint32_t memoryType,
    1101  VkDeviceMemory memory,
    1102  VkDeviceSize size);
    1103 
    1111 typedef struct VmaDeviceMemoryCallbacks {
    1113  PFN_vmaAllocateDeviceMemoryFunction pfnAllocate;
    1115  PFN_vmaFreeDeviceMemoryFunction pfnFree;
    1116 } VmaDeviceMemoryCallbacks;
    1117 
    1119 typedef enum VmaAllocatorCreateFlagBits {
    1124  VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT = 0x00000001,
    1146  VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT = 0x00000002,
    1147 
    1148  VMA_ALLOCATOR_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
    1149 } VmaAllocatorCreateFlagBits;
    1150 typedef VkFlags VmaAllocatorCreateFlags;
    1151 
    1156 typedef struct VmaVulkanFunctions {
    1157  PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties;
    1158  PFN_vkGetPhysicalDeviceMemoryProperties vkGetPhysicalDeviceMemoryProperties;
    1159  PFN_vkAllocateMemory vkAllocateMemory;
    1160  PFN_vkFreeMemory vkFreeMemory;
    1161  PFN_vkMapMemory vkMapMemory;
    1162  PFN_vkUnmapMemory vkUnmapMemory;
    1163  PFN_vkBindBufferMemory vkBindBufferMemory;
    1164  PFN_vkBindImageMemory vkBindImageMemory;
    1165  PFN_vkGetBufferMemoryRequirements vkGetBufferMemoryRequirements;
    1166  PFN_vkGetImageMemoryRequirements vkGetImageMemoryRequirements;
    1167  PFN_vkCreateBuffer vkCreateBuffer;
    1168  PFN_vkDestroyBuffer vkDestroyBuffer;
    1169  PFN_vkCreateImage vkCreateImage;
    1170  PFN_vkDestroyImage vkDestroyImage;
    1171  PFN_vkGetBufferMemoryRequirements2KHR vkGetBufferMemoryRequirements2KHR;
    1172  PFN_vkGetImageMemoryRequirements2KHR vkGetImageMemoryRequirements2KHR;
    1173 } VmaVulkanFunctions;
    1174 
    1176 typedef struct VmaAllocatorCreateInfo
    1177 {
    1179  VmaAllocatorCreateFlags flags;
    1181 
    1182  VkPhysicalDevice physicalDevice;
    1184 
    1185  VkDevice device;
    1187 
    1188  VkDeviceSize preferredLargeHeapBlockSize;
    1190 
    1191  const VkAllocationCallbacks* pAllocationCallbacks;
    1193 
    1194  const VmaDeviceMemoryCallbacks* pDeviceMemoryCallbacks;
    1208  uint32_t frameInUseCount;
    1232  const VkDeviceSize* pHeapSizeLimit;
    1244  const VmaVulkanFunctions* pVulkanFunctions;
    1245 } VmaAllocatorCreateInfo;
    1246 
    1248 VkResult vmaCreateAllocator(
    1249  const VmaAllocatorCreateInfo* pCreateInfo,
    1250  VmaAllocator* pAllocator);
    1251 
    1253 void vmaDestroyAllocator(
    1254  VmaAllocator allocator);
    1255 
    1260 void vmaGetPhysicalDeviceProperties(
    1261  VmaAllocator allocator,
    1262  const VkPhysicalDeviceProperties** ppPhysicalDeviceProperties);
    1263 
    1268 void vmaGetMemoryProperties(
    1269  VmaAllocator allocator,
    1270  const VkPhysicalDeviceMemoryProperties** ppPhysicalDeviceMemoryProperties);
    1271 
    1278 void vmaGetMemoryTypeProperties(
    1279  VmaAllocator allocator,
    1280  uint32_t memoryTypeIndex,
    1281  VkMemoryPropertyFlags* pFlags);
    1282 
    1291 void vmaSetCurrentFrameIndex(
    1292  VmaAllocator allocator,
    1293  uint32_t frameIndex);
    1294 
    1297 typedef struct VmaStatInfo
    1298 {
    1300  uint32_t blockCount;
    1302  uint32_t allocationCount;
    1304  uint32_t unusedRangeCount;
    1306  VkDeviceSize usedBytes;
    1308  VkDeviceSize unusedBytes;
    1309  VkDeviceSize allocationSizeMin, allocationSizeAvg, allocationSizeMax;
    1310  VkDeviceSize unusedRangeSizeMin, unusedRangeSizeAvg, unusedRangeSizeMax;
    1311 } VmaStatInfo;
    1312 
    1314 typedef struct VmaStats
    1315 {
    1316  VmaStatInfo memoryType[VK_MAX_MEMORY_TYPES];
    1317  VmaStatInfo memoryHeap[VK_MAX_MEMORY_HEAPS];
    1318  VmaStatInfo total;
    1319 } VmaStats;
    1320 
    1322 void vmaCalculateStats(
    1323  VmaAllocator allocator,
    1324  VmaStats* pStats);
    1325 
    1326 #define VMA_STATS_STRING_ENABLED 1
    1327 
    1328 #if VMA_STATS_STRING_ENABLED
    1329 
    1331 
    1333 void vmaBuildStatsString(
    1334  VmaAllocator allocator,
    1335  char** ppStatsString,
    1336  VkBool32 detailedMap);
    1337 
    1338 void vmaFreeStatsString(
    1339  VmaAllocator allocator,
    1340  char* pStatsString);
    1341 
    1342 #endif // #if VMA_STATS_STRING_ENABLED
    1343 
    1352 VK_DEFINE_HANDLE(VmaPool)
    1353 
    1354 typedef enum VmaMemoryUsage
    1355 {
    1359  VMA_MEMORY_USAGE_UNKNOWN = 0,
    1376  VMA_MEMORY_USAGE_GPU_ONLY = 1,
    1386  VMA_MEMORY_USAGE_CPU_ONLY = 2,
    1393  VMA_MEMORY_USAGE_CPU_TO_GPU = 3,
    1402  VMA_MEMORY_USAGE_GPU_TO_CPU = 4,
    1403  VMA_MEMORY_USAGE_MAX_ENUM = 0x7FFFFFFF
    1404 } VmaMemoryUsage;
    1405 
    1407 typedef enum VmaAllocationCreateFlagBits {
    1419  VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT = 0x00000001,
    1420 
    1430  VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT = 0x00000002,
    1443  VMA_ALLOCATION_CREATE_MAPPED_BIT = 0x00000004,
    1456  VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT = 0x00000008,
    1463  VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT = 0x00000010,
    1469  VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT = 0x00000020,
    1470 
    1471  VMA_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
    1472 } VmaAllocationCreateFlagBits;
    1473 typedef VkFlags VmaAllocationCreateFlags;
    1474 
    1475 typedef struct VmaAllocationCreateInfo
    1476 {
    1478  VmaAllocationCreateFlags flags;
    1484  VmaMemoryUsage usage;
    1489  VkMemoryPropertyFlags requiredFlags;
    1494  VkMemoryPropertyFlags preferredFlags;
    1502  uint32_t memoryTypeBits;
    1508  VmaPool pool;
    1515  void* pUserData;
    1516 } VmaAllocationCreateInfo;
    1517 
    1534 VkResult vmaFindMemoryTypeIndex(
    1535  VmaAllocator allocator,
    1536  uint32_t memoryTypeBits,
    1537  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    1538  uint32_t* pMemoryTypeIndex);
    1539 
    1552 VkResult vmaFindMemoryTypeIndexForBufferInfo(
    1553  VmaAllocator allocator,
    1554  const VkBufferCreateInfo* pBufferCreateInfo,
    1555  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    1556  uint32_t* pMemoryTypeIndex);
    1557 
    1570 VkResult vmaFindMemoryTypeIndexForImageInfo(
    1571  VmaAllocator allocator,
    1572  const VkImageCreateInfo* pImageCreateInfo,
    1573  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    1574  uint32_t* pMemoryTypeIndex);
    1575 
    1577 typedef enum VmaPoolCreateFlagBits {
    1595  VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT = 0x00000002,
    1596 
    1597  VMA_POOL_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
    1598 } VmaPoolCreateFlagBits;
    1599 typedef VkFlags VmaPoolCreateFlags;
    1600 
    1603 typedef struct VmaPoolCreateInfo {
    1606  uint32_t memoryTypeIndex;
    1609  VmaPoolCreateFlags flags;
    1614  VkDeviceSize blockSize;
    1619  size_t minBlockCount;
    1627  size_t maxBlockCount;
    1641  uint32_t frameInUseCount;
    1642 } VmaPoolCreateInfo;
    1643 
    1646 typedef struct VmaPoolStats {
    1649  VkDeviceSize size;
    1652  VkDeviceSize unusedSize;
    1655  size_t allocationCount;
    1658  size_t unusedRangeCount;
    1665  VkDeviceSize unusedRangeSizeMax;
    1666 } VmaPoolStats;
    1667 
    1674 VkResult vmaCreatePool(
    1675  VmaAllocator allocator,
    1676  const VmaPoolCreateInfo* pCreateInfo,
    1677  VmaPool* pPool);
    1678 
    1681 void vmaDestroyPool(
    1682  VmaAllocator allocator,
    1683  VmaPool pool);
    1684 
    1691 void vmaGetPoolStats(
    1692  VmaAllocator allocator,
    1693  VmaPool pool,
    1694  VmaPoolStats* pPoolStats);
    1695 
    1702 void vmaMakePoolAllocationsLost(
    1703  VmaAllocator allocator,
    1704  VmaPool pool,
    1705  size_t* pLostAllocationCount);
    1706 
    1731 VK_DEFINE_HANDLE(VmaAllocation)
    1732 
    1733 
    1735 typedef struct VmaAllocationInfo {
    1740  uint32_t memoryType;
    1749  VkDeviceMemory deviceMemory;
    1754  VkDeviceSize offset;
    1759  VkDeviceSize size;
    1768  void* pMappedData;
    1773  void* pUserData;
    1774 } VmaAllocationInfo;
    1775 
    1786 VkResult vmaAllocateMemory(
    1787  VmaAllocator allocator,
    1788  const VkMemoryRequirements* pVkMemoryRequirements,
    1789  const VmaAllocationCreateInfo* pCreateInfo,
    1790  VmaAllocation* pAllocation,
    1791  VmaAllocationInfo* pAllocationInfo);
    1792 
    1799 VkResult vmaAllocateMemoryForBuffer(
    1800  VmaAllocator allocator,
    1801  VkBuffer buffer,
    1802  const VmaAllocationCreateInfo* pCreateInfo,
    1803  VmaAllocation* pAllocation,
    1804  VmaAllocationInfo* pAllocationInfo);
    1805 
    1807 VkResult vmaAllocateMemoryForImage(
    1808  VmaAllocator allocator,
    1809  VkImage image,
    1810  const VmaAllocationCreateInfo* pCreateInfo,
    1811  VmaAllocation* pAllocation,
    1812  VmaAllocationInfo* pAllocationInfo);
    1813 
    1815 void vmaFreeMemory(
    1816  VmaAllocator allocator,
    1817  VmaAllocation allocation);
    1818 
    1835 void vmaGetAllocationInfo(
    1836  VmaAllocator allocator,
    1837  VmaAllocation allocation,
    1838  VmaAllocationInfo* pAllocationInfo);
    1839 
    1854 VkBool32 vmaTouchAllocation(
    1855  VmaAllocator allocator,
    1856  VmaAllocation allocation);
    1857 
    1871 void vmaSetAllocationUserData(
    1872  VmaAllocator allocator,
    1873  VmaAllocation allocation,
    1874  void* pUserData);
    1875 
    1886 void vmaCreateLostAllocation(
    1887  VmaAllocator allocator,
    1888  VmaAllocation* pAllocation);
    1889 
    1924 VkResult vmaMapMemory(
    1925  VmaAllocator allocator,
    1926  VmaAllocation allocation,
    1927  void** ppData);
    1928 
    1933 void vmaUnmapMemory(
    1934  VmaAllocator allocator,
    1935  VmaAllocation allocation);
    1936 
    1938 typedef struct VmaDefragmentationInfo {
    1943  VkDeviceSize maxBytesToMove;
    1948  uint32_t maxAllocationsToMove;
    1949 } VmaDefragmentationInfo;
    1950 
    1952 typedef struct VmaDefragmentationStats {
    1954  VkDeviceSize bytesMoved;
    1956  VkDeviceSize bytesFreed;
    1958  uint32_t allocationsMoved;
    1960  uint32_t deviceMemoryBlocksFreed;
    1961 } VmaDefragmentationStats;
    1962 
    2045 VkResult vmaDefragment(
    2046  VmaAllocator allocator,
    2047  VmaAllocation* pAllocations,
    2048  size_t allocationCount,
    2049  VkBool32* pAllocationsChanged,
    2050  const VmaDefragmentationInfo *pDefragmentationInfo,
    2051  VmaDefragmentationStats* pDefragmentationStats);
    2052 
    2065 VkResult vmaBindBufferMemory(
    2066  VmaAllocator allocator,
    2067  VmaAllocation allocation,
    2068  VkBuffer buffer);
    2069 
    2082 VkResult vmaBindImageMemory(
    2083  VmaAllocator allocator,
    2084  VmaAllocation allocation,
    2085  VkImage image);
    2086 
    2113 VkResult vmaCreateBuffer(
    2114  VmaAllocator allocator,
    2115  const VkBufferCreateInfo* pBufferCreateInfo,
    2116  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    2117  VkBuffer* pBuffer,
    2118  VmaAllocation* pAllocation,
    2119  VmaAllocationInfo* pAllocationInfo);
    2120 
    2132 void vmaDestroyBuffer(
    2133  VmaAllocator allocator,
    2134  VkBuffer buffer,
    2135  VmaAllocation allocation);
    2136 
    2138 VkResult vmaCreateImage(
    2139  VmaAllocator allocator,
    2140  const VkImageCreateInfo* pImageCreateInfo,
    2141  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    2142  VkImage* pImage,
    2143  VmaAllocation* pAllocation,
    2144  VmaAllocationInfo* pAllocationInfo);
    2145 
    2157 void vmaDestroyImage(
    2158  VmaAllocator allocator,
    2159  VkImage image,
    2160  VmaAllocation allocation);
    2161 
    2162 #ifdef __cplusplus
    2163 }
    2164 #endif
    2165 
    2166 #endif // AMD_VULKAN_MEMORY_ALLOCATOR_H
    2167 
    2168 // For Visual Studio IntelliSense.
    2169 #ifdef __INTELLISENSE__
    2170 #define VMA_IMPLEMENTATION
    2171 #endif
    2172 
    2173 #ifdef VMA_IMPLEMENTATION
    2174 #undef VMA_IMPLEMENTATION
    2175 
    2176 #include <cstdint>
    2177 #include <cstdlib>
    2178 #include <cstring>
    2179 
    2180 /*******************************************************************************
    2181 CONFIGURATION SECTION
    2182 
    2183 Define some of these macros before each #include of this header or change them
    2184 here if you need other then default behavior depending on your environment.
    2185 */
    2186 
    2187 /*
    2188 Define this macro to 1 to make the library fetch pointers to Vulkan functions
    2189 internally, like:
    2190 
    2191  vulkanFunctions.vkAllocateMemory = &vkAllocateMemory;
    2192 
    2193 Define to 0 if you are going to provide you own pointers to Vulkan functions via
    2194 VmaAllocatorCreateInfo::pVulkanFunctions.
    2195 */
    2196 #if !defined(VMA_STATIC_VULKAN_FUNCTIONS) && !defined(VK_NO_PROTOTYPES)
    2197 #define VMA_STATIC_VULKAN_FUNCTIONS 1
    2198 #endif
    2199 
    2200 // Define this macro to 1 to make the library use STL containers instead of its own implementation.
    2201 //#define VMA_USE_STL_CONTAINERS 1
    2202 
    2203 /* Set this macro to 1 to make the library including and using STL containers:
    2204 std::pair, std::vector, std::list, std::unordered_map.
    2205 
    2206 Set it to 0 or undefined to make the library using its own implementation of
    2207 the containers.
    2208 */
    2209 #if VMA_USE_STL_CONTAINERS
    2210  #define VMA_USE_STL_VECTOR 1
    2211  #define VMA_USE_STL_UNORDERED_MAP 1
    2212  #define VMA_USE_STL_LIST 1
    2213 #endif
    2214 
    2215 #if VMA_USE_STL_VECTOR
    2216  #include <vector>
    2217 #endif
    2218 
    2219 #if VMA_USE_STL_UNORDERED_MAP
    2220  #include <unordered_map>
    2221 #endif
    2222 
    2223 #if VMA_USE_STL_LIST
    2224  #include <list>
    2225 #endif
    2226 
    2227 /*
    2228 Following headers are used in this CONFIGURATION section only, so feel free to
    2229 remove them if not needed.
    2230 */
    2231 #include <cassert> // for assert
    2232 #include <algorithm> // for min, max
    2233 #include <mutex> // for std::mutex
    2234 #include <atomic> // for std::atomic
    2235 
    2236 #if !defined(_WIN32) && !defined(__APPLE__)
    2237  #include <malloc.h> // for aligned_alloc()
    2238 #endif
    2239 
    2240 #ifndef VMA_NULL
    2241  // Value used as null pointer. Define it to e.g.: nullptr, NULL, 0, (void*)0.
    2242  #define VMA_NULL nullptr
    2243 #endif
    2244 
    2245 #if defined(__APPLE__) || defined(__ANDROID__)
    2246 #include <cstdlib>
    2247 void *aligned_alloc(size_t alignment, size_t size)
    2248 {
    2249  // alignment must be >= sizeof(void*)
    2250  if(alignment < sizeof(void*))
    2251  {
    2252  alignment = sizeof(void*);
    2253  }
    2254 
    2255  void *pointer;
    2256  if(posix_memalign(&pointer, alignment, size) == 0)
    2257  return pointer;
    2258  return VMA_NULL;
    2259 }
    2260 #endif
    2261 
    2262 // Normal assert to check for programmer's errors, especially in Debug configuration.
    2263 #ifndef VMA_ASSERT
    2264  #ifdef _DEBUG
    2265  #define VMA_ASSERT(expr) assert(expr)
    2266  #else
    2267  #define VMA_ASSERT(expr)
    2268  #endif
    2269 #endif
    2270 
    2271 // Assert that will be called very often, like inside data structures e.g. operator[].
    2272 // Making it non-empty can make program slow.
    2273 #ifndef VMA_HEAVY_ASSERT
    2274  #ifdef _DEBUG
    2275  #define VMA_HEAVY_ASSERT(expr) //VMA_ASSERT(expr)
    2276  #else
    2277  #define VMA_HEAVY_ASSERT(expr)
    2278  #endif
    2279 #endif
    2280 
    2281 #ifndef VMA_ALIGN_OF
    2282  #define VMA_ALIGN_OF(type) (__alignof(type))
    2283 #endif
    2284 
    2285 #ifndef VMA_SYSTEM_ALIGNED_MALLOC
    2286  #if defined(_WIN32)
    2287  #define VMA_SYSTEM_ALIGNED_MALLOC(size, alignment) (_aligned_malloc((size), (alignment)))
    2288  #else
    2289  #define VMA_SYSTEM_ALIGNED_MALLOC(size, alignment) (aligned_alloc((alignment), (size) ))
    2290  #endif
    2291 #endif
    2292 
    2293 #ifndef VMA_SYSTEM_FREE
    2294  #if defined(_WIN32)
    2295  #define VMA_SYSTEM_FREE(ptr) _aligned_free(ptr)
    2296  #else
    2297  #define VMA_SYSTEM_FREE(ptr) free(ptr)
    2298  #endif
    2299 #endif
    2300 
    2301 #ifndef VMA_MIN
    2302  #define VMA_MIN(v1, v2) (std::min((v1), (v2)))
    2303 #endif
    2304 
    2305 #ifndef VMA_MAX
    2306  #define VMA_MAX(v1, v2) (std::max((v1), (v2)))
    2307 #endif
    2308 
    2309 #ifndef VMA_SWAP
    2310  #define VMA_SWAP(v1, v2) std::swap((v1), (v2))
    2311 #endif
    2312 
    2313 #ifndef VMA_SORT
    2314  #define VMA_SORT(beg, end, cmp) std::sort(beg, end, cmp)
    2315 #endif
    2316 
    2317 #ifndef VMA_DEBUG_LOG
    2318  #define VMA_DEBUG_LOG(format, ...)
    2319  /*
    2320  #define VMA_DEBUG_LOG(format, ...) do { \
    2321  printf(format, __VA_ARGS__); \
    2322  printf("\n"); \
    2323  } while(false)
    2324  */
    2325 #endif
    2326 
    2327 // Define this macro to 1 to enable functions: vmaBuildStatsString, vmaFreeStatsString.
    2328 #if VMA_STATS_STRING_ENABLED
    2329  static inline void VmaUint32ToStr(char* outStr, size_t strLen, uint32_t num)
    2330  {
    2331  snprintf(outStr, strLen, "%u", static_cast<unsigned int>(num));
    2332  }
    2333  static inline void VmaUint64ToStr(char* outStr, size_t strLen, uint64_t num)
    2334  {
    2335  snprintf(outStr, strLen, "%llu", static_cast<unsigned long long>(num));
    2336  }
    2337  static inline void VmaPtrToStr(char* outStr, size_t strLen, const void* ptr)
    2338  {
    2339  snprintf(outStr, strLen, "%p", ptr);
    2340  }
    2341 #endif
    2342 
    2343 #ifndef VMA_MUTEX
    2344  class VmaMutex
    2345  {
    2346  public:
    2347  VmaMutex() { }
    2348  ~VmaMutex() { }
    2349  void Lock() { m_Mutex.lock(); }
    2350  void Unlock() { m_Mutex.unlock(); }
    2351  private:
    2352  std::mutex m_Mutex;
    2353  };
    2354  #define VMA_MUTEX VmaMutex
    2355 #endif
    2356 
    2357 /*
    2358 If providing your own implementation, you need to implement a subset of std::atomic:
    2359 
    2360 - Constructor(uint32_t desired)
    2361 - uint32_t load() const
    2362 - void store(uint32_t desired)
    2363 - bool compare_exchange_weak(uint32_t& expected, uint32_t desired)
    2364 */
    2365 #ifndef VMA_ATOMIC_UINT32
    2366  #define VMA_ATOMIC_UINT32 std::atomic<uint32_t>
    2367 #endif
    2368 
    2369 #ifndef VMA_BEST_FIT
    2370 
    2382  #define VMA_BEST_FIT (1)
    2383 #endif
    2384 
    2385 #ifndef VMA_DEBUG_ALWAYS_DEDICATED_MEMORY
    2386 
    2390  #define VMA_DEBUG_ALWAYS_DEDICATED_MEMORY (0)
    2391 #endif
    2392 
    2393 #ifndef VMA_DEBUG_ALIGNMENT
    2394 
    2398  #define VMA_DEBUG_ALIGNMENT (1)
    2399 #endif
    2400 
    2401 #ifndef VMA_DEBUG_MARGIN
    2402 
    2406  #define VMA_DEBUG_MARGIN (0)
    2407 #endif
    2408 
    2409 #ifndef VMA_DEBUG_GLOBAL_MUTEX
    2410 
    2414  #define VMA_DEBUG_GLOBAL_MUTEX (0)
    2415 #endif
    2416 
    2417 #ifndef VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY
    2418 
    2422  #define VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY (1)
    2423 #endif
    2424 
    2425 #ifndef VMA_SMALL_HEAP_MAX_SIZE
    2426  #define VMA_SMALL_HEAP_MAX_SIZE (1024ull * 1024 * 1024)
    2428 #endif
    2429 
    2430 #ifndef VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE
    2431  #define VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE (256ull * 1024 * 1024)
    2433 #endif
    2434 
    2435 static const uint32_t VMA_FRAME_INDEX_LOST = UINT32_MAX;
    2436 
    2437 /*******************************************************************************
    2438 END OF CONFIGURATION
    2439 */
    2440 
    2441 static VkAllocationCallbacks VmaEmptyAllocationCallbacks = {
    2442  VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL };
    2443 
    2444 // Returns number of bits set to 1 in (v).
    2445 static inline uint32_t VmaCountBitsSet(uint32_t v)
    2446 {
    2447  uint32_t c = v - ((v >> 1) & 0x55555555);
    2448  c = ((c >> 2) & 0x33333333) + (c & 0x33333333);
    2449  c = ((c >> 4) + c) & 0x0F0F0F0F;
    2450  c = ((c >> 8) + c) & 0x00FF00FF;
    2451  c = ((c >> 16) + c) & 0x0000FFFF;
    2452  return c;
    2453 }
    2454 
    2455 // Aligns given value up to nearest multiply of align value. For example: VmaAlignUp(11, 8) = 16.
    2456 // Use types like uint32_t, uint64_t as T.
    2457 template <typename T>
    2458 static inline T VmaAlignUp(T val, T align)
    2459 {
    2460  return (val + align - 1) / align * align;
    2461 }
    2462 
    2463 // Division with mathematical rounding to nearest number.
    2464 template <typename T>
    2465 inline T VmaRoundDiv(T x, T y)
    2466 {
    2467  return (x + (y / (T)2)) / y;
    2468 }
    2469 
    2470 #ifndef VMA_SORT
    2471 
    2472 template<typename Iterator, typename Compare>
    2473 Iterator VmaQuickSortPartition(Iterator beg, Iterator end, Compare cmp)
    2474 {
    2475  Iterator centerValue = end; --centerValue;
    2476  Iterator insertIndex = beg;
    2477  for(Iterator memTypeIndex = beg; memTypeIndex < centerValue; ++memTypeIndex)
    2478  {
    2479  if(cmp(*memTypeIndex, *centerValue))
    2480  {
    2481  if(insertIndex != memTypeIndex)
    2482  {
    2483  VMA_SWAP(*memTypeIndex, *insertIndex);
    2484  }
    2485  ++insertIndex;
    2486  }
    2487  }
    2488  if(insertIndex != centerValue)
    2489  {
    2490  VMA_SWAP(*insertIndex, *centerValue);
    2491  }
    2492  return insertIndex;
    2493 }
    2494 
    2495 template<typename Iterator, typename Compare>
    2496 void VmaQuickSort(Iterator beg, Iterator end, Compare cmp)
    2497 {
    2498  if(beg < end)
    2499  {
    2500  Iterator it = VmaQuickSortPartition<Iterator, Compare>(beg, end, cmp);
    2501  VmaQuickSort<Iterator, Compare>(beg, it, cmp);
    2502  VmaQuickSort<Iterator, Compare>(it + 1, end, cmp);
    2503  }
    2504 }
    2505 
    2506 #define VMA_SORT(beg, end, cmp) VmaQuickSort(beg, end, cmp)
    2507 
    2508 #endif // #ifndef VMA_SORT
    2509 
    2510 /*
    2511 Returns true if two memory blocks occupy overlapping pages.
    2512 ResourceA must be in less memory offset than ResourceB.
    2513 
    2514 Algorithm is based on "Vulkan 1.0.39 - A Specification (with all registered Vulkan extensions)"
    2515 chapter 11.6 "Resource Memory Association", paragraph "Buffer-Image Granularity".
    2516 */
    2517 static inline bool VmaBlocksOnSamePage(
    2518  VkDeviceSize resourceAOffset,
    2519  VkDeviceSize resourceASize,
    2520  VkDeviceSize resourceBOffset,
    2521  VkDeviceSize pageSize)
    2522 {
    2523  VMA_ASSERT(resourceAOffset + resourceASize <= resourceBOffset && resourceASize > 0 && pageSize > 0);
    2524  VkDeviceSize resourceAEnd = resourceAOffset + resourceASize - 1;
    2525  VkDeviceSize resourceAEndPage = resourceAEnd & ~(pageSize - 1);
    2526  VkDeviceSize resourceBStart = resourceBOffset;
    2527  VkDeviceSize resourceBStartPage = resourceBStart & ~(pageSize - 1);
    2528  return resourceAEndPage == resourceBStartPage;
    2529 }
    2530 
    2531 enum VmaSuballocationType
    2532 {
    2533  VMA_SUBALLOCATION_TYPE_FREE = 0,
    2534  VMA_SUBALLOCATION_TYPE_UNKNOWN = 1,
    2535  VMA_SUBALLOCATION_TYPE_BUFFER = 2,
    2536  VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN = 3,
    2537  VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR = 4,
    2538  VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL = 5,
    2539  VMA_SUBALLOCATION_TYPE_MAX_ENUM = 0x7FFFFFFF
    2540 };
    2541 
    2542 /*
    2543 Returns true if given suballocation types could conflict and must respect
    2544 VkPhysicalDeviceLimits::bufferImageGranularity. They conflict if one is buffer
    2545 or linear image and another one is optimal image. If type is unknown, behave
    2546 conservatively.
    2547 */
    2548 static inline bool VmaIsBufferImageGranularityConflict(
    2549  VmaSuballocationType suballocType1,
    2550  VmaSuballocationType suballocType2)
    2551 {
    2552  if(suballocType1 > suballocType2)
    2553  {
    2554  VMA_SWAP(suballocType1, suballocType2);
    2555  }
    2556 
    2557  switch(suballocType1)
    2558  {
    2559  case VMA_SUBALLOCATION_TYPE_FREE:
    2560  return false;
    2561  case VMA_SUBALLOCATION_TYPE_UNKNOWN:
    2562  return true;
    2563  case VMA_SUBALLOCATION_TYPE_BUFFER:
    2564  return
    2565  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
    2566  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
    2567  case VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN:
    2568  return
    2569  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
    2570  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR ||
    2571  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
    2572  case VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR:
    2573  return
    2574  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
    2575  case VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL:
    2576  return false;
    2577  default:
    2578  VMA_ASSERT(0);
    2579  return true;
    2580  }
    2581 }
    2582 
    2583 // Helper RAII class to lock a mutex in constructor and unlock it in destructor (at the end of scope).
    2584 struct VmaMutexLock
    2585 {
    2586 public:
    2587  VmaMutexLock(VMA_MUTEX& mutex, bool useMutex) :
    2588  m_pMutex(useMutex ? &mutex : VMA_NULL)
    2589  {
    2590  if(m_pMutex)
    2591  {
    2592  m_pMutex->Lock();
    2593  }
    2594  }
    2595 
    2596  ~VmaMutexLock()
    2597  {
    2598  if(m_pMutex)
    2599  {
    2600  m_pMutex->Unlock();
    2601  }
    2602  }
    2603 
    2604 private:
    2605  VMA_MUTEX* m_pMutex;
    2606 };
    2607 
    2608 #if VMA_DEBUG_GLOBAL_MUTEX
    2609  static VMA_MUTEX gDebugGlobalMutex;
    2610  #define VMA_DEBUG_GLOBAL_MUTEX_LOCK VmaMutexLock debugGlobalMutexLock(gDebugGlobalMutex, true);
    2611 #else
    2612  #define VMA_DEBUG_GLOBAL_MUTEX_LOCK
    2613 #endif
    2614 
    2615 // Minimum size of a free suballocation to register it in the free suballocation collection.
    2616 static const VkDeviceSize VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER = 16;
    2617 
    2618 /*
    2619 Performs binary search and returns iterator to first element that is greater or
    2620 equal to (key), according to comparison (cmp).
    2621 
    2622 Cmp should return true if first argument is less than second argument.
    2623 
    2624 Returned value is the found element, if present in the collection or place where
    2625 new element with value (key) should be inserted.
    2626 */
    2627 template <typename IterT, typename KeyT, typename CmpT>
    2628 static IterT VmaBinaryFindFirstNotLess(IterT beg, IterT end, const KeyT &key, CmpT cmp)
    2629 {
    2630  size_t down = 0, up = (end - beg);
    2631  while(down < up)
    2632  {
    2633  const size_t mid = (down + up) / 2;
    2634  if(cmp(*(beg+mid), key))
    2635  {
    2636  down = mid + 1;
    2637  }
    2638  else
    2639  {
    2640  up = mid;
    2641  }
    2642  }
    2643  return beg + down;
    2644 }
    2645 
    2647 // Memory allocation
    2648 
    2649 static void* VmaMalloc(const VkAllocationCallbacks* pAllocationCallbacks, size_t size, size_t alignment)
    2650 {
    2651  if((pAllocationCallbacks != VMA_NULL) &&
    2652  (pAllocationCallbacks->pfnAllocation != VMA_NULL))
    2653  {
    2654  return (*pAllocationCallbacks->pfnAllocation)(
    2655  pAllocationCallbacks->pUserData,
    2656  size,
    2657  alignment,
    2658  VK_SYSTEM_ALLOCATION_SCOPE_OBJECT);
    2659  }
    2660  else
    2661  {
    2662  return VMA_SYSTEM_ALIGNED_MALLOC(size, alignment);
    2663  }
    2664 }
    2665 
    2666 static void VmaFree(const VkAllocationCallbacks* pAllocationCallbacks, void* ptr)
    2667 {
    2668  if((pAllocationCallbacks != VMA_NULL) &&
    2669  (pAllocationCallbacks->pfnFree != VMA_NULL))
    2670  {
    2671  (*pAllocationCallbacks->pfnFree)(pAllocationCallbacks->pUserData, ptr);
    2672  }
    2673  else
    2674  {
    2675  VMA_SYSTEM_FREE(ptr);
    2676  }
    2677 }
    2678 
    2679 template<typename T>
    2680 static T* VmaAllocate(const VkAllocationCallbacks* pAllocationCallbacks)
    2681 {
    2682  return (T*)VmaMalloc(pAllocationCallbacks, sizeof(T), VMA_ALIGN_OF(T));
    2683 }
    2684 
    2685 template<typename T>
    2686 static T* VmaAllocateArray(const VkAllocationCallbacks* pAllocationCallbacks, size_t count)
    2687 {
    2688  return (T*)VmaMalloc(pAllocationCallbacks, sizeof(T) * count, VMA_ALIGN_OF(T));
    2689 }
    2690 
    2691 #define vma_new(allocator, type) new(VmaAllocate<type>(allocator))(type)
    2692 
    2693 #define vma_new_array(allocator, type, count) new(VmaAllocateArray<type>((allocator), (count)))(type)
    2694 
    2695 template<typename T>
    2696 static void vma_delete(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr)
    2697 {
    2698  ptr->~T();
    2699  VmaFree(pAllocationCallbacks, ptr);
    2700 }
    2701 
    2702 template<typename T>
    2703 static void vma_delete_array(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr, size_t count)
    2704 {
    2705  if(ptr != VMA_NULL)
    2706  {
    2707  for(size_t i = count; i--; )
    2708  {
    2709  ptr[i].~T();
    2710  }
    2711  VmaFree(pAllocationCallbacks, ptr);
    2712  }
    2713 }
    2714 
    2715 // STL-compatible allocator.
    2716 template<typename T>
    2717 class VmaStlAllocator
    2718 {
    2719 public:
    2720  const VkAllocationCallbacks* const m_pCallbacks;
    2721  typedef T value_type;
    2722 
    2723  VmaStlAllocator(const VkAllocationCallbacks* pCallbacks) : m_pCallbacks(pCallbacks) { }
    2724  template<typename U> VmaStlAllocator(const VmaStlAllocator<U>& src) : m_pCallbacks(src.m_pCallbacks) { }
    2725 
    2726  T* allocate(size_t n) { return VmaAllocateArray<T>(m_pCallbacks, n); }
    2727  void deallocate(T* p, size_t n) { VmaFree(m_pCallbacks, p); }
    2728 
    2729  template<typename U>
    2730  bool operator==(const VmaStlAllocator<U>& rhs) const
    2731  {
    2732  return m_pCallbacks == rhs.m_pCallbacks;
    2733  }
    2734  template<typename U>
    2735  bool operator!=(const VmaStlAllocator<U>& rhs) const
    2736  {
    2737  return m_pCallbacks != rhs.m_pCallbacks;
    2738  }
    2739 
    2740  VmaStlAllocator& operator=(const VmaStlAllocator& x) = delete;
    2741 };
    2742 
    2743 #if VMA_USE_STL_VECTOR
    2744 
    2745 #define VmaVector std::vector
    2746 
    2747 template<typename T, typename allocatorT>
    2748 static void VmaVectorInsert(std::vector<T, allocatorT>& vec, size_t index, const T& item)
    2749 {
    2750  vec.insert(vec.begin() + index, item);
    2751 }
    2752 
    2753 template<typename T, typename allocatorT>
    2754 static void VmaVectorRemove(std::vector<T, allocatorT>& vec, size_t index)
    2755 {
    2756  vec.erase(vec.begin() + index);
    2757 }
    2758 
    2759 #else // #if VMA_USE_STL_VECTOR
    2760 
    2761 /* Class with interface compatible with subset of std::vector.
    2762 T must be POD because constructors and destructors are not called and memcpy is
    2763 used for these objects. */
    2764 template<typename T, typename AllocatorT>
    2765 class VmaVector
    2766 {
    2767 public:
    2768  typedef T value_type;
    2769 
    2770  VmaVector(const AllocatorT& allocator) :
    2771  m_Allocator(allocator),
    2772  m_pArray(VMA_NULL),
    2773  m_Count(0),
    2774  m_Capacity(0)
    2775  {
    2776  }
    2777 
    2778  VmaVector(size_t count, const AllocatorT& allocator) :
    2779  m_Allocator(allocator),
    2780  m_pArray(count ? (T*)VmaAllocateArray<T>(allocator.m_pCallbacks, count) : VMA_NULL),
    2781  m_Count(count),
    2782  m_Capacity(count)
    2783  {
    2784  }
    2785 
    2786  VmaVector(const VmaVector<T, AllocatorT>& src) :
    2787  m_Allocator(src.m_Allocator),
    2788  m_pArray(src.m_Count ? (T*)VmaAllocateArray<T>(src.m_Allocator.m_pCallbacks, src.m_Count) : VMA_NULL),
    2789  m_Count(src.m_Count),
    2790  m_Capacity(src.m_Count)
    2791  {
    2792  if(m_Count != 0)
    2793  {
    2794  memcpy(m_pArray, src.m_pArray, m_Count * sizeof(T));
    2795  }
    2796  }
    2797 
    2798  ~VmaVector()
    2799  {
    2800  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
    2801  }
    2802 
    2803  VmaVector& operator=(const VmaVector<T, AllocatorT>& rhs)
    2804  {
    2805  if(&rhs != this)
    2806  {
    2807  resize(rhs.m_Count);
    2808  if(m_Count != 0)
    2809  {
    2810  memcpy(m_pArray, rhs.m_pArray, m_Count * sizeof(T));
    2811  }
    2812  }
    2813  return *this;
    2814  }
    2815 
    2816  bool empty() const { return m_Count == 0; }
    2817  size_t size() const { return m_Count; }
    2818  T* data() { return m_pArray; }
    2819  const T* data() const { return m_pArray; }
    2820 
    2821  T& operator[](size_t index)
    2822  {
    2823  VMA_HEAVY_ASSERT(index < m_Count);
    2824  return m_pArray[index];
    2825  }
    2826  const T& operator[](size_t index) const
    2827  {
    2828  VMA_HEAVY_ASSERT(index < m_Count);
    2829  return m_pArray[index];
    2830  }
    2831 
    2832  T& front()
    2833  {
    2834  VMA_HEAVY_ASSERT(m_Count > 0);
    2835  return m_pArray[0];
    2836  }
    2837  const T& front() const
    2838  {
    2839  VMA_HEAVY_ASSERT(m_Count > 0);
    2840  return m_pArray[0];
    2841  }
    2842  T& back()
    2843  {
    2844  VMA_HEAVY_ASSERT(m_Count > 0);
    2845  return m_pArray[m_Count - 1];
    2846  }
    2847  const T& back() const
    2848  {
    2849  VMA_HEAVY_ASSERT(m_Count > 0);
    2850  return m_pArray[m_Count - 1];
    2851  }
    2852 
    2853  void reserve(size_t newCapacity, bool freeMemory = false)
    2854  {
    2855  newCapacity = VMA_MAX(newCapacity, m_Count);
    2856 
    2857  if((newCapacity < m_Capacity) && !freeMemory)
    2858  {
    2859  newCapacity = m_Capacity;
    2860  }
    2861 
    2862  if(newCapacity != m_Capacity)
    2863  {
    2864  T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator, newCapacity) : VMA_NULL;
    2865  if(m_Count != 0)
    2866  {
    2867  memcpy(newArray, m_pArray, m_Count * sizeof(T));
    2868  }
    2869  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
    2870  m_Capacity = newCapacity;
    2871  m_pArray = newArray;
    2872  }
    2873  }
    2874 
    2875  void resize(size_t newCount, bool freeMemory = false)
    2876  {
    2877  size_t newCapacity = m_Capacity;
    2878  if(newCount > m_Capacity)
    2879  {
    2880  newCapacity = VMA_MAX(newCount, VMA_MAX(m_Capacity * 3 / 2, (size_t)8));
    2881  }
    2882  else if(freeMemory)
    2883  {
    2884  newCapacity = newCount;
    2885  }
    2886 
    2887  if(newCapacity != m_Capacity)
    2888  {
    2889  T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator.m_pCallbacks, newCapacity) : VMA_NULL;
    2890  const size_t elementsToCopy = VMA_MIN(m_Count, newCount);
    2891  if(elementsToCopy != 0)
    2892  {
    2893  memcpy(newArray, m_pArray, elementsToCopy * sizeof(T));
    2894  }
    2895  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
    2896  m_Capacity = newCapacity;
    2897  m_pArray = newArray;
    2898  }
    2899 
    2900  m_Count = newCount;
    2901  }
    2902 
    2903  void clear(bool freeMemory = false)
    2904  {
    2905  resize(0, freeMemory);
    2906  }
    2907 
    2908  void insert(size_t index, const T& src)
    2909  {
    2910  VMA_HEAVY_ASSERT(index <= m_Count);
    2911  const size_t oldCount = size();
    2912  resize(oldCount + 1);
    2913  if(index < oldCount)
    2914  {
    2915  memmove(m_pArray + (index + 1), m_pArray + index, (oldCount - index) * sizeof(T));
    2916  }
    2917  m_pArray[index] = src;
    2918  }
    2919 
    2920  void remove(size_t index)
    2921  {
    2922  VMA_HEAVY_ASSERT(index < m_Count);
    2923  const size_t oldCount = size();
    2924  if(index < oldCount - 1)
    2925  {
    2926  memmove(m_pArray + index, m_pArray + (index + 1), (oldCount - index - 1) * sizeof(T));
    2927  }
    2928  resize(oldCount - 1);
    2929  }
    2930 
    2931  void push_back(const T& src)
    2932  {
    2933  const size_t newIndex = size();
    2934  resize(newIndex + 1);
    2935  m_pArray[newIndex] = src;
    2936  }
    2937 
    2938  void pop_back()
    2939  {
    2940  VMA_HEAVY_ASSERT(m_Count > 0);
    2941  resize(size() - 1);
    2942  }
    2943 
    2944  void push_front(const T& src)
    2945  {
    2946  insert(0, src);
    2947  }
    2948 
    2949  void pop_front()
    2950  {
    2951  VMA_HEAVY_ASSERT(m_Count > 0);
    2952  remove(0);
    2953  }
    2954 
    2955  typedef T* iterator;
    2956 
    2957  iterator begin() { return m_pArray; }
    2958  iterator end() { return m_pArray + m_Count; }
    2959 
    2960 private:
    2961  AllocatorT m_Allocator;
    2962  T* m_pArray;
    2963  size_t m_Count;
    2964  size_t m_Capacity;
    2965 };
    2966 
    2967 template<typename T, typename allocatorT>
    2968 static void VmaVectorInsert(VmaVector<T, allocatorT>& vec, size_t index, const T& item)
    2969 {
    2970  vec.insert(index, item);
    2971 }
    2972 
    2973 template<typename T, typename allocatorT>
    2974 static void VmaVectorRemove(VmaVector<T, allocatorT>& vec, size_t index)
    2975 {
    2976  vec.remove(index);
    2977 }
    2978 
    2979 #endif // #if VMA_USE_STL_VECTOR
    2980 
    2981 template<typename CmpLess, typename VectorT>
    2982 size_t VmaVectorInsertSorted(VectorT& vector, const typename VectorT::value_type& value)
    2983 {
    2984  const size_t indexToInsert = VmaBinaryFindFirstNotLess(
    2985  vector.data(),
    2986  vector.data() + vector.size(),
    2987  value,
    2988  CmpLess()) - vector.data();
    2989  VmaVectorInsert(vector, indexToInsert, value);
    2990  return indexToInsert;
    2991 }
    2992 
    2993 template<typename CmpLess, typename VectorT>
    2994 bool VmaVectorRemoveSorted(VectorT& vector, const typename VectorT::value_type& value)
    2995 {
    2996  CmpLess comparator;
    2997  typename VectorT::iterator it = VmaBinaryFindFirstNotLess(
    2998  vector.begin(),
    2999  vector.end(),
    3000  value,
    3001  comparator);
    3002  if((it != vector.end()) && !comparator(*it, value) && !comparator(value, *it))
    3003  {
    3004  size_t indexToRemove = it - vector.begin();
    3005  VmaVectorRemove(vector, indexToRemove);
    3006  return true;
    3007  }
    3008  return false;
    3009 }
    3010 
    3011 template<typename CmpLess, typename VectorT>
    3012 size_t VmaVectorFindSorted(const VectorT& vector, const typename VectorT::value_type& value)
    3013 {
    3014  CmpLess comparator;
    3015  typename VectorT::iterator it = VmaBinaryFindFirstNotLess(
    3016  vector.data(),
    3017  vector.data() + vector.size(),
    3018  value,
    3019  comparator);
    3020  if(it != vector.size() && !comparator(*it, value) && !comparator(value, *it))
    3021  {
    3022  return it - vector.begin();
    3023  }
    3024  else
    3025  {
    3026  return vector.size();
    3027  }
    3028 }
    3029 
    3031 // class VmaPoolAllocator
    3032 
    3033 /*
    3034 Allocator for objects of type T using a list of arrays (pools) to speed up
    3035 allocation. Number of elements that can be allocated is not bounded because
    3036 allocator can create multiple blocks.
    3037 */
    3038 template<typename T>
    3039 class VmaPoolAllocator
    3040 {
    3041 public:
    3042  VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, size_t itemsPerBlock);
    3043  ~VmaPoolAllocator();
    3044  void Clear();
    3045  T* Alloc();
    3046  void Free(T* ptr);
    3047 
    3048 private:
    3049  union Item
    3050  {
    3051  uint32_t NextFreeIndex;
    3052  T Value;
    3053  };
    3054 
    3055  struct ItemBlock
    3056  {
    3057  Item* pItems;
    3058  uint32_t FirstFreeIndex;
    3059  };
    3060 
    3061  const VkAllocationCallbacks* m_pAllocationCallbacks;
    3062  size_t m_ItemsPerBlock;
    3063  VmaVector< ItemBlock, VmaStlAllocator<ItemBlock> > m_ItemBlocks;
    3064 
    3065  ItemBlock& CreateNewBlock();
    3066 };
    3067 
    3068 template<typename T>
    3069 VmaPoolAllocator<T>::VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, size_t itemsPerBlock) :
    3070  m_pAllocationCallbacks(pAllocationCallbacks),
    3071  m_ItemsPerBlock(itemsPerBlock),
    3072  m_ItemBlocks(VmaStlAllocator<ItemBlock>(pAllocationCallbacks))
    3073 {
    3074  VMA_ASSERT(itemsPerBlock > 0);
    3075 }
    3076 
    3077 template<typename T>
    3078 VmaPoolAllocator<T>::~VmaPoolAllocator()
    3079 {
    3080  Clear();
    3081 }
    3082 
    3083 template<typename T>
    3084 void VmaPoolAllocator<T>::Clear()
    3085 {
    3086  for(size_t i = m_ItemBlocks.size(); i--; )
    3087  vma_delete_array(m_pAllocationCallbacks, m_ItemBlocks[i].pItems, m_ItemsPerBlock);
    3088  m_ItemBlocks.clear();
    3089 }
    3090 
    3091 template<typename T>
    3092 T* VmaPoolAllocator<T>::Alloc()
    3093 {
    3094  for(size_t i = m_ItemBlocks.size(); i--; )
    3095  {
    3096  ItemBlock& block = m_ItemBlocks[i];
    3097  // This block has some free items: Use first one.
    3098  if(block.FirstFreeIndex != UINT32_MAX)
    3099  {
    3100  Item* const pItem = &block.pItems[block.FirstFreeIndex];
    3101  block.FirstFreeIndex = pItem->NextFreeIndex;
    3102  return &pItem->Value;
    3103  }
    3104  }
    3105 
    3106  // No block has free item: Create new one and use it.
    3107  ItemBlock& newBlock = CreateNewBlock();
    3108  Item* const pItem = &newBlock.pItems[0];
    3109  newBlock.FirstFreeIndex = pItem->NextFreeIndex;
    3110  return &pItem->Value;
    3111 }
    3112 
    3113 template<typename T>
    3114 void VmaPoolAllocator<T>::Free(T* ptr)
    3115 {
    3116  // Search all memory blocks to find ptr.
    3117  for(size_t i = 0; i < m_ItemBlocks.size(); ++i)
    3118  {
    3119  ItemBlock& block = m_ItemBlocks[i];
    3120 
    3121  // Casting to union.
    3122  Item* pItemPtr;
    3123  memcpy(&pItemPtr, &ptr, sizeof(pItemPtr));
    3124 
    3125  // Check if pItemPtr is in address range of this block.
    3126  if((pItemPtr >= block.pItems) && (pItemPtr < block.pItems + m_ItemsPerBlock))
    3127  {
    3128  const uint32_t index = static_cast<uint32_t>(pItemPtr - block.pItems);
    3129  pItemPtr->NextFreeIndex = block.FirstFreeIndex;
    3130  block.FirstFreeIndex = index;
    3131  return;
    3132  }
    3133  }
    3134  VMA_ASSERT(0 && "Pointer doesn't belong to this memory pool.");
    3135 }
    3136 
    3137 template<typename T>
    3138 typename VmaPoolAllocator<T>::ItemBlock& VmaPoolAllocator<T>::CreateNewBlock()
    3139 {
    3140  ItemBlock newBlock = {
    3141  vma_new_array(m_pAllocationCallbacks, Item, m_ItemsPerBlock), 0 };
    3142 
    3143  m_ItemBlocks.push_back(newBlock);
    3144 
    3145  // Setup singly-linked list of all free items in this block.
    3146  for(uint32_t i = 0; i < m_ItemsPerBlock - 1; ++i)
    3147  newBlock.pItems[i].NextFreeIndex = i + 1;
    3148  newBlock.pItems[m_ItemsPerBlock - 1].NextFreeIndex = UINT32_MAX;
    3149  return m_ItemBlocks.back();
    3150 }
    3151 
    3153 // class VmaRawList, VmaList
    3154 
    3155 #if VMA_USE_STL_LIST
    3156 
    3157 #define VmaList std::list
    3158 
    3159 #else // #if VMA_USE_STL_LIST
    3160 
    3161 template<typename T>
    3162 struct VmaListItem
    3163 {
    3164  VmaListItem* pPrev;
    3165  VmaListItem* pNext;
    3166  T Value;
    3167 };
    3168 
    3169 // Doubly linked list.
    3170 template<typename T>
    3171 class VmaRawList
    3172 {
    3173 public:
    3174  typedef VmaListItem<T> ItemType;
    3175 
    3176  VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks);
    3177  ~VmaRawList();
    3178  void Clear();
    3179 
    3180  size_t GetCount() const { return m_Count; }
    3181  bool IsEmpty() const { return m_Count == 0; }
    3182 
    3183  ItemType* Front() { return m_pFront; }
    3184  const ItemType* Front() const { return m_pFront; }
    3185  ItemType* Back() { return m_pBack; }
    3186  const ItemType* Back() const { return m_pBack; }
    3187 
    3188  ItemType* PushBack();
    3189  ItemType* PushFront();
    3190  ItemType* PushBack(const T& value);
    3191  ItemType* PushFront(const T& value);
    3192  void PopBack();
    3193  void PopFront();
    3194 
    3195  // Item can be null - it means PushBack.
    3196  ItemType* InsertBefore(ItemType* pItem);
    3197  // Item can be null - it means PushFront.
    3198  ItemType* InsertAfter(ItemType* pItem);
    3199 
    3200  ItemType* InsertBefore(ItemType* pItem, const T& value);
    3201  ItemType* InsertAfter(ItemType* pItem, const T& value);
    3202 
    3203  void Remove(ItemType* pItem);
    3204 
    3205 private:
    3206  const VkAllocationCallbacks* const m_pAllocationCallbacks;
    3207  VmaPoolAllocator<ItemType> m_ItemAllocator;
    3208  ItemType* m_pFront;
    3209  ItemType* m_pBack;
    3210  size_t m_Count;
    3211 
    3212  // Declared not defined, to block copy constructor and assignment operator.
    3213  VmaRawList(const VmaRawList<T>& src);
    3214  VmaRawList<T>& operator=(const VmaRawList<T>& rhs);
    3215 };
    3216 
    3217 template<typename T>
    3218 VmaRawList<T>::VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks) :
    3219  m_pAllocationCallbacks(pAllocationCallbacks),
    3220  m_ItemAllocator(pAllocationCallbacks, 128),
    3221  m_pFront(VMA_NULL),
    3222  m_pBack(VMA_NULL),
    3223  m_Count(0)
    3224 {
    3225 }
    3226 
    3227 template<typename T>
    3228 VmaRawList<T>::~VmaRawList()
    3229 {
    3230  // Intentionally not calling Clear, because that would be unnecessary
    3231  // computations to return all items to m_ItemAllocator as free.
    3232 }
    3233 
    3234 template<typename T>
    3235 void VmaRawList<T>::Clear()
    3236 {
    3237  if(IsEmpty() == false)
    3238  {
    3239  ItemType* pItem = m_pBack;
    3240  while(pItem != VMA_NULL)
    3241  {
    3242  ItemType* const pPrevItem = pItem->pPrev;
    3243  m_ItemAllocator.Free(pItem);
    3244  pItem = pPrevItem;
    3245  }
    3246  m_pFront = VMA_NULL;
    3247  m_pBack = VMA_NULL;
    3248  m_Count = 0;
    3249  }
    3250 }
    3251 
    3252 template<typename T>
    3253 VmaListItem<T>* VmaRawList<T>::PushBack()
    3254 {
    3255  ItemType* const pNewItem = m_ItemAllocator.Alloc();
    3256  pNewItem->pNext = VMA_NULL;
    3257  if(IsEmpty())
    3258  {
    3259  pNewItem->pPrev = VMA_NULL;
    3260  m_pFront = pNewItem;
    3261  m_pBack = pNewItem;
    3262  m_Count = 1;
    3263  }
    3264  else
    3265  {
    3266  pNewItem->pPrev = m_pBack;
    3267  m_pBack->pNext = pNewItem;
    3268  m_pBack = pNewItem;
    3269  ++m_Count;
    3270  }
    3271  return pNewItem;
    3272 }
    3273 
    3274 template<typename T>
    3275 VmaListItem<T>* VmaRawList<T>::PushFront()
    3276 {
    3277  ItemType* const pNewItem = m_ItemAllocator.Alloc();
    3278  pNewItem->pPrev = VMA_NULL;
    3279  if(IsEmpty())
    3280  {
    3281  pNewItem->pNext = VMA_NULL;
    3282  m_pFront = pNewItem;
    3283  m_pBack = pNewItem;
    3284  m_Count = 1;
    3285  }
    3286  else
    3287  {
    3288  pNewItem->pNext = m_pFront;
    3289  m_pFront->pPrev = pNewItem;
    3290  m_pFront = pNewItem;
    3291  ++m_Count;
    3292  }
    3293  return pNewItem;
    3294 }
    3295 
    3296 template<typename T>
    3297 VmaListItem<T>* VmaRawList<T>::PushBack(const T& value)
    3298 {
    3299  ItemType* const pNewItem = PushBack();
    3300  pNewItem->Value = value;
    3301  return pNewItem;
    3302 }
    3303 
    3304 template<typename T>
    3305 VmaListItem<T>* VmaRawList<T>::PushFront(const T& value)
    3306 {
    3307  ItemType* const pNewItem = PushFront();
    3308  pNewItem->Value = value;
    3309  return pNewItem;
    3310 }
    3311 
    3312 template<typename T>
    3313 void VmaRawList<T>::PopBack()
    3314 {
    3315  VMA_HEAVY_ASSERT(m_Count > 0);
    3316  ItemType* const pBackItem = m_pBack;
    3317  ItemType* const pPrevItem = pBackItem->pPrev;
    3318  if(pPrevItem != VMA_NULL)
    3319  {
    3320  pPrevItem->pNext = VMA_NULL;
    3321  }
    3322  m_pBack = pPrevItem;
    3323  m_ItemAllocator.Free(pBackItem);
    3324  --m_Count;
    3325 }
    3326 
    3327 template<typename T>
    3328 void VmaRawList<T>::PopFront()
    3329 {
    3330  VMA_HEAVY_ASSERT(m_Count > 0);
    3331  ItemType* const pFrontItem = m_pFront;
    3332  ItemType* const pNextItem = pFrontItem->pNext;
    3333  if(pNextItem != VMA_NULL)
    3334  {
    3335  pNextItem->pPrev = VMA_NULL;
    3336  }
    3337  m_pFront = pNextItem;
    3338  m_ItemAllocator.Free(pFrontItem);
    3339  --m_Count;
    3340 }
    3341 
    3342 template<typename T>
    3343 void VmaRawList<T>::Remove(ItemType* pItem)
    3344 {
    3345  VMA_HEAVY_ASSERT(pItem != VMA_NULL);
    3346  VMA_HEAVY_ASSERT(m_Count > 0);
    3347 
    3348  if(pItem->pPrev != VMA_NULL)
    3349  {
    3350  pItem->pPrev->pNext = pItem->pNext;
    3351  }
    3352  else
    3353  {
    3354  VMA_HEAVY_ASSERT(m_pFront == pItem);
    3355  m_pFront = pItem->pNext;
    3356  }
    3357 
    3358  if(pItem->pNext != VMA_NULL)
    3359  {
    3360  pItem->pNext->pPrev = pItem->pPrev;
    3361  }
    3362  else
    3363  {
    3364  VMA_HEAVY_ASSERT(m_pBack == pItem);
    3365  m_pBack = pItem->pPrev;
    3366  }
    3367 
    3368  m_ItemAllocator.Free(pItem);
    3369  --m_Count;
    3370 }
    3371 
    3372 template<typename T>
    3373 VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem)
    3374 {
    3375  if(pItem != VMA_NULL)
    3376  {
    3377  ItemType* const prevItem = pItem->pPrev;
    3378  ItemType* const newItem = m_ItemAllocator.Alloc();
    3379  newItem->pPrev = prevItem;
    3380  newItem->pNext = pItem;
    3381  pItem->pPrev = newItem;
    3382  if(prevItem != VMA_NULL)
    3383  {
    3384  prevItem->pNext = newItem;
    3385  }
    3386  else
    3387  {
    3388  VMA_HEAVY_ASSERT(m_pFront == pItem);
    3389  m_pFront = newItem;
    3390  }
    3391  ++m_Count;
    3392  return newItem;
    3393  }
    3394  else
    3395  return PushBack();
    3396 }
    3397 
    3398 template<typename T>
    3399 VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem)
    3400 {
    3401  if(pItem != VMA_NULL)
    3402  {
    3403  ItemType* const nextItem = pItem->pNext;
    3404  ItemType* const newItem = m_ItemAllocator.Alloc();
    3405  newItem->pNext = nextItem;
    3406  newItem->pPrev = pItem;
    3407  pItem->pNext = newItem;
    3408  if(nextItem != VMA_NULL)
    3409  {
    3410  nextItem->pPrev = newItem;
    3411  }
    3412  else
    3413  {
    3414  VMA_HEAVY_ASSERT(m_pBack == pItem);
    3415  m_pBack = newItem;
    3416  }
    3417  ++m_Count;
    3418  return newItem;
    3419  }
    3420  else
    3421  return PushFront();
    3422 }
    3423 
    3424 template<typename T>
    3425 VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem, const T& value)
    3426 {
    3427  ItemType* const newItem = InsertBefore(pItem);
    3428  newItem->Value = value;
    3429  return newItem;
    3430 }
    3431 
    3432 template<typename T>
    3433 VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem, const T& value)
    3434 {
    3435  ItemType* const newItem = InsertAfter(pItem);
    3436  newItem->Value = value;
    3437  return newItem;
    3438 }
    3439 
    3440 template<typename T, typename AllocatorT>
    3441 class VmaList
    3442 {
    3443 public:
    3444  class iterator
    3445  {
    3446  public:
    3447  iterator() :
    3448  m_pList(VMA_NULL),
    3449  m_pItem(VMA_NULL)
    3450  {
    3451  }
    3452 
    3453  T& operator*() const
    3454  {
    3455  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3456  return m_pItem->Value;
    3457  }
    3458  T* operator->() const
    3459  {
    3460  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3461  return &m_pItem->Value;
    3462  }
    3463 
    3464  iterator& operator++()
    3465  {
    3466  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3467  m_pItem = m_pItem->pNext;
    3468  return *this;
    3469  }
    3470  iterator& operator--()
    3471  {
    3472  if(m_pItem != VMA_NULL)
    3473  {
    3474  m_pItem = m_pItem->pPrev;
    3475  }
    3476  else
    3477  {
    3478  VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
    3479  m_pItem = m_pList->Back();
    3480  }
    3481  return *this;
    3482  }
    3483 
    3484  iterator operator++(int)
    3485  {
    3486  iterator result = *this;
    3487  ++*this;
    3488  return result;
    3489  }
    3490  iterator operator--(int)
    3491  {
    3492  iterator result = *this;
    3493  --*this;
    3494  return result;
    3495  }
    3496 
    3497  bool operator==(const iterator& rhs) const
    3498  {
    3499  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
    3500  return m_pItem == rhs.m_pItem;
    3501  }
    3502  bool operator!=(const iterator& rhs) const
    3503  {
    3504  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
    3505  return m_pItem != rhs.m_pItem;
    3506  }
    3507 
    3508  private:
    3509  VmaRawList<T>* m_pList;
    3510  VmaListItem<T>* m_pItem;
    3511 
    3512  iterator(VmaRawList<T>* pList, VmaListItem<T>* pItem) :
    3513  m_pList(pList),
    3514  m_pItem(pItem)
    3515  {
    3516  }
    3517 
    3518  friend class VmaList<T, AllocatorT>;
    3519  };
    3520 
    3521  class const_iterator
    3522  {
    3523  public:
    3524  const_iterator() :
    3525  m_pList(VMA_NULL),
    3526  m_pItem(VMA_NULL)
    3527  {
    3528  }
    3529 
    3530  const_iterator(const iterator& src) :
    3531  m_pList(src.m_pList),
    3532  m_pItem(src.m_pItem)
    3533  {
    3534  }
    3535 
    3536  const T& operator*() const
    3537  {
    3538  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3539  return m_pItem->Value;
    3540  }
    3541  const T* operator->() const
    3542  {
    3543  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3544  return &m_pItem->Value;
    3545  }
    3546 
    3547  const_iterator& operator++()
    3548  {
    3549  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3550  m_pItem = m_pItem->pNext;
    3551  return *this;
    3552  }
    3553  const_iterator& operator--()
    3554  {
    3555  if(m_pItem != VMA_NULL)
    3556  {
    3557  m_pItem = m_pItem->pPrev;
    3558  }
    3559  else
    3560  {
    3561  VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
    3562  m_pItem = m_pList->Back();
    3563  }
    3564  return *this;
    3565  }
    3566 
    3567  const_iterator operator++(int)
    3568  {
    3569  const_iterator result = *this;
    3570  ++*this;
    3571  return result;
    3572  }
    3573  const_iterator operator--(int)
    3574  {
    3575  const_iterator result = *this;
    3576  --*this;
    3577  return result;
    3578  }
    3579 
    3580  bool operator==(const const_iterator& rhs) const
    3581  {
    3582  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
    3583  return m_pItem == rhs.m_pItem;
    3584  }
    3585  bool operator!=(const const_iterator& rhs) const
    3586  {
    3587  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
    3588  return m_pItem != rhs.m_pItem;
    3589  }
    3590 
    3591  private:
    3592  const_iterator(const VmaRawList<T>* pList, const VmaListItem<T>* pItem) :
    3593  m_pList(pList),
    3594  m_pItem(pItem)
    3595  {
    3596  }
    3597 
    3598  const VmaRawList<T>* m_pList;
    3599  const VmaListItem<T>* m_pItem;
    3600 
    3601  friend class VmaList<T, AllocatorT>;
    3602  };
    3603 
    3604  VmaList(const AllocatorT& allocator) : m_RawList(allocator.m_pCallbacks) { }
    3605 
    3606  bool empty() const { return m_RawList.IsEmpty(); }
    3607  size_t size() const { return m_RawList.GetCount(); }
    3608 
    3609  iterator begin() { return iterator(&m_RawList, m_RawList.Front()); }
    3610  iterator end() { return iterator(&m_RawList, VMA_NULL); }
    3611 
    3612  const_iterator cbegin() const { return const_iterator(&m_RawList, m_RawList.Front()); }
    3613  const_iterator cend() const { return const_iterator(&m_RawList, VMA_NULL); }
    3614 
    3615  void clear() { m_RawList.Clear(); }
    3616  void push_back(const T& value) { m_RawList.PushBack(value); }
    3617  void erase(iterator it) { m_RawList.Remove(it.m_pItem); }
    3618  iterator insert(iterator it, const T& value) { return iterator(&m_RawList, m_RawList.InsertBefore(it.m_pItem, value)); }
    3619 
    3620 private:
    3621  VmaRawList<T> m_RawList;
    3622 };
    3623 
    3624 #endif // #if VMA_USE_STL_LIST
    3625 
    3627 // class VmaMap
    3628 
    3629 // Unused in this version.
    3630 #if 0
    3631 
    3632 #if VMA_USE_STL_UNORDERED_MAP
    3633 
    3634 #define VmaPair std::pair
    3635 
    3636 #define VMA_MAP_TYPE(KeyT, ValueT) \
    3637  std::unordered_map< KeyT, ValueT, std::hash<KeyT>, std::equal_to<KeyT>, VmaStlAllocator< std::pair<KeyT, ValueT> > >
    3638 
    3639 #else // #if VMA_USE_STL_UNORDERED_MAP
    3640 
    3641 template<typename T1, typename T2>
    3642 struct VmaPair
    3643 {
    3644  T1 first;
    3645  T2 second;
    3646 
    3647  VmaPair() : first(), second() { }
    3648  VmaPair(const T1& firstSrc, const T2& secondSrc) : first(firstSrc), second(secondSrc) { }
    3649 };
    3650 
    3651 /* Class compatible with subset of interface of std::unordered_map.
    3652 KeyT, ValueT must be POD because they will be stored in VmaVector.
    3653 */
    3654 template<typename KeyT, typename ValueT>
    3655 class VmaMap
    3656 {
    3657 public:
    3658  typedef VmaPair<KeyT, ValueT> PairType;
    3659  typedef PairType* iterator;
    3660 
    3661  VmaMap(const VmaStlAllocator<PairType>& allocator) : m_Vector(allocator) { }
    3662 
    3663  iterator begin() { return m_Vector.begin(); }
    3664  iterator end() { return m_Vector.end(); }
    3665 
    3666  void insert(const PairType& pair);
    3667  iterator find(const KeyT& key);
    3668  void erase(iterator it);
    3669 
    3670 private:
    3671  VmaVector< PairType, VmaStlAllocator<PairType> > m_Vector;
    3672 };
    3673 
    3674 #define VMA_MAP_TYPE(KeyT, ValueT) VmaMap<KeyT, ValueT>
    3675 
    3676 template<typename FirstT, typename SecondT>
    3677 struct VmaPairFirstLess
    3678 {
    3679  bool operator()(const VmaPair<FirstT, SecondT>& lhs, const VmaPair<FirstT, SecondT>& rhs) const
    3680  {
    3681  return lhs.first < rhs.first;
    3682  }
    3683  bool operator()(const VmaPair<FirstT, SecondT>& lhs, const FirstT& rhsFirst) const
    3684  {
    3685  return lhs.first < rhsFirst;
    3686  }
    3687 };
    3688 
    3689 template<typename KeyT, typename ValueT>
    3690 void VmaMap<KeyT, ValueT>::insert(const PairType& pair)
    3691 {
    3692  const size_t indexToInsert = VmaBinaryFindFirstNotLess(
    3693  m_Vector.data(),
    3694  m_Vector.data() + m_Vector.size(),
    3695  pair,
    3696  VmaPairFirstLess<KeyT, ValueT>()) - m_Vector.data();
    3697  VmaVectorInsert(m_Vector, indexToInsert, pair);
    3698 }
    3699 
    3700 template<typename KeyT, typename ValueT>
    3701 VmaPair<KeyT, ValueT>* VmaMap<KeyT, ValueT>::find(const KeyT& key)
    3702 {
    3703  PairType* it = VmaBinaryFindFirstNotLess(
    3704  m_Vector.data(),
    3705  m_Vector.data() + m_Vector.size(),
    3706  key,
    3707  VmaPairFirstLess<KeyT, ValueT>());
    3708  if((it != m_Vector.end()) && (it->first == key))
    3709  {
    3710  return it;
    3711  }
    3712  else
    3713  {
    3714  return m_Vector.end();
    3715  }
    3716 }
    3717 
    3718 template<typename KeyT, typename ValueT>
    3719 void VmaMap<KeyT, ValueT>::erase(iterator it)
    3720 {
    3721  VmaVectorRemove(m_Vector, it - m_Vector.begin());
    3722 }
    3723 
    3724 #endif // #if VMA_USE_STL_UNORDERED_MAP
    3725 
    3726 #endif // #if 0
    3727 
    3729 
    3730 class VmaDeviceMemoryBlock;
    3731 
    3732 struct VmaAllocation_T
    3733 {
    3734 private:
    3735  static const uint8_t MAP_COUNT_FLAG_PERSISTENT_MAP = 0x80;
    3736 
    3737  enum FLAGS
    3738  {
    3739  FLAG_USER_DATA_STRING = 0x01,
    3740  };
    3741 
    3742 public:
    3743  enum ALLOCATION_TYPE
    3744  {
    3745  ALLOCATION_TYPE_NONE,
    3746  ALLOCATION_TYPE_BLOCK,
    3747  ALLOCATION_TYPE_DEDICATED,
    3748  };
    3749 
    3750  VmaAllocation_T(uint32_t currentFrameIndex, bool userDataString) :
    3751  m_Alignment(1),
    3752  m_Size(0),
    3753  m_pUserData(VMA_NULL),
    3754  m_LastUseFrameIndex(currentFrameIndex),
    3755  m_Type((uint8_t)ALLOCATION_TYPE_NONE),
    3756  m_SuballocationType((uint8_t)VMA_SUBALLOCATION_TYPE_UNKNOWN),
    3757  m_MapCount(0),
    3758  m_Flags(userDataString ? (uint8_t)FLAG_USER_DATA_STRING : 0)
    3759  {
    3760  }
    3761 
    3762  ~VmaAllocation_T()
    3763  {
    3764  VMA_ASSERT((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) == 0 && "Allocation was not unmapped before destruction.");
    3765 
    3766  // Check if owned string was freed.
    3767  VMA_ASSERT(m_pUserData == VMA_NULL);
    3768  }
    3769 
    3770  void InitBlockAllocation(
    3771  VmaPool hPool,
    3772  VmaDeviceMemoryBlock* block,
    3773  VkDeviceSize offset,
    3774  VkDeviceSize alignment,
    3775  VkDeviceSize size,
    3776  VmaSuballocationType suballocationType,
    3777  bool mapped,
    3778  bool canBecomeLost)
    3779  {
    3780  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
    3781  VMA_ASSERT(block != VMA_NULL);
    3782  m_Type = (uint8_t)ALLOCATION_TYPE_BLOCK;
    3783  m_Alignment = alignment;
    3784  m_Size = size;
    3785  m_MapCount = mapped ? MAP_COUNT_FLAG_PERSISTENT_MAP : 0;
    3786  m_SuballocationType = (uint8_t)suballocationType;
    3787  m_BlockAllocation.m_hPool = hPool;
    3788  m_BlockAllocation.m_Block = block;
    3789  m_BlockAllocation.m_Offset = offset;
    3790  m_BlockAllocation.m_CanBecomeLost = canBecomeLost;
    3791  }
    3792 
    3793  void InitLost()
    3794  {
    3795  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
    3796  VMA_ASSERT(m_LastUseFrameIndex.load() == VMA_FRAME_INDEX_LOST);
    3797  m_Type = (uint8_t)ALLOCATION_TYPE_BLOCK;
    3798  m_BlockAllocation.m_hPool = VK_NULL_HANDLE;
    3799  m_BlockAllocation.m_Block = VMA_NULL;
    3800  m_BlockAllocation.m_Offset = 0;
    3801  m_BlockAllocation.m_CanBecomeLost = true;
    3802  }
    3803 
    3804  void ChangeBlockAllocation(
    3805  VmaAllocator hAllocator,
    3806  VmaDeviceMemoryBlock* block,
    3807  VkDeviceSize offset);
    3808 
    3809  // pMappedData not null means allocation is created with MAPPED flag.
    3810  void InitDedicatedAllocation(
    3811  uint32_t memoryTypeIndex,
    3812  VkDeviceMemory hMemory,
    3813  VmaSuballocationType suballocationType,
    3814  void* pMappedData,
    3815  VkDeviceSize size)
    3816  {
    3817  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
    3818  VMA_ASSERT(hMemory != VK_NULL_HANDLE);
    3819  m_Type = (uint8_t)ALLOCATION_TYPE_DEDICATED;
    3820  m_Alignment = 0;
    3821  m_Size = size;
    3822  m_SuballocationType = (uint8_t)suballocationType;
    3823  m_MapCount = (pMappedData != VMA_NULL) ? MAP_COUNT_FLAG_PERSISTENT_MAP : 0;
    3824  m_DedicatedAllocation.m_MemoryTypeIndex = memoryTypeIndex;
    3825  m_DedicatedAllocation.m_hMemory = hMemory;
    3826  m_DedicatedAllocation.m_pMappedData = pMappedData;
    3827  }
    3828 
    3829  ALLOCATION_TYPE GetType() const { return (ALLOCATION_TYPE)m_Type; }
    3830  VkDeviceSize GetAlignment() const { return m_Alignment; }
    3831  VkDeviceSize GetSize() const { return m_Size; }
    3832  bool IsUserDataString() const { return (m_Flags & FLAG_USER_DATA_STRING) != 0; }
    3833  void* GetUserData() const { return m_pUserData; }
    3834  void SetUserData(VmaAllocator hAllocator, void* pUserData);
    3835  VmaSuballocationType GetSuballocationType() const { return (VmaSuballocationType)m_SuballocationType; }
    3836 
    3837  VmaDeviceMemoryBlock* GetBlock() const
    3838  {
    3839  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
    3840  return m_BlockAllocation.m_Block;
    3841  }
    3842  VkDeviceSize GetOffset() const;
    3843  VkDeviceMemory GetMemory() const;
    3844  uint32_t GetMemoryTypeIndex() const;
    3845  bool IsPersistentMap() const { return (m_MapCount & MAP_COUNT_FLAG_PERSISTENT_MAP) != 0; }
    3846  void* GetMappedData() const;
    3847  bool CanBecomeLost() const;
    3848  VmaPool GetPool() const;
    3849 
    3850  uint32_t GetLastUseFrameIndex() const
    3851  {
    3852  return m_LastUseFrameIndex.load();
    3853  }
    3854  bool CompareExchangeLastUseFrameIndex(uint32_t& expected, uint32_t desired)
    3855  {
    3856  return m_LastUseFrameIndex.compare_exchange_weak(expected, desired);
    3857  }
    3858  /*
    3859  - If hAllocation.LastUseFrameIndex + frameInUseCount < allocator.CurrentFrameIndex,
    3860  makes it lost by setting LastUseFrameIndex = VMA_FRAME_INDEX_LOST and returns true.
    3861  - Else, returns false.
    3862 
    3863  If hAllocation is already lost, assert - you should not call it then.
    3864  If hAllocation was not created with CAN_BECOME_LOST_BIT, assert.
    3865  */
    3866  bool MakeLost(uint32_t currentFrameIndex, uint32_t frameInUseCount);
    3867 
    3868  void DedicatedAllocCalcStatsInfo(VmaStatInfo& outInfo)
    3869  {
    3870  VMA_ASSERT(m_Type == ALLOCATION_TYPE_DEDICATED);
    3871  outInfo.blockCount = 1;
    3872  outInfo.allocationCount = 1;
    3873  outInfo.unusedRangeCount = 0;
    3874  outInfo.usedBytes = m_Size;
    3875  outInfo.unusedBytes = 0;
    3876  outInfo.allocationSizeMin = outInfo.allocationSizeMax = m_Size;
    3877  outInfo.unusedRangeSizeMin = UINT64_MAX;
    3878  outInfo.unusedRangeSizeMax = 0;
    3879  }
    3880 
    3881  void BlockAllocMap();
    3882  void BlockAllocUnmap();
    3883  VkResult DedicatedAllocMap(VmaAllocator hAllocator, void** ppData);
    3884  void DedicatedAllocUnmap(VmaAllocator hAllocator);
    3885 
    3886 private:
    3887  VkDeviceSize m_Alignment;
    3888  VkDeviceSize m_Size;
    3889  void* m_pUserData;
    3890  VMA_ATOMIC_UINT32 m_LastUseFrameIndex;
    3891  uint8_t m_Type; // ALLOCATION_TYPE
    3892  uint8_t m_SuballocationType; // VmaSuballocationType
    3893  // Bit 0x80 is set when allocation was created with VMA_ALLOCATION_CREATE_MAPPED_BIT.
    3894  // Bits with mask 0x7F are reference counter for vmaMapMemory()/vmaUnmapMemory().
    3895  uint8_t m_MapCount;
    3896  uint8_t m_Flags; // enum FLAGS
    3897 
    3898  // Allocation out of VmaDeviceMemoryBlock.
    3899  struct BlockAllocation
    3900  {
    3901  VmaPool m_hPool; // Null if belongs to general memory.
    3902  VmaDeviceMemoryBlock* m_Block;
    3903  VkDeviceSize m_Offset;
    3904  bool m_CanBecomeLost;
    3905  };
    3906 
    3907  // Allocation for an object that has its own private VkDeviceMemory.
    3908  struct DedicatedAllocation
    3909  {
    3910  uint32_t m_MemoryTypeIndex;
    3911  VkDeviceMemory m_hMemory;
    3912  void* m_pMappedData; // Not null means memory is mapped.
    3913  };
    3914 
    3915  union
    3916  {
    3917  // Allocation out of VmaDeviceMemoryBlock.
    3918  BlockAllocation m_BlockAllocation;
    3919  // Allocation for an object that has its own private VkDeviceMemory.
    3920  DedicatedAllocation m_DedicatedAllocation;
    3921  };
    3922 
    3923  void FreeUserDataString(VmaAllocator hAllocator);
    3924 };
    3925 
    3926 /*
    3927 Represents a region of VmaDeviceMemoryBlock that is either assigned and returned as
    3928 allocated memory block or free.
    3929 */
    3930 struct VmaSuballocation
    3931 {
    3932  VkDeviceSize offset;
    3933  VkDeviceSize size;
    3934  VmaAllocation hAllocation;
    3935  VmaSuballocationType type;
    3936 };
    3937 
    3938 typedef VmaList< VmaSuballocation, VmaStlAllocator<VmaSuballocation> > VmaSuballocationList;
    3939 
    3940 // Cost of one additional allocation lost, as equivalent in bytes.
    3941 static const VkDeviceSize VMA_LOST_ALLOCATION_COST = 1048576;
    3942 
    3943 /*
    3944 Parameters of planned allocation inside a VmaDeviceMemoryBlock.
    3945 
    3946 If canMakeOtherLost was false:
    3947 - item points to a FREE suballocation.
    3948 - itemsToMakeLostCount is 0.
    3949 
    3950 If canMakeOtherLost was true:
    3951 - item points to first of sequence of suballocations, which are either FREE,
    3952  or point to VmaAllocations that can become lost.
    3953 - itemsToMakeLostCount is the number of VmaAllocations that need to be made lost for
    3954  the requested allocation to succeed.
    3955 */
    3956 struct VmaAllocationRequest
    3957 {
    3958  VkDeviceSize offset;
    3959  VkDeviceSize sumFreeSize; // Sum size of free items that overlap with proposed allocation.
    3960  VkDeviceSize sumItemSize; // Sum size of items to make lost that overlap with proposed allocation.
    3961  VmaSuballocationList::iterator item;
    3962  size_t itemsToMakeLostCount;
    3963 
    3964  VkDeviceSize CalcCost() const
    3965  {
    3966  return sumItemSize + itemsToMakeLostCount * VMA_LOST_ALLOCATION_COST;
    3967  }
    3968 };
    3969 
    3970 /*
    3971 Data structure used for bookkeeping of allocations and unused ranges of memory
    3972 in a single VkDeviceMemory block.
    3973 */
    3974 class VmaBlockMetadata
    3975 {
    3976 public:
    3977  VmaBlockMetadata(VmaAllocator hAllocator);
    3978  ~VmaBlockMetadata();
    3979  void Init(VkDeviceSize size);
    3980 
    3981  // Validates all data structures inside this object. If not valid, returns false.
    3982  bool Validate() const;
    3983  VkDeviceSize GetSize() const { return m_Size; }
    3984  size_t GetAllocationCount() const { return m_Suballocations.size() - m_FreeCount; }
    3985  VkDeviceSize GetSumFreeSize() const { return m_SumFreeSize; }
    3986  VkDeviceSize GetUnusedRangeSizeMax() const;
    3987  // Returns true if this block is empty - contains only single free suballocation.
    3988  bool IsEmpty() const;
    3989 
    3990  void CalcAllocationStatInfo(VmaStatInfo& outInfo) const;
    3991  void AddPoolStats(VmaPoolStats& inoutStats) const;
    3992 
    3993 #if VMA_STATS_STRING_ENABLED
    3994  void PrintDetailedMap(class VmaJsonWriter& json) const;
    3995 #endif
    3996 
    3997  // Creates trivial request for case when block is empty.
    3998  void CreateFirstAllocationRequest(VmaAllocationRequest* pAllocationRequest);
    3999 
    4000  // Tries to find a place for suballocation with given parameters inside this block.
    4001  // If succeeded, fills pAllocationRequest and returns true.
    4002  // If failed, returns false.
    4003  bool CreateAllocationRequest(
    4004  uint32_t currentFrameIndex,
    4005  uint32_t frameInUseCount,
    4006  VkDeviceSize bufferImageGranularity,
    4007  VkDeviceSize allocSize,
    4008  VkDeviceSize allocAlignment,
    4009  VmaSuballocationType allocType,
    4010  bool canMakeOtherLost,
    4011  VmaAllocationRequest* pAllocationRequest);
    4012 
    4013  bool MakeRequestedAllocationsLost(
    4014  uint32_t currentFrameIndex,
    4015  uint32_t frameInUseCount,
    4016  VmaAllocationRequest* pAllocationRequest);
    4017 
    4018  uint32_t MakeAllocationsLost(uint32_t currentFrameIndex, uint32_t frameInUseCount);
    4019 
    4020  // Makes actual allocation based on request. Request must already be checked and valid.
    4021  void Alloc(
    4022  const VmaAllocationRequest& request,
    4023  VmaSuballocationType type,
    4024  VkDeviceSize allocSize,
    4025  VmaAllocation hAllocation);
    4026 
    4027  // Frees suballocation assigned to given memory region.
    4028  void Free(const VmaAllocation allocation);
    4029  void FreeAtOffset(VkDeviceSize offset);
    4030 
    4031 private:
    4032  VkDeviceSize m_Size;
    4033  uint32_t m_FreeCount;
    4034  VkDeviceSize m_SumFreeSize;
    4035  VmaSuballocationList m_Suballocations;
    4036  // Suballocations that are free and have size greater than certain threshold.
    4037  // Sorted by size, ascending.
    4038  VmaVector< VmaSuballocationList::iterator, VmaStlAllocator< VmaSuballocationList::iterator > > m_FreeSuballocationsBySize;
    4039 
    4040  bool ValidateFreeSuballocationList() const;
    4041 
    4042  // Checks if requested suballocation with given parameters can be placed in given pFreeSuballocItem.
    4043  // If yes, fills pOffset and returns true. If no, returns false.
    4044  bool CheckAllocation(
    4045  uint32_t currentFrameIndex,
    4046  uint32_t frameInUseCount,
    4047  VkDeviceSize bufferImageGranularity,
    4048  VkDeviceSize allocSize,
    4049  VkDeviceSize allocAlignment,
    4050  VmaSuballocationType allocType,
    4051  VmaSuballocationList::const_iterator suballocItem,
    4052  bool canMakeOtherLost,
    4053  VkDeviceSize* pOffset,
    4054  size_t* itemsToMakeLostCount,
    4055  VkDeviceSize* pSumFreeSize,
    4056  VkDeviceSize* pSumItemSize) const;
    4057  // Given free suballocation, it merges it with following one, which must also be free.
    4058  void MergeFreeWithNext(VmaSuballocationList::iterator item);
    4059  // Releases given suballocation, making it free.
    4060  // Merges it with adjacent free suballocations if applicable.
    4061  // Returns iterator to new free suballocation at this place.
    4062  VmaSuballocationList::iterator FreeSuballocation(VmaSuballocationList::iterator suballocItem);
    4063  // Given free suballocation, it inserts it into sorted list of
    4064  // m_FreeSuballocationsBySize if it's suitable.
    4065  void RegisterFreeSuballocation(VmaSuballocationList::iterator item);
    4066  // Given free suballocation, it removes it from sorted list of
    4067  // m_FreeSuballocationsBySize if it's suitable.
    4068  void UnregisterFreeSuballocation(VmaSuballocationList::iterator item);
    4069 };
    4070 
    4071 /*
    4072 Represents a single block of device memory (`VkDeviceMemory`) with all the
    4073 data about its regions (aka suballocations, #VmaAllocation), assigned and free.
    4074 
    4075 Thread-safety: This class must be externally synchronized.
    4076 */
    4077 class VmaDeviceMemoryBlock
    4078 {
    4079 public:
    4080  VmaBlockMetadata m_Metadata;
    4081 
    4082  VmaDeviceMemoryBlock(VmaAllocator hAllocator);
    4083 
    4084  ~VmaDeviceMemoryBlock()
    4085  {
    4086  VMA_ASSERT(m_MapCount == 0 && "VkDeviceMemory block is being destroyed while it is still mapped.");
    4087  VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
    4088  }
    4089 
    4090  // Always call after construction.
    4091  void Init(
    4092  uint32_t newMemoryTypeIndex,
    4093  VkDeviceMemory newMemory,
    4094  VkDeviceSize newSize);
    4095  // Always call before destruction.
    4096  void Destroy(VmaAllocator allocator);
    4097 
    4098  VkDeviceMemory GetDeviceMemory() const { return m_hMemory; }
    4099  uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
    4100  void* GetMappedData() const { return m_pMappedData; }
    4101 
    4102  // Validates all data structures inside this object. If not valid, returns false.
    4103  bool Validate() const;
    4104 
    4105  // ppData can be null.
    4106  VkResult Map(VmaAllocator hAllocator, uint32_t count, void** ppData);
    4107  void Unmap(VmaAllocator hAllocator, uint32_t count);
    4108 
    4109  VkResult BindBufferMemory(
    4110  const VmaAllocator hAllocator,
    4111  const VmaAllocation hAllocation,
    4112  VkBuffer hBuffer);
    4113  VkResult BindImageMemory(
    4114  const VmaAllocator hAllocator,
    4115  const VmaAllocation hAllocation,
    4116  VkImage hImage);
    4117 
    4118 private:
    4119  uint32_t m_MemoryTypeIndex;
    4120  VkDeviceMemory m_hMemory;
    4121 
    4122  // Protects access to m_hMemory so it's not used by multiple threads simultaneously, e.g. vkMapMemory, vkBindBufferMemory.
    4123  // Also protects m_MapCount, m_pMappedData.
    4124  VMA_MUTEX m_Mutex;
    4125  uint32_t m_MapCount;
    4126  void* m_pMappedData;
    4127 };
    4128 
    4129 struct VmaPointerLess
    4130 {
    4131  bool operator()(const void* lhs, const void* rhs) const
    4132  {
    4133  return lhs < rhs;
    4134  }
    4135 };
    4136 
    4137 class VmaDefragmentator;
    4138 
    4139 /*
    4140 Sequence of VmaDeviceMemoryBlock. Represents memory blocks allocated for a specific
    4141 Vulkan memory type.
    4142 
    4143 Synchronized internally with a mutex.
    4144 */
    4145 struct VmaBlockVector
    4146 {
    4147  VmaBlockVector(
    4148  VmaAllocator hAllocator,
    4149  uint32_t memoryTypeIndex,
    4150  VkDeviceSize preferredBlockSize,
    4151  size_t minBlockCount,
    4152  size_t maxBlockCount,
    4153  VkDeviceSize bufferImageGranularity,
    4154  uint32_t frameInUseCount,
    4155  bool isCustomPool);
    4156  ~VmaBlockVector();
    4157 
    4158  VkResult CreateMinBlocks();
    4159 
    4160  uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
    4161  VkDeviceSize GetPreferredBlockSize() const { return m_PreferredBlockSize; }
    4162  VkDeviceSize GetBufferImageGranularity() const { return m_BufferImageGranularity; }
    4163  uint32_t GetFrameInUseCount() const { return m_FrameInUseCount; }
    4164 
    4165  void GetPoolStats(VmaPoolStats* pStats);
    4166 
    4167  bool IsEmpty() const { return m_Blocks.empty(); }
    4168 
    4169  VkResult Allocate(
    4170  VmaPool hCurrentPool,
    4171  uint32_t currentFrameIndex,
    4172  const VkMemoryRequirements& vkMemReq,
    4173  const VmaAllocationCreateInfo& createInfo,
    4174  VmaSuballocationType suballocType,
    4175  VmaAllocation* pAllocation);
    4176 
    4177  void Free(
    4178  VmaAllocation hAllocation);
    4179 
    4180  // Adds statistics of this BlockVector to pStats.
    4181  void AddStats(VmaStats* pStats);
    4182 
    4183 #if VMA_STATS_STRING_ENABLED
    4184  void PrintDetailedMap(class VmaJsonWriter& json);
    4185 #endif
    4186 
    4187  void MakePoolAllocationsLost(
    4188  uint32_t currentFrameIndex,
    4189  size_t* pLostAllocationCount);
    4190 
    4191  VmaDefragmentator* EnsureDefragmentator(
    4192  VmaAllocator hAllocator,
    4193  uint32_t currentFrameIndex);
    4194 
    4195  VkResult Defragment(
    4196  VmaDefragmentationStats* pDefragmentationStats,
    4197  VkDeviceSize& maxBytesToMove,
    4198  uint32_t& maxAllocationsToMove);
    4199 
    4200  void DestroyDefragmentator();
    4201 
    4202 private:
    4203  friend class VmaDefragmentator;
    4204 
    4205  const VmaAllocator m_hAllocator;
    4206  const uint32_t m_MemoryTypeIndex;
    4207  const VkDeviceSize m_PreferredBlockSize;
    4208  const size_t m_MinBlockCount;
    4209  const size_t m_MaxBlockCount;
    4210  const VkDeviceSize m_BufferImageGranularity;
    4211  const uint32_t m_FrameInUseCount;
    4212  const bool m_IsCustomPool;
    4213  VMA_MUTEX m_Mutex;
    4214  // Incrementally sorted by sumFreeSize, ascending.
    4215  VmaVector< VmaDeviceMemoryBlock*, VmaStlAllocator<VmaDeviceMemoryBlock*> > m_Blocks;
    4216  /* There can be at most one allocation that is completely empty - a
    4217  hysteresis to avoid pessimistic case of alternating creation and destruction
    4218  of a VkDeviceMemory. */
    4219  bool m_HasEmptyBlock;
    4220  VmaDefragmentator* m_pDefragmentator;
    4221 
    4222  size_t CalcMaxBlockSize() const;
    4223 
    4224  // Finds and removes given block from vector.
    4225  void Remove(VmaDeviceMemoryBlock* pBlock);
    4226 
    4227  // Performs single step in sorting m_Blocks. They may not be fully sorted
    4228  // after this call.
    4229  void IncrementallySortBlocks();
    4230 
    4231  VkResult CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex);
    4232 };
    4233 
    4234 struct VmaPool_T
    4235 {
    4236 public:
    4237  VmaBlockVector m_BlockVector;
    4238 
    4239  // Takes ownership.
    4240  VmaPool_T(
    4241  VmaAllocator hAllocator,
    4242  const VmaPoolCreateInfo& createInfo);
    4243  ~VmaPool_T();
    4244 
    4245  VmaBlockVector& GetBlockVector() { return m_BlockVector; }
    4246 
    4247 #if VMA_STATS_STRING_ENABLED
    4248  //void PrintDetailedMap(class VmaStringBuilder& sb);
    4249 #endif
    4250 };
    4251 
    4252 class VmaDefragmentator
    4253 {
    4254  const VmaAllocator m_hAllocator;
    4255  VmaBlockVector* const m_pBlockVector;
    4256  uint32_t m_CurrentFrameIndex;
    4257  VkDeviceSize m_BytesMoved;
    4258  uint32_t m_AllocationsMoved;
    4259 
    4260  struct AllocationInfo
    4261  {
    4262  VmaAllocation m_hAllocation;
    4263  VkBool32* m_pChanged;
    4264 
    4265  AllocationInfo() :
    4266  m_hAllocation(VK_NULL_HANDLE),
    4267  m_pChanged(VMA_NULL)
    4268  {
    4269  }
    4270  };
    4271 
    4272  struct AllocationInfoSizeGreater
    4273  {
    4274  bool operator()(const AllocationInfo& lhs, const AllocationInfo& rhs) const
    4275  {
    4276  return lhs.m_hAllocation->GetSize() > rhs.m_hAllocation->GetSize();
    4277  }
    4278  };
    4279 
    4280  // Used between AddAllocation and Defragment.
    4281  VmaVector< AllocationInfo, VmaStlAllocator<AllocationInfo> > m_Allocations;
    4282 
    4283  struct BlockInfo
    4284  {
    4285  VmaDeviceMemoryBlock* m_pBlock;
    4286  bool m_HasNonMovableAllocations;
    4287  VmaVector< AllocationInfo, VmaStlAllocator<AllocationInfo> > m_Allocations;
    4288 
    4289  BlockInfo(const VkAllocationCallbacks* pAllocationCallbacks) :
    4290  m_pBlock(VMA_NULL),
    4291  m_HasNonMovableAllocations(true),
    4292  m_Allocations(pAllocationCallbacks),
    4293  m_pMappedDataForDefragmentation(VMA_NULL)
    4294  {
    4295  }
    4296 
    4297  void CalcHasNonMovableAllocations()
    4298  {
    4299  const size_t blockAllocCount = m_pBlock->m_Metadata.GetAllocationCount();
    4300  const size_t defragmentAllocCount = m_Allocations.size();
    4301  m_HasNonMovableAllocations = blockAllocCount != defragmentAllocCount;
    4302  }
    4303 
    4304  void SortAllocationsBySizeDescecnding()
    4305  {
    4306  VMA_SORT(m_Allocations.begin(), m_Allocations.end(), AllocationInfoSizeGreater());
    4307  }
    4308 
    4309  VkResult EnsureMapping(VmaAllocator hAllocator, void** ppMappedData);
    4310  void Unmap(VmaAllocator hAllocator);
    4311 
    4312  private:
    4313  // Not null if mapped for defragmentation only, not originally mapped.
    4314  void* m_pMappedDataForDefragmentation;
    4315  };
    4316 
    4317  struct BlockPointerLess
    4318  {
    4319  bool operator()(const BlockInfo* pLhsBlockInfo, const VmaDeviceMemoryBlock* pRhsBlock) const
    4320  {
    4321  return pLhsBlockInfo->m_pBlock < pRhsBlock;
    4322  }
    4323  bool operator()(const BlockInfo* pLhsBlockInfo, const BlockInfo* pRhsBlockInfo) const
    4324  {
    4325  return pLhsBlockInfo->m_pBlock < pRhsBlockInfo->m_pBlock;
    4326  }
    4327  };
    4328 
    4329  // 1. Blocks with some non-movable allocations go first.
    4330  // 2. Blocks with smaller sumFreeSize go first.
    4331  struct BlockInfoCompareMoveDestination
    4332  {
    4333  bool operator()(const BlockInfo* pLhsBlockInfo, const BlockInfo* pRhsBlockInfo) const
    4334  {
    4335  if(pLhsBlockInfo->m_HasNonMovableAllocations && !pRhsBlockInfo->m_HasNonMovableAllocations)
    4336  {
    4337  return true;
    4338  }
    4339  if(!pLhsBlockInfo->m_HasNonMovableAllocations && pRhsBlockInfo->m_HasNonMovableAllocations)
    4340  {
    4341  return false;
    4342  }
    4343  if(pLhsBlockInfo->m_pBlock->m_Metadata.GetSumFreeSize() < pRhsBlockInfo->m_pBlock->m_Metadata.GetSumFreeSize())
    4344  {
    4345  return true;
    4346  }
    4347  return false;
    4348  }
    4349  };
    4350 
    4351  typedef VmaVector< BlockInfo*, VmaStlAllocator<BlockInfo*> > BlockInfoVector;
    4352  BlockInfoVector m_Blocks;
    4353 
    4354  VkResult DefragmentRound(
    4355  VkDeviceSize maxBytesToMove,
    4356  uint32_t maxAllocationsToMove);
    4357 
    4358  static bool MoveMakesSense(
    4359  size_t dstBlockIndex, VkDeviceSize dstOffset,
    4360  size_t srcBlockIndex, VkDeviceSize srcOffset);
    4361 
    4362 public:
    4363  VmaDefragmentator(
    4364  VmaAllocator hAllocator,
    4365  VmaBlockVector* pBlockVector,
    4366  uint32_t currentFrameIndex);
    4367 
    4368  ~VmaDefragmentator();
    4369 
    4370  VkDeviceSize GetBytesMoved() const { return m_BytesMoved; }
    4371  uint32_t GetAllocationsMoved() const { return m_AllocationsMoved; }
    4372 
    4373  void AddAllocation(VmaAllocation hAlloc, VkBool32* pChanged);
    4374 
    4375  VkResult Defragment(
    4376  VkDeviceSize maxBytesToMove,
    4377  uint32_t maxAllocationsToMove);
    4378 };
    4379 
    4380 // Main allocator object.
    4381 struct VmaAllocator_T
    4382 {
    4383  bool m_UseMutex;
    4384  bool m_UseKhrDedicatedAllocation;
    4385  VkDevice m_hDevice;
    4386  bool m_AllocationCallbacksSpecified;
    4387  VkAllocationCallbacks m_AllocationCallbacks;
    4388  VmaDeviceMemoryCallbacks m_DeviceMemoryCallbacks;
    4389 
    4390  // Number of bytes free out of limit, or VK_WHOLE_SIZE if not limit for that heap.
    4391  VkDeviceSize m_HeapSizeLimit[VK_MAX_MEMORY_HEAPS];
    4392  VMA_MUTEX m_HeapSizeLimitMutex;
    4393 
    4394  VkPhysicalDeviceProperties m_PhysicalDeviceProperties;
    4395  VkPhysicalDeviceMemoryProperties m_MemProps;
    4396 
    4397  // Default pools.
    4398  VmaBlockVector* m_pBlockVectors[VK_MAX_MEMORY_TYPES];
    4399 
    4400  // Each vector is sorted by memory (handle value).
    4401  typedef VmaVector< VmaAllocation, VmaStlAllocator<VmaAllocation> > AllocationVectorType;
    4402  AllocationVectorType* m_pDedicatedAllocations[VK_MAX_MEMORY_TYPES];
    4403  VMA_MUTEX m_DedicatedAllocationsMutex[VK_MAX_MEMORY_TYPES];
    4404 
    4405  VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo);
    4406  ~VmaAllocator_T();
    4407 
    4408  const VkAllocationCallbacks* GetAllocationCallbacks() const
    4409  {
    4410  return m_AllocationCallbacksSpecified ? &m_AllocationCallbacks : 0;
    4411  }
    4412  const VmaVulkanFunctions& GetVulkanFunctions() const
    4413  {
    4414  return m_VulkanFunctions;
    4415  }
    4416 
    4417  VkDeviceSize GetBufferImageGranularity() const
    4418  {
    4419  return VMA_MAX(
    4420  static_cast<VkDeviceSize>(VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY),
    4421  m_PhysicalDeviceProperties.limits.bufferImageGranularity);
    4422  }
    4423 
    4424  uint32_t GetMemoryHeapCount() const { return m_MemProps.memoryHeapCount; }
    4425  uint32_t GetMemoryTypeCount() const { return m_MemProps.memoryTypeCount; }
    4426 
    4427  uint32_t MemoryTypeIndexToHeapIndex(uint32_t memTypeIndex) const
    4428  {
    4429  VMA_ASSERT(memTypeIndex < m_MemProps.memoryTypeCount);
    4430  return m_MemProps.memoryTypes[memTypeIndex].heapIndex;
    4431  }
    4432 
    4433  void GetBufferMemoryRequirements(
    4434  VkBuffer hBuffer,
    4435  VkMemoryRequirements& memReq,
    4436  bool& requiresDedicatedAllocation,
    4437  bool& prefersDedicatedAllocation) const;
    4438  void GetImageMemoryRequirements(
    4439  VkImage hImage,
    4440  VkMemoryRequirements& memReq,
    4441  bool& requiresDedicatedAllocation,
    4442  bool& prefersDedicatedAllocation) const;
    4443 
    4444  // Main allocation function.
    4445  VkResult AllocateMemory(
    4446  const VkMemoryRequirements& vkMemReq,
    4447  bool requiresDedicatedAllocation,
    4448  bool prefersDedicatedAllocation,
    4449  VkBuffer dedicatedBuffer,
    4450  VkImage dedicatedImage,
    4451  const VmaAllocationCreateInfo& createInfo,
    4452  VmaSuballocationType suballocType,
    4453  VmaAllocation* pAllocation);
    4454 
    4455  // Main deallocation function.
    4456  void FreeMemory(const VmaAllocation allocation);
    4457 
    4458  void CalculateStats(VmaStats* pStats);
    4459 
    4460 #if VMA_STATS_STRING_ENABLED
    4461  void PrintDetailedMap(class VmaJsonWriter& json);
    4462 #endif
    4463 
    4464  VkResult Defragment(
    4465  VmaAllocation* pAllocations,
    4466  size_t allocationCount,
    4467  VkBool32* pAllocationsChanged,
    4468  const VmaDefragmentationInfo* pDefragmentationInfo,
    4469  VmaDefragmentationStats* pDefragmentationStats);
    4470 
    4471  void GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo);
    4472  bool TouchAllocation(VmaAllocation hAllocation);
    4473 
    4474  VkResult CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool);
    4475  void DestroyPool(VmaPool pool);
    4476  void GetPoolStats(VmaPool pool, VmaPoolStats* pPoolStats);
    4477 
    4478  void SetCurrentFrameIndex(uint32_t frameIndex);
    4479 
    4480  void MakePoolAllocationsLost(
    4481  VmaPool hPool,
    4482  size_t* pLostAllocationCount);
    4483 
    4484  void CreateLostAllocation(VmaAllocation* pAllocation);
    4485 
    4486  VkResult AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory);
    4487  void FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory);
    4488 
    4489  VkResult Map(VmaAllocation hAllocation, void** ppData);
    4490  void Unmap(VmaAllocation hAllocation);
    4491 
    4492  VkResult BindBufferMemory(VmaAllocation hAllocation, VkBuffer hBuffer);
    4493  VkResult BindImageMemory(VmaAllocation hAllocation, VkImage hImage);
    4494 
    4495 private:
    4496  VkDeviceSize m_PreferredLargeHeapBlockSize;
    4497 
    4498  VkPhysicalDevice m_PhysicalDevice;
    4499  VMA_ATOMIC_UINT32 m_CurrentFrameIndex;
    4500 
    4501  VMA_MUTEX m_PoolsMutex;
    4502  // Protected by m_PoolsMutex. Sorted by pointer value.
    4503  VmaVector<VmaPool, VmaStlAllocator<VmaPool> > m_Pools;
    4504 
    4505  VmaVulkanFunctions m_VulkanFunctions;
    4506 
    4507  void ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions);
    4508 
    4509  VkDeviceSize CalcPreferredBlockSize(uint32_t memTypeIndex);
    4510 
    4511  VkResult AllocateMemoryOfType(
    4512  const VkMemoryRequirements& vkMemReq,
    4513  bool dedicatedAllocation,
    4514  VkBuffer dedicatedBuffer,
    4515  VkImage dedicatedImage,
    4516  const VmaAllocationCreateInfo& createInfo,
    4517  uint32_t memTypeIndex,
    4518  VmaSuballocationType suballocType,
    4519  VmaAllocation* pAllocation);
    4520 
    4521  // Allocates and registers new VkDeviceMemory specifically for single allocation.
    4522  VkResult AllocateDedicatedMemory(
    4523  VkDeviceSize size,
    4524  VmaSuballocationType suballocType,
    4525  uint32_t memTypeIndex,
    4526  bool map,
    4527  bool isUserDataString,
    4528  void* pUserData,
    4529  VkBuffer dedicatedBuffer,
    4530  VkImage dedicatedImage,
    4531  VmaAllocation* pAllocation);
    4532 
    4533  // Tries to free pMemory as Dedicated Memory. Returns true if found and freed.
    4534  void FreeDedicatedMemory(VmaAllocation allocation);
    4535 };
    4536 
    4538 // Memory allocation #2 after VmaAllocator_T definition
    4539 
    4540 static void* VmaMalloc(VmaAllocator hAllocator, size_t size, size_t alignment)
    4541 {
    4542  return VmaMalloc(&hAllocator->m_AllocationCallbacks, size, alignment);
    4543 }
    4544 
    4545 static void VmaFree(VmaAllocator hAllocator, void* ptr)
    4546 {
    4547  VmaFree(&hAllocator->m_AllocationCallbacks, ptr);
    4548 }
    4549 
    4550 template<typename T>
    4551 static T* VmaAllocate(VmaAllocator hAllocator)
    4552 {
    4553  return (T*)VmaMalloc(hAllocator, sizeof(T), VMA_ALIGN_OF(T));
    4554 }
    4555 
    4556 template<typename T>
    4557 static T* VmaAllocateArray(VmaAllocator hAllocator, size_t count)
    4558 {
    4559  return (T*)VmaMalloc(hAllocator, sizeof(T) * count, VMA_ALIGN_OF(T));
    4560 }
    4561 
    4562 template<typename T>
    4563 static void vma_delete(VmaAllocator hAllocator, T* ptr)
    4564 {
    4565  if(ptr != VMA_NULL)
    4566  {
    4567  ptr->~T();
    4568  VmaFree(hAllocator, ptr);
    4569  }
    4570 }
    4571 
    4572 template<typename T>
    4573 static void vma_delete_array(VmaAllocator hAllocator, T* ptr, size_t count)
    4574 {
    4575  if(ptr != VMA_NULL)
    4576  {
    4577  for(size_t i = count; i--; )
    4578  ptr[i].~T();
    4579  VmaFree(hAllocator, ptr);
    4580  }
    4581 }
    4582 
    4584 // VmaStringBuilder
    4585 
    4586 #if VMA_STATS_STRING_ENABLED
    4587 
    4588 class VmaStringBuilder
    4589 {
    4590 public:
    4591  VmaStringBuilder(VmaAllocator alloc) : m_Data(VmaStlAllocator<char>(alloc->GetAllocationCallbacks())) { }
    4592  size_t GetLength() const { return m_Data.size(); }
    4593  const char* GetData() const { return m_Data.data(); }
    4594 
    4595  void Add(char ch) { m_Data.push_back(ch); }
    4596  void Add(const char* pStr);
    4597  void AddNewLine() { Add('\n'); }
    4598  void AddNumber(uint32_t num);
    4599  void AddNumber(uint64_t num);
    4600  void AddPointer(const void* ptr);
    4601 
    4602 private:
    4603  VmaVector< char, VmaStlAllocator<char> > m_Data;
    4604 };
    4605 
    4606 void VmaStringBuilder::Add(const char* pStr)
    4607 {
    4608  const size_t strLen = strlen(pStr);
    4609  if(strLen > 0)
    4610  {
    4611  const size_t oldCount = m_Data.size();
    4612  m_Data.resize(oldCount + strLen);
    4613  memcpy(m_Data.data() + oldCount, pStr, strLen);
    4614  }
    4615 }
    4616 
    4617 void VmaStringBuilder::AddNumber(uint32_t num)
    4618 {
    4619  char buf[11];
    4620  VmaUint32ToStr(buf, sizeof(buf), num);
    4621  Add(buf);
    4622 }
    4623 
    4624 void VmaStringBuilder::AddNumber(uint64_t num)
    4625 {
    4626  char buf[21];
    4627  VmaUint64ToStr(buf, sizeof(buf), num);
    4628  Add(buf);
    4629 }
    4630 
    4631 void VmaStringBuilder::AddPointer(const void* ptr)
    4632 {
    4633  char buf[21];
    4634  VmaPtrToStr(buf, sizeof(buf), ptr);
    4635  Add(buf);
    4636 }
    4637 
    4638 #endif // #if VMA_STATS_STRING_ENABLED
    4639 
    4641 // VmaJsonWriter
    4642 
    4643 #if VMA_STATS_STRING_ENABLED
    4644 
    4645 class VmaJsonWriter
    4646 {
    4647 public:
    4648  VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb);
    4649  ~VmaJsonWriter();
    4650 
    4651  void BeginObject(bool singleLine = false);
    4652  void EndObject();
    4653 
    4654  void BeginArray(bool singleLine = false);
    4655  void EndArray();
    4656 
    4657  void WriteString(const char* pStr);
    4658  void BeginString(const char* pStr = VMA_NULL);
    4659  void ContinueString(const char* pStr);
    4660  void ContinueString(uint32_t n);
    4661  void ContinueString(uint64_t n);
    4662  void ContinueString_Pointer(const void* ptr);
    4663  void EndString(const char* pStr = VMA_NULL);
    4664 
    4665  void WriteNumber(uint32_t n);
    4666  void WriteNumber(uint64_t n);
    4667  void WriteBool(bool b);
    4668  void WriteNull();
    4669 
    4670 private:
    4671  static const char* const INDENT;
    4672 
    4673  enum COLLECTION_TYPE
    4674  {
    4675  COLLECTION_TYPE_OBJECT,
    4676  COLLECTION_TYPE_ARRAY,
    4677  };
    4678  struct StackItem
    4679  {
    4680  COLLECTION_TYPE type;
    4681  uint32_t valueCount;
    4682  bool singleLineMode;
    4683  };
    4684 
    4685  VmaStringBuilder& m_SB;
    4686  VmaVector< StackItem, VmaStlAllocator<StackItem> > m_Stack;
    4687  bool m_InsideString;
    4688 
    4689  void BeginValue(bool isString);
    4690  void WriteIndent(bool oneLess = false);
    4691 };
    4692 
    4693 const char* const VmaJsonWriter::INDENT = " ";
    4694 
    4695 VmaJsonWriter::VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb) :
    4696  m_SB(sb),
    4697  m_Stack(VmaStlAllocator<StackItem>(pAllocationCallbacks)),
    4698  m_InsideString(false)
    4699 {
    4700 }
    4701 
    4702 VmaJsonWriter::~VmaJsonWriter()
    4703 {
    4704  VMA_ASSERT(!m_InsideString);
    4705  VMA_ASSERT(m_Stack.empty());
    4706 }
    4707 
    4708 void VmaJsonWriter::BeginObject(bool singleLine)
    4709 {
    4710  VMA_ASSERT(!m_InsideString);
    4711 
    4712  BeginValue(false);
    4713  m_SB.Add('{');
    4714 
    4715  StackItem item;
    4716  item.type = COLLECTION_TYPE_OBJECT;
    4717  item.valueCount = 0;
    4718  item.singleLineMode = singleLine;
    4719  m_Stack.push_back(item);
    4720 }
    4721 
    4722 void VmaJsonWriter::EndObject()
    4723 {
    4724  VMA_ASSERT(!m_InsideString);
    4725 
    4726  WriteIndent(true);
    4727  m_SB.Add('}');
    4728 
    4729  VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_OBJECT);
    4730  m_Stack.pop_back();
    4731 }
    4732 
    4733 void VmaJsonWriter::BeginArray(bool singleLine)
    4734 {
    4735  VMA_ASSERT(!m_InsideString);
    4736 
    4737  BeginValue(false);
    4738  m_SB.Add('[');
    4739 
    4740  StackItem item;
    4741  item.type = COLLECTION_TYPE_ARRAY;
    4742  item.valueCount = 0;
    4743  item.singleLineMode = singleLine;
    4744  m_Stack.push_back(item);
    4745 }
    4746 
    4747 void VmaJsonWriter::EndArray()
    4748 {
    4749  VMA_ASSERT(!m_InsideString);
    4750 
    4751  WriteIndent(true);
    4752  m_SB.Add(']');
    4753 
    4754  VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_ARRAY);
    4755  m_Stack.pop_back();
    4756 }
    4757 
    4758 void VmaJsonWriter::WriteString(const char* pStr)
    4759 {
    4760  BeginString(pStr);
    4761  EndString();
    4762 }
    4763 
    4764 void VmaJsonWriter::BeginString(const char* pStr)
    4765 {
    4766  VMA_ASSERT(!m_InsideString);
    4767 
    4768  BeginValue(true);
    4769  m_SB.Add('"');
    4770  m_InsideString = true;
    4771  if(pStr != VMA_NULL && pStr[0] != '\0')
    4772  {
    4773  ContinueString(pStr);
    4774  }
    4775 }
    4776 
    4777 void VmaJsonWriter::ContinueString(const char* pStr)
    4778 {
    4779  VMA_ASSERT(m_InsideString);
    4780 
    4781  const size_t strLen = strlen(pStr);
    4782  for(size_t i = 0; i < strLen; ++i)
    4783  {
    4784  char ch = pStr[i];
    4785  if(ch == '\'')
    4786  {
    4787  m_SB.Add("\\\\");
    4788  }
    4789  else if(ch == '"')
    4790  {
    4791  m_SB.Add("\\\"");
    4792  }
    4793  else if(ch >= 32)
    4794  {
    4795  m_SB.Add(ch);
    4796  }
    4797  else switch(ch)
    4798  {
    4799  case '\b':
    4800  m_SB.Add("\\b");
    4801  break;
    4802  case '\f':
    4803  m_SB.Add("\\f");
    4804  break;
    4805  case '\n':
    4806  m_SB.Add("\\n");
    4807  break;
    4808  case '\r':
    4809  m_SB.Add("\\r");
    4810  break;
    4811  case '\t':
    4812  m_SB.Add("\\t");
    4813  break;
    4814  default:
    4815  VMA_ASSERT(0 && "Character not currently supported.");
    4816  break;
    4817  }
    4818  }
    4819 }
    4820 
    4821 void VmaJsonWriter::ContinueString(uint32_t n)
    4822 {
    4823  VMA_ASSERT(m_InsideString);
    4824  m_SB.AddNumber(n);
    4825 }
    4826 
    4827 void VmaJsonWriter::ContinueString(uint64_t n)
    4828 {
    4829  VMA_ASSERT(m_InsideString);
    4830  m_SB.AddNumber(n);
    4831 }
    4832 
    4833 void VmaJsonWriter::ContinueString_Pointer(const void* ptr)
    4834 {
    4835  VMA_ASSERT(m_InsideString);
    4836  m_SB.AddPointer(ptr);
    4837 }
    4838 
    4839 void VmaJsonWriter::EndString(const char* pStr)
    4840 {
    4841  VMA_ASSERT(m_InsideString);
    4842  if(pStr != VMA_NULL && pStr[0] != '\0')
    4843  {
    4844  ContinueString(pStr);
    4845  }
    4846  m_SB.Add('"');
    4847  m_InsideString = false;
    4848 }
    4849 
    4850 void VmaJsonWriter::WriteNumber(uint32_t n)
    4851 {
    4852  VMA_ASSERT(!m_InsideString);
    4853  BeginValue(false);
    4854  m_SB.AddNumber(n);
    4855 }
    4856 
    4857 void VmaJsonWriter::WriteNumber(uint64_t n)
    4858 {
    4859  VMA_ASSERT(!m_InsideString);
    4860  BeginValue(false);
    4861  m_SB.AddNumber(n);
    4862 }
    4863 
    4864 void VmaJsonWriter::WriteBool(bool b)
    4865 {
    4866  VMA_ASSERT(!m_InsideString);
    4867  BeginValue(false);
    4868  m_SB.Add(b ? "true" : "false");
    4869 }
    4870 
    4871 void VmaJsonWriter::WriteNull()
    4872 {
    4873  VMA_ASSERT(!m_InsideString);
    4874  BeginValue(false);
    4875  m_SB.Add("null");
    4876 }
    4877 
    4878 void VmaJsonWriter::BeginValue(bool isString)
    4879 {
    4880  if(!m_Stack.empty())
    4881  {
    4882  StackItem& currItem = m_Stack.back();
    4883  if(currItem.type == COLLECTION_TYPE_OBJECT &&
    4884  currItem.valueCount % 2 == 0)
    4885  {
    4886  VMA_ASSERT(isString);
    4887  }
    4888 
    4889  if(currItem.type == COLLECTION_TYPE_OBJECT &&
    4890  currItem.valueCount % 2 != 0)
    4891  {
    4892  m_SB.Add(": ");
    4893  }
    4894  else if(currItem.valueCount > 0)
    4895  {
    4896  m_SB.Add(", ");
    4897  WriteIndent();
    4898  }
    4899  else
    4900  {
    4901  WriteIndent();
    4902  }
    4903  ++currItem.valueCount;
    4904  }
    4905 }
    4906 
    4907 void VmaJsonWriter::WriteIndent(bool oneLess)
    4908 {
    4909  if(!m_Stack.empty() && !m_Stack.back().singleLineMode)
    4910  {
    4911  m_SB.AddNewLine();
    4912 
    4913  size_t count = m_Stack.size();
    4914  if(count > 0 && oneLess)
    4915  {
    4916  --count;
    4917  }
    4918  for(size_t i = 0; i < count; ++i)
    4919  {
    4920  m_SB.Add(INDENT);
    4921  }
    4922  }
    4923 }
    4924 
    4925 #endif // #if VMA_STATS_STRING_ENABLED
    4926 
    4928 
    4929 void VmaAllocation_T::SetUserData(VmaAllocator hAllocator, void* pUserData)
    4930 {
    4931  if(IsUserDataString())
    4932  {
    4933  VMA_ASSERT(pUserData == VMA_NULL || pUserData != m_pUserData);
    4934 
    4935  FreeUserDataString(hAllocator);
    4936 
    4937  if(pUserData != VMA_NULL)
    4938  {
    4939  const char* const newStrSrc = (char*)pUserData;
    4940  const size_t newStrLen = strlen(newStrSrc);
    4941  char* const newStrDst = vma_new_array(hAllocator, char, newStrLen + 1);
    4942  memcpy(newStrDst, newStrSrc, newStrLen + 1);
    4943  m_pUserData = newStrDst;
    4944  }
    4945  }
    4946  else
    4947  {
    4948  m_pUserData = pUserData;
    4949  }
    4950 }
    4951 
    4952 void VmaAllocation_T::ChangeBlockAllocation(
    4953  VmaAllocator hAllocator,
    4954  VmaDeviceMemoryBlock* block,
    4955  VkDeviceSize offset)
    4956 {
    4957  VMA_ASSERT(block != VMA_NULL);
    4958  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
    4959 
    4960  // Move mapping reference counter from old block to new block.
    4961  if(block != m_BlockAllocation.m_Block)
    4962  {
    4963  uint32_t mapRefCount = m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP;
    4964  if(IsPersistentMap())
    4965  ++mapRefCount;
    4966  m_BlockAllocation.m_Block->Unmap(hAllocator, mapRefCount);
    4967  block->Map(hAllocator, mapRefCount, VMA_NULL);
    4968  }
    4969 
    4970  m_BlockAllocation.m_Block = block;
    4971  m_BlockAllocation.m_Offset = offset;
    4972 }
    4973 
    4974 VkDeviceSize VmaAllocation_T::GetOffset() const
    4975 {
    4976  switch(m_Type)
    4977  {
    4978  case ALLOCATION_TYPE_BLOCK:
    4979  return m_BlockAllocation.m_Offset;
    4980  case ALLOCATION_TYPE_DEDICATED:
    4981  return 0;
    4982  default:
    4983  VMA_ASSERT(0);
    4984  return 0;
    4985  }
    4986 }
    4987 
    4988 VkDeviceMemory VmaAllocation_T::GetMemory() const
    4989 {
    4990  switch(m_Type)
    4991  {
    4992  case ALLOCATION_TYPE_BLOCK:
    4993  return m_BlockAllocation.m_Block->GetDeviceMemory();
    4994  case ALLOCATION_TYPE_DEDICATED:
    4995  return m_DedicatedAllocation.m_hMemory;
    4996  default:
    4997  VMA_ASSERT(0);
    4998  return VK_NULL_HANDLE;
    4999  }
    5000 }
    5001 
    5002 uint32_t VmaAllocation_T::GetMemoryTypeIndex() const
    5003 {
    5004  switch(m_Type)
    5005  {
    5006  case ALLOCATION_TYPE_BLOCK:
    5007  return m_BlockAllocation.m_Block->GetMemoryTypeIndex();
    5008  case ALLOCATION_TYPE_DEDICATED:
    5009  return m_DedicatedAllocation.m_MemoryTypeIndex;
    5010  default:
    5011  VMA_ASSERT(0);
    5012  return UINT32_MAX;
    5013  }
    5014 }
    5015 
    5016 void* VmaAllocation_T::GetMappedData() const
    5017 {
    5018  switch(m_Type)
    5019  {
    5020  case ALLOCATION_TYPE_BLOCK:
    5021  if(m_MapCount != 0)
    5022  {
    5023  void* pBlockData = m_BlockAllocation.m_Block->GetMappedData();
    5024  VMA_ASSERT(pBlockData != VMA_NULL);
    5025  return (char*)pBlockData + m_BlockAllocation.m_Offset;
    5026  }
    5027  else
    5028  {
    5029  return VMA_NULL;
    5030  }
    5031  break;
    5032  case ALLOCATION_TYPE_DEDICATED:
    5033  VMA_ASSERT((m_DedicatedAllocation.m_pMappedData != VMA_NULL) == (m_MapCount != 0));
    5034  return m_DedicatedAllocation.m_pMappedData;
    5035  default:
    5036  VMA_ASSERT(0);
    5037  return VMA_NULL;
    5038  }
    5039 }
    5040 
    5041 bool VmaAllocation_T::CanBecomeLost() const
    5042 {
    5043  switch(m_Type)
    5044  {
    5045  case ALLOCATION_TYPE_BLOCK:
    5046  return m_BlockAllocation.m_CanBecomeLost;
    5047  case ALLOCATION_TYPE_DEDICATED:
    5048  return false;
    5049  default:
    5050  VMA_ASSERT(0);
    5051  return false;
    5052  }
    5053 }
    5054 
    5055 VmaPool VmaAllocation_T::GetPool() const
    5056 {
    5057  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
    5058  return m_BlockAllocation.m_hPool;
    5059 }
    5060 
    5061 bool VmaAllocation_T::MakeLost(uint32_t currentFrameIndex, uint32_t frameInUseCount)
    5062 {
    5063  VMA_ASSERT(CanBecomeLost());
    5064 
    5065  /*
    5066  Warning: This is a carefully designed algorithm.
    5067  Do not modify unless you really know what you're doing :)
    5068  */
    5069  uint32_t localLastUseFrameIndex = GetLastUseFrameIndex();
    5070  for(;;)
    5071  {
    5072  if(localLastUseFrameIndex == VMA_FRAME_INDEX_LOST)
    5073  {
    5074  VMA_ASSERT(0);
    5075  return false;
    5076  }
    5077  else if(localLastUseFrameIndex + frameInUseCount >= currentFrameIndex)
    5078  {
    5079  return false;
    5080  }
    5081  else // Last use time earlier than current time.
    5082  {
    5083  if(CompareExchangeLastUseFrameIndex(localLastUseFrameIndex, VMA_FRAME_INDEX_LOST))
    5084  {
    5085  // Setting hAllocation.LastUseFrameIndex atomic to VMA_FRAME_INDEX_LOST is enough to mark it as LOST.
    5086  // Calling code just needs to unregister this allocation in owning VmaDeviceMemoryBlock.
    5087  return true;
    5088  }
    5089  }
    5090  }
    5091 }
    5092 
    5093 void VmaAllocation_T::FreeUserDataString(VmaAllocator hAllocator)
    5094 {
    5095  VMA_ASSERT(IsUserDataString());
    5096  if(m_pUserData != VMA_NULL)
    5097  {
    5098  char* const oldStr = (char*)m_pUserData;
    5099  const size_t oldStrLen = strlen(oldStr);
    5100  vma_delete_array(hAllocator, oldStr, oldStrLen + 1);
    5101  m_pUserData = VMA_NULL;
    5102  }
    5103 }
    5104 
    5105 void VmaAllocation_T::BlockAllocMap()
    5106 {
    5107  VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
    5108 
    5109  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) < 0x7F)
    5110  {
    5111  ++m_MapCount;
    5112  }
    5113  else
    5114  {
    5115  VMA_ASSERT(0 && "Allocation mapped too many times simultaneously.");
    5116  }
    5117 }
    5118 
    5119 void VmaAllocation_T::BlockAllocUnmap()
    5120 {
    5121  VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
    5122 
    5123  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) != 0)
    5124  {
    5125  --m_MapCount;
    5126  }
    5127  else
    5128  {
    5129  VMA_ASSERT(0 && "Unmapping allocation not previously mapped.");
    5130  }
    5131 }
    5132 
    5133 VkResult VmaAllocation_T::DedicatedAllocMap(VmaAllocator hAllocator, void** ppData)
    5134 {
    5135  VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
    5136 
    5137  if(m_MapCount != 0)
    5138  {
    5139  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) < 0x7F)
    5140  {
    5141  VMA_ASSERT(m_DedicatedAllocation.m_pMappedData != VMA_NULL);
    5142  *ppData = m_DedicatedAllocation.m_pMappedData;
    5143  ++m_MapCount;
    5144  return VK_SUCCESS;
    5145  }
    5146  else
    5147  {
    5148  VMA_ASSERT(0 && "Dedicated allocation mapped too many times simultaneously.");
    5149  return VK_ERROR_MEMORY_MAP_FAILED;
    5150  }
    5151  }
    5152  else
    5153  {
    5154  VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
    5155  hAllocator->m_hDevice,
    5156  m_DedicatedAllocation.m_hMemory,
    5157  0, // offset
    5158  VK_WHOLE_SIZE,
    5159  0, // flags
    5160  ppData);
    5161  if(result == VK_SUCCESS)
    5162  {
    5163  m_DedicatedAllocation.m_pMappedData = *ppData;
    5164  m_MapCount = 1;
    5165  }
    5166  return result;
    5167  }
    5168 }
    5169 
    5170 void VmaAllocation_T::DedicatedAllocUnmap(VmaAllocator hAllocator)
    5171 {
    5172  VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
    5173 
    5174  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) != 0)
    5175  {
    5176  --m_MapCount;
    5177  if(m_MapCount == 0)
    5178  {
    5179  m_DedicatedAllocation.m_pMappedData = VMA_NULL;
    5180  (*hAllocator->GetVulkanFunctions().vkUnmapMemory)(
    5181  hAllocator->m_hDevice,
    5182  m_DedicatedAllocation.m_hMemory);
    5183  }
    5184  }
    5185  else
    5186  {
    5187  VMA_ASSERT(0 && "Unmapping dedicated allocation not previously mapped.");
    5188  }
    5189 }
    5190 
    5191 #if VMA_STATS_STRING_ENABLED
    5192 
    5193 // Correspond to values of enum VmaSuballocationType.
    5194 static const char* VMA_SUBALLOCATION_TYPE_NAMES[] = {
    5195  "FREE",
    5196  "UNKNOWN",
    5197  "BUFFER",
    5198  "IMAGE_UNKNOWN",
    5199  "IMAGE_LINEAR",
    5200  "IMAGE_OPTIMAL",
    5201 };
    5202 
    5203 static void VmaPrintStatInfo(VmaJsonWriter& json, const VmaStatInfo& stat)
    5204 {
    5205  json.BeginObject();
    5206 
    5207  json.WriteString("Blocks");
    5208  json.WriteNumber(stat.blockCount);
    5209 
    5210  json.WriteString("Allocations");
    5211  json.WriteNumber(stat.allocationCount);
    5212 
    5213  json.WriteString("UnusedRanges");
    5214  json.WriteNumber(stat.unusedRangeCount);
    5215 
    5216  json.WriteString("UsedBytes");
    5217  json.WriteNumber(stat.usedBytes);
    5218 
    5219  json.WriteString("UnusedBytes");
    5220  json.WriteNumber(stat.unusedBytes);
    5221 
    5222  if(stat.allocationCount > 1)
    5223  {
    5224  json.WriteString("AllocationSize");
    5225  json.BeginObject(true);
    5226  json.WriteString("Min");
    5227  json.WriteNumber(stat.allocationSizeMin);
    5228  json.WriteString("Avg");
    5229  json.WriteNumber(stat.allocationSizeAvg);
    5230  json.WriteString("Max");
    5231  json.WriteNumber(stat.allocationSizeMax);
    5232  json.EndObject();
    5233  }
    5234 
    5235  if(stat.unusedRangeCount > 1)
    5236  {
    5237  json.WriteString("UnusedRangeSize");
    5238  json.BeginObject(true);
    5239  json.WriteString("Min");
    5240  json.WriteNumber(stat.unusedRangeSizeMin);
    5241  json.WriteString("Avg");
    5242  json.WriteNumber(stat.unusedRangeSizeAvg);
    5243  json.WriteString("Max");
    5244  json.WriteNumber(stat.unusedRangeSizeMax);
    5245  json.EndObject();
    5246  }
    5247 
    5248  json.EndObject();
    5249 }
    5250 
    5251 #endif // #if VMA_STATS_STRING_ENABLED
    5252 
    5253 struct VmaSuballocationItemSizeLess
    5254 {
    5255  bool operator()(
    5256  const VmaSuballocationList::iterator lhs,
    5257  const VmaSuballocationList::iterator rhs) const
    5258  {
    5259  return lhs->size < rhs->size;
    5260  }
    5261  bool operator()(
    5262  const VmaSuballocationList::iterator lhs,
    5263  VkDeviceSize rhsSize) const
    5264  {
    5265  return lhs->size < rhsSize;
    5266  }
    5267 };
    5268 
    5270 // class VmaBlockMetadata
    5271 
    5272 VmaBlockMetadata::VmaBlockMetadata(VmaAllocator hAllocator) :
    5273  m_Size(0),
    5274  m_FreeCount(0),
    5275  m_SumFreeSize(0),
    5276  m_Suballocations(VmaStlAllocator<VmaSuballocation>(hAllocator->GetAllocationCallbacks())),
    5277  m_FreeSuballocationsBySize(VmaStlAllocator<VmaSuballocationList::iterator>(hAllocator->GetAllocationCallbacks()))
    5278 {
    5279 }
    5280 
    5281 VmaBlockMetadata::~VmaBlockMetadata()
    5282 {
    5283 }
    5284 
    5285 void VmaBlockMetadata::Init(VkDeviceSize size)
    5286 {
    5287  m_Size = size;
    5288  m_FreeCount = 1;
    5289  m_SumFreeSize = size;
    5290 
    5291  VmaSuballocation suballoc = {};
    5292  suballoc.offset = 0;
    5293  suballoc.size = size;
    5294  suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
    5295  suballoc.hAllocation = VK_NULL_HANDLE;
    5296 
    5297  m_Suballocations.push_back(suballoc);
    5298  VmaSuballocationList::iterator suballocItem = m_Suballocations.end();
    5299  --suballocItem;
    5300  m_FreeSuballocationsBySize.push_back(suballocItem);
    5301 }
    5302 
    5303 bool VmaBlockMetadata::Validate() const
    5304 {
    5305  if(m_Suballocations.empty())
    5306  {
    5307  return false;
    5308  }
    5309 
    5310  // Expected offset of new suballocation as calculates from previous ones.
    5311  VkDeviceSize calculatedOffset = 0;
    5312  // Expected number of free suballocations as calculated from traversing their list.
    5313  uint32_t calculatedFreeCount = 0;
    5314  // Expected sum size of free suballocations as calculated from traversing their list.
    5315  VkDeviceSize calculatedSumFreeSize = 0;
    5316  // Expected number of free suballocations that should be registered in
    5317  // m_FreeSuballocationsBySize calculated from traversing their list.
    5318  size_t freeSuballocationsToRegister = 0;
    5319  // True if previous visisted suballocation was free.
    5320  bool prevFree = false;
    5321 
    5322  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
    5323  suballocItem != m_Suballocations.cend();
    5324  ++suballocItem)
    5325  {
    5326  const VmaSuballocation& subAlloc = *suballocItem;
    5327 
    5328  // Actual offset of this suballocation doesn't match expected one.
    5329  if(subAlloc.offset != calculatedOffset)
    5330  {
    5331  return false;
    5332  }
    5333 
    5334  const bool currFree = (subAlloc.type == VMA_SUBALLOCATION_TYPE_FREE);
    5335  // Two adjacent free suballocations are invalid. They should be merged.
    5336  if(prevFree && currFree)
    5337  {
    5338  return false;
    5339  }
    5340 
    5341  if(currFree != (subAlloc.hAllocation == VK_NULL_HANDLE))
    5342  {
    5343  return false;
    5344  }
    5345 
    5346  if(currFree)
    5347  {
    5348  calculatedSumFreeSize += subAlloc.size;
    5349  ++calculatedFreeCount;
    5350  if(subAlloc.size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
    5351  {
    5352  ++freeSuballocationsToRegister;
    5353  }
    5354  }
    5355  else
    5356  {
    5357  if(subAlloc.hAllocation->GetOffset() != subAlloc.offset)
    5358  {
    5359  return false;
    5360  }
    5361  if(subAlloc.hAllocation->GetSize() != subAlloc.size)
    5362  {
    5363  return false;
    5364  }
    5365  }
    5366 
    5367  calculatedOffset += subAlloc.size;
    5368  prevFree = currFree;
    5369  }
    5370 
    5371  // Number of free suballocations registered in m_FreeSuballocationsBySize doesn't
    5372  // match expected one.
    5373  if(m_FreeSuballocationsBySize.size() != freeSuballocationsToRegister)
    5374  {
    5375  return false;
    5376  }
    5377 
    5378  VkDeviceSize lastSize = 0;
    5379  for(size_t i = 0; i < m_FreeSuballocationsBySize.size(); ++i)
    5380  {
    5381  VmaSuballocationList::iterator suballocItem = m_FreeSuballocationsBySize[i];
    5382 
    5383  // Only free suballocations can be registered in m_FreeSuballocationsBySize.
    5384  if(suballocItem->type != VMA_SUBALLOCATION_TYPE_FREE)
    5385  {
    5386  return false;
    5387  }
    5388  // They must be sorted by size ascending.
    5389  if(suballocItem->size < lastSize)
    5390  {
    5391  return false;
    5392  }
    5393 
    5394  lastSize = suballocItem->size;
    5395  }
    5396 
    5397  // Check if totals match calculacted values.
    5398  if(!ValidateFreeSuballocationList() ||
    5399  (calculatedOffset != m_Size) ||
    5400  (calculatedSumFreeSize != m_SumFreeSize) ||
    5401  (calculatedFreeCount != m_FreeCount))
    5402  {
    5403  return false;
    5404  }
    5405 
    5406  return true;
    5407 }
    5408 
    5409 VkDeviceSize VmaBlockMetadata::GetUnusedRangeSizeMax() const
    5410 {
    5411  if(!m_FreeSuballocationsBySize.empty())
    5412  {
    5413  return m_FreeSuballocationsBySize.back()->size;
    5414  }
    5415  else
    5416  {
    5417  return 0;
    5418  }
    5419 }
    5420 
    5421 bool VmaBlockMetadata::IsEmpty() const
    5422 {
    5423  return (m_Suballocations.size() == 1) && (m_FreeCount == 1);
    5424 }
    5425 
    5426 void VmaBlockMetadata::CalcAllocationStatInfo(VmaStatInfo& outInfo) const
    5427 {
    5428  outInfo.blockCount = 1;
    5429 
    5430  const uint32_t rangeCount = (uint32_t)m_Suballocations.size();
    5431  outInfo.allocationCount = rangeCount - m_FreeCount;
    5432  outInfo.unusedRangeCount = m_FreeCount;
    5433 
    5434  outInfo.unusedBytes = m_SumFreeSize;
    5435  outInfo.usedBytes = m_Size - outInfo.unusedBytes;
    5436 
    5437  outInfo.allocationSizeMin = UINT64_MAX;
    5438  outInfo.allocationSizeMax = 0;
    5439  outInfo.unusedRangeSizeMin = UINT64_MAX;
    5440  outInfo.unusedRangeSizeMax = 0;
    5441 
    5442  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
    5443  suballocItem != m_Suballocations.cend();
    5444  ++suballocItem)
    5445  {
    5446  const VmaSuballocation& suballoc = *suballocItem;
    5447  if(suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
    5448  {
    5449  outInfo.allocationSizeMin = VMA_MIN(outInfo.allocationSizeMin, suballoc.size);
    5450  outInfo.allocationSizeMax = VMA_MAX(outInfo.allocationSizeMax, suballoc.size);
    5451  }
    5452  else
    5453  {
    5454  outInfo.unusedRangeSizeMin = VMA_MIN(outInfo.unusedRangeSizeMin, suballoc.size);
    5455  outInfo.unusedRangeSizeMax = VMA_MAX(outInfo.unusedRangeSizeMax, suballoc.size);
    5456  }
    5457  }
    5458 }
    5459 
    5460 void VmaBlockMetadata::AddPoolStats(VmaPoolStats& inoutStats) const
    5461 {
    5462  const uint32_t rangeCount = (uint32_t)m_Suballocations.size();
    5463 
    5464  inoutStats.size += m_Size;
    5465  inoutStats.unusedSize += m_SumFreeSize;
    5466  inoutStats.allocationCount += rangeCount - m_FreeCount;
    5467  inoutStats.unusedRangeCount += m_FreeCount;
    5468  inoutStats.unusedRangeSizeMax = VMA_MAX(inoutStats.unusedRangeSizeMax, GetUnusedRangeSizeMax());
    5469 }
    5470 
    5471 #if VMA_STATS_STRING_ENABLED
    5472 
    5473 void VmaBlockMetadata::PrintDetailedMap(class VmaJsonWriter& json) const
    5474 {
    5475  json.BeginObject();
    5476 
    5477  json.WriteString("TotalBytes");
    5478  json.WriteNumber(m_Size);
    5479 
    5480  json.WriteString("UnusedBytes");
    5481  json.WriteNumber(m_SumFreeSize);
    5482 
    5483  json.WriteString("Allocations");
    5484  json.WriteNumber((uint64_t)m_Suballocations.size() - m_FreeCount);
    5485 
    5486  json.WriteString("UnusedRanges");
    5487  json.WriteNumber(m_FreeCount);
    5488 
    5489  json.WriteString("Suballocations");
    5490  json.BeginArray();
    5491  size_t i = 0;
    5492  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
    5493  suballocItem != m_Suballocations.cend();
    5494  ++suballocItem, ++i)
    5495  {
    5496  json.BeginObject(true);
    5497 
    5498  json.WriteString("Type");
    5499  json.WriteString(VMA_SUBALLOCATION_TYPE_NAMES[suballocItem->type]);
    5500 
    5501  json.WriteString("Size");
    5502  json.WriteNumber(suballocItem->size);
    5503 
    5504  json.WriteString("Offset");
    5505  json.WriteNumber(suballocItem->offset);
    5506 
    5507  if(suballocItem->type != VMA_SUBALLOCATION_TYPE_FREE)
    5508  {
    5509  const void* pUserData = suballocItem->hAllocation->GetUserData();
    5510  if(pUserData != VMA_NULL)
    5511  {
    5512  json.WriteString("UserData");
    5513  if(suballocItem->hAllocation->IsUserDataString())
    5514  {
    5515  json.WriteString((const char*)pUserData);
    5516  }
    5517  else
    5518  {
    5519  json.BeginString();
    5520  json.ContinueString_Pointer(pUserData);
    5521  json.EndString();
    5522  }
    5523  }
    5524  }
    5525 
    5526  json.EndObject();
    5527  }
    5528  json.EndArray();
    5529 
    5530  json.EndObject();
    5531 }
    5532 
    5533 #endif // #if VMA_STATS_STRING_ENABLED
    5534 
    5535 /*
    5536 How many suitable free suballocations to analyze before choosing best one.
    5537 - Set to 1 to use First-Fit algorithm - first suitable free suballocation will
    5538  be chosen.
    5539 - Set to UINT32_MAX to use Best-Fit/Worst-Fit algorithm - all suitable free
    5540  suballocations will be analized and best one will be chosen.
    5541 - Any other value is also acceptable.
    5542 */
    5543 //static const uint32_t MAX_SUITABLE_SUBALLOCATIONS_TO_CHECK = 8;
    5544 
    5545 void VmaBlockMetadata::CreateFirstAllocationRequest(VmaAllocationRequest* pAllocationRequest)
    5546 {
    5547  VMA_ASSERT(IsEmpty());
    5548  pAllocationRequest->offset = 0;
    5549  pAllocationRequest->sumFreeSize = m_SumFreeSize;
    5550  pAllocationRequest->sumItemSize = 0;
    5551  pAllocationRequest->item = m_Suballocations.begin();
    5552  pAllocationRequest->itemsToMakeLostCount = 0;
    5553 }
    5554 
    5555 bool VmaBlockMetadata::CreateAllocationRequest(
    5556  uint32_t currentFrameIndex,
    5557  uint32_t frameInUseCount,
    5558  VkDeviceSize bufferImageGranularity,
    5559  VkDeviceSize allocSize,
    5560  VkDeviceSize allocAlignment,
    5561  VmaSuballocationType allocType,
    5562  bool canMakeOtherLost,
    5563  VmaAllocationRequest* pAllocationRequest)
    5564 {
    5565  VMA_ASSERT(allocSize > 0);
    5566  VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
    5567  VMA_ASSERT(pAllocationRequest != VMA_NULL);
    5568  VMA_HEAVY_ASSERT(Validate());
    5569 
    5570  // There is not enough total free space in this block to fullfill the request: Early return.
    5571  if(canMakeOtherLost == false && m_SumFreeSize < allocSize)
    5572  {
    5573  return false;
    5574  }
    5575 
    5576  // New algorithm, efficiently searching freeSuballocationsBySize.
    5577  const size_t freeSuballocCount = m_FreeSuballocationsBySize.size();
    5578  if(freeSuballocCount > 0)
    5579  {
    5580  if(VMA_BEST_FIT)
    5581  {
    5582  // Find first free suballocation with size not less than allocSize.
    5583  VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
    5584  m_FreeSuballocationsBySize.data(),
    5585  m_FreeSuballocationsBySize.data() + freeSuballocCount,
    5586  allocSize,
    5587  VmaSuballocationItemSizeLess());
    5588  size_t index = it - m_FreeSuballocationsBySize.data();
    5589  for(; index < freeSuballocCount; ++index)
    5590  {
    5591  if(CheckAllocation(
    5592  currentFrameIndex,
    5593  frameInUseCount,
    5594  bufferImageGranularity,
    5595  allocSize,
    5596  allocAlignment,
    5597  allocType,
    5598  m_FreeSuballocationsBySize[index],
    5599  false, // canMakeOtherLost
    5600  &pAllocationRequest->offset,
    5601  &pAllocationRequest->itemsToMakeLostCount,
    5602  &pAllocationRequest->sumFreeSize,
    5603  &pAllocationRequest->sumItemSize))
    5604  {
    5605  pAllocationRequest->item = m_FreeSuballocationsBySize[index];
    5606  return true;
    5607  }
    5608  }
    5609  }
    5610  else
    5611  {
    5612  // Search staring from biggest suballocations.
    5613  for(size_t index = freeSuballocCount; index--; )
    5614  {
    5615  if(CheckAllocation(
    5616  currentFrameIndex,
    5617  frameInUseCount,
    5618  bufferImageGranularity,
    5619  allocSize,
    5620  allocAlignment,
    5621  allocType,
    5622  m_FreeSuballocationsBySize[index],
    5623  false, // canMakeOtherLost
    5624  &pAllocationRequest->offset,
    5625  &pAllocationRequest->itemsToMakeLostCount,
    5626  &pAllocationRequest->sumFreeSize,
    5627  &pAllocationRequest->sumItemSize))
    5628  {
    5629  pAllocationRequest->item = m_FreeSuballocationsBySize[index];
    5630  return true;
    5631  }
    5632  }
    5633  }
    5634  }
    5635 
    5636  if(canMakeOtherLost)
    5637  {
    5638  // Brute-force algorithm. TODO: Come up with something better.
    5639 
    5640  pAllocationRequest->sumFreeSize = VK_WHOLE_SIZE;
    5641  pAllocationRequest->sumItemSize = VK_WHOLE_SIZE;
    5642 
    5643  VmaAllocationRequest tmpAllocRequest = {};
    5644  for(VmaSuballocationList::iterator suballocIt = m_Suballocations.begin();
    5645  suballocIt != m_Suballocations.end();
    5646  ++suballocIt)
    5647  {
    5648  if(suballocIt->type == VMA_SUBALLOCATION_TYPE_FREE ||
    5649  suballocIt->hAllocation->CanBecomeLost())
    5650  {
    5651  if(CheckAllocation(
    5652  currentFrameIndex,
    5653  frameInUseCount,
    5654  bufferImageGranularity,
    5655  allocSize,
    5656  allocAlignment,
    5657  allocType,
    5658  suballocIt,
    5659  canMakeOtherLost,
    5660  &tmpAllocRequest.offset,
    5661  &tmpAllocRequest.itemsToMakeLostCount,
    5662  &tmpAllocRequest.sumFreeSize,
    5663  &tmpAllocRequest.sumItemSize))
    5664  {
    5665  tmpAllocRequest.item = suballocIt;
    5666 
    5667  if(tmpAllocRequest.CalcCost() < pAllocationRequest->CalcCost())
    5668  {
    5669  *pAllocationRequest = tmpAllocRequest;
    5670  }
    5671  }
    5672  }
    5673  }
    5674 
    5675  if(pAllocationRequest->sumItemSize != VK_WHOLE_SIZE)
    5676  {
    5677  return true;
    5678  }
    5679  }
    5680 
    5681  return false;
    5682 }
    5683 
    5684 bool VmaBlockMetadata::MakeRequestedAllocationsLost(
    5685  uint32_t currentFrameIndex,
    5686  uint32_t frameInUseCount,
    5687  VmaAllocationRequest* pAllocationRequest)
    5688 {
    5689  while(pAllocationRequest->itemsToMakeLostCount > 0)
    5690  {
    5691  if(pAllocationRequest->item->type == VMA_SUBALLOCATION_TYPE_FREE)
    5692  {
    5693  ++pAllocationRequest->item;
    5694  }
    5695  VMA_ASSERT(pAllocationRequest->item != m_Suballocations.end());
    5696  VMA_ASSERT(pAllocationRequest->item->hAllocation != VK_NULL_HANDLE);
    5697  VMA_ASSERT(pAllocationRequest->item->hAllocation->CanBecomeLost());
    5698  if(pAllocationRequest->item->hAllocation->MakeLost(currentFrameIndex, frameInUseCount))
    5699  {
    5700  pAllocationRequest->item = FreeSuballocation(pAllocationRequest->item);
    5701  --pAllocationRequest->itemsToMakeLostCount;
    5702  }
    5703  else
    5704  {
    5705  return false;
    5706  }
    5707  }
    5708 
    5709  VMA_HEAVY_ASSERT(Validate());
    5710  VMA_ASSERT(pAllocationRequest->item != m_Suballocations.end());
    5711  VMA_ASSERT(pAllocationRequest->item->type == VMA_SUBALLOCATION_TYPE_FREE);
    5712 
    5713  return true;
    5714 }
    5715 
    5716 uint32_t VmaBlockMetadata::MakeAllocationsLost(uint32_t currentFrameIndex, uint32_t frameInUseCount)
    5717 {
    5718  uint32_t lostAllocationCount = 0;
    5719  for(VmaSuballocationList::iterator it = m_Suballocations.begin();
    5720  it != m_Suballocations.end();
    5721  ++it)
    5722  {
    5723  if(it->type != VMA_SUBALLOCATION_TYPE_FREE &&
    5724  it->hAllocation->CanBecomeLost() &&
    5725  it->hAllocation->MakeLost(currentFrameIndex, frameInUseCount))
    5726  {
    5727  it = FreeSuballocation(it);
    5728  ++lostAllocationCount;
    5729  }
    5730  }
    5731  return lostAllocationCount;
    5732 }
    5733 
    5734 void VmaBlockMetadata::Alloc(
    5735  const VmaAllocationRequest& request,
    5736  VmaSuballocationType type,
    5737  VkDeviceSize allocSize,
    5738  VmaAllocation hAllocation)
    5739 {
    5740  VMA_ASSERT(request.item != m_Suballocations.end());
    5741  VmaSuballocation& suballoc = *request.item;
    5742  // Given suballocation is a free block.
    5743  VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
    5744  // Given offset is inside this suballocation.
    5745  VMA_ASSERT(request.offset >= suballoc.offset);
    5746  const VkDeviceSize paddingBegin = request.offset - suballoc.offset;
    5747  VMA_ASSERT(suballoc.size >= paddingBegin + allocSize);
    5748  const VkDeviceSize paddingEnd = suballoc.size - paddingBegin - allocSize;
    5749 
    5750  // Unregister this free suballocation from m_FreeSuballocationsBySize and update
    5751  // it to become used.
    5752  UnregisterFreeSuballocation(request.item);
    5753 
    5754  suballoc.offset = request.offset;
    5755  suballoc.size = allocSize;
    5756  suballoc.type = type;
    5757  suballoc.hAllocation = hAllocation;
    5758 
    5759  // If there are any free bytes remaining at the end, insert new free suballocation after current one.
    5760  if(paddingEnd)
    5761  {
    5762  VmaSuballocation paddingSuballoc = {};
    5763  paddingSuballoc.offset = request.offset + allocSize;
    5764  paddingSuballoc.size = paddingEnd;
    5765  paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
    5766  VmaSuballocationList::iterator next = request.item;
    5767  ++next;
    5768  const VmaSuballocationList::iterator paddingEndItem =
    5769  m_Suballocations.insert(next, paddingSuballoc);
    5770  RegisterFreeSuballocation(paddingEndItem);
    5771  }
    5772 
    5773  // If there are any free bytes remaining at the beginning, insert new free suballocation before current one.
    5774  if(paddingBegin)
    5775  {
    5776  VmaSuballocation paddingSuballoc = {};
    5777  paddingSuballoc.offset = request.offset - paddingBegin;
    5778  paddingSuballoc.size = paddingBegin;
    5779  paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
    5780  const VmaSuballocationList::iterator paddingBeginItem =
    5781  m_Suballocations.insert(request.item, paddingSuballoc);
    5782  RegisterFreeSuballocation(paddingBeginItem);
    5783  }
    5784 
    5785  // Update totals.
    5786  m_FreeCount = m_FreeCount - 1;
    5787  if(paddingBegin > 0)
    5788  {
    5789  ++m_FreeCount;
    5790  }
    5791  if(paddingEnd > 0)
    5792  {
    5793  ++m_FreeCount;
    5794  }
    5795  m_SumFreeSize -= allocSize;
    5796 }
    5797 
    5798 void VmaBlockMetadata::Free(const VmaAllocation allocation)
    5799 {
    5800  for(VmaSuballocationList::iterator suballocItem = m_Suballocations.begin();
    5801  suballocItem != m_Suballocations.end();
    5802  ++suballocItem)
    5803  {
    5804  VmaSuballocation& suballoc = *suballocItem;
    5805  if(suballoc.hAllocation == allocation)
    5806  {
    5807  FreeSuballocation(suballocItem);
    5808  VMA_HEAVY_ASSERT(Validate());
    5809  return;
    5810  }
    5811  }
    5812  VMA_ASSERT(0 && "Not found!");
    5813 }
    5814 
    5815 void VmaBlockMetadata::FreeAtOffset(VkDeviceSize offset)
    5816 {
    5817  for(VmaSuballocationList::iterator suballocItem = m_Suballocations.begin();
    5818  suballocItem != m_Suballocations.end();
    5819  ++suballocItem)
    5820  {
    5821  VmaSuballocation& suballoc = *suballocItem;
    5822  if(suballoc.offset == offset)
    5823  {
    5824  FreeSuballocation(suballocItem);
    5825  return;
    5826  }
    5827  }
    5828  VMA_ASSERT(0 && "Not found!");
    5829 }
    5830 
    5831 bool VmaBlockMetadata::ValidateFreeSuballocationList() const
    5832 {
    5833  VkDeviceSize lastSize = 0;
    5834  for(size_t i = 0, count = m_FreeSuballocationsBySize.size(); i < count; ++i)
    5835  {
    5836  const VmaSuballocationList::iterator it = m_FreeSuballocationsBySize[i];
    5837 
    5838  if(it->type != VMA_SUBALLOCATION_TYPE_FREE)
    5839  {
    5840  VMA_ASSERT(0);
    5841  return false;
    5842  }
    5843  if(it->size < VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
    5844  {
    5845  VMA_ASSERT(0);
    5846  return false;
    5847  }
    5848  if(it->size < lastSize)
    5849  {
    5850  VMA_ASSERT(0);
    5851  return false;
    5852  }
    5853 
    5854  lastSize = it->size;
    5855  }
    5856  return true;
    5857 }
    5858 
    5859 bool VmaBlockMetadata::CheckAllocation(
    5860  uint32_t currentFrameIndex,
    5861  uint32_t frameInUseCount,
    5862  VkDeviceSize bufferImageGranularity,
    5863  VkDeviceSize allocSize,
    5864  VkDeviceSize allocAlignment,
    5865  VmaSuballocationType allocType,
    5866  VmaSuballocationList::const_iterator suballocItem,
    5867  bool canMakeOtherLost,
    5868  VkDeviceSize* pOffset,
    5869  size_t* itemsToMakeLostCount,
    5870  VkDeviceSize* pSumFreeSize,
    5871  VkDeviceSize* pSumItemSize) const
    5872 {
    5873  VMA_ASSERT(allocSize > 0);
    5874  VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
    5875  VMA_ASSERT(suballocItem != m_Suballocations.cend());
    5876  VMA_ASSERT(pOffset != VMA_NULL);
    5877 
    5878  *itemsToMakeLostCount = 0;
    5879  *pSumFreeSize = 0;
    5880  *pSumItemSize = 0;
    5881 
    5882  if(canMakeOtherLost)
    5883  {
    5884  if(suballocItem->type == VMA_SUBALLOCATION_TYPE_FREE)
    5885  {
    5886  *pSumFreeSize = suballocItem->size;
    5887  }
    5888  else
    5889  {
    5890  if(suballocItem->hAllocation->CanBecomeLost() &&
    5891  suballocItem->hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
    5892  {
    5893  ++*itemsToMakeLostCount;
    5894  *pSumItemSize = suballocItem->size;
    5895  }
    5896  else
    5897  {
    5898  return false;
    5899  }
    5900  }
    5901 
    5902  // Remaining size is too small for this request: Early return.
    5903  if(m_Size - suballocItem->offset < allocSize)
    5904  {
    5905  return false;
    5906  }
    5907 
    5908  // Start from offset equal to beginning of this suballocation.
    5909  *pOffset = suballocItem->offset;
    5910 
    5911  // Apply VMA_DEBUG_MARGIN at the beginning.
    5912  if((VMA_DEBUG_MARGIN > 0) && suballocItem != m_Suballocations.cbegin())
    5913  {
    5914  *pOffset += VMA_DEBUG_MARGIN;
    5915  }
    5916 
    5917  // Apply alignment.
    5918  const VkDeviceSize alignment = VMA_MAX(allocAlignment, static_cast<VkDeviceSize>(VMA_DEBUG_ALIGNMENT));
    5919  *pOffset = VmaAlignUp(*pOffset, alignment);
    5920 
    5921  // Check previous suballocations for BufferImageGranularity conflicts.
    5922  // Make bigger alignment if necessary.
    5923  if(bufferImageGranularity > 1)
    5924  {
    5925  bool bufferImageGranularityConflict = false;
    5926  VmaSuballocationList::const_iterator prevSuballocItem = suballocItem;
    5927  while(prevSuballocItem != m_Suballocations.cbegin())
    5928  {
    5929  --prevSuballocItem;
    5930  const VmaSuballocation& prevSuballoc = *prevSuballocItem;
    5931  if(VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, *pOffset, bufferImageGranularity))
    5932  {
    5933  if(VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
    5934  {
    5935  bufferImageGranularityConflict = true;
    5936  break;
    5937  }
    5938  }
    5939  else
    5940  // Already on previous page.
    5941  break;
    5942  }
    5943  if(bufferImageGranularityConflict)
    5944  {
    5945  *pOffset = VmaAlignUp(*pOffset, bufferImageGranularity);
    5946  }
    5947  }
    5948 
    5949  // Now that we have final *pOffset, check if we are past suballocItem.
    5950  // If yes, return false - this function should be called for another suballocItem as starting point.
    5951  if(*pOffset >= suballocItem->offset + suballocItem->size)
    5952  {
    5953  return false;
    5954  }
    5955 
    5956  // Calculate padding at the beginning based on current offset.
    5957  const VkDeviceSize paddingBegin = *pOffset - suballocItem->offset;
    5958 
    5959  // Calculate required margin at the end if this is not last suballocation.
    5960  VmaSuballocationList::const_iterator next = suballocItem;
    5961  ++next;
    5962  const VkDeviceSize requiredEndMargin =
    5963  (next != m_Suballocations.cend()) ? VMA_DEBUG_MARGIN : 0;
    5964 
    5965  const VkDeviceSize totalSize = paddingBegin + allocSize + requiredEndMargin;
    5966  // Another early return check.
    5967  if(suballocItem->offset + totalSize > m_Size)
    5968  {
    5969  return false;
    5970  }
    5971 
    5972  // Advance lastSuballocItem until desired size is reached.
    5973  // Update itemsToMakeLostCount.
    5974  VmaSuballocationList::const_iterator lastSuballocItem = suballocItem;
    5975  if(totalSize > suballocItem->size)
    5976  {
    5977  VkDeviceSize remainingSize = totalSize - suballocItem->size;
    5978  while(remainingSize > 0)
    5979  {
    5980  ++lastSuballocItem;
    5981  if(lastSuballocItem == m_Suballocations.cend())
    5982  {
    5983  return false;
    5984  }
    5985  if(lastSuballocItem->type == VMA_SUBALLOCATION_TYPE_FREE)
    5986  {
    5987  *pSumFreeSize += lastSuballocItem->size;
    5988  }
    5989  else
    5990  {
    5991  VMA_ASSERT(lastSuballocItem->hAllocation != VK_NULL_HANDLE);
    5992  if(lastSuballocItem->hAllocation->CanBecomeLost() &&
    5993  lastSuballocItem->hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
    5994  {
    5995  ++*itemsToMakeLostCount;
    5996  *pSumItemSize += lastSuballocItem->size;
    5997  }
    5998  else
    5999  {
    6000  return false;
    6001  }
    6002  }
    6003  remainingSize = (lastSuballocItem->size < remainingSize) ?
    6004  remainingSize - lastSuballocItem->size : 0;
    6005  }
    6006  }
    6007 
    6008  // Check next suballocations for BufferImageGranularity conflicts.
    6009  // If conflict exists, we must mark more allocations lost or fail.
    6010  if(bufferImageGranularity > 1)
    6011  {
    6012  VmaSuballocationList::const_iterator nextSuballocItem = lastSuballocItem;
    6013  ++nextSuballocItem;
    6014  while(nextSuballocItem != m_Suballocations.cend())
    6015  {
    6016  const VmaSuballocation& nextSuballoc = *nextSuballocItem;
    6017  if(VmaBlocksOnSamePage(*pOffset, allocSize, nextSuballoc.offset, bufferImageGranularity))
    6018  {
    6019  if(VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
    6020  {
    6021  VMA_ASSERT(nextSuballoc.hAllocation != VK_NULL_HANDLE);
    6022  if(nextSuballoc.hAllocation->CanBecomeLost() &&
    6023  nextSuballoc.hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
    6024  {
    6025  ++*itemsToMakeLostCount;
    6026  }
    6027  else
    6028  {
    6029  return false;
    6030  }
    6031  }
    6032  }
    6033  else
    6034  {
    6035  // Already on next page.
    6036  break;
    6037  }
    6038  ++nextSuballocItem;
    6039  }
    6040  }
    6041  }
    6042  else
    6043  {
    6044  const VmaSuballocation& suballoc = *suballocItem;
    6045  VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
    6046 
    6047  *pSumFreeSize = suballoc.size;
    6048 
    6049  // Size of this suballocation is too small for this request: Early return.
    6050  if(suballoc.size < allocSize)
    6051  {
    6052  return false;
    6053  }
    6054 
    6055  // Start from offset equal to beginning of this suballocation.
    6056  *pOffset = suballoc.offset;
    6057 
    6058  // Apply VMA_DEBUG_MARGIN at the beginning.
    6059  if((VMA_DEBUG_MARGIN > 0) && suballocItem != m_Suballocations.cbegin())
    6060  {
    6061  *pOffset += VMA_DEBUG_MARGIN;
    6062  }
    6063 
    6064  // Apply alignment.
    6065  const VkDeviceSize alignment = VMA_MAX(allocAlignment, static_cast<VkDeviceSize>(VMA_DEBUG_ALIGNMENT));
    6066  *pOffset = VmaAlignUp(*pOffset, alignment);
    6067 
    6068  // Check previous suballocations for BufferImageGranularity conflicts.
    6069  // Make bigger alignment if necessary.
    6070  if(bufferImageGranularity > 1)
    6071  {
    6072  bool bufferImageGranularityConflict = false;
    6073  VmaSuballocationList::const_iterator prevSuballocItem = suballocItem;
    6074  while(prevSuballocItem != m_Suballocations.cbegin())
    6075  {
    6076  --prevSuballocItem;
    6077  const VmaSuballocation& prevSuballoc = *prevSuballocItem;
    6078  if(VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, *pOffset, bufferImageGranularity))
    6079  {
    6080  if(VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
    6081  {
    6082  bufferImageGranularityConflict = true;
    6083  break;
    6084  }
    6085  }
    6086  else
    6087  // Already on previous page.
    6088  break;
    6089  }
    6090  if(bufferImageGranularityConflict)
    6091  {
    6092  *pOffset = VmaAlignUp(*pOffset, bufferImageGranularity);
    6093  }
    6094  }
    6095 
    6096  // Calculate padding at the beginning based on current offset.
    6097  const VkDeviceSize paddingBegin = *pOffset - suballoc.offset;
    6098 
    6099  // Calculate required margin at the end if this is not last suballocation.
    6100  VmaSuballocationList::const_iterator next = suballocItem;
    6101  ++next;
    6102  const VkDeviceSize requiredEndMargin =
    6103  (next != m_Suballocations.cend()) ? VMA_DEBUG_MARGIN : 0;
    6104 
    6105  // Fail if requested size plus margin before and after is bigger than size of this suballocation.
    6106  if(paddingBegin + allocSize + requiredEndMargin > suballoc.size)
    6107  {
    6108  return false;
    6109  }
    6110 
    6111  // Check next suballocations for BufferImageGranularity conflicts.
    6112  // If conflict exists, allocation cannot be made here.
    6113  if(bufferImageGranularity > 1)
    6114  {
    6115  VmaSuballocationList::const_iterator nextSuballocItem = suballocItem;
    6116  ++nextSuballocItem;
    6117  while(nextSuballocItem != m_Suballocations.cend())
    6118  {
    6119  const VmaSuballocation& nextSuballoc = *nextSuballocItem;
    6120  if(VmaBlocksOnSamePage(*pOffset, allocSize, nextSuballoc.offset, bufferImageGranularity))
    6121  {
    6122  if(VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
    6123  {
    6124  return false;
    6125  }
    6126  }
    6127  else
    6128  {
    6129  // Already on next page.
    6130  break;
    6131  }
    6132  ++nextSuballocItem;
    6133  }
    6134  }
    6135  }
    6136 
    6137  // All tests passed: Success. pOffset is already filled.
    6138  return true;
    6139 }
    6140 
    6141 void VmaBlockMetadata::MergeFreeWithNext(VmaSuballocationList::iterator item)
    6142 {
    6143  VMA_ASSERT(item != m_Suballocations.end());
    6144  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
    6145 
    6146  VmaSuballocationList::iterator nextItem = item;
    6147  ++nextItem;
    6148  VMA_ASSERT(nextItem != m_Suballocations.end());
    6149  VMA_ASSERT(nextItem->type == VMA_SUBALLOCATION_TYPE_FREE);
    6150 
    6151  item->size += nextItem->size;
    6152  --m_FreeCount;
    6153  m_Suballocations.erase(nextItem);
    6154 }
    6155 
    6156 VmaSuballocationList::iterator VmaBlockMetadata::FreeSuballocation(VmaSuballocationList::iterator suballocItem)
    6157 {
    6158  // Change this suballocation to be marked as free.
    6159  VmaSuballocation& suballoc = *suballocItem;
    6160  suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
    6161  suballoc.hAllocation = VK_NULL_HANDLE;
    6162 
    6163  // Update totals.
    6164  ++m_FreeCount;
    6165  m_SumFreeSize += suballoc.size;
    6166 
    6167  // Merge with previous and/or next suballocation if it's also free.
    6168  bool mergeWithNext = false;
    6169  bool mergeWithPrev = false;
    6170 
    6171  VmaSuballocationList::iterator nextItem = suballocItem;
    6172  ++nextItem;
    6173  if((nextItem != m_Suballocations.end()) && (nextItem->type == VMA_SUBALLOCATION_TYPE_FREE))
    6174  {
    6175  mergeWithNext = true;
    6176  }
    6177 
    6178  VmaSuballocationList::iterator prevItem = suballocItem;
    6179  if(suballocItem != m_Suballocations.begin())
    6180  {
    6181  --prevItem;
    6182  if(prevItem->type == VMA_SUBALLOCATION_TYPE_FREE)
    6183  {
    6184  mergeWithPrev = true;
    6185  }
    6186  }
    6187 
    6188  if(mergeWithNext)
    6189  {
    6190  UnregisterFreeSuballocation(nextItem);
    6191  MergeFreeWithNext(suballocItem);
    6192  }
    6193 
    6194  if(mergeWithPrev)
    6195  {
    6196  UnregisterFreeSuballocation(prevItem);
    6197  MergeFreeWithNext(prevItem);
    6198  RegisterFreeSuballocation(prevItem);
    6199  return prevItem;
    6200  }
    6201  else
    6202  {
    6203  RegisterFreeSuballocation(suballocItem);
    6204  return suballocItem;
    6205  }
    6206 }
    6207 
    6208 void VmaBlockMetadata::RegisterFreeSuballocation(VmaSuballocationList::iterator item)
    6209 {
    6210  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
    6211  VMA_ASSERT(item->size > 0);
    6212 
    6213  // You may want to enable this validation at the beginning or at the end of
    6214  // this function, depending on what do you want to check.
    6215  VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
    6216 
    6217  if(item->size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
    6218  {
    6219  if(m_FreeSuballocationsBySize.empty())
    6220  {
    6221  m_FreeSuballocationsBySize.push_back(item);
    6222  }
    6223  else
    6224  {
    6225  VmaVectorInsertSorted<VmaSuballocationItemSizeLess>(m_FreeSuballocationsBySize, item);
    6226  }
    6227  }
    6228 
    6229  //VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
    6230 }
    6231 
    6232 
    6233 void VmaBlockMetadata::UnregisterFreeSuballocation(VmaSuballocationList::iterator item)
    6234 {
    6235  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
    6236  VMA_ASSERT(item->size > 0);
    6237 
    6238  // You may want to enable this validation at the beginning or at the end of
    6239  // this function, depending on what do you want to check.
    6240  VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
    6241 
    6242  if(item->size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
    6243  {
    6244  VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
    6245  m_FreeSuballocationsBySize.data(),
    6246  m_FreeSuballocationsBySize.data() + m_FreeSuballocationsBySize.size(),
    6247  item,
    6248  VmaSuballocationItemSizeLess());
    6249  for(size_t index = it - m_FreeSuballocationsBySize.data();
    6250  index < m_FreeSuballocationsBySize.size();
    6251  ++index)
    6252  {
    6253  if(m_FreeSuballocationsBySize[index] == item)
    6254  {
    6255  VmaVectorRemove(m_FreeSuballocationsBySize, index);
    6256  return;
    6257  }
    6258  VMA_ASSERT((m_FreeSuballocationsBySize[index]->size == item->size) && "Not found.");
    6259  }
    6260  VMA_ASSERT(0 && "Not found.");
    6261  }
    6262 
    6263  //VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
    6264 }
    6265 
    6267 // class VmaDeviceMemoryBlock
    6268 
    6269 VmaDeviceMemoryBlock::VmaDeviceMemoryBlock(VmaAllocator hAllocator) :
    6270  m_Metadata(hAllocator),
    6271  m_MemoryTypeIndex(UINT32_MAX),
    6272  m_hMemory(VK_NULL_HANDLE),
    6273  m_MapCount(0),
    6274  m_pMappedData(VMA_NULL)
    6275 {
    6276 }
    6277 
    6278 void VmaDeviceMemoryBlock::Init(
    6279  uint32_t newMemoryTypeIndex,
    6280  VkDeviceMemory newMemory,
    6281  VkDeviceSize newSize)
    6282 {
    6283  VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
    6284 
    6285  m_MemoryTypeIndex = newMemoryTypeIndex;
    6286  m_hMemory = newMemory;
    6287 
    6288  m_Metadata.Init(newSize);
    6289 }
    6290 
    6291 void VmaDeviceMemoryBlock::Destroy(VmaAllocator allocator)
    6292 {
    6293  // This is the most important assert in the entire library.
    6294  // Hitting it means you have some memory leak - unreleased VmaAllocation objects.
    6295  VMA_ASSERT(m_Metadata.IsEmpty() && "Some allocations were not freed before destruction of this memory block!");
    6296 
    6297  VMA_ASSERT(m_hMemory != VK_NULL_HANDLE);
    6298  allocator->FreeVulkanMemory(m_MemoryTypeIndex, m_Metadata.GetSize(), m_hMemory);
    6299  m_hMemory = VK_NULL_HANDLE;
    6300 }
    6301 
    6302 bool VmaDeviceMemoryBlock::Validate() const
    6303 {
    6304  if((m_hMemory == VK_NULL_HANDLE) ||
    6305  (m_Metadata.GetSize() == 0))
    6306  {
    6307  return false;
    6308  }
    6309 
    6310  return m_Metadata.Validate();
    6311 }
    6312 
    6313 VkResult VmaDeviceMemoryBlock::Map(VmaAllocator hAllocator, uint32_t count, void** ppData)
    6314 {
    6315  if(count == 0)
    6316  {
    6317  return VK_SUCCESS;
    6318  }
    6319 
    6320  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
    6321  if(m_MapCount != 0)
    6322  {
    6323  m_MapCount += count;
    6324  VMA_ASSERT(m_pMappedData != VMA_NULL);
    6325  if(ppData != VMA_NULL)
    6326  {
    6327  *ppData = m_pMappedData;
    6328  }
    6329  return VK_SUCCESS;
    6330  }
    6331  else
    6332  {
    6333  VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
    6334  hAllocator->m_hDevice,
    6335  m_hMemory,
    6336  0, // offset
    6337  VK_WHOLE_SIZE,
    6338  0, // flags
    6339  &m_pMappedData);
    6340  if(result == VK_SUCCESS)
    6341  {
    6342  if(ppData != VMA_NULL)
    6343  {
    6344  *ppData = m_pMappedData;
    6345  }
    6346  m_MapCount = count;
    6347  }
    6348  return result;
    6349  }
    6350 }
    6351 
    6352 void VmaDeviceMemoryBlock::Unmap(VmaAllocator hAllocator, uint32_t count)
    6353 {
    6354  if(count == 0)
    6355  {
    6356  return;
    6357  }
    6358 
    6359  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
    6360  if(m_MapCount >= count)
    6361  {
    6362  m_MapCount -= count;
    6363  if(m_MapCount == 0)
    6364  {
    6365  m_pMappedData = VMA_NULL;
    6366  (*hAllocator->GetVulkanFunctions().vkUnmapMemory)(hAllocator->m_hDevice, m_hMemory);
    6367  }
    6368  }
    6369  else
    6370  {
    6371  VMA_ASSERT(0 && "VkDeviceMemory block is being unmapped while it was not previously mapped.");
    6372  }
    6373 }
    6374 
    6375 VkResult VmaDeviceMemoryBlock::BindBufferMemory(
    6376  const VmaAllocator hAllocator,
    6377  const VmaAllocation hAllocation,
    6378  VkBuffer hBuffer)
    6379 {
    6380  VMA_ASSERT(hAllocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK &&
    6381  hAllocation->GetBlock() == this);
    6382  // This lock is important so that we don't call vkBind... and/or vkMap... simultaneously on the same VkDeviceMemory from multiple threads.
    6383  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
    6384  return hAllocator->GetVulkanFunctions().vkBindBufferMemory(
    6385  hAllocator->m_hDevice,
    6386  hBuffer,
    6387  m_hMemory,
    6388  hAllocation->GetOffset());
    6389 }
    6390 
    6391 VkResult VmaDeviceMemoryBlock::BindImageMemory(
    6392  const VmaAllocator hAllocator,
    6393  const VmaAllocation hAllocation,
    6394  VkImage hImage)
    6395 {
    6396  VMA_ASSERT(hAllocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK &&
    6397  hAllocation->GetBlock() == this);
    6398  // This lock is important so that we don't call vkBind... and/or vkMap... simultaneously on the same VkDeviceMemory from multiple threads.
    6399  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
    6400  return hAllocator->GetVulkanFunctions().vkBindImageMemory(
    6401  hAllocator->m_hDevice,
    6402  hImage,
    6403  m_hMemory,
    6404  hAllocation->GetOffset());
    6405 }
    6406 
    6407 static void InitStatInfo(VmaStatInfo& outInfo)
    6408 {
    6409  memset(&outInfo, 0, sizeof(outInfo));
    6410  outInfo.allocationSizeMin = UINT64_MAX;
    6411  outInfo.unusedRangeSizeMin = UINT64_MAX;
    6412 }
    6413 
    6414 // Adds statistics srcInfo into inoutInfo, like: inoutInfo += srcInfo.
    6415 static void VmaAddStatInfo(VmaStatInfo& inoutInfo, const VmaStatInfo& srcInfo)
    6416 {
    6417  inoutInfo.blockCount += srcInfo.blockCount;
    6418  inoutInfo.allocationCount += srcInfo.allocationCount;
    6419  inoutInfo.unusedRangeCount += srcInfo.unusedRangeCount;
    6420  inoutInfo.usedBytes += srcInfo.usedBytes;
    6421  inoutInfo.unusedBytes += srcInfo.unusedBytes;
    6422  inoutInfo.allocationSizeMin = VMA_MIN(inoutInfo.allocationSizeMin, srcInfo.allocationSizeMin);
    6423  inoutInfo.allocationSizeMax = VMA_MAX(inoutInfo.allocationSizeMax, srcInfo.allocationSizeMax);
    6424  inoutInfo.unusedRangeSizeMin = VMA_MIN(inoutInfo.unusedRangeSizeMin, srcInfo.unusedRangeSizeMin);
    6425  inoutInfo.unusedRangeSizeMax = VMA_MAX(inoutInfo.unusedRangeSizeMax, srcInfo.unusedRangeSizeMax);
    6426 }
    6427 
    6428 static void VmaPostprocessCalcStatInfo(VmaStatInfo& inoutInfo)
    6429 {
    6430  inoutInfo.allocationSizeAvg = (inoutInfo.allocationCount > 0) ?
    6431  VmaRoundDiv<VkDeviceSize>(inoutInfo.usedBytes, inoutInfo.allocationCount) : 0;
    6432  inoutInfo.unusedRangeSizeAvg = (inoutInfo.unusedRangeCount > 0) ?
    6433  VmaRoundDiv<VkDeviceSize>(inoutInfo.unusedBytes, inoutInfo.unusedRangeCount) : 0;
    6434 }
    6435 
    6436 VmaPool_T::VmaPool_T(
    6437  VmaAllocator hAllocator,
    6438  const VmaPoolCreateInfo& createInfo) :
    6439  m_BlockVector(
    6440  hAllocator,
    6441  createInfo.memoryTypeIndex,
    6442  createInfo.blockSize,
    6443  createInfo.minBlockCount,
    6444  createInfo.maxBlockCount,
    6445  (createInfo.flags & VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT) != 0 ? 1 : hAllocator->GetBufferImageGranularity(),
    6446  createInfo.frameInUseCount,
    6447  true) // isCustomPool
    6448 {
    6449 }
    6450 
    6451 VmaPool_T::~VmaPool_T()
    6452 {
    6453 }
    6454 
    6455 #if VMA_STATS_STRING_ENABLED
    6456 
    6457 #endif // #if VMA_STATS_STRING_ENABLED
    6458 
    6459 VmaBlockVector::VmaBlockVector(
    6460  VmaAllocator hAllocator,
    6461  uint32_t memoryTypeIndex,
    6462  VkDeviceSize preferredBlockSize,
    6463  size_t minBlockCount,
    6464  size_t maxBlockCount,
    6465  VkDeviceSize bufferImageGranularity,
    6466  uint32_t frameInUseCount,
    6467  bool isCustomPool) :
    6468  m_hAllocator(hAllocator),
    6469  m_MemoryTypeIndex(memoryTypeIndex),
    6470  m_PreferredBlockSize(preferredBlockSize),
    6471  m_MinBlockCount(minBlockCount),
    6472  m_MaxBlockCount(maxBlockCount),
    6473  m_BufferImageGranularity(bufferImageGranularity),
    6474  m_FrameInUseCount(frameInUseCount),
    6475  m_IsCustomPool(isCustomPool),
    6476  m_Blocks(VmaStlAllocator<VmaDeviceMemoryBlock*>(hAllocator->GetAllocationCallbacks())),
    6477  m_HasEmptyBlock(false),
    6478  m_pDefragmentator(VMA_NULL)
    6479 {
    6480 }
    6481 
    6482 VmaBlockVector::~VmaBlockVector()
    6483 {
    6484  VMA_ASSERT(m_pDefragmentator == VMA_NULL);
    6485 
    6486  for(size_t i = m_Blocks.size(); i--; )
    6487  {
    6488  m_Blocks[i]->Destroy(m_hAllocator);
    6489  vma_delete(m_hAllocator, m_Blocks[i]);
    6490  }
    6491 }
    6492 
    6493 VkResult VmaBlockVector::CreateMinBlocks()
    6494 {
    6495  for(size_t i = 0; i < m_MinBlockCount; ++i)
    6496  {
    6497  VkResult res = CreateBlock(m_PreferredBlockSize, VMA_NULL);
    6498  if(res != VK_SUCCESS)
    6499  {
    6500  return res;
    6501  }
    6502  }
    6503  return VK_SUCCESS;
    6504 }
    6505 
    6506 void VmaBlockVector::GetPoolStats(VmaPoolStats* pStats)
    6507 {
    6508  pStats->size = 0;
    6509  pStats->unusedSize = 0;
    6510  pStats->allocationCount = 0;
    6511  pStats->unusedRangeCount = 0;
    6512  pStats->unusedRangeSizeMax = 0;
    6513 
    6514  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    6515 
    6516  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
    6517  {
    6518  const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
    6519  VMA_ASSERT(pBlock);
    6520  VMA_HEAVY_ASSERT(pBlock->Validate());
    6521  pBlock->m_Metadata.AddPoolStats(*pStats);
    6522  }
    6523 }
    6524 
    6525 static const uint32_t VMA_ALLOCATION_TRY_COUNT = 32;
    6526 
    6527 VkResult VmaBlockVector::Allocate(
    6528  VmaPool hCurrentPool,
    6529  uint32_t currentFrameIndex,
    6530  const VkMemoryRequirements& vkMemReq,
    6531  const VmaAllocationCreateInfo& createInfo,
    6532  VmaSuballocationType suballocType,
    6533  VmaAllocation* pAllocation)
    6534 {
    6535  const bool mapped = (createInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0;
    6536  const bool isUserDataString = (createInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0;
    6537 
    6538  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    6539 
    6540  // 1. Search existing allocations. Try to allocate without making other allocations lost.
    6541  // Forward order in m_Blocks - prefer blocks with smallest amount of free space.
    6542  for(size_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex )
    6543  {
    6544  VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
    6545  VMA_ASSERT(pCurrBlock);
    6546  VmaAllocationRequest currRequest = {};
    6547  if(pCurrBlock->m_Metadata.CreateAllocationRequest(
    6548  currentFrameIndex,
    6549  m_FrameInUseCount,
    6550  m_BufferImageGranularity,
    6551  vkMemReq.size,
    6552  vkMemReq.alignment,
    6553  suballocType,
    6554  false, // canMakeOtherLost
    6555  &currRequest))
    6556  {
    6557  // Allocate from pCurrBlock.
    6558  VMA_ASSERT(currRequest.itemsToMakeLostCount == 0);
    6559 
    6560  if(mapped)
    6561  {
    6562  VkResult res = pCurrBlock->Map(m_hAllocator, 1, VMA_NULL);
    6563  if(res != VK_SUCCESS)
    6564  {
    6565  return res;
    6566  }
    6567  }
    6568 
    6569  // We no longer have an empty Allocation.
    6570  if(pCurrBlock->m_Metadata.IsEmpty())
    6571  {
    6572  m_HasEmptyBlock = false;
    6573  }
    6574 
    6575  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
    6576  pCurrBlock->m_Metadata.Alloc(currRequest, suballocType, vkMemReq.size, *pAllocation);
    6577  (*pAllocation)->InitBlockAllocation(
    6578  hCurrentPool,
    6579  pCurrBlock,
    6580  currRequest.offset,
    6581  vkMemReq.alignment,
    6582  vkMemReq.size,
    6583  suballocType,
    6584  mapped,
    6585  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
    6586  VMA_HEAVY_ASSERT(pCurrBlock->Validate());
    6587  VMA_DEBUG_LOG(" Returned from existing allocation #%u", (uint32_t)blockIndex);
    6588  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
    6589  return VK_SUCCESS;
    6590  }
    6591  }
    6592 
    6593  const bool canCreateNewBlock =
    6594  ((createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == 0) &&
    6595  (m_Blocks.size() < m_MaxBlockCount);
    6596 
    6597  // 2. Try to create new block.
    6598  if(canCreateNewBlock)
    6599  {
    6600  // Calculate optimal size for new block.
    6601  VkDeviceSize newBlockSize = m_PreferredBlockSize;
    6602  uint32_t newBlockSizeShift = 0;
    6603  const uint32_t NEW_BLOCK_SIZE_SHIFT_MAX = 3;
    6604 
    6605  // Allocating blocks of other sizes is allowed only in default pools.
    6606  // In custom pools block size is fixed.
    6607  if(m_IsCustomPool == false)
    6608  {
    6609  // Allocate 1/8, 1/4, 1/2 as first blocks.
    6610  const VkDeviceSize maxExistingBlockSize = CalcMaxBlockSize();
    6611  for(uint32_t i = 0; i < NEW_BLOCK_SIZE_SHIFT_MAX; ++i)
    6612  {
    6613  const VkDeviceSize smallerNewBlockSize = newBlockSize / 2;
    6614  if(smallerNewBlockSize > maxExistingBlockSize && smallerNewBlockSize >= vkMemReq.size * 2)
    6615  {
    6616  newBlockSize = smallerNewBlockSize;
    6617  ++newBlockSizeShift;
    6618  }
    6619  else
    6620  {
    6621  break;
    6622  }
    6623  }
    6624  }
    6625 
    6626  size_t newBlockIndex = 0;
    6627  VkResult res = CreateBlock(newBlockSize, &newBlockIndex);
    6628  // Allocation of this size failed? Try 1/2, 1/4, 1/8 of m_PreferredBlockSize.
    6629  if(m_IsCustomPool == false)
    6630  {
    6631  while(res < 0 && newBlockSizeShift < NEW_BLOCK_SIZE_SHIFT_MAX)
    6632  {
    6633  const VkDeviceSize smallerNewBlockSize = newBlockSize / 2;
    6634  if(smallerNewBlockSize >= vkMemReq.size)
    6635  {
    6636  newBlockSize = smallerNewBlockSize;
    6637  ++newBlockSizeShift;
    6638  res = CreateBlock(newBlockSize, &newBlockIndex);
    6639  }
    6640  else
    6641  {
    6642  break;
    6643  }
    6644  }
    6645  }
    6646 
    6647  if(res == VK_SUCCESS)
    6648  {
    6649  VmaDeviceMemoryBlock* const pBlock = m_Blocks[newBlockIndex];
    6650  VMA_ASSERT(pBlock->m_Metadata.GetSize() >= vkMemReq.size);
    6651 
    6652  if(mapped)
    6653  {
    6654  res = pBlock->Map(m_hAllocator, 1, VMA_NULL);
    6655  if(res != VK_SUCCESS)
    6656  {
    6657  return res;
    6658  }
    6659  }
    6660 
    6661  // Allocate from pBlock. Because it is empty, dstAllocRequest can be trivially filled.
    6662  VmaAllocationRequest allocRequest;
    6663  pBlock->m_Metadata.CreateFirstAllocationRequest(&allocRequest);
    6664  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
    6665  pBlock->m_Metadata.Alloc(allocRequest, suballocType, vkMemReq.size, *pAllocation);
    6666  (*pAllocation)->InitBlockAllocation(
    6667  hCurrentPool,
    6668  pBlock,
    6669  allocRequest.offset,
    6670  vkMemReq.alignment,
    6671  vkMemReq.size,
    6672  suballocType,
    6673  mapped,
    6674  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
    6675  VMA_HEAVY_ASSERT(pBlock->Validate());
    6676  VMA_DEBUG_LOG(" Created new allocation Size=%llu", allocInfo.allocationSize);
    6677  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
    6678  return VK_SUCCESS;
    6679  }
    6680  }
    6681 
    6682  const bool canMakeOtherLost = (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT) != 0;
    6683 
    6684  // 3. Try to allocate from existing blocks with making other allocations lost.
    6685  if(canMakeOtherLost)
    6686  {
    6687  uint32_t tryIndex = 0;
    6688  for(; tryIndex < VMA_ALLOCATION_TRY_COUNT; ++tryIndex)
    6689  {
    6690  VmaDeviceMemoryBlock* pBestRequestBlock = VMA_NULL;
    6691  VmaAllocationRequest bestRequest = {};
    6692  VkDeviceSize bestRequestCost = VK_WHOLE_SIZE;
    6693 
    6694  // 1. Search existing allocations.
    6695  // Forward order in m_Blocks - prefer blocks with smallest amount of free space.
    6696  for(size_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex )
    6697  {
    6698  VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
    6699  VMA_ASSERT(pCurrBlock);
    6700  VmaAllocationRequest currRequest = {};
    6701  if(pCurrBlock->m_Metadata.CreateAllocationRequest(
    6702  currentFrameIndex,
    6703  m_FrameInUseCount,
    6704  m_BufferImageGranularity,
    6705  vkMemReq.size,
    6706  vkMemReq.alignment,
    6707  suballocType,
    6708  canMakeOtherLost,
    6709  &currRequest))
    6710  {
    6711  const VkDeviceSize currRequestCost = currRequest.CalcCost();
    6712  if(pBestRequestBlock == VMA_NULL ||
    6713  currRequestCost < bestRequestCost)
    6714  {
    6715  pBestRequestBlock = pCurrBlock;
    6716  bestRequest = currRequest;
    6717  bestRequestCost = currRequestCost;
    6718 
    6719  if(bestRequestCost == 0)
    6720  {
    6721  break;
    6722  }
    6723  }
    6724  }
    6725  }
    6726 
    6727  if(pBestRequestBlock != VMA_NULL)
    6728  {
    6729  if(mapped)
    6730  {
    6731  VkResult res = pBestRequestBlock->Map(m_hAllocator, 1, VMA_NULL);
    6732  if(res != VK_SUCCESS)
    6733  {
    6734  return res;
    6735  }
    6736  }
    6737 
    6738  if(pBestRequestBlock->m_Metadata.MakeRequestedAllocationsLost(
    6739  currentFrameIndex,
    6740  m_FrameInUseCount,
    6741  &bestRequest))
    6742  {
    6743  // We no longer have an empty Allocation.
    6744  if(pBestRequestBlock->m_Metadata.IsEmpty())
    6745  {
    6746  m_HasEmptyBlock = false;
    6747  }
    6748  // Allocate from this pBlock.
    6749  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
    6750  pBestRequestBlock->m_Metadata.Alloc(bestRequest, suballocType, vkMemReq.size, *pAllocation);
    6751  (*pAllocation)->InitBlockAllocation(
    6752  hCurrentPool,
    6753  pBestRequestBlock,
    6754  bestRequest.offset,
    6755  vkMemReq.alignment,
    6756  vkMemReq.size,
    6757  suballocType,
    6758  mapped,
    6759  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
    6760  VMA_HEAVY_ASSERT(pBestRequestBlock->Validate());
    6761  VMA_DEBUG_LOG(" Returned from existing allocation #%u", (uint32_t)blockIndex);
    6762  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
    6763  return VK_SUCCESS;
    6764  }
    6765  // else: Some allocations must have been touched while we are here. Next try.
    6766  }
    6767  else
    6768  {
    6769  // Could not find place in any of the blocks - break outer loop.
    6770  break;
    6771  }
    6772  }
    6773  /* Maximum number of tries exceeded - a very unlike event when many other
    6774  threads are simultaneously touching allocations making it impossible to make
    6775  lost at the same time as we try to allocate. */
    6776  if(tryIndex == VMA_ALLOCATION_TRY_COUNT)
    6777  {
    6778  return VK_ERROR_TOO_MANY_OBJECTS;
    6779  }
    6780  }
    6781 
    6782  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    6783 }
    6784 
    6785 void VmaBlockVector::Free(
    6786  VmaAllocation hAllocation)
    6787 {
    6788  VmaDeviceMemoryBlock* pBlockToDelete = VMA_NULL;
    6789 
    6790  // Scope for lock.
    6791  {
    6792  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    6793 
    6794  VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
    6795 
    6796  if(hAllocation->IsPersistentMap())
    6797  {
    6798  pBlock->Unmap(m_hAllocator, 1);
    6799  }
    6800 
    6801  pBlock->m_Metadata.Free(hAllocation);
    6802  VMA_HEAVY_ASSERT(pBlock->Validate());
    6803 
    6804  VMA_DEBUG_LOG(" Freed from MemoryTypeIndex=%u", memTypeIndex);
    6805 
    6806  // pBlock became empty after this deallocation.
    6807  if(pBlock->m_Metadata.IsEmpty())
    6808  {
    6809  // Already has empty Allocation. We don't want to have two, so delete this one.
    6810  if(m_HasEmptyBlock && m_Blocks.size() > m_MinBlockCount)
    6811  {
    6812  pBlockToDelete = pBlock;
    6813  Remove(pBlock);
    6814  }
    6815  // We now have first empty Allocation.
    6816  else
    6817  {
    6818  m_HasEmptyBlock = true;
    6819  }
    6820  }
    6821  // pBlock didn't become empty, but we have another empty block - find and free that one.
    6822  // (This is optional, heuristics.)
    6823  else if(m_HasEmptyBlock)
    6824  {
    6825  VmaDeviceMemoryBlock* pLastBlock = m_Blocks.back();
    6826  if(pLastBlock->m_Metadata.IsEmpty() && m_Blocks.size() > m_MinBlockCount)
    6827  {
    6828  pBlockToDelete = pLastBlock;
    6829  m_Blocks.pop_back();
    6830  m_HasEmptyBlock = false;
    6831  }
    6832  }
    6833 
    6834  IncrementallySortBlocks();
    6835  }
    6836 
    6837  // Destruction of a free Allocation. Deferred until this point, outside of mutex
    6838  // lock, for performance reason.
    6839  if(pBlockToDelete != VMA_NULL)
    6840  {
    6841  VMA_DEBUG_LOG(" Deleted empty allocation");
    6842  pBlockToDelete->Destroy(m_hAllocator);
    6843  vma_delete(m_hAllocator, pBlockToDelete);
    6844  }
    6845 }
    6846 
    6847 size_t VmaBlockVector::CalcMaxBlockSize() const
    6848 {
    6849  size_t result = 0;
    6850  for(size_t i = m_Blocks.size(); i--; )
    6851  {
    6852  result = VMA_MAX((uint64_t)result, (uint64_t)m_Blocks[i]->m_Metadata.GetSize());
    6853  if(result >= m_PreferredBlockSize)
    6854  {
    6855  break;
    6856  }
    6857  }
    6858  return result;
    6859 }
    6860 
    6861 void VmaBlockVector::Remove(VmaDeviceMemoryBlock* pBlock)
    6862 {
    6863  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
    6864  {
    6865  if(m_Blocks[blockIndex] == pBlock)
    6866  {
    6867  VmaVectorRemove(m_Blocks, blockIndex);
    6868  return;
    6869  }
    6870  }
    6871  VMA_ASSERT(0);
    6872 }
    6873 
    6874 void VmaBlockVector::IncrementallySortBlocks()
    6875 {
    6876  // Bubble sort only until first swap.
    6877  for(size_t i = 1; i < m_Blocks.size(); ++i)
    6878  {
    6879  if(m_Blocks[i - 1]->m_Metadata.GetSumFreeSize() > m_Blocks[i]->m_Metadata.GetSumFreeSize())
    6880  {
    6881  VMA_SWAP(m_Blocks[i - 1], m_Blocks[i]);
    6882  return;
    6883  }
    6884  }
    6885 }
    6886 
    6887 VkResult VmaBlockVector::CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex)
    6888 {
    6889  VkMemoryAllocateInfo allocInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
    6890  allocInfo.memoryTypeIndex = m_MemoryTypeIndex;
    6891  allocInfo.allocationSize = blockSize;
    6892  VkDeviceMemory mem = VK_NULL_HANDLE;
    6893  VkResult res = m_hAllocator->AllocateVulkanMemory(&allocInfo, &mem);
    6894  if(res < 0)
    6895  {
    6896  return res;
    6897  }
    6898 
    6899  // New VkDeviceMemory successfully created.
    6900 
    6901  // Create new Allocation for it.
    6902  VmaDeviceMemoryBlock* const pBlock = vma_new(m_hAllocator, VmaDeviceMemoryBlock)(m_hAllocator);
    6903  pBlock->Init(
    6904  m_MemoryTypeIndex,
    6905  mem,
    6906  allocInfo.allocationSize);
    6907 
    6908  m_Blocks.push_back(pBlock);
    6909  if(pNewBlockIndex != VMA_NULL)
    6910  {
    6911  *pNewBlockIndex = m_Blocks.size() - 1;
    6912  }
    6913 
    6914  return VK_SUCCESS;
    6915 }
    6916 
    6917 #if VMA_STATS_STRING_ENABLED
    6918 
    6919 void VmaBlockVector::PrintDetailedMap(class VmaJsonWriter& json)
    6920 {
    6921  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    6922 
    6923  json.BeginObject();
    6924 
    6925  if(m_IsCustomPool)
    6926  {
    6927  json.WriteString("MemoryTypeIndex");
    6928  json.WriteNumber(m_MemoryTypeIndex);
    6929 
    6930  json.WriteString("BlockSize");
    6931  json.WriteNumber(m_PreferredBlockSize);
    6932 
    6933  json.WriteString("BlockCount");
    6934  json.BeginObject(true);
    6935  if(m_MinBlockCount > 0)
    6936  {
    6937  json.WriteString("Min");
    6938  json.WriteNumber((uint64_t)m_MinBlockCount);
    6939  }
    6940  if(m_MaxBlockCount < SIZE_MAX)
    6941  {
    6942  json.WriteString("Max");
    6943  json.WriteNumber((uint64_t)m_MaxBlockCount);
    6944  }
    6945  json.WriteString("Cur");
    6946  json.WriteNumber((uint64_t)m_Blocks.size());
    6947  json.EndObject();
    6948 
    6949  if(m_FrameInUseCount > 0)
    6950  {
    6951  json.WriteString("FrameInUseCount");
    6952  json.WriteNumber(m_FrameInUseCount);
    6953  }
    6954  }
    6955  else
    6956  {
    6957  json.WriteString("PreferredBlockSize");
    6958  json.WriteNumber(m_PreferredBlockSize);
    6959  }
    6960 
    6961  json.WriteString("Blocks");
    6962  json.BeginArray();
    6963  for(size_t i = 0; i < m_Blocks.size(); ++i)
    6964  {
    6965  m_Blocks[i]->m_Metadata.PrintDetailedMap(json);
    6966  }
    6967  json.EndArray();
    6968 
    6969  json.EndObject();
    6970 }
    6971 
    6972 #endif // #if VMA_STATS_STRING_ENABLED
    6973 
    6974 VmaDefragmentator* VmaBlockVector::EnsureDefragmentator(
    6975  VmaAllocator hAllocator,
    6976  uint32_t currentFrameIndex)
    6977 {
    6978  if(m_pDefragmentator == VMA_NULL)
    6979  {
    6980  m_pDefragmentator = vma_new(m_hAllocator, VmaDefragmentator)(
    6981  hAllocator,
    6982  this,
    6983  currentFrameIndex);
    6984  }
    6985 
    6986  return m_pDefragmentator;
    6987 }
    6988 
    6989 VkResult VmaBlockVector::Defragment(
    6990  VmaDefragmentationStats* pDefragmentationStats,
    6991  VkDeviceSize& maxBytesToMove,
    6992  uint32_t& maxAllocationsToMove)
    6993 {
    6994  if(m_pDefragmentator == VMA_NULL)
    6995  {
    6996  return VK_SUCCESS;
    6997  }
    6998 
    6999  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    7000 
    7001  // Defragment.
    7002  VkResult result = m_pDefragmentator->Defragment(maxBytesToMove, maxAllocationsToMove);
    7003 
    7004  // Accumulate statistics.
    7005  if(pDefragmentationStats != VMA_NULL)
    7006  {
    7007  const VkDeviceSize bytesMoved = m_pDefragmentator->GetBytesMoved();
    7008  const uint32_t allocationsMoved = m_pDefragmentator->GetAllocationsMoved();
    7009  pDefragmentationStats->bytesMoved += bytesMoved;
    7010  pDefragmentationStats->allocationsMoved += allocationsMoved;
    7011  VMA_ASSERT(bytesMoved <= maxBytesToMove);
    7012  VMA_ASSERT(allocationsMoved <= maxAllocationsToMove);
    7013  maxBytesToMove -= bytesMoved;
    7014  maxAllocationsToMove -= allocationsMoved;
    7015  }
    7016 
    7017  // Free empty blocks.
    7018  m_HasEmptyBlock = false;
    7019  for(size_t blockIndex = m_Blocks.size(); blockIndex--; )
    7020  {
    7021  VmaDeviceMemoryBlock* pBlock = m_Blocks[blockIndex];
    7022  if(pBlock->m_Metadata.IsEmpty())
    7023  {
    7024  if(m_Blocks.size() > m_MinBlockCount)
    7025  {
    7026  if(pDefragmentationStats != VMA_NULL)
    7027  {
    7028  ++pDefragmentationStats->deviceMemoryBlocksFreed;
    7029  pDefragmentationStats->bytesFreed += pBlock->m_Metadata.GetSize();
    7030  }
    7031 
    7032  VmaVectorRemove(m_Blocks, blockIndex);
    7033  pBlock->Destroy(m_hAllocator);
    7034  vma_delete(m_hAllocator, pBlock);
    7035  }
    7036  else
    7037  {
    7038  m_HasEmptyBlock = true;
    7039  }
    7040  }
    7041  }
    7042 
    7043  return result;
    7044 }
    7045 
    7046 void VmaBlockVector::DestroyDefragmentator()
    7047 {
    7048  if(m_pDefragmentator != VMA_NULL)
    7049  {
    7050  vma_delete(m_hAllocator, m_pDefragmentator);
    7051  m_pDefragmentator = VMA_NULL;
    7052  }
    7053 }
    7054 
    7055 void VmaBlockVector::MakePoolAllocationsLost(
    7056  uint32_t currentFrameIndex,
    7057  size_t* pLostAllocationCount)
    7058 {
    7059  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    7060  size_t lostAllocationCount = 0;
    7061  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
    7062  {
    7063  VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
    7064  VMA_ASSERT(pBlock);
    7065  lostAllocationCount += pBlock->m_Metadata.MakeAllocationsLost(currentFrameIndex, m_FrameInUseCount);
    7066  }
    7067  if(pLostAllocationCount != VMA_NULL)
    7068  {
    7069  *pLostAllocationCount = lostAllocationCount;
    7070  }
    7071 }
    7072 
    7073 void VmaBlockVector::AddStats(VmaStats* pStats)
    7074 {
    7075  const uint32_t memTypeIndex = m_MemoryTypeIndex;
    7076  const uint32_t memHeapIndex = m_hAllocator->MemoryTypeIndexToHeapIndex(memTypeIndex);
    7077 
    7078  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    7079 
    7080  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
    7081  {
    7082  const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
    7083  VMA_ASSERT(pBlock);
    7084  VMA_HEAVY_ASSERT(pBlock->Validate());
    7085  VmaStatInfo allocationStatInfo;
    7086  pBlock->m_Metadata.CalcAllocationStatInfo(allocationStatInfo);
    7087  VmaAddStatInfo(pStats->total, allocationStatInfo);
    7088  VmaAddStatInfo(pStats->memoryType[memTypeIndex], allocationStatInfo);
    7089  VmaAddStatInfo(pStats->memoryHeap[memHeapIndex], allocationStatInfo);
    7090  }
    7091 }
    7092 
    7094 // VmaDefragmentator members definition
    7095 
    7096 VmaDefragmentator::VmaDefragmentator(
    7097  VmaAllocator hAllocator,
    7098  VmaBlockVector* pBlockVector,
    7099  uint32_t currentFrameIndex) :
    7100  m_hAllocator(hAllocator),
    7101  m_pBlockVector(pBlockVector),
    7102  m_CurrentFrameIndex(currentFrameIndex),
    7103  m_BytesMoved(0),
    7104  m_AllocationsMoved(0),
    7105  m_Allocations(VmaStlAllocator<AllocationInfo>(hAllocator->GetAllocationCallbacks())),
    7106  m_Blocks(VmaStlAllocator<BlockInfo*>(hAllocator->GetAllocationCallbacks()))
    7107 {
    7108 }
    7109 
    7110 VmaDefragmentator::~VmaDefragmentator()
    7111 {
    7112  for(size_t i = m_Blocks.size(); i--; )
    7113  {
    7114  vma_delete(m_hAllocator, m_Blocks[i]);
    7115  }
    7116 }
    7117 
    7118 void VmaDefragmentator::AddAllocation(VmaAllocation hAlloc, VkBool32* pChanged)
    7119 {
    7120  AllocationInfo allocInfo;
    7121  allocInfo.m_hAllocation = hAlloc;
    7122  allocInfo.m_pChanged = pChanged;
    7123  m_Allocations.push_back(allocInfo);
    7124 }
    7125 
    7126 VkResult VmaDefragmentator::BlockInfo::EnsureMapping(VmaAllocator hAllocator, void** ppMappedData)
    7127 {
    7128  // It has already been mapped for defragmentation.
    7129  if(m_pMappedDataForDefragmentation)
    7130  {
    7131  *ppMappedData = m_pMappedDataForDefragmentation;
    7132  return VK_SUCCESS;
    7133  }
    7134 
    7135  // It is originally mapped.
    7136  if(m_pBlock->GetMappedData())
    7137  {
    7138  *ppMappedData = m_pBlock->GetMappedData();
    7139  return VK_SUCCESS;
    7140  }
    7141 
    7142  // Map on first usage.
    7143  VkResult res = m_pBlock->Map(hAllocator, 1, &m_pMappedDataForDefragmentation);
    7144  *ppMappedData = m_pMappedDataForDefragmentation;
    7145  return res;
    7146 }
    7147 
    7148 void VmaDefragmentator::BlockInfo::Unmap(VmaAllocator hAllocator)
    7149 {
    7150  if(m_pMappedDataForDefragmentation != VMA_NULL)
    7151  {
    7152  m_pBlock->Unmap(hAllocator, 1);
    7153  }
    7154 }
    7155 
    7156 VkResult VmaDefragmentator::DefragmentRound(
    7157  VkDeviceSize maxBytesToMove,
    7158  uint32_t maxAllocationsToMove)
    7159 {
    7160  if(m_Blocks.empty())
    7161  {
    7162  return VK_SUCCESS;
    7163  }
    7164 
    7165  size_t srcBlockIndex = m_Blocks.size() - 1;
    7166  size_t srcAllocIndex = SIZE_MAX;
    7167  for(;;)
    7168  {
    7169  // 1. Find next allocation to move.
    7170  // 1.1. Start from last to first m_Blocks - they are sorted from most "destination" to most "source".
    7171  // 1.2. Then start from last to first m_Allocations - they are sorted from largest to smallest.
    7172  while(srcAllocIndex >= m_Blocks[srcBlockIndex]->m_Allocations.size())
    7173  {
    7174  if(m_Blocks[srcBlockIndex]->m_Allocations.empty())
    7175  {
    7176  // Finished: no more allocations to process.
    7177  if(srcBlockIndex == 0)
    7178  {
    7179  return VK_SUCCESS;
    7180  }
    7181  else
    7182  {
    7183  --srcBlockIndex;
    7184  srcAllocIndex = SIZE_MAX;
    7185  }
    7186  }
    7187  else
    7188  {
    7189  srcAllocIndex = m_Blocks[srcBlockIndex]->m_Allocations.size() - 1;
    7190  }
    7191  }
    7192 
    7193  BlockInfo* pSrcBlockInfo = m_Blocks[srcBlockIndex];
    7194  AllocationInfo& allocInfo = pSrcBlockInfo->m_Allocations[srcAllocIndex];
    7195 
    7196  const VkDeviceSize size = allocInfo.m_hAllocation->GetSize();
    7197  const VkDeviceSize srcOffset = allocInfo.m_hAllocation->GetOffset();
    7198  const VkDeviceSize alignment = allocInfo.m_hAllocation->GetAlignment();
    7199  const VmaSuballocationType suballocType = allocInfo.m_hAllocation->GetSuballocationType();
    7200 
    7201  // 2. Try to find new place for this allocation in preceding or current block.
    7202  for(size_t dstBlockIndex = 0; dstBlockIndex <= srcBlockIndex; ++dstBlockIndex)
    7203  {
    7204  BlockInfo* pDstBlockInfo = m_Blocks[dstBlockIndex];
    7205  VmaAllocationRequest dstAllocRequest;
    7206  if(pDstBlockInfo->m_pBlock->m_Metadata.CreateAllocationRequest(
    7207  m_CurrentFrameIndex,
    7208  m_pBlockVector->GetFrameInUseCount(),
    7209  m_pBlockVector->GetBufferImageGranularity(),
    7210  size,
    7211  alignment,
    7212  suballocType,
    7213  false, // canMakeOtherLost
    7214  &dstAllocRequest) &&
    7215  MoveMakesSense(
    7216  dstBlockIndex, dstAllocRequest.offset, srcBlockIndex, srcOffset))
    7217  {
    7218  VMA_ASSERT(dstAllocRequest.itemsToMakeLostCount == 0);
    7219 
    7220  // Reached limit on number of allocations or bytes to move.
    7221  if((m_AllocationsMoved + 1 > maxAllocationsToMove) ||
    7222  (m_BytesMoved + size > maxBytesToMove))
    7223  {
    7224  return VK_INCOMPLETE;
    7225  }
    7226 
    7227  void* pDstMappedData = VMA_NULL;
    7228  VkResult res = pDstBlockInfo->EnsureMapping(m_hAllocator, &pDstMappedData);
    7229  if(res != VK_SUCCESS)
    7230  {
    7231  return res;
    7232  }
    7233 
    7234  void* pSrcMappedData = VMA_NULL;
    7235  res = pSrcBlockInfo->EnsureMapping(m_hAllocator, &pSrcMappedData);
    7236  if(res != VK_SUCCESS)
    7237  {
    7238  return res;
    7239  }
    7240 
    7241  // THE PLACE WHERE ACTUAL DATA COPY HAPPENS.
    7242  memcpy(
    7243  reinterpret_cast<char*>(pDstMappedData) + dstAllocRequest.offset,
    7244  reinterpret_cast<char*>(pSrcMappedData) + srcOffset,
    7245  static_cast<size_t>(size));
    7246 
    7247  pDstBlockInfo->m_pBlock->m_Metadata.Alloc(dstAllocRequest, suballocType, size, allocInfo.m_hAllocation);
    7248  pSrcBlockInfo->m_pBlock->m_Metadata.FreeAtOffset(srcOffset);
    7249 
    7250  allocInfo.m_hAllocation->ChangeBlockAllocation(m_hAllocator, pDstBlockInfo->m_pBlock, dstAllocRequest.offset);
    7251 
    7252  if(allocInfo.m_pChanged != VMA_NULL)
    7253  {
    7254  *allocInfo.m_pChanged = VK_TRUE;
    7255  }
    7256 
    7257  ++m_AllocationsMoved;
    7258  m_BytesMoved += size;
    7259 
    7260  VmaVectorRemove(pSrcBlockInfo->m_Allocations, srcAllocIndex);
    7261 
    7262  break;
    7263  }
    7264  }
    7265 
    7266  // If not processed, this allocInfo remains in pBlockInfo->m_Allocations for next round.
    7267 
    7268  if(srcAllocIndex > 0)
    7269  {
    7270  --srcAllocIndex;
    7271  }
    7272  else
    7273  {
    7274  if(srcBlockIndex > 0)
    7275  {
    7276  --srcBlockIndex;
    7277  srcAllocIndex = SIZE_MAX;
    7278  }
    7279  else
    7280  {
    7281  return VK_SUCCESS;
    7282  }
    7283  }
    7284  }
    7285 }
    7286 
    7287 VkResult VmaDefragmentator::Defragment(
    7288  VkDeviceSize maxBytesToMove,
    7289  uint32_t maxAllocationsToMove)
    7290 {
    7291  if(m_Allocations.empty())
    7292  {
    7293  return VK_SUCCESS;
    7294  }
    7295 
    7296  // Create block info for each block.
    7297  const size_t blockCount = m_pBlockVector->m_Blocks.size();
    7298  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
    7299  {
    7300  BlockInfo* pBlockInfo = vma_new(m_hAllocator, BlockInfo)(m_hAllocator->GetAllocationCallbacks());
    7301  pBlockInfo->m_pBlock = m_pBlockVector->m_Blocks[blockIndex];
    7302  m_Blocks.push_back(pBlockInfo);
    7303  }
    7304 
    7305  // Sort them by m_pBlock pointer value.
    7306  VMA_SORT(m_Blocks.begin(), m_Blocks.end(), BlockPointerLess());
    7307 
    7308  // Move allocation infos from m_Allocations to appropriate m_Blocks[memTypeIndex].m_Allocations.
    7309  for(size_t blockIndex = 0, allocCount = m_Allocations.size(); blockIndex < allocCount; ++blockIndex)
    7310  {
    7311  AllocationInfo& allocInfo = m_Allocations[blockIndex];
    7312  // Now as we are inside VmaBlockVector::m_Mutex, we can make final check if this allocation was not lost.
    7313  if(allocInfo.m_hAllocation->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST)
    7314  {
    7315  VmaDeviceMemoryBlock* pBlock = allocInfo.m_hAllocation->GetBlock();
    7316  BlockInfoVector::iterator it = VmaBinaryFindFirstNotLess(m_Blocks.begin(), m_Blocks.end(), pBlock, BlockPointerLess());
    7317  if(it != m_Blocks.end() && (*it)->m_pBlock == pBlock)
    7318  {
    7319  (*it)->m_Allocations.push_back(allocInfo);
    7320  }
    7321  else
    7322  {
    7323  VMA_ASSERT(0);
    7324  }
    7325  }
    7326  }
    7327  m_Allocations.clear();
    7328 
    7329  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
    7330  {
    7331  BlockInfo* pBlockInfo = m_Blocks[blockIndex];
    7332  pBlockInfo->CalcHasNonMovableAllocations();
    7333  pBlockInfo->SortAllocationsBySizeDescecnding();
    7334  }
    7335 
    7336  // Sort m_Blocks this time by the main criterium, from most "destination" to most "source" blocks.
    7337  VMA_SORT(m_Blocks.begin(), m_Blocks.end(), BlockInfoCompareMoveDestination());
    7338 
    7339  // Execute defragmentation rounds (the main part).
    7340  VkResult result = VK_SUCCESS;
    7341  for(size_t round = 0; (round < 2) && (result == VK_SUCCESS); ++round)
    7342  {
    7343  result = DefragmentRound(maxBytesToMove, maxAllocationsToMove);
    7344  }
    7345 
    7346  // Unmap blocks that were mapped for defragmentation.
    7347  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
    7348  {
    7349  m_Blocks[blockIndex]->Unmap(m_hAllocator);
    7350  }
    7351 
    7352  return result;
    7353 }
    7354 
    7355 bool VmaDefragmentator::MoveMakesSense(
    7356  size_t dstBlockIndex, VkDeviceSize dstOffset,
    7357  size_t srcBlockIndex, VkDeviceSize srcOffset)
    7358 {
    7359  if(dstBlockIndex < srcBlockIndex)
    7360  {
    7361  return true;
    7362  }
    7363  if(dstBlockIndex > srcBlockIndex)
    7364  {
    7365  return false;
    7366  }
    7367  if(dstOffset < srcOffset)
    7368  {
    7369  return true;
    7370  }
    7371  return false;
    7372 }
    7373 
    7375 // VmaAllocator_T
    7376 
    7377 VmaAllocator_T::VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo) :
    7378  m_UseMutex((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT) == 0),
    7379  m_UseKhrDedicatedAllocation((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT) != 0),
    7380  m_hDevice(pCreateInfo->device),
    7381  m_AllocationCallbacksSpecified(pCreateInfo->pAllocationCallbacks != VMA_NULL),
    7382  m_AllocationCallbacks(pCreateInfo->pAllocationCallbacks ?
    7383  *pCreateInfo->pAllocationCallbacks : VmaEmptyAllocationCallbacks),
    7384  m_PreferredLargeHeapBlockSize(0),
    7385  m_PhysicalDevice(pCreateInfo->physicalDevice),
    7386  m_CurrentFrameIndex(0),
    7387  m_Pools(VmaStlAllocator<VmaPool>(GetAllocationCallbacks()))
    7388 {
    7389  VMA_ASSERT(pCreateInfo->physicalDevice && pCreateInfo->device);
    7390 
    7391  memset(&m_DeviceMemoryCallbacks, 0 ,sizeof(m_DeviceMemoryCallbacks));
    7392  memset(&m_MemProps, 0, sizeof(m_MemProps));
    7393  memset(&m_PhysicalDeviceProperties, 0, sizeof(m_PhysicalDeviceProperties));
    7394 
    7395  memset(&m_pBlockVectors, 0, sizeof(m_pBlockVectors));
    7396  memset(&m_pDedicatedAllocations, 0, sizeof(m_pDedicatedAllocations));
    7397 
    7398  for(uint32_t i = 0; i < VK_MAX_MEMORY_HEAPS; ++i)
    7399  {
    7400  m_HeapSizeLimit[i] = VK_WHOLE_SIZE;
    7401  }
    7402 
    7403  if(pCreateInfo->pDeviceMemoryCallbacks != VMA_NULL)
    7404  {
    7405  m_DeviceMemoryCallbacks.pfnAllocate = pCreateInfo->pDeviceMemoryCallbacks->pfnAllocate;
    7406  m_DeviceMemoryCallbacks.pfnFree = pCreateInfo->pDeviceMemoryCallbacks->pfnFree;
    7407  }
    7408 
    7409  ImportVulkanFunctions(pCreateInfo->pVulkanFunctions);
    7410 
    7411  (*m_VulkanFunctions.vkGetPhysicalDeviceProperties)(m_PhysicalDevice, &m_PhysicalDeviceProperties);
    7412  (*m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties)(m_PhysicalDevice, &m_MemProps);
    7413 
    7414  m_PreferredLargeHeapBlockSize = (pCreateInfo->preferredLargeHeapBlockSize != 0) ?
    7415  pCreateInfo->preferredLargeHeapBlockSize : static_cast<VkDeviceSize>(VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE);
    7416 
    7417  if(pCreateInfo->pHeapSizeLimit != VMA_NULL)
    7418  {
    7419  for(uint32_t heapIndex = 0; heapIndex < GetMemoryHeapCount(); ++heapIndex)
    7420  {
    7421  const VkDeviceSize limit = pCreateInfo->pHeapSizeLimit[heapIndex];
    7422  if(limit != VK_WHOLE_SIZE)
    7423  {
    7424  m_HeapSizeLimit[heapIndex] = limit;
    7425  if(limit < m_MemProps.memoryHeaps[heapIndex].size)
    7426  {
    7427  m_MemProps.memoryHeaps[heapIndex].size = limit;
    7428  }
    7429  }
    7430  }
    7431  }
    7432 
    7433  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
    7434  {
    7435  const VkDeviceSize preferredBlockSize = CalcPreferredBlockSize(memTypeIndex);
    7436 
    7437  m_pBlockVectors[memTypeIndex] = vma_new(this, VmaBlockVector)(
    7438  this,
    7439  memTypeIndex,
    7440  preferredBlockSize,
    7441  0,
    7442  SIZE_MAX,
    7443  GetBufferImageGranularity(),
    7444  pCreateInfo->frameInUseCount,
    7445  false); // isCustomPool
    7446  // No need to call m_pBlockVectors[memTypeIndex][blockVectorTypeIndex]->CreateMinBlocks here,
    7447  // becase minBlockCount is 0.
    7448  m_pDedicatedAllocations[memTypeIndex] = vma_new(this, AllocationVectorType)(VmaStlAllocator<VmaAllocation>(GetAllocationCallbacks()));
    7449  }
    7450 }
    7451 
    7452 VmaAllocator_T::~VmaAllocator_T()
    7453 {
    7454  VMA_ASSERT(m_Pools.empty());
    7455 
    7456  for(size_t i = GetMemoryTypeCount(); i--; )
    7457  {
    7458  vma_delete(this, m_pDedicatedAllocations[i]);
    7459  vma_delete(this, m_pBlockVectors[i]);
    7460  }
    7461 }
    7462 
    7463 void VmaAllocator_T::ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions)
    7464 {
    7465 #if VMA_STATIC_VULKAN_FUNCTIONS == 1
    7466  m_VulkanFunctions.vkGetPhysicalDeviceProperties = &vkGetPhysicalDeviceProperties;
    7467  m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties = &vkGetPhysicalDeviceMemoryProperties;
    7468  m_VulkanFunctions.vkAllocateMemory = &vkAllocateMemory;
    7469  m_VulkanFunctions.vkFreeMemory = &vkFreeMemory;
    7470  m_VulkanFunctions.vkMapMemory = &vkMapMemory;
    7471  m_VulkanFunctions.vkUnmapMemory = &vkUnmapMemory;
    7472  m_VulkanFunctions.vkBindBufferMemory = &vkBindBufferMemory;
    7473  m_VulkanFunctions.vkBindImageMemory = &vkBindImageMemory;
    7474  m_VulkanFunctions.vkGetBufferMemoryRequirements = &vkGetBufferMemoryRequirements;
    7475  m_VulkanFunctions.vkGetImageMemoryRequirements = &vkGetImageMemoryRequirements;
    7476  m_VulkanFunctions.vkCreateBuffer = &vkCreateBuffer;
    7477  m_VulkanFunctions.vkDestroyBuffer = &vkDestroyBuffer;
    7478  m_VulkanFunctions.vkCreateImage = &vkCreateImage;
    7479  m_VulkanFunctions.vkDestroyImage = &vkDestroyImage;
    7480  if(m_UseKhrDedicatedAllocation)
    7481  {
    7482  m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR =
    7483  (PFN_vkGetBufferMemoryRequirements2KHR)vkGetDeviceProcAddr(m_hDevice, "vkGetBufferMemoryRequirements2KHR");
    7484  m_VulkanFunctions.vkGetImageMemoryRequirements2KHR =
    7485  (PFN_vkGetImageMemoryRequirements2KHR)vkGetDeviceProcAddr(m_hDevice, "vkGetImageMemoryRequirements2KHR");
    7486  }
    7487 #endif // #if VMA_STATIC_VULKAN_FUNCTIONS == 1
    7488 
    7489 #define VMA_COPY_IF_NOT_NULL(funcName) \
    7490  if(pVulkanFunctions->funcName != VMA_NULL) m_VulkanFunctions.funcName = pVulkanFunctions->funcName;
    7491 
    7492  if(pVulkanFunctions != VMA_NULL)
    7493  {
    7494  VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceProperties);
    7495  VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceMemoryProperties);
    7496  VMA_COPY_IF_NOT_NULL(vkAllocateMemory);
    7497  VMA_COPY_IF_NOT_NULL(vkFreeMemory);
    7498  VMA_COPY_IF_NOT_NULL(vkMapMemory);
    7499  VMA_COPY_IF_NOT_NULL(vkUnmapMemory);
    7500  VMA_COPY_IF_NOT_NULL(vkBindBufferMemory);
    7501  VMA_COPY_IF_NOT_NULL(vkBindImageMemory);
    7502  VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements);
    7503  VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements);
    7504  VMA_COPY_IF_NOT_NULL(vkCreateBuffer);
    7505  VMA_COPY_IF_NOT_NULL(vkDestroyBuffer);
    7506  VMA_COPY_IF_NOT_NULL(vkCreateImage);
    7507  VMA_COPY_IF_NOT_NULL(vkDestroyImage);
    7508  VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements2KHR);
    7509  VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements2KHR);
    7510  }
    7511 
    7512 #undef VMA_COPY_IF_NOT_NULL
    7513 
    7514  // If these asserts are hit, you must either #define VMA_STATIC_VULKAN_FUNCTIONS 1
    7515  // or pass valid pointers as VmaAllocatorCreateInfo::pVulkanFunctions.
    7516  VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceProperties != VMA_NULL);
    7517  VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties != VMA_NULL);
    7518  VMA_ASSERT(m_VulkanFunctions.vkAllocateMemory != VMA_NULL);
    7519  VMA_ASSERT(m_VulkanFunctions.vkFreeMemory != VMA_NULL);
    7520  VMA_ASSERT(m_VulkanFunctions.vkMapMemory != VMA_NULL);
    7521  VMA_ASSERT(m_VulkanFunctions.vkUnmapMemory != VMA_NULL);
    7522  VMA_ASSERT(m_VulkanFunctions.vkBindBufferMemory != VMA_NULL);
    7523  VMA_ASSERT(m_VulkanFunctions.vkBindImageMemory != VMA_NULL);
    7524  VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements != VMA_NULL);
    7525  VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements != VMA_NULL);
    7526  VMA_ASSERT(m_VulkanFunctions.vkCreateBuffer != VMA_NULL);
    7527  VMA_ASSERT(m_VulkanFunctions.vkDestroyBuffer != VMA_NULL);
    7528  VMA_ASSERT(m_VulkanFunctions.vkCreateImage != VMA_NULL);
    7529  VMA_ASSERT(m_VulkanFunctions.vkDestroyImage != VMA_NULL);
    7530  if(m_UseKhrDedicatedAllocation)
    7531  {
    7532  VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR != VMA_NULL);
    7533  VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements2KHR != VMA_NULL);
    7534  }
    7535 }
    7536 
    7537 VkDeviceSize VmaAllocator_T::CalcPreferredBlockSize(uint32_t memTypeIndex)
    7538 {
    7539  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
    7540  const VkDeviceSize heapSize = m_MemProps.memoryHeaps[heapIndex].size;
    7541  const bool isSmallHeap = heapSize <= VMA_SMALL_HEAP_MAX_SIZE;
    7542  return isSmallHeap ? (heapSize / 8) : m_PreferredLargeHeapBlockSize;
    7543 }
    7544 
    7545 VkResult VmaAllocator_T::AllocateMemoryOfType(
    7546  const VkMemoryRequirements& vkMemReq,
    7547  bool dedicatedAllocation,
    7548  VkBuffer dedicatedBuffer,
    7549  VkImage dedicatedImage,
    7550  const VmaAllocationCreateInfo& createInfo,
    7551  uint32_t memTypeIndex,
    7552  VmaSuballocationType suballocType,
    7553  VmaAllocation* pAllocation)
    7554 {
    7555  VMA_ASSERT(pAllocation != VMA_NULL);
    7556  VMA_DEBUG_LOG(" AllocateMemory: MemoryTypeIndex=%u, Size=%llu", memTypeIndex, vkMemReq.size);
    7557 
    7558  VmaAllocationCreateInfo finalCreateInfo = createInfo;
    7559 
    7560  // If memory type is not HOST_VISIBLE, disable MAPPED.
    7561  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0 &&
    7562  (m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) == 0)
    7563  {
    7564  finalCreateInfo.flags &= ~VMA_ALLOCATION_CREATE_MAPPED_BIT;
    7565  }
    7566 
    7567  VmaBlockVector* const blockVector = m_pBlockVectors[memTypeIndex];
    7568  VMA_ASSERT(blockVector);
    7569 
    7570  const VkDeviceSize preferredBlockSize = blockVector->GetPreferredBlockSize();
    7571  bool preferDedicatedMemory =
    7572  VMA_DEBUG_ALWAYS_DEDICATED_MEMORY ||
    7573  dedicatedAllocation ||
    7574  // Heuristics: Allocate dedicated memory if requested size if greater than half of preferred block size.
    7575  vkMemReq.size > preferredBlockSize / 2;
    7576 
    7577  if(preferDedicatedMemory &&
    7578  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == 0 &&
    7579  finalCreateInfo.pool == VK_NULL_HANDLE)
    7580  {
    7581  finalCreateInfo.flags |= VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
    7582  }
    7583 
    7584  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0)
    7585  {
    7586  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
    7587  {
    7588  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7589  }
    7590  else
    7591  {
    7592  return AllocateDedicatedMemory(
    7593  vkMemReq.size,
    7594  suballocType,
    7595  memTypeIndex,
    7596  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0,
    7597  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0,
    7598  finalCreateInfo.pUserData,
    7599  dedicatedBuffer,
    7600  dedicatedImage,
    7601  pAllocation);
    7602  }
    7603  }
    7604  else
    7605  {
    7606  VkResult res = blockVector->Allocate(
    7607  VK_NULL_HANDLE, // hCurrentPool
    7608  m_CurrentFrameIndex.load(),
    7609  vkMemReq,
    7610  finalCreateInfo,
    7611  suballocType,
    7612  pAllocation);
    7613  if(res == VK_SUCCESS)
    7614  {
    7615  return res;
    7616  }
    7617 
    7618  // 5. Try dedicated memory.
    7619  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
    7620  {
    7621  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7622  }
    7623  else
    7624  {
    7625  res = AllocateDedicatedMemory(
    7626  vkMemReq.size,
    7627  suballocType,
    7628  memTypeIndex,
    7629  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0,
    7630  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0,
    7631  finalCreateInfo.pUserData,
    7632  dedicatedBuffer,
    7633  dedicatedImage,
    7634  pAllocation);
    7635  if(res == VK_SUCCESS)
    7636  {
    7637  // Succeeded: AllocateDedicatedMemory function already filld pMemory, nothing more to do here.
    7638  VMA_DEBUG_LOG(" Allocated as DedicatedMemory");
    7639  return VK_SUCCESS;
    7640  }
    7641  else
    7642  {
    7643  // Everything failed: Return error code.
    7644  VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
    7645  return res;
    7646  }
    7647  }
    7648  }
    7649 }
    7650 
    7651 VkResult VmaAllocator_T::AllocateDedicatedMemory(
    7652  VkDeviceSize size,
    7653  VmaSuballocationType suballocType,
    7654  uint32_t memTypeIndex,
    7655  bool map,
    7656  bool isUserDataString,
    7657  void* pUserData,
    7658  VkBuffer dedicatedBuffer,
    7659  VkImage dedicatedImage,
    7660  VmaAllocation* pAllocation)
    7661 {
    7662  VMA_ASSERT(pAllocation);
    7663 
    7664  VkMemoryAllocateInfo allocInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
    7665  allocInfo.memoryTypeIndex = memTypeIndex;
    7666  allocInfo.allocationSize = size;
    7667 
    7668  VkMemoryDedicatedAllocateInfoKHR dedicatedAllocInfo = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO_KHR };
    7669  if(m_UseKhrDedicatedAllocation)
    7670  {
    7671  if(dedicatedBuffer != VK_NULL_HANDLE)
    7672  {
    7673  VMA_ASSERT(dedicatedImage == VK_NULL_HANDLE);
    7674  dedicatedAllocInfo.buffer = dedicatedBuffer;
    7675  allocInfo.pNext = &dedicatedAllocInfo;
    7676  }
    7677  else if(dedicatedImage != VK_NULL_HANDLE)
    7678  {
    7679  dedicatedAllocInfo.image = dedicatedImage;
    7680  allocInfo.pNext = &dedicatedAllocInfo;
    7681  }
    7682  }
    7683 
    7684  // Allocate VkDeviceMemory.
    7685  VkDeviceMemory hMemory = VK_NULL_HANDLE;
    7686  VkResult res = AllocateVulkanMemory(&allocInfo, &hMemory);
    7687  if(res < 0)
    7688  {
    7689  VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
    7690  return res;
    7691  }
    7692 
    7693  void* pMappedData = VMA_NULL;
    7694  if(map)
    7695  {
    7696  res = (*m_VulkanFunctions.vkMapMemory)(
    7697  m_hDevice,
    7698  hMemory,
    7699  0,
    7700  VK_WHOLE_SIZE,
    7701  0,
    7702  &pMappedData);
    7703  if(res < 0)
    7704  {
    7705  VMA_DEBUG_LOG(" vkMapMemory FAILED");
    7706  FreeVulkanMemory(memTypeIndex, size, hMemory);
    7707  return res;
    7708  }
    7709  }
    7710 
    7711  *pAllocation = vma_new(this, VmaAllocation_T)(m_CurrentFrameIndex.load(), isUserDataString);
    7712  (*pAllocation)->InitDedicatedAllocation(memTypeIndex, hMemory, suballocType, pMappedData, size);
    7713  (*pAllocation)->SetUserData(this, pUserData);
    7714 
    7715  // Register it in m_pDedicatedAllocations.
    7716  {
    7717  VmaMutexLock lock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
    7718  AllocationVectorType* pDedicatedAllocations = m_pDedicatedAllocations[memTypeIndex];
    7719  VMA_ASSERT(pDedicatedAllocations);
    7720  VmaVectorInsertSorted<VmaPointerLess>(*pDedicatedAllocations, *pAllocation);
    7721  }
    7722 
    7723  VMA_DEBUG_LOG(" Allocated DedicatedMemory MemoryTypeIndex=#%u", memTypeIndex);
    7724 
    7725  return VK_SUCCESS;
    7726 }
    7727 
    7728 void VmaAllocator_T::GetBufferMemoryRequirements(
    7729  VkBuffer hBuffer,
    7730  VkMemoryRequirements& memReq,
    7731  bool& requiresDedicatedAllocation,
    7732  bool& prefersDedicatedAllocation) const
    7733 {
    7734  if(m_UseKhrDedicatedAllocation)
    7735  {
    7736  VkBufferMemoryRequirementsInfo2KHR memReqInfo = { VK_STRUCTURE_TYPE_BUFFER_MEMORY_REQUIREMENTS_INFO_2_KHR };
    7737  memReqInfo.buffer = hBuffer;
    7738 
    7739  VkMemoryDedicatedRequirementsKHR memDedicatedReq = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
    7740 
    7741  VkMemoryRequirements2KHR memReq2 = { VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
    7742  memReq2.pNext = &memDedicatedReq;
    7743 
    7744  (*m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
    7745 
    7746  memReq = memReq2.memoryRequirements;
    7747  requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
    7748  prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
    7749  }
    7750  else
    7751  {
    7752  (*m_VulkanFunctions.vkGetBufferMemoryRequirements)(m_hDevice, hBuffer, &memReq);
    7753  requiresDedicatedAllocation = false;
    7754  prefersDedicatedAllocation = false;
    7755  }
    7756 }
    7757 
    7758 void VmaAllocator_T::GetImageMemoryRequirements(
    7759  VkImage hImage,
    7760  VkMemoryRequirements& memReq,
    7761  bool& requiresDedicatedAllocation,
    7762  bool& prefersDedicatedAllocation) const
    7763 {
    7764  if(m_UseKhrDedicatedAllocation)
    7765  {
    7766  VkImageMemoryRequirementsInfo2KHR memReqInfo = { VK_STRUCTURE_TYPE_IMAGE_MEMORY_REQUIREMENTS_INFO_2_KHR };
    7767  memReqInfo.image = hImage;
    7768 
    7769  VkMemoryDedicatedRequirementsKHR memDedicatedReq = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
    7770 
    7771  VkMemoryRequirements2KHR memReq2 = { VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
    7772  memReq2.pNext = &memDedicatedReq;
    7773 
    7774  (*m_VulkanFunctions.vkGetImageMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
    7775 
    7776  memReq = memReq2.memoryRequirements;
    7777  requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
    7778  prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
    7779  }
    7780  else
    7781  {
    7782  (*m_VulkanFunctions.vkGetImageMemoryRequirements)(m_hDevice, hImage, &memReq);
    7783  requiresDedicatedAllocation = false;
    7784  prefersDedicatedAllocation = false;
    7785  }
    7786 }
    7787 
    7788 VkResult VmaAllocator_T::AllocateMemory(
    7789  const VkMemoryRequirements& vkMemReq,
    7790  bool requiresDedicatedAllocation,
    7791  bool prefersDedicatedAllocation,
    7792  VkBuffer dedicatedBuffer,
    7793  VkImage dedicatedImage,
    7794  const VmaAllocationCreateInfo& createInfo,
    7795  VmaSuballocationType suballocType,
    7796  VmaAllocation* pAllocation)
    7797 {
    7798  if((createInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0 &&
    7799  (createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
    7800  {
    7801  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT together with VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT makes no sense.");
    7802  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7803  }
    7804  if((createInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0 &&
    7805  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0)
    7806  {
    7807  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_MAPPED_BIT together with VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT is invalid.");
    7808  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7809  }
    7810  if(requiresDedicatedAllocation)
    7811  {
    7812  if((createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
    7813  {
    7814  VMA_ASSERT(0 && "VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT specified while dedicated allocation is required.");
    7815  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7816  }
    7817  if(createInfo.pool != VK_NULL_HANDLE)
    7818  {
    7819  VMA_ASSERT(0 && "Pool specified while dedicated allocation is required.");
    7820  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7821  }
    7822  }
    7823  if((createInfo.pool != VK_NULL_HANDLE) &&
    7824  ((createInfo.flags & (VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT)) != 0))
    7825  {
    7826  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT when pool != null is invalid.");
    7827  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7828  }
    7829 
    7830  if(createInfo.pool != VK_NULL_HANDLE)
    7831  {
    7832  return createInfo.pool->m_BlockVector.Allocate(
    7833  createInfo.pool,
    7834  m_CurrentFrameIndex.load(),
    7835  vkMemReq,
    7836  createInfo,
    7837  suballocType,
    7838  pAllocation);
    7839  }
    7840  else
    7841  {
    7842  // Bit mask of memory Vulkan types acceptable for this allocation.
    7843  uint32_t memoryTypeBits = vkMemReq.memoryTypeBits;
    7844  uint32_t memTypeIndex = UINT32_MAX;
    7845  VkResult res = vmaFindMemoryTypeIndex(this, memoryTypeBits, &createInfo, &memTypeIndex);
    7846  if(res == VK_SUCCESS)
    7847  {
    7848  res = AllocateMemoryOfType(
    7849  vkMemReq,
    7850  requiresDedicatedAllocation || prefersDedicatedAllocation,
    7851  dedicatedBuffer,
    7852  dedicatedImage,
    7853  createInfo,
    7854  memTypeIndex,
    7855  suballocType,
    7856  pAllocation);
    7857  // Succeeded on first try.
    7858  if(res == VK_SUCCESS)
    7859  {
    7860  return res;
    7861  }
    7862  // Allocation from this memory type failed. Try other compatible memory types.
    7863  else
    7864  {
    7865  for(;;)
    7866  {
    7867  // Remove old memTypeIndex from list of possibilities.
    7868  memoryTypeBits &= ~(1u << memTypeIndex);
    7869  // Find alternative memTypeIndex.
    7870  res = vmaFindMemoryTypeIndex(this, memoryTypeBits, &createInfo, &memTypeIndex);
    7871  if(res == VK_SUCCESS)
    7872  {
    7873  res = AllocateMemoryOfType(
    7874  vkMemReq,
    7875  requiresDedicatedAllocation || prefersDedicatedAllocation,
    7876  dedicatedBuffer,
    7877  dedicatedImage,
    7878  createInfo,
    7879  memTypeIndex,
    7880  suballocType,
    7881  pAllocation);
    7882  // Allocation from this alternative memory type succeeded.
    7883  if(res == VK_SUCCESS)
    7884  {
    7885  return res;
    7886  }
    7887  // else: Allocation from this memory type failed. Try next one - next loop iteration.
    7888  }
    7889  // No other matching memory type index could be found.
    7890  else
    7891  {
    7892  // Not returning res, which is VK_ERROR_FEATURE_NOT_PRESENT, because we already failed to allocate once.
    7893  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7894  }
    7895  }
    7896  }
    7897  }
    7898  // Can't find any single memory type maching requirements. res is VK_ERROR_FEATURE_NOT_PRESENT.
    7899  else
    7900  return res;
    7901  }
    7902 }
    7903 
    7904 void VmaAllocator_T::FreeMemory(const VmaAllocation allocation)
    7905 {
    7906  VMA_ASSERT(allocation);
    7907 
    7908  if(allocation->CanBecomeLost() == false ||
    7909  allocation->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST)
    7910  {
    7911  switch(allocation->GetType())
    7912  {
    7913  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
    7914  {
    7915  VmaBlockVector* pBlockVector = VMA_NULL;
    7916  VmaPool hPool = allocation->GetPool();
    7917  if(hPool != VK_NULL_HANDLE)
    7918  {
    7919  pBlockVector = &hPool->m_BlockVector;
    7920  }
    7921  else
    7922  {
    7923  const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
    7924  pBlockVector = m_pBlockVectors[memTypeIndex];
    7925  }
    7926  pBlockVector->Free(allocation);
    7927  }
    7928  break;
    7929  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
    7930  FreeDedicatedMemory(allocation);
    7931  break;
    7932  default:
    7933  VMA_ASSERT(0);
    7934  }
    7935  }
    7936 
    7937  allocation->SetUserData(this, VMA_NULL);
    7938  vma_delete(this, allocation);
    7939 }
    7940 
    7941 void VmaAllocator_T::CalculateStats(VmaStats* pStats)
    7942 {
    7943  // Initialize.
    7944  InitStatInfo(pStats->total);
    7945  for(size_t i = 0; i < VK_MAX_MEMORY_TYPES; ++i)
    7946  InitStatInfo(pStats->memoryType[i]);
    7947  for(size_t i = 0; i < VK_MAX_MEMORY_HEAPS; ++i)
    7948  InitStatInfo(pStats->memoryHeap[i]);
    7949 
    7950  // Process default pools.
    7951  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
    7952  {
    7953  VmaBlockVector* const pBlockVector = m_pBlockVectors[memTypeIndex];
    7954  VMA_ASSERT(pBlockVector);
    7955  pBlockVector->AddStats(pStats);
    7956  }
    7957 
    7958  // Process custom pools.
    7959  {
    7960  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
    7961  for(size_t poolIndex = 0, poolCount = m_Pools.size(); poolIndex < poolCount; ++poolIndex)
    7962  {
    7963  m_Pools[poolIndex]->GetBlockVector().AddStats(pStats);
    7964  }
    7965  }
    7966 
    7967  // Process dedicated allocations.
    7968  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
    7969  {
    7970  const uint32_t memHeapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
    7971  VmaMutexLock dedicatedAllocationsLock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
    7972  AllocationVectorType* const pDedicatedAllocVector = m_pDedicatedAllocations[memTypeIndex];
    7973  VMA_ASSERT(pDedicatedAllocVector);
    7974  for(size_t allocIndex = 0, allocCount = pDedicatedAllocVector->size(); allocIndex < allocCount; ++allocIndex)
    7975  {
    7976  VmaStatInfo allocationStatInfo;
    7977  (*pDedicatedAllocVector)[allocIndex]->DedicatedAllocCalcStatsInfo(allocationStatInfo);
    7978  VmaAddStatInfo(pStats->total, allocationStatInfo);
    7979  VmaAddStatInfo(pStats->memoryType[memTypeIndex], allocationStatInfo);
    7980  VmaAddStatInfo(pStats->memoryHeap[memHeapIndex], allocationStatInfo);
    7981  }
    7982  }
    7983 
    7984  // Postprocess.
    7985  VmaPostprocessCalcStatInfo(pStats->total);
    7986  for(size_t i = 0; i < GetMemoryTypeCount(); ++i)
    7987  VmaPostprocessCalcStatInfo(pStats->memoryType[i]);
    7988  for(size_t i = 0; i < GetMemoryHeapCount(); ++i)
    7989  VmaPostprocessCalcStatInfo(pStats->memoryHeap[i]);
    7990 }
    7991 
    7992 static const uint32_t VMA_VENDOR_ID_AMD = 4098;
    7993 
    7994 VkResult VmaAllocator_T::Defragment(
    7995  VmaAllocation* pAllocations,
    7996  size_t allocationCount,
    7997  VkBool32* pAllocationsChanged,
    7998  const VmaDefragmentationInfo* pDefragmentationInfo,
    7999  VmaDefragmentationStats* pDefragmentationStats)
    8000 {
    8001  if(pAllocationsChanged != VMA_NULL)
    8002  {
    8003  memset(pAllocationsChanged, 0, sizeof(*pAllocationsChanged));
    8004  }
    8005  if(pDefragmentationStats != VMA_NULL)
    8006  {
    8007  memset(pDefragmentationStats, 0, sizeof(*pDefragmentationStats));
    8008  }
    8009 
    8010  const uint32_t currentFrameIndex = m_CurrentFrameIndex.load();
    8011 
    8012  VmaMutexLock poolsLock(m_PoolsMutex, m_UseMutex);
    8013 
    8014  const size_t poolCount = m_Pools.size();
    8015 
    8016  // Dispatch pAllocations among defragmentators. Create them in BlockVectors when necessary.
    8017  for(size_t allocIndex = 0; allocIndex < allocationCount; ++allocIndex)
    8018  {
    8019  VmaAllocation hAlloc = pAllocations[allocIndex];
    8020  VMA_ASSERT(hAlloc);
    8021  const uint32_t memTypeIndex = hAlloc->GetMemoryTypeIndex();
    8022  // DedicatedAlloc cannot be defragmented.
    8023  if((hAlloc->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK) &&
    8024  // Only HOST_VISIBLE memory types can be defragmented.
    8025  ((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0) &&
    8026  // Lost allocation cannot be defragmented.
    8027  (hAlloc->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST))
    8028  {
    8029  VmaBlockVector* pAllocBlockVector = VMA_NULL;
    8030 
    8031  const VmaPool hAllocPool = hAlloc->GetPool();
    8032  // This allocation belongs to custom pool.
    8033  if(hAllocPool != VK_NULL_HANDLE)
    8034  {
    8035  pAllocBlockVector = &hAllocPool->GetBlockVector();
    8036  }
    8037  // This allocation belongs to general pool.
    8038  else
    8039  {
    8040  pAllocBlockVector = m_pBlockVectors[memTypeIndex];
    8041  }
    8042 
    8043  VmaDefragmentator* const pDefragmentator = pAllocBlockVector->EnsureDefragmentator(this, currentFrameIndex);
    8044 
    8045  VkBool32* const pChanged = (pAllocationsChanged != VMA_NULL) ?
    8046  &pAllocationsChanged[allocIndex] : VMA_NULL;
    8047  pDefragmentator->AddAllocation(hAlloc, pChanged);
    8048  }
    8049  }
    8050 
    8051  VkResult result = VK_SUCCESS;
    8052 
    8053  // ======== Main processing.
    8054 
    8055  VkDeviceSize maxBytesToMove = SIZE_MAX;
    8056  uint32_t maxAllocationsToMove = UINT32_MAX;
    8057  if(pDefragmentationInfo != VMA_NULL)
    8058  {
    8059  maxBytesToMove = pDefragmentationInfo->maxBytesToMove;
    8060  maxAllocationsToMove = pDefragmentationInfo->maxAllocationsToMove;
    8061  }
    8062 
    8063  // Process standard memory.
    8064  for(uint32_t memTypeIndex = 0;
    8065  (memTypeIndex < GetMemoryTypeCount()) && (result == VK_SUCCESS);
    8066  ++memTypeIndex)
    8067  {
    8068  // Only HOST_VISIBLE memory types can be defragmented.
    8069  if((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
    8070  {
    8071  result = m_pBlockVectors[memTypeIndex]->Defragment(
    8072  pDefragmentationStats,
    8073  maxBytesToMove,
    8074  maxAllocationsToMove);
    8075  }
    8076  }
    8077 
    8078  // Process custom pools.
    8079  for(size_t poolIndex = 0; (poolIndex < poolCount) && (result == VK_SUCCESS); ++poolIndex)
    8080  {
    8081  result = m_Pools[poolIndex]->GetBlockVector().Defragment(
    8082  pDefragmentationStats,
    8083  maxBytesToMove,
    8084  maxAllocationsToMove);
    8085  }
    8086 
    8087  // ======== Destroy defragmentators.
    8088 
    8089  // Process custom pools.
    8090  for(size_t poolIndex = poolCount; poolIndex--; )
    8091  {
    8092  m_Pools[poolIndex]->GetBlockVector().DestroyDefragmentator();
    8093  }
    8094 
    8095  // Process standard memory.
    8096  for(uint32_t memTypeIndex = GetMemoryTypeCount(); memTypeIndex--; )
    8097  {
    8098  if((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
    8099  {
    8100  m_pBlockVectors[memTypeIndex]->DestroyDefragmentator();
    8101  }
    8102  }
    8103 
    8104  return result;
    8105 }
    8106 
    8107 void VmaAllocator_T::GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo)
    8108 {
    8109  if(hAllocation->CanBecomeLost())
    8110  {
    8111  /*
    8112  Warning: This is a carefully designed algorithm.
    8113  Do not modify unless you really know what you're doing :)
    8114  */
    8115  uint32_t localCurrFrameIndex = m_CurrentFrameIndex.load();
    8116  uint32_t localLastUseFrameIndex = hAllocation->GetLastUseFrameIndex();
    8117  for(;;)
    8118  {
    8119  if(localLastUseFrameIndex == VMA_FRAME_INDEX_LOST)
    8120  {
    8121  pAllocationInfo->memoryType = UINT32_MAX;
    8122  pAllocationInfo->deviceMemory = VK_NULL_HANDLE;
    8123  pAllocationInfo->offset = 0;
    8124  pAllocationInfo->size = hAllocation->GetSize();
    8125  pAllocationInfo->pMappedData = VMA_NULL;
    8126  pAllocationInfo->pUserData = hAllocation->GetUserData();
    8127  return;
    8128  }
    8129  else if(localLastUseFrameIndex == localCurrFrameIndex)
    8130  {
    8131  pAllocationInfo->memoryType = hAllocation->GetMemoryTypeIndex();
    8132  pAllocationInfo->deviceMemory = hAllocation->GetMemory();
    8133  pAllocationInfo->offset = hAllocation->GetOffset();
    8134  pAllocationInfo->size = hAllocation->GetSize();
    8135  pAllocationInfo->pMappedData = VMA_NULL;
    8136  pAllocationInfo->pUserData = hAllocation->GetUserData();
    8137  return;
    8138  }
    8139  else // Last use time earlier than current time.
    8140  {
    8141  if(hAllocation->CompareExchangeLastUseFrameIndex(localLastUseFrameIndex, localCurrFrameIndex))
    8142  {
    8143  localLastUseFrameIndex = localCurrFrameIndex;
    8144  }
    8145  }
    8146  }
    8147  }
    8148  else
    8149  {
    8150  pAllocationInfo->memoryType = hAllocation->GetMemoryTypeIndex();
    8151  pAllocationInfo->deviceMemory = hAllocation->GetMemory();
    8152  pAllocationInfo->offset = hAllocation->GetOffset();
    8153  pAllocationInfo->size = hAllocation->GetSize();
    8154  pAllocationInfo->pMappedData = hAllocation->GetMappedData();
    8155  pAllocationInfo->pUserData = hAllocation->GetUserData();
    8156  }
    8157 }
    8158 
    8159 bool VmaAllocator_T::TouchAllocation(VmaAllocation hAllocation)
    8160 {
    8161  // This is a stripped-down version of VmaAllocator_T::GetAllocationInfo.
    8162  if(hAllocation->CanBecomeLost())
    8163  {
    8164  uint32_t localCurrFrameIndex = m_CurrentFrameIndex.load();
    8165  uint32_t localLastUseFrameIndex = hAllocation->GetLastUseFrameIndex();
    8166  for(;;)
    8167  {
    8168  if(localLastUseFrameIndex == VMA_FRAME_INDEX_LOST)
    8169  {
    8170  return false;
    8171  }
    8172  else if(localLastUseFrameIndex == localCurrFrameIndex)
    8173  {
    8174  return true;
    8175  }
    8176  else // Last use time earlier than current time.
    8177  {
    8178  if(hAllocation->CompareExchangeLastUseFrameIndex(localLastUseFrameIndex, localCurrFrameIndex))
    8179  {
    8180  localLastUseFrameIndex = localCurrFrameIndex;
    8181  }
    8182  }
    8183  }
    8184  }
    8185  else
    8186  {
    8187  return true;
    8188  }
    8189 }
    8190 
    8191 VkResult VmaAllocator_T::CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool)
    8192 {
    8193  VMA_DEBUG_LOG(" CreatePool: MemoryTypeIndex=%u", pCreateInfo->memoryTypeIndex);
    8194 
    8195  VmaPoolCreateInfo newCreateInfo = *pCreateInfo;
    8196 
    8197  if(newCreateInfo.maxBlockCount == 0)
    8198  {
    8199  newCreateInfo.maxBlockCount = SIZE_MAX;
    8200  }
    8201  if(newCreateInfo.blockSize == 0)
    8202  {
    8203  newCreateInfo.blockSize = CalcPreferredBlockSize(newCreateInfo.memoryTypeIndex);
    8204  }
    8205 
    8206  *pPool = vma_new(this, VmaPool_T)(this, newCreateInfo);
    8207 
    8208  VkResult res = (*pPool)->m_BlockVector.CreateMinBlocks();
    8209  if(res != VK_SUCCESS)
    8210  {
    8211  vma_delete(this, *pPool);
    8212  *pPool = VMA_NULL;
    8213  return res;
    8214  }
    8215 
    8216  // Add to m_Pools.
    8217  {
    8218  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
    8219  VmaVectorInsertSorted<VmaPointerLess>(m_Pools, *pPool);
    8220  }
    8221 
    8222  return VK_SUCCESS;
    8223 }
    8224 
    8225 void VmaAllocator_T::DestroyPool(VmaPool pool)
    8226 {
    8227  // Remove from m_Pools.
    8228  {
    8229  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
    8230  bool success = VmaVectorRemoveSorted<VmaPointerLess>(m_Pools, pool);
    8231  VMA_ASSERT(success && "Pool not found in Allocator.");
    8232  }
    8233 
    8234  vma_delete(this, pool);
    8235 }
    8236 
    8237 void VmaAllocator_T::GetPoolStats(VmaPool pool, VmaPoolStats* pPoolStats)
    8238 {
    8239  pool->m_BlockVector.GetPoolStats(pPoolStats);
    8240 }
    8241 
    8242 void VmaAllocator_T::SetCurrentFrameIndex(uint32_t frameIndex)
    8243 {
    8244  m_CurrentFrameIndex.store(frameIndex);
    8245 }
    8246 
    8247 void VmaAllocator_T::MakePoolAllocationsLost(
    8248  VmaPool hPool,
    8249  size_t* pLostAllocationCount)
    8250 {
    8251  hPool->m_BlockVector.MakePoolAllocationsLost(
    8252  m_CurrentFrameIndex.load(),
    8253  pLostAllocationCount);
    8254 }
    8255 
    8256 void VmaAllocator_T::CreateLostAllocation(VmaAllocation* pAllocation)
    8257 {
    8258  *pAllocation = vma_new(this, VmaAllocation_T)(VMA_FRAME_INDEX_LOST, false);
    8259  (*pAllocation)->InitLost();
    8260 }
    8261 
    8262 VkResult VmaAllocator_T::AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory)
    8263 {
    8264  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(pAllocateInfo->memoryTypeIndex);
    8265 
    8266  VkResult res;
    8267  if(m_HeapSizeLimit[heapIndex] != VK_WHOLE_SIZE)
    8268  {
    8269  VmaMutexLock lock(m_HeapSizeLimitMutex, m_UseMutex);
    8270  if(m_HeapSizeLimit[heapIndex] >= pAllocateInfo->allocationSize)
    8271  {
    8272  res = (*m_VulkanFunctions.vkAllocateMemory)(m_hDevice, pAllocateInfo, GetAllocationCallbacks(), pMemory);
    8273  if(res == VK_SUCCESS)
    8274  {
    8275  m_HeapSizeLimit[heapIndex] -= pAllocateInfo->allocationSize;
    8276  }
    8277  }
    8278  else
    8279  {
    8280  res = VK_ERROR_OUT_OF_DEVICE_MEMORY;
    8281  }
    8282  }
    8283  else
    8284  {
    8285  res = (*m_VulkanFunctions.vkAllocateMemory)(m_hDevice, pAllocateInfo, GetAllocationCallbacks(), pMemory);
    8286  }
    8287 
    8288  if(res == VK_SUCCESS && m_DeviceMemoryCallbacks.pfnAllocate != VMA_NULL)
    8289  {
    8290  (*m_DeviceMemoryCallbacks.pfnAllocate)(this, pAllocateInfo->memoryTypeIndex, *pMemory, pAllocateInfo->allocationSize);
    8291  }
    8292 
    8293  return res;
    8294 }
    8295 
    8296 void VmaAllocator_T::FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory)
    8297 {
    8298  if(m_DeviceMemoryCallbacks.pfnFree != VMA_NULL)
    8299  {
    8300  (*m_DeviceMemoryCallbacks.pfnFree)(this, memoryType, hMemory, size);
    8301  }
    8302 
    8303  (*m_VulkanFunctions.vkFreeMemory)(m_hDevice, hMemory, GetAllocationCallbacks());
    8304 
    8305  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memoryType);
    8306  if(m_HeapSizeLimit[heapIndex] != VK_WHOLE_SIZE)
    8307  {
    8308  VmaMutexLock lock(m_HeapSizeLimitMutex, m_UseMutex);
    8309  m_HeapSizeLimit[heapIndex] += size;
    8310  }
    8311 }
    8312 
    8313 VkResult VmaAllocator_T::Map(VmaAllocation hAllocation, void** ppData)
    8314 {
    8315  if(hAllocation->CanBecomeLost())
    8316  {
    8317  return VK_ERROR_MEMORY_MAP_FAILED;
    8318  }
    8319 
    8320  switch(hAllocation->GetType())
    8321  {
    8322  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
    8323  {
    8324  VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
    8325  char *pBytes = VMA_NULL;
    8326  VkResult res = pBlock->Map(this, 1, (void**)&pBytes);
    8327  if(res == VK_SUCCESS)
    8328  {
    8329  *ppData = pBytes + (ptrdiff_t)hAllocation->GetOffset();
    8330  hAllocation->BlockAllocMap();
    8331  }
    8332  return res;
    8333  }
    8334  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
    8335  return hAllocation->DedicatedAllocMap(this, ppData);
    8336  default:
    8337  VMA_ASSERT(0);
    8338  return VK_ERROR_MEMORY_MAP_FAILED;
    8339  }
    8340 }
    8341 
    8342 void VmaAllocator_T::Unmap(VmaAllocation hAllocation)
    8343 {
    8344  switch(hAllocation->GetType())
    8345  {
    8346  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
    8347  {
    8348  VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
    8349  hAllocation->BlockAllocUnmap();
    8350  pBlock->Unmap(this, 1);
    8351  }
    8352  break;
    8353  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
    8354  hAllocation->DedicatedAllocUnmap(this);
    8355  break;
    8356  default:
    8357  VMA_ASSERT(0);
    8358  }
    8359 }
    8360 
    8361 VkResult VmaAllocator_T::BindBufferMemory(VmaAllocation hAllocation, VkBuffer hBuffer)
    8362 {
    8363  VkResult res = VK_SUCCESS;
    8364  switch(hAllocation->GetType())
    8365  {
    8366  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
    8367  res = GetVulkanFunctions().vkBindBufferMemory(
    8368  m_hDevice,
    8369  hBuffer,
    8370  hAllocation->GetMemory(),
    8371  0); //memoryOffset
    8372  break;
    8373  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
    8374  {
    8375  VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
    8376  VMA_ASSERT(pBlock && "Binding buffer to allocation that doesn't belong to any block. Is the allocation lost?");
    8377  res = pBlock->BindBufferMemory(this, hAllocation, hBuffer);
    8378  break;
    8379  }
    8380  default:
    8381  VMA_ASSERT(0);
    8382  }
    8383  return res;
    8384 }
    8385 
    8386 VkResult VmaAllocator_T::BindImageMemory(VmaAllocation hAllocation, VkImage hImage)
    8387 {
    8388  VkResult res = VK_SUCCESS;
    8389  switch(hAllocation->GetType())
    8390  {
    8391  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
    8392  res = GetVulkanFunctions().vkBindImageMemory(
    8393  m_hDevice,
    8394  hImage,
    8395  hAllocation->GetMemory(),
    8396  0); //memoryOffset
    8397  break;
    8398  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
    8399  {
    8400  VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
    8401  VMA_ASSERT(pBlock && "Binding image to allocation that doesn't belong to any block. Is the allocation lost?");
    8402  res = pBlock->BindImageMemory(this, hAllocation, hImage);
    8403  break;
    8404  }
    8405  default:
    8406  VMA_ASSERT(0);
    8407  }
    8408  return res;
    8409 }
    8410 
    8411 void VmaAllocator_T::FreeDedicatedMemory(VmaAllocation allocation)
    8412 {
    8413  VMA_ASSERT(allocation && allocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
    8414 
    8415  const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
    8416  {
    8417  VmaMutexLock lock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
    8418  AllocationVectorType* const pDedicatedAllocations = m_pDedicatedAllocations[memTypeIndex];
    8419  VMA_ASSERT(pDedicatedAllocations);
    8420  bool success = VmaVectorRemoveSorted<VmaPointerLess>(*pDedicatedAllocations, allocation);
    8421  VMA_ASSERT(success);
    8422  }
    8423 
    8424  VkDeviceMemory hMemory = allocation->GetMemory();
    8425 
    8426  if(allocation->GetMappedData() != VMA_NULL)
    8427  {
    8428  (*m_VulkanFunctions.vkUnmapMemory)(m_hDevice, hMemory);
    8429  }
    8430 
    8431  FreeVulkanMemory(memTypeIndex, allocation->GetSize(), hMemory);
    8432 
    8433  VMA_DEBUG_LOG(" Freed DedicatedMemory MemoryTypeIndex=%u", memTypeIndex);
    8434 }
    8435 
    8436 #if VMA_STATS_STRING_ENABLED
    8437 
    8438 void VmaAllocator_T::PrintDetailedMap(VmaJsonWriter& json)
    8439 {
    8440  bool dedicatedAllocationsStarted = false;
    8441  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
    8442  {
    8443  VmaMutexLock dedicatedAllocationsLock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
    8444  AllocationVectorType* const pDedicatedAllocVector = m_pDedicatedAllocations[memTypeIndex];
    8445  VMA_ASSERT(pDedicatedAllocVector);
    8446  if(pDedicatedAllocVector->empty() == false)
    8447  {
    8448  if(dedicatedAllocationsStarted == false)
    8449  {
    8450  dedicatedAllocationsStarted = true;
    8451  json.WriteString("DedicatedAllocations");
    8452  json.BeginObject();
    8453  }
    8454 
    8455  json.BeginString("Type ");
    8456  json.ContinueString(memTypeIndex);
    8457  json.EndString();
    8458 
    8459  json.BeginArray();
    8460 
    8461  for(size_t i = 0; i < pDedicatedAllocVector->size(); ++i)
    8462  {
    8463  const VmaAllocation hAlloc = (*pDedicatedAllocVector)[i];
    8464  json.BeginObject(true);
    8465 
    8466  json.WriteString("Type");
    8467  json.WriteString(VMA_SUBALLOCATION_TYPE_NAMES[hAlloc->GetSuballocationType()]);
    8468 
    8469  json.WriteString("Size");
    8470  json.WriteNumber(hAlloc->GetSize());
    8471 
    8472  const void* pUserData = hAlloc->GetUserData();
    8473  if(pUserData != VMA_NULL)
    8474  {
    8475  json.WriteString("UserData");
    8476  if(hAlloc->IsUserDataString())
    8477  {
    8478  json.WriteString((const char*)pUserData);
    8479  }
    8480  else
    8481  {
    8482  json.BeginString();
    8483  json.ContinueString_Pointer(pUserData);
    8484  json.EndString();
    8485  }
    8486  }
    8487 
    8488  json.EndObject();
    8489  }
    8490 
    8491  json.EndArray();
    8492  }
    8493  }
    8494  if(dedicatedAllocationsStarted)
    8495  {
    8496  json.EndObject();
    8497  }
    8498 
    8499  {
    8500  bool allocationsStarted = false;
    8501  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
    8502  {
    8503  if(m_pBlockVectors[memTypeIndex]->IsEmpty() == false)
    8504  {
    8505  if(allocationsStarted == false)
    8506  {
    8507  allocationsStarted = true;
    8508  json.WriteString("DefaultPools");
    8509  json.BeginObject();
    8510  }
    8511 
    8512  json.BeginString("Type ");
    8513  json.ContinueString(memTypeIndex);
    8514  json.EndString();
    8515 
    8516  m_pBlockVectors[memTypeIndex]->PrintDetailedMap(json);
    8517  }
    8518  }
    8519  if(allocationsStarted)
    8520  {
    8521  json.EndObject();
    8522  }
    8523  }
    8524 
    8525  {
    8526  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
    8527  const size_t poolCount = m_Pools.size();
    8528  if(poolCount > 0)
    8529  {
    8530  json.WriteString("Pools");
    8531  json.BeginArray();
    8532  for(size_t poolIndex = 0; poolIndex < poolCount; ++poolIndex)
    8533  {
    8534  m_Pools[poolIndex]->m_BlockVector.PrintDetailedMap(json);
    8535  }
    8536  json.EndArray();
    8537  }
    8538  }
    8539 }
    8540 
    8541 #endif // #if VMA_STATS_STRING_ENABLED
    8542 
    8543 static VkResult AllocateMemoryForImage(
    8544  VmaAllocator allocator,
    8545  VkImage image,
    8546  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    8547  VmaSuballocationType suballocType,
    8548  VmaAllocation* pAllocation)
    8549 {
    8550  VMA_ASSERT(allocator && (image != VK_NULL_HANDLE) && pAllocationCreateInfo && pAllocation);
    8551 
    8552  VkMemoryRequirements vkMemReq = {};
    8553  bool requiresDedicatedAllocation = false;
    8554  bool prefersDedicatedAllocation = false;
    8555  allocator->GetImageMemoryRequirements(image, vkMemReq,
    8556  requiresDedicatedAllocation, prefersDedicatedAllocation);
    8557 
    8558  return allocator->AllocateMemory(
    8559  vkMemReq,
    8560  requiresDedicatedAllocation,
    8561  prefersDedicatedAllocation,
    8562  VK_NULL_HANDLE, // dedicatedBuffer
    8563  image, // dedicatedImage
    8564  *pAllocationCreateInfo,
    8565  suballocType,
    8566  pAllocation);
    8567 }
    8568 
    8570 // Public interface
    8571 
    8572 VkResult vmaCreateAllocator(
    8573  const VmaAllocatorCreateInfo* pCreateInfo,
    8574  VmaAllocator* pAllocator)
    8575 {
    8576  VMA_ASSERT(pCreateInfo && pAllocator);
    8577  VMA_DEBUG_LOG("vmaCreateAllocator");
    8578  *pAllocator = vma_new(pCreateInfo->pAllocationCallbacks, VmaAllocator_T)(pCreateInfo);
    8579  return VK_SUCCESS;
    8580 }
    8581 
    8582 void vmaDestroyAllocator(
    8583  VmaAllocator allocator)
    8584 {
    8585  if(allocator != VK_NULL_HANDLE)
    8586  {
    8587  VMA_DEBUG_LOG("vmaDestroyAllocator");
    8588  VkAllocationCallbacks allocationCallbacks = allocator->m_AllocationCallbacks;
    8589  vma_delete(&allocationCallbacks, allocator);
    8590  }
    8591 }
    8592 
    8593 void vmaGetPhysicalDeviceProperties(
    8594  VmaAllocator allocator,
    8595  const VkPhysicalDeviceProperties **ppPhysicalDeviceProperties)
    8596 {
    8597  VMA_ASSERT(allocator && ppPhysicalDeviceProperties);
    8598  *ppPhysicalDeviceProperties = &allocator->m_PhysicalDeviceProperties;
    8599 }
    8600 
    8601 void vmaGetMemoryProperties(
    8602  VmaAllocator allocator,
    8603  const VkPhysicalDeviceMemoryProperties** ppPhysicalDeviceMemoryProperties)
    8604 {
    8605  VMA_ASSERT(allocator && ppPhysicalDeviceMemoryProperties);
    8606  *ppPhysicalDeviceMemoryProperties = &allocator->m_MemProps;
    8607 }
    8608 
    8609 void vmaGetMemoryTypeProperties(
    8610  VmaAllocator allocator,
    8611  uint32_t memoryTypeIndex,
    8612  VkMemoryPropertyFlags* pFlags)
    8613 {
    8614  VMA_ASSERT(allocator && pFlags);
    8615  VMA_ASSERT(memoryTypeIndex < allocator->GetMemoryTypeCount());
    8616  *pFlags = allocator->m_MemProps.memoryTypes[memoryTypeIndex].propertyFlags;
    8617 }
    8618 
    8619 void vmaSetCurrentFrameIndex(
    8620  VmaAllocator allocator,
    8621  uint32_t frameIndex)
    8622 {
    8623  VMA_ASSERT(allocator);
    8624  VMA_ASSERT(frameIndex != VMA_FRAME_INDEX_LOST);
    8625 
    8626  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8627 
    8628  allocator->SetCurrentFrameIndex(frameIndex);
    8629 }
    8630 
    8631 void vmaCalculateStats(
    8632  VmaAllocator allocator,
    8633  VmaStats* pStats)
    8634 {
    8635  VMA_ASSERT(allocator && pStats);
    8636  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8637  allocator->CalculateStats(pStats);
    8638 }
    8639 
    8640 #if VMA_STATS_STRING_ENABLED
    8641 
    8642 void vmaBuildStatsString(
    8643  VmaAllocator allocator,
    8644  char** ppStatsString,
    8645  VkBool32 detailedMap)
    8646 {
    8647  VMA_ASSERT(allocator && ppStatsString);
    8648  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8649 
    8650  VmaStringBuilder sb(allocator);
    8651  {
    8652  VmaJsonWriter json(allocator->GetAllocationCallbacks(), sb);
    8653  json.BeginObject();
    8654 
    8655  VmaStats stats;
    8656  allocator->CalculateStats(&stats);
    8657 
    8658  json.WriteString("Total");
    8659  VmaPrintStatInfo(json, stats.total);
    8660 
    8661  for(uint32_t heapIndex = 0; heapIndex < allocator->GetMemoryHeapCount(); ++heapIndex)
    8662  {
    8663  json.BeginString("Heap ");
    8664  json.ContinueString(heapIndex);
    8665  json.EndString();
    8666  json.BeginObject();
    8667 
    8668  json.WriteString("Size");
    8669  json.WriteNumber(allocator->m_MemProps.memoryHeaps[heapIndex].size);
    8670 
    8671  json.WriteString("Flags");
    8672  json.BeginArray(true);
    8673  if((allocator->m_MemProps.memoryHeaps[heapIndex].flags & VK_MEMORY_HEAP_DEVICE_LOCAL_BIT) != 0)
    8674  {
    8675  json.WriteString("DEVICE_LOCAL");
    8676  }
    8677  json.EndArray();
    8678 
    8679  if(stats.memoryHeap[heapIndex].blockCount > 0)
    8680  {
    8681  json.WriteString("Stats");
    8682  VmaPrintStatInfo(json, stats.memoryHeap[heapIndex]);
    8683  }
    8684 
    8685  for(uint32_t typeIndex = 0; typeIndex < allocator->GetMemoryTypeCount(); ++typeIndex)
    8686  {
    8687  if(allocator->MemoryTypeIndexToHeapIndex(typeIndex) == heapIndex)
    8688  {
    8689  json.BeginString("Type ");
    8690  json.ContinueString(typeIndex);
    8691  json.EndString();
    8692 
    8693  json.BeginObject();
    8694 
    8695  json.WriteString("Flags");
    8696  json.BeginArray(true);
    8697  VkMemoryPropertyFlags flags = allocator->m_MemProps.memoryTypes[typeIndex].propertyFlags;
    8698  if((flags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT) != 0)
    8699  {
    8700  json.WriteString("DEVICE_LOCAL");
    8701  }
    8702  if((flags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
    8703  {
    8704  json.WriteString("HOST_VISIBLE");
    8705  }
    8706  if((flags & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT) != 0)
    8707  {
    8708  json.WriteString("HOST_COHERENT");
    8709  }
    8710  if((flags & VK_MEMORY_PROPERTY_HOST_CACHED_BIT) != 0)
    8711  {
    8712  json.WriteString("HOST_CACHED");
    8713  }
    8714  if((flags & VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT) != 0)
    8715  {
    8716  json.WriteString("LAZILY_ALLOCATED");
    8717  }
    8718  json.EndArray();
    8719 
    8720  if(stats.memoryType[typeIndex].blockCount > 0)
    8721  {
    8722  json.WriteString("Stats");
    8723  VmaPrintStatInfo(json, stats.memoryType[typeIndex]);
    8724  }
    8725 
    8726  json.EndObject();
    8727  }
    8728  }
    8729 
    8730  json.EndObject();
    8731  }
    8732  if(detailedMap == VK_TRUE)
    8733  {
    8734  allocator->PrintDetailedMap(json);
    8735  }
    8736 
    8737  json.EndObject();
    8738  }
    8739 
    8740  const size_t len = sb.GetLength();
    8741  char* const pChars = vma_new_array(allocator, char, len + 1);
    8742  if(len > 0)
    8743  {
    8744  memcpy(pChars, sb.GetData(), len);
    8745  }
    8746  pChars[len] = '\0';
    8747  *ppStatsString = pChars;
    8748 }
    8749 
    8750 void vmaFreeStatsString(
    8751  VmaAllocator allocator,
    8752  char* pStatsString)
    8753 {
    8754  if(pStatsString != VMA_NULL)
    8755  {
    8756  VMA_ASSERT(allocator);
    8757  size_t len = strlen(pStatsString);
    8758  vma_delete_array(allocator, pStatsString, len + 1);
    8759  }
    8760 }
    8761 
    8762 #endif // #if VMA_STATS_STRING_ENABLED
    8763 
    8764 /*
    8765 This function is not protected by any mutex because it just reads immutable data.
    8766 */
    8767 VkResult vmaFindMemoryTypeIndex(
    8768  VmaAllocator allocator,
    8769  uint32_t memoryTypeBits,
    8770  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    8771  uint32_t* pMemoryTypeIndex)
    8772 {
    8773  VMA_ASSERT(allocator != VK_NULL_HANDLE);
    8774  VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
    8775  VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
    8776 
    8777  if(pAllocationCreateInfo->memoryTypeBits != 0)
    8778  {
    8779  memoryTypeBits &= pAllocationCreateInfo->memoryTypeBits;
    8780  }
    8781 
    8782  uint32_t requiredFlags = pAllocationCreateInfo->requiredFlags;
    8783  uint32_t preferredFlags = pAllocationCreateInfo->preferredFlags;
    8784 
    8785  // Convert usage to requiredFlags and preferredFlags.
    8786  switch(pAllocationCreateInfo->usage)
    8787  {
    8788  case VMA_MEMORY_USAGE_UNKNOWN:
    8789  break;
    8790  case VMA_MEMORY_USAGE_GPU_ONLY:
    8791  preferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
    8792  break;
    8793  case VMA_MEMORY_USAGE_CPU_ONLY:
    8794  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
    8795  break;
    8796  case VMA_MEMORY_USAGE_CPU_TO_GPU:
    8797  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
    8798  preferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
    8799  break;
    8800  case VMA_MEMORY_USAGE_GPU_TO_CPU:
    8801  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
    8802  preferredFlags |= VK_MEMORY_PROPERTY_HOST_COHERENT_BIT | VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
    8803  break;
    8804  default:
    8805  break;
    8806  }
    8807 
    8808  *pMemoryTypeIndex = UINT32_MAX;
    8809  uint32_t minCost = UINT32_MAX;
    8810  for(uint32_t memTypeIndex = 0, memTypeBit = 1;
    8811  memTypeIndex < allocator->GetMemoryTypeCount();
    8812  ++memTypeIndex, memTypeBit <<= 1)
    8813  {
    8814  // This memory type is acceptable according to memoryTypeBits bitmask.
    8815  if((memTypeBit & memoryTypeBits) != 0)
    8816  {
    8817  const VkMemoryPropertyFlags currFlags =
    8818  allocator->m_MemProps.memoryTypes[memTypeIndex].propertyFlags;
    8819  // This memory type contains requiredFlags.
    8820  if((requiredFlags & ~currFlags) == 0)
    8821  {
    8822  // Calculate cost as number of bits from preferredFlags not present in this memory type.
    8823  uint32_t currCost = VmaCountBitsSet(preferredFlags & ~currFlags);
    8824  // Remember memory type with lowest cost.
    8825  if(currCost < minCost)
    8826  {
    8827  *pMemoryTypeIndex = memTypeIndex;
    8828  if(currCost == 0)
    8829  {
    8830  return VK_SUCCESS;
    8831  }
    8832  minCost = currCost;
    8833  }
    8834  }
    8835  }
    8836  }
    8837  return (*pMemoryTypeIndex != UINT32_MAX) ? VK_SUCCESS : VK_ERROR_FEATURE_NOT_PRESENT;
    8838 }
    8839 
    8840 VkResult vmaFindMemoryTypeIndexForBufferInfo(
    8841  VmaAllocator allocator,
    8842  const VkBufferCreateInfo* pBufferCreateInfo,
    8843  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    8844  uint32_t* pMemoryTypeIndex)
    8845 {
    8846  VMA_ASSERT(allocator != VK_NULL_HANDLE);
    8847  VMA_ASSERT(pBufferCreateInfo != VMA_NULL);
    8848  VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
    8849  VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
    8850 
    8851  const VkDevice hDev = allocator->m_hDevice;
    8852  VkBuffer hBuffer = VK_NULL_HANDLE;
    8853  VkResult res = allocator->GetVulkanFunctions().vkCreateBuffer(
    8854  hDev, pBufferCreateInfo, allocator->GetAllocationCallbacks(), &hBuffer);
    8855  if(res == VK_SUCCESS)
    8856  {
    8857  VkMemoryRequirements memReq = {};
    8858  allocator->GetVulkanFunctions().vkGetBufferMemoryRequirements(
    8859  hDev, hBuffer, &memReq);
    8860 
    8861  res = vmaFindMemoryTypeIndex(
    8862  allocator,
    8863  memReq.memoryTypeBits,
    8864  pAllocationCreateInfo,
    8865  pMemoryTypeIndex);
    8866 
    8867  allocator->GetVulkanFunctions().vkDestroyBuffer(
    8868  hDev, hBuffer, allocator->GetAllocationCallbacks());
    8869  }
    8870  return res;
    8871 }
    8872 
    8873 VkResult vmaFindMemoryTypeIndexForImageInfo(
    8874  VmaAllocator allocator,
    8875  const VkImageCreateInfo* pImageCreateInfo,
    8876  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    8877  uint32_t* pMemoryTypeIndex)
    8878 {
    8879  VMA_ASSERT(allocator != VK_NULL_HANDLE);
    8880  VMA_ASSERT(pImageCreateInfo != VMA_NULL);
    8881  VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
    8882  VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
    8883 
    8884  const VkDevice hDev = allocator->m_hDevice;
    8885  VkImage hImage = VK_NULL_HANDLE;
    8886  VkResult res = allocator->GetVulkanFunctions().vkCreateImage(
    8887  hDev, pImageCreateInfo, allocator->GetAllocationCallbacks(), &hImage);
    8888  if(res == VK_SUCCESS)
    8889  {
    8890  VkMemoryRequirements memReq = {};
    8891  allocator->GetVulkanFunctions().vkGetImageMemoryRequirements(
    8892  hDev, hImage, &memReq);
    8893 
    8894  res = vmaFindMemoryTypeIndex(
    8895  allocator,
    8896  memReq.memoryTypeBits,
    8897  pAllocationCreateInfo,
    8898  pMemoryTypeIndex);
    8899 
    8900  allocator->GetVulkanFunctions().vkDestroyImage(
    8901  hDev, hImage, allocator->GetAllocationCallbacks());
    8902  }
    8903  return res;
    8904 }
    8905 
    8906 VkResult vmaCreatePool(
    8907  VmaAllocator allocator,
    8908  const VmaPoolCreateInfo* pCreateInfo,
    8909  VmaPool* pPool)
    8910 {
    8911  VMA_ASSERT(allocator && pCreateInfo && pPool);
    8912 
    8913  VMA_DEBUG_LOG("vmaCreatePool");
    8914 
    8915  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8916 
    8917  return allocator->CreatePool(pCreateInfo, pPool);
    8918 }
    8919 
    8920 void vmaDestroyPool(
    8921  VmaAllocator allocator,
    8922  VmaPool pool)
    8923 {
    8924  VMA_ASSERT(allocator);
    8925 
    8926  if(pool == VK_NULL_HANDLE)
    8927  {
    8928  return;
    8929  }
    8930 
    8931  VMA_DEBUG_LOG("vmaDestroyPool");
    8932 
    8933  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8934 
    8935  allocator->DestroyPool(pool);
    8936 }
    8937 
    8938 void vmaGetPoolStats(
    8939  VmaAllocator allocator,
    8940  VmaPool pool,
    8941  VmaPoolStats* pPoolStats)
    8942 {
    8943  VMA_ASSERT(allocator && pool && pPoolStats);
    8944 
    8945  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8946 
    8947  allocator->GetPoolStats(pool, pPoolStats);
    8948 }
    8949 
    8950 void vmaMakePoolAllocationsLost(
    8951  VmaAllocator allocator,
    8952  VmaPool pool,
    8953  size_t* pLostAllocationCount)
    8954 {
    8955  VMA_ASSERT(allocator && pool);
    8956 
    8957  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8958 
    8959  allocator->MakePoolAllocationsLost(pool, pLostAllocationCount);
    8960 }
    8961 
    8962 VkResult vmaAllocateMemory(
    8963  VmaAllocator allocator,
    8964  const VkMemoryRequirements* pVkMemoryRequirements,
    8965  const VmaAllocationCreateInfo* pCreateInfo,
    8966  VmaAllocation* pAllocation,
    8967  VmaAllocationInfo* pAllocationInfo)
    8968 {
    8969  VMA_ASSERT(allocator && pVkMemoryRequirements && pCreateInfo && pAllocation);
    8970 
    8971  VMA_DEBUG_LOG("vmaAllocateMemory");
    8972 
    8973  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8974 
    8975  VkResult result = allocator->AllocateMemory(
    8976  *pVkMemoryRequirements,
    8977  false, // requiresDedicatedAllocation
    8978  false, // prefersDedicatedAllocation
    8979  VK_NULL_HANDLE, // dedicatedBuffer
    8980  VK_NULL_HANDLE, // dedicatedImage
    8981  *pCreateInfo,
    8982  VMA_SUBALLOCATION_TYPE_UNKNOWN,
    8983  pAllocation);
    8984 
    8985  if(pAllocationInfo && result == VK_SUCCESS)
    8986  {
    8987  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
    8988  }
    8989 
    8990  return result;
    8991 }
    8992 
    8993 VkResult vmaAllocateMemoryForBuffer(
    8994  VmaAllocator allocator,
    8995  VkBuffer buffer,
    8996  const VmaAllocationCreateInfo* pCreateInfo,
    8997  VmaAllocation* pAllocation,
    8998  VmaAllocationInfo* pAllocationInfo)
    8999 {
    9000  VMA_ASSERT(allocator && buffer != VK_NULL_HANDLE && pCreateInfo && pAllocation);
    9001 
    9002  VMA_DEBUG_LOG("vmaAllocateMemoryForBuffer");
    9003 
    9004  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9005 
    9006  VkMemoryRequirements vkMemReq = {};
    9007  bool requiresDedicatedAllocation = false;
    9008  bool prefersDedicatedAllocation = false;
    9009  allocator->GetBufferMemoryRequirements(buffer, vkMemReq,
    9010  requiresDedicatedAllocation,
    9011  prefersDedicatedAllocation);
    9012 
    9013  VkResult result = allocator->AllocateMemory(
    9014  vkMemReq,
    9015  requiresDedicatedAllocation,
    9016  prefersDedicatedAllocation,
    9017  buffer, // dedicatedBuffer
    9018  VK_NULL_HANDLE, // dedicatedImage
    9019  *pCreateInfo,
    9020  VMA_SUBALLOCATION_TYPE_BUFFER,
    9021  pAllocation);
    9022 
    9023  if(pAllocationInfo && result == VK_SUCCESS)
    9024  {
    9025  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
    9026  }
    9027 
    9028  return result;
    9029 }
    9030 
    9031 VkResult vmaAllocateMemoryForImage(
    9032  VmaAllocator allocator,
    9033  VkImage image,
    9034  const VmaAllocationCreateInfo* pCreateInfo,
    9035  VmaAllocation* pAllocation,
    9036  VmaAllocationInfo* pAllocationInfo)
    9037 {
    9038  VMA_ASSERT(allocator && image != VK_NULL_HANDLE && pCreateInfo && pAllocation);
    9039 
    9040  VMA_DEBUG_LOG("vmaAllocateMemoryForImage");
    9041 
    9042  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9043 
    9044  VkResult result = AllocateMemoryForImage(
    9045  allocator,
    9046  image,
    9047  pCreateInfo,
    9048  VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN,
    9049  pAllocation);
    9050 
    9051  if(pAllocationInfo && result == VK_SUCCESS)
    9052  {
    9053  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
    9054  }
    9055 
    9056  return result;
    9057 }
    9058 
    9059 void vmaFreeMemory(
    9060  VmaAllocator allocator,
    9061  VmaAllocation allocation)
    9062 {
    9063  VMA_ASSERT(allocator && allocation);
    9064 
    9065  VMA_DEBUG_LOG("vmaFreeMemory");
    9066 
    9067  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9068 
    9069  allocator->FreeMemory(allocation);
    9070 }
    9071 
    9072 void vmaGetAllocationInfo(
    9073  VmaAllocator allocator,
    9074  VmaAllocation allocation,
    9075  VmaAllocationInfo* pAllocationInfo)
    9076 {
    9077  VMA_ASSERT(allocator && allocation && pAllocationInfo);
    9078 
    9079  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9080 
    9081  allocator->GetAllocationInfo(allocation, pAllocationInfo);
    9082 }
    9083 
    9084 VkBool32 vmaTouchAllocation(
    9085  VmaAllocator allocator,
    9086  VmaAllocation allocation)
    9087 {
    9088  VMA_ASSERT(allocator && allocation);
    9089 
    9090  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9091 
    9092  return allocator->TouchAllocation(allocation);
    9093 }
    9094 
    9095 void vmaSetAllocationUserData(
    9096  VmaAllocator allocator,
    9097  VmaAllocation allocation,
    9098  void* pUserData)
    9099 {
    9100  VMA_ASSERT(allocator && allocation);
    9101 
    9102  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9103 
    9104  allocation->SetUserData(allocator, pUserData);
    9105 }
    9106 
    9107 void vmaCreateLostAllocation(
    9108  VmaAllocator allocator,
    9109  VmaAllocation* pAllocation)
    9110 {
    9111  VMA_ASSERT(allocator && pAllocation);
    9112 
    9113  VMA_DEBUG_GLOBAL_MUTEX_LOCK;
    9114 
    9115  allocator->CreateLostAllocation(pAllocation);
    9116 }
    9117 
    9118 VkResult vmaMapMemory(
    9119  VmaAllocator allocator,
    9120  VmaAllocation allocation,
    9121  void** ppData)
    9122 {
    9123  VMA_ASSERT(allocator && allocation && ppData);
    9124 
    9125  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9126 
    9127  return allocator->Map(allocation, ppData);
    9128 }
    9129 
    9130 void vmaUnmapMemory(
    9131  VmaAllocator allocator,
    9132  VmaAllocation allocation)
    9133 {
    9134  VMA_ASSERT(allocator && allocation);
    9135 
    9136  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9137 
    9138  allocator->Unmap(allocation);
    9139 }
    9140 
    9141 VkResult vmaDefragment(
    9142  VmaAllocator allocator,
    9143  VmaAllocation* pAllocations,
    9144  size_t allocationCount,
    9145  VkBool32* pAllocationsChanged,
    9146  const VmaDefragmentationInfo *pDefragmentationInfo,
    9147  VmaDefragmentationStats* pDefragmentationStats)
    9148 {
    9149  VMA_ASSERT(allocator && pAllocations);
    9150 
    9151  VMA_DEBUG_LOG("vmaDefragment");
    9152 
    9153  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9154 
    9155  return allocator->Defragment(pAllocations, allocationCount, pAllocationsChanged, pDefragmentationInfo, pDefragmentationStats);
    9156 }
    9157 
    9158 VkResult vmaBindBufferMemory(
    9159  VmaAllocator allocator,
    9160  VmaAllocation allocation,
    9161  VkBuffer buffer)
    9162 {
    9163  VMA_ASSERT(allocator && allocation && buffer);
    9164 
    9165  VMA_DEBUG_LOG("vmaBindBufferMemory");
    9166 
    9167  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9168 
    9169  return allocator->BindBufferMemory(allocation, buffer);
    9170 }
    9171 
    9172 VkResult vmaBindImageMemory(
    9173  VmaAllocator allocator,
    9174  VmaAllocation allocation,
    9175  VkImage image)
    9176 {
    9177  VMA_ASSERT(allocator && allocation && image);
    9178 
    9179  VMA_DEBUG_LOG("vmaBindImageMemory");
    9180 
    9181  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9182 
    9183  return allocator->BindImageMemory(allocation, image);
    9184 }
    9185 
    9186 VkResult vmaCreateBuffer(
    9187  VmaAllocator allocator,
    9188  const VkBufferCreateInfo* pBufferCreateInfo,
    9189  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    9190  VkBuffer* pBuffer,
    9191  VmaAllocation* pAllocation,
    9192  VmaAllocationInfo* pAllocationInfo)
    9193 {
    9194  VMA_ASSERT(allocator && pBufferCreateInfo && pAllocationCreateInfo && pBuffer && pAllocation);
    9195 
    9196  VMA_DEBUG_LOG("vmaCreateBuffer");
    9197 
    9198  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9199 
    9200  *pBuffer = VK_NULL_HANDLE;
    9201  *pAllocation = VK_NULL_HANDLE;
    9202 
    9203  // 1. Create VkBuffer.
    9204  VkResult res = (*allocator->GetVulkanFunctions().vkCreateBuffer)(
    9205  allocator->m_hDevice,
    9206  pBufferCreateInfo,
    9207  allocator->GetAllocationCallbacks(),
    9208  pBuffer);
    9209  if(res >= 0)
    9210  {
    9211  // 2. vkGetBufferMemoryRequirements.
    9212  VkMemoryRequirements vkMemReq = {};
    9213  bool requiresDedicatedAllocation = false;
    9214  bool prefersDedicatedAllocation = false;
    9215  allocator->GetBufferMemoryRequirements(*pBuffer, vkMemReq,
    9216  requiresDedicatedAllocation, prefersDedicatedAllocation);
    9217 
    9218  // Make sure alignment requirements for specific buffer usages reported
    9219  // in Physical Device Properties are included in alignment reported by memory requirements.
    9220  if((pBufferCreateInfo->usage & VK_BUFFER_USAGE_UNIFORM_TEXEL_BUFFER_BIT) != 0)
    9221  {
    9222  VMA_ASSERT(vkMemReq.alignment %
    9223  allocator->m_PhysicalDeviceProperties.limits.minTexelBufferOffsetAlignment == 0);
    9224  }
    9225  if((pBufferCreateInfo->usage & VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT) != 0)
    9226  {
    9227  VMA_ASSERT(vkMemReq.alignment %
    9228  allocator->m_PhysicalDeviceProperties.limits.minUniformBufferOffsetAlignment == 0);
    9229  }
    9230  if((pBufferCreateInfo->usage & VK_BUFFER_USAGE_STORAGE_BUFFER_BIT) != 0)
    9231  {
    9232  VMA_ASSERT(vkMemReq.alignment %
    9233  allocator->m_PhysicalDeviceProperties.limits.minStorageBufferOffsetAlignment == 0);
    9234  }
    9235 
    9236  // 3. Allocate memory using allocator.
    9237  res = allocator->AllocateMemory(
    9238  vkMemReq,
    9239  requiresDedicatedAllocation,
    9240  prefersDedicatedAllocation,
    9241  *pBuffer, // dedicatedBuffer
    9242  VK_NULL_HANDLE, // dedicatedImage
    9243  *pAllocationCreateInfo,
    9244  VMA_SUBALLOCATION_TYPE_BUFFER,
    9245  pAllocation);
    9246  if(res >= 0)
    9247  {
    9248  // 3. Bind buffer with memory.
    9249  res = allocator->BindBufferMemory(*pAllocation, *pBuffer);
    9250  if(res >= 0)
    9251  {
    9252  // All steps succeeded.
    9253  if(pAllocationInfo != VMA_NULL)
    9254  {
    9255  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
    9256  }
    9257  return VK_SUCCESS;
    9258  }
    9259  allocator->FreeMemory(*pAllocation);
    9260  *pAllocation = VK_NULL_HANDLE;
    9261  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
    9262  *pBuffer = VK_NULL_HANDLE;
    9263  return res;
    9264  }
    9265  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
    9266  *pBuffer = VK_NULL_HANDLE;
    9267  return res;
    9268  }
    9269  return res;
    9270 }
    9271 
    9272 void vmaDestroyBuffer(
    9273  VmaAllocator allocator,
    9274  VkBuffer buffer,
    9275  VmaAllocation allocation)
    9276 {
    9277  if(buffer != VK_NULL_HANDLE)
    9278  {
    9279  VMA_ASSERT(allocator);
    9280 
    9281  VMA_DEBUG_LOG("vmaDestroyBuffer");
    9282 
    9283  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9284 
    9285  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, buffer, allocator->GetAllocationCallbacks());
    9286 
    9287  allocator->FreeMemory(allocation);
    9288  }
    9289 }
    9290 
    9291 VkResult vmaCreateImage(
    9292  VmaAllocator allocator,
    9293  const VkImageCreateInfo* pImageCreateInfo,
    9294  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    9295  VkImage* pImage,
    9296  VmaAllocation* pAllocation,
    9297  VmaAllocationInfo* pAllocationInfo)
    9298 {
    9299  VMA_ASSERT(allocator && pImageCreateInfo && pAllocationCreateInfo && pImage && pAllocation);
    9300 
    9301  VMA_DEBUG_LOG("vmaCreateImage");
    9302 
    9303  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9304 
    9305  *pImage = VK_NULL_HANDLE;
    9306  *pAllocation = VK_NULL_HANDLE;
    9307 
    9308  // 1. Create VkImage.
    9309  VkResult res = (*allocator->GetVulkanFunctions().vkCreateImage)(
    9310  allocator->m_hDevice,
    9311  pImageCreateInfo,
    9312  allocator->GetAllocationCallbacks(),
    9313  pImage);
    9314  if(res >= 0)
    9315  {
    9316  VmaSuballocationType suballocType = pImageCreateInfo->tiling == VK_IMAGE_TILING_OPTIMAL ?
    9317  VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL :
    9318  VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR;
    9319 
    9320  // 2. Allocate memory using allocator.
    9321  res = AllocateMemoryForImage(allocator, *pImage, pAllocationCreateInfo, suballocType, pAllocation);
    9322  if(res >= 0)
    9323  {
    9324  // 3. Bind image with memory.
    9325  res = allocator->BindImageMemory(*pAllocation, *pImage);
    9326  if(res >= 0)
    9327  {
    9328  // All steps succeeded.
    9329  if(pAllocationInfo != VMA_NULL)
    9330  {
    9331  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
    9332  }
    9333  return VK_SUCCESS;
    9334  }
    9335  allocator->FreeMemory(*pAllocation);
    9336  *pAllocation = VK_NULL_HANDLE;
    9337  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, *pImage, allocator->GetAllocationCallbacks());
    9338  *pImage = VK_NULL_HANDLE;
    9339  return res;
    9340  }
    9341  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, *pImage, allocator->GetAllocationCallbacks());
    9342  *pImage = VK_NULL_HANDLE;
    9343  return res;
    9344  }
    9345  return res;
    9346 }
    9347 
    9348 void vmaDestroyImage(
    9349  VmaAllocator allocator,
    9350  VkImage image,
    9351  VmaAllocation allocation)
    9352 {
    9353  if(image != VK_NULL_HANDLE)
    9354  {
    9355  VMA_ASSERT(allocator);
    9356 
    9357  VMA_DEBUG_LOG("vmaDestroyImage");
    9358 
    9359  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9360 
    9361  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, image, allocator->GetAllocationCallbacks());
    9362 
    9363  allocator->FreeMemory(allocation);
    9364  }
    9365 }
    9366 
    9367 #endif // #ifdef VMA_IMPLEMENTATION
    VmaVulkanFunctions::vkGetPhysicalDeviceProperties
    PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties
    Definition: vk_mem_alloc.h:1157
    -
    VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT
    Set this flag if the allocation should have its own memory block.
    Definition: vk_mem_alloc.h:1419
    +Go to the documentation of this file.
    1 //
    2 // Copyright (c) 2017-2018 Advanced Micro Devices, Inc. All rights reserved.
    3 //
    4 // Permission is hereby granted, free of charge, to any person obtaining a copy
    5 // of this software and associated documentation files (the "Software"), to deal
    6 // in the Software without restriction, including without limitation the rights
    7 // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
    8 // copies of the Software, and to permit persons to whom the Software is
    9 // furnished to do so, subject to the following conditions:
    10 //
    11 // The above copyright notice and this permission notice shall be included in
    12 // all copies or substantial portions of the Software.
    13 //
    14 // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    15 // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    16 // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
    17 // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    18 // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
    19 // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
    20 // THE SOFTWARE.
    21 //
    22 
    23 #ifndef AMD_VULKAN_MEMORY_ALLOCATOR_H
    24 #define AMD_VULKAN_MEMORY_ALLOCATOR_H
    25 
    26 #ifdef __cplusplus
    27 extern "C" {
    28 #endif
    29 
    1082 #include <vulkan/vulkan.h>
    1083 
    1084 #if !defined(VMA_DEDICATED_ALLOCATION)
    1085  #if VK_KHR_get_memory_requirements2 && VK_KHR_dedicated_allocation
    1086  #define VMA_DEDICATED_ALLOCATION 1
    1087  #else
    1088  #define VMA_DEDICATED_ALLOCATION 0
    1089  #endif
    1090 #endif
    1091 
    1101 VK_DEFINE_HANDLE(VmaAllocator)
    1102 
    1103 typedef void (VKAPI_PTR *PFN_vmaAllocateDeviceMemoryFunction)(
    1105  VmaAllocator allocator,
    1106  uint32_t memoryType,
    1107  VkDeviceMemory memory,
    1108  VkDeviceSize size);
    1110 typedef void (VKAPI_PTR *PFN_vmaFreeDeviceMemoryFunction)(
    1111  VmaAllocator allocator,
    1112  uint32_t memoryType,
    1113  VkDeviceMemory memory,
    1114  VkDeviceSize size);
    1115 
    1123 typedef struct VmaDeviceMemoryCallbacks {
    1125  PFN_vmaAllocateDeviceMemoryFunction pfnAllocate;
    1127  PFN_vmaFreeDeviceMemoryFunction pfnFree;
    1128 } VmaDeviceMemoryCallbacks;
    1129 
    1131 typedef enum VmaAllocatorCreateFlagBits {
    1136  VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT = 0x00000001,
    1158  VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT = 0x00000002,
    1159 
    1160  VMA_ALLOCATOR_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
    1161 } VmaAllocatorCreateFlagBits;
    1162 typedef VkFlags VmaAllocatorCreateFlags;
    1163 
    1168 typedef struct VmaVulkanFunctions {
    1169  PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties;
    1170  PFN_vkGetPhysicalDeviceMemoryProperties vkGetPhysicalDeviceMemoryProperties;
    1171  PFN_vkAllocateMemory vkAllocateMemory;
    1172  PFN_vkFreeMemory vkFreeMemory;
    1173  PFN_vkMapMemory vkMapMemory;
    1174  PFN_vkUnmapMemory vkUnmapMemory;
    1175  PFN_vkBindBufferMemory vkBindBufferMemory;
    1176  PFN_vkBindImageMemory vkBindImageMemory;
    1177  PFN_vkGetBufferMemoryRequirements vkGetBufferMemoryRequirements;
    1178  PFN_vkGetImageMemoryRequirements vkGetImageMemoryRequirements;
    1179  PFN_vkCreateBuffer vkCreateBuffer;
    1180  PFN_vkDestroyBuffer vkDestroyBuffer;
    1181  PFN_vkCreateImage vkCreateImage;
    1182  PFN_vkDestroyImage vkDestroyImage;
    1183 #if VMA_DEDICATED_ALLOCATION
    1184  PFN_vkGetBufferMemoryRequirements2KHR vkGetBufferMemoryRequirements2KHR;
    1185  PFN_vkGetImageMemoryRequirements2KHR vkGetImageMemoryRequirements2KHR;
    1186 #endif
    1187 } VmaVulkanFunctions;
    1188 
    1190 typedef struct VmaAllocatorCreateInfo
    1191 {
    1193  VmaAllocatorCreateFlags flags;
    1195 
    1196  VkPhysicalDevice physicalDevice;
    1198 
    1199  VkDevice device;
    1201 
    1202  VkDeviceSize preferredLargeHeapBlockSize;
    1204 
    1205  const VkAllocationCallbacks* pAllocationCallbacks;
    1207 
    1208  const VmaDeviceMemoryCallbacks* pDeviceMemoryCallbacks;
    1222  uint32_t frameInUseCount;
    1246  const VkDeviceSize* pHeapSizeLimit;
    1258  const VmaVulkanFunctions* pVulkanFunctions;
    1259 } VmaAllocatorCreateInfo;
    1260 
    1262 VkResult vmaCreateAllocator(
    1263  const VmaAllocatorCreateInfo* pCreateInfo,
    1264  VmaAllocator* pAllocator);
    1265 
    1267 void vmaDestroyAllocator(
    1268  VmaAllocator allocator);
    1269 
    1274 void vmaGetPhysicalDeviceProperties(
    1275  VmaAllocator allocator,
    1276  const VkPhysicalDeviceProperties** ppPhysicalDeviceProperties);
    1277 
    1282 void vmaGetMemoryProperties(
    1283  VmaAllocator allocator,
    1284  const VkPhysicalDeviceMemoryProperties** ppPhysicalDeviceMemoryProperties);
    1285 
    1292 void vmaGetMemoryTypeProperties(
    1293  VmaAllocator allocator,
    1294  uint32_t memoryTypeIndex,
    1295  VkMemoryPropertyFlags* pFlags);
    1296 
    1305 void vmaSetCurrentFrameIndex(
    1306  VmaAllocator allocator,
    1307  uint32_t frameIndex);
    1308 
    1311 typedef struct VmaStatInfo
    1312 {
    1314  uint32_t blockCount;
    1316  uint32_t allocationCount;
    1318  uint32_t unusedRangeCount;
    1320  VkDeviceSize usedBytes;
    1322  VkDeviceSize unusedBytes;
    1323  VkDeviceSize allocationSizeMin, allocationSizeAvg, allocationSizeMax;
    1324  VkDeviceSize unusedRangeSizeMin, unusedRangeSizeAvg, unusedRangeSizeMax;
    1325 } VmaStatInfo;
    1326 
    1328 typedef struct VmaStats
    1329 {
    1330  VmaStatInfo memoryType[VK_MAX_MEMORY_TYPES];
    1331  VmaStatInfo memoryHeap[VK_MAX_MEMORY_HEAPS];
    1332  VmaStatInfo total;
    1333 } VmaStats;
    1334 
    1336 void vmaCalculateStats(
    1337  VmaAllocator allocator,
    1338  VmaStats* pStats);
    1339 
    1340 #define VMA_STATS_STRING_ENABLED 1
    1341 
    1342 #if VMA_STATS_STRING_ENABLED
    1343 
    1345 
    1347 void vmaBuildStatsString(
    1348  VmaAllocator allocator,
    1349  char** ppStatsString,
    1350  VkBool32 detailedMap);
    1351 
    1352 void vmaFreeStatsString(
    1353  VmaAllocator allocator,
    1354  char* pStatsString);
    1355 
    1356 #endif // #if VMA_STATS_STRING_ENABLED
    1357 
    1366 VK_DEFINE_HANDLE(VmaPool)
    1367 
    1368 typedef enum VmaMemoryUsage
    1369 {
    1373  VMA_MEMORY_USAGE_UNKNOWN = 0,
    1390  VMA_MEMORY_USAGE_GPU_ONLY = 1,
    1400  VMA_MEMORY_USAGE_CPU_ONLY = 2,
    1407  VMA_MEMORY_USAGE_CPU_TO_GPU = 3,
    1416  VMA_MEMORY_USAGE_GPU_TO_CPU = 4,
    1417  VMA_MEMORY_USAGE_MAX_ENUM = 0x7FFFFFFF
    1418 } VmaMemoryUsage;
    1419 
    1421 typedef enum VmaAllocationCreateFlagBits {
    1433  VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT = 0x00000001,
    1434 
    1444  VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT = 0x00000002,
    1457  VMA_ALLOCATION_CREATE_MAPPED_BIT = 0x00000004,
    1470  VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT = 0x00000008,
    1477  VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT = 0x00000010,
    1483  VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT = 0x00000020,
    1484 
    1485  VMA_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
    1486 } VmaAllocationCreateFlagBits;
    1487 typedef VkFlags VmaAllocationCreateFlags;
    1488 
    1489 typedef struct VmaAllocationCreateInfo
    1490 {
    1492  VmaAllocationCreateFlags flags;
    1498  VmaMemoryUsage usage;
    1503  VkMemoryPropertyFlags requiredFlags;
    1508  VkMemoryPropertyFlags preferredFlags;
    1516  uint32_t memoryTypeBits;
    1522  VmaPool pool;
    1529  void* pUserData;
    1530 } VmaAllocationCreateInfo;
    1531 
    1548 VkResult vmaFindMemoryTypeIndex(
    1549  VmaAllocator allocator,
    1550  uint32_t memoryTypeBits,
    1551  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    1552  uint32_t* pMemoryTypeIndex);
    1553 
    1566 VkResult vmaFindMemoryTypeIndexForBufferInfo(
    1567  VmaAllocator allocator,
    1568  const VkBufferCreateInfo* pBufferCreateInfo,
    1569  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    1570  uint32_t* pMemoryTypeIndex);
    1571 
    1584 VkResult vmaFindMemoryTypeIndexForImageInfo(
    1585  VmaAllocator allocator,
    1586  const VkImageCreateInfo* pImageCreateInfo,
    1587  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    1588  uint32_t* pMemoryTypeIndex);
    1589 
    1591 typedef enum VmaPoolCreateFlagBits {
    1609  VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT = 0x00000002,
    1610 
    1611  VMA_POOL_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
    1612 } VmaPoolCreateFlagBits;
    1613 typedef VkFlags VmaPoolCreateFlags;
    1614 
    1617 typedef struct VmaPoolCreateInfo {
    1620  uint32_t memoryTypeIndex;
    1623  VmaPoolCreateFlags flags;
    1628  VkDeviceSize blockSize;
    1633  size_t minBlockCount;
    1641  size_t maxBlockCount;
    1655  uint32_t frameInUseCount;
    1656 } VmaPoolCreateInfo;
    1657 
    1660 typedef struct VmaPoolStats {
    1663  VkDeviceSize size;
    1666  VkDeviceSize unusedSize;
    1669  size_t allocationCount;
    1672  size_t unusedRangeCount;
    1679  VkDeviceSize unusedRangeSizeMax;
    1680 } VmaPoolStats;
    1681 
    1688 VkResult vmaCreatePool(
    1689  VmaAllocator allocator,
    1690  const VmaPoolCreateInfo* pCreateInfo,
    1691  VmaPool* pPool);
    1692 
    1695 void vmaDestroyPool(
    1696  VmaAllocator allocator,
    1697  VmaPool pool);
    1698 
    1705 void vmaGetPoolStats(
    1706  VmaAllocator allocator,
    1707  VmaPool pool,
    1708  VmaPoolStats* pPoolStats);
    1709 
    1716 void vmaMakePoolAllocationsLost(
    1717  VmaAllocator allocator,
    1718  VmaPool pool,
    1719  size_t* pLostAllocationCount);
    1720 
    1745 VK_DEFINE_HANDLE(VmaAllocation)
    1746 
    1747 
    1749 typedef struct VmaAllocationInfo {
    1754  uint32_t memoryType;
    1763  VkDeviceMemory deviceMemory;
    1768  VkDeviceSize offset;
    1773  VkDeviceSize size;
    1782  void* pMappedData;
    1787  void* pUserData;
    1788 } VmaAllocationInfo;
    1789 
    1800 VkResult vmaAllocateMemory(
    1801  VmaAllocator allocator,
    1802  const VkMemoryRequirements* pVkMemoryRequirements,
    1803  const VmaAllocationCreateInfo* pCreateInfo,
    1804  VmaAllocation* pAllocation,
    1805  VmaAllocationInfo* pAllocationInfo);
    1806 
    1813 VkResult vmaAllocateMemoryForBuffer(
    1814  VmaAllocator allocator,
    1815  VkBuffer buffer,
    1816  const VmaAllocationCreateInfo* pCreateInfo,
    1817  VmaAllocation* pAllocation,
    1818  VmaAllocationInfo* pAllocationInfo);
    1819 
    1821 VkResult vmaAllocateMemoryForImage(
    1822  VmaAllocator allocator,
    1823  VkImage image,
    1824  const VmaAllocationCreateInfo* pCreateInfo,
    1825  VmaAllocation* pAllocation,
    1826  VmaAllocationInfo* pAllocationInfo);
    1827 
    1829 void vmaFreeMemory(
    1830  VmaAllocator allocator,
    1831  VmaAllocation allocation);
    1832 
    1849 void vmaGetAllocationInfo(
    1850  VmaAllocator allocator,
    1851  VmaAllocation allocation,
    1852  VmaAllocationInfo* pAllocationInfo);
    1853 
    1868 VkBool32 vmaTouchAllocation(
    1869  VmaAllocator allocator,
    1870  VmaAllocation allocation);
    1871 
    1885 void vmaSetAllocationUserData(
    1886  VmaAllocator allocator,
    1887  VmaAllocation allocation,
    1888  void* pUserData);
    1889 
    1900 void vmaCreateLostAllocation(
    1901  VmaAllocator allocator,
    1902  VmaAllocation* pAllocation);
    1903 
    1938 VkResult vmaMapMemory(
    1939  VmaAllocator allocator,
    1940  VmaAllocation allocation,
    1941  void** ppData);
    1942 
    1947 void vmaUnmapMemory(
    1948  VmaAllocator allocator,
    1949  VmaAllocation allocation);
    1950 
    1952 typedef struct VmaDefragmentationInfo {
    1957  VkDeviceSize maxBytesToMove;
    1962  uint32_t maxAllocationsToMove;
    1963 } VmaDefragmentationInfo;
    1964 
    1966 typedef struct VmaDefragmentationStats {
    1968  VkDeviceSize bytesMoved;
    1970  VkDeviceSize bytesFreed;
    1972  uint32_t allocationsMoved;
    1974  uint32_t deviceMemoryBlocksFreed;
    1975 } VmaDefragmentationStats;
    1976 
    2059 VkResult vmaDefragment(
    2060  VmaAllocator allocator,
    2061  VmaAllocation* pAllocations,
    2062  size_t allocationCount,
    2063  VkBool32* pAllocationsChanged,
    2064  const VmaDefragmentationInfo *pDefragmentationInfo,
    2065  VmaDefragmentationStats* pDefragmentationStats);
    2066 
    2079 VkResult vmaBindBufferMemory(
    2080  VmaAllocator allocator,
    2081  VmaAllocation allocation,
    2082  VkBuffer buffer);
    2083 
    2096 VkResult vmaBindImageMemory(
    2097  VmaAllocator allocator,
    2098  VmaAllocation allocation,
    2099  VkImage image);
    2100 
    2127 VkResult vmaCreateBuffer(
    2128  VmaAllocator allocator,
    2129  const VkBufferCreateInfo* pBufferCreateInfo,
    2130  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    2131  VkBuffer* pBuffer,
    2132  VmaAllocation* pAllocation,
    2133  VmaAllocationInfo* pAllocationInfo);
    2134 
    2146 void vmaDestroyBuffer(
    2147  VmaAllocator allocator,
    2148  VkBuffer buffer,
    2149  VmaAllocation allocation);
    2150 
    2152 VkResult vmaCreateImage(
    2153  VmaAllocator allocator,
    2154  const VkImageCreateInfo* pImageCreateInfo,
    2155  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    2156  VkImage* pImage,
    2157  VmaAllocation* pAllocation,
    2158  VmaAllocationInfo* pAllocationInfo);
    2159 
    2171 void vmaDestroyImage(
    2172  VmaAllocator allocator,
    2173  VkImage image,
    2174  VmaAllocation allocation);
    2175 
    2176 #ifdef __cplusplus
    2177 }
    2178 #endif
    2179 
    2180 #endif // AMD_VULKAN_MEMORY_ALLOCATOR_H
    2181 
    2182 // For Visual Studio IntelliSense.
    2183 #ifdef __INTELLISENSE__
    2184 #define VMA_IMPLEMENTATION
    2185 #endif
    2186 
    2187 #ifdef VMA_IMPLEMENTATION
    2188 #undef VMA_IMPLEMENTATION
    2189 
    2190 #include <cstdint>
    2191 #include <cstdlib>
    2192 #include <cstring>
    2193 
    2194 /*******************************************************************************
    2195 CONFIGURATION SECTION
    2196 
    2197 Define some of these macros before each #include of this header or change them
    2198 here if you need other then default behavior depending on your environment.
    2199 */
    2200 
    2201 /*
    2202 Define this macro to 1 to make the library fetch pointers to Vulkan functions
    2203 internally, like:
    2204 
    2205  vulkanFunctions.vkAllocateMemory = &vkAllocateMemory;
    2206 
    2207 Define to 0 if you are going to provide you own pointers to Vulkan functions via
    2208 VmaAllocatorCreateInfo::pVulkanFunctions.
    2209 */
    2210 #if !defined(VMA_STATIC_VULKAN_FUNCTIONS) && !defined(VK_NO_PROTOTYPES)
    2211 #define VMA_STATIC_VULKAN_FUNCTIONS 1
    2212 #endif
    2213 
    2214 // Define this macro to 1 to make the library use STL containers instead of its own implementation.
    2215 //#define VMA_USE_STL_CONTAINERS 1
    2216 
    2217 /* Set this macro to 1 to make the library including and using STL containers:
    2218 std::pair, std::vector, std::list, std::unordered_map.
    2219 
    2220 Set it to 0 or undefined to make the library using its own implementation of
    2221 the containers.
    2222 */
    2223 #if VMA_USE_STL_CONTAINERS
    2224  #define VMA_USE_STL_VECTOR 1
    2225  #define VMA_USE_STL_UNORDERED_MAP 1
    2226  #define VMA_USE_STL_LIST 1
    2227 #endif
    2228 
    2229 #if VMA_USE_STL_VECTOR
    2230  #include <vector>
    2231 #endif
    2232 
    2233 #if VMA_USE_STL_UNORDERED_MAP
    2234  #include <unordered_map>
    2235 #endif
    2236 
    2237 #if VMA_USE_STL_LIST
    2238  #include <list>
    2239 #endif
    2240 
    2241 /*
    2242 Following headers are used in this CONFIGURATION section only, so feel free to
    2243 remove them if not needed.
    2244 */
    2245 #include <cassert> // for assert
    2246 #include <algorithm> // for min, max
    2247 #include <mutex> // for std::mutex
    2248 #include <atomic> // for std::atomic
    2249 
    2250 #if !defined(_WIN32) && !defined(__APPLE__)
    2251  #include <malloc.h> // for aligned_alloc()
    2252 #endif
    2253 
    2254 #ifndef VMA_NULL
    2255  // Value used as null pointer. Define it to e.g.: nullptr, NULL, 0, (void*)0.
    2256  #define VMA_NULL nullptr
    2257 #endif
    2258 
    2259 #if defined(__APPLE__) || defined(__ANDROID__)
    2260 #include <cstdlib>
    2261 void *aligned_alloc(size_t alignment, size_t size)
    2262 {
    2263  // alignment must be >= sizeof(void*)
    2264  if(alignment < sizeof(void*))
    2265  {
    2266  alignment = sizeof(void*);
    2267  }
    2268 
    2269  void *pointer;
    2270  if(posix_memalign(&pointer, alignment, size) == 0)
    2271  return pointer;
    2272  return VMA_NULL;
    2273 }
    2274 #endif
    2275 
    2276 // Normal assert to check for programmer's errors, especially in Debug configuration.
    2277 #ifndef VMA_ASSERT
    2278  #ifdef _DEBUG
    2279  #define VMA_ASSERT(expr) assert(expr)
    2280  #else
    2281  #define VMA_ASSERT(expr)
    2282  #endif
    2283 #endif
    2284 
    2285 // Assert that will be called very often, like inside data structures e.g. operator[].
    2286 // Making it non-empty can make program slow.
    2287 #ifndef VMA_HEAVY_ASSERT
    2288  #ifdef _DEBUG
    2289  #define VMA_HEAVY_ASSERT(expr) //VMA_ASSERT(expr)
    2290  #else
    2291  #define VMA_HEAVY_ASSERT(expr)
    2292  #endif
    2293 #endif
    2294 
    2295 #ifndef VMA_ALIGN_OF
    2296  #define VMA_ALIGN_OF(type) (__alignof(type))
    2297 #endif
    2298 
    2299 #ifndef VMA_SYSTEM_ALIGNED_MALLOC
    2300  #if defined(_WIN32)
    2301  #define VMA_SYSTEM_ALIGNED_MALLOC(size, alignment) (_aligned_malloc((size), (alignment)))
    2302  #else
    2303  #define VMA_SYSTEM_ALIGNED_MALLOC(size, alignment) (aligned_alloc((alignment), (size) ))
    2304  #endif
    2305 #endif
    2306 
    2307 #ifndef VMA_SYSTEM_FREE
    2308  #if defined(_WIN32)
    2309  #define VMA_SYSTEM_FREE(ptr) _aligned_free(ptr)
    2310  #else
    2311  #define VMA_SYSTEM_FREE(ptr) free(ptr)
    2312  #endif
    2313 #endif
    2314 
    2315 #ifndef VMA_MIN
    2316  #define VMA_MIN(v1, v2) (std::min((v1), (v2)))
    2317 #endif
    2318 
    2319 #ifndef VMA_MAX
    2320  #define VMA_MAX(v1, v2) (std::max((v1), (v2)))
    2321 #endif
    2322 
    2323 #ifndef VMA_SWAP
    2324  #define VMA_SWAP(v1, v2) std::swap((v1), (v2))
    2325 #endif
    2326 
    2327 #ifndef VMA_SORT
    2328  #define VMA_SORT(beg, end, cmp) std::sort(beg, end, cmp)
    2329 #endif
    2330 
    2331 #ifndef VMA_DEBUG_LOG
    2332  #define VMA_DEBUG_LOG(format, ...)
    2333  /*
    2334  #define VMA_DEBUG_LOG(format, ...) do { \
    2335  printf(format, __VA_ARGS__); \
    2336  printf("\n"); \
    2337  } while(false)
    2338  */
    2339 #endif
    2340 
    2341 // Define this macro to 1 to enable functions: vmaBuildStatsString, vmaFreeStatsString.
    2342 #if VMA_STATS_STRING_ENABLED
    2343  static inline void VmaUint32ToStr(char* outStr, size_t strLen, uint32_t num)
    2344  {
    2345  snprintf(outStr, strLen, "%u", static_cast<unsigned int>(num));
    2346  }
    2347  static inline void VmaUint64ToStr(char* outStr, size_t strLen, uint64_t num)
    2348  {
    2349  snprintf(outStr, strLen, "%llu", static_cast<unsigned long long>(num));
    2350  }
    2351  static inline void VmaPtrToStr(char* outStr, size_t strLen, const void* ptr)
    2352  {
    2353  snprintf(outStr, strLen, "%p", ptr);
    2354  }
    2355 #endif
    2356 
    2357 #ifndef VMA_MUTEX
    2358  class VmaMutex
    2359  {
    2360  public:
    2361  VmaMutex() { }
    2362  ~VmaMutex() { }
    2363  void Lock() { m_Mutex.lock(); }
    2364  void Unlock() { m_Mutex.unlock(); }
    2365  private:
    2366  std::mutex m_Mutex;
    2367  };
    2368  #define VMA_MUTEX VmaMutex
    2369 #endif
    2370 
    2371 /*
    2372 If providing your own implementation, you need to implement a subset of std::atomic:
    2373 
    2374 - Constructor(uint32_t desired)
    2375 - uint32_t load() const
    2376 - void store(uint32_t desired)
    2377 - bool compare_exchange_weak(uint32_t& expected, uint32_t desired)
    2378 */
    2379 #ifndef VMA_ATOMIC_UINT32
    2380  #define VMA_ATOMIC_UINT32 std::atomic<uint32_t>
    2381 #endif
    2382 
    2383 #ifndef VMA_BEST_FIT
    2384 
    2396  #define VMA_BEST_FIT (1)
    2397 #endif
    2398 
    2399 #ifndef VMA_DEBUG_ALWAYS_DEDICATED_MEMORY
    2400 
    2404  #define VMA_DEBUG_ALWAYS_DEDICATED_MEMORY (0)
    2405 #endif
    2406 
    2407 #ifndef VMA_DEBUG_ALIGNMENT
    2408 
    2412  #define VMA_DEBUG_ALIGNMENT (1)
    2413 #endif
    2414 
    2415 #ifndef VMA_DEBUG_MARGIN
    2416 
    2420  #define VMA_DEBUG_MARGIN (0)
    2421 #endif
    2422 
    2423 #ifndef VMA_DEBUG_GLOBAL_MUTEX
    2424 
    2428  #define VMA_DEBUG_GLOBAL_MUTEX (0)
    2429 #endif
    2430 
    2431 #ifndef VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY
    2432 
    2436  #define VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY (1)
    2437 #endif
    2438 
    2439 #ifndef VMA_SMALL_HEAP_MAX_SIZE
    2440  #define VMA_SMALL_HEAP_MAX_SIZE (1024ull * 1024 * 1024)
    2442 #endif
    2443 
    2444 #ifndef VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE
    2445  #define VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE (256ull * 1024 * 1024)
    2447 #endif
    2448 
    2449 static const uint32_t VMA_FRAME_INDEX_LOST = UINT32_MAX;
    2450 
    2451 /*******************************************************************************
    2452 END OF CONFIGURATION
    2453 */
    2454 
    2455 static VkAllocationCallbacks VmaEmptyAllocationCallbacks = {
    2456  VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL };
    2457 
    2458 // Returns number of bits set to 1 in (v).
    2459 static inline uint32_t VmaCountBitsSet(uint32_t v)
    2460 {
    2461  uint32_t c = v - ((v >> 1) & 0x55555555);
    2462  c = ((c >> 2) & 0x33333333) + (c & 0x33333333);
    2463  c = ((c >> 4) + c) & 0x0F0F0F0F;
    2464  c = ((c >> 8) + c) & 0x00FF00FF;
    2465  c = ((c >> 16) + c) & 0x0000FFFF;
    2466  return c;
    2467 }
    2468 
    2469 // Aligns given value up to nearest multiply of align value. For example: VmaAlignUp(11, 8) = 16.
    2470 // Use types like uint32_t, uint64_t as T.
    2471 template <typename T>
    2472 static inline T VmaAlignUp(T val, T align)
    2473 {
    2474  return (val + align - 1) / align * align;
    2475 }
    2476 
    2477 // Division with mathematical rounding to nearest number.
    2478 template <typename T>
    2479 inline T VmaRoundDiv(T x, T y)
    2480 {
    2481  return (x + (y / (T)2)) / y;
    2482 }
    2483 
    2484 #ifndef VMA_SORT
    2485 
    2486 template<typename Iterator, typename Compare>
    2487 Iterator VmaQuickSortPartition(Iterator beg, Iterator end, Compare cmp)
    2488 {
    2489  Iterator centerValue = end; --centerValue;
    2490  Iterator insertIndex = beg;
    2491  for(Iterator memTypeIndex = beg; memTypeIndex < centerValue; ++memTypeIndex)
    2492  {
    2493  if(cmp(*memTypeIndex, *centerValue))
    2494  {
    2495  if(insertIndex != memTypeIndex)
    2496  {
    2497  VMA_SWAP(*memTypeIndex, *insertIndex);
    2498  }
    2499  ++insertIndex;
    2500  }
    2501  }
    2502  if(insertIndex != centerValue)
    2503  {
    2504  VMA_SWAP(*insertIndex, *centerValue);
    2505  }
    2506  return insertIndex;
    2507 }
    2508 
    2509 template<typename Iterator, typename Compare>
    2510 void VmaQuickSort(Iterator beg, Iterator end, Compare cmp)
    2511 {
    2512  if(beg < end)
    2513  {
    2514  Iterator it = VmaQuickSortPartition<Iterator, Compare>(beg, end, cmp);
    2515  VmaQuickSort<Iterator, Compare>(beg, it, cmp);
    2516  VmaQuickSort<Iterator, Compare>(it + 1, end, cmp);
    2517  }
    2518 }
    2519 
    2520 #define VMA_SORT(beg, end, cmp) VmaQuickSort(beg, end, cmp)
    2521 
    2522 #endif // #ifndef VMA_SORT
    2523 
    2524 /*
    2525 Returns true if two memory blocks occupy overlapping pages.
    2526 ResourceA must be in less memory offset than ResourceB.
    2527 
    2528 Algorithm is based on "Vulkan 1.0.39 - A Specification (with all registered Vulkan extensions)"
    2529 chapter 11.6 "Resource Memory Association", paragraph "Buffer-Image Granularity".
    2530 */
    2531 static inline bool VmaBlocksOnSamePage(
    2532  VkDeviceSize resourceAOffset,
    2533  VkDeviceSize resourceASize,
    2534  VkDeviceSize resourceBOffset,
    2535  VkDeviceSize pageSize)
    2536 {
    2537  VMA_ASSERT(resourceAOffset + resourceASize <= resourceBOffset && resourceASize > 0 && pageSize > 0);
    2538  VkDeviceSize resourceAEnd = resourceAOffset + resourceASize - 1;
    2539  VkDeviceSize resourceAEndPage = resourceAEnd & ~(pageSize - 1);
    2540  VkDeviceSize resourceBStart = resourceBOffset;
    2541  VkDeviceSize resourceBStartPage = resourceBStart & ~(pageSize - 1);
    2542  return resourceAEndPage == resourceBStartPage;
    2543 }
    2544 
    2545 enum VmaSuballocationType
    2546 {
    2547  VMA_SUBALLOCATION_TYPE_FREE = 0,
    2548  VMA_SUBALLOCATION_TYPE_UNKNOWN = 1,
    2549  VMA_SUBALLOCATION_TYPE_BUFFER = 2,
    2550  VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN = 3,
    2551  VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR = 4,
    2552  VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL = 5,
    2553  VMA_SUBALLOCATION_TYPE_MAX_ENUM = 0x7FFFFFFF
    2554 };
    2555 
    2556 /*
    2557 Returns true if given suballocation types could conflict and must respect
    2558 VkPhysicalDeviceLimits::bufferImageGranularity. They conflict if one is buffer
    2559 or linear image and another one is optimal image. If type is unknown, behave
    2560 conservatively.
    2561 */
    2562 static inline bool VmaIsBufferImageGranularityConflict(
    2563  VmaSuballocationType suballocType1,
    2564  VmaSuballocationType suballocType2)
    2565 {
    2566  if(suballocType1 > suballocType2)
    2567  {
    2568  VMA_SWAP(suballocType1, suballocType2);
    2569  }
    2570 
    2571  switch(suballocType1)
    2572  {
    2573  case VMA_SUBALLOCATION_TYPE_FREE:
    2574  return false;
    2575  case VMA_SUBALLOCATION_TYPE_UNKNOWN:
    2576  return true;
    2577  case VMA_SUBALLOCATION_TYPE_BUFFER:
    2578  return
    2579  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
    2580  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
    2581  case VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN:
    2582  return
    2583  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
    2584  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR ||
    2585  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
    2586  case VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR:
    2587  return
    2588  suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
    2589  case VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL:
    2590  return false;
    2591  default:
    2592  VMA_ASSERT(0);
    2593  return true;
    2594  }
    2595 }
    2596 
    2597 // Helper RAII class to lock a mutex in constructor and unlock it in destructor (at the end of scope).
    2598 struct VmaMutexLock
    2599 {
    2600 public:
    2601  VmaMutexLock(VMA_MUTEX& mutex, bool useMutex) :
    2602  m_pMutex(useMutex ? &mutex : VMA_NULL)
    2603  {
    2604  if(m_pMutex)
    2605  {
    2606  m_pMutex->Lock();
    2607  }
    2608  }
    2609 
    2610  ~VmaMutexLock()
    2611  {
    2612  if(m_pMutex)
    2613  {
    2614  m_pMutex->Unlock();
    2615  }
    2616  }
    2617 
    2618 private:
    2619  VMA_MUTEX* m_pMutex;
    2620 };
    2621 
    2622 #if VMA_DEBUG_GLOBAL_MUTEX
    2623  static VMA_MUTEX gDebugGlobalMutex;
    2624  #define VMA_DEBUG_GLOBAL_MUTEX_LOCK VmaMutexLock debugGlobalMutexLock(gDebugGlobalMutex, true);
    2625 #else
    2626  #define VMA_DEBUG_GLOBAL_MUTEX_LOCK
    2627 #endif
    2628 
    2629 // Minimum size of a free suballocation to register it in the free suballocation collection.
    2630 static const VkDeviceSize VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER = 16;
    2631 
    2632 /*
    2633 Performs binary search and returns iterator to first element that is greater or
    2634 equal to (key), according to comparison (cmp).
    2635 
    2636 Cmp should return true if first argument is less than second argument.
    2637 
    2638 Returned value is the found element, if present in the collection or place where
    2639 new element with value (key) should be inserted.
    2640 */
    2641 template <typename IterT, typename KeyT, typename CmpT>
    2642 static IterT VmaBinaryFindFirstNotLess(IterT beg, IterT end, const KeyT &key, CmpT cmp)
    2643 {
    2644  size_t down = 0, up = (end - beg);
    2645  while(down < up)
    2646  {
    2647  const size_t mid = (down + up) / 2;
    2648  if(cmp(*(beg+mid), key))
    2649  {
    2650  down = mid + 1;
    2651  }
    2652  else
    2653  {
    2654  up = mid;
    2655  }
    2656  }
    2657  return beg + down;
    2658 }
    2659 
    2661 // Memory allocation
    2662 
    2663 static void* VmaMalloc(const VkAllocationCallbacks* pAllocationCallbacks, size_t size, size_t alignment)
    2664 {
    2665  if((pAllocationCallbacks != VMA_NULL) &&
    2666  (pAllocationCallbacks->pfnAllocation != VMA_NULL))
    2667  {
    2668  return (*pAllocationCallbacks->pfnAllocation)(
    2669  pAllocationCallbacks->pUserData,
    2670  size,
    2671  alignment,
    2672  VK_SYSTEM_ALLOCATION_SCOPE_OBJECT);
    2673  }
    2674  else
    2675  {
    2676  return VMA_SYSTEM_ALIGNED_MALLOC(size, alignment);
    2677  }
    2678 }
    2679 
    2680 static void VmaFree(const VkAllocationCallbacks* pAllocationCallbacks, void* ptr)
    2681 {
    2682  if((pAllocationCallbacks != VMA_NULL) &&
    2683  (pAllocationCallbacks->pfnFree != VMA_NULL))
    2684  {
    2685  (*pAllocationCallbacks->pfnFree)(pAllocationCallbacks->pUserData, ptr);
    2686  }
    2687  else
    2688  {
    2689  VMA_SYSTEM_FREE(ptr);
    2690  }
    2691 }
    2692 
    2693 template<typename T>
    2694 static T* VmaAllocate(const VkAllocationCallbacks* pAllocationCallbacks)
    2695 {
    2696  return (T*)VmaMalloc(pAllocationCallbacks, sizeof(T), VMA_ALIGN_OF(T));
    2697 }
    2698 
    2699 template<typename T>
    2700 static T* VmaAllocateArray(const VkAllocationCallbacks* pAllocationCallbacks, size_t count)
    2701 {
    2702  return (T*)VmaMalloc(pAllocationCallbacks, sizeof(T) * count, VMA_ALIGN_OF(T));
    2703 }
    2704 
    2705 #define vma_new(allocator, type) new(VmaAllocate<type>(allocator))(type)
    2706 
    2707 #define vma_new_array(allocator, type, count) new(VmaAllocateArray<type>((allocator), (count)))(type)
    2708 
    2709 template<typename T>
    2710 static void vma_delete(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr)
    2711 {
    2712  ptr->~T();
    2713  VmaFree(pAllocationCallbacks, ptr);
    2714 }
    2715 
    2716 template<typename T>
    2717 static void vma_delete_array(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr, size_t count)
    2718 {
    2719  if(ptr != VMA_NULL)
    2720  {
    2721  for(size_t i = count; i--; )
    2722  {
    2723  ptr[i].~T();
    2724  }
    2725  VmaFree(pAllocationCallbacks, ptr);
    2726  }
    2727 }
    2728 
    2729 // STL-compatible allocator.
    2730 template<typename T>
    2731 class VmaStlAllocator
    2732 {
    2733 public:
    2734  const VkAllocationCallbacks* const m_pCallbacks;
    2735  typedef T value_type;
    2736 
    2737  VmaStlAllocator(const VkAllocationCallbacks* pCallbacks) : m_pCallbacks(pCallbacks) { }
    2738  template<typename U> VmaStlAllocator(const VmaStlAllocator<U>& src) : m_pCallbacks(src.m_pCallbacks) { }
    2739 
    2740  T* allocate(size_t n) { return VmaAllocateArray<T>(m_pCallbacks, n); }
    2741  void deallocate(T* p, size_t n) { VmaFree(m_pCallbacks, p); }
    2742 
    2743  template<typename U>
    2744  bool operator==(const VmaStlAllocator<U>& rhs) const
    2745  {
    2746  return m_pCallbacks == rhs.m_pCallbacks;
    2747  }
    2748  template<typename U>
    2749  bool operator!=(const VmaStlAllocator<U>& rhs) const
    2750  {
    2751  return m_pCallbacks != rhs.m_pCallbacks;
    2752  }
    2753 
    2754  VmaStlAllocator& operator=(const VmaStlAllocator& x) = delete;
    2755 };
    2756 
    2757 #if VMA_USE_STL_VECTOR
    2758 
    2759 #define VmaVector std::vector
    2760 
    2761 template<typename T, typename allocatorT>
    2762 static void VmaVectorInsert(std::vector<T, allocatorT>& vec, size_t index, const T& item)
    2763 {
    2764  vec.insert(vec.begin() + index, item);
    2765 }
    2766 
    2767 template<typename T, typename allocatorT>
    2768 static void VmaVectorRemove(std::vector<T, allocatorT>& vec, size_t index)
    2769 {
    2770  vec.erase(vec.begin() + index);
    2771 }
    2772 
    2773 #else // #if VMA_USE_STL_VECTOR
    2774 
    2775 /* Class with interface compatible with subset of std::vector.
    2776 T must be POD because constructors and destructors are not called and memcpy is
    2777 used for these objects. */
    2778 template<typename T, typename AllocatorT>
    2779 class VmaVector
    2780 {
    2781 public:
    2782  typedef T value_type;
    2783 
    2784  VmaVector(const AllocatorT& allocator) :
    2785  m_Allocator(allocator),
    2786  m_pArray(VMA_NULL),
    2787  m_Count(0),
    2788  m_Capacity(0)
    2789  {
    2790  }
    2791 
    2792  VmaVector(size_t count, const AllocatorT& allocator) :
    2793  m_Allocator(allocator),
    2794  m_pArray(count ? (T*)VmaAllocateArray<T>(allocator.m_pCallbacks, count) : VMA_NULL),
    2795  m_Count(count),
    2796  m_Capacity(count)
    2797  {
    2798  }
    2799 
    2800  VmaVector(const VmaVector<T, AllocatorT>& src) :
    2801  m_Allocator(src.m_Allocator),
    2802  m_pArray(src.m_Count ? (T*)VmaAllocateArray<T>(src.m_Allocator.m_pCallbacks, src.m_Count) : VMA_NULL),
    2803  m_Count(src.m_Count),
    2804  m_Capacity(src.m_Count)
    2805  {
    2806  if(m_Count != 0)
    2807  {
    2808  memcpy(m_pArray, src.m_pArray, m_Count * sizeof(T));
    2809  }
    2810  }
    2811 
    2812  ~VmaVector()
    2813  {
    2814  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
    2815  }
    2816 
    2817  VmaVector& operator=(const VmaVector<T, AllocatorT>& rhs)
    2818  {
    2819  if(&rhs != this)
    2820  {
    2821  resize(rhs.m_Count);
    2822  if(m_Count != 0)
    2823  {
    2824  memcpy(m_pArray, rhs.m_pArray, m_Count * sizeof(T));
    2825  }
    2826  }
    2827  return *this;
    2828  }
    2829 
    2830  bool empty() const { return m_Count == 0; }
    2831  size_t size() const { return m_Count; }
    2832  T* data() { return m_pArray; }
    2833  const T* data() const { return m_pArray; }
    2834 
    2835  T& operator[](size_t index)
    2836  {
    2837  VMA_HEAVY_ASSERT(index < m_Count);
    2838  return m_pArray[index];
    2839  }
    2840  const T& operator[](size_t index) const
    2841  {
    2842  VMA_HEAVY_ASSERT(index < m_Count);
    2843  return m_pArray[index];
    2844  }
    2845 
    2846  T& front()
    2847  {
    2848  VMA_HEAVY_ASSERT(m_Count > 0);
    2849  return m_pArray[0];
    2850  }
    2851  const T& front() const
    2852  {
    2853  VMA_HEAVY_ASSERT(m_Count > 0);
    2854  return m_pArray[0];
    2855  }
    2856  T& back()
    2857  {
    2858  VMA_HEAVY_ASSERT(m_Count > 0);
    2859  return m_pArray[m_Count - 1];
    2860  }
    2861  const T& back() const
    2862  {
    2863  VMA_HEAVY_ASSERT(m_Count > 0);
    2864  return m_pArray[m_Count - 1];
    2865  }
    2866 
    2867  void reserve(size_t newCapacity, bool freeMemory = false)
    2868  {
    2869  newCapacity = VMA_MAX(newCapacity, m_Count);
    2870 
    2871  if((newCapacity < m_Capacity) && !freeMemory)
    2872  {
    2873  newCapacity = m_Capacity;
    2874  }
    2875 
    2876  if(newCapacity != m_Capacity)
    2877  {
    2878  T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator, newCapacity) : VMA_NULL;
    2879  if(m_Count != 0)
    2880  {
    2881  memcpy(newArray, m_pArray, m_Count * sizeof(T));
    2882  }
    2883  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
    2884  m_Capacity = newCapacity;
    2885  m_pArray = newArray;
    2886  }
    2887  }
    2888 
    2889  void resize(size_t newCount, bool freeMemory = false)
    2890  {
    2891  size_t newCapacity = m_Capacity;
    2892  if(newCount > m_Capacity)
    2893  {
    2894  newCapacity = VMA_MAX(newCount, VMA_MAX(m_Capacity * 3 / 2, (size_t)8));
    2895  }
    2896  else if(freeMemory)
    2897  {
    2898  newCapacity = newCount;
    2899  }
    2900 
    2901  if(newCapacity != m_Capacity)
    2902  {
    2903  T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator.m_pCallbacks, newCapacity) : VMA_NULL;
    2904  const size_t elementsToCopy = VMA_MIN(m_Count, newCount);
    2905  if(elementsToCopy != 0)
    2906  {
    2907  memcpy(newArray, m_pArray, elementsToCopy * sizeof(T));
    2908  }
    2909  VmaFree(m_Allocator.m_pCallbacks, m_pArray);
    2910  m_Capacity = newCapacity;
    2911  m_pArray = newArray;
    2912  }
    2913 
    2914  m_Count = newCount;
    2915  }
    2916 
    2917  void clear(bool freeMemory = false)
    2918  {
    2919  resize(0, freeMemory);
    2920  }
    2921 
    2922  void insert(size_t index, const T& src)
    2923  {
    2924  VMA_HEAVY_ASSERT(index <= m_Count);
    2925  const size_t oldCount = size();
    2926  resize(oldCount + 1);
    2927  if(index < oldCount)
    2928  {
    2929  memmove(m_pArray + (index + 1), m_pArray + index, (oldCount - index) * sizeof(T));
    2930  }
    2931  m_pArray[index] = src;
    2932  }
    2933 
    2934  void remove(size_t index)
    2935  {
    2936  VMA_HEAVY_ASSERT(index < m_Count);
    2937  const size_t oldCount = size();
    2938  if(index < oldCount - 1)
    2939  {
    2940  memmove(m_pArray + index, m_pArray + (index + 1), (oldCount - index - 1) * sizeof(T));
    2941  }
    2942  resize(oldCount - 1);
    2943  }
    2944 
    2945  void push_back(const T& src)
    2946  {
    2947  const size_t newIndex = size();
    2948  resize(newIndex + 1);
    2949  m_pArray[newIndex] = src;
    2950  }
    2951 
    2952  void pop_back()
    2953  {
    2954  VMA_HEAVY_ASSERT(m_Count > 0);
    2955  resize(size() - 1);
    2956  }
    2957 
    2958  void push_front(const T& src)
    2959  {
    2960  insert(0, src);
    2961  }
    2962 
    2963  void pop_front()
    2964  {
    2965  VMA_HEAVY_ASSERT(m_Count > 0);
    2966  remove(0);
    2967  }
    2968 
    2969  typedef T* iterator;
    2970 
    2971  iterator begin() { return m_pArray; }
    2972  iterator end() { return m_pArray + m_Count; }
    2973 
    2974 private:
    2975  AllocatorT m_Allocator;
    2976  T* m_pArray;
    2977  size_t m_Count;
    2978  size_t m_Capacity;
    2979 };
    2980 
    2981 template<typename T, typename allocatorT>
    2982 static void VmaVectorInsert(VmaVector<T, allocatorT>& vec, size_t index, const T& item)
    2983 {
    2984  vec.insert(index, item);
    2985 }
    2986 
    2987 template<typename T, typename allocatorT>
    2988 static void VmaVectorRemove(VmaVector<T, allocatorT>& vec, size_t index)
    2989 {
    2990  vec.remove(index);
    2991 }
    2992 
    2993 #endif // #if VMA_USE_STL_VECTOR
    2994 
    2995 template<typename CmpLess, typename VectorT>
    2996 size_t VmaVectorInsertSorted(VectorT& vector, const typename VectorT::value_type& value)
    2997 {
    2998  const size_t indexToInsert = VmaBinaryFindFirstNotLess(
    2999  vector.data(),
    3000  vector.data() + vector.size(),
    3001  value,
    3002  CmpLess()) - vector.data();
    3003  VmaVectorInsert(vector, indexToInsert, value);
    3004  return indexToInsert;
    3005 }
    3006 
    3007 template<typename CmpLess, typename VectorT>
    3008 bool VmaVectorRemoveSorted(VectorT& vector, const typename VectorT::value_type& value)
    3009 {
    3010  CmpLess comparator;
    3011  typename VectorT::iterator it = VmaBinaryFindFirstNotLess(
    3012  vector.begin(),
    3013  vector.end(),
    3014  value,
    3015  comparator);
    3016  if((it != vector.end()) && !comparator(*it, value) && !comparator(value, *it))
    3017  {
    3018  size_t indexToRemove = it - vector.begin();
    3019  VmaVectorRemove(vector, indexToRemove);
    3020  return true;
    3021  }
    3022  return false;
    3023 }
    3024 
    3025 template<typename CmpLess, typename VectorT>
    3026 size_t VmaVectorFindSorted(const VectorT& vector, const typename VectorT::value_type& value)
    3027 {
    3028  CmpLess comparator;
    3029  typename VectorT::iterator it = VmaBinaryFindFirstNotLess(
    3030  vector.data(),
    3031  vector.data() + vector.size(),
    3032  value,
    3033  comparator);
    3034  if(it != vector.size() && !comparator(*it, value) && !comparator(value, *it))
    3035  {
    3036  return it - vector.begin();
    3037  }
    3038  else
    3039  {
    3040  return vector.size();
    3041  }
    3042 }
    3043 
    3045 // class VmaPoolAllocator
    3046 
    3047 /*
    3048 Allocator for objects of type T using a list of arrays (pools) to speed up
    3049 allocation. Number of elements that can be allocated is not bounded because
    3050 allocator can create multiple blocks.
    3051 */
    3052 template<typename T>
    3053 class VmaPoolAllocator
    3054 {
    3055 public:
    3056  VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, size_t itemsPerBlock);
    3057  ~VmaPoolAllocator();
    3058  void Clear();
    3059  T* Alloc();
    3060  void Free(T* ptr);
    3061 
    3062 private:
    3063  union Item
    3064  {
    3065  uint32_t NextFreeIndex;
    3066  T Value;
    3067  };
    3068 
    3069  struct ItemBlock
    3070  {
    3071  Item* pItems;
    3072  uint32_t FirstFreeIndex;
    3073  };
    3074 
    3075  const VkAllocationCallbacks* m_pAllocationCallbacks;
    3076  size_t m_ItemsPerBlock;
    3077  VmaVector< ItemBlock, VmaStlAllocator<ItemBlock> > m_ItemBlocks;
    3078 
    3079  ItemBlock& CreateNewBlock();
    3080 };
    3081 
    3082 template<typename T>
    3083 VmaPoolAllocator<T>::VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, size_t itemsPerBlock) :
    3084  m_pAllocationCallbacks(pAllocationCallbacks),
    3085  m_ItemsPerBlock(itemsPerBlock),
    3086  m_ItemBlocks(VmaStlAllocator<ItemBlock>(pAllocationCallbacks))
    3087 {
    3088  VMA_ASSERT(itemsPerBlock > 0);
    3089 }
    3090 
    3091 template<typename T>
    3092 VmaPoolAllocator<T>::~VmaPoolAllocator()
    3093 {
    3094  Clear();
    3095 }
    3096 
    3097 template<typename T>
    3098 void VmaPoolAllocator<T>::Clear()
    3099 {
    3100  for(size_t i = m_ItemBlocks.size(); i--; )
    3101  vma_delete_array(m_pAllocationCallbacks, m_ItemBlocks[i].pItems, m_ItemsPerBlock);
    3102  m_ItemBlocks.clear();
    3103 }
    3104 
    3105 template<typename T>
    3106 T* VmaPoolAllocator<T>::Alloc()
    3107 {
    3108  for(size_t i = m_ItemBlocks.size(); i--; )
    3109  {
    3110  ItemBlock& block = m_ItemBlocks[i];
    3111  // This block has some free items: Use first one.
    3112  if(block.FirstFreeIndex != UINT32_MAX)
    3113  {
    3114  Item* const pItem = &block.pItems[block.FirstFreeIndex];
    3115  block.FirstFreeIndex = pItem->NextFreeIndex;
    3116  return &pItem->Value;
    3117  }
    3118  }
    3119 
    3120  // No block has free item: Create new one and use it.
    3121  ItemBlock& newBlock = CreateNewBlock();
    3122  Item* const pItem = &newBlock.pItems[0];
    3123  newBlock.FirstFreeIndex = pItem->NextFreeIndex;
    3124  return &pItem->Value;
    3125 }
    3126 
    3127 template<typename T>
    3128 void VmaPoolAllocator<T>::Free(T* ptr)
    3129 {
    3130  // Search all memory blocks to find ptr.
    3131  for(size_t i = 0; i < m_ItemBlocks.size(); ++i)
    3132  {
    3133  ItemBlock& block = m_ItemBlocks[i];
    3134 
    3135  // Casting to union.
    3136  Item* pItemPtr;
    3137  memcpy(&pItemPtr, &ptr, sizeof(pItemPtr));
    3138 
    3139  // Check if pItemPtr is in address range of this block.
    3140  if((pItemPtr >= block.pItems) && (pItemPtr < block.pItems + m_ItemsPerBlock))
    3141  {
    3142  const uint32_t index = static_cast<uint32_t>(pItemPtr - block.pItems);
    3143  pItemPtr->NextFreeIndex = block.FirstFreeIndex;
    3144  block.FirstFreeIndex = index;
    3145  return;
    3146  }
    3147  }
    3148  VMA_ASSERT(0 && "Pointer doesn't belong to this memory pool.");
    3149 }
    3150 
    3151 template<typename T>
    3152 typename VmaPoolAllocator<T>::ItemBlock& VmaPoolAllocator<T>::CreateNewBlock()
    3153 {
    3154  ItemBlock newBlock = {
    3155  vma_new_array(m_pAllocationCallbacks, Item, m_ItemsPerBlock), 0 };
    3156 
    3157  m_ItemBlocks.push_back(newBlock);
    3158 
    3159  // Setup singly-linked list of all free items in this block.
    3160  for(uint32_t i = 0; i < m_ItemsPerBlock - 1; ++i)
    3161  newBlock.pItems[i].NextFreeIndex = i + 1;
    3162  newBlock.pItems[m_ItemsPerBlock - 1].NextFreeIndex = UINT32_MAX;
    3163  return m_ItemBlocks.back();
    3164 }
    3165 
    3167 // class VmaRawList, VmaList
    3168 
    3169 #if VMA_USE_STL_LIST
    3170 
    3171 #define VmaList std::list
    3172 
    3173 #else // #if VMA_USE_STL_LIST
    3174 
    3175 template<typename T>
    3176 struct VmaListItem
    3177 {
    3178  VmaListItem* pPrev;
    3179  VmaListItem* pNext;
    3180  T Value;
    3181 };
    3182 
    3183 // Doubly linked list.
    3184 template<typename T>
    3185 class VmaRawList
    3186 {
    3187 public:
    3188  typedef VmaListItem<T> ItemType;
    3189 
    3190  VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks);
    3191  ~VmaRawList();
    3192  void Clear();
    3193 
    3194  size_t GetCount() const { return m_Count; }
    3195  bool IsEmpty() const { return m_Count == 0; }
    3196 
    3197  ItemType* Front() { return m_pFront; }
    3198  const ItemType* Front() const { return m_pFront; }
    3199  ItemType* Back() { return m_pBack; }
    3200  const ItemType* Back() const { return m_pBack; }
    3201 
    3202  ItemType* PushBack();
    3203  ItemType* PushFront();
    3204  ItemType* PushBack(const T& value);
    3205  ItemType* PushFront(const T& value);
    3206  void PopBack();
    3207  void PopFront();
    3208 
    3209  // Item can be null - it means PushBack.
    3210  ItemType* InsertBefore(ItemType* pItem);
    3211  // Item can be null - it means PushFront.
    3212  ItemType* InsertAfter(ItemType* pItem);
    3213 
    3214  ItemType* InsertBefore(ItemType* pItem, const T& value);
    3215  ItemType* InsertAfter(ItemType* pItem, const T& value);
    3216 
    3217  void Remove(ItemType* pItem);
    3218 
    3219 private:
    3220  const VkAllocationCallbacks* const m_pAllocationCallbacks;
    3221  VmaPoolAllocator<ItemType> m_ItemAllocator;
    3222  ItemType* m_pFront;
    3223  ItemType* m_pBack;
    3224  size_t m_Count;
    3225 
    3226  // Declared not defined, to block copy constructor and assignment operator.
    3227  VmaRawList(const VmaRawList<T>& src);
    3228  VmaRawList<T>& operator=(const VmaRawList<T>& rhs);
    3229 };
    3230 
    3231 template<typename T>
    3232 VmaRawList<T>::VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks) :
    3233  m_pAllocationCallbacks(pAllocationCallbacks),
    3234  m_ItemAllocator(pAllocationCallbacks, 128),
    3235  m_pFront(VMA_NULL),
    3236  m_pBack(VMA_NULL),
    3237  m_Count(0)
    3238 {
    3239 }
    3240 
    3241 template<typename T>
    3242 VmaRawList<T>::~VmaRawList()
    3243 {
    3244  // Intentionally not calling Clear, because that would be unnecessary
    3245  // computations to return all items to m_ItemAllocator as free.
    3246 }
    3247 
    3248 template<typename T>
    3249 void VmaRawList<T>::Clear()
    3250 {
    3251  if(IsEmpty() == false)
    3252  {
    3253  ItemType* pItem = m_pBack;
    3254  while(pItem != VMA_NULL)
    3255  {
    3256  ItemType* const pPrevItem = pItem->pPrev;
    3257  m_ItemAllocator.Free(pItem);
    3258  pItem = pPrevItem;
    3259  }
    3260  m_pFront = VMA_NULL;
    3261  m_pBack = VMA_NULL;
    3262  m_Count = 0;
    3263  }
    3264 }
    3265 
    3266 template<typename T>
    3267 VmaListItem<T>* VmaRawList<T>::PushBack()
    3268 {
    3269  ItemType* const pNewItem = m_ItemAllocator.Alloc();
    3270  pNewItem->pNext = VMA_NULL;
    3271  if(IsEmpty())
    3272  {
    3273  pNewItem->pPrev = VMA_NULL;
    3274  m_pFront = pNewItem;
    3275  m_pBack = pNewItem;
    3276  m_Count = 1;
    3277  }
    3278  else
    3279  {
    3280  pNewItem->pPrev = m_pBack;
    3281  m_pBack->pNext = pNewItem;
    3282  m_pBack = pNewItem;
    3283  ++m_Count;
    3284  }
    3285  return pNewItem;
    3286 }
    3287 
    3288 template<typename T>
    3289 VmaListItem<T>* VmaRawList<T>::PushFront()
    3290 {
    3291  ItemType* const pNewItem = m_ItemAllocator.Alloc();
    3292  pNewItem->pPrev = VMA_NULL;
    3293  if(IsEmpty())
    3294  {
    3295  pNewItem->pNext = VMA_NULL;
    3296  m_pFront = pNewItem;
    3297  m_pBack = pNewItem;
    3298  m_Count = 1;
    3299  }
    3300  else
    3301  {
    3302  pNewItem->pNext = m_pFront;
    3303  m_pFront->pPrev = pNewItem;
    3304  m_pFront = pNewItem;
    3305  ++m_Count;
    3306  }
    3307  return pNewItem;
    3308 }
    3309 
    3310 template<typename T>
    3311 VmaListItem<T>* VmaRawList<T>::PushBack(const T& value)
    3312 {
    3313  ItemType* const pNewItem = PushBack();
    3314  pNewItem->Value = value;
    3315  return pNewItem;
    3316 }
    3317 
    3318 template<typename T>
    3319 VmaListItem<T>* VmaRawList<T>::PushFront(const T& value)
    3320 {
    3321  ItemType* const pNewItem = PushFront();
    3322  pNewItem->Value = value;
    3323  return pNewItem;
    3324 }
    3325 
    3326 template<typename T>
    3327 void VmaRawList<T>::PopBack()
    3328 {
    3329  VMA_HEAVY_ASSERT(m_Count > 0);
    3330  ItemType* const pBackItem = m_pBack;
    3331  ItemType* const pPrevItem = pBackItem->pPrev;
    3332  if(pPrevItem != VMA_NULL)
    3333  {
    3334  pPrevItem->pNext = VMA_NULL;
    3335  }
    3336  m_pBack = pPrevItem;
    3337  m_ItemAllocator.Free(pBackItem);
    3338  --m_Count;
    3339 }
    3340 
    3341 template<typename T>
    3342 void VmaRawList<T>::PopFront()
    3343 {
    3344  VMA_HEAVY_ASSERT(m_Count > 0);
    3345  ItemType* const pFrontItem = m_pFront;
    3346  ItemType* const pNextItem = pFrontItem->pNext;
    3347  if(pNextItem != VMA_NULL)
    3348  {
    3349  pNextItem->pPrev = VMA_NULL;
    3350  }
    3351  m_pFront = pNextItem;
    3352  m_ItemAllocator.Free(pFrontItem);
    3353  --m_Count;
    3354 }
    3355 
    3356 template<typename T>
    3357 void VmaRawList<T>::Remove(ItemType* pItem)
    3358 {
    3359  VMA_HEAVY_ASSERT(pItem != VMA_NULL);
    3360  VMA_HEAVY_ASSERT(m_Count > 0);
    3361 
    3362  if(pItem->pPrev != VMA_NULL)
    3363  {
    3364  pItem->pPrev->pNext = pItem->pNext;
    3365  }
    3366  else
    3367  {
    3368  VMA_HEAVY_ASSERT(m_pFront == pItem);
    3369  m_pFront = pItem->pNext;
    3370  }
    3371 
    3372  if(pItem->pNext != VMA_NULL)
    3373  {
    3374  pItem->pNext->pPrev = pItem->pPrev;
    3375  }
    3376  else
    3377  {
    3378  VMA_HEAVY_ASSERT(m_pBack == pItem);
    3379  m_pBack = pItem->pPrev;
    3380  }
    3381 
    3382  m_ItemAllocator.Free(pItem);
    3383  --m_Count;
    3384 }
    3385 
    3386 template<typename T>
    3387 VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem)
    3388 {
    3389  if(pItem != VMA_NULL)
    3390  {
    3391  ItemType* const prevItem = pItem->pPrev;
    3392  ItemType* const newItem = m_ItemAllocator.Alloc();
    3393  newItem->pPrev = prevItem;
    3394  newItem->pNext = pItem;
    3395  pItem->pPrev = newItem;
    3396  if(prevItem != VMA_NULL)
    3397  {
    3398  prevItem->pNext = newItem;
    3399  }
    3400  else
    3401  {
    3402  VMA_HEAVY_ASSERT(m_pFront == pItem);
    3403  m_pFront = newItem;
    3404  }
    3405  ++m_Count;
    3406  return newItem;
    3407  }
    3408  else
    3409  return PushBack();
    3410 }
    3411 
    3412 template<typename T>
    3413 VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem)
    3414 {
    3415  if(pItem != VMA_NULL)
    3416  {
    3417  ItemType* const nextItem = pItem->pNext;
    3418  ItemType* const newItem = m_ItemAllocator.Alloc();
    3419  newItem->pNext = nextItem;
    3420  newItem->pPrev = pItem;
    3421  pItem->pNext = newItem;
    3422  if(nextItem != VMA_NULL)
    3423  {
    3424  nextItem->pPrev = newItem;
    3425  }
    3426  else
    3427  {
    3428  VMA_HEAVY_ASSERT(m_pBack == pItem);
    3429  m_pBack = newItem;
    3430  }
    3431  ++m_Count;
    3432  return newItem;
    3433  }
    3434  else
    3435  return PushFront();
    3436 }
    3437 
    3438 template<typename T>
    3439 VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem, const T& value)
    3440 {
    3441  ItemType* const newItem = InsertBefore(pItem);
    3442  newItem->Value = value;
    3443  return newItem;
    3444 }
    3445 
    3446 template<typename T>
    3447 VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem, const T& value)
    3448 {
    3449  ItemType* const newItem = InsertAfter(pItem);
    3450  newItem->Value = value;
    3451  return newItem;
    3452 }
    3453 
    3454 template<typename T, typename AllocatorT>
    3455 class VmaList
    3456 {
    3457 public:
    3458  class iterator
    3459  {
    3460  public:
    3461  iterator() :
    3462  m_pList(VMA_NULL),
    3463  m_pItem(VMA_NULL)
    3464  {
    3465  }
    3466 
    3467  T& operator*() const
    3468  {
    3469  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3470  return m_pItem->Value;
    3471  }
    3472  T* operator->() const
    3473  {
    3474  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3475  return &m_pItem->Value;
    3476  }
    3477 
    3478  iterator& operator++()
    3479  {
    3480  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3481  m_pItem = m_pItem->pNext;
    3482  return *this;
    3483  }
    3484  iterator& operator--()
    3485  {
    3486  if(m_pItem != VMA_NULL)
    3487  {
    3488  m_pItem = m_pItem->pPrev;
    3489  }
    3490  else
    3491  {
    3492  VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
    3493  m_pItem = m_pList->Back();
    3494  }
    3495  return *this;
    3496  }
    3497 
    3498  iterator operator++(int)
    3499  {
    3500  iterator result = *this;
    3501  ++*this;
    3502  return result;
    3503  }
    3504  iterator operator--(int)
    3505  {
    3506  iterator result = *this;
    3507  --*this;
    3508  return result;
    3509  }
    3510 
    3511  bool operator==(const iterator& rhs) const
    3512  {
    3513  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
    3514  return m_pItem == rhs.m_pItem;
    3515  }
    3516  bool operator!=(const iterator& rhs) const
    3517  {
    3518  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
    3519  return m_pItem != rhs.m_pItem;
    3520  }
    3521 
    3522  private:
    3523  VmaRawList<T>* m_pList;
    3524  VmaListItem<T>* m_pItem;
    3525 
    3526  iterator(VmaRawList<T>* pList, VmaListItem<T>* pItem) :
    3527  m_pList(pList),
    3528  m_pItem(pItem)
    3529  {
    3530  }
    3531 
    3532  friend class VmaList<T, AllocatorT>;
    3533  };
    3534 
    3535  class const_iterator
    3536  {
    3537  public:
    3538  const_iterator() :
    3539  m_pList(VMA_NULL),
    3540  m_pItem(VMA_NULL)
    3541  {
    3542  }
    3543 
    3544  const_iterator(const iterator& src) :
    3545  m_pList(src.m_pList),
    3546  m_pItem(src.m_pItem)
    3547  {
    3548  }
    3549 
    3550  const T& operator*() const
    3551  {
    3552  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3553  return m_pItem->Value;
    3554  }
    3555  const T* operator->() const
    3556  {
    3557  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3558  return &m_pItem->Value;
    3559  }
    3560 
    3561  const_iterator& operator++()
    3562  {
    3563  VMA_HEAVY_ASSERT(m_pItem != VMA_NULL);
    3564  m_pItem = m_pItem->pNext;
    3565  return *this;
    3566  }
    3567  const_iterator& operator--()
    3568  {
    3569  if(m_pItem != VMA_NULL)
    3570  {
    3571  m_pItem = m_pItem->pPrev;
    3572  }
    3573  else
    3574  {
    3575  VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
    3576  m_pItem = m_pList->Back();
    3577  }
    3578  return *this;
    3579  }
    3580 
    3581  const_iterator operator++(int)
    3582  {
    3583  const_iterator result = *this;
    3584  ++*this;
    3585  return result;
    3586  }
    3587  const_iterator operator--(int)
    3588  {
    3589  const_iterator result = *this;
    3590  --*this;
    3591  return result;
    3592  }
    3593 
    3594  bool operator==(const const_iterator& rhs) const
    3595  {
    3596  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
    3597  return m_pItem == rhs.m_pItem;
    3598  }
    3599  bool operator!=(const const_iterator& rhs) const
    3600  {
    3601  VMA_HEAVY_ASSERT(m_pList == rhs.m_pList);
    3602  return m_pItem != rhs.m_pItem;
    3603  }
    3604 
    3605  private:
    3606  const_iterator(const VmaRawList<T>* pList, const VmaListItem<T>* pItem) :
    3607  m_pList(pList),
    3608  m_pItem(pItem)
    3609  {
    3610  }
    3611 
    3612  const VmaRawList<T>* m_pList;
    3613  const VmaListItem<T>* m_pItem;
    3614 
    3615  friend class VmaList<T, AllocatorT>;
    3616  };
    3617 
    3618  VmaList(const AllocatorT& allocator) : m_RawList(allocator.m_pCallbacks) { }
    3619 
    3620  bool empty() const { return m_RawList.IsEmpty(); }
    3621  size_t size() const { return m_RawList.GetCount(); }
    3622 
    3623  iterator begin() { return iterator(&m_RawList, m_RawList.Front()); }
    3624  iterator end() { return iterator(&m_RawList, VMA_NULL); }
    3625 
    3626  const_iterator cbegin() const { return const_iterator(&m_RawList, m_RawList.Front()); }
    3627  const_iterator cend() const { return const_iterator(&m_RawList, VMA_NULL); }
    3628 
    3629  void clear() { m_RawList.Clear(); }
    3630  void push_back(const T& value) { m_RawList.PushBack(value); }
    3631  void erase(iterator it) { m_RawList.Remove(it.m_pItem); }
    3632  iterator insert(iterator it, const T& value) { return iterator(&m_RawList, m_RawList.InsertBefore(it.m_pItem, value)); }
    3633 
    3634 private:
    3635  VmaRawList<T> m_RawList;
    3636 };
    3637 
    3638 #endif // #if VMA_USE_STL_LIST
    3639 
    3641 // class VmaMap
    3642 
    3643 // Unused in this version.
    3644 #if 0
    3645 
    3646 #if VMA_USE_STL_UNORDERED_MAP
    3647 
    3648 #define VmaPair std::pair
    3649 
    3650 #define VMA_MAP_TYPE(KeyT, ValueT) \
    3651  std::unordered_map< KeyT, ValueT, std::hash<KeyT>, std::equal_to<KeyT>, VmaStlAllocator< std::pair<KeyT, ValueT> > >
    3652 
    3653 #else // #if VMA_USE_STL_UNORDERED_MAP
    3654 
    3655 template<typename T1, typename T2>
    3656 struct VmaPair
    3657 {
    3658  T1 first;
    3659  T2 second;
    3660 
    3661  VmaPair() : first(), second() { }
    3662  VmaPair(const T1& firstSrc, const T2& secondSrc) : first(firstSrc), second(secondSrc) { }
    3663 };
    3664 
    3665 /* Class compatible with subset of interface of std::unordered_map.
    3666 KeyT, ValueT must be POD because they will be stored in VmaVector.
    3667 */
    3668 template<typename KeyT, typename ValueT>
    3669 class VmaMap
    3670 {
    3671 public:
    3672  typedef VmaPair<KeyT, ValueT> PairType;
    3673  typedef PairType* iterator;
    3674 
    3675  VmaMap(const VmaStlAllocator<PairType>& allocator) : m_Vector(allocator) { }
    3676 
    3677  iterator begin() { return m_Vector.begin(); }
    3678  iterator end() { return m_Vector.end(); }
    3679 
    3680  void insert(const PairType& pair);
    3681  iterator find(const KeyT& key);
    3682  void erase(iterator it);
    3683 
    3684 private:
    3685  VmaVector< PairType, VmaStlAllocator<PairType> > m_Vector;
    3686 };
    3687 
    3688 #define VMA_MAP_TYPE(KeyT, ValueT) VmaMap<KeyT, ValueT>
    3689 
    3690 template<typename FirstT, typename SecondT>
    3691 struct VmaPairFirstLess
    3692 {
    3693  bool operator()(const VmaPair<FirstT, SecondT>& lhs, const VmaPair<FirstT, SecondT>& rhs) const
    3694  {
    3695  return lhs.first < rhs.first;
    3696  }
    3697  bool operator()(const VmaPair<FirstT, SecondT>& lhs, const FirstT& rhsFirst) const
    3698  {
    3699  return lhs.first < rhsFirst;
    3700  }
    3701 };
    3702 
    3703 template<typename KeyT, typename ValueT>
    3704 void VmaMap<KeyT, ValueT>::insert(const PairType& pair)
    3705 {
    3706  const size_t indexToInsert = VmaBinaryFindFirstNotLess(
    3707  m_Vector.data(),
    3708  m_Vector.data() + m_Vector.size(),
    3709  pair,
    3710  VmaPairFirstLess<KeyT, ValueT>()) - m_Vector.data();
    3711  VmaVectorInsert(m_Vector, indexToInsert, pair);
    3712 }
    3713 
    3714 template<typename KeyT, typename ValueT>
    3715 VmaPair<KeyT, ValueT>* VmaMap<KeyT, ValueT>::find(const KeyT& key)
    3716 {
    3717  PairType* it = VmaBinaryFindFirstNotLess(
    3718  m_Vector.data(),
    3719  m_Vector.data() + m_Vector.size(),
    3720  key,
    3721  VmaPairFirstLess<KeyT, ValueT>());
    3722  if((it != m_Vector.end()) && (it->first == key))
    3723  {
    3724  return it;
    3725  }
    3726  else
    3727  {
    3728  return m_Vector.end();
    3729  }
    3730 }
    3731 
    3732 template<typename KeyT, typename ValueT>
    3733 void VmaMap<KeyT, ValueT>::erase(iterator it)
    3734 {
    3735  VmaVectorRemove(m_Vector, it - m_Vector.begin());
    3736 }
    3737 
    3738 #endif // #if VMA_USE_STL_UNORDERED_MAP
    3739 
    3740 #endif // #if 0
    3741 
    3743 
    3744 class VmaDeviceMemoryBlock;
    3745 
    3746 struct VmaAllocation_T
    3747 {
    3748 private:
    3749  static const uint8_t MAP_COUNT_FLAG_PERSISTENT_MAP = 0x80;
    3750 
    3751  enum FLAGS
    3752  {
    3753  FLAG_USER_DATA_STRING = 0x01,
    3754  };
    3755 
    3756 public:
    3757  enum ALLOCATION_TYPE
    3758  {
    3759  ALLOCATION_TYPE_NONE,
    3760  ALLOCATION_TYPE_BLOCK,
    3761  ALLOCATION_TYPE_DEDICATED,
    3762  };
    3763 
    3764  VmaAllocation_T(uint32_t currentFrameIndex, bool userDataString) :
    3765  m_Alignment(1),
    3766  m_Size(0),
    3767  m_pUserData(VMA_NULL),
    3768  m_LastUseFrameIndex(currentFrameIndex),
    3769  m_Type((uint8_t)ALLOCATION_TYPE_NONE),
    3770  m_SuballocationType((uint8_t)VMA_SUBALLOCATION_TYPE_UNKNOWN),
    3771  m_MapCount(0),
    3772  m_Flags(userDataString ? (uint8_t)FLAG_USER_DATA_STRING : 0)
    3773  {
    3774  }
    3775 
    3776  ~VmaAllocation_T()
    3777  {
    3778  VMA_ASSERT((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) == 0 && "Allocation was not unmapped before destruction.");
    3779 
    3780  // Check if owned string was freed.
    3781  VMA_ASSERT(m_pUserData == VMA_NULL);
    3782  }
    3783 
    3784  void InitBlockAllocation(
    3785  VmaPool hPool,
    3786  VmaDeviceMemoryBlock* block,
    3787  VkDeviceSize offset,
    3788  VkDeviceSize alignment,
    3789  VkDeviceSize size,
    3790  VmaSuballocationType suballocationType,
    3791  bool mapped,
    3792  bool canBecomeLost)
    3793  {
    3794  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
    3795  VMA_ASSERT(block != VMA_NULL);
    3796  m_Type = (uint8_t)ALLOCATION_TYPE_BLOCK;
    3797  m_Alignment = alignment;
    3798  m_Size = size;
    3799  m_MapCount = mapped ? MAP_COUNT_FLAG_PERSISTENT_MAP : 0;
    3800  m_SuballocationType = (uint8_t)suballocationType;
    3801  m_BlockAllocation.m_hPool = hPool;
    3802  m_BlockAllocation.m_Block = block;
    3803  m_BlockAllocation.m_Offset = offset;
    3804  m_BlockAllocation.m_CanBecomeLost = canBecomeLost;
    3805  }
    3806 
    3807  void InitLost()
    3808  {
    3809  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
    3810  VMA_ASSERT(m_LastUseFrameIndex.load() == VMA_FRAME_INDEX_LOST);
    3811  m_Type = (uint8_t)ALLOCATION_TYPE_BLOCK;
    3812  m_BlockAllocation.m_hPool = VK_NULL_HANDLE;
    3813  m_BlockAllocation.m_Block = VMA_NULL;
    3814  m_BlockAllocation.m_Offset = 0;
    3815  m_BlockAllocation.m_CanBecomeLost = true;
    3816  }
    3817 
    3818  void ChangeBlockAllocation(
    3819  VmaAllocator hAllocator,
    3820  VmaDeviceMemoryBlock* block,
    3821  VkDeviceSize offset);
    3822 
    3823  // pMappedData not null means allocation is created with MAPPED flag.
    3824  void InitDedicatedAllocation(
    3825  uint32_t memoryTypeIndex,
    3826  VkDeviceMemory hMemory,
    3827  VmaSuballocationType suballocationType,
    3828  void* pMappedData,
    3829  VkDeviceSize size)
    3830  {
    3831  VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
    3832  VMA_ASSERT(hMemory != VK_NULL_HANDLE);
    3833  m_Type = (uint8_t)ALLOCATION_TYPE_DEDICATED;
    3834  m_Alignment = 0;
    3835  m_Size = size;
    3836  m_SuballocationType = (uint8_t)suballocationType;
    3837  m_MapCount = (pMappedData != VMA_NULL) ? MAP_COUNT_FLAG_PERSISTENT_MAP : 0;
    3838  m_DedicatedAllocation.m_MemoryTypeIndex = memoryTypeIndex;
    3839  m_DedicatedAllocation.m_hMemory = hMemory;
    3840  m_DedicatedAllocation.m_pMappedData = pMappedData;
    3841  }
    3842 
    3843  ALLOCATION_TYPE GetType() const { return (ALLOCATION_TYPE)m_Type; }
    3844  VkDeviceSize GetAlignment() const { return m_Alignment; }
    3845  VkDeviceSize GetSize() const { return m_Size; }
    3846  bool IsUserDataString() const { return (m_Flags & FLAG_USER_DATA_STRING) != 0; }
    3847  void* GetUserData() const { return m_pUserData; }
    3848  void SetUserData(VmaAllocator hAllocator, void* pUserData);
    3849  VmaSuballocationType GetSuballocationType() const { return (VmaSuballocationType)m_SuballocationType; }
    3850 
    3851  VmaDeviceMemoryBlock* GetBlock() const
    3852  {
    3853  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
    3854  return m_BlockAllocation.m_Block;
    3855  }
    3856  VkDeviceSize GetOffset() const;
    3857  VkDeviceMemory GetMemory() const;
    3858  uint32_t GetMemoryTypeIndex() const;
    3859  bool IsPersistentMap() const { return (m_MapCount & MAP_COUNT_FLAG_PERSISTENT_MAP) != 0; }
    3860  void* GetMappedData() const;
    3861  bool CanBecomeLost() const;
    3862  VmaPool GetPool() const;
    3863 
    3864  uint32_t GetLastUseFrameIndex() const
    3865  {
    3866  return m_LastUseFrameIndex.load();
    3867  }
    3868  bool CompareExchangeLastUseFrameIndex(uint32_t& expected, uint32_t desired)
    3869  {
    3870  return m_LastUseFrameIndex.compare_exchange_weak(expected, desired);
    3871  }
    3872  /*
    3873  - If hAllocation.LastUseFrameIndex + frameInUseCount < allocator.CurrentFrameIndex,
    3874  makes it lost by setting LastUseFrameIndex = VMA_FRAME_INDEX_LOST and returns true.
    3875  - Else, returns false.
    3876 
    3877  If hAllocation is already lost, assert - you should not call it then.
    3878  If hAllocation was not created with CAN_BECOME_LOST_BIT, assert.
    3879  */
    3880  bool MakeLost(uint32_t currentFrameIndex, uint32_t frameInUseCount);
    3881 
    3882  void DedicatedAllocCalcStatsInfo(VmaStatInfo& outInfo)
    3883  {
    3884  VMA_ASSERT(m_Type == ALLOCATION_TYPE_DEDICATED);
    3885  outInfo.blockCount = 1;
    3886  outInfo.allocationCount = 1;
    3887  outInfo.unusedRangeCount = 0;
    3888  outInfo.usedBytes = m_Size;
    3889  outInfo.unusedBytes = 0;
    3890  outInfo.allocationSizeMin = outInfo.allocationSizeMax = m_Size;
    3891  outInfo.unusedRangeSizeMin = UINT64_MAX;
    3892  outInfo.unusedRangeSizeMax = 0;
    3893  }
    3894 
    3895  void BlockAllocMap();
    3896  void BlockAllocUnmap();
    3897  VkResult DedicatedAllocMap(VmaAllocator hAllocator, void** ppData);
    3898  void DedicatedAllocUnmap(VmaAllocator hAllocator);
    3899 
    3900 private:
    3901  VkDeviceSize m_Alignment;
    3902  VkDeviceSize m_Size;
    3903  void* m_pUserData;
    3904  VMA_ATOMIC_UINT32 m_LastUseFrameIndex;
    3905  uint8_t m_Type; // ALLOCATION_TYPE
    3906  uint8_t m_SuballocationType; // VmaSuballocationType
    3907  // Bit 0x80 is set when allocation was created with VMA_ALLOCATION_CREATE_MAPPED_BIT.
    3908  // Bits with mask 0x7F are reference counter for vmaMapMemory()/vmaUnmapMemory().
    3909  uint8_t m_MapCount;
    3910  uint8_t m_Flags; // enum FLAGS
    3911 
    3912  // Allocation out of VmaDeviceMemoryBlock.
    3913  struct BlockAllocation
    3914  {
    3915  VmaPool m_hPool; // Null if belongs to general memory.
    3916  VmaDeviceMemoryBlock* m_Block;
    3917  VkDeviceSize m_Offset;
    3918  bool m_CanBecomeLost;
    3919  };
    3920 
    3921  // Allocation for an object that has its own private VkDeviceMemory.
    3922  struct DedicatedAllocation
    3923  {
    3924  uint32_t m_MemoryTypeIndex;
    3925  VkDeviceMemory m_hMemory;
    3926  void* m_pMappedData; // Not null means memory is mapped.
    3927  };
    3928 
    3929  union
    3930  {
    3931  // Allocation out of VmaDeviceMemoryBlock.
    3932  BlockAllocation m_BlockAllocation;
    3933  // Allocation for an object that has its own private VkDeviceMemory.
    3934  DedicatedAllocation m_DedicatedAllocation;
    3935  };
    3936 
    3937  void FreeUserDataString(VmaAllocator hAllocator);
    3938 };
    3939 
    3940 /*
    3941 Represents a region of VmaDeviceMemoryBlock that is either assigned and returned as
    3942 allocated memory block or free.
    3943 */
    3944 struct VmaSuballocation
    3945 {
    3946  VkDeviceSize offset;
    3947  VkDeviceSize size;
    3948  VmaAllocation hAllocation;
    3949  VmaSuballocationType type;
    3950 };
    3951 
    3952 typedef VmaList< VmaSuballocation, VmaStlAllocator<VmaSuballocation> > VmaSuballocationList;
    3953 
    3954 // Cost of one additional allocation lost, as equivalent in bytes.
    3955 static const VkDeviceSize VMA_LOST_ALLOCATION_COST = 1048576;
    3956 
    3957 /*
    3958 Parameters of planned allocation inside a VmaDeviceMemoryBlock.
    3959 
    3960 If canMakeOtherLost was false:
    3961 - item points to a FREE suballocation.
    3962 - itemsToMakeLostCount is 0.
    3963 
    3964 If canMakeOtherLost was true:
    3965 - item points to first of sequence of suballocations, which are either FREE,
    3966  or point to VmaAllocations that can become lost.
    3967 - itemsToMakeLostCount is the number of VmaAllocations that need to be made lost for
    3968  the requested allocation to succeed.
    3969 */
    3970 struct VmaAllocationRequest
    3971 {
    3972  VkDeviceSize offset;
    3973  VkDeviceSize sumFreeSize; // Sum size of free items that overlap with proposed allocation.
    3974  VkDeviceSize sumItemSize; // Sum size of items to make lost that overlap with proposed allocation.
    3975  VmaSuballocationList::iterator item;
    3976  size_t itemsToMakeLostCount;
    3977 
    3978  VkDeviceSize CalcCost() const
    3979  {
    3980  return sumItemSize + itemsToMakeLostCount * VMA_LOST_ALLOCATION_COST;
    3981  }
    3982 };
    3983 
    3984 /*
    3985 Data structure used for bookkeeping of allocations and unused ranges of memory
    3986 in a single VkDeviceMemory block.
    3987 */
    3988 class VmaBlockMetadata
    3989 {
    3990 public:
    3991  VmaBlockMetadata(VmaAllocator hAllocator);
    3992  ~VmaBlockMetadata();
    3993  void Init(VkDeviceSize size);
    3994 
    3995  // Validates all data structures inside this object. If not valid, returns false.
    3996  bool Validate() const;
    3997  VkDeviceSize GetSize() const { return m_Size; }
    3998  size_t GetAllocationCount() const { return m_Suballocations.size() - m_FreeCount; }
    3999  VkDeviceSize GetSumFreeSize() const { return m_SumFreeSize; }
    4000  VkDeviceSize GetUnusedRangeSizeMax() const;
    4001  // Returns true if this block is empty - contains only single free suballocation.
    4002  bool IsEmpty() const;
    4003 
    4004  void CalcAllocationStatInfo(VmaStatInfo& outInfo) const;
    4005  void AddPoolStats(VmaPoolStats& inoutStats) const;
    4006 
    4007 #if VMA_STATS_STRING_ENABLED
    4008  void PrintDetailedMap(class VmaJsonWriter& json) const;
    4009 #endif
    4010 
    4011  // Creates trivial request for case when block is empty.
    4012  void CreateFirstAllocationRequest(VmaAllocationRequest* pAllocationRequest);
    4013 
    4014  // Tries to find a place for suballocation with given parameters inside this block.
    4015  // If succeeded, fills pAllocationRequest and returns true.
    4016  // If failed, returns false.
    4017  bool CreateAllocationRequest(
    4018  uint32_t currentFrameIndex,
    4019  uint32_t frameInUseCount,
    4020  VkDeviceSize bufferImageGranularity,
    4021  VkDeviceSize allocSize,
    4022  VkDeviceSize allocAlignment,
    4023  VmaSuballocationType allocType,
    4024  bool canMakeOtherLost,
    4025  VmaAllocationRequest* pAllocationRequest);
    4026 
    4027  bool MakeRequestedAllocationsLost(
    4028  uint32_t currentFrameIndex,
    4029  uint32_t frameInUseCount,
    4030  VmaAllocationRequest* pAllocationRequest);
    4031 
    4032  uint32_t MakeAllocationsLost(uint32_t currentFrameIndex, uint32_t frameInUseCount);
    4033 
    4034  // Makes actual allocation based on request. Request must already be checked and valid.
    4035  void Alloc(
    4036  const VmaAllocationRequest& request,
    4037  VmaSuballocationType type,
    4038  VkDeviceSize allocSize,
    4039  VmaAllocation hAllocation);
    4040 
    4041  // Frees suballocation assigned to given memory region.
    4042  void Free(const VmaAllocation allocation);
    4043  void FreeAtOffset(VkDeviceSize offset);
    4044 
    4045 private:
    4046  VkDeviceSize m_Size;
    4047  uint32_t m_FreeCount;
    4048  VkDeviceSize m_SumFreeSize;
    4049  VmaSuballocationList m_Suballocations;
    4050  // Suballocations that are free and have size greater than certain threshold.
    4051  // Sorted by size, ascending.
    4052  VmaVector< VmaSuballocationList::iterator, VmaStlAllocator< VmaSuballocationList::iterator > > m_FreeSuballocationsBySize;
    4053 
    4054  bool ValidateFreeSuballocationList() const;
    4055 
    4056  // Checks if requested suballocation with given parameters can be placed in given pFreeSuballocItem.
    4057  // If yes, fills pOffset and returns true. If no, returns false.
    4058  bool CheckAllocation(
    4059  uint32_t currentFrameIndex,
    4060  uint32_t frameInUseCount,
    4061  VkDeviceSize bufferImageGranularity,
    4062  VkDeviceSize allocSize,
    4063  VkDeviceSize allocAlignment,
    4064  VmaSuballocationType allocType,
    4065  VmaSuballocationList::const_iterator suballocItem,
    4066  bool canMakeOtherLost,
    4067  VkDeviceSize* pOffset,
    4068  size_t* itemsToMakeLostCount,
    4069  VkDeviceSize* pSumFreeSize,
    4070  VkDeviceSize* pSumItemSize) const;
    4071  // Given free suballocation, it merges it with following one, which must also be free.
    4072  void MergeFreeWithNext(VmaSuballocationList::iterator item);
    4073  // Releases given suballocation, making it free.
    4074  // Merges it with adjacent free suballocations if applicable.
    4075  // Returns iterator to new free suballocation at this place.
    4076  VmaSuballocationList::iterator FreeSuballocation(VmaSuballocationList::iterator suballocItem);
    4077  // Given free suballocation, it inserts it into sorted list of
    4078  // m_FreeSuballocationsBySize if it's suitable.
    4079  void RegisterFreeSuballocation(VmaSuballocationList::iterator item);
    4080  // Given free suballocation, it removes it from sorted list of
    4081  // m_FreeSuballocationsBySize if it's suitable.
    4082  void UnregisterFreeSuballocation(VmaSuballocationList::iterator item);
    4083 };
    4084 
    4085 /*
    4086 Represents a single block of device memory (`VkDeviceMemory`) with all the
    4087 data about its regions (aka suballocations, #VmaAllocation), assigned and free.
    4088 
    4089 Thread-safety: This class must be externally synchronized.
    4090 */
    4091 class VmaDeviceMemoryBlock
    4092 {
    4093 public:
    4094  VmaBlockMetadata m_Metadata;
    4095 
    4096  VmaDeviceMemoryBlock(VmaAllocator hAllocator);
    4097 
    4098  ~VmaDeviceMemoryBlock()
    4099  {
    4100  VMA_ASSERT(m_MapCount == 0 && "VkDeviceMemory block is being destroyed while it is still mapped.");
    4101  VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
    4102  }
    4103 
    4104  // Always call after construction.
    4105  void Init(
    4106  uint32_t newMemoryTypeIndex,
    4107  VkDeviceMemory newMemory,
    4108  VkDeviceSize newSize);
    4109  // Always call before destruction.
    4110  void Destroy(VmaAllocator allocator);
    4111 
    4112  VkDeviceMemory GetDeviceMemory() const { return m_hMemory; }
    4113  uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
    4114  void* GetMappedData() const { return m_pMappedData; }
    4115 
    4116  // Validates all data structures inside this object. If not valid, returns false.
    4117  bool Validate() const;
    4118 
    4119  // ppData can be null.
    4120  VkResult Map(VmaAllocator hAllocator, uint32_t count, void** ppData);
    4121  void Unmap(VmaAllocator hAllocator, uint32_t count);
    4122 
    4123  VkResult BindBufferMemory(
    4124  const VmaAllocator hAllocator,
    4125  const VmaAllocation hAllocation,
    4126  VkBuffer hBuffer);
    4127  VkResult BindImageMemory(
    4128  const VmaAllocator hAllocator,
    4129  const VmaAllocation hAllocation,
    4130  VkImage hImage);
    4131 
    4132 private:
    4133  uint32_t m_MemoryTypeIndex;
    4134  VkDeviceMemory m_hMemory;
    4135 
    4136  // Protects access to m_hMemory so it's not used by multiple threads simultaneously, e.g. vkMapMemory, vkBindBufferMemory.
    4137  // Also protects m_MapCount, m_pMappedData.
    4138  VMA_MUTEX m_Mutex;
    4139  uint32_t m_MapCount;
    4140  void* m_pMappedData;
    4141 };
    4142 
    4143 struct VmaPointerLess
    4144 {
    4145  bool operator()(const void* lhs, const void* rhs) const
    4146  {
    4147  return lhs < rhs;
    4148  }
    4149 };
    4150 
    4151 class VmaDefragmentator;
    4152 
    4153 /*
    4154 Sequence of VmaDeviceMemoryBlock. Represents memory blocks allocated for a specific
    4155 Vulkan memory type.
    4156 
    4157 Synchronized internally with a mutex.
    4158 */
    4159 struct VmaBlockVector
    4160 {
    4161  VmaBlockVector(
    4162  VmaAllocator hAllocator,
    4163  uint32_t memoryTypeIndex,
    4164  VkDeviceSize preferredBlockSize,
    4165  size_t minBlockCount,
    4166  size_t maxBlockCount,
    4167  VkDeviceSize bufferImageGranularity,
    4168  uint32_t frameInUseCount,
    4169  bool isCustomPool);
    4170  ~VmaBlockVector();
    4171 
    4172  VkResult CreateMinBlocks();
    4173 
    4174  uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
    4175  VkDeviceSize GetPreferredBlockSize() const { return m_PreferredBlockSize; }
    4176  VkDeviceSize GetBufferImageGranularity() const { return m_BufferImageGranularity; }
    4177  uint32_t GetFrameInUseCount() const { return m_FrameInUseCount; }
    4178 
    4179  void GetPoolStats(VmaPoolStats* pStats);
    4180 
    4181  bool IsEmpty() const { return m_Blocks.empty(); }
    4182 
    4183  VkResult Allocate(
    4184  VmaPool hCurrentPool,
    4185  uint32_t currentFrameIndex,
    4186  const VkMemoryRequirements& vkMemReq,
    4187  const VmaAllocationCreateInfo& createInfo,
    4188  VmaSuballocationType suballocType,
    4189  VmaAllocation* pAllocation);
    4190 
    4191  void Free(
    4192  VmaAllocation hAllocation);
    4193 
    4194  // Adds statistics of this BlockVector to pStats.
    4195  void AddStats(VmaStats* pStats);
    4196 
    4197 #if VMA_STATS_STRING_ENABLED
    4198  void PrintDetailedMap(class VmaJsonWriter& json);
    4199 #endif
    4200 
    4201  void MakePoolAllocationsLost(
    4202  uint32_t currentFrameIndex,
    4203  size_t* pLostAllocationCount);
    4204 
    4205  VmaDefragmentator* EnsureDefragmentator(
    4206  VmaAllocator hAllocator,
    4207  uint32_t currentFrameIndex);
    4208 
    4209  VkResult Defragment(
    4210  VmaDefragmentationStats* pDefragmentationStats,
    4211  VkDeviceSize& maxBytesToMove,
    4212  uint32_t& maxAllocationsToMove);
    4213 
    4214  void DestroyDefragmentator();
    4215 
    4216 private:
    4217  friend class VmaDefragmentator;
    4218 
    4219  const VmaAllocator m_hAllocator;
    4220  const uint32_t m_MemoryTypeIndex;
    4221  const VkDeviceSize m_PreferredBlockSize;
    4222  const size_t m_MinBlockCount;
    4223  const size_t m_MaxBlockCount;
    4224  const VkDeviceSize m_BufferImageGranularity;
    4225  const uint32_t m_FrameInUseCount;
    4226  const bool m_IsCustomPool;
    4227  VMA_MUTEX m_Mutex;
    4228  // Incrementally sorted by sumFreeSize, ascending.
    4229  VmaVector< VmaDeviceMemoryBlock*, VmaStlAllocator<VmaDeviceMemoryBlock*> > m_Blocks;
    4230  /* There can be at most one allocation that is completely empty - a
    4231  hysteresis to avoid pessimistic case of alternating creation and destruction
    4232  of a VkDeviceMemory. */
    4233  bool m_HasEmptyBlock;
    4234  VmaDefragmentator* m_pDefragmentator;
    4235 
    4236  size_t CalcMaxBlockSize() const;
    4237 
    4238  // Finds and removes given block from vector.
    4239  void Remove(VmaDeviceMemoryBlock* pBlock);
    4240 
    4241  // Performs single step in sorting m_Blocks. They may not be fully sorted
    4242  // after this call.
    4243  void IncrementallySortBlocks();
    4244 
    4245  VkResult CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex);
    4246 };
    4247 
    4248 struct VmaPool_T
    4249 {
    4250 public:
    4251  VmaBlockVector m_BlockVector;
    4252 
    4253  // Takes ownership.
    4254  VmaPool_T(
    4255  VmaAllocator hAllocator,
    4256  const VmaPoolCreateInfo& createInfo);
    4257  ~VmaPool_T();
    4258 
    4259  VmaBlockVector& GetBlockVector() { return m_BlockVector; }
    4260 
    4261 #if VMA_STATS_STRING_ENABLED
    4262  //void PrintDetailedMap(class VmaStringBuilder& sb);
    4263 #endif
    4264 };
    4265 
    4266 class VmaDefragmentator
    4267 {
    4268  const VmaAllocator m_hAllocator;
    4269  VmaBlockVector* const m_pBlockVector;
    4270  uint32_t m_CurrentFrameIndex;
    4271  VkDeviceSize m_BytesMoved;
    4272  uint32_t m_AllocationsMoved;
    4273 
    4274  struct AllocationInfo
    4275  {
    4276  VmaAllocation m_hAllocation;
    4277  VkBool32* m_pChanged;
    4278 
    4279  AllocationInfo() :
    4280  m_hAllocation(VK_NULL_HANDLE),
    4281  m_pChanged(VMA_NULL)
    4282  {
    4283  }
    4284  };
    4285 
    4286  struct AllocationInfoSizeGreater
    4287  {
    4288  bool operator()(const AllocationInfo& lhs, const AllocationInfo& rhs) const
    4289  {
    4290  return lhs.m_hAllocation->GetSize() > rhs.m_hAllocation->GetSize();
    4291  }
    4292  };
    4293 
    4294  // Used between AddAllocation and Defragment.
    4295  VmaVector< AllocationInfo, VmaStlAllocator<AllocationInfo> > m_Allocations;
    4296 
    4297  struct BlockInfo
    4298  {
    4299  VmaDeviceMemoryBlock* m_pBlock;
    4300  bool m_HasNonMovableAllocations;
    4301  VmaVector< AllocationInfo, VmaStlAllocator<AllocationInfo> > m_Allocations;
    4302 
    4303  BlockInfo(const VkAllocationCallbacks* pAllocationCallbacks) :
    4304  m_pBlock(VMA_NULL),
    4305  m_HasNonMovableAllocations(true),
    4306  m_Allocations(pAllocationCallbacks),
    4307  m_pMappedDataForDefragmentation(VMA_NULL)
    4308  {
    4309  }
    4310 
    4311  void CalcHasNonMovableAllocations()
    4312  {
    4313  const size_t blockAllocCount = m_pBlock->m_Metadata.GetAllocationCount();
    4314  const size_t defragmentAllocCount = m_Allocations.size();
    4315  m_HasNonMovableAllocations = blockAllocCount != defragmentAllocCount;
    4316  }
    4317 
    4318  void SortAllocationsBySizeDescecnding()
    4319  {
    4320  VMA_SORT(m_Allocations.begin(), m_Allocations.end(), AllocationInfoSizeGreater());
    4321  }
    4322 
    4323  VkResult EnsureMapping(VmaAllocator hAllocator, void** ppMappedData);
    4324  void Unmap(VmaAllocator hAllocator);
    4325 
    4326  private:
    4327  // Not null if mapped for defragmentation only, not originally mapped.
    4328  void* m_pMappedDataForDefragmentation;
    4329  };
    4330 
    4331  struct BlockPointerLess
    4332  {
    4333  bool operator()(const BlockInfo* pLhsBlockInfo, const VmaDeviceMemoryBlock* pRhsBlock) const
    4334  {
    4335  return pLhsBlockInfo->m_pBlock < pRhsBlock;
    4336  }
    4337  bool operator()(const BlockInfo* pLhsBlockInfo, const BlockInfo* pRhsBlockInfo) const
    4338  {
    4339  return pLhsBlockInfo->m_pBlock < pRhsBlockInfo->m_pBlock;
    4340  }
    4341  };
    4342 
    4343  // 1. Blocks with some non-movable allocations go first.
    4344  // 2. Blocks with smaller sumFreeSize go first.
    4345  struct BlockInfoCompareMoveDestination
    4346  {
    4347  bool operator()(const BlockInfo* pLhsBlockInfo, const BlockInfo* pRhsBlockInfo) const
    4348  {
    4349  if(pLhsBlockInfo->m_HasNonMovableAllocations && !pRhsBlockInfo->m_HasNonMovableAllocations)
    4350  {
    4351  return true;
    4352  }
    4353  if(!pLhsBlockInfo->m_HasNonMovableAllocations && pRhsBlockInfo->m_HasNonMovableAllocations)
    4354  {
    4355  return false;
    4356  }
    4357  if(pLhsBlockInfo->m_pBlock->m_Metadata.GetSumFreeSize() < pRhsBlockInfo->m_pBlock->m_Metadata.GetSumFreeSize())
    4358  {
    4359  return true;
    4360  }
    4361  return false;
    4362  }
    4363  };
    4364 
    4365  typedef VmaVector< BlockInfo*, VmaStlAllocator<BlockInfo*> > BlockInfoVector;
    4366  BlockInfoVector m_Blocks;
    4367 
    4368  VkResult DefragmentRound(
    4369  VkDeviceSize maxBytesToMove,
    4370  uint32_t maxAllocationsToMove);
    4371 
    4372  static bool MoveMakesSense(
    4373  size_t dstBlockIndex, VkDeviceSize dstOffset,
    4374  size_t srcBlockIndex, VkDeviceSize srcOffset);
    4375 
    4376 public:
    4377  VmaDefragmentator(
    4378  VmaAllocator hAllocator,
    4379  VmaBlockVector* pBlockVector,
    4380  uint32_t currentFrameIndex);
    4381 
    4382  ~VmaDefragmentator();
    4383 
    4384  VkDeviceSize GetBytesMoved() const { return m_BytesMoved; }
    4385  uint32_t GetAllocationsMoved() const { return m_AllocationsMoved; }
    4386 
    4387  void AddAllocation(VmaAllocation hAlloc, VkBool32* pChanged);
    4388 
    4389  VkResult Defragment(
    4390  VkDeviceSize maxBytesToMove,
    4391  uint32_t maxAllocationsToMove);
    4392 };
    4393 
    4394 // Main allocator object.
    4395 struct VmaAllocator_T
    4396 {
    4397  bool m_UseMutex;
    4398  bool m_UseKhrDedicatedAllocation;
    4399  VkDevice m_hDevice;
    4400  bool m_AllocationCallbacksSpecified;
    4401  VkAllocationCallbacks m_AllocationCallbacks;
    4402  VmaDeviceMemoryCallbacks m_DeviceMemoryCallbacks;
    4403 
    4404  // Number of bytes free out of limit, or VK_WHOLE_SIZE if not limit for that heap.
    4405  VkDeviceSize m_HeapSizeLimit[VK_MAX_MEMORY_HEAPS];
    4406  VMA_MUTEX m_HeapSizeLimitMutex;
    4407 
    4408  VkPhysicalDeviceProperties m_PhysicalDeviceProperties;
    4409  VkPhysicalDeviceMemoryProperties m_MemProps;
    4410 
    4411  // Default pools.
    4412  VmaBlockVector* m_pBlockVectors[VK_MAX_MEMORY_TYPES];
    4413 
    4414  // Each vector is sorted by memory (handle value).
    4415  typedef VmaVector< VmaAllocation, VmaStlAllocator<VmaAllocation> > AllocationVectorType;
    4416  AllocationVectorType* m_pDedicatedAllocations[VK_MAX_MEMORY_TYPES];
    4417  VMA_MUTEX m_DedicatedAllocationsMutex[VK_MAX_MEMORY_TYPES];
    4418 
    4419  VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo);
    4420  ~VmaAllocator_T();
    4421 
    4422  const VkAllocationCallbacks* GetAllocationCallbacks() const
    4423  {
    4424  return m_AllocationCallbacksSpecified ? &m_AllocationCallbacks : 0;
    4425  }
    4426  const VmaVulkanFunctions& GetVulkanFunctions() const
    4427  {
    4428  return m_VulkanFunctions;
    4429  }
    4430 
    4431  VkDeviceSize GetBufferImageGranularity() const
    4432  {
    4433  return VMA_MAX(
    4434  static_cast<VkDeviceSize>(VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY),
    4435  m_PhysicalDeviceProperties.limits.bufferImageGranularity);
    4436  }
    4437 
    4438  uint32_t GetMemoryHeapCount() const { return m_MemProps.memoryHeapCount; }
    4439  uint32_t GetMemoryTypeCount() const { return m_MemProps.memoryTypeCount; }
    4440 
    4441  uint32_t MemoryTypeIndexToHeapIndex(uint32_t memTypeIndex) const
    4442  {
    4443  VMA_ASSERT(memTypeIndex < m_MemProps.memoryTypeCount);
    4444  return m_MemProps.memoryTypes[memTypeIndex].heapIndex;
    4445  }
    4446 
    4447  void GetBufferMemoryRequirements(
    4448  VkBuffer hBuffer,
    4449  VkMemoryRequirements& memReq,
    4450  bool& requiresDedicatedAllocation,
    4451  bool& prefersDedicatedAllocation) const;
    4452  void GetImageMemoryRequirements(
    4453  VkImage hImage,
    4454  VkMemoryRequirements& memReq,
    4455  bool& requiresDedicatedAllocation,
    4456  bool& prefersDedicatedAllocation) const;
    4457 
    4458  // Main allocation function.
    4459  VkResult AllocateMemory(
    4460  const VkMemoryRequirements& vkMemReq,
    4461  bool requiresDedicatedAllocation,
    4462  bool prefersDedicatedAllocation,
    4463  VkBuffer dedicatedBuffer,
    4464  VkImage dedicatedImage,
    4465  const VmaAllocationCreateInfo& createInfo,
    4466  VmaSuballocationType suballocType,
    4467  VmaAllocation* pAllocation);
    4468 
    4469  // Main deallocation function.
    4470  void FreeMemory(const VmaAllocation allocation);
    4471 
    4472  void CalculateStats(VmaStats* pStats);
    4473 
    4474 #if VMA_STATS_STRING_ENABLED
    4475  void PrintDetailedMap(class VmaJsonWriter& json);
    4476 #endif
    4477 
    4478  VkResult Defragment(
    4479  VmaAllocation* pAllocations,
    4480  size_t allocationCount,
    4481  VkBool32* pAllocationsChanged,
    4482  const VmaDefragmentationInfo* pDefragmentationInfo,
    4483  VmaDefragmentationStats* pDefragmentationStats);
    4484 
    4485  void GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo);
    4486  bool TouchAllocation(VmaAllocation hAllocation);
    4487 
    4488  VkResult CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool);
    4489  void DestroyPool(VmaPool pool);
    4490  void GetPoolStats(VmaPool pool, VmaPoolStats* pPoolStats);
    4491 
    4492  void SetCurrentFrameIndex(uint32_t frameIndex);
    4493 
    4494  void MakePoolAllocationsLost(
    4495  VmaPool hPool,
    4496  size_t* pLostAllocationCount);
    4497 
    4498  void CreateLostAllocation(VmaAllocation* pAllocation);
    4499 
    4500  VkResult AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory);
    4501  void FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory);
    4502 
    4503  VkResult Map(VmaAllocation hAllocation, void** ppData);
    4504  void Unmap(VmaAllocation hAllocation);
    4505 
    4506  VkResult BindBufferMemory(VmaAllocation hAllocation, VkBuffer hBuffer);
    4507  VkResult BindImageMemory(VmaAllocation hAllocation, VkImage hImage);
    4508 
    4509 private:
    4510  VkDeviceSize m_PreferredLargeHeapBlockSize;
    4511 
    4512  VkPhysicalDevice m_PhysicalDevice;
    4513  VMA_ATOMIC_UINT32 m_CurrentFrameIndex;
    4514 
    4515  VMA_MUTEX m_PoolsMutex;
    4516  // Protected by m_PoolsMutex. Sorted by pointer value.
    4517  VmaVector<VmaPool, VmaStlAllocator<VmaPool> > m_Pools;
    4518 
    4519  VmaVulkanFunctions m_VulkanFunctions;
    4520 
    4521  void ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions);
    4522 
    4523  VkDeviceSize CalcPreferredBlockSize(uint32_t memTypeIndex);
    4524 
    4525  VkResult AllocateMemoryOfType(
    4526  const VkMemoryRequirements& vkMemReq,
    4527  bool dedicatedAllocation,
    4528  VkBuffer dedicatedBuffer,
    4529  VkImage dedicatedImage,
    4530  const VmaAllocationCreateInfo& createInfo,
    4531  uint32_t memTypeIndex,
    4532  VmaSuballocationType suballocType,
    4533  VmaAllocation* pAllocation);
    4534 
    4535  // Allocates and registers new VkDeviceMemory specifically for single allocation.
    4536  VkResult AllocateDedicatedMemory(
    4537  VkDeviceSize size,
    4538  VmaSuballocationType suballocType,
    4539  uint32_t memTypeIndex,
    4540  bool map,
    4541  bool isUserDataString,
    4542  void* pUserData,
    4543  VkBuffer dedicatedBuffer,
    4544  VkImage dedicatedImage,
    4545  VmaAllocation* pAllocation);
    4546 
    4547  // Tries to free pMemory as Dedicated Memory. Returns true if found and freed.
    4548  void FreeDedicatedMemory(VmaAllocation allocation);
    4549 };
    4550 
    4552 // Memory allocation #2 after VmaAllocator_T definition
    4553 
    4554 static void* VmaMalloc(VmaAllocator hAllocator, size_t size, size_t alignment)
    4555 {
    4556  return VmaMalloc(&hAllocator->m_AllocationCallbacks, size, alignment);
    4557 }
    4558 
    4559 static void VmaFree(VmaAllocator hAllocator, void* ptr)
    4560 {
    4561  VmaFree(&hAllocator->m_AllocationCallbacks, ptr);
    4562 }
    4563 
    4564 template<typename T>
    4565 static T* VmaAllocate(VmaAllocator hAllocator)
    4566 {
    4567  return (T*)VmaMalloc(hAllocator, sizeof(T), VMA_ALIGN_OF(T));
    4568 }
    4569 
    4570 template<typename T>
    4571 static T* VmaAllocateArray(VmaAllocator hAllocator, size_t count)
    4572 {
    4573  return (T*)VmaMalloc(hAllocator, sizeof(T) * count, VMA_ALIGN_OF(T));
    4574 }
    4575 
    4576 template<typename T>
    4577 static void vma_delete(VmaAllocator hAllocator, T* ptr)
    4578 {
    4579  if(ptr != VMA_NULL)
    4580  {
    4581  ptr->~T();
    4582  VmaFree(hAllocator, ptr);
    4583  }
    4584 }
    4585 
    4586 template<typename T>
    4587 static void vma_delete_array(VmaAllocator hAllocator, T* ptr, size_t count)
    4588 {
    4589  if(ptr != VMA_NULL)
    4590  {
    4591  for(size_t i = count; i--; )
    4592  ptr[i].~T();
    4593  VmaFree(hAllocator, ptr);
    4594  }
    4595 }
    4596 
    4598 // VmaStringBuilder
    4599 
    4600 #if VMA_STATS_STRING_ENABLED
    4601 
    4602 class VmaStringBuilder
    4603 {
    4604 public:
    4605  VmaStringBuilder(VmaAllocator alloc) : m_Data(VmaStlAllocator<char>(alloc->GetAllocationCallbacks())) { }
    4606  size_t GetLength() const { return m_Data.size(); }
    4607  const char* GetData() const { return m_Data.data(); }
    4608 
    4609  void Add(char ch) { m_Data.push_back(ch); }
    4610  void Add(const char* pStr);
    4611  void AddNewLine() { Add('\n'); }
    4612  void AddNumber(uint32_t num);
    4613  void AddNumber(uint64_t num);
    4614  void AddPointer(const void* ptr);
    4615 
    4616 private:
    4617  VmaVector< char, VmaStlAllocator<char> > m_Data;
    4618 };
    4619 
    4620 void VmaStringBuilder::Add(const char* pStr)
    4621 {
    4622  const size_t strLen = strlen(pStr);
    4623  if(strLen > 0)
    4624  {
    4625  const size_t oldCount = m_Data.size();
    4626  m_Data.resize(oldCount + strLen);
    4627  memcpy(m_Data.data() + oldCount, pStr, strLen);
    4628  }
    4629 }
    4630 
    4631 void VmaStringBuilder::AddNumber(uint32_t num)
    4632 {
    4633  char buf[11];
    4634  VmaUint32ToStr(buf, sizeof(buf), num);
    4635  Add(buf);
    4636 }
    4637 
    4638 void VmaStringBuilder::AddNumber(uint64_t num)
    4639 {
    4640  char buf[21];
    4641  VmaUint64ToStr(buf, sizeof(buf), num);
    4642  Add(buf);
    4643 }
    4644 
    4645 void VmaStringBuilder::AddPointer(const void* ptr)
    4646 {
    4647  char buf[21];
    4648  VmaPtrToStr(buf, sizeof(buf), ptr);
    4649  Add(buf);
    4650 }
    4651 
    4652 #endif // #if VMA_STATS_STRING_ENABLED
    4653 
    4655 // VmaJsonWriter
    4656 
    4657 #if VMA_STATS_STRING_ENABLED
    4658 
    4659 class VmaJsonWriter
    4660 {
    4661 public:
    4662  VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb);
    4663  ~VmaJsonWriter();
    4664 
    4665  void BeginObject(bool singleLine = false);
    4666  void EndObject();
    4667 
    4668  void BeginArray(bool singleLine = false);
    4669  void EndArray();
    4670 
    4671  void WriteString(const char* pStr);
    4672  void BeginString(const char* pStr = VMA_NULL);
    4673  void ContinueString(const char* pStr);
    4674  void ContinueString(uint32_t n);
    4675  void ContinueString(uint64_t n);
    4676  void ContinueString_Pointer(const void* ptr);
    4677  void EndString(const char* pStr = VMA_NULL);
    4678 
    4679  void WriteNumber(uint32_t n);
    4680  void WriteNumber(uint64_t n);
    4681  void WriteBool(bool b);
    4682  void WriteNull();
    4683 
    4684 private:
    4685  static const char* const INDENT;
    4686 
    4687  enum COLLECTION_TYPE
    4688  {
    4689  COLLECTION_TYPE_OBJECT,
    4690  COLLECTION_TYPE_ARRAY,
    4691  };
    4692  struct StackItem
    4693  {
    4694  COLLECTION_TYPE type;
    4695  uint32_t valueCount;
    4696  bool singleLineMode;
    4697  };
    4698 
    4699  VmaStringBuilder& m_SB;
    4700  VmaVector< StackItem, VmaStlAllocator<StackItem> > m_Stack;
    4701  bool m_InsideString;
    4702 
    4703  void BeginValue(bool isString);
    4704  void WriteIndent(bool oneLess = false);
    4705 };
    4706 
    4707 const char* const VmaJsonWriter::INDENT = " ";
    4708 
    4709 VmaJsonWriter::VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb) :
    4710  m_SB(sb),
    4711  m_Stack(VmaStlAllocator<StackItem>(pAllocationCallbacks)),
    4712  m_InsideString(false)
    4713 {
    4714 }
    4715 
    4716 VmaJsonWriter::~VmaJsonWriter()
    4717 {
    4718  VMA_ASSERT(!m_InsideString);
    4719  VMA_ASSERT(m_Stack.empty());
    4720 }
    4721 
    4722 void VmaJsonWriter::BeginObject(bool singleLine)
    4723 {
    4724  VMA_ASSERT(!m_InsideString);
    4725 
    4726  BeginValue(false);
    4727  m_SB.Add('{');
    4728 
    4729  StackItem item;
    4730  item.type = COLLECTION_TYPE_OBJECT;
    4731  item.valueCount = 0;
    4732  item.singleLineMode = singleLine;
    4733  m_Stack.push_back(item);
    4734 }
    4735 
    4736 void VmaJsonWriter::EndObject()
    4737 {
    4738  VMA_ASSERT(!m_InsideString);
    4739 
    4740  WriteIndent(true);
    4741  m_SB.Add('}');
    4742 
    4743  VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_OBJECT);
    4744  m_Stack.pop_back();
    4745 }
    4746 
    4747 void VmaJsonWriter::BeginArray(bool singleLine)
    4748 {
    4749  VMA_ASSERT(!m_InsideString);
    4750 
    4751  BeginValue(false);
    4752  m_SB.Add('[');
    4753 
    4754  StackItem item;
    4755  item.type = COLLECTION_TYPE_ARRAY;
    4756  item.valueCount = 0;
    4757  item.singleLineMode = singleLine;
    4758  m_Stack.push_back(item);
    4759 }
    4760 
    4761 void VmaJsonWriter::EndArray()
    4762 {
    4763  VMA_ASSERT(!m_InsideString);
    4764 
    4765  WriteIndent(true);
    4766  m_SB.Add(']');
    4767 
    4768  VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_ARRAY);
    4769  m_Stack.pop_back();
    4770 }
    4771 
    4772 void VmaJsonWriter::WriteString(const char* pStr)
    4773 {
    4774  BeginString(pStr);
    4775  EndString();
    4776 }
    4777 
    4778 void VmaJsonWriter::BeginString(const char* pStr)
    4779 {
    4780  VMA_ASSERT(!m_InsideString);
    4781 
    4782  BeginValue(true);
    4783  m_SB.Add('"');
    4784  m_InsideString = true;
    4785  if(pStr != VMA_NULL && pStr[0] != '\0')
    4786  {
    4787  ContinueString(pStr);
    4788  }
    4789 }
    4790 
    4791 void VmaJsonWriter::ContinueString(const char* pStr)
    4792 {
    4793  VMA_ASSERT(m_InsideString);
    4794 
    4795  const size_t strLen = strlen(pStr);
    4796  for(size_t i = 0; i < strLen; ++i)
    4797  {
    4798  char ch = pStr[i];
    4799  if(ch == '\'')
    4800  {
    4801  m_SB.Add("\\\\");
    4802  }
    4803  else if(ch == '"')
    4804  {
    4805  m_SB.Add("\\\"");
    4806  }
    4807  else if(ch >= 32)
    4808  {
    4809  m_SB.Add(ch);
    4810  }
    4811  else switch(ch)
    4812  {
    4813  case '\b':
    4814  m_SB.Add("\\b");
    4815  break;
    4816  case '\f':
    4817  m_SB.Add("\\f");
    4818  break;
    4819  case '\n':
    4820  m_SB.Add("\\n");
    4821  break;
    4822  case '\r':
    4823  m_SB.Add("\\r");
    4824  break;
    4825  case '\t':
    4826  m_SB.Add("\\t");
    4827  break;
    4828  default:
    4829  VMA_ASSERT(0 && "Character not currently supported.");
    4830  break;
    4831  }
    4832  }
    4833 }
    4834 
    4835 void VmaJsonWriter::ContinueString(uint32_t n)
    4836 {
    4837  VMA_ASSERT(m_InsideString);
    4838  m_SB.AddNumber(n);
    4839 }
    4840 
    4841 void VmaJsonWriter::ContinueString(uint64_t n)
    4842 {
    4843  VMA_ASSERT(m_InsideString);
    4844  m_SB.AddNumber(n);
    4845 }
    4846 
    4847 void VmaJsonWriter::ContinueString_Pointer(const void* ptr)
    4848 {
    4849  VMA_ASSERT(m_InsideString);
    4850  m_SB.AddPointer(ptr);
    4851 }
    4852 
    4853 void VmaJsonWriter::EndString(const char* pStr)
    4854 {
    4855  VMA_ASSERT(m_InsideString);
    4856  if(pStr != VMA_NULL && pStr[0] != '\0')
    4857  {
    4858  ContinueString(pStr);
    4859  }
    4860  m_SB.Add('"');
    4861  m_InsideString = false;
    4862 }
    4863 
    4864 void VmaJsonWriter::WriteNumber(uint32_t n)
    4865 {
    4866  VMA_ASSERT(!m_InsideString);
    4867  BeginValue(false);
    4868  m_SB.AddNumber(n);
    4869 }
    4870 
    4871 void VmaJsonWriter::WriteNumber(uint64_t n)
    4872 {
    4873  VMA_ASSERT(!m_InsideString);
    4874  BeginValue(false);
    4875  m_SB.AddNumber(n);
    4876 }
    4877 
    4878 void VmaJsonWriter::WriteBool(bool b)
    4879 {
    4880  VMA_ASSERT(!m_InsideString);
    4881  BeginValue(false);
    4882  m_SB.Add(b ? "true" : "false");
    4883 }
    4884 
    4885 void VmaJsonWriter::WriteNull()
    4886 {
    4887  VMA_ASSERT(!m_InsideString);
    4888  BeginValue(false);
    4889  m_SB.Add("null");
    4890 }
    4891 
    4892 void VmaJsonWriter::BeginValue(bool isString)
    4893 {
    4894  if(!m_Stack.empty())
    4895  {
    4896  StackItem& currItem = m_Stack.back();
    4897  if(currItem.type == COLLECTION_TYPE_OBJECT &&
    4898  currItem.valueCount % 2 == 0)
    4899  {
    4900  VMA_ASSERT(isString);
    4901  }
    4902 
    4903  if(currItem.type == COLLECTION_TYPE_OBJECT &&
    4904  currItem.valueCount % 2 != 0)
    4905  {
    4906  m_SB.Add(": ");
    4907  }
    4908  else if(currItem.valueCount > 0)
    4909  {
    4910  m_SB.Add(", ");
    4911  WriteIndent();
    4912  }
    4913  else
    4914  {
    4915  WriteIndent();
    4916  }
    4917  ++currItem.valueCount;
    4918  }
    4919 }
    4920 
    4921 void VmaJsonWriter::WriteIndent(bool oneLess)
    4922 {
    4923  if(!m_Stack.empty() && !m_Stack.back().singleLineMode)
    4924  {
    4925  m_SB.AddNewLine();
    4926 
    4927  size_t count = m_Stack.size();
    4928  if(count > 0 && oneLess)
    4929  {
    4930  --count;
    4931  }
    4932  for(size_t i = 0; i < count; ++i)
    4933  {
    4934  m_SB.Add(INDENT);
    4935  }
    4936  }
    4937 }
    4938 
    4939 #endif // #if VMA_STATS_STRING_ENABLED
    4940 
    4942 
    4943 void VmaAllocation_T::SetUserData(VmaAllocator hAllocator, void* pUserData)
    4944 {
    4945  if(IsUserDataString())
    4946  {
    4947  VMA_ASSERT(pUserData == VMA_NULL || pUserData != m_pUserData);
    4948 
    4949  FreeUserDataString(hAllocator);
    4950 
    4951  if(pUserData != VMA_NULL)
    4952  {
    4953  const char* const newStrSrc = (char*)pUserData;
    4954  const size_t newStrLen = strlen(newStrSrc);
    4955  char* const newStrDst = vma_new_array(hAllocator, char, newStrLen + 1);
    4956  memcpy(newStrDst, newStrSrc, newStrLen + 1);
    4957  m_pUserData = newStrDst;
    4958  }
    4959  }
    4960  else
    4961  {
    4962  m_pUserData = pUserData;
    4963  }
    4964 }
    4965 
    4966 void VmaAllocation_T::ChangeBlockAllocation(
    4967  VmaAllocator hAllocator,
    4968  VmaDeviceMemoryBlock* block,
    4969  VkDeviceSize offset)
    4970 {
    4971  VMA_ASSERT(block != VMA_NULL);
    4972  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
    4973 
    4974  // Move mapping reference counter from old block to new block.
    4975  if(block != m_BlockAllocation.m_Block)
    4976  {
    4977  uint32_t mapRefCount = m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP;
    4978  if(IsPersistentMap())
    4979  ++mapRefCount;
    4980  m_BlockAllocation.m_Block->Unmap(hAllocator, mapRefCount);
    4981  block->Map(hAllocator, mapRefCount, VMA_NULL);
    4982  }
    4983 
    4984  m_BlockAllocation.m_Block = block;
    4985  m_BlockAllocation.m_Offset = offset;
    4986 }
    4987 
    4988 VkDeviceSize VmaAllocation_T::GetOffset() const
    4989 {
    4990  switch(m_Type)
    4991  {
    4992  case ALLOCATION_TYPE_BLOCK:
    4993  return m_BlockAllocation.m_Offset;
    4994  case ALLOCATION_TYPE_DEDICATED:
    4995  return 0;
    4996  default:
    4997  VMA_ASSERT(0);
    4998  return 0;
    4999  }
    5000 }
    5001 
    5002 VkDeviceMemory VmaAllocation_T::GetMemory() const
    5003 {
    5004  switch(m_Type)
    5005  {
    5006  case ALLOCATION_TYPE_BLOCK:
    5007  return m_BlockAllocation.m_Block->GetDeviceMemory();
    5008  case ALLOCATION_TYPE_DEDICATED:
    5009  return m_DedicatedAllocation.m_hMemory;
    5010  default:
    5011  VMA_ASSERT(0);
    5012  return VK_NULL_HANDLE;
    5013  }
    5014 }
    5015 
    5016 uint32_t VmaAllocation_T::GetMemoryTypeIndex() const
    5017 {
    5018  switch(m_Type)
    5019  {
    5020  case ALLOCATION_TYPE_BLOCK:
    5021  return m_BlockAllocation.m_Block->GetMemoryTypeIndex();
    5022  case ALLOCATION_TYPE_DEDICATED:
    5023  return m_DedicatedAllocation.m_MemoryTypeIndex;
    5024  default:
    5025  VMA_ASSERT(0);
    5026  return UINT32_MAX;
    5027  }
    5028 }
    5029 
    5030 void* VmaAllocation_T::GetMappedData() const
    5031 {
    5032  switch(m_Type)
    5033  {
    5034  case ALLOCATION_TYPE_BLOCK:
    5035  if(m_MapCount != 0)
    5036  {
    5037  void* pBlockData = m_BlockAllocation.m_Block->GetMappedData();
    5038  VMA_ASSERT(pBlockData != VMA_NULL);
    5039  return (char*)pBlockData + m_BlockAllocation.m_Offset;
    5040  }
    5041  else
    5042  {
    5043  return VMA_NULL;
    5044  }
    5045  break;
    5046  case ALLOCATION_TYPE_DEDICATED:
    5047  VMA_ASSERT((m_DedicatedAllocation.m_pMappedData != VMA_NULL) == (m_MapCount != 0));
    5048  return m_DedicatedAllocation.m_pMappedData;
    5049  default:
    5050  VMA_ASSERT(0);
    5051  return VMA_NULL;
    5052  }
    5053 }
    5054 
    5055 bool VmaAllocation_T::CanBecomeLost() const
    5056 {
    5057  switch(m_Type)
    5058  {
    5059  case ALLOCATION_TYPE_BLOCK:
    5060  return m_BlockAllocation.m_CanBecomeLost;
    5061  case ALLOCATION_TYPE_DEDICATED:
    5062  return false;
    5063  default:
    5064  VMA_ASSERT(0);
    5065  return false;
    5066  }
    5067 }
    5068 
    5069 VmaPool VmaAllocation_T::GetPool() const
    5070 {
    5071  VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
    5072  return m_BlockAllocation.m_hPool;
    5073 }
    5074 
    5075 bool VmaAllocation_T::MakeLost(uint32_t currentFrameIndex, uint32_t frameInUseCount)
    5076 {
    5077  VMA_ASSERT(CanBecomeLost());
    5078 
    5079  /*
    5080  Warning: This is a carefully designed algorithm.
    5081  Do not modify unless you really know what you're doing :)
    5082  */
    5083  uint32_t localLastUseFrameIndex = GetLastUseFrameIndex();
    5084  for(;;)
    5085  {
    5086  if(localLastUseFrameIndex == VMA_FRAME_INDEX_LOST)
    5087  {
    5088  VMA_ASSERT(0);
    5089  return false;
    5090  }
    5091  else if(localLastUseFrameIndex + frameInUseCount >= currentFrameIndex)
    5092  {
    5093  return false;
    5094  }
    5095  else // Last use time earlier than current time.
    5096  {
    5097  if(CompareExchangeLastUseFrameIndex(localLastUseFrameIndex, VMA_FRAME_INDEX_LOST))
    5098  {
    5099  // Setting hAllocation.LastUseFrameIndex atomic to VMA_FRAME_INDEX_LOST is enough to mark it as LOST.
    5100  // Calling code just needs to unregister this allocation in owning VmaDeviceMemoryBlock.
    5101  return true;
    5102  }
    5103  }
    5104  }
    5105 }
    5106 
    5107 void VmaAllocation_T::FreeUserDataString(VmaAllocator hAllocator)
    5108 {
    5109  VMA_ASSERT(IsUserDataString());
    5110  if(m_pUserData != VMA_NULL)
    5111  {
    5112  char* const oldStr = (char*)m_pUserData;
    5113  const size_t oldStrLen = strlen(oldStr);
    5114  vma_delete_array(hAllocator, oldStr, oldStrLen + 1);
    5115  m_pUserData = VMA_NULL;
    5116  }
    5117 }
    5118 
    5119 void VmaAllocation_T::BlockAllocMap()
    5120 {
    5121  VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
    5122 
    5123  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) < 0x7F)
    5124  {
    5125  ++m_MapCount;
    5126  }
    5127  else
    5128  {
    5129  VMA_ASSERT(0 && "Allocation mapped too many times simultaneously.");
    5130  }
    5131 }
    5132 
    5133 void VmaAllocation_T::BlockAllocUnmap()
    5134 {
    5135  VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
    5136 
    5137  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) != 0)
    5138  {
    5139  --m_MapCount;
    5140  }
    5141  else
    5142  {
    5143  VMA_ASSERT(0 && "Unmapping allocation not previously mapped.");
    5144  }
    5145 }
    5146 
    5147 VkResult VmaAllocation_T::DedicatedAllocMap(VmaAllocator hAllocator, void** ppData)
    5148 {
    5149  VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
    5150 
    5151  if(m_MapCount != 0)
    5152  {
    5153  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) < 0x7F)
    5154  {
    5155  VMA_ASSERT(m_DedicatedAllocation.m_pMappedData != VMA_NULL);
    5156  *ppData = m_DedicatedAllocation.m_pMappedData;
    5157  ++m_MapCount;
    5158  return VK_SUCCESS;
    5159  }
    5160  else
    5161  {
    5162  VMA_ASSERT(0 && "Dedicated allocation mapped too many times simultaneously.");
    5163  return VK_ERROR_MEMORY_MAP_FAILED;
    5164  }
    5165  }
    5166  else
    5167  {
    5168  VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
    5169  hAllocator->m_hDevice,
    5170  m_DedicatedAllocation.m_hMemory,
    5171  0, // offset
    5172  VK_WHOLE_SIZE,
    5173  0, // flags
    5174  ppData);
    5175  if(result == VK_SUCCESS)
    5176  {
    5177  m_DedicatedAllocation.m_pMappedData = *ppData;
    5178  m_MapCount = 1;
    5179  }
    5180  return result;
    5181  }
    5182 }
    5183 
    5184 void VmaAllocation_T::DedicatedAllocUnmap(VmaAllocator hAllocator)
    5185 {
    5186  VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
    5187 
    5188  if((m_MapCount & ~MAP_COUNT_FLAG_PERSISTENT_MAP) != 0)
    5189  {
    5190  --m_MapCount;
    5191  if(m_MapCount == 0)
    5192  {
    5193  m_DedicatedAllocation.m_pMappedData = VMA_NULL;
    5194  (*hAllocator->GetVulkanFunctions().vkUnmapMemory)(
    5195  hAllocator->m_hDevice,
    5196  m_DedicatedAllocation.m_hMemory);
    5197  }
    5198  }
    5199  else
    5200  {
    5201  VMA_ASSERT(0 && "Unmapping dedicated allocation not previously mapped.");
    5202  }
    5203 }
    5204 
    5205 #if VMA_STATS_STRING_ENABLED
    5206 
    5207 // Correspond to values of enum VmaSuballocationType.
    5208 static const char* VMA_SUBALLOCATION_TYPE_NAMES[] = {
    5209  "FREE",
    5210  "UNKNOWN",
    5211  "BUFFER",
    5212  "IMAGE_UNKNOWN",
    5213  "IMAGE_LINEAR",
    5214  "IMAGE_OPTIMAL",
    5215 };
    5216 
    5217 static void VmaPrintStatInfo(VmaJsonWriter& json, const VmaStatInfo& stat)
    5218 {
    5219  json.BeginObject();
    5220 
    5221  json.WriteString("Blocks");
    5222  json.WriteNumber(stat.blockCount);
    5223 
    5224  json.WriteString("Allocations");
    5225  json.WriteNumber(stat.allocationCount);
    5226 
    5227  json.WriteString("UnusedRanges");
    5228  json.WriteNumber(stat.unusedRangeCount);
    5229 
    5230  json.WriteString("UsedBytes");
    5231  json.WriteNumber(stat.usedBytes);
    5232 
    5233  json.WriteString("UnusedBytes");
    5234  json.WriteNumber(stat.unusedBytes);
    5235 
    5236  if(stat.allocationCount > 1)
    5237  {
    5238  json.WriteString("AllocationSize");
    5239  json.BeginObject(true);
    5240  json.WriteString("Min");
    5241  json.WriteNumber(stat.allocationSizeMin);
    5242  json.WriteString("Avg");
    5243  json.WriteNumber(stat.allocationSizeAvg);
    5244  json.WriteString("Max");
    5245  json.WriteNumber(stat.allocationSizeMax);
    5246  json.EndObject();
    5247  }
    5248 
    5249  if(stat.unusedRangeCount > 1)
    5250  {
    5251  json.WriteString("UnusedRangeSize");
    5252  json.BeginObject(true);
    5253  json.WriteString("Min");
    5254  json.WriteNumber(stat.unusedRangeSizeMin);
    5255  json.WriteString("Avg");
    5256  json.WriteNumber(stat.unusedRangeSizeAvg);
    5257  json.WriteString("Max");
    5258  json.WriteNumber(stat.unusedRangeSizeMax);
    5259  json.EndObject();
    5260  }
    5261 
    5262  json.EndObject();
    5263 }
    5264 
    5265 #endif // #if VMA_STATS_STRING_ENABLED
    5266 
    5267 struct VmaSuballocationItemSizeLess
    5268 {
    5269  bool operator()(
    5270  const VmaSuballocationList::iterator lhs,
    5271  const VmaSuballocationList::iterator rhs) const
    5272  {
    5273  return lhs->size < rhs->size;
    5274  }
    5275  bool operator()(
    5276  const VmaSuballocationList::iterator lhs,
    5277  VkDeviceSize rhsSize) const
    5278  {
    5279  return lhs->size < rhsSize;
    5280  }
    5281 };
    5282 
    5284 // class VmaBlockMetadata
    5285 
    5286 VmaBlockMetadata::VmaBlockMetadata(VmaAllocator hAllocator) :
    5287  m_Size(0),
    5288  m_FreeCount(0),
    5289  m_SumFreeSize(0),
    5290  m_Suballocations(VmaStlAllocator<VmaSuballocation>(hAllocator->GetAllocationCallbacks())),
    5291  m_FreeSuballocationsBySize(VmaStlAllocator<VmaSuballocationList::iterator>(hAllocator->GetAllocationCallbacks()))
    5292 {
    5293 }
    5294 
    5295 VmaBlockMetadata::~VmaBlockMetadata()
    5296 {
    5297 }
    5298 
    5299 void VmaBlockMetadata::Init(VkDeviceSize size)
    5300 {
    5301  m_Size = size;
    5302  m_FreeCount = 1;
    5303  m_SumFreeSize = size;
    5304 
    5305  VmaSuballocation suballoc = {};
    5306  suballoc.offset = 0;
    5307  suballoc.size = size;
    5308  suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
    5309  suballoc.hAllocation = VK_NULL_HANDLE;
    5310 
    5311  m_Suballocations.push_back(suballoc);
    5312  VmaSuballocationList::iterator suballocItem = m_Suballocations.end();
    5313  --suballocItem;
    5314  m_FreeSuballocationsBySize.push_back(suballocItem);
    5315 }
    5316 
    5317 bool VmaBlockMetadata::Validate() const
    5318 {
    5319  if(m_Suballocations.empty())
    5320  {
    5321  return false;
    5322  }
    5323 
    5324  // Expected offset of new suballocation as calculates from previous ones.
    5325  VkDeviceSize calculatedOffset = 0;
    5326  // Expected number of free suballocations as calculated from traversing their list.
    5327  uint32_t calculatedFreeCount = 0;
    5328  // Expected sum size of free suballocations as calculated from traversing their list.
    5329  VkDeviceSize calculatedSumFreeSize = 0;
    5330  // Expected number of free suballocations that should be registered in
    5331  // m_FreeSuballocationsBySize calculated from traversing their list.
    5332  size_t freeSuballocationsToRegister = 0;
    5333  // True if previous visisted suballocation was free.
    5334  bool prevFree = false;
    5335 
    5336  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
    5337  suballocItem != m_Suballocations.cend();
    5338  ++suballocItem)
    5339  {
    5340  const VmaSuballocation& subAlloc = *suballocItem;
    5341 
    5342  // Actual offset of this suballocation doesn't match expected one.
    5343  if(subAlloc.offset != calculatedOffset)
    5344  {
    5345  return false;
    5346  }
    5347 
    5348  const bool currFree = (subAlloc.type == VMA_SUBALLOCATION_TYPE_FREE);
    5349  // Two adjacent free suballocations are invalid. They should be merged.
    5350  if(prevFree && currFree)
    5351  {
    5352  return false;
    5353  }
    5354 
    5355  if(currFree != (subAlloc.hAllocation == VK_NULL_HANDLE))
    5356  {
    5357  return false;
    5358  }
    5359 
    5360  if(currFree)
    5361  {
    5362  calculatedSumFreeSize += subAlloc.size;
    5363  ++calculatedFreeCount;
    5364  if(subAlloc.size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
    5365  {
    5366  ++freeSuballocationsToRegister;
    5367  }
    5368  }
    5369  else
    5370  {
    5371  if(subAlloc.hAllocation->GetOffset() != subAlloc.offset)
    5372  {
    5373  return false;
    5374  }
    5375  if(subAlloc.hAllocation->GetSize() != subAlloc.size)
    5376  {
    5377  return false;
    5378  }
    5379  }
    5380 
    5381  calculatedOffset += subAlloc.size;
    5382  prevFree = currFree;
    5383  }
    5384 
    5385  // Number of free suballocations registered in m_FreeSuballocationsBySize doesn't
    5386  // match expected one.
    5387  if(m_FreeSuballocationsBySize.size() != freeSuballocationsToRegister)
    5388  {
    5389  return false;
    5390  }
    5391 
    5392  VkDeviceSize lastSize = 0;
    5393  for(size_t i = 0; i < m_FreeSuballocationsBySize.size(); ++i)
    5394  {
    5395  VmaSuballocationList::iterator suballocItem = m_FreeSuballocationsBySize[i];
    5396 
    5397  // Only free suballocations can be registered in m_FreeSuballocationsBySize.
    5398  if(suballocItem->type != VMA_SUBALLOCATION_TYPE_FREE)
    5399  {
    5400  return false;
    5401  }
    5402  // They must be sorted by size ascending.
    5403  if(suballocItem->size < lastSize)
    5404  {
    5405  return false;
    5406  }
    5407 
    5408  lastSize = suballocItem->size;
    5409  }
    5410 
    5411  // Check if totals match calculacted values.
    5412  if(!ValidateFreeSuballocationList() ||
    5413  (calculatedOffset != m_Size) ||
    5414  (calculatedSumFreeSize != m_SumFreeSize) ||
    5415  (calculatedFreeCount != m_FreeCount))
    5416  {
    5417  return false;
    5418  }
    5419 
    5420  return true;
    5421 }
    5422 
    5423 VkDeviceSize VmaBlockMetadata::GetUnusedRangeSizeMax() const
    5424 {
    5425  if(!m_FreeSuballocationsBySize.empty())
    5426  {
    5427  return m_FreeSuballocationsBySize.back()->size;
    5428  }
    5429  else
    5430  {
    5431  return 0;
    5432  }
    5433 }
    5434 
    5435 bool VmaBlockMetadata::IsEmpty() const
    5436 {
    5437  return (m_Suballocations.size() == 1) && (m_FreeCount == 1);
    5438 }
    5439 
    5440 void VmaBlockMetadata::CalcAllocationStatInfo(VmaStatInfo& outInfo) const
    5441 {
    5442  outInfo.blockCount = 1;
    5443 
    5444  const uint32_t rangeCount = (uint32_t)m_Suballocations.size();
    5445  outInfo.allocationCount = rangeCount - m_FreeCount;
    5446  outInfo.unusedRangeCount = m_FreeCount;
    5447 
    5448  outInfo.unusedBytes = m_SumFreeSize;
    5449  outInfo.usedBytes = m_Size - outInfo.unusedBytes;
    5450 
    5451  outInfo.allocationSizeMin = UINT64_MAX;
    5452  outInfo.allocationSizeMax = 0;
    5453  outInfo.unusedRangeSizeMin = UINT64_MAX;
    5454  outInfo.unusedRangeSizeMax = 0;
    5455 
    5456  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
    5457  suballocItem != m_Suballocations.cend();
    5458  ++suballocItem)
    5459  {
    5460  const VmaSuballocation& suballoc = *suballocItem;
    5461  if(suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
    5462  {
    5463  outInfo.allocationSizeMin = VMA_MIN(outInfo.allocationSizeMin, suballoc.size);
    5464  outInfo.allocationSizeMax = VMA_MAX(outInfo.allocationSizeMax, suballoc.size);
    5465  }
    5466  else
    5467  {
    5468  outInfo.unusedRangeSizeMin = VMA_MIN(outInfo.unusedRangeSizeMin, suballoc.size);
    5469  outInfo.unusedRangeSizeMax = VMA_MAX(outInfo.unusedRangeSizeMax, suballoc.size);
    5470  }
    5471  }
    5472 }
    5473 
    5474 void VmaBlockMetadata::AddPoolStats(VmaPoolStats& inoutStats) const
    5475 {
    5476  const uint32_t rangeCount = (uint32_t)m_Suballocations.size();
    5477 
    5478  inoutStats.size += m_Size;
    5479  inoutStats.unusedSize += m_SumFreeSize;
    5480  inoutStats.allocationCount += rangeCount - m_FreeCount;
    5481  inoutStats.unusedRangeCount += m_FreeCount;
    5482  inoutStats.unusedRangeSizeMax = VMA_MAX(inoutStats.unusedRangeSizeMax, GetUnusedRangeSizeMax());
    5483 }
    5484 
    5485 #if VMA_STATS_STRING_ENABLED
    5486 
    5487 void VmaBlockMetadata::PrintDetailedMap(class VmaJsonWriter& json) const
    5488 {
    5489  json.BeginObject();
    5490 
    5491  json.WriteString("TotalBytes");
    5492  json.WriteNumber(m_Size);
    5493 
    5494  json.WriteString("UnusedBytes");
    5495  json.WriteNumber(m_SumFreeSize);
    5496 
    5497  json.WriteString("Allocations");
    5498  json.WriteNumber((uint64_t)m_Suballocations.size() - m_FreeCount);
    5499 
    5500  json.WriteString("UnusedRanges");
    5501  json.WriteNumber(m_FreeCount);
    5502 
    5503  json.WriteString("Suballocations");
    5504  json.BeginArray();
    5505  size_t i = 0;
    5506  for(VmaSuballocationList::const_iterator suballocItem = m_Suballocations.cbegin();
    5507  suballocItem != m_Suballocations.cend();
    5508  ++suballocItem, ++i)
    5509  {
    5510  json.BeginObject(true);
    5511 
    5512  json.WriteString("Type");
    5513  json.WriteString(VMA_SUBALLOCATION_TYPE_NAMES[suballocItem->type]);
    5514 
    5515  json.WriteString("Size");
    5516  json.WriteNumber(suballocItem->size);
    5517 
    5518  json.WriteString("Offset");
    5519  json.WriteNumber(suballocItem->offset);
    5520 
    5521  if(suballocItem->type != VMA_SUBALLOCATION_TYPE_FREE)
    5522  {
    5523  const void* pUserData = suballocItem->hAllocation->GetUserData();
    5524  if(pUserData != VMA_NULL)
    5525  {
    5526  json.WriteString("UserData");
    5527  if(suballocItem->hAllocation->IsUserDataString())
    5528  {
    5529  json.WriteString((const char*)pUserData);
    5530  }
    5531  else
    5532  {
    5533  json.BeginString();
    5534  json.ContinueString_Pointer(pUserData);
    5535  json.EndString();
    5536  }
    5537  }
    5538  }
    5539 
    5540  json.EndObject();
    5541  }
    5542  json.EndArray();
    5543 
    5544  json.EndObject();
    5545 }
    5546 
    5547 #endif // #if VMA_STATS_STRING_ENABLED
    5548 
    5549 /*
    5550 How many suitable free suballocations to analyze before choosing best one.
    5551 - Set to 1 to use First-Fit algorithm - first suitable free suballocation will
    5552  be chosen.
    5553 - Set to UINT32_MAX to use Best-Fit/Worst-Fit algorithm - all suitable free
    5554  suballocations will be analized and best one will be chosen.
    5555 - Any other value is also acceptable.
    5556 */
    5557 //static const uint32_t MAX_SUITABLE_SUBALLOCATIONS_TO_CHECK = 8;
    5558 
    5559 void VmaBlockMetadata::CreateFirstAllocationRequest(VmaAllocationRequest* pAllocationRequest)
    5560 {
    5561  VMA_ASSERT(IsEmpty());
    5562  pAllocationRequest->offset = 0;
    5563  pAllocationRequest->sumFreeSize = m_SumFreeSize;
    5564  pAllocationRequest->sumItemSize = 0;
    5565  pAllocationRequest->item = m_Suballocations.begin();
    5566  pAllocationRequest->itemsToMakeLostCount = 0;
    5567 }
    5568 
    5569 bool VmaBlockMetadata::CreateAllocationRequest(
    5570  uint32_t currentFrameIndex,
    5571  uint32_t frameInUseCount,
    5572  VkDeviceSize bufferImageGranularity,
    5573  VkDeviceSize allocSize,
    5574  VkDeviceSize allocAlignment,
    5575  VmaSuballocationType allocType,
    5576  bool canMakeOtherLost,
    5577  VmaAllocationRequest* pAllocationRequest)
    5578 {
    5579  VMA_ASSERT(allocSize > 0);
    5580  VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
    5581  VMA_ASSERT(pAllocationRequest != VMA_NULL);
    5582  VMA_HEAVY_ASSERT(Validate());
    5583 
    5584  // There is not enough total free space in this block to fullfill the request: Early return.
    5585  if(canMakeOtherLost == false && m_SumFreeSize < allocSize)
    5586  {
    5587  return false;
    5588  }
    5589 
    5590  // New algorithm, efficiently searching freeSuballocationsBySize.
    5591  const size_t freeSuballocCount = m_FreeSuballocationsBySize.size();
    5592  if(freeSuballocCount > 0)
    5593  {
    5594  if(VMA_BEST_FIT)
    5595  {
    5596  // Find first free suballocation with size not less than allocSize.
    5597  VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
    5598  m_FreeSuballocationsBySize.data(),
    5599  m_FreeSuballocationsBySize.data() + freeSuballocCount,
    5600  allocSize,
    5601  VmaSuballocationItemSizeLess());
    5602  size_t index = it - m_FreeSuballocationsBySize.data();
    5603  for(; index < freeSuballocCount; ++index)
    5604  {
    5605  if(CheckAllocation(
    5606  currentFrameIndex,
    5607  frameInUseCount,
    5608  bufferImageGranularity,
    5609  allocSize,
    5610  allocAlignment,
    5611  allocType,
    5612  m_FreeSuballocationsBySize[index],
    5613  false, // canMakeOtherLost
    5614  &pAllocationRequest->offset,
    5615  &pAllocationRequest->itemsToMakeLostCount,
    5616  &pAllocationRequest->sumFreeSize,
    5617  &pAllocationRequest->sumItemSize))
    5618  {
    5619  pAllocationRequest->item = m_FreeSuballocationsBySize[index];
    5620  return true;
    5621  }
    5622  }
    5623  }
    5624  else
    5625  {
    5626  // Search staring from biggest suballocations.
    5627  for(size_t index = freeSuballocCount; index--; )
    5628  {
    5629  if(CheckAllocation(
    5630  currentFrameIndex,
    5631  frameInUseCount,
    5632  bufferImageGranularity,
    5633  allocSize,
    5634  allocAlignment,
    5635  allocType,
    5636  m_FreeSuballocationsBySize[index],
    5637  false, // canMakeOtherLost
    5638  &pAllocationRequest->offset,
    5639  &pAllocationRequest->itemsToMakeLostCount,
    5640  &pAllocationRequest->sumFreeSize,
    5641  &pAllocationRequest->sumItemSize))
    5642  {
    5643  pAllocationRequest->item = m_FreeSuballocationsBySize[index];
    5644  return true;
    5645  }
    5646  }
    5647  }
    5648  }
    5649 
    5650  if(canMakeOtherLost)
    5651  {
    5652  // Brute-force algorithm. TODO: Come up with something better.
    5653 
    5654  pAllocationRequest->sumFreeSize = VK_WHOLE_SIZE;
    5655  pAllocationRequest->sumItemSize = VK_WHOLE_SIZE;
    5656 
    5657  VmaAllocationRequest tmpAllocRequest = {};
    5658  for(VmaSuballocationList::iterator suballocIt = m_Suballocations.begin();
    5659  suballocIt != m_Suballocations.end();
    5660  ++suballocIt)
    5661  {
    5662  if(suballocIt->type == VMA_SUBALLOCATION_TYPE_FREE ||
    5663  suballocIt->hAllocation->CanBecomeLost())
    5664  {
    5665  if(CheckAllocation(
    5666  currentFrameIndex,
    5667  frameInUseCount,
    5668  bufferImageGranularity,
    5669  allocSize,
    5670  allocAlignment,
    5671  allocType,
    5672  suballocIt,
    5673  canMakeOtherLost,
    5674  &tmpAllocRequest.offset,
    5675  &tmpAllocRequest.itemsToMakeLostCount,
    5676  &tmpAllocRequest.sumFreeSize,
    5677  &tmpAllocRequest.sumItemSize))
    5678  {
    5679  tmpAllocRequest.item = suballocIt;
    5680 
    5681  if(tmpAllocRequest.CalcCost() < pAllocationRequest->CalcCost())
    5682  {
    5683  *pAllocationRequest = tmpAllocRequest;
    5684  }
    5685  }
    5686  }
    5687  }
    5688 
    5689  if(pAllocationRequest->sumItemSize != VK_WHOLE_SIZE)
    5690  {
    5691  return true;
    5692  }
    5693  }
    5694 
    5695  return false;
    5696 }
    5697 
    5698 bool VmaBlockMetadata::MakeRequestedAllocationsLost(
    5699  uint32_t currentFrameIndex,
    5700  uint32_t frameInUseCount,
    5701  VmaAllocationRequest* pAllocationRequest)
    5702 {
    5703  while(pAllocationRequest->itemsToMakeLostCount > 0)
    5704  {
    5705  if(pAllocationRequest->item->type == VMA_SUBALLOCATION_TYPE_FREE)
    5706  {
    5707  ++pAllocationRequest->item;
    5708  }
    5709  VMA_ASSERT(pAllocationRequest->item != m_Suballocations.end());
    5710  VMA_ASSERT(pAllocationRequest->item->hAllocation != VK_NULL_HANDLE);
    5711  VMA_ASSERT(pAllocationRequest->item->hAllocation->CanBecomeLost());
    5712  if(pAllocationRequest->item->hAllocation->MakeLost(currentFrameIndex, frameInUseCount))
    5713  {
    5714  pAllocationRequest->item = FreeSuballocation(pAllocationRequest->item);
    5715  --pAllocationRequest->itemsToMakeLostCount;
    5716  }
    5717  else
    5718  {
    5719  return false;
    5720  }
    5721  }
    5722 
    5723  VMA_HEAVY_ASSERT(Validate());
    5724  VMA_ASSERT(pAllocationRequest->item != m_Suballocations.end());
    5725  VMA_ASSERT(pAllocationRequest->item->type == VMA_SUBALLOCATION_TYPE_FREE);
    5726 
    5727  return true;
    5728 }
    5729 
    5730 uint32_t VmaBlockMetadata::MakeAllocationsLost(uint32_t currentFrameIndex, uint32_t frameInUseCount)
    5731 {
    5732  uint32_t lostAllocationCount = 0;
    5733  for(VmaSuballocationList::iterator it = m_Suballocations.begin();
    5734  it != m_Suballocations.end();
    5735  ++it)
    5736  {
    5737  if(it->type != VMA_SUBALLOCATION_TYPE_FREE &&
    5738  it->hAllocation->CanBecomeLost() &&
    5739  it->hAllocation->MakeLost(currentFrameIndex, frameInUseCount))
    5740  {
    5741  it = FreeSuballocation(it);
    5742  ++lostAllocationCount;
    5743  }
    5744  }
    5745  return lostAllocationCount;
    5746 }
    5747 
    5748 void VmaBlockMetadata::Alloc(
    5749  const VmaAllocationRequest& request,
    5750  VmaSuballocationType type,
    5751  VkDeviceSize allocSize,
    5752  VmaAllocation hAllocation)
    5753 {
    5754  VMA_ASSERT(request.item != m_Suballocations.end());
    5755  VmaSuballocation& suballoc = *request.item;
    5756  // Given suballocation is a free block.
    5757  VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
    5758  // Given offset is inside this suballocation.
    5759  VMA_ASSERT(request.offset >= suballoc.offset);
    5760  const VkDeviceSize paddingBegin = request.offset - suballoc.offset;
    5761  VMA_ASSERT(suballoc.size >= paddingBegin + allocSize);
    5762  const VkDeviceSize paddingEnd = suballoc.size - paddingBegin - allocSize;
    5763 
    5764  // Unregister this free suballocation from m_FreeSuballocationsBySize and update
    5765  // it to become used.
    5766  UnregisterFreeSuballocation(request.item);
    5767 
    5768  suballoc.offset = request.offset;
    5769  suballoc.size = allocSize;
    5770  suballoc.type = type;
    5771  suballoc.hAllocation = hAllocation;
    5772 
    5773  // If there are any free bytes remaining at the end, insert new free suballocation after current one.
    5774  if(paddingEnd)
    5775  {
    5776  VmaSuballocation paddingSuballoc = {};
    5777  paddingSuballoc.offset = request.offset + allocSize;
    5778  paddingSuballoc.size = paddingEnd;
    5779  paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
    5780  VmaSuballocationList::iterator next = request.item;
    5781  ++next;
    5782  const VmaSuballocationList::iterator paddingEndItem =
    5783  m_Suballocations.insert(next, paddingSuballoc);
    5784  RegisterFreeSuballocation(paddingEndItem);
    5785  }
    5786 
    5787  // If there are any free bytes remaining at the beginning, insert new free suballocation before current one.
    5788  if(paddingBegin)
    5789  {
    5790  VmaSuballocation paddingSuballoc = {};
    5791  paddingSuballoc.offset = request.offset - paddingBegin;
    5792  paddingSuballoc.size = paddingBegin;
    5793  paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
    5794  const VmaSuballocationList::iterator paddingBeginItem =
    5795  m_Suballocations.insert(request.item, paddingSuballoc);
    5796  RegisterFreeSuballocation(paddingBeginItem);
    5797  }
    5798 
    5799  // Update totals.
    5800  m_FreeCount = m_FreeCount - 1;
    5801  if(paddingBegin > 0)
    5802  {
    5803  ++m_FreeCount;
    5804  }
    5805  if(paddingEnd > 0)
    5806  {
    5807  ++m_FreeCount;
    5808  }
    5809  m_SumFreeSize -= allocSize;
    5810 }
    5811 
    5812 void VmaBlockMetadata::Free(const VmaAllocation allocation)
    5813 {
    5814  for(VmaSuballocationList::iterator suballocItem = m_Suballocations.begin();
    5815  suballocItem != m_Suballocations.end();
    5816  ++suballocItem)
    5817  {
    5818  VmaSuballocation& suballoc = *suballocItem;
    5819  if(suballoc.hAllocation == allocation)
    5820  {
    5821  FreeSuballocation(suballocItem);
    5822  VMA_HEAVY_ASSERT(Validate());
    5823  return;
    5824  }
    5825  }
    5826  VMA_ASSERT(0 && "Not found!");
    5827 }
    5828 
    5829 void VmaBlockMetadata::FreeAtOffset(VkDeviceSize offset)
    5830 {
    5831  for(VmaSuballocationList::iterator suballocItem = m_Suballocations.begin();
    5832  suballocItem != m_Suballocations.end();
    5833  ++suballocItem)
    5834  {
    5835  VmaSuballocation& suballoc = *suballocItem;
    5836  if(suballoc.offset == offset)
    5837  {
    5838  FreeSuballocation(suballocItem);
    5839  return;
    5840  }
    5841  }
    5842  VMA_ASSERT(0 && "Not found!");
    5843 }
    5844 
    5845 bool VmaBlockMetadata::ValidateFreeSuballocationList() const
    5846 {
    5847  VkDeviceSize lastSize = 0;
    5848  for(size_t i = 0, count = m_FreeSuballocationsBySize.size(); i < count; ++i)
    5849  {
    5850  const VmaSuballocationList::iterator it = m_FreeSuballocationsBySize[i];
    5851 
    5852  if(it->type != VMA_SUBALLOCATION_TYPE_FREE)
    5853  {
    5854  VMA_ASSERT(0);
    5855  return false;
    5856  }
    5857  if(it->size < VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
    5858  {
    5859  VMA_ASSERT(0);
    5860  return false;
    5861  }
    5862  if(it->size < lastSize)
    5863  {
    5864  VMA_ASSERT(0);
    5865  return false;
    5866  }
    5867 
    5868  lastSize = it->size;
    5869  }
    5870  return true;
    5871 }
    5872 
    5873 bool VmaBlockMetadata::CheckAllocation(
    5874  uint32_t currentFrameIndex,
    5875  uint32_t frameInUseCount,
    5876  VkDeviceSize bufferImageGranularity,
    5877  VkDeviceSize allocSize,
    5878  VkDeviceSize allocAlignment,
    5879  VmaSuballocationType allocType,
    5880  VmaSuballocationList::const_iterator suballocItem,
    5881  bool canMakeOtherLost,
    5882  VkDeviceSize* pOffset,
    5883  size_t* itemsToMakeLostCount,
    5884  VkDeviceSize* pSumFreeSize,
    5885  VkDeviceSize* pSumItemSize) const
    5886 {
    5887  VMA_ASSERT(allocSize > 0);
    5888  VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
    5889  VMA_ASSERT(suballocItem != m_Suballocations.cend());
    5890  VMA_ASSERT(pOffset != VMA_NULL);
    5891 
    5892  *itemsToMakeLostCount = 0;
    5893  *pSumFreeSize = 0;
    5894  *pSumItemSize = 0;
    5895 
    5896  if(canMakeOtherLost)
    5897  {
    5898  if(suballocItem->type == VMA_SUBALLOCATION_TYPE_FREE)
    5899  {
    5900  *pSumFreeSize = suballocItem->size;
    5901  }
    5902  else
    5903  {
    5904  if(suballocItem->hAllocation->CanBecomeLost() &&
    5905  suballocItem->hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
    5906  {
    5907  ++*itemsToMakeLostCount;
    5908  *pSumItemSize = suballocItem->size;
    5909  }
    5910  else
    5911  {
    5912  return false;
    5913  }
    5914  }
    5915 
    5916  // Remaining size is too small for this request: Early return.
    5917  if(m_Size - suballocItem->offset < allocSize)
    5918  {
    5919  return false;
    5920  }
    5921 
    5922  // Start from offset equal to beginning of this suballocation.
    5923  *pOffset = suballocItem->offset;
    5924 
    5925  // Apply VMA_DEBUG_MARGIN at the beginning.
    5926  if((VMA_DEBUG_MARGIN > 0) && suballocItem != m_Suballocations.cbegin())
    5927  {
    5928  *pOffset += VMA_DEBUG_MARGIN;
    5929  }
    5930 
    5931  // Apply alignment.
    5932  const VkDeviceSize alignment = VMA_MAX(allocAlignment, static_cast<VkDeviceSize>(VMA_DEBUG_ALIGNMENT));
    5933  *pOffset = VmaAlignUp(*pOffset, alignment);
    5934 
    5935  // Check previous suballocations for BufferImageGranularity conflicts.
    5936  // Make bigger alignment if necessary.
    5937  if(bufferImageGranularity > 1)
    5938  {
    5939  bool bufferImageGranularityConflict = false;
    5940  VmaSuballocationList::const_iterator prevSuballocItem = suballocItem;
    5941  while(prevSuballocItem != m_Suballocations.cbegin())
    5942  {
    5943  --prevSuballocItem;
    5944  const VmaSuballocation& prevSuballoc = *prevSuballocItem;
    5945  if(VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, *pOffset, bufferImageGranularity))
    5946  {
    5947  if(VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
    5948  {
    5949  bufferImageGranularityConflict = true;
    5950  break;
    5951  }
    5952  }
    5953  else
    5954  // Already on previous page.
    5955  break;
    5956  }
    5957  if(bufferImageGranularityConflict)
    5958  {
    5959  *pOffset = VmaAlignUp(*pOffset, bufferImageGranularity);
    5960  }
    5961  }
    5962 
    5963  // Now that we have final *pOffset, check if we are past suballocItem.
    5964  // If yes, return false - this function should be called for another suballocItem as starting point.
    5965  if(*pOffset >= suballocItem->offset + suballocItem->size)
    5966  {
    5967  return false;
    5968  }
    5969 
    5970  // Calculate padding at the beginning based on current offset.
    5971  const VkDeviceSize paddingBegin = *pOffset - suballocItem->offset;
    5972 
    5973  // Calculate required margin at the end if this is not last suballocation.
    5974  VmaSuballocationList::const_iterator next = suballocItem;
    5975  ++next;
    5976  const VkDeviceSize requiredEndMargin =
    5977  (next != m_Suballocations.cend()) ? VMA_DEBUG_MARGIN : 0;
    5978 
    5979  const VkDeviceSize totalSize = paddingBegin + allocSize + requiredEndMargin;
    5980  // Another early return check.
    5981  if(suballocItem->offset + totalSize > m_Size)
    5982  {
    5983  return false;
    5984  }
    5985 
    5986  // Advance lastSuballocItem until desired size is reached.
    5987  // Update itemsToMakeLostCount.
    5988  VmaSuballocationList::const_iterator lastSuballocItem = suballocItem;
    5989  if(totalSize > suballocItem->size)
    5990  {
    5991  VkDeviceSize remainingSize = totalSize - suballocItem->size;
    5992  while(remainingSize > 0)
    5993  {
    5994  ++lastSuballocItem;
    5995  if(lastSuballocItem == m_Suballocations.cend())
    5996  {
    5997  return false;
    5998  }
    5999  if(lastSuballocItem->type == VMA_SUBALLOCATION_TYPE_FREE)
    6000  {
    6001  *pSumFreeSize += lastSuballocItem->size;
    6002  }
    6003  else
    6004  {
    6005  VMA_ASSERT(lastSuballocItem->hAllocation != VK_NULL_HANDLE);
    6006  if(lastSuballocItem->hAllocation->CanBecomeLost() &&
    6007  lastSuballocItem->hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
    6008  {
    6009  ++*itemsToMakeLostCount;
    6010  *pSumItemSize += lastSuballocItem->size;
    6011  }
    6012  else
    6013  {
    6014  return false;
    6015  }
    6016  }
    6017  remainingSize = (lastSuballocItem->size < remainingSize) ?
    6018  remainingSize - lastSuballocItem->size : 0;
    6019  }
    6020  }
    6021 
    6022  // Check next suballocations for BufferImageGranularity conflicts.
    6023  // If conflict exists, we must mark more allocations lost or fail.
    6024  if(bufferImageGranularity > 1)
    6025  {
    6026  VmaSuballocationList::const_iterator nextSuballocItem = lastSuballocItem;
    6027  ++nextSuballocItem;
    6028  while(nextSuballocItem != m_Suballocations.cend())
    6029  {
    6030  const VmaSuballocation& nextSuballoc = *nextSuballocItem;
    6031  if(VmaBlocksOnSamePage(*pOffset, allocSize, nextSuballoc.offset, bufferImageGranularity))
    6032  {
    6033  if(VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
    6034  {
    6035  VMA_ASSERT(nextSuballoc.hAllocation != VK_NULL_HANDLE);
    6036  if(nextSuballoc.hAllocation->CanBecomeLost() &&
    6037  nextSuballoc.hAllocation->GetLastUseFrameIndex() + frameInUseCount < currentFrameIndex)
    6038  {
    6039  ++*itemsToMakeLostCount;
    6040  }
    6041  else
    6042  {
    6043  return false;
    6044  }
    6045  }
    6046  }
    6047  else
    6048  {
    6049  // Already on next page.
    6050  break;
    6051  }
    6052  ++nextSuballocItem;
    6053  }
    6054  }
    6055  }
    6056  else
    6057  {
    6058  const VmaSuballocation& suballoc = *suballocItem;
    6059  VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
    6060 
    6061  *pSumFreeSize = suballoc.size;
    6062 
    6063  // Size of this suballocation is too small for this request: Early return.
    6064  if(suballoc.size < allocSize)
    6065  {
    6066  return false;
    6067  }
    6068 
    6069  // Start from offset equal to beginning of this suballocation.
    6070  *pOffset = suballoc.offset;
    6071 
    6072  // Apply VMA_DEBUG_MARGIN at the beginning.
    6073  if((VMA_DEBUG_MARGIN > 0) && suballocItem != m_Suballocations.cbegin())
    6074  {
    6075  *pOffset += VMA_DEBUG_MARGIN;
    6076  }
    6077 
    6078  // Apply alignment.
    6079  const VkDeviceSize alignment = VMA_MAX(allocAlignment, static_cast<VkDeviceSize>(VMA_DEBUG_ALIGNMENT));
    6080  *pOffset = VmaAlignUp(*pOffset, alignment);
    6081 
    6082  // Check previous suballocations for BufferImageGranularity conflicts.
    6083  // Make bigger alignment if necessary.
    6084  if(bufferImageGranularity > 1)
    6085  {
    6086  bool bufferImageGranularityConflict = false;
    6087  VmaSuballocationList::const_iterator prevSuballocItem = suballocItem;
    6088  while(prevSuballocItem != m_Suballocations.cbegin())
    6089  {
    6090  --prevSuballocItem;
    6091  const VmaSuballocation& prevSuballoc = *prevSuballocItem;
    6092  if(VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, *pOffset, bufferImageGranularity))
    6093  {
    6094  if(VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
    6095  {
    6096  bufferImageGranularityConflict = true;
    6097  break;
    6098  }
    6099  }
    6100  else
    6101  // Already on previous page.
    6102  break;
    6103  }
    6104  if(bufferImageGranularityConflict)
    6105  {
    6106  *pOffset = VmaAlignUp(*pOffset, bufferImageGranularity);
    6107  }
    6108  }
    6109 
    6110  // Calculate padding at the beginning based on current offset.
    6111  const VkDeviceSize paddingBegin = *pOffset - suballoc.offset;
    6112 
    6113  // Calculate required margin at the end if this is not last suballocation.
    6114  VmaSuballocationList::const_iterator next = suballocItem;
    6115  ++next;
    6116  const VkDeviceSize requiredEndMargin =
    6117  (next != m_Suballocations.cend()) ? VMA_DEBUG_MARGIN : 0;
    6118 
    6119  // Fail if requested size plus margin before and after is bigger than size of this suballocation.
    6120  if(paddingBegin + allocSize + requiredEndMargin > suballoc.size)
    6121  {
    6122  return false;
    6123  }
    6124 
    6125  // Check next suballocations for BufferImageGranularity conflicts.
    6126  // If conflict exists, allocation cannot be made here.
    6127  if(bufferImageGranularity > 1)
    6128  {
    6129  VmaSuballocationList::const_iterator nextSuballocItem = suballocItem;
    6130  ++nextSuballocItem;
    6131  while(nextSuballocItem != m_Suballocations.cend())
    6132  {
    6133  const VmaSuballocation& nextSuballoc = *nextSuballocItem;
    6134  if(VmaBlocksOnSamePage(*pOffset, allocSize, nextSuballoc.offset, bufferImageGranularity))
    6135  {
    6136  if(VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
    6137  {
    6138  return false;
    6139  }
    6140  }
    6141  else
    6142  {
    6143  // Already on next page.
    6144  break;
    6145  }
    6146  ++nextSuballocItem;
    6147  }
    6148  }
    6149  }
    6150 
    6151  // All tests passed: Success. pOffset is already filled.
    6152  return true;
    6153 }
    6154 
    6155 void VmaBlockMetadata::MergeFreeWithNext(VmaSuballocationList::iterator item)
    6156 {
    6157  VMA_ASSERT(item != m_Suballocations.end());
    6158  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
    6159 
    6160  VmaSuballocationList::iterator nextItem = item;
    6161  ++nextItem;
    6162  VMA_ASSERT(nextItem != m_Suballocations.end());
    6163  VMA_ASSERT(nextItem->type == VMA_SUBALLOCATION_TYPE_FREE);
    6164 
    6165  item->size += nextItem->size;
    6166  --m_FreeCount;
    6167  m_Suballocations.erase(nextItem);
    6168 }
    6169 
    6170 VmaSuballocationList::iterator VmaBlockMetadata::FreeSuballocation(VmaSuballocationList::iterator suballocItem)
    6171 {
    6172  // Change this suballocation to be marked as free.
    6173  VmaSuballocation& suballoc = *suballocItem;
    6174  suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
    6175  suballoc.hAllocation = VK_NULL_HANDLE;
    6176 
    6177  // Update totals.
    6178  ++m_FreeCount;
    6179  m_SumFreeSize += suballoc.size;
    6180 
    6181  // Merge with previous and/or next suballocation if it's also free.
    6182  bool mergeWithNext = false;
    6183  bool mergeWithPrev = false;
    6184 
    6185  VmaSuballocationList::iterator nextItem = suballocItem;
    6186  ++nextItem;
    6187  if((nextItem != m_Suballocations.end()) && (nextItem->type == VMA_SUBALLOCATION_TYPE_FREE))
    6188  {
    6189  mergeWithNext = true;
    6190  }
    6191 
    6192  VmaSuballocationList::iterator prevItem = suballocItem;
    6193  if(suballocItem != m_Suballocations.begin())
    6194  {
    6195  --prevItem;
    6196  if(prevItem->type == VMA_SUBALLOCATION_TYPE_FREE)
    6197  {
    6198  mergeWithPrev = true;
    6199  }
    6200  }
    6201 
    6202  if(mergeWithNext)
    6203  {
    6204  UnregisterFreeSuballocation(nextItem);
    6205  MergeFreeWithNext(suballocItem);
    6206  }
    6207 
    6208  if(mergeWithPrev)
    6209  {
    6210  UnregisterFreeSuballocation(prevItem);
    6211  MergeFreeWithNext(prevItem);
    6212  RegisterFreeSuballocation(prevItem);
    6213  return prevItem;
    6214  }
    6215  else
    6216  {
    6217  RegisterFreeSuballocation(suballocItem);
    6218  return suballocItem;
    6219  }
    6220 }
    6221 
    6222 void VmaBlockMetadata::RegisterFreeSuballocation(VmaSuballocationList::iterator item)
    6223 {
    6224  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
    6225  VMA_ASSERT(item->size > 0);
    6226 
    6227  // You may want to enable this validation at the beginning or at the end of
    6228  // this function, depending on what do you want to check.
    6229  VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
    6230 
    6231  if(item->size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
    6232  {
    6233  if(m_FreeSuballocationsBySize.empty())
    6234  {
    6235  m_FreeSuballocationsBySize.push_back(item);
    6236  }
    6237  else
    6238  {
    6239  VmaVectorInsertSorted<VmaSuballocationItemSizeLess>(m_FreeSuballocationsBySize, item);
    6240  }
    6241  }
    6242 
    6243  //VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
    6244 }
    6245 
    6246 
    6247 void VmaBlockMetadata::UnregisterFreeSuballocation(VmaSuballocationList::iterator item)
    6248 {
    6249  VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
    6250  VMA_ASSERT(item->size > 0);
    6251 
    6252  // You may want to enable this validation at the beginning or at the end of
    6253  // this function, depending on what do you want to check.
    6254  VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
    6255 
    6256  if(item->size >= VMA_MIN_FREE_SUBALLOCATION_SIZE_TO_REGISTER)
    6257  {
    6258  VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
    6259  m_FreeSuballocationsBySize.data(),
    6260  m_FreeSuballocationsBySize.data() + m_FreeSuballocationsBySize.size(),
    6261  item,
    6262  VmaSuballocationItemSizeLess());
    6263  for(size_t index = it - m_FreeSuballocationsBySize.data();
    6264  index < m_FreeSuballocationsBySize.size();
    6265  ++index)
    6266  {
    6267  if(m_FreeSuballocationsBySize[index] == item)
    6268  {
    6269  VmaVectorRemove(m_FreeSuballocationsBySize, index);
    6270  return;
    6271  }
    6272  VMA_ASSERT((m_FreeSuballocationsBySize[index]->size == item->size) && "Not found.");
    6273  }
    6274  VMA_ASSERT(0 && "Not found.");
    6275  }
    6276 
    6277  //VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
    6278 }
    6279 
    6281 // class VmaDeviceMemoryBlock
    6282 
    6283 VmaDeviceMemoryBlock::VmaDeviceMemoryBlock(VmaAllocator hAllocator) :
    6284  m_Metadata(hAllocator),
    6285  m_MemoryTypeIndex(UINT32_MAX),
    6286  m_hMemory(VK_NULL_HANDLE),
    6287  m_MapCount(0),
    6288  m_pMappedData(VMA_NULL)
    6289 {
    6290 }
    6291 
    6292 void VmaDeviceMemoryBlock::Init(
    6293  uint32_t newMemoryTypeIndex,
    6294  VkDeviceMemory newMemory,
    6295  VkDeviceSize newSize)
    6296 {
    6297  VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
    6298 
    6299  m_MemoryTypeIndex = newMemoryTypeIndex;
    6300  m_hMemory = newMemory;
    6301 
    6302  m_Metadata.Init(newSize);
    6303 }
    6304 
    6305 void VmaDeviceMemoryBlock::Destroy(VmaAllocator allocator)
    6306 {
    6307  // This is the most important assert in the entire library.
    6308  // Hitting it means you have some memory leak - unreleased VmaAllocation objects.
    6309  VMA_ASSERT(m_Metadata.IsEmpty() && "Some allocations were not freed before destruction of this memory block!");
    6310 
    6311  VMA_ASSERT(m_hMemory != VK_NULL_HANDLE);
    6312  allocator->FreeVulkanMemory(m_MemoryTypeIndex, m_Metadata.GetSize(), m_hMemory);
    6313  m_hMemory = VK_NULL_HANDLE;
    6314 }
    6315 
    6316 bool VmaDeviceMemoryBlock::Validate() const
    6317 {
    6318  if((m_hMemory == VK_NULL_HANDLE) ||
    6319  (m_Metadata.GetSize() == 0))
    6320  {
    6321  return false;
    6322  }
    6323 
    6324  return m_Metadata.Validate();
    6325 }
    6326 
    6327 VkResult VmaDeviceMemoryBlock::Map(VmaAllocator hAllocator, uint32_t count, void** ppData)
    6328 {
    6329  if(count == 0)
    6330  {
    6331  return VK_SUCCESS;
    6332  }
    6333 
    6334  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
    6335  if(m_MapCount != 0)
    6336  {
    6337  m_MapCount += count;
    6338  VMA_ASSERT(m_pMappedData != VMA_NULL);
    6339  if(ppData != VMA_NULL)
    6340  {
    6341  *ppData = m_pMappedData;
    6342  }
    6343  return VK_SUCCESS;
    6344  }
    6345  else
    6346  {
    6347  VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
    6348  hAllocator->m_hDevice,
    6349  m_hMemory,
    6350  0, // offset
    6351  VK_WHOLE_SIZE,
    6352  0, // flags
    6353  &m_pMappedData);
    6354  if(result == VK_SUCCESS)
    6355  {
    6356  if(ppData != VMA_NULL)
    6357  {
    6358  *ppData = m_pMappedData;
    6359  }
    6360  m_MapCount = count;
    6361  }
    6362  return result;
    6363  }
    6364 }
    6365 
    6366 void VmaDeviceMemoryBlock::Unmap(VmaAllocator hAllocator, uint32_t count)
    6367 {
    6368  if(count == 0)
    6369  {
    6370  return;
    6371  }
    6372 
    6373  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
    6374  if(m_MapCount >= count)
    6375  {
    6376  m_MapCount -= count;
    6377  if(m_MapCount == 0)
    6378  {
    6379  m_pMappedData = VMA_NULL;
    6380  (*hAllocator->GetVulkanFunctions().vkUnmapMemory)(hAllocator->m_hDevice, m_hMemory);
    6381  }
    6382  }
    6383  else
    6384  {
    6385  VMA_ASSERT(0 && "VkDeviceMemory block is being unmapped while it was not previously mapped.");
    6386  }
    6387 }
    6388 
    6389 VkResult VmaDeviceMemoryBlock::BindBufferMemory(
    6390  const VmaAllocator hAllocator,
    6391  const VmaAllocation hAllocation,
    6392  VkBuffer hBuffer)
    6393 {
    6394  VMA_ASSERT(hAllocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK &&
    6395  hAllocation->GetBlock() == this);
    6396  // This lock is important so that we don't call vkBind... and/or vkMap... simultaneously on the same VkDeviceMemory from multiple threads.
    6397  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
    6398  return hAllocator->GetVulkanFunctions().vkBindBufferMemory(
    6399  hAllocator->m_hDevice,
    6400  hBuffer,
    6401  m_hMemory,
    6402  hAllocation->GetOffset());
    6403 }
    6404 
    6405 VkResult VmaDeviceMemoryBlock::BindImageMemory(
    6406  const VmaAllocator hAllocator,
    6407  const VmaAllocation hAllocation,
    6408  VkImage hImage)
    6409 {
    6410  VMA_ASSERT(hAllocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK &&
    6411  hAllocation->GetBlock() == this);
    6412  // This lock is important so that we don't call vkBind... and/or vkMap... simultaneously on the same VkDeviceMemory from multiple threads.
    6413  VmaMutexLock lock(m_Mutex, hAllocator->m_UseMutex);
    6414  return hAllocator->GetVulkanFunctions().vkBindImageMemory(
    6415  hAllocator->m_hDevice,
    6416  hImage,
    6417  m_hMemory,
    6418  hAllocation->GetOffset());
    6419 }
    6420 
    6421 static void InitStatInfo(VmaStatInfo& outInfo)
    6422 {
    6423  memset(&outInfo, 0, sizeof(outInfo));
    6424  outInfo.allocationSizeMin = UINT64_MAX;
    6425  outInfo.unusedRangeSizeMin = UINT64_MAX;
    6426 }
    6427 
    6428 // Adds statistics srcInfo into inoutInfo, like: inoutInfo += srcInfo.
    6429 static void VmaAddStatInfo(VmaStatInfo& inoutInfo, const VmaStatInfo& srcInfo)
    6430 {
    6431  inoutInfo.blockCount += srcInfo.blockCount;
    6432  inoutInfo.allocationCount += srcInfo.allocationCount;
    6433  inoutInfo.unusedRangeCount += srcInfo.unusedRangeCount;
    6434  inoutInfo.usedBytes += srcInfo.usedBytes;
    6435  inoutInfo.unusedBytes += srcInfo.unusedBytes;
    6436  inoutInfo.allocationSizeMin = VMA_MIN(inoutInfo.allocationSizeMin, srcInfo.allocationSizeMin);
    6437  inoutInfo.allocationSizeMax = VMA_MAX(inoutInfo.allocationSizeMax, srcInfo.allocationSizeMax);
    6438  inoutInfo.unusedRangeSizeMin = VMA_MIN(inoutInfo.unusedRangeSizeMin, srcInfo.unusedRangeSizeMin);
    6439  inoutInfo.unusedRangeSizeMax = VMA_MAX(inoutInfo.unusedRangeSizeMax, srcInfo.unusedRangeSizeMax);
    6440 }
    6441 
    6442 static void VmaPostprocessCalcStatInfo(VmaStatInfo& inoutInfo)
    6443 {
    6444  inoutInfo.allocationSizeAvg = (inoutInfo.allocationCount > 0) ?
    6445  VmaRoundDiv<VkDeviceSize>(inoutInfo.usedBytes, inoutInfo.allocationCount) : 0;
    6446  inoutInfo.unusedRangeSizeAvg = (inoutInfo.unusedRangeCount > 0) ?
    6447  VmaRoundDiv<VkDeviceSize>(inoutInfo.unusedBytes, inoutInfo.unusedRangeCount) : 0;
    6448 }
    6449 
    6450 VmaPool_T::VmaPool_T(
    6451  VmaAllocator hAllocator,
    6452  const VmaPoolCreateInfo& createInfo) :
    6453  m_BlockVector(
    6454  hAllocator,
    6455  createInfo.memoryTypeIndex,
    6456  createInfo.blockSize,
    6457  createInfo.minBlockCount,
    6458  createInfo.maxBlockCount,
    6459  (createInfo.flags & VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT) != 0 ? 1 : hAllocator->GetBufferImageGranularity(),
    6460  createInfo.frameInUseCount,
    6461  true) // isCustomPool
    6462 {
    6463 }
    6464 
    6465 VmaPool_T::~VmaPool_T()
    6466 {
    6467 }
    6468 
    6469 #if VMA_STATS_STRING_ENABLED
    6470 
    6471 #endif // #if VMA_STATS_STRING_ENABLED
    6472 
    6473 VmaBlockVector::VmaBlockVector(
    6474  VmaAllocator hAllocator,
    6475  uint32_t memoryTypeIndex,
    6476  VkDeviceSize preferredBlockSize,
    6477  size_t minBlockCount,
    6478  size_t maxBlockCount,
    6479  VkDeviceSize bufferImageGranularity,
    6480  uint32_t frameInUseCount,
    6481  bool isCustomPool) :
    6482  m_hAllocator(hAllocator),
    6483  m_MemoryTypeIndex(memoryTypeIndex),
    6484  m_PreferredBlockSize(preferredBlockSize),
    6485  m_MinBlockCount(minBlockCount),
    6486  m_MaxBlockCount(maxBlockCount),
    6487  m_BufferImageGranularity(bufferImageGranularity),
    6488  m_FrameInUseCount(frameInUseCount),
    6489  m_IsCustomPool(isCustomPool),
    6490  m_Blocks(VmaStlAllocator<VmaDeviceMemoryBlock*>(hAllocator->GetAllocationCallbacks())),
    6491  m_HasEmptyBlock(false),
    6492  m_pDefragmentator(VMA_NULL)
    6493 {
    6494 }
    6495 
    6496 VmaBlockVector::~VmaBlockVector()
    6497 {
    6498  VMA_ASSERT(m_pDefragmentator == VMA_NULL);
    6499 
    6500  for(size_t i = m_Blocks.size(); i--; )
    6501  {
    6502  m_Blocks[i]->Destroy(m_hAllocator);
    6503  vma_delete(m_hAllocator, m_Blocks[i]);
    6504  }
    6505 }
    6506 
    6507 VkResult VmaBlockVector::CreateMinBlocks()
    6508 {
    6509  for(size_t i = 0; i < m_MinBlockCount; ++i)
    6510  {
    6511  VkResult res = CreateBlock(m_PreferredBlockSize, VMA_NULL);
    6512  if(res != VK_SUCCESS)
    6513  {
    6514  return res;
    6515  }
    6516  }
    6517  return VK_SUCCESS;
    6518 }
    6519 
    6520 void VmaBlockVector::GetPoolStats(VmaPoolStats* pStats)
    6521 {
    6522  pStats->size = 0;
    6523  pStats->unusedSize = 0;
    6524  pStats->allocationCount = 0;
    6525  pStats->unusedRangeCount = 0;
    6526  pStats->unusedRangeSizeMax = 0;
    6527 
    6528  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    6529 
    6530  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
    6531  {
    6532  const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
    6533  VMA_ASSERT(pBlock);
    6534  VMA_HEAVY_ASSERT(pBlock->Validate());
    6535  pBlock->m_Metadata.AddPoolStats(*pStats);
    6536  }
    6537 }
    6538 
    6539 static const uint32_t VMA_ALLOCATION_TRY_COUNT = 32;
    6540 
    6541 VkResult VmaBlockVector::Allocate(
    6542  VmaPool hCurrentPool,
    6543  uint32_t currentFrameIndex,
    6544  const VkMemoryRequirements& vkMemReq,
    6545  const VmaAllocationCreateInfo& createInfo,
    6546  VmaSuballocationType suballocType,
    6547  VmaAllocation* pAllocation)
    6548 {
    6549  const bool mapped = (createInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0;
    6550  const bool isUserDataString = (createInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0;
    6551 
    6552  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    6553 
    6554  // 1. Search existing allocations. Try to allocate without making other allocations lost.
    6555  // Forward order in m_Blocks - prefer blocks with smallest amount of free space.
    6556  for(size_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex )
    6557  {
    6558  VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
    6559  VMA_ASSERT(pCurrBlock);
    6560  VmaAllocationRequest currRequest = {};
    6561  if(pCurrBlock->m_Metadata.CreateAllocationRequest(
    6562  currentFrameIndex,
    6563  m_FrameInUseCount,
    6564  m_BufferImageGranularity,
    6565  vkMemReq.size,
    6566  vkMemReq.alignment,
    6567  suballocType,
    6568  false, // canMakeOtherLost
    6569  &currRequest))
    6570  {
    6571  // Allocate from pCurrBlock.
    6572  VMA_ASSERT(currRequest.itemsToMakeLostCount == 0);
    6573 
    6574  if(mapped)
    6575  {
    6576  VkResult res = pCurrBlock->Map(m_hAllocator, 1, VMA_NULL);
    6577  if(res != VK_SUCCESS)
    6578  {
    6579  return res;
    6580  }
    6581  }
    6582 
    6583  // We no longer have an empty Allocation.
    6584  if(pCurrBlock->m_Metadata.IsEmpty())
    6585  {
    6586  m_HasEmptyBlock = false;
    6587  }
    6588 
    6589  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
    6590  pCurrBlock->m_Metadata.Alloc(currRequest, suballocType, vkMemReq.size, *pAllocation);
    6591  (*pAllocation)->InitBlockAllocation(
    6592  hCurrentPool,
    6593  pCurrBlock,
    6594  currRequest.offset,
    6595  vkMemReq.alignment,
    6596  vkMemReq.size,
    6597  suballocType,
    6598  mapped,
    6599  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
    6600  VMA_HEAVY_ASSERT(pCurrBlock->Validate());
    6601  VMA_DEBUG_LOG(" Returned from existing allocation #%u", (uint32_t)blockIndex);
    6602  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
    6603  return VK_SUCCESS;
    6604  }
    6605  }
    6606 
    6607  const bool canCreateNewBlock =
    6608  ((createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == 0) &&
    6609  (m_Blocks.size() < m_MaxBlockCount);
    6610 
    6611  // 2. Try to create new block.
    6612  if(canCreateNewBlock)
    6613  {
    6614  // Calculate optimal size for new block.
    6615  VkDeviceSize newBlockSize = m_PreferredBlockSize;
    6616  uint32_t newBlockSizeShift = 0;
    6617  const uint32_t NEW_BLOCK_SIZE_SHIFT_MAX = 3;
    6618 
    6619  // Allocating blocks of other sizes is allowed only in default pools.
    6620  // In custom pools block size is fixed.
    6621  if(m_IsCustomPool == false)
    6622  {
    6623  // Allocate 1/8, 1/4, 1/2 as first blocks.
    6624  const VkDeviceSize maxExistingBlockSize = CalcMaxBlockSize();
    6625  for(uint32_t i = 0; i < NEW_BLOCK_SIZE_SHIFT_MAX; ++i)
    6626  {
    6627  const VkDeviceSize smallerNewBlockSize = newBlockSize / 2;
    6628  if(smallerNewBlockSize > maxExistingBlockSize && smallerNewBlockSize >= vkMemReq.size * 2)
    6629  {
    6630  newBlockSize = smallerNewBlockSize;
    6631  ++newBlockSizeShift;
    6632  }
    6633  else
    6634  {
    6635  break;
    6636  }
    6637  }
    6638  }
    6639 
    6640  size_t newBlockIndex = 0;
    6641  VkResult res = CreateBlock(newBlockSize, &newBlockIndex);
    6642  // Allocation of this size failed? Try 1/2, 1/4, 1/8 of m_PreferredBlockSize.
    6643  if(m_IsCustomPool == false)
    6644  {
    6645  while(res < 0 && newBlockSizeShift < NEW_BLOCK_SIZE_SHIFT_MAX)
    6646  {
    6647  const VkDeviceSize smallerNewBlockSize = newBlockSize / 2;
    6648  if(smallerNewBlockSize >= vkMemReq.size)
    6649  {
    6650  newBlockSize = smallerNewBlockSize;
    6651  ++newBlockSizeShift;
    6652  res = CreateBlock(newBlockSize, &newBlockIndex);
    6653  }
    6654  else
    6655  {
    6656  break;
    6657  }
    6658  }
    6659  }
    6660 
    6661  if(res == VK_SUCCESS)
    6662  {
    6663  VmaDeviceMemoryBlock* const pBlock = m_Blocks[newBlockIndex];
    6664  VMA_ASSERT(pBlock->m_Metadata.GetSize() >= vkMemReq.size);
    6665 
    6666  if(mapped)
    6667  {
    6668  res = pBlock->Map(m_hAllocator, 1, VMA_NULL);
    6669  if(res != VK_SUCCESS)
    6670  {
    6671  return res;
    6672  }
    6673  }
    6674 
    6675  // Allocate from pBlock. Because it is empty, dstAllocRequest can be trivially filled.
    6676  VmaAllocationRequest allocRequest;
    6677  pBlock->m_Metadata.CreateFirstAllocationRequest(&allocRequest);
    6678  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
    6679  pBlock->m_Metadata.Alloc(allocRequest, suballocType, vkMemReq.size, *pAllocation);
    6680  (*pAllocation)->InitBlockAllocation(
    6681  hCurrentPool,
    6682  pBlock,
    6683  allocRequest.offset,
    6684  vkMemReq.alignment,
    6685  vkMemReq.size,
    6686  suballocType,
    6687  mapped,
    6688  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
    6689  VMA_HEAVY_ASSERT(pBlock->Validate());
    6690  VMA_DEBUG_LOG(" Created new allocation Size=%llu", allocInfo.allocationSize);
    6691  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
    6692  return VK_SUCCESS;
    6693  }
    6694  }
    6695 
    6696  const bool canMakeOtherLost = (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT) != 0;
    6697 
    6698  // 3. Try to allocate from existing blocks with making other allocations lost.
    6699  if(canMakeOtherLost)
    6700  {
    6701  uint32_t tryIndex = 0;
    6702  for(; tryIndex < VMA_ALLOCATION_TRY_COUNT; ++tryIndex)
    6703  {
    6704  VmaDeviceMemoryBlock* pBestRequestBlock = VMA_NULL;
    6705  VmaAllocationRequest bestRequest = {};
    6706  VkDeviceSize bestRequestCost = VK_WHOLE_SIZE;
    6707 
    6708  // 1. Search existing allocations.
    6709  // Forward order in m_Blocks - prefer blocks with smallest amount of free space.
    6710  for(size_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex )
    6711  {
    6712  VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
    6713  VMA_ASSERT(pCurrBlock);
    6714  VmaAllocationRequest currRequest = {};
    6715  if(pCurrBlock->m_Metadata.CreateAllocationRequest(
    6716  currentFrameIndex,
    6717  m_FrameInUseCount,
    6718  m_BufferImageGranularity,
    6719  vkMemReq.size,
    6720  vkMemReq.alignment,
    6721  suballocType,
    6722  canMakeOtherLost,
    6723  &currRequest))
    6724  {
    6725  const VkDeviceSize currRequestCost = currRequest.CalcCost();
    6726  if(pBestRequestBlock == VMA_NULL ||
    6727  currRequestCost < bestRequestCost)
    6728  {
    6729  pBestRequestBlock = pCurrBlock;
    6730  bestRequest = currRequest;
    6731  bestRequestCost = currRequestCost;
    6732 
    6733  if(bestRequestCost == 0)
    6734  {
    6735  break;
    6736  }
    6737  }
    6738  }
    6739  }
    6740 
    6741  if(pBestRequestBlock != VMA_NULL)
    6742  {
    6743  if(mapped)
    6744  {
    6745  VkResult res = pBestRequestBlock->Map(m_hAllocator, 1, VMA_NULL);
    6746  if(res != VK_SUCCESS)
    6747  {
    6748  return res;
    6749  }
    6750  }
    6751 
    6752  if(pBestRequestBlock->m_Metadata.MakeRequestedAllocationsLost(
    6753  currentFrameIndex,
    6754  m_FrameInUseCount,
    6755  &bestRequest))
    6756  {
    6757  // We no longer have an empty Allocation.
    6758  if(pBestRequestBlock->m_Metadata.IsEmpty())
    6759  {
    6760  m_HasEmptyBlock = false;
    6761  }
    6762  // Allocate from this pBlock.
    6763  *pAllocation = vma_new(m_hAllocator, VmaAllocation_T)(currentFrameIndex, isUserDataString);
    6764  pBestRequestBlock->m_Metadata.Alloc(bestRequest, suballocType, vkMemReq.size, *pAllocation);
    6765  (*pAllocation)->InitBlockAllocation(
    6766  hCurrentPool,
    6767  pBestRequestBlock,
    6768  bestRequest.offset,
    6769  vkMemReq.alignment,
    6770  vkMemReq.size,
    6771  suballocType,
    6772  mapped,
    6773  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0);
    6774  VMA_HEAVY_ASSERT(pBestRequestBlock->Validate());
    6775  VMA_DEBUG_LOG(" Returned from existing allocation #%u", (uint32_t)blockIndex);
    6776  (*pAllocation)->SetUserData(m_hAllocator, createInfo.pUserData);
    6777  return VK_SUCCESS;
    6778  }
    6779  // else: Some allocations must have been touched while we are here. Next try.
    6780  }
    6781  else
    6782  {
    6783  // Could not find place in any of the blocks - break outer loop.
    6784  break;
    6785  }
    6786  }
    6787  /* Maximum number of tries exceeded - a very unlike event when many other
    6788  threads are simultaneously touching allocations making it impossible to make
    6789  lost at the same time as we try to allocate. */
    6790  if(tryIndex == VMA_ALLOCATION_TRY_COUNT)
    6791  {
    6792  return VK_ERROR_TOO_MANY_OBJECTS;
    6793  }
    6794  }
    6795 
    6796  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    6797 }
    6798 
    6799 void VmaBlockVector::Free(
    6800  VmaAllocation hAllocation)
    6801 {
    6802  VmaDeviceMemoryBlock* pBlockToDelete = VMA_NULL;
    6803 
    6804  // Scope for lock.
    6805  {
    6806  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    6807 
    6808  VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
    6809 
    6810  if(hAllocation->IsPersistentMap())
    6811  {
    6812  pBlock->Unmap(m_hAllocator, 1);
    6813  }
    6814 
    6815  pBlock->m_Metadata.Free(hAllocation);
    6816  VMA_HEAVY_ASSERT(pBlock->Validate());
    6817 
    6818  VMA_DEBUG_LOG(" Freed from MemoryTypeIndex=%u", memTypeIndex);
    6819 
    6820  // pBlock became empty after this deallocation.
    6821  if(pBlock->m_Metadata.IsEmpty())
    6822  {
    6823  // Already has empty Allocation. We don't want to have two, so delete this one.
    6824  if(m_HasEmptyBlock && m_Blocks.size() > m_MinBlockCount)
    6825  {
    6826  pBlockToDelete = pBlock;
    6827  Remove(pBlock);
    6828  }
    6829  // We now have first empty Allocation.
    6830  else
    6831  {
    6832  m_HasEmptyBlock = true;
    6833  }
    6834  }
    6835  // pBlock didn't become empty, but we have another empty block - find and free that one.
    6836  // (This is optional, heuristics.)
    6837  else if(m_HasEmptyBlock)
    6838  {
    6839  VmaDeviceMemoryBlock* pLastBlock = m_Blocks.back();
    6840  if(pLastBlock->m_Metadata.IsEmpty() && m_Blocks.size() > m_MinBlockCount)
    6841  {
    6842  pBlockToDelete = pLastBlock;
    6843  m_Blocks.pop_back();
    6844  m_HasEmptyBlock = false;
    6845  }
    6846  }
    6847 
    6848  IncrementallySortBlocks();
    6849  }
    6850 
    6851  // Destruction of a free Allocation. Deferred until this point, outside of mutex
    6852  // lock, for performance reason.
    6853  if(pBlockToDelete != VMA_NULL)
    6854  {
    6855  VMA_DEBUG_LOG(" Deleted empty allocation");
    6856  pBlockToDelete->Destroy(m_hAllocator);
    6857  vma_delete(m_hAllocator, pBlockToDelete);
    6858  }
    6859 }
    6860 
    6861 size_t VmaBlockVector::CalcMaxBlockSize() const
    6862 {
    6863  size_t result = 0;
    6864  for(size_t i = m_Blocks.size(); i--; )
    6865  {
    6866  result = VMA_MAX((uint64_t)result, (uint64_t)m_Blocks[i]->m_Metadata.GetSize());
    6867  if(result >= m_PreferredBlockSize)
    6868  {
    6869  break;
    6870  }
    6871  }
    6872  return result;
    6873 }
    6874 
    6875 void VmaBlockVector::Remove(VmaDeviceMemoryBlock* pBlock)
    6876 {
    6877  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
    6878  {
    6879  if(m_Blocks[blockIndex] == pBlock)
    6880  {
    6881  VmaVectorRemove(m_Blocks, blockIndex);
    6882  return;
    6883  }
    6884  }
    6885  VMA_ASSERT(0);
    6886 }
    6887 
    6888 void VmaBlockVector::IncrementallySortBlocks()
    6889 {
    6890  // Bubble sort only until first swap.
    6891  for(size_t i = 1; i < m_Blocks.size(); ++i)
    6892  {
    6893  if(m_Blocks[i - 1]->m_Metadata.GetSumFreeSize() > m_Blocks[i]->m_Metadata.GetSumFreeSize())
    6894  {
    6895  VMA_SWAP(m_Blocks[i - 1], m_Blocks[i]);
    6896  return;
    6897  }
    6898  }
    6899 }
    6900 
    6901 VkResult VmaBlockVector::CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex)
    6902 {
    6903  VkMemoryAllocateInfo allocInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
    6904  allocInfo.memoryTypeIndex = m_MemoryTypeIndex;
    6905  allocInfo.allocationSize = blockSize;
    6906  VkDeviceMemory mem = VK_NULL_HANDLE;
    6907  VkResult res = m_hAllocator->AllocateVulkanMemory(&allocInfo, &mem);
    6908  if(res < 0)
    6909  {
    6910  return res;
    6911  }
    6912 
    6913  // New VkDeviceMemory successfully created.
    6914 
    6915  // Create new Allocation for it.
    6916  VmaDeviceMemoryBlock* const pBlock = vma_new(m_hAllocator, VmaDeviceMemoryBlock)(m_hAllocator);
    6917  pBlock->Init(
    6918  m_MemoryTypeIndex,
    6919  mem,
    6920  allocInfo.allocationSize);
    6921 
    6922  m_Blocks.push_back(pBlock);
    6923  if(pNewBlockIndex != VMA_NULL)
    6924  {
    6925  *pNewBlockIndex = m_Blocks.size() - 1;
    6926  }
    6927 
    6928  return VK_SUCCESS;
    6929 }
    6930 
    6931 #if VMA_STATS_STRING_ENABLED
    6932 
    6933 void VmaBlockVector::PrintDetailedMap(class VmaJsonWriter& json)
    6934 {
    6935  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    6936 
    6937  json.BeginObject();
    6938 
    6939  if(m_IsCustomPool)
    6940  {
    6941  json.WriteString("MemoryTypeIndex");
    6942  json.WriteNumber(m_MemoryTypeIndex);
    6943 
    6944  json.WriteString("BlockSize");
    6945  json.WriteNumber(m_PreferredBlockSize);
    6946 
    6947  json.WriteString("BlockCount");
    6948  json.BeginObject(true);
    6949  if(m_MinBlockCount > 0)
    6950  {
    6951  json.WriteString("Min");
    6952  json.WriteNumber((uint64_t)m_MinBlockCount);
    6953  }
    6954  if(m_MaxBlockCount < SIZE_MAX)
    6955  {
    6956  json.WriteString("Max");
    6957  json.WriteNumber((uint64_t)m_MaxBlockCount);
    6958  }
    6959  json.WriteString("Cur");
    6960  json.WriteNumber((uint64_t)m_Blocks.size());
    6961  json.EndObject();
    6962 
    6963  if(m_FrameInUseCount > 0)
    6964  {
    6965  json.WriteString("FrameInUseCount");
    6966  json.WriteNumber(m_FrameInUseCount);
    6967  }
    6968  }
    6969  else
    6970  {
    6971  json.WriteString("PreferredBlockSize");
    6972  json.WriteNumber(m_PreferredBlockSize);
    6973  }
    6974 
    6975  json.WriteString("Blocks");
    6976  json.BeginArray();
    6977  for(size_t i = 0; i < m_Blocks.size(); ++i)
    6978  {
    6979  m_Blocks[i]->m_Metadata.PrintDetailedMap(json);
    6980  }
    6981  json.EndArray();
    6982 
    6983  json.EndObject();
    6984 }
    6985 
    6986 #endif // #if VMA_STATS_STRING_ENABLED
    6987 
    6988 VmaDefragmentator* VmaBlockVector::EnsureDefragmentator(
    6989  VmaAllocator hAllocator,
    6990  uint32_t currentFrameIndex)
    6991 {
    6992  if(m_pDefragmentator == VMA_NULL)
    6993  {
    6994  m_pDefragmentator = vma_new(m_hAllocator, VmaDefragmentator)(
    6995  hAllocator,
    6996  this,
    6997  currentFrameIndex);
    6998  }
    6999 
    7000  return m_pDefragmentator;
    7001 }
    7002 
    7003 VkResult VmaBlockVector::Defragment(
    7004  VmaDefragmentationStats* pDefragmentationStats,
    7005  VkDeviceSize& maxBytesToMove,
    7006  uint32_t& maxAllocationsToMove)
    7007 {
    7008  if(m_pDefragmentator == VMA_NULL)
    7009  {
    7010  return VK_SUCCESS;
    7011  }
    7012 
    7013  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    7014 
    7015  // Defragment.
    7016  VkResult result = m_pDefragmentator->Defragment(maxBytesToMove, maxAllocationsToMove);
    7017 
    7018  // Accumulate statistics.
    7019  if(pDefragmentationStats != VMA_NULL)
    7020  {
    7021  const VkDeviceSize bytesMoved = m_pDefragmentator->GetBytesMoved();
    7022  const uint32_t allocationsMoved = m_pDefragmentator->GetAllocationsMoved();
    7023  pDefragmentationStats->bytesMoved += bytesMoved;
    7024  pDefragmentationStats->allocationsMoved += allocationsMoved;
    7025  VMA_ASSERT(bytesMoved <= maxBytesToMove);
    7026  VMA_ASSERT(allocationsMoved <= maxAllocationsToMove);
    7027  maxBytesToMove -= bytesMoved;
    7028  maxAllocationsToMove -= allocationsMoved;
    7029  }
    7030 
    7031  // Free empty blocks.
    7032  m_HasEmptyBlock = false;
    7033  for(size_t blockIndex = m_Blocks.size(); blockIndex--; )
    7034  {
    7035  VmaDeviceMemoryBlock* pBlock = m_Blocks[blockIndex];
    7036  if(pBlock->m_Metadata.IsEmpty())
    7037  {
    7038  if(m_Blocks.size() > m_MinBlockCount)
    7039  {
    7040  if(pDefragmentationStats != VMA_NULL)
    7041  {
    7042  ++pDefragmentationStats->deviceMemoryBlocksFreed;
    7043  pDefragmentationStats->bytesFreed += pBlock->m_Metadata.GetSize();
    7044  }
    7045 
    7046  VmaVectorRemove(m_Blocks, blockIndex);
    7047  pBlock->Destroy(m_hAllocator);
    7048  vma_delete(m_hAllocator, pBlock);
    7049  }
    7050  else
    7051  {
    7052  m_HasEmptyBlock = true;
    7053  }
    7054  }
    7055  }
    7056 
    7057  return result;
    7058 }
    7059 
    7060 void VmaBlockVector::DestroyDefragmentator()
    7061 {
    7062  if(m_pDefragmentator != VMA_NULL)
    7063  {
    7064  vma_delete(m_hAllocator, m_pDefragmentator);
    7065  m_pDefragmentator = VMA_NULL;
    7066  }
    7067 }
    7068 
    7069 void VmaBlockVector::MakePoolAllocationsLost(
    7070  uint32_t currentFrameIndex,
    7071  size_t* pLostAllocationCount)
    7072 {
    7073  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    7074  size_t lostAllocationCount = 0;
    7075  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
    7076  {
    7077  VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
    7078  VMA_ASSERT(pBlock);
    7079  lostAllocationCount += pBlock->m_Metadata.MakeAllocationsLost(currentFrameIndex, m_FrameInUseCount);
    7080  }
    7081  if(pLostAllocationCount != VMA_NULL)
    7082  {
    7083  *pLostAllocationCount = lostAllocationCount;
    7084  }
    7085 }
    7086 
    7087 void VmaBlockVector::AddStats(VmaStats* pStats)
    7088 {
    7089  const uint32_t memTypeIndex = m_MemoryTypeIndex;
    7090  const uint32_t memHeapIndex = m_hAllocator->MemoryTypeIndexToHeapIndex(memTypeIndex);
    7091 
    7092  VmaMutexLock lock(m_Mutex, m_hAllocator->m_UseMutex);
    7093 
    7094  for(uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
    7095  {
    7096  const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
    7097  VMA_ASSERT(pBlock);
    7098  VMA_HEAVY_ASSERT(pBlock->Validate());
    7099  VmaStatInfo allocationStatInfo;
    7100  pBlock->m_Metadata.CalcAllocationStatInfo(allocationStatInfo);
    7101  VmaAddStatInfo(pStats->total, allocationStatInfo);
    7102  VmaAddStatInfo(pStats->memoryType[memTypeIndex], allocationStatInfo);
    7103  VmaAddStatInfo(pStats->memoryHeap[memHeapIndex], allocationStatInfo);
    7104  }
    7105 }
    7106 
    7108 // VmaDefragmentator members definition
    7109 
    7110 VmaDefragmentator::VmaDefragmentator(
    7111  VmaAllocator hAllocator,
    7112  VmaBlockVector* pBlockVector,
    7113  uint32_t currentFrameIndex) :
    7114  m_hAllocator(hAllocator),
    7115  m_pBlockVector(pBlockVector),
    7116  m_CurrentFrameIndex(currentFrameIndex),
    7117  m_BytesMoved(0),
    7118  m_AllocationsMoved(0),
    7119  m_Allocations(VmaStlAllocator<AllocationInfo>(hAllocator->GetAllocationCallbacks())),
    7120  m_Blocks(VmaStlAllocator<BlockInfo*>(hAllocator->GetAllocationCallbacks()))
    7121 {
    7122 }
    7123 
    7124 VmaDefragmentator::~VmaDefragmentator()
    7125 {
    7126  for(size_t i = m_Blocks.size(); i--; )
    7127  {
    7128  vma_delete(m_hAllocator, m_Blocks[i]);
    7129  }
    7130 }
    7131 
    7132 void VmaDefragmentator::AddAllocation(VmaAllocation hAlloc, VkBool32* pChanged)
    7133 {
    7134  AllocationInfo allocInfo;
    7135  allocInfo.m_hAllocation = hAlloc;
    7136  allocInfo.m_pChanged = pChanged;
    7137  m_Allocations.push_back(allocInfo);
    7138 }
    7139 
    7140 VkResult VmaDefragmentator::BlockInfo::EnsureMapping(VmaAllocator hAllocator, void** ppMappedData)
    7141 {
    7142  // It has already been mapped for defragmentation.
    7143  if(m_pMappedDataForDefragmentation)
    7144  {
    7145  *ppMappedData = m_pMappedDataForDefragmentation;
    7146  return VK_SUCCESS;
    7147  }
    7148 
    7149  // It is originally mapped.
    7150  if(m_pBlock->GetMappedData())
    7151  {
    7152  *ppMappedData = m_pBlock->GetMappedData();
    7153  return VK_SUCCESS;
    7154  }
    7155 
    7156  // Map on first usage.
    7157  VkResult res = m_pBlock->Map(hAllocator, 1, &m_pMappedDataForDefragmentation);
    7158  *ppMappedData = m_pMappedDataForDefragmentation;
    7159  return res;
    7160 }
    7161 
    7162 void VmaDefragmentator::BlockInfo::Unmap(VmaAllocator hAllocator)
    7163 {
    7164  if(m_pMappedDataForDefragmentation != VMA_NULL)
    7165  {
    7166  m_pBlock->Unmap(hAllocator, 1);
    7167  }
    7168 }
    7169 
    7170 VkResult VmaDefragmentator::DefragmentRound(
    7171  VkDeviceSize maxBytesToMove,
    7172  uint32_t maxAllocationsToMove)
    7173 {
    7174  if(m_Blocks.empty())
    7175  {
    7176  return VK_SUCCESS;
    7177  }
    7178 
    7179  size_t srcBlockIndex = m_Blocks.size() - 1;
    7180  size_t srcAllocIndex = SIZE_MAX;
    7181  for(;;)
    7182  {
    7183  // 1. Find next allocation to move.
    7184  // 1.1. Start from last to first m_Blocks - they are sorted from most "destination" to most "source".
    7185  // 1.2. Then start from last to first m_Allocations - they are sorted from largest to smallest.
    7186  while(srcAllocIndex >= m_Blocks[srcBlockIndex]->m_Allocations.size())
    7187  {
    7188  if(m_Blocks[srcBlockIndex]->m_Allocations.empty())
    7189  {
    7190  // Finished: no more allocations to process.
    7191  if(srcBlockIndex == 0)
    7192  {
    7193  return VK_SUCCESS;
    7194  }
    7195  else
    7196  {
    7197  --srcBlockIndex;
    7198  srcAllocIndex = SIZE_MAX;
    7199  }
    7200  }
    7201  else
    7202  {
    7203  srcAllocIndex = m_Blocks[srcBlockIndex]->m_Allocations.size() - 1;
    7204  }
    7205  }
    7206 
    7207  BlockInfo* pSrcBlockInfo = m_Blocks[srcBlockIndex];
    7208  AllocationInfo& allocInfo = pSrcBlockInfo->m_Allocations[srcAllocIndex];
    7209 
    7210  const VkDeviceSize size = allocInfo.m_hAllocation->GetSize();
    7211  const VkDeviceSize srcOffset = allocInfo.m_hAllocation->GetOffset();
    7212  const VkDeviceSize alignment = allocInfo.m_hAllocation->GetAlignment();
    7213  const VmaSuballocationType suballocType = allocInfo.m_hAllocation->GetSuballocationType();
    7214 
    7215  // 2. Try to find new place for this allocation in preceding or current block.
    7216  for(size_t dstBlockIndex = 0; dstBlockIndex <= srcBlockIndex; ++dstBlockIndex)
    7217  {
    7218  BlockInfo* pDstBlockInfo = m_Blocks[dstBlockIndex];
    7219  VmaAllocationRequest dstAllocRequest;
    7220  if(pDstBlockInfo->m_pBlock->m_Metadata.CreateAllocationRequest(
    7221  m_CurrentFrameIndex,
    7222  m_pBlockVector->GetFrameInUseCount(),
    7223  m_pBlockVector->GetBufferImageGranularity(),
    7224  size,
    7225  alignment,
    7226  suballocType,
    7227  false, // canMakeOtherLost
    7228  &dstAllocRequest) &&
    7229  MoveMakesSense(
    7230  dstBlockIndex, dstAllocRequest.offset, srcBlockIndex, srcOffset))
    7231  {
    7232  VMA_ASSERT(dstAllocRequest.itemsToMakeLostCount == 0);
    7233 
    7234  // Reached limit on number of allocations or bytes to move.
    7235  if((m_AllocationsMoved + 1 > maxAllocationsToMove) ||
    7236  (m_BytesMoved + size > maxBytesToMove))
    7237  {
    7238  return VK_INCOMPLETE;
    7239  }
    7240 
    7241  void* pDstMappedData = VMA_NULL;
    7242  VkResult res = pDstBlockInfo->EnsureMapping(m_hAllocator, &pDstMappedData);
    7243  if(res != VK_SUCCESS)
    7244  {
    7245  return res;
    7246  }
    7247 
    7248  void* pSrcMappedData = VMA_NULL;
    7249  res = pSrcBlockInfo->EnsureMapping(m_hAllocator, &pSrcMappedData);
    7250  if(res != VK_SUCCESS)
    7251  {
    7252  return res;
    7253  }
    7254 
    7255  // THE PLACE WHERE ACTUAL DATA COPY HAPPENS.
    7256  memcpy(
    7257  reinterpret_cast<char*>(pDstMappedData) + dstAllocRequest.offset,
    7258  reinterpret_cast<char*>(pSrcMappedData) + srcOffset,
    7259  static_cast<size_t>(size));
    7260 
    7261  pDstBlockInfo->m_pBlock->m_Metadata.Alloc(dstAllocRequest, suballocType, size, allocInfo.m_hAllocation);
    7262  pSrcBlockInfo->m_pBlock->m_Metadata.FreeAtOffset(srcOffset);
    7263 
    7264  allocInfo.m_hAllocation->ChangeBlockAllocation(m_hAllocator, pDstBlockInfo->m_pBlock, dstAllocRequest.offset);
    7265 
    7266  if(allocInfo.m_pChanged != VMA_NULL)
    7267  {
    7268  *allocInfo.m_pChanged = VK_TRUE;
    7269  }
    7270 
    7271  ++m_AllocationsMoved;
    7272  m_BytesMoved += size;
    7273 
    7274  VmaVectorRemove(pSrcBlockInfo->m_Allocations, srcAllocIndex);
    7275 
    7276  break;
    7277  }
    7278  }
    7279 
    7280  // If not processed, this allocInfo remains in pBlockInfo->m_Allocations for next round.
    7281 
    7282  if(srcAllocIndex > 0)
    7283  {
    7284  --srcAllocIndex;
    7285  }
    7286  else
    7287  {
    7288  if(srcBlockIndex > 0)
    7289  {
    7290  --srcBlockIndex;
    7291  srcAllocIndex = SIZE_MAX;
    7292  }
    7293  else
    7294  {
    7295  return VK_SUCCESS;
    7296  }
    7297  }
    7298  }
    7299 }
    7300 
    7301 VkResult VmaDefragmentator::Defragment(
    7302  VkDeviceSize maxBytesToMove,
    7303  uint32_t maxAllocationsToMove)
    7304 {
    7305  if(m_Allocations.empty())
    7306  {
    7307  return VK_SUCCESS;
    7308  }
    7309 
    7310  // Create block info for each block.
    7311  const size_t blockCount = m_pBlockVector->m_Blocks.size();
    7312  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
    7313  {
    7314  BlockInfo* pBlockInfo = vma_new(m_hAllocator, BlockInfo)(m_hAllocator->GetAllocationCallbacks());
    7315  pBlockInfo->m_pBlock = m_pBlockVector->m_Blocks[blockIndex];
    7316  m_Blocks.push_back(pBlockInfo);
    7317  }
    7318 
    7319  // Sort them by m_pBlock pointer value.
    7320  VMA_SORT(m_Blocks.begin(), m_Blocks.end(), BlockPointerLess());
    7321 
    7322  // Move allocation infos from m_Allocations to appropriate m_Blocks[memTypeIndex].m_Allocations.
    7323  for(size_t blockIndex = 0, allocCount = m_Allocations.size(); blockIndex < allocCount; ++blockIndex)
    7324  {
    7325  AllocationInfo& allocInfo = m_Allocations[blockIndex];
    7326  // Now as we are inside VmaBlockVector::m_Mutex, we can make final check if this allocation was not lost.
    7327  if(allocInfo.m_hAllocation->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST)
    7328  {
    7329  VmaDeviceMemoryBlock* pBlock = allocInfo.m_hAllocation->GetBlock();
    7330  BlockInfoVector::iterator it = VmaBinaryFindFirstNotLess(m_Blocks.begin(), m_Blocks.end(), pBlock, BlockPointerLess());
    7331  if(it != m_Blocks.end() && (*it)->m_pBlock == pBlock)
    7332  {
    7333  (*it)->m_Allocations.push_back(allocInfo);
    7334  }
    7335  else
    7336  {
    7337  VMA_ASSERT(0);
    7338  }
    7339  }
    7340  }
    7341  m_Allocations.clear();
    7342 
    7343  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
    7344  {
    7345  BlockInfo* pBlockInfo = m_Blocks[blockIndex];
    7346  pBlockInfo->CalcHasNonMovableAllocations();
    7347  pBlockInfo->SortAllocationsBySizeDescecnding();
    7348  }
    7349 
    7350  // Sort m_Blocks this time by the main criterium, from most "destination" to most "source" blocks.
    7351  VMA_SORT(m_Blocks.begin(), m_Blocks.end(), BlockInfoCompareMoveDestination());
    7352 
    7353  // Execute defragmentation rounds (the main part).
    7354  VkResult result = VK_SUCCESS;
    7355  for(size_t round = 0; (round < 2) && (result == VK_SUCCESS); ++round)
    7356  {
    7357  result = DefragmentRound(maxBytesToMove, maxAllocationsToMove);
    7358  }
    7359 
    7360  // Unmap blocks that were mapped for defragmentation.
    7361  for(size_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
    7362  {
    7363  m_Blocks[blockIndex]->Unmap(m_hAllocator);
    7364  }
    7365 
    7366  return result;
    7367 }
    7368 
    7369 bool VmaDefragmentator::MoveMakesSense(
    7370  size_t dstBlockIndex, VkDeviceSize dstOffset,
    7371  size_t srcBlockIndex, VkDeviceSize srcOffset)
    7372 {
    7373  if(dstBlockIndex < srcBlockIndex)
    7374  {
    7375  return true;
    7376  }
    7377  if(dstBlockIndex > srcBlockIndex)
    7378  {
    7379  return false;
    7380  }
    7381  if(dstOffset < srcOffset)
    7382  {
    7383  return true;
    7384  }
    7385  return false;
    7386 }
    7387 
    7389 // VmaAllocator_T
    7390 
    7391 VmaAllocator_T::VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo) :
    7392  m_UseMutex((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT) == 0),
    7393  m_UseKhrDedicatedAllocation((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT) != 0),
    7394  m_hDevice(pCreateInfo->device),
    7395  m_AllocationCallbacksSpecified(pCreateInfo->pAllocationCallbacks != VMA_NULL),
    7396  m_AllocationCallbacks(pCreateInfo->pAllocationCallbacks ?
    7397  *pCreateInfo->pAllocationCallbacks : VmaEmptyAllocationCallbacks),
    7398  m_PreferredLargeHeapBlockSize(0),
    7399  m_PhysicalDevice(pCreateInfo->physicalDevice),
    7400  m_CurrentFrameIndex(0),
    7401  m_Pools(VmaStlAllocator<VmaPool>(GetAllocationCallbacks()))
    7402 {
    7403  VMA_ASSERT(pCreateInfo->physicalDevice && pCreateInfo->device);
    7404 
    7405 #if !(VMA_DEDICATED_ALLOCATION)
    7406  if((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT) != 0)
    7407  {
    7408  VMA_ASSERT(0 && "VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT set but required extensions are disabled by preprocessor macros.");
    7409  }
    7410 #endif
    7411 
    7412  memset(&m_DeviceMemoryCallbacks, 0 ,sizeof(m_DeviceMemoryCallbacks));
    7413  memset(&m_MemProps, 0, sizeof(m_MemProps));
    7414  memset(&m_PhysicalDeviceProperties, 0, sizeof(m_PhysicalDeviceProperties));
    7415 
    7416  memset(&m_pBlockVectors, 0, sizeof(m_pBlockVectors));
    7417  memset(&m_pDedicatedAllocations, 0, sizeof(m_pDedicatedAllocations));
    7418 
    7419  for(uint32_t i = 0; i < VK_MAX_MEMORY_HEAPS; ++i)
    7420  {
    7421  m_HeapSizeLimit[i] = VK_WHOLE_SIZE;
    7422  }
    7423 
    7424  if(pCreateInfo->pDeviceMemoryCallbacks != VMA_NULL)
    7425  {
    7426  m_DeviceMemoryCallbacks.pfnAllocate = pCreateInfo->pDeviceMemoryCallbacks->pfnAllocate;
    7427  m_DeviceMemoryCallbacks.pfnFree = pCreateInfo->pDeviceMemoryCallbacks->pfnFree;
    7428  }
    7429 
    7430  ImportVulkanFunctions(pCreateInfo->pVulkanFunctions);
    7431 
    7432  (*m_VulkanFunctions.vkGetPhysicalDeviceProperties)(m_PhysicalDevice, &m_PhysicalDeviceProperties);
    7433  (*m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties)(m_PhysicalDevice, &m_MemProps);
    7434 
    7435  m_PreferredLargeHeapBlockSize = (pCreateInfo->preferredLargeHeapBlockSize != 0) ?
    7436  pCreateInfo->preferredLargeHeapBlockSize : static_cast<VkDeviceSize>(VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE);
    7437 
    7438  if(pCreateInfo->pHeapSizeLimit != VMA_NULL)
    7439  {
    7440  for(uint32_t heapIndex = 0; heapIndex < GetMemoryHeapCount(); ++heapIndex)
    7441  {
    7442  const VkDeviceSize limit = pCreateInfo->pHeapSizeLimit[heapIndex];
    7443  if(limit != VK_WHOLE_SIZE)
    7444  {
    7445  m_HeapSizeLimit[heapIndex] = limit;
    7446  if(limit < m_MemProps.memoryHeaps[heapIndex].size)
    7447  {
    7448  m_MemProps.memoryHeaps[heapIndex].size = limit;
    7449  }
    7450  }
    7451  }
    7452  }
    7453 
    7454  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
    7455  {
    7456  const VkDeviceSize preferredBlockSize = CalcPreferredBlockSize(memTypeIndex);
    7457 
    7458  m_pBlockVectors[memTypeIndex] = vma_new(this, VmaBlockVector)(
    7459  this,
    7460  memTypeIndex,
    7461  preferredBlockSize,
    7462  0,
    7463  SIZE_MAX,
    7464  GetBufferImageGranularity(),
    7465  pCreateInfo->frameInUseCount,
    7466  false); // isCustomPool
    7467  // No need to call m_pBlockVectors[memTypeIndex][blockVectorTypeIndex]->CreateMinBlocks here,
    7468  // becase minBlockCount is 0.
    7469  m_pDedicatedAllocations[memTypeIndex] = vma_new(this, AllocationVectorType)(VmaStlAllocator<VmaAllocation>(GetAllocationCallbacks()));
    7470  }
    7471 }
    7472 
    7473 VmaAllocator_T::~VmaAllocator_T()
    7474 {
    7475  VMA_ASSERT(m_Pools.empty());
    7476 
    7477  for(size_t i = GetMemoryTypeCount(); i--; )
    7478  {
    7479  vma_delete(this, m_pDedicatedAllocations[i]);
    7480  vma_delete(this, m_pBlockVectors[i]);
    7481  }
    7482 }
    7483 
    7484 void VmaAllocator_T::ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions)
    7485 {
    7486 #if VMA_STATIC_VULKAN_FUNCTIONS == 1
    7487  m_VulkanFunctions.vkGetPhysicalDeviceProperties = &vkGetPhysicalDeviceProperties;
    7488  m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties = &vkGetPhysicalDeviceMemoryProperties;
    7489  m_VulkanFunctions.vkAllocateMemory = &vkAllocateMemory;
    7490  m_VulkanFunctions.vkFreeMemory = &vkFreeMemory;
    7491  m_VulkanFunctions.vkMapMemory = &vkMapMemory;
    7492  m_VulkanFunctions.vkUnmapMemory = &vkUnmapMemory;
    7493  m_VulkanFunctions.vkBindBufferMemory = &vkBindBufferMemory;
    7494  m_VulkanFunctions.vkBindImageMemory = &vkBindImageMemory;
    7495  m_VulkanFunctions.vkGetBufferMemoryRequirements = &vkGetBufferMemoryRequirements;
    7496  m_VulkanFunctions.vkGetImageMemoryRequirements = &vkGetImageMemoryRequirements;
    7497  m_VulkanFunctions.vkCreateBuffer = &vkCreateBuffer;
    7498  m_VulkanFunctions.vkDestroyBuffer = &vkDestroyBuffer;
    7499  m_VulkanFunctions.vkCreateImage = &vkCreateImage;
    7500  m_VulkanFunctions.vkDestroyImage = &vkDestroyImage;
    7501 #if VMA_DEDICATED_ALLOCATION
    7502  if(m_UseKhrDedicatedAllocation)
    7503  {
    7504  m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR =
    7505  (PFN_vkGetBufferMemoryRequirements2KHR)vkGetDeviceProcAddr(m_hDevice, "vkGetBufferMemoryRequirements2KHR");
    7506  m_VulkanFunctions.vkGetImageMemoryRequirements2KHR =
    7507  (PFN_vkGetImageMemoryRequirements2KHR)vkGetDeviceProcAddr(m_hDevice, "vkGetImageMemoryRequirements2KHR");
    7508  }
    7509 #endif // #if VMA_DEDICATED_ALLOCATION
    7510 #endif // #if VMA_STATIC_VULKAN_FUNCTIONS == 1
    7511 
    7512 #define VMA_COPY_IF_NOT_NULL(funcName) \
    7513  if(pVulkanFunctions->funcName != VMA_NULL) m_VulkanFunctions.funcName = pVulkanFunctions->funcName;
    7514 
    7515  if(pVulkanFunctions != VMA_NULL)
    7516  {
    7517  VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceProperties);
    7518  VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceMemoryProperties);
    7519  VMA_COPY_IF_NOT_NULL(vkAllocateMemory);
    7520  VMA_COPY_IF_NOT_NULL(vkFreeMemory);
    7521  VMA_COPY_IF_NOT_NULL(vkMapMemory);
    7522  VMA_COPY_IF_NOT_NULL(vkUnmapMemory);
    7523  VMA_COPY_IF_NOT_NULL(vkBindBufferMemory);
    7524  VMA_COPY_IF_NOT_NULL(vkBindImageMemory);
    7525  VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements);
    7526  VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements);
    7527  VMA_COPY_IF_NOT_NULL(vkCreateBuffer);
    7528  VMA_COPY_IF_NOT_NULL(vkDestroyBuffer);
    7529  VMA_COPY_IF_NOT_NULL(vkCreateImage);
    7530  VMA_COPY_IF_NOT_NULL(vkDestroyImage);
    7531 #if VMA_DEDICATED_ALLOCATION
    7532  VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements2KHR);
    7533  VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements2KHR);
    7534 #endif
    7535  }
    7536 
    7537 #undef VMA_COPY_IF_NOT_NULL
    7538 
    7539  // If these asserts are hit, you must either #define VMA_STATIC_VULKAN_FUNCTIONS 1
    7540  // or pass valid pointers as VmaAllocatorCreateInfo::pVulkanFunctions.
    7541  VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceProperties != VMA_NULL);
    7542  VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties != VMA_NULL);
    7543  VMA_ASSERT(m_VulkanFunctions.vkAllocateMemory != VMA_NULL);
    7544  VMA_ASSERT(m_VulkanFunctions.vkFreeMemory != VMA_NULL);
    7545  VMA_ASSERT(m_VulkanFunctions.vkMapMemory != VMA_NULL);
    7546  VMA_ASSERT(m_VulkanFunctions.vkUnmapMemory != VMA_NULL);
    7547  VMA_ASSERT(m_VulkanFunctions.vkBindBufferMemory != VMA_NULL);
    7548  VMA_ASSERT(m_VulkanFunctions.vkBindImageMemory != VMA_NULL);
    7549  VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements != VMA_NULL);
    7550  VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements != VMA_NULL);
    7551  VMA_ASSERT(m_VulkanFunctions.vkCreateBuffer != VMA_NULL);
    7552  VMA_ASSERT(m_VulkanFunctions.vkDestroyBuffer != VMA_NULL);
    7553  VMA_ASSERT(m_VulkanFunctions.vkCreateImage != VMA_NULL);
    7554  VMA_ASSERT(m_VulkanFunctions.vkDestroyImage != VMA_NULL);
    7555 #if VMA_DEDICATED_ALLOCATION
    7556  if(m_UseKhrDedicatedAllocation)
    7557  {
    7558  VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR != VMA_NULL);
    7559  VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements2KHR != VMA_NULL);
    7560  }
    7561 #endif
    7562 }
    7563 
    7564 VkDeviceSize VmaAllocator_T::CalcPreferredBlockSize(uint32_t memTypeIndex)
    7565 {
    7566  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
    7567  const VkDeviceSize heapSize = m_MemProps.memoryHeaps[heapIndex].size;
    7568  const bool isSmallHeap = heapSize <= VMA_SMALL_HEAP_MAX_SIZE;
    7569  return isSmallHeap ? (heapSize / 8) : m_PreferredLargeHeapBlockSize;
    7570 }
    7571 
    7572 VkResult VmaAllocator_T::AllocateMemoryOfType(
    7573  const VkMemoryRequirements& vkMemReq,
    7574  bool dedicatedAllocation,
    7575  VkBuffer dedicatedBuffer,
    7576  VkImage dedicatedImage,
    7577  const VmaAllocationCreateInfo& createInfo,
    7578  uint32_t memTypeIndex,
    7579  VmaSuballocationType suballocType,
    7580  VmaAllocation* pAllocation)
    7581 {
    7582  VMA_ASSERT(pAllocation != VMA_NULL);
    7583  VMA_DEBUG_LOG(" AllocateMemory: MemoryTypeIndex=%u, Size=%llu", memTypeIndex, vkMemReq.size);
    7584 
    7585  VmaAllocationCreateInfo finalCreateInfo = createInfo;
    7586 
    7587  // If memory type is not HOST_VISIBLE, disable MAPPED.
    7588  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0 &&
    7589  (m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) == 0)
    7590  {
    7591  finalCreateInfo.flags &= ~VMA_ALLOCATION_CREATE_MAPPED_BIT;
    7592  }
    7593 
    7594  VmaBlockVector* const blockVector = m_pBlockVectors[memTypeIndex];
    7595  VMA_ASSERT(blockVector);
    7596 
    7597  const VkDeviceSize preferredBlockSize = blockVector->GetPreferredBlockSize();
    7598  bool preferDedicatedMemory =
    7599  VMA_DEBUG_ALWAYS_DEDICATED_MEMORY ||
    7600  dedicatedAllocation ||
    7601  // Heuristics: Allocate dedicated memory if requested size if greater than half of preferred block size.
    7602  vkMemReq.size > preferredBlockSize / 2;
    7603 
    7604  if(preferDedicatedMemory &&
    7605  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == 0 &&
    7606  finalCreateInfo.pool == VK_NULL_HANDLE)
    7607  {
    7608  finalCreateInfo.flags |= VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
    7609  }
    7610 
    7611  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0)
    7612  {
    7613  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
    7614  {
    7615  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7616  }
    7617  else
    7618  {
    7619  return AllocateDedicatedMemory(
    7620  vkMemReq.size,
    7621  suballocType,
    7622  memTypeIndex,
    7623  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0,
    7624  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0,
    7625  finalCreateInfo.pUserData,
    7626  dedicatedBuffer,
    7627  dedicatedImage,
    7628  pAllocation);
    7629  }
    7630  }
    7631  else
    7632  {
    7633  VkResult res = blockVector->Allocate(
    7634  VK_NULL_HANDLE, // hCurrentPool
    7635  m_CurrentFrameIndex.load(),
    7636  vkMemReq,
    7637  finalCreateInfo,
    7638  suballocType,
    7639  pAllocation);
    7640  if(res == VK_SUCCESS)
    7641  {
    7642  return res;
    7643  }
    7644 
    7645  // 5. Try dedicated memory.
    7646  if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
    7647  {
    7648  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7649  }
    7650  else
    7651  {
    7652  res = AllocateDedicatedMemory(
    7653  vkMemReq.size,
    7654  suballocType,
    7655  memTypeIndex,
    7656  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0,
    7657  (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0,
    7658  finalCreateInfo.pUserData,
    7659  dedicatedBuffer,
    7660  dedicatedImage,
    7661  pAllocation);
    7662  if(res == VK_SUCCESS)
    7663  {
    7664  // Succeeded: AllocateDedicatedMemory function already filld pMemory, nothing more to do here.
    7665  VMA_DEBUG_LOG(" Allocated as DedicatedMemory");
    7666  return VK_SUCCESS;
    7667  }
    7668  else
    7669  {
    7670  // Everything failed: Return error code.
    7671  VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
    7672  return res;
    7673  }
    7674  }
    7675  }
    7676 }
    7677 
    7678 VkResult VmaAllocator_T::AllocateDedicatedMemory(
    7679  VkDeviceSize size,
    7680  VmaSuballocationType suballocType,
    7681  uint32_t memTypeIndex,
    7682  bool map,
    7683  bool isUserDataString,
    7684  void* pUserData,
    7685  VkBuffer dedicatedBuffer,
    7686  VkImage dedicatedImage,
    7687  VmaAllocation* pAllocation)
    7688 {
    7689  VMA_ASSERT(pAllocation);
    7690 
    7691  VkMemoryAllocateInfo allocInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
    7692  allocInfo.memoryTypeIndex = memTypeIndex;
    7693  allocInfo.allocationSize = size;
    7694 
    7695 #if VMA_DEDICATED_ALLOCATION
    7696  VkMemoryDedicatedAllocateInfoKHR dedicatedAllocInfo = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO_KHR };
    7697  if(m_UseKhrDedicatedAllocation)
    7698  {
    7699  if(dedicatedBuffer != VK_NULL_HANDLE)
    7700  {
    7701  VMA_ASSERT(dedicatedImage == VK_NULL_HANDLE);
    7702  dedicatedAllocInfo.buffer = dedicatedBuffer;
    7703  allocInfo.pNext = &dedicatedAllocInfo;
    7704  }
    7705  else if(dedicatedImage != VK_NULL_HANDLE)
    7706  {
    7707  dedicatedAllocInfo.image = dedicatedImage;
    7708  allocInfo.pNext = &dedicatedAllocInfo;
    7709  }
    7710  }
    7711 #endif // #if VMA_DEDICATED_ALLOCATION
    7712 
    7713  // Allocate VkDeviceMemory.
    7714  VkDeviceMemory hMemory = VK_NULL_HANDLE;
    7715  VkResult res = AllocateVulkanMemory(&allocInfo, &hMemory);
    7716  if(res < 0)
    7717  {
    7718  VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
    7719  return res;
    7720  }
    7721 
    7722  void* pMappedData = VMA_NULL;
    7723  if(map)
    7724  {
    7725  res = (*m_VulkanFunctions.vkMapMemory)(
    7726  m_hDevice,
    7727  hMemory,
    7728  0,
    7729  VK_WHOLE_SIZE,
    7730  0,
    7731  &pMappedData);
    7732  if(res < 0)
    7733  {
    7734  VMA_DEBUG_LOG(" vkMapMemory FAILED");
    7735  FreeVulkanMemory(memTypeIndex, size, hMemory);
    7736  return res;
    7737  }
    7738  }
    7739 
    7740  *pAllocation = vma_new(this, VmaAllocation_T)(m_CurrentFrameIndex.load(), isUserDataString);
    7741  (*pAllocation)->InitDedicatedAllocation(memTypeIndex, hMemory, suballocType, pMappedData, size);
    7742  (*pAllocation)->SetUserData(this, pUserData);
    7743 
    7744  // Register it in m_pDedicatedAllocations.
    7745  {
    7746  VmaMutexLock lock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
    7747  AllocationVectorType* pDedicatedAllocations = m_pDedicatedAllocations[memTypeIndex];
    7748  VMA_ASSERT(pDedicatedAllocations);
    7749  VmaVectorInsertSorted<VmaPointerLess>(*pDedicatedAllocations, *pAllocation);
    7750  }
    7751 
    7752  VMA_DEBUG_LOG(" Allocated DedicatedMemory MemoryTypeIndex=#%u", memTypeIndex);
    7753 
    7754  return VK_SUCCESS;
    7755 }
    7756 
    7757 void VmaAllocator_T::GetBufferMemoryRequirements(
    7758  VkBuffer hBuffer,
    7759  VkMemoryRequirements& memReq,
    7760  bool& requiresDedicatedAllocation,
    7761  bool& prefersDedicatedAllocation) const
    7762 {
    7763 #if VMA_DEDICATED_ALLOCATION
    7764  if(m_UseKhrDedicatedAllocation)
    7765  {
    7766  VkBufferMemoryRequirementsInfo2KHR memReqInfo = { VK_STRUCTURE_TYPE_BUFFER_MEMORY_REQUIREMENTS_INFO_2_KHR };
    7767  memReqInfo.buffer = hBuffer;
    7768 
    7769  VkMemoryDedicatedRequirementsKHR memDedicatedReq = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
    7770 
    7771  VkMemoryRequirements2KHR memReq2 = { VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
    7772  memReq2.pNext = &memDedicatedReq;
    7773 
    7774  (*m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
    7775 
    7776  memReq = memReq2.memoryRequirements;
    7777  requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
    7778  prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
    7779  }
    7780  else
    7781 #endif // #if VMA_DEDICATED_ALLOCATION
    7782  {
    7783  (*m_VulkanFunctions.vkGetBufferMemoryRequirements)(m_hDevice, hBuffer, &memReq);
    7784  requiresDedicatedAllocation = false;
    7785  prefersDedicatedAllocation = false;
    7786  }
    7787 }
    7788 
    7789 void VmaAllocator_T::GetImageMemoryRequirements(
    7790  VkImage hImage,
    7791  VkMemoryRequirements& memReq,
    7792  bool& requiresDedicatedAllocation,
    7793  bool& prefersDedicatedAllocation) const
    7794 {
    7795 #if VMA_DEDICATED_ALLOCATION
    7796  if(m_UseKhrDedicatedAllocation)
    7797  {
    7798  VkImageMemoryRequirementsInfo2KHR memReqInfo = { VK_STRUCTURE_TYPE_IMAGE_MEMORY_REQUIREMENTS_INFO_2_KHR };
    7799  memReqInfo.image = hImage;
    7800 
    7801  VkMemoryDedicatedRequirementsKHR memDedicatedReq = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
    7802 
    7803  VkMemoryRequirements2KHR memReq2 = { VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
    7804  memReq2.pNext = &memDedicatedReq;
    7805 
    7806  (*m_VulkanFunctions.vkGetImageMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
    7807 
    7808  memReq = memReq2.memoryRequirements;
    7809  requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
    7810  prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
    7811  }
    7812  else
    7813 #endif // #if VMA_DEDICATED_ALLOCATION
    7814  {
    7815  (*m_VulkanFunctions.vkGetImageMemoryRequirements)(m_hDevice, hImage, &memReq);
    7816  requiresDedicatedAllocation = false;
    7817  prefersDedicatedAllocation = false;
    7818  }
    7819 }
    7820 
    7821 VkResult VmaAllocator_T::AllocateMemory(
    7822  const VkMemoryRequirements& vkMemReq,
    7823  bool requiresDedicatedAllocation,
    7824  bool prefersDedicatedAllocation,
    7825  VkBuffer dedicatedBuffer,
    7826  VkImage dedicatedImage,
    7827  const VmaAllocationCreateInfo& createInfo,
    7828  VmaSuballocationType suballocType,
    7829  VmaAllocation* pAllocation)
    7830 {
    7831  if((createInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0 &&
    7832  (createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
    7833  {
    7834  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT together with VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT makes no sense.");
    7835  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7836  }
    7837  if((createInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0 &&
    7838  (createInfo.flags & VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT) != 0)
    7839  {
    7840  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_MAPPED_BIT together with VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT is invalid.");
    7841  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7842  }
    7843  if(requiresDedicatedAllocation)
    7844  {
    7845  if((createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
    7846  {
    7847  VMA_ASSERT(0 && "VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT specified while dedicated allocation is required.");
    7848  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7849  }
    7850  if(createInfo.pool != VK_NULL_HANDLE)
    7851  {
    7852  VMA_ASSERT(0 && "Pool specified while dedicated allocation is required.");
    7853  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7854  }
    7855  }
    7856  if((createInfo.pool != VK_NULL_HANDLE) &&
    7857  ((createInfo.flags & (VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT)) != 0))
    7858  {
    7859  VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT when pool != null is invalid.");
    7860  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7861  }
    7862 
    7863  if(createInfo.pool != VK_NULL_HANDLE)
    7864  {
    7865  return createInfo.pool->m_BlockVector.Allocate(
    7866  createInfo.pool,
    7867  m_CurrentFrameIndex.load(),
    7868  vkMemReq,
    7869  createInfo,
    7870  suballocType,
    7871  pAllocation);
    7872  }
    7873  else
    7874  {
    7875  // Bit mask of memory Vulkan types acceptable for this allocation.
    7876  uint32_t memoryTypeBits = vkMemReq.memoryTypeBits;
    7877  uint32_t memTypeIndex = UINT32_MAX;
    7878  VkResult res = vmaFindMemoryTypeIndex(this, memoryTypeBits, &createInfo, &memTypeIndex);
    7879  if(res == VK_SUCCESS)
    7880  {
    7881  res = AllocateMemoryOfType(
    7882  vkMemReq,
    7883  requiresDedicatedAllocation || prefersDedicatedAllocation,
    7884  dedicatedBuffer,
    7885  dedicatedImage,
    7886  createInfo,
    7887  memTypeIndex,
    7888  suballocType,
    7889  pAllocation);
    7890  // Succeeded on first try.
    7891  if(res == VK_SUCCESS)
    7892  {
    7893  return res;
    7894  }
    7895  // Allocation from this memory type failed. Try other compatible memory types.
    7896  else
    7897  {
    7898  for(;;)
    7899  {
    7900  // Remove old memTypeIndex from list of possibilities.
    7901  memoryTypeBits &= ~(1u << memTypeIndex);
    7902  // Find alternative memTypeIndex.
    7903  res = vmaFindMemoryTypeIndex(this, memoryTypeBits, &createInfo, &memTypeIndex);
    7904  if(res == VK_SUCCESS)
    7905  {
    7906  res = AllocateMemoryOfType(
    7907  vkMemReq,
    7908  requiresDedicatedAllocation || prefersDedicatedAllocation,
    7909  dedicatedBuffer,
    7910  dedicatedImage,
    7911  createInfo,
    7912  memTypeIndex,
    7913  suballocType,
    7914  pAllocation);
    7915  // Allocation from this alternative memory type succeeded.
    7916  if(res == VK_SUCCESS)
    7917  {
    7918  return res;
    7919  }
    7920  // else: Allocation from this memory type failed. Try next one - next loop iteration.
    7921  }
    7922  // No other matching memory type index could be found.
    7923  else
    7924  {
    7925  // Not returning res, which is VK_ERROR_FEATURE_NOT_PRESENT, because we already failed to allocate once.
    7926  return VK_ERROR_OUT_OF_DEVICE_MEMORY;
    7927  }
    7928  }
    7929  }
    7930  }
    7931  // Can't find any single memory type maching requirements. res is VK_ERROR_FEATURE_NOT_PRESENT.
    7932  else
    7933  return res;
    7934  }
    7935 }
    7936 
    7937 void VmaAllocator_T::FreeMemory(const VmaAllocation allocation)
    7938 {
    7939  VMA_ASSERT(allocation);
    7940 
    7941  if(allocation->CanBecomeLost() == false ||
    7942  allocation->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST)
    7943  {
    7944  switch(allocation->GetType())
    7945  {
    7946  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
    7947  {
    7948  VmaBlockVector* pBlockVector = VMA_NULL;
    7949  VmaPool hPool = allocation->GetPool();
    7950  if(hPool != VK_NULL_HANDLE)
    7951  {
    7952  pBlockVector = &hPool->m_BlockVector;
    7953  }
    7954  else
    7955  {
    7956  const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
    7957  pBlockVector = m_pBlockVectors[memTypeIndex];
    7958  }
    7959  pBlockVector->Free(allocation);
    7960  }
    7961  break;
    7962  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
    7963  FreeDedicatedMemory(allocation);
    7964  break;
    7965  default:
    7966  VMA_ASSERT(0);
    7967  }
    7968  }
    7969 
    7970  allocation->SetUserData(this, VMA_NULL);
    7971  vma_delete(this, allocation);
    7972 }
    7973 
    7974 void VmaAllocator_T::CalculateStats(VmaStats* pStats)
    7975 {
    7976  // Initialize.
    7977  InitStatInfo(pStats->total);
    7978  for(size_t i = 0; i < VK_MAX_MEMORY_TYPES; ++i)
    7979  InitStatInfo(pStats->memoryType[i]);
    7980  for(size_t i = 0; i < VK_MAX_MEMORY_HEAPS; ++i)
    7981  InitStatInfo(pStats->memoryHeap[i]);
    7982 
    7983  // Process default pools.
    7984  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
    7985  {
    7986  VmaBlockVector* const pBlockVector = m_pBlockVectors[memTypeIndex];
    7987  VMA_ASSERT(pBlockVector);
    7988  pBlockVector->AddStats(pStats);
    7989  }
    7990 
    7991  // Process custom pools.
    7992  {
    7993  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
    7994  for(size_t poolIndex = 0, poolCount = m_Pools.size(); poolIndex < poolCount; ++poolIndex)
    7995  {
    7996  m_Pools[poolIndex]->GetBlockVector().AddStats(pStats);
    7997  }
    7998  }
    7999 
    8000  // Process dedicated allocations.
    8001  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
    8002  {
    8003  const uint32_t memHeapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
    8004  VmaMutexLock dedicatedAllocationsLock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
    8005  AllocationVectorType* const pDedicatedAllocVector = m_pDedicatedAllocations[memTypeIndex];
    8006  VMA_ASSERT(pDedicatedAllocVector);
    8007  for(size_t allocIndex = 0, allocCount = pDedicatedAllocVector->size(); allocIndex < allocCount; ++allocIndex)
    8008  {
    8009  VmaStatInfo allocationStatInfo;
    8010  (*pDedicatedAllocVector)[allocIndex]->DedicatedAllocCalcStatsInfo(allocationStatInfo);
    8011  VmaAddStatInfo(pStats->total, allocationStatInfo);
    8012  VmaAddStatInfo(pStats->memoryType[memTypeIndex], allocationStatInfo);
    8013  VmaAddStatInfo(pStats->memoryHeap[memHeapIndex], allocationStatInfo);
    8014  }
    8015  }
    8016 
    8017  // Postprocess.
    8018  VmaPostprocessCalcStatInfo(pStats->total);
    8019  for(size_t i = 0; i < GetMemoryTypeCount(); ++i)
    8020  VmaPostprocessCalcStatInfo(pStats->memoryType[i]);
    8021  for(size_t i = 0; i < GetMemoryHeapCount(); ++i)
    8022  VmaPostprocessCalcStatInfo(pStats->memoryHeap[i]);
    8023 }
    8024 
    8025 static const uint32_t VMA_VENDOR_ID_AMD = 4098;
    8026 
    8027 VkResult VmaAllocator_T::Defragment(
    8028  VmaAllocation* pAllocations,
    8029  size_t allocationCount,
    8030  VkBool32* pAllocationsChanged,
    8031  const VmaDefragmentationInfo* pDefragmentationInfo,
    8032  VmaDefragmentationStats* pDefragmentationStats)
    8033 {
    8034  if(pAllocationsChanged != VMA_NULL)
    8035  {
    8036  memset(pAllocationsChanged, 0, sizeof(*pAllocationsChanged));
    8037  }
    8038  if(pDefragmentationStats != VMA_NULL)
    8039  {
    8040  memset(pDefragmentationStats, 0, sizeof(*pDefragmentationStats));
    8041  }
    8042 
    8043  const uint32_t currentFrameIndex = m_CurrentFrameIndex.load();
    8044 
    8045  VmaMutexLock poolsLock(m_PoolsMutex, m_UseMutex);
    8046 
    8047  const size_t poolCount = m_Pools.size();
    8048 
    8049  // Dispatch pAllocations among defragmentators. Create them in BlockVectors when necessary.
    8050  for(size_t allocIndex = 0; allocIndex < allocationCount; ++allocIndex)
    8051  {
    8052  VmaAllocation hAlloc = pAllocations[allocIndex];
    8053  VMA_ASSERT(hAlloc);
    8054  const uint32_t memTypeIndex = hAlloc->GetMemoryTypeIndex();
    8055  // DedicatedAlloc cannot be defragmented.
    8056  if((hAlloc->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK) &&
    8057  // Only HOST_VISIBLE memory types can be defragmented.
    8058  ((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0) &&
    8059  // Lost allocation cannot be defragmented.
    8060  (hAlloc->GetLastUseFrameIndex() != VMA_FRAME_INDEX_LOST))
    8061  {
    8062  VmaBlockVector* pAllocBlockVector = VMA_NULL;
    8063 
    8064  const VmaPool hAllocPool = hAlloc->GetPool();
    8065  // This allocation belongs to custom pool.
    8066  if(hAllocPool != VK_NULL_HANDLE)
    8067  {
    8068  pAllocBlockVector = &hAllocPool->GetBlockVector();
    8069  }
    8070  // This allocation belongs to general pool.
    8071  else
    8072  {
    8073  pAllocBlockVector = m_pBlockVectors[memTypeIndex];
    8074  }
    8075 
    8076  VmaDefragmentator* const pDefragmentator = pAllocBlockVector->EnsureDefragmentator(this, currentFrameIndex);
    8077 
    8078  VkBool32* const pChanged = (pAllocationsChanged != VMA_NULL) ?
    8079  &pAllocationsChanged[allocIndex] : VMA_NULL;
    8080  pDefragmentator->AddAllocation(hAlloc, pChanged);
    8081  }
    8082  }
    8083 
    8084  VkResult result = VK_SUCCESS;
    8085 
    8086  // ======== Main processing.
    8087 
    8088  VkDeviceSize maxBytesToMove = SIZE_MAX;
    8089  uint32_t maxAllocationsToMove = UINT32_MAX;
    8090  if(pDefragmentationInfo != VMA_NULL)
    8091  {
    8092  maxBytesToMove = pDefragmentationInfo->maxBytesToMove;
    8093  maxAllocationsToMove = pDefragmentationInfo->maxAllocationsToMove;
    8094  }
    8095 
    8096  // Process standard memory.
    8097  for(uint32_t memTypeIndex = 0;
    8098  (memTypeIndex < GetMemoryTypeCount()) && (result == VK_SUCCESS);
    8099  ++memTypeIndex)
    8100  {
    8101  // Only HOST_VISIBLE memory types can be defragmented.
    8102  if((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
    8103  {
    8104  result = m_pBlockVectors[memTypeIndex]->Defragment(
    8105  pDefragmentationStats,
    8106  maxBytesToMove,
    8107  maxAllocationsToMove);
    8108  }
    8109  }
    8110 
    8111  // Process custom pools.
    8112  for(size_t poolIndex = 0; (poolIndex < poolCount) && (result == VK_SUCCESS); ++poolIndex)
    8113  {
    8114  result = m_Pools[poolIndex]->GetBlockVector().Defragment(
    8115  pDefragmentationStats,
    8116  maxBytesToMove,
    8117  maxAllocationsToMove);
    8118  }
    8119 
    8120  // ======== Destroy defragmentators.
    8121 
    8122  // Process custom pools.
    8123  for(size_t poolIndex = poolCount; poolIndex--; )
    8124  {
    8125  m_Pools[poolIndex]->GetBlockVector().DestroyDefragmentator();
    8126  }
    8127 
    8128  // Process standard memory.
    8129  for(uint32_t memTypeIndex = GetMemoryTypeCount(); memTypeIndex--; )
    8130  {
    8131  if((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
    8132  {
    8133  m_pBlockVectors[memTypeIndex]->DestroyDefragmentator();
    8134  }
    8135  }
    8136 
    8137  return result;
    8138 }
    8139 
    8140 void VmaAllocator_T::GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo)
    8141 {
    8142  if(hAllocation->CanBecomeLost())
    8143  {
    8144  /*
    8145  Warning: This is a carefully designed algorithm.
    8146  Do not modify unless you really know what you're doing :)
    8147  */
    8148  uint32_t localCurrFrameIndex = m_CurrentFrameIndex.load();
    8149  uint32_t localLastUseFrameIndex = hAllocation->GetLastUseFrameIndex();
    8150  for(;;)
    8151  {
    8152  if(localLastUseFrameIndex == VMA_FRAME_INDEX_LOST)
    8153  {
    8154  pAllocationInfo->memoryType = UINT32_MAX;
    8155  pAllocationInfo->deviceMemory = VK_NULL_HANDLE;
    8156  pAllocationInfo->offset = 0;
    8157  pAllocationInfo->size = hAllocation->GetSize();
    8158  pAllocationInfo->pMappedData = VMA_NULL;
    8159  pAllocationInfo->pUserData = hAllocation->GetUserData();
    8160  return;
    8161  }
    8162  else if(localLastUseFrameIndex == localCurrFrameIndex)
    8163  {
    8164  pAllocationInfo->memoryType = hAllocation->GetMemoryTypeIndex();
    8165  pAllocationInfo->deviceMemory = hAllocation->GetMemory();
    8166  pAllocationInfo->offset = hAllocation->GetOffset();
    8167  pAllocationInfo->size = hAllocation->GetSize();
    8168  pAllocationInfo->pMappedData = VMA_NULL;
    8169  pAllocationInfo->pUserData = hAllocation->GetUserData();
    8170  return;
    8171  }
    8172  else // Last use time earlier than current time.
    8173  {
    8174  if(hAllocation->CompareExchangeLastUseFrameIndex(localLastUseFrameIndex, localCurrFrameIndex))
    8175  {
    8176  localLastUseFrameIndex = localCurrFrameIndex;
    8177  }
    8178  }
    8179  }
    8180  }
    8181  else
    8182  {
    8183  pAllocationInfo->memoryType = hAllocation->GetMemoryTypeIndex();
    8184  pAllocationInfo->deviceMemory = hAllocation->GetMemory();
    8185  pAllocationInfo->offset = hAllocation->GetOffset();
    8186  pAllocationInfo->size = hAllocation->GetSize();
    8187  pAllocationInfo->pMappedData = hAllocation->GetMappedData();
    8188  pAllocationInfo->pUserData = hAllocation->GetUserData();
    8189  }
    8190 }
    8191 
    8192 bool VmaAllocator_T::TouchAllocation(VmaAllocation hAllocation)
    8193 {
    8194  // This is a stripped-down version of VmaAllocator_T::GetAllocationInfo.
    8195  if(hAllocation->CanBecomeLost())
    8196  {
    8197  uint32_t localCurrFrameIndex = m_CurrentFrameIndex.load();
    8198  uint32_t localLastUseFrameIndex = hAllocation->GetLastUseFrameIndex();
    8199  for(;;)
    8200  {
    8201  if(localLastUseFrameIndex == VMA_FRAME_INDEX_LOST)
    8202  {
    8203  return false;
    8204  }
    8205  else if(localLastUseFrameIndex == localCurrFrameIndex)
    8206  {
    8207  return true;
    8208  }
    8209  else // Last use time earlier than current time.
    8210  {
    8211  if(hAllocation->CompareExchangeLastUseFrameIndex(localLastUseFrameIndex, localCurrFrameIndex))
    8212  {
    8213  localLastUseFrameIndex = localCurrFrameIndex;
    8214  }
    8215  }
    8216  }
    8217  }
    8218  else
    8219  {
    8220  return true;
    8221  }
    8222 }
    8223 
    8224 VkResult VmaAllocator_T::CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool)
    8225 {
    8226  VMA_DEBUG_LOG(" CreatePool: MemoryTypeIndex=%u", pCreateInfo->memoryTypeIndex);
    8227 
    8228  VmaPoolCreateInfo newCreateInfo = *pCreateInfo;
    8229 
    8230  if(newCreateInfo.maxBlockCount == 0)
    8231  {
    8232  newCreateInfo.maxBlockCount = SIZE_MAX;
    8233  }
    8234  if(newCreateInfo.blockSize == 0)
    8235  {
    8236  newCreateInfo.blockSize = CalcPreferredBlockSize(newCreateInfo.memoryTypeIndex);
    8237  }
    8238 
    8239  *pPool = vma_new(this, VmaPool_T)(this, newCreateInfo);
    8240 
    8241  VkResult res = (*pPool)->m_BlockVector.CreateMinBlocks();
    8242  if(res != VK_SUCCESS)
    8243  {
    8244  vma_delete(this, *pPool);
    8245  *pPool = VMA_NULL;
    8246  return res;
    8247  }
    8248 
    8249  // Add to m_Pools.
    8250  {
    8251  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
    8252  VmaVectorInsertSorted<VmaPointerLess>(m_Pools, *pPool);
    8253  }
    8254 
    8255  return VK_SUCCESS;
    8256 }
    8257 
    8258 void VmaAllocator_T::DestroyPool(VmaPool pool)
    8259 {
    8260  // Remove from m_Pools.
    8261  {
    8262  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
    8263  bool success = VmaVectorRemoveSorted<VmaPointerLess>(m_Pools, pool);
    8264  VMA_ASSERT(success && "Pool not found in Allocator.");
    8265  }
    8266 
    8267  vma_delete(this, pool);
    8268 }
    8269 
    8270 void VmaAllocator_T::GetPoolStats(VmaPool pool, VmaPoolStats* pPoolStats)
    8271 {
    8272  pool->m_BlockVector.GetPoolStats(pPoolStats);
    8273 }
    8274 
    8275 void VmaAllocator_T::SetCurrentFrameIndex(uint32_t frameIndex)
    8276 {
    8277  m_CurrentFrameIndex.store(frameIndex);
    8278 }
    8279 
    8280 void VmaAllocator_T::MakePoolAllocationsLost(
    8281  VmaPool hPool,
    8282  size_t* pLostAllocationCount)
    8283 {
    8284  hPool->m_BlockVector.MakePoolAllocationsLost(
    8285  m_CurrentFrameIndex.load(),
    8286  pLostAllocationCount);
    8287 }
    8288 
    8289 void VmaAllocator_T::CreateLostAllocation(VmaAllocation* pAllocation)
    8290 {
    8291  *pAllocation = vma_new(this, VmaAllocation_T)(VMA_FRAME_INDEX_LOST, false);
    8292  (*pAllocation)->InitLost();
    8293 }
    8294 
    8295 VkResult VmaAllocator_T::AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory)
    8296 {
    8297  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(pAllocateInfo->memoryTypeIndex);
    8298 
    8299  VkResult res;
    8300  if(m_HeapSizeLimit[heapIndex] != VK_WHOLE_SIZE)
    8301  {
    8302  VmaMutexLock lock(m_HeapSizeLimitMutex, m_UseMutex);
    8303  if(m_HeapSizeLimit[heapIndex] >= pAllocateInfo->allocationSize)
    8304  {
    8305  res = (*m_VulkanFunctions.vkAllocateMemory)(m_hDevice, pAllocateInfo, GetAllocationCallbacks(), pMemory);
    8306  if(res == VK_SUCCESS)
    8307  {
    8308  m_HeapSizeLimit[heapIndex] -= pAllocateInfo->allocationSize;
    8309  }
    8310  }
    8311  else
    8312  {
    8313  res = VK_ERROR_OUT_OF_DEVICE_MEMORY;
    8314  }
    8315  }
    8316  else
    8317  {
    8318  res = (*m_VulkanFunctions.vkAllocateMemory)(m_hDevice, pAllocateInfo, GetAllocationCallbacks(), pMemory);
    8319  }
    8320 
    8321  if(res == VK_SUCCESS && m_DeviceMemoryCallbacks.pfnAllocate != VMA_NULL)
    8322  {
    8323  (*m_DeviceMemoryCallbacks.pfnAllocate)(this, pAllocateInfo->memoryTypeIndex, *pMemory, pAllocateInfo->allocationSize);
    8324  }
    8325 
    8326  return res;
    8327 }
    8328 
    8329 void VmaAllocator_T::FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory)
    8330 {
    8331  if(m_DeviceMemoryCallbacks.pfnFree != VMA_NULL)
    8332  {
    8333  (*m_DeviceMemoryCallbacks.pfnFree)(this, memoryType, hMemory, size);
    8334  }
    8335 
    8336  (*m_VulkanFunctions.vkFreeMemory)(m_hDevice, hMemory, GetAllocationCallbacks());
    8337 
    8338  const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memoryType);
    8339  if(m_HeapSizeLimit[heapIndex] != VK_WHOLE_SIZE)
    8340  {
    8341  VmaMutexLock lock(m_HeapSizeLimitMutex, m_UseMutex);
    8342  m_HeapSizeLimit[heapIndex] += size;
    8343  }
    8344 }
    8345 
    8346 VkResult VmaAllocator_T::Map(VmaAllocation hAllocation, void** ppData)
    8347 {
    8348  if(hAllocation->CanBecomeLost())
    8349  {
    8350  return VK_ERROR_MEMORY_MAP_FAILED;
    8351  }
    8352 
    8353  switch(hAllocation->GetType())
    8354  {
    8355  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
    8356  {
    8357  VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
    8358  char *pBytes = VMA_NULL;
    8359  VkResult res = pBlock->Map(this, 1, (void**)&pBytes);
    8360  if(res == VK_SUCCESS)
    8361  {
    8362  *ppData = pBytes + (ptrdiff_t)hAllocation->GetOffset();
    8363  hAllocation->BlockAllocMap();
    8364  }
    8365  return res;
    8366  }
    8367  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
    8368  return hAllocation->DedicatedAllocMap(this, ppData);
    8369  default:
    8370  VMA_ASSERT(0);
    8371  return VK_ERROR_MEMORY_MAP_FAILED;
    8372  }
    8373 }
    8374 
    8375 void VmaAllocator_T::Unmap(VmaAllocation hAllocation)
    8376 {
    8377  switch(hAllocation->GetType())
    8378  {
    8379  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
    8380  {
    8381  VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
    8382  hAllocation->BlockAllocUnmap();
    8383  pBlock->Unmap(this, 1);
    8384  }
    8385  break;
    8386  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
    8387  hAllocation->DedicatedAllocUnmap(this);
    8388  break;
    8389  default:
    8390  VMA_ASSERT(0);
    8391  }
    8392 }
    8393 
    8394 VkResult VmaAllocator_T::BindBufferMemory(VmaAllocation hAllocation, VkBuffer hBuffer)
    8395 {
    8396  VkResult res = VK_SUCCESS;
    8397  switch(hAllocation->GetType())
    8398  {
    8399  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
    8400  res = GetVulkanFunctions().vkBindBufferMemory(
    8401  m_hDevice,
    8402  hBuffer,
    8403  hAllocation->GetMemory(),
    8404  0); //memoryOffset
    8405  break;
    8406  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
    8407  {
    8408  VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
    8409  VMA_ASSERT(pBlock && "Binding buffer to allocation that doesn't belong to any block. Is the allocation lost?");
    8410  res = pBlock->BindBufferMemory(this, hAllocation, hBuffer);
    8411  break;
    8412  }
    8413  default:
    8414  VMA_ASSERT(0);
    8415  }
    8416  return res;
    8417 }
    8418 
    8419 VkResult VmaAllocator_T::BindImageMemory(VmaAllocation hAllocation, VkImage hImage)
    8420 {
    8421  VkResult res = VK_SUCCESS;
    8422  switch(hAllocation->GetType())
    8423  {
    8424  case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
    8425  res = GetVulkanFunctions().vkBindImageMemory(
    8426  m_hDevice,
    8427  hImage,
    8428  hAllocation->GetMemory(),
    8429  0); //memoryOffset
    8430  break;
    8431  case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
    8432  {
    8433  VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
    8434  VMA_ASSERT(pBlock && "Binding image to allocation that doesn't belong to any block. Is the allocation lost?");
    8435  res = pBlock->BindImageMemory(this, hAllocation, hImage);
    8436  break;
    8437  }
    8438  default:
    8439  VMA_ASSERT(0);
    8440  }
    8441  return res;
    8442 }
    8443 
    8444 void VmaAllocator_T::FreeDedicatedMemory(VmaAllocation allocation)
    8445 {
    8446  VMA_ASSERT(allocation && allocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
    8447 
    8448  const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
    8449  {
    8450  VmaMutexLock lock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
    8451  AllocationVectorType* const pDedicatedAllocations = m_pDedicatedAllocations[memTypeIndex];
    8452  VMA_ASSERT(pDedicatedAllocations);
    8453  bool success = VmaVectorRemoveSorted<VmaPointerLess>(*pDedicatedAllocations, allocation);
    8454  VMA_ASSERT(success);
    8455  }
    8456 
    8457  VkDeviceMemory hMemory = allocation->GetMemory();
    8458 
    8459  if(allocation->GetMappedData() != VMA_NULL)
    8460  {
    8461  (*m_VulkanFunctions.vkUnmapMemory)(m_hDevice, hMemory);
    8462  }
    8463 
    8464  FreeVulkanMemory(memTypeIndex, allocation->GetSize(), hMemory);
    8465 
    8466  VMA_DEBUG_LOG(" Freed DedicatedMemory MemoryTypeIndex=%u", memTypeIndex);
    8467 }
    8468 
    8469 #if VMA_STATS_STRING_ENABLED
    8470 
    8471 void VmaAllocator_T::PrintDetailedMap(VmaJsonWriter& json)
    8472 {
    8473  bool dedicatedAllocationsStarted = false;
    8474  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
    8475  {
    8476  VmaMutexLock dedicatedAllocationsLock(m_DedicatedAllocationsMutex[memTypeIndex], m_UseMutex);
    8477  AllocationVectorType* const pDedicatedAllocVector = m_pDedicatedAllocations[memTypeIndex];
    8478  VMA_ASSERT(pDedicatedAllocVector);
    8479  if(pDedicatedAllocVector->empty() == false)
    8480  {
    8481  if(dedicatedAllocationsStarted == false)
    8482  {
    8483  dedicatedAllocationsStarted = true;
    8484  json.WriteString("DedicatedAllocations");
    8485  json.BeginObject();
    8486  }
    8487 
    8488  json.BeginString("Type ");
    8489  json.ContinueString(memTypeIndex);
    8490  json.EndString();
    8491 
    8492  json.BeginArray();
    8493 
    8494  for(size_t i = 0; i < pDedicatedAllocVector->size(); ++i)
    8495  {
    8496  const VmaAllocation hAlloc = (*pDedicatedAllocVector)[i];
    8497  json.BeginObject(true);
    8498 
    8499  json.WriteString("Type");
    8500  json.WriteString(VMA_SUBALLOCATION_TYPE_NAMES[hAlloc->GetSuballocationType()]);
    8501 
    8502  json.WriteString("Size");
    8503  json.WriteNumber(hAlloc->GetSize());
    8504 
    8505  const void* pUserData = hAlloc->GetUserData();
    8506  if(pUserData != VMA_NULL)
    8507  {
    8508  json.WriteString("UserData");
    8509  if(hAlloc->IsUserDataString())
    8510  {
    8511  json.WriteString((const char*)pUserData);
    8512  }
    8513  else
    8514  {
    8515  json.BeginString();
    8516  json.ContinueString_Pointer(pUserData);
    8517  json.EndString();
    8518  }
    8519  }
    8520 
    8521  json.EndObject();
    8522  }
    8523 
    8524  json.EndArray();
    8525  }
    8526  }
    8527  if(dedicatedAllocationsStarted)
    8528  {
    8529  json.EndObject();
    8530  }
    8531 
    8532  {
    8533  bool allocationsStarted = false;
    8534  for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
    8535  {
    8536  if(m_pBlockVectors[memTypeIndex]->IsEmpty() == false)
    8537  {
    8538  if(allocationsStarted == false)
    8539  {
    8540  allocationsStarted = true;
    8541  json.WriteString("DefaultPools");
    8542  json.BeginObject();
    8543  }
    8544 
    8545  json.BeginString("Type ");
    8546  json.ContinueString(memTypeIndex);
    8547  json.EndString();
    8548 
    8549  m_pBlockVectors[memTypeIndex]->PrintDetailedMap(json);
    8550  }
    8551  }
    8552  if(allocationsStarted)
    8553  {
    8554  json.EndObject();
    8555  }
    8556  }
    8557 
    8558  {
    8559  VmaMutexLock lock(m_PoolsMutex, m_UseMutex);
    8560  const size_t poolCount = m_Pools.size();
    8561  if(poolCount > 0)
    8562  {
    8563  json.WriteString("Pools");
    8564  json.BeginArray();
    8565  for(size_t poolIndex = 0; poolIndex < poolCount; ++poolIndex)
    8566  {
    8567  m_Pools[poolIndex]->m_BlockVector.PrintDetailedMap(json);
    8568  }
    8569  json.EndArray();
    8570  }
    8571  }
    8572 }
    8573 
    8574 #endif // #if VMA_STATS_STRING_ENABLED
    8575 
    8576 static VkResult AllocateMemoryForImage(
    8577  VmaAllocator allocator,
    8578  VkImage image,
    8579  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    8580  VmaSuballocationType suballocType,
    8581  VmaAllocation* pAllocation)
    8582 {
    8583  VMA_ASSERT(allocator && (image != VK_NULL_HANDLE) && pAllocationCreateInfo && pAllocation);
    8584 
    8585  VkMemoryRequirements vkMemReq = {};
    8586  bool requiresDedicatedAllocation = false;
    8587  bool prefersDedicatedAllocation = false;
    8588  allocator->GetImageMemoryRequirements(image, vkMemReq,
    8589  requiresDedicatedAllocation, prefersDedicatedAllocation);
    8590 
    8591  return allocator->AllocateMemory(
    8592  vkMemReq,
    8593  requiresDedicatedAllocation,
    8594  prefersDedicatedAllocation,
    8595  VK_NULL_HANDLE, // dedicatedBuffer
    8596  image, // dedicatedImage
    8597  *pAllocationCreateInfo,
    8598  suballocType,
    8599  pAllocation);
    8600 }
    8601 
    8603 // Public interface
    8604 
    8605 VkResult vmaCreateAllocator(
    8606  const VmaAllocatorCreateInfo* pCreateInfo,
    8607  VmaAllocator* pAllocator)
    8608 {
    8609  VMA_ASSERT(pCreateInfo && pAllocator);
    8610  VMA_DEBUG_LOG("vmaCreateAllocator");
    8611  *pAllocator = vma_new(pCreateInfo->pAllocationCallbacks, VmaAllocator_T)(pCreateInfo);
    8612  return VK_SUCCESS;
    8613 }
    8614 
    8615 void vmaDestroyAllocator(
    8616  VmaAllocator allocator)
    8617 {
    8618  if(allocator != VK_NULL_HANDLE)
    8619  {
    8620  VMA_DEBUG_LOG("vmaDestroyAllocator");
    8621  VkAllocationCallbacks allocationCallbacks = allocator->m_AllocationCallbacks;
    8622  vma_delete(&allocationCallbacks, allocator);
    8623  }
    8624 }
    8625 
    8626 void vmaGetPhysicalDeviceProperties(
    8627  VmaAllocator allocator,
    8628  const VkPhysicalDeviceProperties **ppPhysicalDeviceProperties)
    8629 {
    8630  VMA_ASSERT(allocator && ppPhysicalDeviceProperties);
    8631  *ppPhysicalDeviceProperties = &allocator->m_PhysicalDeviceProperties;
    8632 }
    8633 
    8634 void vmaGetMemoryProperties(
    8635  VmaAllocator allocator,
    8636  const VkPhysicalDeviceMemoryProperties** ppPhysicalDeviceMemoryProperties)
    8637 {
    8638  VMA_ASSERT(allocator && ppPhysicalDeviceMemoryProperties);
    8639  *ppPhysicalDeviceMemoryProperties = &allocator->m_MemProps;
    8640 }
    8641 
    8642 void vmaGetMemoryTypeProperties(
    8643  VmaAllocator allocator,
    8644  uint32_t memoryTypeIndex,
    8645  VkMemoryPropertyFlags* pFlags)
    8646 {
    8647  VMA_ASSERT(allocator && pFlags);
    8648  VMA_ASSERT(memoryTypeIndex < allocator->GetMemoryTypeCount());
    8649  *pFlags = allocator->m_MemProps.memoryTypes[memoryTypeIndex].propertyFlags;
    8650 }
    8651 
    8652 void vmaSetCurrentFrameIndex(
    8653  VmaAllocator allocator,
    8654  uint32_t frameIndex)
    8655 {
    8656  VMA_ASSERT(allocator);
    8657  VMA_ASSERT(frameIndex != VMA_FRAME_INDEX_LOST);
    8658 
    8659  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8660 
    8661  allocator->SetCurrentFrameIndex(frameIndex);
    8662 }
    8663 
    8664 void vmaCalculateStats(
    8665  VmaAllocator allocator,
    8666  VmaStats* pStats)
    8667 {
    8668  VMA_ASSERT(allocator && pStats);
    8669  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8670  allocator->CalculateStats(pStats);
    8671 }
    8672 
    8673 #if VMA_STATS_STRING_ENABLED
    8674 
    8675 void vmaBuildStatsString(
    8676  VmaAllocator allocator,
    8677  char** ppStatsString,
    8678  VkBool32 detailedMap)
    8679 {
    8680  VMA_ASSERT(allocator && ppStatsString);
    8681  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8682 
    8683  VmaStringBuilder sb(allocator);
    8684  {
    8685  VmaJsonWriter json(allocator->GetAllocationCallbacks(), sb);
    8686  json.BeginObject();
    8687 
    8688  VmaStats stats;
    8689  allocator->CalculateStats(&stats);
    8690 
    8691  json.WriteString("Total");
    8692  VmaPrintStatInfo(json, stats.total);
    8693 
    8694  for(uint32_t heapIndex = 0; heapIndex < allocator->GetMemoryHeapCount(); ++heapIndex)
    8695  {
    8696  json.BeginString("Heap ");
    8697  json.ContinueString(heapIndex);
    8698  json.EndString();
    8699  json.BeginObject();
    8700 
    8701  json.WriteString("Size");
    8702  json.WriteNumber(allocator->m_MemProps.memoryHeaps[heapIndex].size);
    8703 
    8704  json.WriteString("Flags");
    8705  json.BeginArray(true);
    8706  if((allocator->m_MemProps.memoryHeaps[heapIndex].flags & VK_MEMORY_HEAP_DEVICE_LOCAL_BIT) != 0)
    8707  {
    8708  json.WriteString("DEVICE_LOCAL");
    8709  }
    8710  json.EndArray();
    8711 
    8712  if(stats.memoryHeap[heapIndex].blockCount > 0)
    8713  {
    8714  json.WriteString("Stats");
    8715  VmaPrintStatInfo(json, stats.memoryHeap[heapIndex]);
    8716  }
    8717 
    8718  for(uint32_t typeIndex = 0; typeIndex < allocator->GetMemoryTypeCount(); ++typeIndex)
    8719  {
    8720  if(allocator->MemoryTypeIndexToHeapIndex(typeIndex) == heapIndex)
    8721  {
    8722  json.BeginString("Type ");
    8723  json.ContinueString(typeIndex);
    8724  json.EndString();
    8725 
    8726  json.BeginObject();
    8727 
    8728  json.WriteString("Flags");
    8729  json.BeginArray(true);
    8730  VkMemoryPropertyFlags flags = allocator->m_MemProps.memoryTypes[typeIndex].propertyFlags;
    8731  if((flags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT) != 0)
    8732  {
    8733  json.WriteString("DEVICE_LOCAL");
    8734  }
    8735  if((flags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
    8736  {
    8737  json.WriteString("HOST_VISIBLE");
    8738  }
    8739  if((flags & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT) != 0)
    8740  {
    8741  json.WriteString("HOST_COHERENT");
    8742  }
    8743  if((flags & VK_MEMORY_PROPERTY_HOST_CACHED_BIT) != 0)
    8744  {
    8745  json.WriteString("HOST_CACHED");
    8746  }
    8747  if((flags & VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT) != 0)
    8748  {
    8749  json.WriteString("LAZILY_ALLOCATED");
    8750  }
    8751  json.EndArray();
    8752 
    8753  if(stats.memoryType[typeIndex].blockCount > 0)
    8754  {
    8755  json.WriteString("Stats");
    8756  VmaPrintStatInfo(json, stats.memoryType[typeIndex]);
    8757  }
    8758 
    8759  json.EndObject();
    8760  }
    8761  }
    8762 
    8763  json.EndObject();
    8764  }
    8765  if(detailedMap == VK_TRUE)
    8766  {
    8767  allocator->PrintDetailedMap(json);
    8768  }
    8769 
    8770  json.EndObject();
    8771  }
    8772 
    8773  const size_t len = sb.GetLength();
    8774  char* const pChars = vma_new_array(allocator, char, len + 1);
    8775  if(len > 0)
    8776  {
    8777  memcpy(pChars, sb.GetData(), len);
    8778  }
    8779  pChars[len] = '\0';
    8780  *ppStatsString = pChars;
    8781 }
    8782 
    8783 void vmaFreeStatsString(
    8784  VmaAllocator allocator,
    8785  char* pStatsString)
    8786 {
    8787  if(pStatsString != VMA_NULL)
    8788  {
    8789  VMA_ASSERT(allocator);
    8790  size_t len = strlen(pStatsString);
    8791  vma_delete_array(allocator, pStatsString, len + 1);
    8792  }
    8793 }
    8794 
    8795 #endif // #if VMA_STATS_STRING_ENABLED
    8796 
    8797 /*
    8798 This function is not protected by any mutex because it just reads immutable data.
    8799 */
    8800 VkResult vmaFindMemoryTypeIndex(
    8801  VmaAllocator allocator,
    8802  uint32_t memoryTypeBits,
    8803  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    8804  uint32_t* pMemoryTypeIndex)
    8805 {
    8806  VMA_ASSERT(allocator != VK_NULL_HANDLE);
    8807  VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
    8808  VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
    8809 
    8810  if(pAllocationCreateInfo->memoryTypeBits != 0)
    8811  {
    8812  memoryTypeBits &= pAllocationCreateInfo->memoryTypeBits;
    8813  }
    8814 
    8815  uint32_t requiredFlags = pAllocationCreateInfo->requiredFlags;
    8816  uint32_t preferredFlags = pAllocationCreateInfo->preferredFlags;
    8817 
    8818  // Convert usage to requiredFlags and preferredFlags.
    8819  switch(pAllocationCreateInfo->usage)
    8820  {
    8821  case VMA_MEMORY_USAGE_UNKNOWN:
    8822  break;
    8823  case VMA_MEMORY_USAGE_GPU_ONLY:
    8824  preferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
    8825  break;
    8826  case VMA_MEMORY_USAGE_CPU_ONLY:
    8827  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
    8828  break;
    8829  case VMA_MEMORY_USAGE_CPU_TO_GPU:
    8830  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
    8831  preferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
    8832  break;
    8833  case VMA_MEMORY_USAGE_GPU_TO_CPU:
    8834  requiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
    8835  preferredFlags |= VK_MEMORY_PROPERTY_HOST_COHERENT_BIT | VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
    8836  break;
    8837  default:
    8838  break;
    8839  }
    8840 
    8841  *pMemoryTypeIndex = UINT32_MAX;
    8842  uint32_t minCost = UINT32_MAX;
    8843  for(uint32_t memTypeIndex = 0, memTypeBit = 1;
    8844  memTypeIndex < allocator->GetMemoryTypeCount();
    8845  ++memTypeIndex, memTypeBit <<= 1)
    8846  {
    8847  // This memory type is acceptable according to memoryTypeBits bitmask.
    8848  if((memTypeBit & memoryTypeBits) != 0)
    8849  {
    8850  const VkMemoryPropertyFlags currFlags =
    8851  allocator->m_MemProps.memoryTypes[memTypeIndex].propertyFlags;
    8852  // This memory type contains requiredFlags.
    8853  if((requiredFlags & ~currFlags) == 0)
    8854  {
    8855  // Calculate cost as number of bits from preferredFlags not present in this memory type.
    8856  uint32_t currCost = VmaCountBitsSet(preferredFlags & ~currFlags);
    8857  // Remember memory type with lowest cost.
    8858  if(currCost < minCost)
    8859  {
    8860  *pMemoryTypeIndex = memTypeIndex;
    8861  if(currCost == 0)
    8862  {
    8863  return VK_SUCCESS;
    8864  }
    8865  minCost = currCost;
    8866  }
    8867  }
    8868  }
    8869  }
    8870  return (*pMemoryTypeIndex != UINT32_MAX) ? VK_SUCCESS : VK_ERROR_FEATURE_NOT_PRESENT;
    8871 }
    8872 
    8873 VkResult vmaFindMemoryTypeIndexForBufferInfo(
    8874  VmaAllocator allocator,
    8875  const VkBufferCreateInfo* pBufferCreateInfo,
    8876  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    8877  uint32_t* pMemoryTypeIndex)
    8878 {
    8879  VMA_ASSERT(allocator != VK_NULL_HANDLE);
    8880  VMA_ASSERT(pBufferCreateInfo != VMA_NULL);
    8881  VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
    8882  VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
    8883 
    8884  const VkDevice hDev = allocator->m_hDevice;
    8885  VkBuffer hBuffer = VK_NULL_HANDLE;
    8886  VkResult res = allocator->GetVulkanFunctions().vkCreateBuffer(
    8887  hDev, pBufferCreateInfo, allocator->GetAllocationCallbacks(), &hBuffer);
    8888  if(res == VK_SUCCESS)
    8889  {
    8890  VkMemoryRequirements memReq = {};
    8891  allocator->GetVulkanFunctions().vkGetBufferMemoryRequirements(
    8892  hDev, hBuffer, &memReq);
    8893 
    8894  res = vmaFindMemoryTypeIndex(
    8895  allocator,
    8896  memReq.memoryTypeBits,
    8897  pAllocationCreateInfo,
    8898  pMemoryTypeIndex);
    8899 
    8900  allocator->GetVulkanFunctions().vkDestroyBuffer(
    8901  hDev, hBuffer, allocator->GetAllocationCallbacks());
    8902  }
    8903  return res;
    8904 }
    8905 
    8906 VkResult vmaFindMemoryTypeIndexForImageInfo(
    8907  VmaAllocator allocator,
    8908  const VkImageCreateInfo* pImageCreateInfo,
    8909  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    8910  uint32_t* pMemoryTypeIndex)
    8911 {
    8912  VMA_ASSERT(allocator != VK_NULL_HANDLE);
    8913  VMA_ASSERT(pImageCreateInfo != VMA_NULL);
    8914  VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
    8915  VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
    8916 
    8917  const VkDevice hDev = allocator->m_hDevice;
    8918  VkImage hImage = VK_NULL_HANDLE;
    8919  VkResult res = allocator->GetVulkanFunctions().vkCreateImage(
    8920  hDev, pImageCreateInfo, allocator->GetAllocationCallbacks(), &hImage);
    8921  if(res == VK_SUCCESS)
    8922  {
    8923  VkMemoryRequirements memReq = {};
    8924  allocator->GetVulkanFunctions().vkGetImageMemoryRequirements(
    8925  hDev, hImage, &memReq);
    8926 
    8927  res = vmaFindMemoryTypeIndex(
    8928  allocator,
    8929  memReq.memoryTypeBits,
    8930  pAllocationCreateInfo,
    8931  pMemoryTypeIndex);
    8932 
    8933  allocator->GetVulkanFunctions().vkDestroyImage(
    8934  hDev, hImage, allocator->GetAllocationCallbacks());
    8935  }
    8936  return res;
    8937 }
    8938 
    8939 VkResult vmaCreatePool(
    8940  VmaAllocator allocator,
    8941  const VmaPoolCreateInfo* pCreateInfo,
    8942  VmaPool* pPool)
    8943 {
    8944  VMA_ASSERT(allocator && pCreateInfo && pPool);
    8945 
    8946  VMA_DEBUG_LOG("vmaCreatePool");
    8947 
    8948  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8949 
    8950  return allocator->CreatePool(pCreateInfo, pPool);
    8951 }
    8952 
    8953 void vmaDestroyPool(
    8954  VmaAllocator allocator,
    8955  VmaPool pool)
    8956 {
    8957  VMA_ASSERT(allocator);
    8958 
    8959  if(pool == VK_NULL_HANDLE)
    8960  {
    8961  return;
    8962  }
    8963 
    8964  VMA_DEBUG_LOG("vmaDestroyPool");
    8965 
    8966  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8967 
    8968  allocator->DestroyPool(pool);
    8969 }
    8970 
    8971 void vmaGetPoolStats(
    8972  VmaAllocator allocator,
    8973  VmaPool pool,
    8974  VmaPoolStats* pPoolStats)
    8975 {
    8976  VMA_ASSERT(allocator && pool && pPoolStats);
    8977 
    8978  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8979 
    8980  allocator->GetPoolStats(pool, pPoolStats);
    8981 }
    8982 
    8983 void vmaMakePoolAllocationsLost(
    8984  VmaAllocator allocator,
    8985  VmaPool pool,
    8986  size_t* pLostAllocationCount)
    8987 {
    8988  VMA_ASSERT(allocator && pool);
    8989 
    8990  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    8991 
    8992  allocator->MakePoolAllocationsLost(pool, pLostAllocationCount);
    8993 }
    8994 
    8995 VkResult vmaAllocateMemory(
    8996  VmaAllocator allocator,
    8997  const VkMemoryRequirements* pVkMemoryRequirements,
    8998  const VmaAllocationCreateInfo* pCreateInfo,
    8999  VmaAllocation* pAllocation,
    9000  VmaAllocationInfo* pAllocationInfo)
    9001 {
    9002  VMA_ASSERT(allocator && pVkMemoryRequirements && pCreateInfo && pAllocation);
    9003 
    9004  VMA_DEBUG_LOG("vmaAllocateMemory");
    9005 
    9006  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9007 
    9008  VkResult result = allocator->AllocateMemory(
    9009  *pVkMemoryRequirements,
    9010  false, // requiresDedicatedAllocation
    9011  false, // prefersDedicatedAllocation
    9012  VK_NULL_HANDLE, // dedicatedBuffer
    9013  VK_NULL_HANDLE, // dedicatedImage
    9014  *pCreateInfo,
    9015  VMA_SUBALLOCATION_TYPE_UNKNOWN,
    9016  pAllocation);
    9017 
    9018  if(pAllocationInfo && result == VK_SUCCESS)
    9019  {
    9020  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
    9021  }
    9022 
    9023  return result;
    9024 }
    9025 
    9026 VkResult vmaAllocateMemoryForBuffer(
    9027  VmaAllocator allocator,
    9028  VkBuffer buffer,
    9029  const VmaAllocationCreateInfo* pCreateInfo,
    9030  VmaAllocation* pAllocation,
    9031  VmaAllocationInfo* pAllocationInfo)
    9032 {
    9033  VMA_ASSERT(allocator && buffer != VK_NULL_HANDLE && pCreateInfo && pAllocation);
    9034 
    9035  VMA_DEBUG_LOG("vmaAllocateMemoryForBuffer");
    9036 
    9037  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9038 
    9039  VkMemoryRequirements vkMemReq = {};
    9040  bool requiresDedicatedAllocation = false;
    9041  bool prefersDedicatedAllocation = false;
    9042  allocator->GetBufferMemoryRequirements(buffer, vkMemReq,
    9043  requiresDedicatedAllocation,
    9044  prefersDedicatedAllocation);
    9045 
    9046  VkResult result = allocator->AllocateMemory(
    9047  vkMemReq,
    9048  requiresDedicatedAllocation,
    9049  prefersDedicatedAllocation,
    9050  buffer, // dedicatedBuffer
    9051  VK_NULL_HANDLE, // dedicatedImage
    9052  *pCreateInfo,
    9053  VMA_SUBALLOCATION_TYPE_BUFFER,
    9054  pAllocation);
    9055 
    9056  if(pAllocationInfo && result == VK_SUCCESS)
    9057  {
    9058  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
    9059  }
    9060 
    9061  return result;
    9062 }
    9063 
    9064 VkResult vmaAllocateMemoryForImage(
    9065  VmaAllocator allocator,
    9066  VkImage image,
    9067  const VmaAllocationCreateInfo* pCreateInfo,
    9068  VmaAllocation* pAllocation,
    9069  VmaAllocationInfo* pAllocationInfo)
    9070 {
    9071  VMA_ASSERT(allocator && image != VK_NULL_HANDLE && pCreateInfo && pAllocation);
    9072 
    9073  VMA_DEBUG_LOG("vmaAllocateMemoryForImage");
    9074 
    9075  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9076 
    9077  VkResult result = AllocateMemoryForImage(
    9078  allocator,
    9079  image,
    9080  pCreateInfo,
    9081  VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN,
    9082  pAllocation);
    9083 
    9084  if(pAllocationInfo && result == VK_SUCCESS)
    9085  {
    9086  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
    9087  }
    9088 
    9089  return result;
    9090 }
    9091 
    9092 void vmaFreeMemory(
    9093  VmaAllocator allocator,
    9094  VmaAllocation allocation)
    9095 {
    9096  VMA_ASSERT(allocator);
    9097  VMA_DEBUG_LOG("vmaFreeMemory");
    9098  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9099  if(allocation != VK_NULL_HANDLE)
    9100  {
    9101  allocator->FreeMemory(allocation);
    9102  }
    9103 }
    9104 
    9105 void vmaGetAllocationInfo(
    9106  VmaAllocator allocator,
    9107  VmaAllocation allocation,
    9108  VmaAllocationInfo* pAllocationInfo)
    9109 {
    9110  VMA_ASSERT(allocator && allocation && pAllocationInfo);
    9111 
    9112  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9113 
    9114  allocator->GetAllocationInfo(allocation, pAllocationInfo);
    9115 }
    9116 
    9117 VkBool32 vmaTouchAllocation(
    9118  VmaAllocator allocator,
    9119  VmaAllocation allocation)
    9120 {
    9121  VMA_ASSERT(allocator && allocation);
    9122 
    9123  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9124 
    9125  return allocator->TouchAllocation(allocation);
    9126 }
    9127 
    9128 void vmaSetAllocationUserData(
    9129  VmaAllocator allocator,
    9130  VmaAllocation allocation,
    9131  void* pUserData)
    9132 {
    9133  VMA_ASSERT(allocator && allocation);
    9134 
    9135  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9136 
    9137  allocation->SetUserData(allocator, pUserData);
    9138 }
    9139 
    9140 void vmaCreateLostAllocation(
    9141  VmaAllocator allocator,
    9142  VmaAllocation* pAllocation)
    9143 {
    9144  VMA_ASSERT(allocator && pAllocation);
    9145 
    9146  VMA_DEBUG_GLOBAL_MUTEX_LOCK;
    9147 
    9148  allocator->CreateLostAllocation(pAllocation);
    9149 }
    9150 
    9151 VkResult vmaMapMemory(
    9152  VmaAllocator allocator,
    9153  VmaAllocation allocation,
    9154  void** ppData)
    9155 {
    9156  VMA_ASSERT(allocator && allocation && ppData);
    9157 
    9158  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9159 
    9160  return allocator->Map(allocation, ppData);
    9161 }
    9162 
    9163 void vmaUnmapMemory(
    9164  VmaAllocator allocator,
    9165  VmaAllocation allocation)
    9166 {
    9167  VMA_ASSERT(allocator && allocation);
    9168 
    9169  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9170 
    9171  allocator->Unmap(allocation);
    9172 }
    9173 
    9174 VkResult vmaDefragment(
    9175  VmaAllocator allocator,
    9176  VmaAllocation* pAllocations,
    9177  size_t allocationCount,
    9178  VkBool32* pAllocationsChanged,
    9179  const VmaDefragmentationInfo *pDefragmentationInfo,
    9180  VmaDefragmentationStats* pDefragmentationStats)
    9181 {
    9182  VMA_ASSERT(allocator && pAllocations);
    9183 
    9184  VMA_DEBUG_LOG("vmaDefragment");
    9185 
    9186  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9187 
    9188  return allocator->Defragment(pAllocations, allocationCount, pAllocationsChanged, pDefragmentationInfo, pDefragmentationStats);
    9189 }
    9190 
    9191 VkResult vmaBindBufferMemory(
    9192  VmaAllocator allocator,
    9193  VmaAllocation allocation,
    9194  VkBuffer buffer)
    9195 {
    9196  VMA_ASSERT(allocator && allocation && buffer);
    9197 
    9198  VMA_DEBUG_LOG("vmaBindBufferMemory");
    9199 
    9200  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9201 
    9202  return allocator->BindBufferMemory(allocation, buffer);
    9203 }
    9204 
    9205 VkResult vmaBindImageMemory(
    9206  VmaAllocator allocator,
    9207  VmaAllocation allocation,
    9208  VkImage image)
    9209 {
    9210  VMA_ASSERT(allocator && allocation && image);
    9211 
    9212  VMA_DEBUG_LOG("vmaBindImageMemory");
    9213 
    9214  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9215 
    9216  return allocator->BindImageMemory(allocation, image);
    9217 }
    9218 
    9219 VkResult vmaCreateBuffer(
    9220  VmaAllocator allocator,
    9221  const VkBufferCreateInfo* pBufferCreateInfo,
    9222  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    9223  VkBuffer* pBuffer,
    9224  VmaAllocation* pAllocation,
    9225  VmaAllocationInfo* pAllocationInfo)
    9226 {
    9227  VMA_ASSERT(allocator && pBufferCreateInfo && pAllocationCreateInfo && pBuffer && pAllocation);
    9228 
    9229  VMA_DEBUG_LOG("vmaCreateBuffer");
    9230 
    9231  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9232 
    9233  *pBuffer = VK_NULL_HANDLE;
    9234  *pAllocation = VK_NULL_HANDLE;
    9235 
    9236  // 1. Create VkBuffer.
    9237  VkResult res = (*allocator->GetVulkanFunctions().vkCreateBuffer)(
    9238  allocator->m_hDevice,
    9239  pBufferCreateInfo,
    9240  allocator->GetAllocationCallbacks(),
    9241  pBuffer);
    9242  if(res >= 0)
    9243  {
    9244  // 2. vkGetBufferMemoryRequirements.
    9245  VkMemoryRequirements vkMemReq = {};
    9246  bool requiresDedicatedAllocation = false;
    9247  bool prefersDedicatedAllocation = false;
    9248  allocator->GetBufferMemoryRequirements(*pBuffer, vkMemReq,
    9249  requiresDedicatedAllocation, prefersDedicatedAllocation);
    9250 
    9251  // Make sure alignment requirements for specific buffer usages reported
    9252  // in Physical Device Properties are included in alignment reported by memory requirements.
    9253  if((pBufferCreateInfo->usage & VK_BUFFER_USAGE_UNIFORM_TEXEL_BUFFER_BIT) != 0)
    9254  {
    9255  VMA_ASSERT(vkMemReq.alignment %
    9256  allocator->m_PhysicalDeviceProperties.limits.minTexelBufferOffsetAlignment == 0);
    9257  }
    9258  if((pBufferCreateInfo->usage & VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT) != 0)
    9259  {
    9260  VMA_ASSERT(vkMemReq.alignment %
    9261  allocator->m_PhysicalDeviceProperties.limits.minUniformBufferOffsetAlignment == 0);
    9262  }
    9263  if((pBufferCreateInfo->usage & VK_BUFFER_USAGE_STORAGE_BUFFER_BIT) != 0)
    9264  {
    9265  VMA_ASSERT(vkMemReq.alignment %
    9266  allocator->m_PhysicalDeviceProperties.limits.minStorageBufferOffsetAlignment == 0);
    9267  }
    9268 
    9269  // 3. Allocate memory using allocator.
    9270  res = allocator->AllocateMemory(
    9271  vkMemReq,
    9272  requiresDedicatedAllocation,
    9273  prefersDedicatedAllocation,
    9274  *pBuffer, // dedicatedBuffer
    9275  VK_NULL_HANDLE, // dedicatedImage
    9276  *pAllocationCreateInfo,
    9277  VMA_SUBALLOCATION_TYPE_BUFFER,
    9278  pAllocation);
    9279  if(res >= 0)
    9280  {
    9281  // 3. Bind buffer with memory.
    9282  res = allocator->BindBufferMemory(*pAllocation, *pBuffer);
    9283  if(res >= 0)
    9284  {
    9285  // All steps succeeded.
    9286  if(pAllocationInfo != VMA_NULL)
    9287  {
    9288  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
    9289  }
    9290  return VK_SUCCESS;
    9291  }
    9292  allocator->FreeMemory(*pAllocation);
    9293  *pAllocation = VK_NULL_HANDLE;
    9294  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
    9295  *pBuffer = VK_NULL_HANDLE;
    9296  return res;
    9297  }
    9298  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
    9299  *pBuffer = VK_NULL_HANDLE;
    9300  return res;
    9301  }
    9302  return res;
    9303 }
    9304 
    9305 void vmaDestroyBuffer(
    9306  VmaAllocator allocator,
    9307  VkBuffer buffer,
    9308  VmaAllocation allocation)
    9309 {
    9310  VMA_ASSERT(allocator);
    9311  VMA_DEBUG_LOG("vmaDestroyBuffer");
    9312  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9313  if(buffer != VK_NULL_HANDLE)
    9314  {
    9315  (*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, buffer, allocator->GetAllocationCallbacks());
    9316  }
    9317  if(allocation != VK_NULL_HANDLE)
    9318  {
    9319  allocator->FreeMemory(allocation);
    9320  }
    9321 }
    9322 
    9323 VkResult vmaCreateImage(
    9324  VmaAllocator allocator,
    9325  const VkImageCreateInfo* pImageCreateInfo,
    9326  const VmaAllocationCreateInfo* pAllocationCreateInfo,
    9327  VkImage* pImage,
    9328  VmaAllocation* pAllocation,
    9329  VmaAllocationInfo* pAllocationInfo)
    9330 {
    9331  VMA_ASSERT(allocator && pImageCreateInfo && pAllocationCreateInfo && pImage && pAllocation);
    9332 
    9333  VMA_DEBUG_LOG("vmaCreateImage");
    9334 
    9335  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9336 
    9337  *pImage = VK_NULL_HANDLE;
    9338  *pAllocation = VK_NULL_HANDLE;
    9339 
    9340  // 1. Create VkImage.
    9341  VkResult res = (*allocator->GetVulkanFunctions().vkCreateImage)(
    9342  allocator->m_hDevice,
    9343  pImageCreateInfo,
    9344  allocator->GetAllocationCallbacks(),
    9345  pImage);
    9346  if(res >= 0)
    9347  {
    9348  VmaSuballocationType suballocType = pImageCreateInfo->tiling == VK_IMAGE_TILING_OPTIMAL ?
    9349  VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL :
    9350  VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR;
    9351 
    9352  // 2. Allocate memory using allocator.
    9353  res = AllocateMemoryForImage(allocator, *pImage, pAllocationCreateInfo, suballocType, pAllocation);
    9354  if(res >= 0)
    9355  {
    9356  // 3. Bind image with memory.
    9357  res = allocator->BindImageMemory(*pAllocation, *pImage);
    9358  if(res >= 0)
    9359  {
    9360  // All steps succeeded.
    9361  if(pAllocationInfo != VMA_NULL)
    9362  {
    9363  allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
    9364  }
    9365  return VK_SUCCESS;
    9366  }
    9367  allocator->FreeMemory(*pAllocation);
    9368  *pAllocation = VK_NULL_HANDLE;
    9369  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, *pImage, allocator->GetAllocationCallbacks());
    9370  *pImage = VK_NULL_HANDLE;
    9371  return res;
    9372  }
    9373  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, *pImage, allocator->GetAllocationCallbacks());
    9374  *pImage = VK_NULL_HANDLE;
    9375  return res;
    9376  }
    9377  return res;
    9378 }
    9379 
    9380 void vmaDestroyImage(
    9381  VmaAllocator allocator,
    9382  VkImage image,
    9383  VmaAllocation allocation)
    9384 {
    9385  VMA_ASSERT(allocator);
    9386  VMA_DEBUG_LOG("vmaDestroyImage");
    9387  VMA_DEBUG_GLOBAL_MUTEX_LOCK
    9388  if(image != VK_NULL_HANDLE)
    9389  {
    9390  (*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, image, allocator->GetAllocationCallbacks());
    9391  }
    9392  if(allocation != VK_NULL_HANDLE)
    9393  {
    9394  allocator->FreeMemory(allocation);
    9395  }
    9396 }
    9397 
    9398 #endif // #ifdef VMA_IMPLEMENTATION
    VmaVulkanFunctions::vkGetPhysicalDeviceProperties
    PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties
    Definition: vk_mem_alloc.h:1169
    +
    VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT
    Set this flag if the allocation should have its own memory block.
    Definition: vk_mem_alloc.h:1433
    vmaUnmapMemory
    void vmaUnmapMemory(VmaAllocator allocator, VmaAllocation allocation)
    Unmaps memory represented by given allocation, mapped previously using vmaMapMemory().
    -
    VmaAllocatorCreateInfo::physicalDevice
    VkPhysicalDevice physicalDevice
    Vulkan physical device.
    Definition: vk_mem_alloc.h:1182
    +
    VmaAllocatorCreateInfo::physicalDevice
    VkPhysicalDevice physicalDevice
    Vulkan physical device.
    Definition: vk_mem_alloc.h:1196
    vmaDefragment
    VkResult vmaDefragment(VmaAllocator allocator, VmaAllocation *pAllocations, size_t allocationCount, VkBool32 *pAllocationsChanged, const VmaDefragmentationInfo *pDefragmentationInfo, VmaDefragmentationStats *pDefragmentationStats)
    Compacts memory by moving allocations.
    VmaAllocation
    Represents single memory allocation.
    -
    VmaVulkanFunctions::vkCreateBuffer
    PFN_vkCreateBuffer vkCreateBuffer
    Definition: vk_mem_alloc.h:1167
    +
    VmaVulkanFunctions::vkCreateBuffer
    PFN_vkCreateBuffer vkCreateBuffer
    Definition: vk_mem_alloc.h:1179
    vmaFreeStatsString
    void vmaFreeStatsString(VmaAllocator allocator, char *pStatsString)
    VmaStats
    struct VmaStats VmaStats
    General statistics from current state of Allocator.
    -
    VMA_MEMORY_USAGE_GPU_ONLY
    Definition: vk_mem_alloc.h:1376
    -
    VmaVulkanFunctions::vkMapMemory
    PFN_vkMapMemory vkMapMemory
    Definition: vk_mem_alloc.h:1161
    -
    VmaAllocationInfo::deviceMemory
    VkDeviceMemory deviceMemory
    Handle to Vulkan memory object.
    Definition: vk_mem_alloc.h:1749
    -
    VmaAllocatorCreateInfo::flags
    VmaAllocatorCreateFlags flags
    Flags for created allocator. Use VmaAllocatorCreateFlagBits enum.
    Definition: vk_mem_alloc.h:1179
    -
    VmaDefragmentationInfo::maxAllocationsToMove
    uint32_t maxAllocationsToMove
    Maximum number of allocations that can be moved to different place.
    Definition: vk_mem_alloc.h:1948
    -
    VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT
    Use this flag if you always allocate only buffers and linear images or only optimal images out of thi...
    Definition: vk_mem_alloc.h:1595
    +
    VMA_MEMORY_USAGE_GPU_ONLY
    Definition: vk_mem_alloc.h:1390
    +
    VmaVulkanFunctions::vkMapMemory
    PFN_vkMapMemory vkMapMemory
    Definition: vk_mem_alloc.h:1173
    +
    VmaAllocationInfo::deviceMemory
    VkDeviceMemory deviceMemory
    Handle to Vulkan memory object.
    Definition: vk_mem_alloc.h:1763
    +
    VmaAllocatorCreateInfo::flags
    VmaAllocatorCreateFlags flags
    Flags for created allocator. Use VmaAllocatorCreateFlagBits enum.
    Definition: vk_mem_alloc.h:1193
    +
    VmaDefragmentationInfo::maxAllocationsToMove
    uint32_t maxAllocationsToMove
    Maximum number of allocations that can be moved to different place.
    Definition: vk_mem_alloc.h:1962
    +
    VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT
    Use this flag if you always allocate only buffers and linear images or only optimal images out of thi...
    Definition: vk_mem_alloc.h:1609
    vmaMakePoolAllocationsLost
    void vmaMakePoolAllocationsLost(VmaAllocator allocator, VmaPool pool, size_t *pLostAllocationCount)
    Marks all allocations in given pool as lost if they are not used in current frame or VmaPoolCreateInf...
    -
    VmaPoolStats::size
    VkDeviceSize size
    Total amount of VkDeviceMemory allocated from Vulkan for this pool, in bytes.
    Definition: vk_mem_alloc.h:1649
    -
    VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT
    Definition: vk_mem_alloc.h:1456
    -
    VmaAllocatorCreateFlags
    VkFlags VmaAllocatorCreateFlags
    Definition: vk_mem_alloc.h:1150
    -
    VmaAllocationCreateInfo::preferredFlags
    VkMemoryPropertyFlags preferredFlags
    Flags that preferably should be set in a memory type chosen for an allocation.
    Definition: vk_mem_alloc.h:1494
    -
    VMA_MEMORY_USAGE_MAX_ENUM
    Definition: vk_mem_alloc.h:1403
    -
    VmaAllocatorCreateInfo::pAllocationCallbacks
    const VkAllocationCallbacks * pAllocationCallbacks
    Custom CPU memory allocation callbacks. Optional.
    Definition: vk_mem_alloc.h:1191
    +
    VmaPoolStats::size
    VkDeviceSize size
    Total amount of VkDeviceMemory allocated from Vulkan for this pool, in bytes.
    Definition: vk_mem_alloc.h:1663
    +
    VMA_ALLOCATION_CREATE_CAN_BECOME_LOST_BIT
    Definition: vk_mem_alloc.h:1470
    +
    VmaAllocatorCreateFlags
    VkFlags VmaAllocatorCreateFlags
    Definition: vk_mem_alloc.h:1162
    +
    VmaAllocationCreateInfo::preferredFlags
    VkMemoryPropertyFlags preferredFlags
    Flags that preferably should be set in a memory type chosen for an allocation.
    Definition: vk_mem_alloc.h:1508
    +
    VMA_MEMORY_USAGE_MAX_ENUM
    Definition: vk_mem_alloc.h:1417
    +
    VmaAllocatorCreateInfo::pAllocationCallbacks
    const VkAllocationCallbacks * pAllocationCallbacks
    Custom CPU memory allocation callbacks. Optional.
    Definition: vk_mem_alloc.h:1205
    vmaCalculateStats
    void vmaCalculateStats(VmaAllocator allocator, VmaStats *pStats)
    Retrieves statistics from current state of the Allocator.
    -
    VmaAllocatorCreateInfo::pVulkanFunctions
    const VmaVulkanFunctions * pVulkanFunctions
    Pointers to Vulkan functions. Can be null if you leave define VMA_STATIC_VULKAN_FUNCTIONS 1...
    Definition: vk_mem_alloc.h:1244
    -
    VmaAllocatorCreateInfo
    Description of a Allocator to be created.
    Definition: vk_mem_alloc.h:1176
    +
    VmaAllocatorCreateInfo::pVulkanFunctions
    const VmaVulkanFunctions * pVulkanFunctions
    Pointers to Vulkan functions. Can be null if you leave define VMA_STATIC_VULKAN_FUNCTIONS 1...
    Definition: vk_mem_alloc.h:1258
    +
    VmaAllocatorCreateInfo
    Description of a Allocator to be created.
    Definition: vk_mem_alloc.h:1190
    vmaDestroyAllocator
    void vmaDestroyAllocator(VmaAllocator allocator)
    Destroys allocator object.
    -
    VmaAllocationCreateFlagBits
    VmaAllocationCreateFlagBits
    Flags to be passed as VmaAllocationCreateInfo::flags.
    Definition: vk_mem_alloc.h:1407
    +
    VmaAllocationCreateFlagBits
    VmaAllocationCreateFlagBits
    Flags to be passed as VmaAllocationCreateInfo::flags.
    Definition: vk_mem_alloc.h:1421
    vmaGetAllocationInfo
    void vmaGetAllocationInfo(VmaAllocator allocator, VmaAllocation allocation, VmaAllocationInfo *pAllocationInfo)
    Returns current information about specified allocation and atomically marks it as used in current fra...
    -
    VmaStatInfo::allocationSizeMax
    VkDeviceSize allocationSizeMax
    Definition: vk_mem_alloc.h:1309
    -
    VmaVulkanFunctions::vkBindImageMemory
    PFN_vkBindImageMemory vkBindImageMemory
    Definition: vk_mem_alloc.h:1164
    -
    VmaStatInfo::unusedBytes
    VkDeviceSize unusedBytes
    Total number of bytes occupied by unused ranges.
    Definition: vk_mem_alloc.h:1308
    -
    VmaVulkanFunctions::vkGetImageMemoryRequirements2KHR
    PFN_vkGetImageMemoryRequirements2KHR vkGetImageMemoryRequirements2KHR
    Definition: vk_mem_alloc.h:1172
    -
    VmaDefragmentationStats
    Statistics returned by function vmaDefragment().
    Definition: vk_mem_alloc.h:1952
    +
    VmaStatInfo::allocationSizeMax
    VkDeviceSize allocationSizeMax
    Definition: vk_mem_alloc.h:1323
    +
    VmaVulkanFunctions::vkBindImageMemory
    PFN_vkBindImageMemory vkBindImageMemory
    Definition: vk_mem_alloc.h:1176
    +
    VmaStatInfo::unusedBytes
    VkDeviceSize unusedBytes
    Total number of bytes occupied by unused ranges.
    Definition: vk_mem_alloc.h:1322
    +
    VmaDefragmentationStats
    Statistics returned by function vmaDefragment().
    Definition: vk_mem_alloc.h:1966
    vmaFreeMemory
    void vmaFreeMemory(VmaAllocator allocator, VmaAllocation allocation)
    Frees memory previously allocated using vmaAllocateMemory(), vmaAllocateMemoryForBuffer(), or vmaAllocateMemoryForImage().
    -
    VmaAllocatorCreateInfo::frameInUseCount
    uint32_t frameInUseCount
    Maximum number of additional frames that are in use at the same time as current frame.
    Definition: vk_mem_alloc.h:1208
    -
    VmaStats::total
    VmaStatInfo total
    Definition: vk_mem_alloc.h:1318
    -
    VmaDefragmentationStats::deviceMemoryBlocksFreed
    uint32_t deviceMemoryBlocksFreed
    Number of empty VkDeviceMemory objects that have been released to the system.
    Definition: vk_mem_alloc.h:1960
    -
    VmaAllocationCreateInfo::flags
    VmaAllocationCreateFlags flags
    Use VmaAllocationCreateFlagBits enum.
    Definition: vk_mem_alloc.h:1478
    -
    VmaDefragmentationInfo::maxBytesToMove
    VkDeviceSize maxBytesToMove
    Maximum total numbers of bytes that can be copied while moving allocations to different places...
    Definition: vk_mem_alloc.h:1943
    -
    VmaVulkanFunctions::vkGetBufferMemoryRequirements
    PFN_vkGetBufferMemoryRequirements vkGetBufferMemoryRequirements
    Definition: vk_mem_alloc.h:1165
    -
    PFN_vmaAllocateDeviceMemoryFunction
    void(VKAPI_PTR * PFN_vmaAllocateDeviceMemoryFunction)(VmaAllocator allocator, uint32_t memoryType, VkDeviceMemory memory, VkDeviceSize size)
    Callback function called after successful vkAllocateMemory.
    Definition: vk_mem_alloc.h:1092
    +
    VmaAllocatorCreateInfo::frameInUseCount
    uint32_t frameInUseCount
    Maximum number of additional frames that are in use at the same time as current frame.
    Definition: vk_mem_alloc.h:1222
    +
    VmaStats::total
    VmaStatInfo total
    Definition: vk_mem_alloc.h:1332
    +
    VmaDefragmentationStats::deviceMemoryBlocksFreed
    uint32_t deviceMemoryBlocksFreed
    Number of empty VkDeviceMemory objects that have been released to the system.
    Definition: vk_mem_alloc.h:1974
    +
    VmaAllocationCreateInfo::flags
    VmaAllocationCreateFlags flags
    Use VmaAllocationCreateFlagBits enum.
    Definition: vk_mem_alloc.h:1492
    +
    VmaDefragmentationInfo::maxBytesToMove
    VkDeviceSize maxBytesToMove
    Maximum total numbers of bytes that can be copied while moving allocations to different places...
    Definition: vk_mem_alloc.h:1957
    +
    VmaVulkanFunctions::vkGetBufferMemoryRequirements
    PFN_vkGetBufferMemoryRequirements vkGetBufferMemoryRequirements
    Definition: vk_mem_alloc.h:1177
    +
    PFN_vmaAllocateDeviceMemoryFunction
    void(VKAPI_PTR * PFN_vmaAllocateDeviceMemoryFunction)(VmaAllocator allocator, uint32_t memoryType, VkDeviceMemory memory, VkDeviceSize size)
    Callback function called after successful vkAllocateMemory.
    Definition: vk_mem_alloc.h:1104
    VmaAllocator
    Represents main object of this library initialized.
    -
    VmaAllocatorCreateInfo::device
    VkDevice device
    Vulkan device.
    Definition: vk_mem_alloc.h:1185
    +
    VmaAllocatorCreateInfo::device
    VkDevice device
    Vulkan device.
    Definition: vk_mem_alloc.h:1199
    vmaBindBufferMemory
    VkResult vmaBindBufferMemory(VmaAllocator allocator, VmaAllocation allocation, VkBuffer buffer)
    Binds buffer to allocation.
    -
    VmaPoolCreateInfo
    Describes parameter of created VmaPool.
    Definition: vk_mem_alloc.h:1603
    -
    VMA_POOL_CREATE_FLAG_BITS_MAX_ENUM
    Definition: vk_mem_alloc.h:1597
    -
    VmaAllocationInfo::size
    VkDeviceSize size
    Size of this allocation, in bytes.
    Definition: vk_mem_alloc.h:1759
    +
    VmaPoolCreateInfo
    Describes parameter of created VmaPool.
    Definition: vk_mem_alloc.h:1617
    +
    VMA_POOL_CREATE_FLAG_BITS_MAX_ENUM
    Definition: vk_mem_alloc.h:1611
    +
    VmaAllocationInfo::size
    VkDeviceSize size
    Size of this allocation, in bytes.
    Definition: vk_mem_alloc.h:1773
    vmaGetMemoryTypeProperties
    void vmaGetMemoryTypeProperties(VmaAllocator allocator, uint32_t memoryTypeIndex, VkMemoryPropertyFlags *pFlags)
    Given Memory Type Index, returns Property Flags of this memory type.
    -
    VmaVulkanFunctions::vkUnmapMemory
    PFN_vkUnmapMemory vkUnmapMemory
    Definition: vk_mem_alloc.h:1162
    -
    VmaAllocationCreateInfo::pUserData
    void * pUserData
    Custom general-purpose pointer that will be stored in VmaAllocation, can be read as VmaAllocationInfo...
    Definition: vk_mem_alloc.h:1515
    -
    VmaPoolCreateInfo::minBlockCount
    size_t minBlockCount
    Minimum number of blocks to be always allocated in this pool, even if they stay empty.
    Definition: vk_mem_alloc.h:1619
    -
    VmaPoolStats::allocationCount
    size_t allocationCount
    Number of VmaAllocation objects created from this pool that were not destroyed or lost...
    Definition: vk_mem_alloc.h:1655
    +
    VmaVulkanFunctions::vkUnmapMemory
    PFN_vkUnmapMemory vkUnmapMemory
    Definition: vk_mem_alloc.h:1174
    +
    VmaAllocationCreateInfo::pUserData
    void * pUserData
    Custom general-purpose pointer that will be stored in VmaAllocation, can be read as VmaAllocationInfo...
    Definition: vk_mem_alloc.h:1529
    +
    VmaPoolCreateInfo::minBlockCount
    size_t minBlockCount
    Minimum number of blocks to be always allocated in this pool, even if they stay empty.
    Definition: vk_mem_alloc.h:1633
    +
    VmaPoolStats::allocationCount
    size_t allocationCount
    Number of VmaAllocation objects created from this pool that were not destroyed or lost...
    Definition: vk_mem_alloc.h:1669
    VmaVulkanFunctions
    struct VmaVulkanFunctions VmaVulkanFunctions
    Pointers to some Vulkan functions - a subset used by the library.
    -
    VMA_ALLOCATOR_CREATE_FLAG_BITS_MAX_ENUM
    Definition: vk_mem_alloc.h:1148
    -
    VmaPoolCreateInfo::memoryTypeIndex
    uint32_t memoryTypeIndex
    Vulkan memory type index to allocate this pool from.
    Definition: vk_mem_alloc.h:1606
    +
    VMA_ALLOCATOR_CREATE_FLAG_BITS_MAX_ENUM
    Definition: vk_mem_alloc.h:1160
    +
    VmaPoolCreateInfo::memoryTypeIndex
    uint32_t memoryTypeIndex
    Vulkan memory type index to allocate this pool from.
    Definition: vk_mem_alloc.h:1620
    vmaFindMemoryTypeIndex
    VkResult vmaFindMemoryTypeIndex(VmaAllocator allocator, uint32_t memoryTypeBits, const VmaAllocationCreateInfo *pAllocationCreateInfo, uint32_t *pMemoryTypeIndex)
    Helps to find memoryTypeIndex, given memoryTypeBits and VmaAllocationCreateInfo.
    -
    VmaMemoryUsage
    VmaMemoryUsage
    Definition: vk_mem_alloc.h:1354
    +
    VmaMemoryUsage
    VmaMemoryUsage
    Definition: vk_mem_alloc.h:1368
    VmaAllocationInfo
    struct VmaAllocationInfo VmaAllocationInfo
    Parameters of VmaAllocation objects, that can be retrieved using function vmaGetAllocationInfo().
    -
    VmaDefragmentationInfo
    Optional configuration parameters to be passed to function vmaDefragment().
    Definition: vk_mem_alloc.h:1938
    +
    VmaDefragmentationInfo
    Optional configuration parameters to be passed to function vmaDefragment().
    Definition: vk_mem_alloc.h:1952
    VmaPoolCreateInfo
    struct VmaPoolCreateInfo VmaPoolCreateInfo
    Describes parameter of created VmaPool.
    vmaDestroyPool
    void vmaDestroyPool(VmaAllocator allocator, VmaPool pool)
    Destroys VmaPool object and frees Vulkan device memory.
    -
    VmaDefragmentationStats::bytesFreed
    VkDeviceSize bytesFreed
    Total number of bytes that have been released to the system by freeing empty VkDeviceMemory objects...
    Definition: vk_mem_alloc.h:1956
    -
    VMA_MEMORY_USAGE_CPU_TO_GPU
    Definition: vk_mem_alloc.h:1393
    -
    VmaAllocationCreateInfo::memoryTypeBits
    uint32_t memoryTypeBits
    Bitmask containing one bit set for every memory type acceptable for this allocation.
    Definition: vk_mem_alloc.h:1502
    -
    VmaVulkanFunctions::vkBindBufferMemory
    PFN_vkBindBufferMemory vkBindBufferMemory
    Definition: vk_mem_alloc.h:1163
    +
    VmaDefragmentationStats::bytesFreed
    VkDeviceSize bytesFreed
    Total number of bytes that have been released to the system by freeing empty VkDeviceMemory objects...
    Definition: vk_mem_alloc.h:1970
    +
    VMA_MEMORY_USAGE_CPU_TO_GPU
    Definition: vk_mem_alloc.h:1407
    +
    VmaAllocationCreateInfo::memoryTypeBits
    uint32_t memoryTypeBits
    Bitmask containing one bit set for every memory type acceptable for this allocation.
    Definition: vk_mem_alloc.h:1516
    +
    VmaVulkanFunctions::vkBindBufferMemory
    PFN_vkBindBufferMemory vkBindBufferMemory
    Definition: vk_mem_alloc.h:1175
    VmaPool
    Represents custom memory pool.
    vmaGetPoolStats
    void vmaGetPoolStats(VmaAllocator allocator, VmaPool pool, VmaPoolStats *pPoolStats)
    Retrieves statistics of existing VmaPool object.
    VmaDefragmentationInfo
    struct VmaDefragmentationInfo VmaDefragmentationInfo
    Optional configuration parameters to be passed to function vmaDefragment().
    -
    VmaStats
    General statistics from current state of Allocator.
    Definition: vk_mem_alloc.h:1314
    -
    PFN_vmaFreeDeviceMemoryFunction
    void(VKAPI_PTR * PFN_vmaFreeDeviceMemoryFunction)(VmaAllocator allocator, uint32_t memoryType, VkDeviceMemory memory, VkDeviceSize size)
    Callback function called before vkFreeMemory.
    Definition: vk_mem_alloc.h:1098
    +
    VmaStats
    General statistics from current state of Allocator.
    Definition: vk_mem_alloc.h:1328
    +
    PFN_vmaFreeDeviceMemoryFunction
    void(VKAPI_PTR * PFN_vmaFreeDeviceMemoryFunction)(VmaAllocator allocator, uint32_t memoryType, VkDeviceMemory memory, VkDeviceSize size)
    Callback function called before vkFreeMemory.
    Definition: vk_mem_alloc.h:1110
    vmaSetAllocationUserData
    void vmaSetAllocationUserData(VmaAllocator allocator, VmaAllocation allocation, void *pUserData)
    Sets pUserData in given allocation to new value.
    vmaCreatePool
    VkResult vmaCreatePool(VmaAllocator allocator, const VmaPoolCreateInfo *pCreateInfo, VmaPool *pPool)
    Allocates Vulkan device memory and creates VmaPool object.
    -
    VmaAllocatorCreateFlagBits
    VmaAllocatorCreateFlagBits
    Flags for created VmaAllocator.
    Definition: vk_mem_alloc.h:1119
    +
    VmaAllocatorCreateFlagBits
    VmaAllocatorCreateFlagBits
    Flags for created VmaAllocator.
    Definition: vk_mem_alloc.h:1131
    vmaBindImageMemory
    VkResult vmaBindImageMemory(VmaAllocator allocator, VmaAllocation allocation, VkImage image)
    Binds image to allocation.
    VmaStatInfo
    struct VmaStatInfo VmaStatInfo
    Calculated statistics of memory usage in entire allocator.
    -
    VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT
    Allocator and all objects created from it will not be synchronized internally, so you must guarantee ...
    Definition: vk_mem_alloc.h:1124
    -
    VmaDefragmentationStats::allocationsMoved
    uint32_t allocationsMoved
    Number of allocations that have been moved to different places.
    Definition: vk_mem_alloc.h:1958
    +
    VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT
    Allocator and all objects created from it will not be synchronized internally, so you must guarantee ...
    Definition: vk_mem_alloc.h:1136
    +
    VmaDefragmentationStats::allocationsMoved
    uint32_t allocationsMoved
    Number of allocations that have been moved to different places.
    Definition: vk_mem_alloc.h:1972
    vmaCreateLostAllocation
    void vmaCreateLostAllocation(VmaAllocator allocator, VmaAllocation *pAllocation)
    Creates new allocation that is in lost state from the beginning.
    -
    VmaAllocationCreateInfo::requiredFlags
    VkMemoryPropertyFlags requiredFlags
    Flags that must be set in a Memory Type chosen for an allocation.
    Definition: vk_mem_alloc.h:1489
    -
    VmaPoolStats::unusedRangeSizeMax
    VkDeviceSize unusedRangeSizeMax
    Size of the largest continuous free memory region.
    Definition: vk_mem_alloc.h:1665
    +
    VmaAllocationCreateInfo::requiredFlags
    VkMemoryPropertyFlags requiredFlags
    Flags that must be set in a Memory Type chosen for an allocation.
    Definition: vk_mem_alloc.h:1503
    +
    VmaPoolStats::unusedRangeSizeMax
    VkDeviceSize unusedRangeSizeMax
    Size of the largest continuous free memory region.
    Definition: vk_mem_alloc.h:1679
    vmaBuildStatsString
    void vmaBuildStatsString(VmaAllocator allocator, char **ppStatsString, VkBool32 detailedMap)
    Builds and returns statistics as string in JSON format.
    -
    VmaVulkanFunctions::vkGetPhysicalDeviceMemoryProperties
    PFN_vkGetPhysicalDeviceMemoryProperties vkGetPhysicalDeviceMemoryProperties
    Definition: vk_mem_alloc.h:1158
    -
    VmaStatInfo
    Calculated statistics of memory usage in entire allocator.
    Definition: vk_mem_alloc.h:1297
    -
    VmaPoolCreateInfo::blockSize
    VkDeviceSize blockSize
    Size of a single VkDeviceMemory block to be allocated as part of this pool, in bytes.
    Definition: vk_mem_alloc.h:1614
    -
    VmaDeviceMemoryCallbacks
    Set of callbacks that the library will call for vkAllocateMemory and vkFreeMemory.
    Definition: vk_mem_alloc.h:1111
    +
    VmaVulkanFunctions::vkGetPhysicalDeviceMemoryProperties
    PFN_vkGetPhysicalDeviceMemoryProperties vkGetPhysicalDeviceMemoryProperties
    Definition: vk_mem_alloc.h:1170
    +
    VmaStatInfo
    Calculated statistics of memory usage in entire allocator.
    Definition: vk_mem_alloc.h:1311
    +
    VmaPoolCreateInfo::blockSize
    VkDeviceSize blockSize
    Size of a single VkDeviceMemory block to be allocated as part of this pool, in bytes.
    Definition: vk_mem_alloc.h:1628
    +
    VmaDeviceMemoryCallbacks
    Set of callbacks that the library will call for vkAllocateMemory and vkFreeMemory.
    Definition: vk_mem_alloc.h:1123
    vmaCreateBuffer
    VkResult vmaCreateBuffer(VmaAllocator allocator, const VkBufferCreateInfo *pBufferCreateInfo, const VmaAllocationCreateInfo *pAllocationCreateInfo, VkBuffer *pBuffer, VmaAllocation *pAllocation, VmaAllocationInfo *pAllocationInfo)
    -
    VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT
    Definition: vk_mem_alloc.h:1463
    -
    VmaStatInfo::unusedRangeSizeMin
    VkDeviceSize unusedRangeSizeMin
    Definition: vk_mem_alloc.h:1310
    -
    VmaDeviceMemoryCallbacks::pfnFree
    PFN_vmaFreeDeviceMemoryFunction pfnFree
    Optional, can be null.
    Definition: vk_mem_alloc.h:1115
    -
    VmaPoolCreateInfo::flags
    VmaPoolCreateFlags flags
    Use combination of VmaPoolCreateFlagBits.
    Definition: vk_mem_alloc.h:1609
    -
    VMA_MEMORY_USAGE_GPU_TO_CPU
    Definition: vk_mem_alloc.h:1402
    +
    VMA_ALLOCATION_CREATE_CAN_MAKE_OTHER_LOST_BIT
    Definition: vk_mem_alloc.h:1477
    +
    VmaStatInfo::unusedRangeSizeMin
    VkDeviceSize unusedRangeSizeMin
    Definition: vk_mem_alloc.h:1324
    +
    VmaDeviceMemoryCallbacks::pfnFree
    PFN_vmaFreeDeviceMemoryFunction pfnFree
    Optional, can be null.
    Definition: vk_mem_alloc.h:1127
    +
    VmaPoolCreateInfo::flags
    VmaPoolCreateFlags flags
    Use combination of VmaPoolCreateFlagBits.
    Definition: vk_mem_alloc.h:1623
    +
    VMA_MEMORY_USAGE_GPU_TO_CPU
    Definition: vk_mem_alloc.h:1416
    VmaPoolStats
    struct VmaPoolStats VmaPoolStats
    Describes parameter of existing VmaPool.
    vmaCreateImage
    VkResult vmaCreateImage(VmaAllocator allocator, const VkImageCreateInfo *pImageCreateInfo, const VmaAllocationCreateInfo *pAllocationCreateInfo, VkImage *pImage, VmaAllocation *pAllocation, VmaAllocationInfo *pAllocationInfo)
    Function similar to vmaCreateBuffer().
    -
    VmaAllocationCreateInfo::usage
    VmaMemoryUsage usage
    Intended usage of memory.
    Definition: vk_mem_alloc.h:1484
    -
    VmaAllocationCreateInfo
    Definition: vk_mem_alloc.h:1475
    +
    VmaAllocationCreateInfo::usage
    VmaMemoryUsage usage
    Intended usage of memory.
    Definition: vk_mem_alloc.h:1498
    +
    VmaAllocationCreateInfo
    Definition: vk_mem_alloc.h:1489
    vmaFindMemoryTypeIndexForImageInfo
    VkResult vmaFindMemoryTypeIndexForImageInfo(VmaAllocator allocator, const VkImageCreateInfo *pImageCreateInfo, const VmaAllocationCreateInfo *pAllocationCreateInfo, uint32_t *pMemoryTypeIndex)
    Helps to find memoryTypeIndex, given VkImageCreateInfo and VmaAllocationCreateInfo.
    -
    VmaStatInfo::blockCount
    uint32_t blockCount
    Number of VkDeviceMemory Vulkan memory blocks allocated.
    Definition: vk_mem_alloc.h:1300
    -
    VmaVulkanFunctions::vkFreeMemory
    PFN_vkFreeMemory vkFreeMemory
    Definition: vk_mem_alloc.h:1160
    -
    VmaPoolCreateInfo::maxBlockCount
    size_t maxBlockCount
    Maximum number of blocks that can be allocated in this pool. Optional.
    Definition: vk_mem_alloc.h:1627
    -
    VmaAllocatorCreateInfo::pDeviceMemoryCallbacks
    const VmaDeviceMemoryCallbacks * pDeviceMemoryCallbacks
    Informative callbacks for vkAllocateMemory, vkFreeMemory. Optional.
    Definition: vk_mem_alloc.h:1194
    -
    VmaPoolStats::unusedRangeCount
    size_t unusedRangeCount
    Number of continuous memory ranges in the pool not used by any VmaAllocation.
    Definition: vk_mem_alloc.h:1658
    -
    VmaAllocationCreateFlags
    VkFlags VmaAllocationCreateFlags
    Definition: vk_mem_alloc.h:1473
    -
    VmaAllocationCreateInfo::pool
    VmaPool pool
    Pool that this allocation should be created in.
    Definition: vk_mem_alloc.h:1508
    +
    VmaStatInfo::blockCount
    uint32_t blockCount
    Number of VkDeviceMemory Vulkan memory blocks allocated.
    Definition: vk_mem_alloc.h:1314
    +
    VmaVulkanFunctions::vkFreeMemory
    PFN_vkFreeMemory vkFreeMemory
    Definition: vk_mem_alloc.h:1172
    +
    VmaPoolCreateInfo::maxBlockCount
    size_t maxBlockCount
    Maximum number of blocks that can be allocated in this pool. Optional.
    Definition: vk_mem_alloc.h:1641
    +
    VmaAllocatorCreateInfo::pDeviceMemoryCallbacks
    const VmaDeviceMemoryCallbacks * pDeviceMemoryCallbacks
    Informative callbacks for vkAllocateMemory, vkFreeMemory. Optional.
    Definition: vk_mem_alloc.h:1208
    +
    VmaPoolStats::unusedRangeCount
    size_t unusedRangeCount
    Number of continuous memory ranges in the pool not used by any VmaAllocation.
    Definition: vk_mem_alloc.h:1672
    +
    VmaAllocationCreateFlags
    VkFlags VmaAllocationCreateFlags
    Definition: vk_mem_alloc.h:1487
    +
    VmaAllocationCreateInfo::pool
    VmaPool pool
    Pool that this allocation should be created in.
    Definition: vk_mem_alloc.h:1522
    vmaGetMemoryProperties
    void vmaGetMemoryProperties(VmaAllocator allocator, const VkPhysicalDeviceMemoryProperties **ppPhysicalDeviceMemoryProperties)
    -
    VmaAllocatorCreateInfo::pHeapSizeLimit
    const VkDeviceSize * pHeapSizeLimit
    Either null or a pointer to an array of limits on maximum number of bytes that can be allocated out o...
    Definition: vk_mem_alloc.h:1232
    -
    VmaStats::memoryType
    VmaStatInfo memoryType[VK_MAX_MEMORY_TYPES]
    Definition: vk_mem_alloc.h:1316
    -
    VMA_ALLOCATION_CREATE_MAPPED_BIT
    Set this flag to use a memory that will be persistently mapped and retrieve pointer to it...
    Definition: vk_mem_alloc.h:1443
    -
    VmaStatInfo::allocationSizeMin
    VkDeviceSize allocationSizeMin
    Definition: vk_mem_alloc.h:1309
    +
    VmaAllocatorCreateInfo::pHeapSizeLimit
    const VkDeviceSize * pHeapSizeLimit
    Either null or a pointer to an array of limits on maximum number of bytes that can be allocated out o...
    Definition: vk_mem_alloc.h:1246
    +
    VmaStats::memoryType
    VmaStatInfo memoryType[VK_MAX_MEMORY_TYPES]
    Definition: vk_mem_alloc.h:1330
    +
    VMA_ALLOCATION_CREATE_MAPPED_BIT
    Set this flag to use a memory that will be persistently mapped and retrieve pointer to it...
    Definition: vk_mem_alloc.h:1457
    +
    VmaStatInfo::allocationSizeMin
    VkDeviceSize allocationSizeMin
    Definition: vk_mem_alloc.h:1323
    vmaFindMemoryTypeIndexForBufferInfo
    VkResult vmaFindMemoryTypeIndexForBufferInfo(VmaAllocator allocator, const VkBufferCreateInfo *pBufferCreateInfo, const VmaAllocationCreateInfo *pAllocationCreateInfo, uint32_t *pMemoryTypeIndex)
    Helps to find memoryTypeIndex, given VkBufferCreateInfo and VmaAllocationCreateInfo.
    -
    VmaVulkanFunctions::vkCreateImage
    PFN_vkCreateImage vkCreateImage
    Definition: vk_mem_alloc.h:1169
    -
    VmaDeviceMemoryCallbacks::pfnAllocate
    PFN_vmaAllocateDeviceMemoryFunction pfnAllocate
    Optional, can be null.
    Definition: vk_mem_alloc.h:1113
    -
    VmaVulkanFunctions::vkDestroyBuffer
    PFN_vkDestroyBuffer vkDestroyBuffer
    Definition: vk_mem_alloc.h:1168
    +
    VmaVulkanFunctions::vkCreateImage
    PFN_vkCreateImage vkCreateImage
    Definition: vk_mem_alloc.h:1181
    +
    VmaDeviceMemoryCallbacks::pfnAllocate
    PFN_vmaAllocateDeviceMemoryFunction pfnAllocate
    Optional, can be null.
    Definition: vk_mem_alloc.h:1125
    +
    VmaVulkanFunctions::vkDestroyBuffer
    PFN_vkDestroyBuffer vkDestroyBuffer
    Definition: vk_mem_alloc.h:1180
    vmaMapMemory
    VkResult vmaMapMemory(VmaAllocator allocator, VmaAllocation allocation, void **ppData)
    Maps memory represented by given allocation and returns pointer to it.
    -
    VmaPoolCreateInfo::frameInUseCount
    uint32_t frameInUseCount
    Maximum number of additional frames that are in use at the same time as current frame.
    Definition: vk_mem_alloc.h:1641
    +
    VmaPoolCreateInfo::frameInUseCount
    uint32_t frameInUseCount
    Maximum number of additional frames that are in use at the same time as current frame.
    Definition: vk_mem_alloc.h:1655
    vmaAllocateMemoryForImage
    VkResult vmaAllocateMemoryForImage(VmaAllocator allocator, VkImage image, const VmaAllocationCreateInfo *pCreateInfo, VmaAllocation *pAllocation, VmaAllocationInfo *pAllocationInfo)
    Function similar to vmaAllocateMemoryForBuffer().
    VmaAllocatorCreateInfo
    struct VmaAllocatorCreateInfo VmaAllocatorCreateInfo
    Description of a Allocator to be created.
    -
    VmaAllocationInfo::pUserData
    void * pUserData
    Custom general-purpose pointer that was passed as VmaAllocationCreateInfo::pUserData or set using vma...
    Definition: vk_mem_alloc.h:1773
    -
    VmaAllocatorCreateInfo::preferredLargeHeapBlockSize
    VkDeviceSize preferredLargeHeapBlockSize
    Preferred size of a single VkDeviceMemory block to be allocated from large heaps > 1 GiB...
    Definition: vk_mem_alloc.h:1188
    -
    VmaStatInfo::allocationSizeAvg
    VkDeviceSize allocationSizeAvg
    Definition: vk_mem_alloc.h:1309
    -
    VmaStatInfo::usedBytes
    VkDeviceSize usedBytes
    Total number of bytes occupied by all allocations.
    Definition: vk_mem_alloc.h:1306
    +
    VmaAllocationInfo::pUserData
    void * pUserData
    Custom general-purpose pointer that was passed as VmaAllocationCreateInfo::pUserData or set using vma...
    Definition: vk_mem_alloc.h:1787
    +
    VmaAllocatorCreateInfo::preferredLargeHeapBlockSize
    VkDeviceSize preferredLargeHeapBlockSize
    Preferred size of a single VkDeviceMemory block to be allocated from large heaps > 1 GiB...
    Definition: vk_mem_alloc.h:1202
    +
    VmaStatInfo::allocationSizeAvg
    VkDeviceSize allocationSizeAvg
    Definition: vk_mem_alloc.h:1323
    +
    VmaStatInfo::usedBytes
    VkDeviceSize usedBytes
    Total number of bytes occupied by all allocations.
    Definition: vk_mem_alloc.h:1320
    VmaDeviceMemoryCallbacks
    struct VmaDeviceMemoryCallbacks VmaDeviceMemoryCallbacks
    Set of callbacks that the library will call for vkAllocateMemory and vkFreeMemory.
    -
    VmaPoolStats
    Describes parameter of existing VmaPool.
    Definition: vk_mem_alloc.h:1646
    -
    VmaAllocationInfo::offset
    VkDeviceSize offset
    Offset into deviceMemory object to the beginning of this allocation, in bytes. (deviceMemory, offset) pair is unique to this allocation.
    Definition: vk_mem_alloc.h:1754
    -
    VMA_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM
    Definition: vk_mem_alloc.h:1471
    -
    VmaDefragmentationStats::bytesMoved
    VkDeviceSize bytesMoved
    Total number of bytes that have been copied while moving allocations to different places...
    Definition: vk_mem_alloc.h:1954
    -
    VmaVulkanFunctions
    Pointers to some Vulkan functions - a subset used by the library.
    Definition: vk_mem_alloc.h:1156
    +
    VmaPoolStats
    Describes parameter of existing VmaPool.
    Definition: vk_mem_alloc.h:1660
    +
    VmaAllocationInfo::offset
    VkDeviceSize offset
    Offset into deviceMemory object to the beginning of this allocation, in bytes. (deviceMemory, offset) pair is unique to this allocation.
    Definition: vk_mem_alloc.h:1768
    +
    VMA_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM
    Definition: vk_mem_alloc.h:1485
    +
    VmaDefragmentationStats::bytesMoved
    VkDeviceSize bytesMoved
    Total number of bytes that have been copied while moving allocations to different places...
    Definition: vk_mem_alloc.h:1968
    +
    VmaVulkanFunctions
    Pointers to some Vulkan functions - a subset used by the library.
    Definition: vk_mem_alloc.h:1168
    vmaCreateAllocator
    VkResult vmaCreateAllocator(const VmaAllocatorCreateInfo *pCreateInfo, VmaAllocator *pAllocator)
    Creates Allocator object.
    -
    VmaVulkanFunctions::vkGetBufferMemoryRequirements2KHR
    PFN_vkGetBufferMemoryRequirements2KHR vkGetBufferMemoryRequirements2KHR
    Definition: vk_mem_alloc.h:1171
    -
    VmaStatInfo::unusedRangeCount
    uint32_t unusedRangeCount
    Number of free ranges of memory between allocations.
    Definition: vk_mem_alloc.h:1304
    -
    VMA_MEMORY_USAGE_UNKNOWN
    Definition: vk_mem_alloc.h:1359
    -
    VmaPoolCreateFlags
    VkFlags VmaPoolCreateFlags
    Definition: vk_mem_alloc.h:1599
    +
    VmaStatInfo::unusedRangeCount
    uint32_t unusedRangeCount
    Number of free ranges of memory between allocations.
    Definition: vk_mem_alloc.h:1318
    +
    VMA_MEMORY_USAGE_UNKNOWN
    Definition: vk_mem_alloc.h:1373
    +
    VmaPoolCreateFlags
    VkFlags VmaPoolCreateFlags
    Definition: vk_mem_alloc.h:1613
    vmaGetPhysicalDeviceProperties
    void vmaGetPhysicalDeviceProperties(VmaAllocator allocator, const VkPhysicalDeviceProperties **ppPhysicalDeviceProperties)
    -
    VmaStatInfo::allocationCount
    uint32_t allocationCount
    Number of VmaAllocation allocation objects allocated.
    Definition: vk_mem_alloc.h:1302
    -
    VmaVulkanFunctions::vkGetImageMemoryRequirements
    PFN_vkGetImageMemoryRequirements vkGetImageMemoryRequirements
    Definition: vk_mem_alloc.h:1166
    -
    VmaVulkanFunctions::vkDestroyImage
    PFN_vkDestroyImage vkDestroyImage
    Definition: vk_mem_alloc.h:1170
    -
    VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT
    Set this flag to only try to allocate from existing VkDeviceMemory blocks and never create new such b...
    Definition: vk_mem_alloc.h:1430
    -
    VMA_MEMORY_USAGE_CPU_ONLY
    Definition: vk_mem_alloc.h:1386
    -
    VmaAllocationInfo::pMappedData
    void * pMappedData
    Pointer to the beginning of this allocation as mapped data.
    Definition: vk_mem_alloc.h:1768
    +
    VmaStatInfo::allocationCount
    uint32_t allocationCount
    Number of VmaAllocation allocation objects allocated.
    Definition: vk_mem_alloc.h:1316
    +
    VmaVulkanFunctions::vkGetImageMemoryRequirements
    PFN_vkGetImageMemoryRequirements vkGetImageMemoryRequirements
    Definition: vk_mem_alloc.h:1178
    +
    VmaVulkanFunctions::vkDestroyImage
    PFN_vkDestroyImage vkDestroyImage
    Definition: vk_mem_alloc.h:1182
    +
    VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT
    Set this flag to only try to allocate from existing VkDeviceMemory blocks and never create new such b...
    Definition: vk_mem_alloc.h:1444
    +
    VMA_MEMORY_USAGE_CPU_ONLY
    Definition: vk_mem_alloc.h:1400
    +
    VmaAllocationInfo::pMappedData
    void * pMappedData
    Pointer to the beginning of this allocation as mapped data.
    Definition: vk_mem_alloc.h:1782
    vmaDestroyImage
    void vmaDestroyImage(VmaAllocator allocator, VkImage image, VmaAllocation allocation)
    Destroys Vulkan image and frees allocated memory.
    -
    VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT
    Enables usage of VK_KHR_dedicated_allocation extension.
    Definition: vk_mem_alloc.h:1146
    +
    VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT
    Enables usage of VK_KHR_dedicated_allocation extension.
    Definition: vk_mem_alloc.h:1158
    VmaDefragmentationStats
    struct VmaDefragmentationStats VmaDefragmentationStats
    Statistics returned by function vmaDefragment().
    -
    VmaVulkanFunctions::vkAllocateMemory
    PFN_vkAllocateMemory vkAllocateMemory
    Definition: vk_mem_alloc.h:1159
    -
    VmaAllocationInfo
    Parameters of VmaAllocation objects, that can be retrieved using function vmaGetAllocationInfo().
    Definition: vk_mem_alloc.h:1735
    +
    VmaVulkanFunctions::vkAllocateMemory
    PFN_vkAllocateMemory vkAllocateMemory
    Definition: vk_mem_alloc.h:1171
    +
    VmaAllocationInfo
    Parameters of VmaAllocation objects, that can be retrieved using function vmaGetAllocationInfo().
    Definition: vk_mem_alloc.h:1749
    vmaAllocateMemory
    VkResult vmaAllocateMemory(VmaAllocator allocator, const VkMemoryRequirements *pVkMemoryRequirements, const VmaAllocationCreateInfo *pCreateInfo, VmaAllocation *pAllocation, VmaAllocationInfo *pAllocationInfo)
    General purpose memory allocation.
    vmaSetCurrentFrameIndex
    void vmaSetCurrentFrameIndex(VmaAllocator allocator, uint32_t frameIndex)
    Sets index of the current frame.
    VmaAllocationCreateInfo
    struct VmaAllocationCreateInfo VmaAllocationCreateInfo
    vmaAllocateMemoryForBuffer
    VkResult vmaAllocateMemoryForBuffer(VmaAllocator allocator, VkBuffer buffer, const VmaAllocationCreateInfo *pCreateInfo, VmaAllocation *pAllocation, VmaAllocationInfo *pAllocationInfo)
    -
    VmaPoolCreateFlagBits
    VmaPoolCreateFlagBits
    Flags to be passed as VmaPoolCreateInfo::flags.
    Definition: vk_mem_alloc.h:1577
    -
    VmaStatInfo::unusedRangeSizeAvg
    VkDeviceSize unusedRangeSizeAvg
    Definition: vk_mem_alloc.h:1310
    +
    VmaPoolCreateFlagBits
    VmaPoolCreateFlagBits
    Flags to be passed as VmaPoolCreateInfo::flags.
    Definition: vk_mem_alloc.h:1591
    +
    VmaStatInfo::unusedRangeSizeAvg
    VkDeviceSize unusedRangeSizeAvg
    Definition: vk_mem_alloc.h:1324
    vmaTouchAllocation
    VkBool32 vmaTouchAllocation(VmaAllocator allocator, VmaAllocation allocation)
    Returns VK_TRUE if allocation is not lost and atomically marks it as used in current frame...
    -
    VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT
    Definition: vk_mem_alloc.h:1469
    -
    VmaStats::memoryHeap
    VmaStatInfo memoryHeap[VK_MAX_MEMORY_HEAPS]
    Definition: vk_mem_alloc.h:1317
    +
    VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT
    Definition: vk_mem_alloc.h:1483
    +
    VmaStats::memoryHeap
    VmaStatInfo memoryHeap[VK_MAX_MEMORY_HEAPS]
    Definition: vk_mem_alloc.h:1331
    vmaDestroyBuffer
    void vmaDestroyBuffer(VmaAllocator allocator, VkBuffer buffer, VmaAllocation allocation)
    Destroys Vulkan buffer and frees allocated memory.
    -
    VmaPoolStats::unusedSize
    VkDeviceSize unusedSize
    Total number of bytes in the pool not used by any VmaAllocation.
    Definition: vk_mem_alloc.h:1652
    -
    VmaStatInfo::unusedRangeSizeMax
    VkDeviceSize unusedRangeSizeMax
    Definition: vk_mem_alloc.h:1310
    -
    VmaAllocationInfo::memoryType
    uint32_t memoryType
    Memory type index that this allocation was allocated from.
    Definition: vk_mem_alloc.h:1740
    +
    VmaPoolStats::unusedSize
    VkDeviceSize unusedSize
    Total number of bytes in the pool not used by any VmaAllocation.
    Definition: vk_mem_alloc.h:1666
    +
    VmaStatInfo::unusedRangeSizeMax
    VkDeviceSize unusedRangeSizeMax
    Definition: vk_mem_alloc.h:1324
    +
    VmaAllocationInfo::memoryType
    uint32_t memoryType
    Memory type index that this allocation was allocated from.
    Definition: vk_mem_alloc.h:1754
    diff --git a/src/vk_mem_alloc.h b/src/vk_mem_alloc.h index 2b48da8..94eb078 100644 --- a/src/vk_mem_alloc.h +++ b/src/vk_mem_alloc.h @@ -932,7 +932,7 @@ solutions are possible: You should take some measurements to decide which option is faster in case of your specific resource. -If you don't want to specialize your code for specific types of GPUs, yon can still make +If you don't want to specialize your code for specific types of GPUs, you can still make an simple optimization for cases when your resource ends up in mappable memory to use it directly in this case instead of creating CPU-side staging copy. For details see [Finding out if memory is mappable](@ref memory_mapping_finding_if_memory_mappable). @@ -1070,6 +1070,10 @@ Features deliberately excluded from the scope of this library: - Data transfer - issuing commands that transfer data between buffers or images, any usage of `VkCommandList` or `VkCommandQueue` and related synchronization is responsibility of the user. +- Allocations for imported/exported external memory. They tend to require + explicit memory type index and dedicated allocation anyway, so they don't + interact with main features of this library. Such special purpose allocations + should be made manually, using `vkCreateBuffer()` and `vkAllocateMemory()`. - Support for any programming languages other than C/C++. Bindings to other languages are welcomed as external projects. @@ -1377,7 +1381,7 @@ typedef enum VmaMemoryUsage - Resources written and read by device, e.g. images used as attachments. - Resources transferred from host once (immutable) or infrequently and read by device multiple times, e.g. textures to be sampled, vertex buffers, uniform - (constant) buffers, and majority of other types of resources used by device. + (constant) buffers, and majority of other types of resources used on GPU. Allocation may still end up in `HOST_VISIBLE` memory on some implementations. In such case, you are free to map it. @@ -1386,9 +1390,9 @@ typedef enum VmaMemoryUsage VMA_MEMORY_USAGE_GPU_ONLY = 1, /** Memory will be mappable on host. It usually means CPU (system) memory. - Resources created in this pool may still be accessible to the device, but access to them can be slower. Guarantees to be `HOST_VISIBLE` and `HOST_COHERENT`. - CPU read may be uncached. + CPU access is typically uncached. Writes may be write-combined. + Resources created in this pool may still be accessible to the device, but access to them can be slow. It is roughly equivalent of `D3D12_HEAP_TYPE_UPLOAD`. Usage: Staging copy of resources used as transfer source. @@ -1396,7 +1400,7 @@ typedef enum VmaMemoryUsage VMA_MEMORY_USAGE_CPU_ONLY = 2, /** Memory that is both mappable on host (guarantees to be `HOST_VISIBLE`) and preferably fast to access by GPU. - CPU reads may be uncached and very slow. + CPU access is typically uncached. Writes may be write-combined. Usage: Resources written frequently by host (dynamic), read by device. E.g. textures, vertex buffers, uniform buffers updated every frame or every draw call. */