Abstracted voxel metadata so internals no longer depends on Variant

doc/source/specs/block_format_v3.md

@@ -1,6 +1,9 @@
 Voxel block format
 ====================
 
+!!! warn
+    This document is about an old version of the format. You may check the most recent version.
+
 Version: 3
 
 This page describes the binary format used by default in this module to serialize voxel blocks to files, network or databases.

doc/source/specs/block_format_v4.md

@@ -0,0 +1,130 @@
+Voxel block format
+====================
+
+Version: 4
+
+This page describes the binary format used by default in this module to serialize voxel blocks to files, network or databases.
+
+### Changes from version 3
+
+- Metadata uses a new format which no longer depends on Godot Engine.
+
+
+Specification
+----------------
+
+### Endianess
+
+By default, little-endian.
+
+### Compressed container
+
+A block is usually serialized within a compressed data container.
+This is the format provided by the `VoxelBlockSerializer` utility class. If you don't use compression, the layout will correspond to `BlockData` described in the next listing, and won't have this wrapper.
+See [Compressed container format](#compressed-container) for specification.
+
+### Block format
+
+It starts with version number `4` in one byte, then some info and the actual voxels. Optionally, it is followed by custom metadata.
+
+!!! note
+    The size and formats are present to make the format standalone. When used within a chunked container like region files, it is recommended to check if they match the format expected for the volume as a whole.
+
+```
+BlockData
+- version: uint8_t
+- size_x: uint16_t
+- size_y: uint16_t
+- size_z: uint16_t
+- channels[8]
+- metadata*
+- epilogue
+```
+
+### Channels
+
+Block data starts with exactly 8 channels one after the other, each with the following structure:
+
+```
+Channel
+- format: uint8_t (low nibble = compression, high nibble = depth)
+- data
+```
+
+`format` contains both compression and bit depth, respectively known as `VoxelBuffer::Compression` and `VoxelBuffer::Depth` enums. The low nibble contains compression, and the high nibble contains depth. Depending on those values, `data` will be different.
+
+Depth can be 0 (8-bit), 1 (16-bit), 2 (32-bit) or 3 (64-bit).
+
+If compression is `COMPRESSION_NONE` (0), `data` will be an array of N*S bytes, where N is the number of voxels inside a block, multiplied by the number of bytes corresponding to the bit depth. For example, a block of size 16x16x16 and a channel of 32-bit depth will have `16*16*16*4` bytes to load from the file into this channel.
+The 3D indexing of that data is in order `ZXY`.
+
+If compression is `COMPRESSION_UNIFORM` (1), the data will be a single voxel value, which means all voxels in the block have that same value. Unused channels will always use this mode. The value spans the same number of bytes defined by the depth.
+
+Other compression values are invalid.
+
+#### SDF channel
+
+The second channel (at index 1) is used for SDF data. If depth is 8 or 16 bits, it may contain fixed-point values encoded as `inorm8` or `inorm16`. This is numbers in the range [-1..1].
+
+To obtain a `float` from an `int8`, use `max(i / 127, -1.f)`.
+To obtain a `float` from an `int16`, use `max(i / 32767, -1.f)`.
+
+For 32-bit depth, regular `float` are used.
+For 64-bit depth, regular `double` are used.
+
+### Metadata
+
+After all channels information, block data can contain metadata information. Blocks that don't contain any will only have a fixed amount of bytes left (from the epilogue) before reaching the size of the total data to read. If there is more, the block contains metadata.
+
+```
+Metadata
+- metadata_size: uint32_t
+- block_metadata: MetadataItem
+- voxel_metadata: VoxelMetadataItem[*]
+
+VoxelMetadataItem
+- x: uint16_t
+- y: uint16_t
+- z: uint16_t
+- metadata: MetadataItem
+```
+
+It starts with one 32-bit unsigned integer representing the total size of all metadata there is to read. That data comes in two groups: one for the whole block, and a list that associates one per voxel (not all voxels have metadata).
+
+Each metadata item uses the following format:
+
+```
+MetadataItem
+- type: uint8_t
+- data
+```
+
+It starts with a `type` header, followed by data depending on that type.
+
+- If `type` is `0`, the item is empty and there is no `data` to read.
+- If `type` is `1`, it is followed by 8 bytes (`uint64_t`).
+- If `type` is `32`, it is followed by a Godot Engine `Variant`, encoded using the `encode_variant` function. This is only available when using Godot Engine.
+- If `type` is greater than `32`, the following data is application-defined. The application usually knows which data corresponds to that type and defines how to serialize and deserialize it.
+
+The meaning of metadata is application-defined. Two games using different metadata are not expected to be compatible.
+
+
+### Epilogue
+
+At the very end, block data finishes with a sequence of 4 bytes, which once read into a `uint32_t` integer must match the value `0x900df00d`. If that condition isn't fulfilled, the block must be assumed corrupted.
+
+!!! note
+    On little-endian architectures (like desktop), binary editors will not show the epilogue as `0x900df00d`, but as `0x0df00d90` instead.
+
+
+Current Issues
+----------------
+
+### Endianess
+
+The format is intented to use little-endian, however the implementation of the engine does not fully guarantee this.
+
+Godot's `encode_variant` doesn't seem to care about endianess across architectures, so it's possible it becomes a problem in the future and gets changed to a custom format.
+The implementation of block channels with depth greater than 8-bit currently doesn't consider this either. This might be refined in a later iteration.
+
+This will become important to address if voxel games require communication between mobile and desktop.

edition/funcs.h

@@ -1,5 +1,5 @@
-#ifndef EDITION_FUNCS_H
-#define EDITION_FUNCS_H
+#ifndef VOXEL_EDITION_FUNCS_H
+#define VOXEL_EDITION_FUNCS_H
 
 #include "../storage/funcs.h"
 #include "../util/fixed_array.h"
@@ -110,4 +110,4 @@ inline void blend_texture_packed_u16(
 
 } // namespace zylann::voxel
 
-#endif // EDITION_FUNCS_H
+#endif // VOXEL_EDITION_FUNCS_H

edition/voxel_tool_buffer.cpp

@@ -64,12 +64,12 @@ void VoxelToolBuffer::_post_edit(const Box3i &box) {
 
 void VoxelToolBuffer::set_voxel_metadata(Vector3i pos, Variant meta) {
 	ERR_FAIL_COND(_buffer.is_null());
-	_buffer->get_buffer().set_voxel_metadata(pos, meta);
+	_buffer->set_voxel_metadata(pos, meta);
 }
 
 Variant VoxelToolBuffer::get_voxel_metadata(Vector3i pos) const {
 	ERR_FAIL_COND_V(_buffer.is_null(), Variant());
-	return _buffer->get_buffer().get_voxel_metadata(pos);
+	return _buffer->get_voxel_metadata(pos);
 }
 
 void VoxelToolBuffer::paste(
@@ -116,7 +116,7 @@ void VoxelToolBuffer::paste(
 						dst.set_voxel(v, x, y, z, channel_index);
 
 						// Overwrite previous metadata
-						dst.set_voxel_metadata(Vector3i(x, y, z), Variant());
+						dst.erase_voxel_metadata(Vector3i(x, y, z));
 					}
 				}
 			}

edition/voxel_tool_terrain.cpp

@@ -2,6 +2,7 @@
 #include "../meshers/blocky/voxel_mesher_blocky.h"
 #include "../meshers/cubes/voxel_mesher_cubes.h"
 #include "../storage/voxel_buffer_gd.h"
+#include "../storage/voxel_metadata_variant.h"
 #include "../terrain/fixed_lod/voxel_terrain.h"
 #include "../util/godot/funcs.h"
 #include "../util/voxel_raycast.h"
@@ -211,7 +212,9 @@ void VoxelToolTerrain::set_voxel_metadata(Vector3i pos, Variant meta) {
 	VoxelDataBlock *block = map.get_block(map.voxel_to_block(pos));
 	ERR_FAIL_COND_MSG(block == nullptr, "Area not editable");
 	RWLockWrite lock(block->get_voxels().get_lock());
-	block->get_voxels().set_voxel_metadata(map.to_local(pos), meta);
+	VoxelMetadata *meta_storage = block->get_voxels().get_or_create_voxel_metadata(map.to_local(pos));
+	ERR_FAIL_COND(meta_storage == nullptr);
+	gd::set_as_variant(*meta_storage, meta);
 }
 
 Variant VoxelToolTerrain::get_voxel_metadata(Vector3i pos) const {
@@ -220,7 +223,11 @@ Variant VoxelToolTerrain::get_voxel_metadata(Vector3i pos) const {
 	VoxelDataBlock *block = map.get_block(map.voxel_to_block(pos));
 	ERR_FAIL_COND_V_MSG(block == nullptr, Variant(), "Area not editable");
 	RWLockRead lock(block->get_voxels().get_lock());
-	return block->get_voxels_const().get_voxel_metadata(map.to_local(pos));
+	const VoxelMetadata *meta = block->get_voxels_const().get_voxel_metadata(map.to_local(pos));
+	if (meta == nullptr) {
+		return Variant();
+	}
+	return gd::get_as_variant(*meta);
 }
 
 void VoxelToolTerrain::run_blocky_random_tick_static(VoxelDataMap &map, Box3i voxel_box, const VoxelBlockyLibrary &lib,
@@ -389,9 +396,10 @@ void VoxelToolTerrain::for_each_voxel_metadata_in_area(AABB voxel_area, const Ca
 		// TODO Worth it locking blocks for metadata?
 
 		block->get_voxels().for_each_voxel_metadata_in_area(
-				rel_voxel_box, [&callback, block_origin](Vector3i rel_pos, Variant meta) {
+				rel_voxel_box, [&callback, block_origin](Vector3i rel_pos, const VoxelMetadata &meta) {
+					Variant v = gd::get_as_variant(meta);
 					const Variant key = rel_pos + block_origin;
-					const Variant *args[2] = { &key, &meta };
+					const Variant *args[2] = { &key, &v };
 					Callable::CallError err;
 					Variant retval; // We don't care about the return value, Callable API requires it
 					callback.call(args, 2, retval, err);

register_types.cpp

@@ -21,6 +21,7 @@
 #include "server/voxel_server_gd.h"
 #include "storage/voxel_buffer_gd.h"
 #include "storage/voxel_memory_pool.h"
+#include "storage/voxel_metadata_variant.h"
 #include "streams/region/voxel_stream_region_files.h"
 #include "streams/sqlite/voxel_stream_sqlite.h"
 #include "streams/vox_loader.h"
@@ -77,6 +78,8 @@ void register_voxel_types() {
 
 	Engine::get_singleton()->add_singleton(Engine::Singleton("VoxelServer", gd::VoxelServer::get_singleton()));
 
+	VoxelMetadataFactory::get_singleton().add_constructor_by_type<gd::VoxelMetadataVariant>(gd::METADATA_TYPE_VARIANT);
+
 	// TODO Can I prevent users from instancing it? is "register_virtual_class" correct for a class that's not abstract?
 	ClassDB::register_class<gd::VoxelServer>();
 

storage/voxel_buffer_gd.cpp

@@ -1,6 +1,7 @@
 #include "voxel_buffer_gd.h"
 #include "../edition/voxel_tool_buffer.h"
 #include "../util/memory.h"
+#include "voxel_metadata_variant.h"
 
 #include <core/io/image.h>
 
@@ -105,19 +106,42 @@ VoxelBuffer::Depth VoxelBuffer::get_channel_depth(unsigned int channel_index) co
 	return VoxelBuffer::Depth(_buffer->get_channel_depth(channel_index));
 }
 
+Variant VoxelBuffer::get_block_metadata() const {
+	return get_as_variant(_buffer->get_block_metadata());
+}
+
 void VoxelBuffer::set_block_metadata(Variant meta) {
-	_buffer->set_block_metadata(meta);
+	set_as_variant(_buffer->get_block_metadata(), meta);
+}
+
+Variant VoxelBuffer::get_voxel_metadata(Vector3i pos) const {
+	VoxelMetadata *meta = _buffer->get_voxel_metadata(pos);
+	if (meta == nullptr) {
+		return Variant();
+	}
+	return get_as_variant(*meta);
+}
+
+void VoxelBuffer::set_voxel_metadata(Vector3i pos, Variant meta) {
+	if (meta.get_type() == Variant::NIL) {
+		_buffer->erase_voxel_metadata(pos);
+	} else {
+		VoxelMetadata *mv = _buffer->get_or_create_voxel_metadata(pos);
+		ZN_ASSERT_RETURN(mv != nullptr);
+		set_as_variant(*mv, meta);
+	}
 }
 
 void VoxelBuffer::for_each_voxel_metadata(const Callable &callback) const {
 	ERR_FAIL_COND(callback.is_null());
 	//_buffer->for_each_voxel_metadata(callback);
 
-	const FlatMap<Vector3i, Variant> &metadata = _buffer->get_voxel_metadata();
+	const FlatMapMoveOnly<Vector3i, VoxelMetadata> &metadata = _buffer->get_voxel_metadata();
 
 	for (auto it = metadata.begin(); it != metadata.end(); ++it) {
+		Variant v = get_as_variant(it->value);
 		const Variant key = it->key;
-		const Variant *args[2] = { &key, &it->value };
+		const Variant *args[2] = { &key, &v };
 		Callable::CallError err;
 		Variant retval; // We don't care about the return value, Callable API requires it
 		callback.call(args, 2, retval, err);
@@ -136,9 +160,10 @@ void VoxelBuffer::for_each_voxel_metadata_in_area(const Callable &callback, Vect
 
 	const Box3i box = Box3i::from_min_max(min_pos, max_pos);
 
-	_buffer->for_each_voxel_metadata_in_area(box, [&callback](Vector3i rel_pos, Variant meta) {
+	_buffer->for_each_voxel_metadata_in_area(box, [&callback](Vector3i rel_pos, const VoxelMetadata &meta) {
+		Variant v = get_as_variant(meta);
 		const Variant key = rel_pos;
-		const Variant *args[2] = { &key, &meta };
+		const Variant *args[2] = { &key, &v };
 		Callable::CallError err;
 		Variant retval; // We don't care about the return value, Callable API requires it
 		callback.call(args, 2, retval, err);

storage/voxel_buffer_gd.h

@@ -149,17 +149,12 @@ public:
 
 	// Metadata
 
-	Variant get_block_metadata() const {
-		return _buffer->get_block_metadata();
-	}
+	Variant get_block_metadata() const;
 	void set_block_metadata(Variant meta);
 
-	Variant get_voxel_metadata(Vector3i pos) const {
-		return _buffer->get_voxel_metadata(pos);
-	}
-	void set_voxel_metadata(Vector3i pos, Variant meta) {
-		_buffer->set_voxel_metadata(pos, meta);
-	}
+	Variant get_voxel_metadata(Vector3i pos) const;
+	void set_voxel_metadata(Vector3i pos, Variant meta);
+
 	void for_each_voxel_metadata(const Callable &callback) const;
 	void for_each_voxel_metadata_in_area(const Callable &callback, Vector3i min_pos, Vector3i max_pos);
 	void copy_voxel_metadata_in_area(

storage/voxel_buffer_internal.cpp

@@ -597,13 +597,8 @@ void VoxelBufferInternal::move_to(VoxelBufferInternal &dst) {
 	dst._channels = _channels;
 	dst._size = _size;
 
-	// TODO Optimization: Godot needs move semantics
-	dst._block_metadata = _block_metadata;
-	_block_metadata = BlockMetadata();
-
-	// TODO Optimization: Godot needs move semantics
-	dst._voxel_metadata = _voxel_metadata;
-	_voxel_metadata.clear();
+	dst._block_metadata = std::move(_block_metadata);
+	dst._voxel_metadata = std::move(_voxel_metadata);
 
 	for (unsigned int i = 0; i < _channels.size(); ++i) {
 		Channel &channel = _channels[i];
@@ -780,27 +775,33 @@ float VoxelBufferInternal::get_sdf_quantization_scale(Depth d) {
 	}
 }
 
-void VoxelBufferInternal::set_block_metadata(Variant meta) {
-	_block_metadata.user_data = meta;
+const VoxelMetadata *VoxelBufferInternal::get_voxel_metadata(Vector3i pos) const {
+	ZN_ASSERT_RETURN_V(is_position_valid(pos), nullptr);
+	return _voxel_metadata.find(pos);
 }
 
-Variant VoxelBufferInternal::get_voxel_metadata(Vector3i pos) const {
-	ZN_ASSERT_RETURN_V(is_position_valid(pos), Variant());
-	Variant metadata;
-	_voxel_metadata.find(pos, metadata);
-	return metadata;
+VoxelMetadata *VoxelBufferInternal::get_voxel_metadata(Vector3i pos) {
+	ZN_ASSERT_RETURN_V(is_position_valid(pos), nullptr);
+	return _voxel_metadata.find(pos);
 }
 
-void VoxelBufferInternal::set_voxel_metadata(Vector3i pos, Variant meta) {
-	ZN_ASSERT_RETURN(is_position_valid(pos));
-	if (meta.get_type() == Variant::NIL) {
-		_voxel_metadata.erase(pos);
-	} else {
-		_voxel_metadata.insert_or_assign(pos, meta);
+VoxelMetadata *VoxelBufferInternal::get_or_create_voxel_metadata(Vector3i pos) {
+	ZN_ASSERT_RETURN_V(is_position_valid(pos), nullptr);
+	VoxelMetadata *d = _voxel_metadata.find(pos);
+	if (d != nullptr) {
+		return d;
 	}
+	// TODO Optimize: we know the key should not exist
+	VoxelMetadata &meta = _voxel_metadata.insert_or_assign(pos, VoxelMetadata());
+	return &meta;
 }
 
-void VoxelBufferInternal::clear_and_set_voxel_metadata(Span<FlatMap<Vector3i, Variant>::Pair> pairs) {
+void VoxelBufferInternal::erase_voxel_metadata(Vector3i pos) {
+	ZN_ASSERT_RETURN(is_position_valid(pos));
+	_voxel_metadata.erase(pos);
+}
+
+void VoxelBufferInternal::clear_and_set_voxel_metadata(Span<FlatMapMoveOnly<Vector3i, VoxelMetadata>::Pair> pairs) {
 #ifdef DEBUG_ENABLED
 	for (size_t i = 0; i < pairs.size(); ++i) {
 		ZN_ASSERT_CONTINUE(is_position_valid(pairs[i].key));
@@ -853,7 +854,7 @@ void VoxelBufferInternal::clear_voxel_metadata() {
 }
 
 void VoxelBufferInternal::clear_voxel_metadata_in_area(Box3i box) {
-	_voxel_metadata.remove_if([&box](const FlatMap<Vector3i, Variant>::Pair &p) { //
+	_voxel_metadata.remove_if([&box](const FlatMapMoveOnly<Vector3i, VoxelMetadata>::Pair &p) { //
 		return box.contains(p.key);
 	});
 }
@@ -865,12 +866,14 @@ void VoxelBufferInternal::copy_voxel_metadata_in_area(
 	const Box3i clipped_src_box = src_box.clipped(Box3i(src_box.pos - dst_origin, _size));
 	const Vector3i clipped_dst_offset = dst_origin + clipped_src_box.pos - src_box.pos;
 
-	for (FlatMap<Vector3i, Variant>::ConstIterator src_it = src_buffer._voxel_metadata.begin();
+	for (FlatMapMoveOnly<Vector3i, VoxelMetadata>::ConstIterator src_it = src_buffer._voxel_metadata.begin();
 			src_it != src_buffer._voxel_metadata.end(); ++src_it) {
 		if (src_box.contains(src_it->key)) {
 			const Vector3i dst_pos = src_it->key + clipped_dst_offset;
 			ZN_ASSERT(is_position_valid(dst_pos));
-			_voxel_metadata.insert_or_assign(dst_pos, src_it->value.duplicate());
+
+			VoxelMetadata &meta = _voxel_metadata.insert_or_assign(dst_pos, VoxelMetadata());
+			meta.copy_from(src_it->value);
 		}
 	}
 }
@@ -878,12 +881,13 @@ void VoxelBufferInternal::copy_voxel_metadata_in_area(
 void VoxelBufferInternal::copy_voxel_metadata(const VoxelBufferInternal &src_buffer) {
 	ZN_ASSERT_RETURN(src_buffer.get_size() == _size);
 
-	for (FlatMap<Vector3i, Variant>::ConstIterator src_it = src_buffer._voxel_metadata.begin();
+	for (FlatMapMoveOnly<Vector3i, VoxelMetadata>::ConstIterator src_it = src_buffer._voxel_metadata.begin();
 			src_it != src_buffer._voxel_metadata.end(); ++src_it) {
-		_voxel_metadata.insert_or_assign(src_it->key, src_it->value.duplicate());
+		VoxelMetadata &meta = _voxel_metadata.insert_or_assign(src_it->key, VoxelMetadata());
+		meta.copy_from(src_it->value);
 	}
 
-	_block_metadata.user_data = src_buffer._block_metadata.user_data.duplicate();
+	_block_metadata.copy_from(src_buffer._block_metadata);
 }
 
 } // namespace zylann::voxel

storage/voxel_buffer_internal.h

@@ -6,9 +6,9 @@
 #include "../util/flat_map.h"
 #include "../util/math/box3i.h"
 #include "funcs.h"
+#include "voxel_metadata.h"
 
 #include <core/os/rw_lock.h>
-#include <core/variant/variant.h>
 #include <limits>
 
 namespace zylann::voxel {
@@ -416,20 +416,26 @@ public:
 
 	// Metadata
 
-	Variant get_block_metadata() const {
-		return _block_metadata.user_data;
+	VoxelMetadata &get_block_metadata() {
+		return _block_metadata;
+	}
+	const VoxelMetadata &get_block_metadata() const {
+		return _block_metadata;
 	}
-	void set_block_metadata(Variant meta);
 
-	Variant get_voxel_metadata(Vector3i pos) const;
-	void set_voxel_metadata(Vector3i pos, Variant meta);
+	const VoxelMetadata *get_voxel_metadata(Vector3i pos) const;
+	VoxelMetadata *get_voxel_metadata(Vector3i pos);
+	VoxelMetadata *get_or_create_voxel_metadata(Vector3i pos);
+	void erase_voxel_metadata(Vector3i pos);
 
-	void clear_and_set_voxel_metadata(Span<FlatMap<Vector3i, Variant>::Pair> pairs);
+	void clear_and_set_voxel_metadata(Span<FlatMapMoveOnly<Vector3i, VoxelMetadata>::Pair> pairs);
 
 	template <typename F>
 	void for_each_voxel_metadata_in_area(Box3i box, F callback) const {
-		for (FlatMap<Vector3i, Variant>::ConstIterator it = _voxel_metadata.begin(); it != _voxel_metadata.end();
-				++it) {
+		// TODO For `find`s and this kind of iteration, we may want to separate keys and values in FlatMap's internal
+		// storage, to reduce cache misses
+		for (FlatMapMoveOnly<Vector3i, VoxelMetadata>::ConstIterator it = _voxel_metadata.begin();
+				it != _voxel_metadata.end(); ++it) {
 			if (box.contains(it->key)) {
 				callback(it->key, it->value);
 			}
@@ -447,7 +453,7 @@ public:
 	void copy_voxel_metadata_in_area(const VoxelBufferInternal &src_buffer, Box3i src_box, Vector3i dst_origin);
 	void copy_voxel_metadata(const VoxelBufferInternal &src_buffer);
 
-	const FlatMap<Vector3i, Variant> &get_voxel_metadata() const {
+	const FlatMapMoveOnly<Vector3i, VoxelMetadata> &get_voxel_metadata() const {
 		return _voxel_metadata;
 	}
 
@@ -480,15 +486,10 @@ private:
 	// How many voxels are there in the three directions. All populated channels have the same size.
 	Vector3i _size;
 
-	struct BlockMetadata {
-		// User-defined data, not used by the engine
-		Variant user_data;
-	};
-
 	// TODO Could we separate metadata from VoxelBufferInternal?
-	BlockMetadata _block_metadata;
+	VoxelMetadata _block_metadata;
 	// This metadata is expected to be sparse, with low amount of items.
-	FlatMap<Vector3i, Variant> _voxel_metadata;
+	FlatMapMoveOnly<Vector3i, VoxelMetadata> _voxel_metadata;
 
 	// TODO It may be preferable to actually move away from storing an RWLock in every buffer in the future.
 	// We should be able to find a solution because very few of these locks are actually used at a given time.

storage/voxel_metadata.cpp

@@ -0,0 +1,101 @@
+#include "voxel_metadata.h"
+#include "../util/errors.h"
+#include "../util/string_funcs.h"
+
+namespace zylann::voxel {
+
+void VoxelMetadata::clear() {
+	if (_type >= TYPE_CUSTOM_BEGIN) {
+		ZN_DELETE(_data.custom_data);
+		_data.custom_data = nullptr;
+	}
+	_type = TYPE_EMPTY;
+}
+
+void VoxelMetadata::set_u64(const uint64_t &v) {
+	if (_type != TYPE_U64) {
+		clear();
+		_type = TYPE_U64;
+	}
+	_data.u64_data = v;
+}
+
+uint64_t VoxelMetadata::get_u64() const {
+	ZN_ASSERT(_type == TYPE_U64);
+	return _data.u64_data;
+}
+
+void VoxelMetadata::set_custom(uint8_t type, ICustomVoxelMetadata *custom_data) {
+	ZN_ASSERT(type >= TYPE_CUSTOM_BEGIN);
+	clear();
+	_type = type;
+	_data.custom_data = custom_data;
+}
+
+ICustomVoxelMetadata &VoxelMetadata::get_custom() {
+	ZN_ASSERT(_type >= TYPE_CUSTOM_BEGIN);
+	ZN_ASSERT(_data.custom_data != nullptr);
+	return *_data.custom_data;
+}
+
+const ICustomVoxelMetadata &VoxelMetadata::get_custom() const {
+	ZN_ASSERT(_type >= TYPE_CUSTOM_BEGIN);
+	ZN_ASSERT(_data.custom_data != nullptr);
+	return *_data.custom_data;
+}
+
+void VoxelMetadata::copy_from(const VoxelMetadata &src) {
+	clear();
+	if (src._type >= TYPE_CUSTOM_BEGIN) {
+		ZN_ASSERT(src._data.custom_data != nullptr);
+		_data.custom_data = src._data.custom_data->duplicate();
+	} else {
+		_data = src._data;
+	}
+	_type = src._type;
+}
+
+////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+
+namespace {
+VoxelMetadataFactory g_voxel_metadata_factory;
+}
+
+VoxelMetadataFactory &VoxelMetadataFactory::get_singleton() {
+	return g_voxel_metadata_factory;
+}
+
+VoxelMetadataFactory::VoxelMetadataFactory() {
+	fill(_constructors, (ConstructorFunc) nullptr);
+}
+
+void VoxelMetadataFactory::add_constructor(uint8_t type, ConstructorFunc ctor) {
+	ZN_ASSERT(type >= VoxelMetadata::TYPE_CUSTOM_BEGIN);
+	const unsigned int i = type - VoxelMetadata::TYPE_CUSTOM_BEGIN;
+	ZN_ASSERT_MSG(_constructors[i] == nullptr, "Type already registered");
+	_constructors[i] = ctor;
+}
+
+void VoxelMetadataFactory::remove_constructor(uint8_t type) {
+	ZN_ASSERT(type >= VoxelMetadata::TYPE_CUSTOM_BEGIN);
+	const unsigned int i = type - VoxelMetadata::TYPE_CUSTOM_BEGIN;
+	ZN_ASSERT_MSG(_constructors[i] != nullptr, "Type not registered");
+	_constructors[i] = nullptr;
+}
+
+ICustomVoxelMetadata *VoxelMetadataFactory::try_construct(uint8_t type) const {
+	ZN_ASSERT_RETURN_V_MSG(
+			type >= VoxelMetadata::TYPE_CUSTOM_BEGIN, nullptr, format("Invalid custom metadata type {}", type));
+	const unsigned int i = type - VoxelMetadata::TYPE_CUSTOM_BEGIN;
+
+	const ConstructorFunc ctor = _constructors[i];
+	ZN_ASSERT_RETURN_V_MSG(ctor != nullptr, nullptr, format("Custom metadata constructor not found for type {}", type));
+
+	ICustomVoxelMetadata *m = ctor();
+	ZN_ASSERT_RETURN_V_MSG(
+			m != nullptr, nullptr, format("Custom metadata constructor for type {} returned nullptr", type));
+
+	return ctor();
+}
+
+} // namespace zylann::voxel

storage/voxel_metadata.h

@@ -0,0 +1,131 @@
+#ifndef VOXEL_METADATA_H
+#define VOXEL_METADATA_H
+
+#include "../util/memory.h"
+//#include "../util/non_copyable.h"
+#include "../util/span.h"
+#include <cstdint>
+
+namespace zylann::voxel {
+
+// Voxel metadata is arbitrary, sparse data that can be attached to particular voxels.
+// It is not intented at being an efficient or fast storage method, but rather a versatile one for special cases.
+// For example, it can be used to store text, tags, inventory contents, or other complex states attached to a voxel.
+// If you need to store smaller data much more frequently, you may rely on a data channel instead.
+
+// Base interface for custom data types.
+class ICustomVoxelMetadata {
+public:
+	virtual ~ICustomVoxelMetadata() {}
+
+	// Gets how many bytes this metadata will occupy when serialized.
+	virtual size_t get_serialized_size() const = 0;
+
+	// Serializes this metadata into `dst`. The size of `dst` will be equal or greater than the size returned by
+	// `get_serialized_size()`. Returns how many bytes were written.
+	virtual size_t serialize(Span<uint8_t> dst) const = 0;
+
+	// Deserializes this metadata from the given bytes.
+	// Returns `true` on success, `false` otherwise. `out_read_size` must be assigned to the number of bytes read.
+	virtual bool deserialize(Span<const uint8_t> src, uint64_t &out_read_size) = 0;
+
+	virtual ICustomVoxelMetadata *duplicate() = 0;
+};
+
+// Container for one metadata instance. It owns the data.
+class VoxelMetadata {
+public:
+	enum Type : uint8_t { //
+		TYPE_EMPTY = 0,
+		TYPE_U64 = 1,
+		// Reserved types.
+
+		TYPE_CUSTOM_BEGIN = 32
+		// Types equal or greater will implement `ICustomVoxelMetadata`.
+	};
+
+	static const unsigned int CUSTOM_TYPES_MAX_COUNT = 256 - TYPE_CUSTOM_BEGIN;
+
+	VoxelMetadata() {}
+
+	VoxelMetadata(VoxelMetadata &&other) {
+		_type = other._type;
+		_data = other._data;
+		other._type = TYPE_EMPTY;
+	}
+
+	~VoxelMetadata() {
+		clear();
+	}
+
+	inline void operator=(VoxelMetadata &&other) {
+		clear();
+		_type = other._type;
+		_data = other._data;
+		other._type = TYPE_EMPTY;
+	}
+
+	void clear();
+
+	inline Type get_type() const {
+		return Type(_type);
+	}
+
+	void set_u64(const uint64_t &v);
+	uint64_t get_u64() const;
+
+	void set_custom(uint8_t type, ICustomVoxelMetadata *custom_data);
+	ICustomVoxelMetadata &get_custom();
+	const ICustomVoxelMetadata &get_custom() const;
+
+	// Clears this metadata and makes it a duplicate of the given one.
+	void copy_from(const VoxelMetadata &src);
+
+private:
+	union Data {
+		uint64_t u64_data;
+		ICustomVoxelMetadata *custom_data;
+	};
+
+	uint8_t _type = TYPE_EMPTY;
+	Data _data;
+};
+
+// Registry of custom metadata types, used to deserialize them from saved data.
+class VoxelMetadataFactory {
+public:
+	typedef ICustomVoxelMetadata *(*ConstructorFunc)();
+
+	static VoxelMetadataFactory &get_singleton();
+
+	VoxelMetadataFactory();
+
+	// Registers a custom metadata type.
+	// The `type` you choose should remain the same over time.
+	// It will be used in save files, so changing it could break old saves.
+	// `type` must be greater or equal to `VoxelMetadata::TYPE_CUSTOM_BEGIN`.
+	void add_constructor(uint8_t type, ConstructorFunc ctor);
+
+	template <typename T>
+	void add_constructor_by_type(uint8_t type) {
+		add_constructor(type, []() { //
+			// Doesn't compile if I directly return the newed instance
+			ICustomVoxelMetadata *c = ZN_NEW(T);
+			return c;
+		});
+	}
+
+	void remove_constructor(uint8_t type);
+
+	// Constructs a custom metadata type from the given type ID.
+	// The `type` must be greater or equal to `VoxelMetadata::TYPE_CUSTOM_BEGIN`.
+	// Returns `nullptr` if the type could not be constructed.
+	ICustomVoxelMetadata *try_construct(uint8_t type) const;
+
+private:
+	FixedArray<ConstructorFunc, VoxelMetadata::CUSTOM_TYPES_MAX_COUNT> _constructors;
+};
+
+} //namespace zylann::voxel
+
+#endif // VOXEL_METADATA_H

storage/voxel_metadata_variant.cpp

@@ -0,0 +1,70 @@
+#include "voxel_metadata_variant.h"
+#include <core/io/marshalls.h>
+
+namespace zylann::voxel::gd {
+
+size_t VoxelMetadataVariant::get_serialized_size() const {
+	int len;
+	const Error err = encode_variant(data, nullptr, len, false);
+	ERR_FAIL_COND_V_MSG(err != OK, 0, "Error when trying to encode Variant metadata.");
+	return len;
+}
+
+size_t VoxelMetadataVariant::serialize(Span<uint8_t> dst) const {
+	int written_length;
+	const Error err = encode_variant(data, dst.data(), written_length, false);
+	ERR_FAIL_COND_V(err != OK, 0);
+	return written_length;
+}
+
+bool VoxelMetadataVariant::deserialize(Span<const uint8_t> src, uint64_t &out_read_size) {
+	int read_length;
+	const Error err = decode_variant(data, src.data(), src.size(), &read_length, false);
+	ERR_FAIL_COND_V_MSG(err != OK, false, "Failed to deserialize block metadata");
+	out_read_size = read_length;
+	return true;
+}
+
+ICustomVoxelMetadata *VoxelMetadataVariant::duplicate() {
+	VoxelMetadataVariant *d = ZN_NEW(VoxelMetadataVariant);
+	d->data = data.duplicate();
+	return d;
+}
+
+////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+
+Variant get_as_variant(const VoxelMetadata &meta) {
+	switch (meta.get_type()) {
+		case METADATA_TYPE_VARIANT: {
+			const VoxelMetadataVariant &mv = static_cast<const VoxelMetadataVariant &>(meta.get_custom());
+			return mv.data;
+		}
+		case VoxelMetadata::TYPE_EMPTY: {
+			return Variant();
+		}
+		case VoxelMetadata::TYPE_U64: {
+			return Variant(int64_t(meta.get_u64()));
+		}
+		default:
+			ZN_PRINT_ERROR("Unknown VoxelMetadata type");
+			return Variant();
+	}
+}
+
+void set_as_variant(VoxelMetadata &meta, Variant v) {
+	if (v.get_type() == Variant::NIL) {
+		meta.clear();
+	} else {
+		if (meta.get_type() == METADATA_TYPE_VARIANT) {
+			VoxelMetadataVariant &mv = static_cast<VoxelMetadataVariant &>(meta.get_custom());
+			mv.data = v;
+			return;
+		} else {
+			VoxelMetadataVariant *mv = ZN_NEW(VoxelMetadataVariant);
+			mv->data = v;
+			meta.set_custom(METADATA_TYPE_VARIANT, mv);
+		}
+	}
+}
+
+} // namespace zylann::voxel::gd

storage/voxel_metadata_variant.h

@@ -0,0 +1,31 @@
+#ifndef VOXEL_METADATA_VARIANT_H
+#define VOXEL_METADATA_VARIANT_H
+
+#include "voxel_metadata.h"
+#include <core/variant/variant.h>
+
+namespace zylann::voxel::gd {
+
+// TODO Not sure if that should be a custom type. Custom types are supposed to be specific to a game?
+enum GodotMetadataTypes { //
+	METADATA_TYPE_VARIANT = VoxelMetadata::TYPE_CUSTOM_BEGIN
+};
+
+// Custom metadata holding a Godot Variant (basically, anything recognized by Godot Engine).
+// Serializability depends on the same rules as Godot's `encode_variant`: no invalid objects, no cycles.
+class VoxelMetadataVariant : public ICustomVoxelMetadata {
+public:
+	Variant data;
+
+	size_t get_serialized_size() const override;
+	size_t serialize(Span<uint8_t> dst) const override;
+	bool deserialize(Span<const uint8_t> src, uint64_t &out_read_size) override;
+	ICustomVoxelMetadata *duplicate() override;
+};
+
+Variant get_as_variant(const VoxelMetadata &meta);
+void set_as_variant(VoxelMetadata &meta, Variant v);
+
+} // namespace zylann::voxel::gd
+
+#endif // VOXEL_METADATA_VARIANT_H

streams/voxel_block_serializer.cpp

@@ -5,9 +5,14 @@
 #include "../util/math/vector3i.h"
 #include "../util/profiling.h"
 #include "../util/serialization.h"
+#include "../util/string_funcs.h"
 #include "compressed_data.h"
 
+#ifdef ZN_GODOT
+#include "../storage/voxel_metadata_variant.h"
 #include <core/io/marshalls.h> // For `encode_variant`
+#endif
+
 #include <core/io/stream_peer.h>
 #include <limits>
 
@@ -24,11 +29,32 @@ thread_local std::vector<uint8_t> tls_data;
 thread_local std::vector<uint8_t> tls_compressed_data;
 thread_local std::vector<uint8_t> tls_metadata_tmp;
 
+size_t get_metadata_size_in_bytes(const VoxelMetadata &meta) {
+	size_t size = 1; // Type
+	switch (meta.get_type()) {
+		case VoxelMetadata::TYPE_EMPTY:
+			break;
+		case VoxelMetadata::TYPE_U64:
+			size += sizeof(uint64_t);
+			break;
+		default:
+			if (meta.get_type() >= VoxelMetadata::TYPE_CUSTOM_BEGIN) {
+				const ICustomVoxelMetadata &custom = meta.get_custom();
+				size += custom.get_serialized_size();
+			} else {
+				ZN_PRINT_ERROR("Unknown metadata type");
+				return 0;
+			}
+	}
+	return size;
+}
+
 size_t get_metadata_size_in_bytes(const VoxelBufferInternal &buffer) {
 	size_t size = 0;
 
-	const FlatMap<Vector3i, Variant> &voxel_metadata = buffer.get_voxel_metadata();
-	for (FlatMap<Vector3i, Variant>::ConstIterator it = voxel_metadata.begin(); it != voxel_metadata.end(); ++it) {
+	const FlatMapMoveOnly<Vector3i, VoxelMetadata> &voxel_metadata = buffer.get_voxel_metadata();
+	for (FlatMapMoveOnly<Vector3i, VoxelMetadata>::ConstIterator it = voxel_metadata.begin();
+			it != voxel_metadata.end(); ++it) {
 		const Vector3i pos = it->key;
 
 		ERR_FAIL_COND_V_MSG(pos.x < 0 || static_cast<uint32_t>(pos.x) >= VoxelBufferInternal::MAX_SIZE, 0,
@@ -39,23 +65,17 @@ size_t get_metadata_size_in_bytes(const VoxelBufferInternal &buffer) {
 				"Invalid voxel metadata Z position");
 
 		size += 3 * sizeof(uint16_t); // Positions are stored as 3 unsigned shorts
-
-		int len;
-		const Error err = encode_variant(it->value, nullptr, len, false);
-		ERR_FAIL_COND_V_MSG(err != OK, 0, "Error when trying to encode voxel metadata.");
-		size += len;
+		size += get_metadata_size_in_bytes(it->value);
 	}
 
 	// If no metadata is found at all, nothing is serialized, not even null.
 	// It spares 24 bytes (40 if real_t == double),
 	// and is backward compatible with saves made before introduction of metadata.
 
-	if (size != 0 || buffer.get_block_metadata() != Variant()) {
-		int len;
-		// Get size first by invoking the function is "length mode"
-		const Error err = encode_variant(buffer.get_block_metadata(), nullptr, len, false);
-		ERR_FAIL_COND_V_MSG(err != OK, 0, "Error when trying to encode block metadata.");
-		size += len;
+	const VoxelMetadata &block_meta = buffer.get_block_metadata();
+
+	if (size != 0 || block_meta.get_type() != VoxelMetadata::TYPE_EMPTY) {
+		size += get_metadata_size_in_bytes(block_meta);
 	}
 
 	return size;
@@ -74,43 +94,51 @@ inline T read(uint8_t *&src) {
 	return d;
 }
 
+static void serialize_metadata(const VoxelMetadata &meta, MemoryWriterExistingBuffer &mw) {
+	const uint8_t type = meta.get_type();
+	switch (type) {
+		case VoxelMetadata::TYPE_EMPTY:
+			mw.store_8(type);
+			break;
+		case VoxelMetadata::TYPE_U64:
+			mw.store_8(type);
+			mw.store_64(meta.get_u64());
+			break;
+		default:
+			if (type >= VoxelMetadata::TYPE_CUSTOM_BEGIN) {
+				mw.store_8(type);
+				const size_t written_size = meta.get_custom().serialize(mw.data.data.sub(mw.data.pos));
+				ZN_ASSERT(mw.data.pos + written_size <= mw.data.data.size());
+				mw.data.pos += written_size;
+			} else {
+				ZN_PRINT_ERROR("Unknown metadata type");
+				mw.store_8(VoxelMetadata::TYPE_EMPTY);
+			}
+			break;
+	}
+}
+
 // The target buffer MUST have correct size. Recoverable errors must have been checked before.
 void serialize_metadata(Span<uint8_t> p_dst, const VoxelBufferInternal &buffer) {
-	uint8_t *dst = p_dst.data();
+	ByteSpanWithPosition bs(p_dst, 0);
+	MemoryWriterExistingBuffer mw(bs, ENDIANESS_LITTLE_ENDIAN);
 
-	{
-		int written_length;
-		encode_variant(buffer.get_block_metadata(), dst, written_length, false);
-		dst += written_length;
+	const VoxelMetadata &block_meta = buffer.get_block_metadata();
+	serialize_metadata(block_meta, mw);
 
-		// I chose to cast this way to fix a GCC warning.
-		// If dst - p_dst is negative (which is wrong), it will wrap and cause a justified assertion failure
-		CRASH_COND_MSG(
-				static_cast<size_t>(dst - p_dst.data()) > p_dst.size(), "Wrote block metadata out of expected bounds");
-	}
-
-	const FlatMap<Vector3i, Variant> &voxel_metadata = buffer.get_voxel_metadata();
-	for (FlatMap<Vector3i, Variant>::ConstIterator it = voxel_metadata.begin(); it != voxel_metadata.end(); ++it) {
+	const FlatMapMoveOnly<Vector3i, VoxelMetadata> &voxel_metadata = buffer.get_voxel_metadata();
+	for (FlatMapMoveOnly<Vector3i, VoxelMetadata>::ConstIterator it = voxel_metadata.begin();
+			it != voxel_metadata.end(); ++it) {
 		// Serializing key as ushort because it's more than enough for a 3D dense array
 		static_assert(VoxelBufferInternal::MAX_SIZE <= std::numeric_limits<uint16_t>::max(),
 				"Maximum size exceeds serialization support");
 		const Vector3i pos = it->key;
-		write<uint16_t>(dst, pos.x);
-		write<uint16_t>(dst, pos.y);
-		write<uint16_t>(dst, pos.z);
+		mw.store_16(pos.x);
+		mw.store_16(pos.y);
+		mw.store_16(pos.z);
 
-		int written_length;
-		const Error err = encode_variant(it->value, dst, written_length, false);
-		CRASH_COND_MSG(err != OK, "Error when trying to encode voxel metadata.");
-		dst += written_length;
-
-		CRASH_COND_MSG(
-				static_cast<size_t>(dst - p_dst.data()) > p_dst.size(), "Wrote voxel metadata out of expected bounds");
+		serialize_metadata(it->value, mw);
 	}
-
-	CRASH_COND_MSG(static_cast<size_t>(dst - p_dst.data()) != p_dst.size(),
-			String("Written metadata doesn't match expected count (expected {0}, got {1})")
-					.format(varray(ZN_SIZE_T_TO_VARIANT(p_dst.size()), (int)(dst - p_dst.data()))));
 }
 
 template <typename T>
@@ -123,52 +151,73 @@ struct ClearOnExit {
 
 //#define CLEAR_ON_EXIT(container) ClearOnExit<decltype(container)> clear_on_exit_##__LINE__;
 
-bool deserialize_metadata(uint8_t *p_src, VoxelBufferInternal &buffer, const size_t metadata_size) {
-	uint8_t *src = p_src;
-	size_t remaining_length = metadata_size;
+static bool deserialize_metadata(VoxelMetadata &meta, MemoryReader &mr) {
+	const uint8_t type = mr.get_8();
+	switch (type) {
+		case VoxelMetadata::TYPE_EMPTY:
+			meta.clear();
+			return true;
 
-	{
-		Variant block_metadata;
-		int read_length;
-		const Error err = decode_variant(block_metadata, src, remaining_length, &read_length, false);
-		ERR_FAIL_COND_V_MSG(err != OK, false, "Failed to deserialize block metadata");
-		remaining_length -= read_length;
-		src += read_length;
-		CRASH_COND_MSG(remaining_length > metadata_size, "Block metadata size underflow");
-		buffer.set_block_metadata(block_metadata);
+		case VoxelMetadata::TYPE_U64:
+			meta.set_u64(mr.get_64());
+			return true;
+
+		default:
+			if (type >= VoxelMetadata::TYPE_CUSTOM_BEGIN) {
+				ICustomVoxelMetadata *custom = VoxelMetadataFactory::get_singleton().try_construct(type);
+				ZN_ASSERT_RETURN_V_MSG(
+						custom != nullptr, false, format("Could not deserialize custom metadata with type {}", type));
+
+				// Store in a temporary container so it auto-deletes in case of error
+				VoxelMetadata temp;
+				temp.set_custom(type, custom);
+
+				size_t read_size = 0;
+				ZN_ASSERT_RETURN_V(custom->deserialize(mr.data.sub(mr.pos), read_size), false);
+				ZN_ASSERT_RETURN_V(mr.pos + read_size <= mr.data.size(), false);
+				mr.pos += read_size;
+
+				meta = std::move(temp);
+				return true;
+
+			} else {
+				ZN_PRINT_ERROR("Unknown metadata type");
+				return false;
+			}
 	}
+	return false;
+}
 
-	typedef FlatMap<Vector3i, Variant>::Pair Pair;
+bool deserialize_metadata(Span<const uint8_t> p_src, VoxelBufferInternal &buffer) {
+	MemoryReader mr(p_src, ENDIANESS_LITTLE_ENDIAN);
+
+	ZN_ASSERT_RETURN_V(deserialize_metadata(buffer.get_block_metadata(), mr), false);
+
+	typedef FlatMapMoveOnly<Vector3i, VoxelMetadata>::Pair Pair;
 	static thread_local std::vector<Pair> tls_pairs;
 	// Clear when exiting scope (including cases of error) so we don't store dangling Variants
 	ClearOnExit<std::vector<Pair>> clear_tls_pairs{ tls_pairs };
 
-	while (remaining_length > 0) {
+	while (mr.pos < mr.data.size()) {
 		Vector3i pos;
-		pos.x = read<uint16_t>(src);
-		pos.y = read<uint16_t>(src);
-		pos.z = read<uint16_t>(src);
-		remaining_length -= 3 * sizeof(uint16_t);
+		pos.x = mr.get_16();
+		pos.y = mr.get_16();
+		pos.z = mr.get_16();
 
-		ERR_CONTINUE_MSG(!buffer.is_position_valid(pos),
-				String("Invalid voxel metadata position {0} for buffer of size {1}")
-						.format(varray(pos, buffer.get_size())));
+		ZN_ASSERT_CONTINUE_MSG(buffer.is_position_valid(pos),
+				format("Invalid voxel metadata position {} for buffer of size {}", pos, buffer.get_size()));
 
-		Variant metadata;
-		int read_length;
-		const Error err = decode_variant(metadata, src, remaining_length, &read_length, false);
-		ERR_FAIL_COND_V_MSG(err != OK, false, "Failed to deserialize block metadata");
-		remaining_length -= read_length;
-		src += read_length;
-		CRASH_COND_MSG(remaining_length > metadata_size, "Block metadata size underflow");
-
-		tls_pairs.push_back(Pair{ pos, metadata });
+		//VoxelMetadata &vmeta = buffer.get_or_create_voxel_metadata(pos);
+		tls_pairs.resize(tls_pairs.size() + 1);
+		Pair &p = tls_pairs.back();
+		p.key = pos;
+		ZN_ASSERT_RETURN_V_MSG(
+				deserialize_metadata(p.value, mr), false, format("Failed to deserialize voxel metadata {}", pos));
 	}
 
 	// Set all metadata at once, FlatMap is faster to initialize this way
 	buffer.clear_and_set_voxel_metadata(to_span(tls_pairs));
 
-	CRASH_COND_MSG(remaining_length != 0, "Did not read expected size");
 	return true;
 }
 
@@ -300,6 +349,108 @@ SerializeResult serialize(const VoxelBufferInternal &voxel_buffer) {
 	return SerializeResult(dst_data, true);
 }
 
+namespace legacy {
+
+bool migrate_v3_to_v4(Span<const uint8_t> p_data, std::vector<uint8_t> &dst) {
+	// In v3, metadata was always a Godot Variant. In v4, metadata uses an independent format.
+
+#ifndef ZN_GODOT
+	ZN_PRINT_ERROR("Cannot migrate block from v3 to v4, Godot Engine is required");
+	return false;
+#else
+
+	// Constants used at the time of this version
+	const unsigned int channel_count = 8;
+	const unsigned int no_compression = 0;
+	const unsigned int uniform_compression = 1;
+
+	MemoryReader mr(p_data, ENDIANESS_LITTLE_ENDIAN);
+
+	const uint8_t rv = mr.get_8(); // version
+	ZN_ASSERT(rv == 3);
+
+	const uint16_t size_x = mr.get_16(); // size_x
+	const uint16_t size_y = mr.get_16(); // size_y
+	const uint16_t size_z = mr.get_16(); // size_z
+	const unsigned int volume = size_x * size_y * size_z;
+
+	for (unsigned int channel_index = 0; channel_index < channel_count; ++channel_index) {
+		const uint8_t fmt = mr.get_8();
+
+		const uint8_t compression_value = fmt & 0xf;
+		const uint8_t depth_value = (fmt >> 4) & 0xf;
+
+		ZN_ASSERT_RETURN_V(compression_value < 2, false);
+		ZN_ASSERT_RETURN_V(depth_value < 4, false);
+
+		if (compression_value == no_compression) {
+			mr.pos += volume << depth_value;
+
+		} else if (compression_value == uniform_compression) {
+			mr.pos += size_t(1) << depth_value;
+		}
+	}
+
+	ZN_ASSERT(mr.pos <= mr.data.size());
+
+	// Copy everything up to beginning of metadata
+	dst.resize(mr.pos);
+	memcpy(dst.data(), p_data.data(), mr.pos);
+	// Set version
+	dst[0] = 4;
+
+	// Convert metadata
+
+	const size_t total_metadata_size = mr.data.size() - mr.pos;
+
+	if (total_metadata_size > 0) {
+		MemoryWriter mw(dst, ENDIANESS_LITTLE_ENDIAN);
+
+		struct L {
+			static bool convert_metadata_item(MemoryReader &mr, MemoryWriter &mw) {
+				// Read Variant
+				Variant src_meta;
+				int read_length;
+				const Error err =
+						decode_variant(src_meta, &mr.data[mr.pos], mr.data.size() - mr.pos, &read_length, false);
+				ZN_ASSERT_RETURN_V_MSG(err == OK, false, "Failed to deserialize v3 Variant metadata");
+				mr.pos += read_length;
+				ZN_ASSERT(mr.pos <= mr.data.size());
+
+				// Write v4 equivalent
+				VoxelMetadata dst_meta;
+				gd::VoxelMetadataVariant *custom = ZN_NEW(gd::VoxelMetadataVariant);
+				custom->data = src_meta;
+				dst_meta.set_custom(gd::METADATA_TYPE_VARIANT, custom);
+				mw.store_8(dst_meta.get_type());
+				const size_t ss = custom->get_serialized_size();
+				const size_t prev_size = mw.data.size();
+				mw.data.resize(mw.data.size() + ss);
+				const size_t written_size = custom->serialize(Span<uint8_t>(mw.data.data() + prev_size, ss));
+				mw.data.resize(prev_size + written_size);
+
+				return true;
+			}
+		};
+
+		ZN_ASSERT_RETURN_V(L::convert_metadata_item(mr, mw), false);
+
+		while (mr.pos < mr.data.size()) {
+			const uint16_t pos_x = mr.get_16();
+			const uint16_t pos_y = mr.get_16();
+			const uint16_t pos_z = mr.get_16();
+
+			mw.store_16(pos_x);
+			mw.store_16(pos_y);
+			mw.store_16(pos_z);
+
+			ZN_ASSERT_RETURN_V(L::convert_metadata_item(mr, mw), false);
+		}
+	}
+#endif
+	return true;
+}
+
 bool migrate_v2_to_v3(Span<const uint8_t> p_data, std::vector<uint8_t> &dst) {
 	// In v2, SDF data was using a legacy arbitrary formula to encode fixed-point numbers.
 	// In v3, it now uses inorm8 and inorm16.
@@ -317,7 +468,7 @@ bool migrate_v2_to_v3(Span<const uint8_t> p_data, std::vector<uint8_t> &dst) {
 	MemoryReader mr(p_data, ENDIANESS_LITTLE_ENDIAN);
 
 	const uint8_t rv = mr.get_8(); // version
-	CRASH_COND(rv != 2);
+	ZN_ASSERT(rv == 2);
 
 	dst[0] = 3;
 
@@ -344,12 +495,12 @@ bool migrate_v2_to_v3(Span<const uint8_t> p_data, std::vector<uint8_t> &dst) {
 				switch (depth_value) {
 					case 0:
 						for (unsigned int i = 0; i < volume; ++i) {
-							mw.store_8(snorm_to_s8(legacy::u8_to_snorm(mr.get_8())));
+							mw.store_8(snorm_to_s8(voxel::legacy::u8_to_snorm(mr.get_8())));
 						}
 						break;
 					case 1:
 						for (unsigned int i = 0; i < volume; ++i) {
-							mw.store_16(snorm_to_s16(legacy::u16_to_snorm(mr.get_16())));
+							mw.store_16(snorm_to_s16(voxel::legacy::u16_to_snorm(mr.get_16())));
 						}
 						break;
 					case 2:
@@ -361,10 +512,10 @@ bool migrate_v2_to_v3(Span<const uint8_t> p_data, std::vector<uint8_t> &dst) {
 			} else if (compression_value == uniform_compression) {
 				switch (depth_value) {
 					case 0:
-						mw.store_8(snorm_to_s8(legacy::u8_to_snorm(mr.get_8())));
+						mw.store_8(snorm_to_s8(voxel::legacy::u8_to_snorm(mr.get_8())));
 						break;
 					case 1:
-						mw.store_16(snorm_to_s16(legacy::u16_to_snorm(mr.get_16())));
+						mw.store_16(snorm_to_s16(voxel::legacy::u16_to_snorm(mr.get_16())));
 						break;
 					case 2:
 					case 3:
@@ -387,6 +538,8 @@ bool migrate_v2_to_v3(Span<const uint8_t> p_data, std::vector<uint8_t> &dst) {
 	return true;
 }
 
+} // namespace legacy
+
 bool deserialize(Span<const uint8_t> p_data, VoxelBufferInternal &out_voxel_buffer) {
 	ZN_PROFILE_SCOPE();
 
@@ -400,13 +553,21 @@ bool deserialize(Span<const uint8_t> p_data, VoxelBufferInternal &out_voxel_buff
 
 	const uint8_t format_version = f.get_8();
 
-	if (format_version == 2) {
-		std::vector<uint8_t> migrated_data;
-		ERR_FAIL_COND_V(!migrate_v2_to_v3(p_data, migrated_data), false);
-		return deserialize(to_span_const(migrated_data), out_voxel_buffer);
+	switch (format_version) {
+		case 2: {
+			std::vector<uint8_t> migrated_data;
+			ERR_FAIL_COND_V(!legacy::migrate_v2_to_v3(p_data, migrated_data), false);
+			return deserialize(to_span(migrated_data), out_voxel_buffer);
+		} break;
 
-	} else {
-		ERR_FAIL_COND_V(format_version != BLOCK_FORMAT_VERSION, false);
+		case 3: {
+			std::vector<uint8_t> migrated_data;
+			ERR_FAIL_COND_V(!legacy::migrate_v3_to_v4(p_data, migrated_data), false);
+			return deserialize(to_span(migrated_data), out_voxel_buffer);
+		} break;
+
+		default:
+			ERR_FAIL_COND_V(format_version != BLOCK_FORMAT_VERSION, false);
 	}
 
 	const unsigned int size_x = f.get_16();
@@ -475,7 +636,7 @@ bool deserialize(Span<const uint8_t> p_data, VoxelBufferInternal &out_voxel_buff
 		ERR_FAIL_COND_V(f.get_position() + metadata_size > p_data.size(), false);
 		metadata_tmp.resize(metadata_size);
 		f.get_buffer(to_span(metadata_tmp));
-		deserialize_metadata(metadata_tmp.data(), out_voxel_buffer, metadata_tmp.size());
+		deserialize_metadata(to_span(metadata_tmp), out_voxel_buffer);
 	}
 
 	// Failure at this indicates file corruption

tests/tests.cpp

@@ -5,6 +5,7 @@
 #include "../meshers/blocky/voxel_blocky_library.h"
 #include "../storage/voxel_buffer_gd.h"
 #include "../storage/voxel_data_map.h"
+#include "../storage/voxel_metadata_variant.h"
 #include "../streams/instance_data.h"
 #include "../streams/region/region_file.h"
 #include "../streams/region/voxel_stream_region_files.h"
@@ -1485,10 +1486,11 @@ void test_flat_map() {
 			ZYLANN_TEST_ASSERT_V(sorted_pairs.size() == map.size(), false);
 			for (size_t i = 0; i < sorted_pairs.size(); ++i) {
 				const Pair expected_pair = sorted_pairs[i];
-				Value value;
 				ZYLANN_TEST_ASSERT_V(map.has(expected_pair.key), false);
-				ZYLANN_TEST_ASSERT_V(map.find(expected_pair.key, value), false);
-				ZYLANN_TEST_ASSERT_V(value == expected_pair.value, false);
+				ZYLANN_TEST_ASSERT_V(map.find(expected_pair.key) != nullptr, false);
+				const Value *value = map.find(expected_pair.key);
+				ZYLANN_TEST_ASSERT_V(value != nullptr, false);
+				ZYLANN_TEST_ASSERT_V(*value == expected_pair.value, false);
 			}
 			return true;
 		}
@@ -1829,6 +1831,205 @@ void test_expression_parser() {
 	}
 }
 
+class CustomMetadataTest : public ICustomVoxelMetadata {
+public:
+	static const uint8_t ID = VoxelMetadata::TYPE_CUSTOM_BEGIN + 10;
+
+	uint8_t a;
+	uint8_t b;
+	uint8_t c;
+
+	size_t get_serialized_size() const override {
+		// Note, `sizeof(CustomMetadataTest)` gives 16 here. Probably because of vtable
+		return 3;
+	}
+
+	size_t serialize(Span<uint8_t> dst) const override {
+		dst[0] = a;
+		dst[1] = b;
+		dst[2] = c;
+		return get_serialized_size();
+	}
+
+	bool deserialize(Span<const uint8_t> src, uint64_t &out_read_size) override {
+		a = src[0];
+		b = src[1];
+		c = src[2];
+		out_read_size = get_serialized_size();
+		return true;
+	}
+
+	virtual ICustomVoxelMetadata *duplicate() {
+		CustomMetadataTest *d = ZN_NEW(CustomMetadataTest);
+		*d = *this;
+		return d;
+	}
+
+	bool operator==(const CustomMetadataTest &other) const {
+		return a == other.a && b == other.b && c == other.c;
+	}
+};
+
+void test_voxel_buffer_metadata() {
+	// Basic get and set
+	{
+		VoxelBufferInternal vb;
+		vb.create(10, 10, 10);
+
+		VoxelMetadata *meta = vb.get_or_create_voxel_metadata(Vector3i(1, 2, 3));
+		ZYLANN_TEST_ASSERT(meta != nullptr);
+		meta->set_u64(1234567890);
+
+		const VoxelMetadata *meta2 = vb.get_voxel_metadata(Vector3i(1, 2, 3));
+		ZYLANN_TEST_ASSERT(meta2 != nullptr);
+		ZYLANN_TEST_ASSERT(meta2->get_type() == meta->get_type());
+		ZYLANN_TEST_ASSERT(meta2->get_u64() == meta->get_u64());
+	}
+	// Serialization
+	{
+		VoxelBufferInternal vb;
+		vb.create(10, 10, 10);
+
+		{
+			VoxelMetadata *meta0 = vb.get_or_create_voxel_metadata(Vector3i(1, 2, 3));
+			ZYLANN_TEST_ASSERT(meta0 != nullptr);
+			meta0->set_u64(1234567890);
+		}
+
+		{
+			VoxelMetadata *meta1 = vb.get_or_create_voxel_metadata(Vector3i(4, 5, 6));
+			ZYLANN_TEST_ASSERT(meta1 != nullptr);
+			meta1->clear();
+		}
+
+		struct RemoveTypeOnExit {
+			~RemoveTypeOnExit() {
+				VoxelMetadataFactory::get_singleton().remove_constructor(CustomMetadataTest::ID);
+			}
+		};
+		RemoveTypeOnExit rmtype;
+		VoxelMetadataFactory::get_singleton().add_constructor_by_type<CustomMetadataTest>(CustomMetadataTest::ID);
+		{
+			VoxelMetadata *meta2 = vb.get_or_create_voxel_metadata(Vector3i(7, 8, 9));
+			ZYLANN_TEST_ASSERT(meta2 != nullptr);
+			CustomMetadataTest *custom = ZN_NEW(CustomMetadataTest);
+			custom->a = 10;
+			custom->b = 20;
+			custom->c = 30;
+			meta2->set_custom(CustomMetadataTest::ID, custom);
+		}
+
+		BlockSerializer::SerializeResult sresult = BlockSerializer::serialize(vb);
+		ZYLANN_TEST_ASSERT(sresult.success);
+		std::vector<uint8_t> bytes = sresult.data;
+
+		VoxelBufferInternal rvb;
+		ZYLANN_TEST_ASSERT(BlockSerializer::deserialize(to_span(bytes), rvb));
+
+		const FlatMapMoveOnly<Vector3i, VoxelMetadata> &vb_meta_map = vb.get_voxel_metadata();
+		const FlatMapMoveOnly<Vector3i, VoxelMetadata> &rvb_meta_map = rvb.get_voxel_metadata();
+
+		ZYLANN_TEST_ASSERT(vb_meta_map.size() == rvb_meta_map.size());
+
+		for (auto it = vb_meta_map.begin(); it != vb_meta_map.end(); ++it) {
+			const VoxelMetadata &meta = it->value;
+			const VoxelMetadata *rmeta = rvb_meta_map.find(it->key);
+
+			ZYLANN_TEST_ASSERT(rmeta != nullptr);
+			ZYLANN_TEST_ASSERT(rmeta->get_type() == meta.get_type());
+
+			switch (meta.get_type()) {
+				case VoxelMetadata::TYPE_EMPTY:
+					break;
+				case VoxelMetadata::TYPE_U64:
+					ZYLANN_TEST_ASSERT(meta.get_u64() == rmeta->get_u64());
+					break;
+				case CustomMetadataTest::ID: {
+					const CustomMetadataTest &custom = static_cast<const CustomMetadataTest &>(meta.get_custom());
+					const CustomMetadataTest &rcustom = static_cast<const CustomMetadataTest &>(rmeta->get_custom());
+					ZYLANN_TEST_ASSERT(custom == rcustom);
+				} break;
+				default:
+					ZYLANN_TEST_ASSERT(false);
+					break;
+			}
+		}
+	}
+}
+
+void test_voxel_buffer_metadata_gd() {
+	// Basic get and set (Godot)
+	{
+		Ref<gd::VoxelBuffer> vb;
+		vb.instantiate();
+		vb->create(10, 10, 10);
+
+		Array meta;
+		meta.push_back("Hello");
+		meta.push_back("World");
+		meta.push_back(42);
+
+		vb->set_voxel_metadata(Vector3i(1, 2, 3), meta);
+
+		Array read_meta = vb->get_voxel_metadata(Vector3i(1, 2, 3));
+		ZYLANN_TEST_ASSERT(read_meta.size() == meta.size());
+		ZYLANN_TEST_ASSERT(read_meta == meta);
+	}
+	// Serialization (Godot)
+	{
+		Ref<gd::VoxelBuffer> vb;
+		vb.instantiate();
+		vb->create(10, 10, 10);
+
+		{
+			Array meta0;
+			meta0.push_back("Hello");
+			meta0.push_back("World");
+			meta0.push_back(42);
+			vb->set_voxel_metadata(Vector3i(1, 2, 3), meta0);
+		}
+		{
+			Dictionary meta1;
+			meta1["One"] = 1;
+			meta1["Two"] = 2.5;
+			meta1["Three"] = Basis();
+			vb->set_voxel_metadata(Vector3i(4, 5, 6), meta1);
+		}
+
+		BlockSerializer::SerializeResult sresult = BlockSerializer::serialize(vb->get_buffer());
+		ZYLANN_TEST_ASSERT(sresult.success);
+		std::vector<uint8_t> bytes = sresult.data;
+
+		Ref<gd::VoxelBuffer> vb2;
+		vb2.instantiate();
+
+		ZYLANN_TEST_ASSERT(BlockSerializer::deserialize(to_span(bytes), vb2->get_buffer()));
+
+		ZYLANN_TEST_ASSERT(vb2->get_buffer().equals(vb->get_buffer()));
+
+		// `equals` does not compare metadata at the moment, mainly because it's not trivial and there is no use case
+		// for it apart from this test, so do it manually
+
+		const FlatMapMoveOnly<Vector3i, VoxelMetadata> &vb_meta_map = vb->get_buffer().get_voxel_metadata();
+		const FlatMapMoveOnly<Vector3i, VoxelMetadata> &vb2_meta_map = vb2->get_buffer().get_voxel_metadata();
+
+		ZYLANN_TEST_ASSERT(vb_meta_map.size() == vb2_meta_map.size());
+
+		for (auto it = vb_meta_map.begin(); it != vb_meta_map.end(); ++it) {
+			const VoxelMetadata &meta = it->value;
+			ZYLANN_TEST_ASSERT(meta.get_type() == gd::METADATA_TYPE_VARIANT);
+
+			const VoxelMetadata *meta2 = vb2_meta_map.find(it->key);
+			ZYLANN_TEST_ASSERT(meta2 != nullptr);
+			ZYLANN_TEST_ASSERT(meta2->get_type() == meta.get_type());
+
+			const gd::VoxelMetadataVariant &metav = static_cast<const gd::VoxelMetadataVariant &>(meta.get_custom());
+			const gd::VoxelMetadataVariant &meta2v = static_cast<const gd::VoxelMetadataVariant &>(meta2->get_custom());
+			ZYLANN_TEST_ASSERT(metav.data == meta2v.data);
+		}
+	}
+}
+
 ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
 
 #define VOXEL_TEST(fname)                                                                                              \
@@ -1866,6 +2067,8 @@ void run_voxel_tests() {
 	VOXEL_TEST(test_run_blocky_random_tick);
 	VOXEL_TEST(test_flat_map);
 	VOXEL_TEST(test_expression_parser);
+	VOXEL_TEST(test_voxel_buffer_metadata);
+	VOXEL_TEST(test_voxel_buffer_metadata_gd);
 
 	print_line("------------ Voxel tests end -------------");
 }

util/flat_map.h

@@ -1,5 +1,5 @@
-#ifndef FLAT_MAP_H
-#define FLAT_MAP_H
+#ifndef ZN_FLAT_MAP_H
+#define ZN_FLAT_MAP_H
 
 #include "span.h"
 #include <algorithm>
@@ -42,6 +42,7 @@ public:
 	bool insert(K key, T value) {
 		typename std::vector<Pair>::const_iterator it = std::lower_bound(_items.begin(), _items.end(), key);
 		if (it != _items.end() && it->key == key) {
+			// Item already exists
 			return false;
 		}
 		_items.insert(it, Pair{ key, value });
@@ -49,13 +50,16 @@ public:
 	}
 
 	// If the key already exists, the item will replace the previous value.
-	void insert_or_assign(K key, T value) {
+	T &insert_or_assign(K key, T value) {
 		typename std::vector<Pair>::iterator it = std::lower_bound(_items.begin(), _items.end(), key);
 		if (it != _items.end() && it->key == key) {
+			// Item already exists, assign it
 			it->value = value;
 		} else {
-			_items.insert(it, Pair{ key, value });
+			// Item doesnt exist, insert it
+			it = _items.insert(it, Pair{ key, value });
 		}
+		return it->value;
 	}
 
 	// Initialize from a collection if items.
@@ -69,13 +73,20 @@ public:
 		std::sort(_items.begin(), _items.end());
 	}
 
-	bool find(K key, T &out_value) const {
+	const T *find(K key) const {
 		typename std::vector<Pair>::const_iterator it = std::lower_bound(_items.begin(), _items.end(), key);
 		if (it != _items.end() && it->key == key) {
-			out_value = it->value;
-			return true;
+			return &it->value;
 		}
-		return false;
+		return nullptr;
+	}
+
+	T *find(K key) {
+		typename std::vector<Pair>::const_iterator it = std::lower_bound(_items.begin(), _items.end(), key);
+		if (it != _items.end() && it->key == key) {
+			return &it->value;
+		}
+		return nullptr;
 	}
 
 	bool has(K key) const {
@@ -188,6 +199,198 @@ private:
 	std::vector<Pair> _items;
 };
 
+////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+
+// template <typename T>
+// void insert_default(std::vector<T> &vec, size_t pi) {
+// 	ZN_ASSERT(pi <= vec.size());
+// 	const size_t prev_size = vec.size();
+// 	vec.resize(vec.size() + 1);
+// 	for (size_t i = pi; i < prev_size; ++i) {
+// 		vec[i + 1] = std::move(vec[i]);
+// 	}
+// }
+
+// Specialization of FlatMap where `T` is not copyable, only movable
+template <typename K, typename T, typename KComp = FlatMapDefaultComparator<K>>
+class FlatMapMoveOnly {
+public:
+	struct Pair {
+		K key;
+		T value;
+
+		// For std::sort
+		inline bool operator<(const Pair &other) const {
+			return KComp::less_than(key, other.key);
+		}
+
+		// For std::lower_bound
+		inline bool operator<(const K &other_key) const {
+			return KComp::less_than(key, other_key);
+		}
+
+		Pair() {}
+
+		Pair(const K &p_key, T &&p_value) {
+			key = p_key;
+			value = std::move(p_value);
+		}
+
+		Pair(Pair &&other) {
+			key = other.key;
+			value = std::move(other.value);
+		}
+
+		void operator=(Pair &&other) {
+			key = other.key;
+			value = std::move(other.value);
+		}
+	};
+
+	// If the key already exists, the item is not inserted and returns false.
+	// If insertion was successful, returns true.
+	bool insert(K key, T &&value) {
+		typename std::vector<Pair>::const_iterator it = std::lower_bound(_items.begin(), _items.end(), key);
+		if (it != _items.end() && it->key == key) {
+			// Item already exists
+			return false;
+		}
+		_items.insert(it, std::move(Pair(key, std::move(value))));
+		return true;
+	}
+
+	// If the key already exists, the item will replace the previous value.
+	T &insert_or_assign(K key, T &&value) {
+		typename std::vector<Pair>::iterator it = std::lower_bound(_items.begin(), _items.end(), key);
+		if (it != _items.end() && it->key == key) {
+			// Item already exists, assign it
+			it->value = std::move(value);
+		} else {
+			// Item doesnt exist, insert it
+			it = _items.insert(it, std::move(Pair(key, std::move(value))));
+		}
+		return it->value;
+	}
+
+	// Initialize from a collection if items.
+	// Faster than doing individual insertion of each item.
+	void clear_and_insert(Span<Pair> pairs) {
+		clear();
+		_items.resize(pairs.size());
+		for (size_t i = 0; i < pairs.size(); ++i) {
+			_items[i] = std::move(pairs[i]);
+		}
+		std::sort(_items.begin(), _items.end());
+	}
+
+	const T *find(K key) const {
+		typename std::vector<Pair>::const_iterator it = std::lower_bound(_items.begin(), _items.end(), key);
+		if (it != _items.end() && it->key == key) {
+			return &it->value;
+		}
+		return nullptr;
+	}
+
+	T *find(K key) {
+		typename std::vector<Pair>::iterator it = std::lower_bound(_items.begin(), _items.end(), key);
+		if (it != _items.end() && it->key == key) {
+			return &it->value;
+		}
+		return nullptr;
+	}
+
+	bool has(K key) const {
+		// Using std::binary_search is very annoying.
+		// First, we don't want to pass a Pair because it would require constructing a T.
+		// Using just the key as the "value" to search doesn't compile because Pair only has comparison with K as second
+		// argument.
+		// Specifying a comparison lambda is also not viable because both arguments need to be convertible to K.
+		// Making it work would require passing a struct with two operators() with arguments in both orders...
+		//return std::binary_search(_items.cbegin(), _items.cend(), key);
+
+		typename std::vector<Pair>::const_iterator it = std::lower_bound(_items.begin(), _items.end(), key);
+		return it != _items.end() && it->key == key;
+	}
+
+	bool erase(K key) {
+		typename std::vector<Pair>::iterator it = std::lower_bound(_items.begin(), _items.end(), key);
+		if (it != _items.end() && it->key == key) {
+			_items.erase(it);
+			return true;
+		}
+		return false;
+	}
+
+	inline size_t size() const {
+		return _items.size();
+	}
+
+	void clear() {
+		_items.clear();
+	}
+
+	// `bool predicate(FlatMap<K, T>::Pair)`
+	template <typename F>
+	inline void remove_if(F predicate) {
+		_items.erase(std::remove_if(_items.begin(), _items.end(), predicate));
+	}
+
+	void operator=(const FlatMap<K, T> &other) {
+		_items = other._items;
+	}
+
+	bool operator==(const FlatMap<K, T> &other) const {
+		return _items == other._items;
+	}
+
+	class ConstIterator {
+	public:
+		ConstIterator(const Pair *p) : _current(p) {}
+
+		inline const Pair &operator*() {
+#ifdef DEBUG_ENABLED
+			ZN_ASSERT(_current != nullptr);
+#endif
+			return *_current;
+		}
+
+		inline const Pair *operator->() {
+#ifdef DEBUG_ENABLED
+			ZN_ASSERT(_current != nullptr);
+#endif
+			return _current;
+		}
+
+		inline ConstIterator &operator++() {
+			++_current;
+			return *this;
+		}
+
+		inline bool operator==(const ConstIterator other) const {
+			return _current == other._current;
+		}
+
+		inline bool operator!=(const ConstIterator other) const {
+			return _current != other._current;
+		}
+
+	private:
+		const Pair *_current;
+	};
+
+	inline ConstIterator begin() const {
+		return ConstIterator(_items.empty() ? nullptr : &_items[0]);
+	}
+
+	inline ConstIterator end() const {
+		return ConstIterator(_items.empty() ? nullptr : (&_items[0] + _items.size()));
+	}
+
+private:
+	// Sorted by key
+	std::vector<Pair> _items;
+};
+
 } // namespace zylann
 
-#endif // FLAT_MAP_H
+#endif // ZN_FLAT_MAP_H