Some changes and fixes related modifiers

- VoxelLodTerrain no longer caches generated voxels by default, so
  generating on the fly is no longer exclusive to full load mode.
  Might add an option later, but not for now (VoxelTerrain is still
  unaffected and keeps caching them)
- The "Cached" state is represented with blocks having no voxel data,
  so it needs extra checks in some areas to avoid null access
- Fix generate task was not including modifiers after the base generator
- The "save_generator_output" option on streams now means such blocks are
  considered edited
- Modifying modifiers now clears cached generated blocks
  intersecting with them.
- Fix "re-generate" was erasing the internal stack of modifiers
- Added docs
master
Marc Gilleron 2022-06-18 23:14:18 +01:00
parent 714525724f
commit 86ba74ce3a
29 changed files with 449 additions and 143 deletions

View File

@ -417,3 +417,26 @@ Node name | Description
Curve | Returns the value of a custom `curve` at coordinate `x`, where `x` is in the range `[0..1]`. The `curve` is specified with a `Curve` resource.
Image2D | Returns the value of the red channel of an `image` at coordinates `(x, y)`, where `x` and `y` are in pixels and the return value is in the range `[0..1]` (or more if the image has an HDR format). If coordinates are outside the image, they will be wrapped around. No filtering is performed. The image must have an uncompressed format.
Modifiers
-----------
Modifiers are generators that affect a limited region of the volume. They can stack on top of base generated voxels or other modifiers, and affect the final result. This is a workflow that mostly serves if your world has a finite size, and you want to set up specific shapes of the landscape in a non-destructive way from the editor.
!!! note
This feature is only implemented with `VoxelLodTerrain` at the moment, and only works to sculpt smooth voxels.
Modifiers can be added with nodes as child of the terrain. `VoxelModifierSphere` adds or subtracts a sphere, while `VoxelModifierMesh` adds or subtracts a mesh. For the latter, the mesh must be baked into an SDF volume first, using the `VoxelMeshSDF` resource.
Because modifiers are part of the procedural generation stack, destructive edits will always override them. If a block is edited, modifiers cannot affect it. It is then assumed that such edits would come from players at runtime, and that modifiers don't change.
Caching
---------
Generators are designed to be deterministic: if the same area is generated twice, the result must be the same. This means, ultimately, we only need to store edited voxels (aka "destructive" editing), while non-edited regions can be recomputed on the fly. Even if you want to access one voxel and it happens to be in a non-edited location, then the generator will be called just to obtain that voxel.
However, if a generator is too expensive or not expected to run this way, it may be desirable to store the output in memory so that querying the same area again picks up the cached data.
By default, `VoxelTerrain` caches blocks in memory until they get far from any viewer. `VoxelLodTerrain` does not cache blocks by default. There is no option yet to change that behavior.
It is also possible to tell a `VoxelGenerator` to save its outputs to the current `VoxelStream`, if any is setup. However, these blocks will act as edited ones, so they will behave as if it was changes done destructively.

View File

@ -12,7 +12,6 @@ class SceneTree;
namespace zylann::voxel {
// Contains the baked signed distance field of a mesh, which can be used to sculpt terrain.
// TODO Make it a resource so we can pre-build, save and load the baked data more easily
class VoxelMeshSDF : public Resource {
GDCLASS(VoxelMeshSDF, Resource)
public:
@ -62,7 +61,14 @@ public:
// It is currently needed to ensure `VoxelServerUpdater` gets created so it can tick the task system...
void bake_async(SceneTree *scene_tree);
// Accesses baked SDF data.
// WARNING: don't modify this buffer. Only read from it.
// There are some usages (like modifiers) that will read it from different threads,
// but there is no thread safety in case of direct modification.
// TODO Introduce a VoxelBufferReadOnly? Since that's likely the only way in an object-oriented script API...
Ref<gd::VoxelBuffer> get_voxel_buffer() const;
// Gets the padded bounding box of the model. This is important to know for signed distances to be coherent.
AABB get_aabb() const;
Array debug_check_sdf(Ref<Mesh> mesh);

View File

@ -237,9 +237,7 @@ void VoxelToolLodTerrain::do_sphere(Vector3 center, float radius) {
ERR_FAIL_COND(data == nullptr);
VoxelDataLodMap::Lod &data_lod = data->lods[0];
if (_terrain->is_full_load_mode_enabled()) {
preload_box(*data, box, _terrain->get_generator().ptr());
}
preload_box(*data, box, _terrain->get_generator().ptr(), !_terrain->is_full_load_mode_enabled());
ops::DoSphere op;
op.box = box;
@ -768,9 +766,7 @@ void VoxelToolLodTerrain::stamp_sdf(
ERR_FAIL_COND(data == nullptr);
VoxelDataLodMap::Lod &data_lod = data->lods[0];
if (_terrain->is_full_load_mode_enabled()) {
preload_box(*data, voxel_box, _terrain->get_generator().ptr());
}
preload_box(*data, voxel_box, _terrain->get_generator().ptr(), !_terrain->is_full_load_mode_enabled());
// TODO Maybe more efficient to "rasterize" the box? We're going to iterate voxels the box doesnt intersect
// TODO Maybe we should scale SDF values based on the scale of the transform too

View File

@ -212,6 +212,8 @@ void VoxelToolTerrain::set_voxel_metadata(Vector3i pos, Variant meta) {
VoxelDataMap &map = _terrain->get_storage();
VoxelDataBlock *block = map.get_block(map.voxel_to_block(pos));
ERR_FAIL_COND_MSG(block == nullptr, "Area not editable");
// TODO In this situation, the generator would need to be invoked to fill in the blank
ERR_FAIL_COND_MSG(!block->has_voxels(), "Area not cached");
RWLockWrite lock(block->get_voxels().get_lock());
VoxelMetadata *meta_storage = block->get_voxels().get_or_create_voxel_metadata(map.to_local(pos));
ERR_FAIL_COND(meta_storage == nullptr);
@ -223,6 +225,8 @@ Variant VoxelToolTerrain::get_voxel_metadata(Vector3i pos) const {
VoxelDataMap &map = _terrain->get_storage();
VoxelDataBlock *block = map.get_block(map.voxel_to_block(pos));
ERR_FAIL_COND_V_MSG(block == nullptr, Variant(), "Area not editable");
// TODO In this situation, the generator would need to be invoked to fill in the blank
ERR_FAIL_COND_V_MSG(!block->has_voxels(), Variant(), "Area not cached");
RWLockRead lock(block->get_voxels().get_lock());
const VoxelMetadata *meta = block->get_voxels_const().get_voxel_metadata(map.to_local(pos));
if (meta == nullptr) {
@ -273,7 +277,8 @@ void VoxelToolTerrain::run_blocky_random_tick_static(VoxelDataMap &map, Box3i vo
const Vector3i block_origin = map.block_to_voxel(block_pos);
VoxelDataBlock *block = map.get_block(block_pos);
if (block != nullptr) {
if (block != nullptr && block->has_voxels()) {
// Doing ONLY reads here.
{
RWLockRead lock(block->get_voxels().get_lock());
@ -395,7 +400,7 @@ void VoxelToolTerrain::for_each_voxel_metadata_in_area(AABB voxel_area, const Ca
data_block_box.for_each_cell([&map, &callback, voxel_box](Vector3i block_pos) {
VoxelDataBlock *block = map.get_block(block_pos);
if (block == nullptr) {
if (block == nullptr || !block->has_voxels()) {
return;
}

View File

@ -42,7 +42,6 @@ public:
};
virtual Result generate_block(VoxelQueryData &input);
// TODO Single sample
virtual bool supports_single_generation() const {
return false;

View File

@ -145,7 +145,10 @@ struct DeepSampler : transvoxel::IDeepSDFSampler {
RWLockRead rlock(lod.map_lock);
const Vector3i lod_bpos = lod_pos >> lod.map.get_block_size_pow2();
const VoxelDataBlock *block = lod.map.get_block(lod_bpos);
if (block != nullptr) {
// TODO Thread-safety: this checking presence of voxels is not safe.
// It can change while meshing takes place if a modifier is moved in the same area,
// because it invalidates cached data.
if (block != nullptr && block->has_voxels()) {
voxels = block->get_voxels_shared();
bsm = lod.map.get_block_size_mask();
}

View File

@ -1,5 +1,6 @@
#include "generate_block_task.h"
#include "../storage/voxel_buffer_internal.h"
#include "../storage/voxel_data_map.h"
#include "../util/godot/funcs.h"
#include "../util/log.h"
#include "../util/profiling.h"
@ -43,6 +44,11 @@ void GenerateBlockTask::run(zylann::ThreadedTaskContext ctx) {
const VoxelGenerator::Result result = generator->generate_block(query_data);
max_lod_hint = result.max_lod_hint;
if (data != nullptr) {
data->modifiers.apply(
query_data.voxel_buffer, AABB(query_data.origin_in_voxels, query_data.voxel_buffer.get_size() << lod));
}
if (stream_dependency->valid) {
Ref<VoxelStream> stream = stream_dependency->stream;
@ -87,12 +93,20 @@ void GenerateBlockTask::apply_result() {
// The request response must match the dependency it would have been requested with.
// If it doesn't match, we are no longer interested in the result.
if (stream_dependency->valid) {
Ref<VoxelStream> stream = stream_dependency->stream;
VoxelServer::BlockDataOutput o;
o.voxels = voxels;
o.position = position;
o.lod = lod;
o.dropped = !has_run;
o.type = VoxelServer::BlockDataOutput::TYPE_GENERATED;
if (stream.is_valid() && stream->get_save_generator_output()) {
// We can't consider the block as "generated" since there is no state to tell that once saved,
// so it has to be considered an edited block
o.type = VoxelServer::BlockDataOutput::TYPE_LOADED;
} else {
o.type = VoxelServer::BlockDataOutput::TYPE_GENERATED;
}
o.max_lod_hint = max_lod_hint;
o.initial_load = false;

View File

@ -8,6 +8,8 @@
namespace zylann::voxel {
struct VoxelDataLodMap;
class GenerateBlockTask : public IThreadedTask {
public:
GenerateBlockTask();
@ -31,6 +33,7 @@ public:
bool drop_beyond_max_distance = true;
PriorityDependency priority_dependency;
std::shared_ptr<StreamingDependency> stream_dependency;
std::shared_ptr<VoxelDataLodMap> data;
std::shared_ptr<AsyncDependencyTracker> tracker;
};

View File

@ -15,13 +15,14 @@ std::atomic_int g_debug_load_block_tasks_count;
LoadBlockDataTask::LoadBlockDataTask(uint32_t p_volume_id, Vector3i p_block_pos, uint8_t p_lod, uint8_t p_block_size,
bool p_request_instances, std::shared_ptr<StreamingDependency> p_stream_dependency,
PriorityDependency p_priority_dependency) :
PriorityDependency p_priority_dependency, bool generate_cache_data) :
_priority_dependency(p_priority_dependency),
_position(p_block_pos),
_volume_id(p_volume_id),
_lod(p_lod),
_block_size(p_block_size),
_request_instances(p_request_instances),
_generate_cache_data(generate_cache_data),
//_request_voxels(true),
_stream_dependency(p_stream_dependency) {
//
@ -62,7 +63,7 @@ void LoadBlockDataTask::run(zylann::ThreadedTaskContext ctx) {
if (voxel_query_data.result == VoxelStream::RESULT_ERROR) {
ERR_PRINT("Error loading voxel block");
} else if (voxel_query_data.result == VoxelStream::RESULT_BLOCK_NOT_FOUND) {
} else if (voxel_query_data.result == VoxelStream::RESULT_BLOCK_NOT_FOUND && _generate_cache_data) {
Ref<VoxelGenerator> generator = _stream_dependency->generator;
if (generator.is_valid()) {
@ -76,7 +77,7 @@ void LoadBlockDataTask::run(zylann::ThreadedTaskContext ctx) {
task->priority_dependency = _priority_dependency;
VoxelServer::get_singleton().push_async_task(task);
_fallback_on_generator = true;
_requested_generator_task = true;
} else {
// If there is no generator... what do we do? What defines the format of that empty block?
@ -123,7 +124,7 @@ void LoadBlockDataTask::apply_result() {
// TODO Comparing pointer may not be guaranteed
// The request response must match the dependency it would have been requested with.
// If it doesn't match, we are no longer interested in the result.
if (_stream_dependency->valid && !_fallback_on_generator) {
if (_stream_dependency->valid && !_requested_generator_task) {
VoxelServer::BlockDataOutput o;
o.voxels = _voxels;
o.instances = std::move(_instances);

View File

@ -12,7 +12,7 @@ class LoadBlockDataTask : public IThreadedTask {
public:
LoadBlockDataTask(uint32_t p_volume_id, Vector3i p_block_pos, uint8_t p_lod, uint8_t p_block_size,
bool p_request_instances, std::shared_ptr<StreamingDependency> p_stream_dependency,
PriorityDependency p_priority_dependency);
PriorityDependency p_priority_dependency, bool generate_cache_data);
~LoadBlockDataTask();
@ -36,7 +36,8 @@ private:
bool _request_instances = false;
//bool _request_voxels = false;
bool _max_lod_hint = false;
bool _fallback_on_generator = false;
bool _generate_cache_data = true;
bool _requested_generator_task = false;
std::shared_ptr<StreamingDependency> _stream_dependency;
};

View File

@ -7,24 +7,21 @@
namespace zylann::voxel {
// Takes a list of blocks and interprets it as a cube of blocks centered around the area we want to create a mesh from.
// Voxels from central blocks are copied, and part of side blocks are also copied so we get a temporary buffer
// which includes enough neighbors for the mesher to avoid doing bound checks.
static void copy_block_and_neighbors(Span<std::shared_ptr<VoxelBufferInternal>> blocks, VoxelBufferInternal &dst,
int min_padding, int max_padding, int channels_mask, Ref<VoxelGenerator> generator, int data_block_size,
uint8_t lod_index, Vector3i mesh_block_pos) {
ZN_DSTACK();
ZN_PROFILE_SCOPE();
struct CubicAreaInfo {
int edge_size; // In data blocks
int mesh_block_size_factor;
unsigned int anchor_buffer_index;
// Extract wanted channels in a list
unsigned int channels_count = 0;
FixedArray<uint8_t, VoxelBufferInternal::MAX_CHANNELS> channels =
VoxelBufferInternal::mask_to_channels_list(channels_mask, channels_count);
inline bool is_valid() const {
return edge_size != 0;
}
};
CubicAreaInfo get_cubic_area_info_from_size(unsigned int size) {
// Determine size of the cube of blocks
int edge_size;
int mesh_block_size_factor;
switch (blocks.size()) {
switch (size) {
case 3 * 3 * 3:
edge_size = 3;
mesh_block_size_factor = 1;
@ -34,19 +31,41 @@ static void copy_block_and_neighbors(Span<std::shared_ptr<VoxelBufferInternal>>
mesh_block_size_factor = 2;
break;
default:
ERR_FAIL_MSG("Unsupported block count");
ZN_PRINT_ERROR("Unsupported block count");
return CubicAreaInfo{ 0, 0, 0 };
}
// Pick anchor block, usually within the central part of the cube (that block must be valid)
const unsigned int anchor_buffer_index = edge_size * edge_size + edge_size + 1;
std::shared_ptr<VoxelBufferInternal> &central_buffer = blocks[anchor_buffer_index];
return { edge_size, mesh_block_size_factor, anchor_buffer_index };
}
// Takes a list of blocks and interprets it as a cube of blocks centered around the area we want to create a mesh from.
// Voxels from central blocks are copied, and part of side blocks are also copied so we get a temporary buffer
// which includes enough neighbors for the mesher to avoid doing bound checks.
static void copy_block_and_neighbors(Span<std::shared_ptr<VoxelBufferInternal>> blocks, VoxelBufferInternal &dst,
int min_padding, int max_padding, int channels_mask, Ref<VoxelGenerator> generator,
const VoxelModifierStack *modifiers, int data_block_size, uint8_t lod_index, Vector3i mesh_block_pos) {
ZN_DSTACK();
ZN_PROFILE_SCOPE();
// Extract wanted channels in a list
unsigned int channels_count = 0;
FixedArray<uint8_t, VoxelBufferInternal::MAX_CHANNELS> channels =
VoxelBufferInternal::mask_to_channels_list(channels_mask, channels_count);
// Determine size of the cube of blocks
const CubicAreaInfo area_info = get_cubic_area_info_from_size(blocks.size());
ERR_FAIL_COND(!area_info.is_valid());
std::shared_ptr<VoxelBufferInternal> &central_buffer = blocks[area_info.anchor_buffer_index];
ERR_FAIL_COND_MSG(central_buffer == nullptr && generator.is_null(), "Central buffer must be valid");
if (central_buffer != nullptr) {
ERR_FAIL_COND_MSG(
Vector3iUtil::all_members_equal(central_buffer->get_size()) == false, "Central buffer must be cubic");
}
const int mesh_block_size = data_block_size * mesh_block_size_factor;
const int mesh_block_size = data_block_size * area_info.mesh_block_size_factor;
const int padded_mesh_block_size = mesh_block_size + min_padding + max_padding;
dst.create(padded_mesh_block_size, padded_mesh_block_size, padded_mesh_block_size);
@ -60,9 +79,7 @@ static void copy_block_and_neighbors(Span<std::shared_ptr<VoxelBufferInternal>>
const std::shared_ptr<VoxelBufferInternal> &buffer = blocks[i];
if (buffer != nullptr) {
// Initialize channel depths from the first non-null block found
for (unsigned int ci = 0; ci < channels.size(); ++ci) {
dst.set_channel_depth(ci, buffer->get_channel_depth(ci));
}
dst.copy_format(*buffer);
break;
}
}
@ -72,15 +89,16 @@ static void copy_block_and_neighbors(Span<std::shared_ptr<VoxelBufferInternal>>
std::vector<Box3i> boxes_to_generate;
const Box3i mesh_data_box = Box3i::from_min_max(min_pos, max_pos);
if (generator.is_valid()) {
const bool has_generator = generator.is_valid() || modifiers != nullptr;
if (has_generator) {
boxes_to_generate.push_back(mesh_data_box);
}
// Using ZXY as convention to reconstruct positions with thread locking consistency
unsigned int block_index = 0;
for (int z = -1; z < edge_size - 1; ++z) {
for (int x = -1; x < edge_size - 1; ++x) {
for (int y = -1; y < edge_size - 1; ++y) {
for (int z = -1; z < area_info.edge_size - 1; ++z) {
for (int x = -1; x < area_info.edge_size - 1; ++x) {
for (int y = -1; y < area_info.edge_size - 1; ++y) {
const Vector3i offset = data_block_size * Vector3i(x, y, z);
const std::shared_ptr<VoxelBufferInternal> &src = blocks[block_index];
++block_index;
@ -99,7 +117,7 @@ static void copy_block_and_neighbors(Span<std::shared_ptr<VoxelBufferInternal>>
}
}
if (generator.is_valid()) {
if (has_generator) {
// Subtract edited box from the area to generate
// TODO This approach allows to batch boxes if necessary,
// but is it just better to do it anyways for every clipped box?
@ -121,12 +139,13 @@ static void copy_block_and_neighbors(Span<std::shared_ptr<VoxelBufferInternal>>
}
}
if (generator.is_valid()) {
if (has_generator) {
// Complete data with generated voxels
ZN_PROFILE_SCOPE_NAMED("Generate");
VoxelBufferInternal generated_voxels;
const Vector3i origin_in_voxels = mesh_block_pos * (mesh_block_size_factor * data_block_size << lod_index);
const Vector3i origin_in_voxels =
mesh_block_pos * (area_info.mesh_block_size_factor * data_block_size << lod_index);
for (unsigned int i = 0; i < boxes_to_generate.size(); ++i) {
const Box3i &box = boxes_to_generate[i];
@ -135,7 +154,13 @@ static void copy_block_and_neighbors(Span<std::shared_ptr<VoxelBufferInternal>>
//generated_voxels.set_voxel_f(2.0f, box.size.x / 2, box.size.y / 2, box.size.z / 2,
//VoxelBufferInternal::CHANNEL_SDF);
VoxelGenerator::VoxelQueryData q{ generated_voxels, (box.pos << lod_index) + origin_in_voxels, lod_index };
generator->generate_block(q);
if (generator.is_valid()) {
generator->generate_block(q);
}
if (modifiers != nullptr) {
modifiers->apply(q.voxel_buffer, AABB(q.origin_in_voxels, q.voxel_buffer.get_size() << lod_index));
}
for (unsigned int ci = 0; ci < channels_count; ++ci) {
dst.copy_from(generated_voxels, Vector3i(), generated_voxels.get_size(),
@ -173,19 +198,50 @@ void MeshBlockTask::run(zylann::ThreadedTaskContext ctx) {
const unsigned int min_padding = mesher->get_minimum_padding();
const unsigned int max_padding = mesher->get_maximum_padding();
// TODO Cache?
const VoxelModifierStack *modifiers = data != nullptr ? &data->modifiers : nullptr;
VoxelBufferInternal voxels;
copy_block_and_neighbors(to_span(blocks, blocks_count), voxels, min_padding, max_padding,
mesher->get_used_channels_mask(), meshing_dependency->generator, data_block_size, lod, position);
mesher->get_used_channels_mask(), meshing_dependency->generator, modifiers, data_block_size, lod_index,
position);
const Vector3i origin_in_voxels = position * (int(data_block_size) << lod);
// Could cache generator data from here if it was safe to write into the map
/*if (data != nullptr && cache_generated_blocks) {
const CubicAreaInfo area_info = get_cubic_area_info_from_size(blocks.size());
ERR_FAIL_COND(!area_info.is_valid());
if (data != nullptr) {
data->modifiers.apply(voxels, AABB(origin_in_voxels, voxels.get_size() << lod));
}
VoxelDataLodMap::Lod &lod = data->lods[lod_index];
const VoxelMesher::Input input = { voxels, meshing_dependency->generator.ptr(), data.get(), origin_in_voxels, lod,
collision_hint };
// Note, this box does not include neighbors!
const Vector3i min_bpos = position * area_info.mesh_block_size_factor;
const Vector3i max_bpos = min_bpos + Vector3iUtil::create(area_info.edge_size - 2);
Vector3i bpos;
for (bpos.z = min_bpos.z; bpos.z < max_bpos.z; ++bpos.z) {
for (bpos.x = min_bpos.x; bpos.x < max_bpos.x; ++bpos.x) {
for (bpos.y = min_bpos.y; bpos.y < max_bpos.y; ++bpos.y) {
// {
// RWLockRead rlock(lod.map_lock);
// VoxelDataBlock *block = lod.map.get_block(bpos);
// if (block != nullptr && (block->is_edited() || block->is_modified())) {
// continue;
// }
// }
std::shared_ptr<VoxelBufferInternal> &cache_buffer = make_shared_instance<VoxelBufferInternal>();
cache_buffer->copy_format(voxels);
const Vector3i min_src_pos =
(bpos - min_bpos) * data_block_size + Vector3iUtil::create(min_padding);
cache_buffer->copy_from(voxels, min_src_pos, min_src_pos + cache_buffer->get_size(), Vector3i());
// TODO Where to put voxels? Can't safely write to data at the moment.
}
}
}
}*/
const Vector3i origin_in_voxels = position * (int(data_block_size) << lod_index);
const VoxelMesher::Input input = { voxels, meshing_dependency->generator.ptr(), data.get(), origin_in_voxels,
lod_index, collision_hint };
mesher->build(_surfaces_output, input);
_has_run = true;
@ -193,7 +249,7 @@ void MeshBlockTask::run(zylann::ThreadedTaskContext ctx) {
int MeshBlockTask::get_priority() {
float closest_viewer_distance_sq;
const int p = priority_dependency.evaluate(lod, &closest_viewer_distance_sq);
const int p = priority_dependency.evaluate(lod_index, &closest_viewer_distance_sq);
_too_far = closest_viewer_distance_sq > priority_dependency.drop_distance_squared;
return p;
}
@ -218,7 +274,7 @@ void MeshBlockTask::apply_result() {
}
o.position = position;
o.lod = lod;
o.lod = lod_index;
o.surfaces = std::move(_surfaces_output);
VoxelServer::VolumeCallbacks callbacks = VoxelServer::get_singleton().get_volume_callbacks(volume_id);

View File

@ -28,7 +28,7 @@ public:
//FixedArray<uint8_t, VoxelBufferInternal::MAX_CHANNELS> channel_depths;
Vector3i position; // In mesh blocks of the specified lod
uint32_t volume_id;
uint8_t lod = 0;
uint8_t lod_index = 0;
uint8_t blocks_count = 0;
uint8_t data_block_size = 0;
bool collision_hint = false;

View File

@ -36,6 +36,8 @@ public:
};
Type type;
// If voxels are null with TYPE_LOADED, it means no block was found in the stream (if any) and no generator task
// was scheduled. This is the case when we don't want to cache blocks of generated data.
std::shared_ptr<VoxelBufferInternal> voxels;
UniquePtr<InstanceBlockData> instances;
Vector3i position;

View File

@ -134,6 +134,28 @@ void store_sdf(VoxelBufferInternal &voxels, Span<float> sdf) {
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
VoxelModifierStack::VoxelModifierStack() {}
VoxelModifierStack::VoxelModifierStack(VoxelModifierStack &&other) {
move_from_noclear(other);
}
VoxelModifierStack &VoxelModifierStack::operator=(VoxelModifierStack &&other) {
clear();
move_from_noclear(other);
_next_id = other._next_id;
return *this;
}
void VoxelModifierStack::move_from_noclear(VoxelModifierStack &other) {
{
RWLockRead rlock(other._stack_lock);
_modifiers = std::move(other._modifiers);
_stack = std::move(other._stack);
}
_next_id = other._next_id;
}
uint32_t VoxelModifierStack::allocate_id() {
return ++_next_id;
}
@ -224,6 +246,12 @@ void VoxelModifierStack::apply(float &sdf, Vector3 position) const {
}
}
void VoxelModifierStack::clear() {
RWLockWrite lock(_stack_lock);
_stack.clear();
_modifiers.clear();
}
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
void VoxelModifierSphere::set_radius(real_t radius) {
@ -303,6 +331,10 @@ void VoxelModifierBuffer::apply(VoxelModifierContext ctx) const {
return;
}
// TODO VoxelMeshSDF isn't preventing scripts from writing into this buffer from a different thread.
// I can't think of a reason to manually modify the buffer of a VoxelMeshSDF at the moment.
RWLockRead buffer_rlock(_buffer->get_lock());
const Transform3D model_to_world = get_transform();
const Transform3D buffer_to_model =
Transform3D(Basis().scaled(to_vec3(_max_pos - _min_pos) / to_vec3(_buffer->get_size())), to_vec3(_min_pos));

View File

@ -146,6 +146,11 @@ class VoxelModifierStack {
public:
uint32_t allocate_id();
VoxelModifierStack();
VoxelModifierStack(VoxelModifierStack &&other);
VoxelModifierStack &operator=(VoxelModifierStack &&other);
template <typename T>
T *add_modifier(uint32_t id) {
ZN_ASSERT(!has_modifier(id));
@ -162,8 +167,11 @@ public:
VoxelModifier *get_modifier(uint32_t id) const;
void apply(VoxelBufferInternal &voxels, AABB aabb) const;
void apply(float &sdf, Vector3 position) const;
void clear();
private:
void move_from_noclear(VoxelModifierStack &other);
std::unordered_map<uint32_t, UniquePtr<VoxelModifier>> _modifiers;
uint32_t _next_id = 1;
// TODO Later, replace this with a spatial acceleration structure based on AABBs, like BVH

View File

@ -116,7 +116,7 @@ void VoxelModifier::_notification(int p_what) {
std::shared_ptr<VoxelDataLodMap> data = _volume->get_storage();
VoxelModifierStack &modifiers = data->modifiers;
zylann::voxel::VoxelModifier *modifier = modifiers.get_modifier(_modifier_id);
ZN_ASSERT_RETURN(modifier != nullptr);
ZN_ASSERT_RETURN_MSG(modifier != nullptr, "The modifier node wasn't linked properly");
post_edit_modifier(*_volume, modifier->get_aabb());
modifiers.remove_modifier(_modifier_id);
_volume = nullptr;

View File

@ -7,13 +7,18 @@
namespace zylann::voxel {
// Stores loaded voxel data for a chunk of the volume. Mesh and colliders are stored separately.
// Stores voxel data for a chunk of the volume. Mesh and colliders are stored separately.
// Voxel data can be present, or not. If not present, it means we know the block contains no edits, and voxels can be
// obtained by querying generators.
// Voxel data can also be present as a cache of generators, for cheaper repeated queries.
class VoxelDataBlock {
public:
RefCount viewers;
VoxelDataBlock() {}
VoxelDataBlock(unsigned int p_lod_index) : _lod_index(p_lod_index) {}
VoxelDataBlock(std::shared_ptr<VoxelBufferInternal> &buffer, unsigned int p_lod_index) :
_voxels(buffer), _lod_index(p_lod_index) {}
@ -39,6 +44,14 @@ public:
return _lod_index;
}
// Tests if voxel data is present.
// If false, it means the block has no edits and does not contain cached generated data,
// so we may fallback on procedural generators on the fly or request a cache.
inline bool has_voxels() const {
return _voxels != nullptr;
}
// Get voxels, expecting them to be present
VoxelBufferInternal &get_voxels() {
#ifdef DEBUG_ENABLED
ZN_ASSERT(_voxels != nullptr);
@ -46,6 +59,7 @@ public:
return *_voxels;
}
// Get voxels, expecting them to be present
const VoxelBufferInternal &get_voxels_const() const {
#ifdef DEBUG_ENABLED
ZN_ASSERT(_voxels != nullptr);
@ -53,6 +67,7 @@ public:
return *_voxels;
}
// Get voxels, expecting them to be present
std::shared_ptr<VoxelBufferInternal> get_voxels_shared() const {
#ifdef DEBUG_ENABLED
ZN_ASSERT(_voxels != nullptr);
@ -65,6 +80,11 @@ public:
_voxels = buffer;
}
void clear_voxels() {
_voxels = nullptr;
_edited = false;
}
void set_modified(bool modified);
inline bool is_modified() const {
@ -90,16 +110,17 @@ public:
private:
std::shared_ptr<VoxelBufferInternal> _voxels;
// TODO Storing lod index here might not be necessary, it is known since we have to get the map first
uint8_t _lod_index = 0;
// The block was edited, which requires its LOD counterparts to be recomputed
// Indicates mipmaps need to be computed since this block was modified.
bool _needs_lodding = false;
// Indicates if this block is different from the time it was loaded (should be saved)
bool _modified = false;
// Tells if the block has ever been edited.
// If `false`, the same data can be obtained by running the generator.
// If `false`, then the data is a cache of generators and modifiers. It can be re-generated.
// Once it becomes `true`, it usually never comes back to `false` unless reverted.
bool _edited = false;

View File

@ -16,7 +16,9 @@ public:
_offset_in_blocks = blocks_box.pos;
blocks_box.for_each_cell_zxy([&map, this](const Vector3i pos) {
VoxelDataBlock *block = map.get_block(pos);
if (block != nullptr) {
// TODO Might need to invoke the generator at some level for present blocks without voxels,
// or make sure all blocks contain voxel data
if (block != nullptr && block->has_voxels()) {
set_block(pos, block->get_voxels_shared());
} else {
set_block(pos, nullptr);

View File

@ -54,7 +54,7 @@ unsigned int VoxelDataMap::get_lod_index() const {
int VoxelDataMap::get_voxel(Vector3i pos, unsigned int c) const {
Vector3i bpos = voxel_to_block(pos);
const VoxelDataBlock *block = get_block(bpos);
if (block == nullptr) {
if (block == nullptr || !block->has_voxels()) {
return _default_voxel[c];
}
RWLockRead lock(block->get_voxels_const().get_lock());
@ -93,7 +93,8 @@ void VoxelDataMap::set_voxel(int value, Vector3i pos, unsigned int c) {
float VoxelDataMap::get_voxel_f(Vector3i pos, unsigned int c) const {
Vector3i bpos = voxel_to_block(pos);
const VoxelDataBlock *block = get_block(bpos);
if (block == nullptr) {
// TODO The generator needs to be invoked if the block has no voxels
if (block == nullptr || !block->has_voxels()) {
// TODO Not valid for a float return value
return _default_voxel[c];
}
@ -105,6 +106,8 @@ float VoxelDataMap::get_voxel_f(Vector3i pos, unsigned int c) const {
void VoxelDataMap::set_voxel_f(real_t value, Vector3i pos, unsigned int c) {
VoxelDataBlock *block = get_or_create_block_at_voxel_pos(pos);
Vector3i lpos = to_local(pos);
// TODO In this situation, the generator must be invoked to fill the block
ZN_ASSERT_RETURN_MSG(block->has_voxels(), "Block not cached");
VoxelBufferInternal &voxels = block->get_voxels();
RWLockWrite lock(voxels.get_lock());
voxels.set_voxel_f(value, lpos.x, lpos.y, lpos.z, c);
@ -159,6 +162,26 @@ VoxelDataBlock *VoxelDataMap::set_block_buffer(
return block;
}
VoxelDataBlock *VoxelDataMap::set_empty_block(Vector3i bpos, bool overwrite) {
VoxelDataBlock *block = get_block(bpos);
if (block == nullptr) {
VoxelDataBlock &map_block = _blocks_map[bpos];
map_block = std::move(VoxelDataBlock(_lod_index));
block = &map_block;
} else if (overwrite) {
block->clear_voxels();
} else {
ZN_PROFILE_MESSAGE("Redundant data block");
ZN_PRINT_VERBOSE(format(
"Discarded block {} lod {}, there was already data and overwriting is not enabled", bpos, _lod_index));
}
return block;
}
bool VoxelDataMap::has_block(Vector3i pos) const {
return _blocks_map.find(pos) != _blocks_map.end();
}
@ -194,7 +217,7 @@ void VoxelDataMap::copy(Vector3i min_pos, VoxelBufferInternal &dst_buffer, unsig
const VoxelDataBlock *block = get_block(bpos);
const Vector3i src_block_origin = block_to_voxel(bpos);
if (block != nullptr) {
if (block != nullptr && block->has_voxels()) {
const VoxelBufferInternal &src_buffer = block->get_voxels_const();
RWLockRead rlock(src_buffer.get_lock());
@ -260,6 +283,9 @@ void VoxelDataMap::paste(Vector3i min_pos, VoxelBufferInternal &src_buffer, unsi
}
}
// TODO In this situation, the generator has to be invoked to fill the blanks
ZN_ASSERT_CONTINUE_MSG(block->has_voxels(), "Area not cached");
const Vector3i dst_block_origin = block_to_voxel(bpos);
VoxelBufferInternal &dst_buffer = block->get_voxels();
@ -306,7 +332,7 @@ bool VoxelDataMap::is_area_fully_loaded(const Box3i voxels_box) const {
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
void preload_box(VoxelDataLodMap &data, Box3i voxel_box, VoxelGenerator *generator) {
void preload_box(VoxelDataLodMap &data, Box3i voxel_box, VoxelGenerator *generator, bool is_streaming) {
ZN_PROFILE_SCOPE();
//ERR_FAIL_COND_MSG(_full_load_mode == false, nullptr, "This function can only be used in full load mode");
@ -333,10 +359,20 @@ void preload_box(VoxelDataLodMap &data, Box3i voxel_box, VoxelGenerator *generat
{
RWLockRead rlock(data_lod.map_lock);
block_box.for_each_cell([&data_lod, lod_index, &todo](Vector3i block_pos) {
block_box.for_each_cell([&data_lod, lod_index, &todo, is_streaming](Vector3i block_pos) {
// We don't check "loading blocks", because this function wants to complete the task right now.
if (!data_lod.map.has_block(block_pos)) {
todo.push_back(Task{ block_pos, lod_index, nullptr });
const VoxelDataBlock *block = data_lod.map.get_block(block_pos);
if (is_streaming) {
// Non-resident blocks must not be touched because we don't know what's in them.
// We can generate caches if resident ones have no voxel data.
if (block != nullptr && !block->has_voxels()) {
todo.push_back(Task{ block_pos, lod_index, nullptr });
}
} else {
// We can generate anywhere voxel data is not in memory
if (block == nullptr || !block->has_voxels()) {
todo.push_back(Task{ block_pos, lod_index, nullptr });
}
}
});
}
@ -386,4 +422,20 @@ void preload_box(VoxelDataLodMap &data, Box3i voxel_box, VoxelGenerator *generat
}
}
void clear_cached_blocks_in_voxel_area(VoxelDataLodMap &data, Box3i p_voxel_box) {
for (unsigned int lod_index = 0; lod_index < data.lod_count; ++lod_index) {
VoxelDataLodMap::Lod &lod = data.lods[lod_index];
RWLockRead rlock(lod.map_lock);
const Box3i blocks_box = p_voxel_box.downscaled(lod.map.get_block_size() << lod_index);
blocks_box.for_each_cell_zxy([&lod](const Vector3i bpos) {
VoxelDataBlock *block = lod.map.get_block(bpos);
if (block == nullptr || block->is_edited() || block->is_modified()) {
return;
}
block->clear_voxels();
});
}
}
} // namespace zylann::voxel

View File

@ -12,9 +12,17 @@ namespace zylann::voxel {
class VoxelGenerator;
// Infinite voxel storage by means of octants like Gridmap, within a constant LOD.
// Sparse voxel storage by means of cubic chunks, within a constant LOD.
//
// Convenience functions to access VoxelBuffers internally will lock them to protect against multithreaded access.
// However, the map itself is not thread-safe.
//
// When doing data streaming, the volume is *partially* loaded. If a block is not found at some coordinates,
// it means we don't know if it contains edits or not. Knowing this is important to avoid writing or caching voxel data
// in blank areas, that may be completely different once loaded.
// When using "full load" of edits, it doesn't matter. If all edits are loaded, we know up-front that everything else
// isn't edited (which also means we may not find blocks without data in them).
//
class VoxelDataMap {
public:
// Converts voxel coodinates into block coordinates.
@ -76,6 +84,7 @@ public:
// Moves the given buffer into a block of the map. The buffer is referenced, no copy is made.
VoxelDataBlock *set_block_buffer(Vector3i bpos, std::shared_ptr<VoxelBufferInternal> &buffer, bool overwrite);
VoxelDataBlock *set_empty_block(Vector3i bpos, bool overwrite);
struct NoAction {
inline void operator()(VoxelDataBlock &block) {}
@ -193,6 +202,7 @@ private:
// To prevent too much hashing, this reference is checked before.
//mutable VoxelDataBlock *_last_accessed_block = nullptr;
// This is block size in VOXELS. To convert to space units, use `block_size << lod_index`.
unsigned int _block_size;
unsigned int _block_size_pow2;
unsigned int _block_size_mask;
@ -218,7 +228,10 @@ struct VoxelDataLodMap {
// Every block intersecting with the box at every LOD will be checked.
// This function runs sequentially and should be thread-safe. May be used if blocks are immediately needed.
// It will block if other threads are accessing the same data.
void preload_box(VoxelDataLodMap &data, Box3i voxel_box, VoxelGenerator *generator);
void preload_box(VoxelDataLodMap &data, Box3i voxel_box, VoxelGenerator *generator, bool is_streaming);
// Clears voxel data from blocks that are pure results of generators and modifiers.
void clear_cached_blocks_in_voxel_area(VoxelDataLodMap &data, Box3i p_voxel_box);
} // namespace zylann::voxel

View File

@ -110,6 +110,8 @@ public:
virtual int get_lod_count() const;
// Should generated blocks be saved immediately? If not, they will be saved only when modified.
// If this is enabled, generated blocks will immediately be considered edited and will be saved to the stream.
// Warning: this is incompatible with non-destructive workflows such as modifiers.
void set_save_generator_output(bool enabled);
bool get_save_generator_output() const;

View File

@ -575,12 +575,15 @@ struct ScheduleSaveAction {
if (block.is_modified()) {
//print_line(String("Scheduling save for block {0}").format(varray(block->position.to_vec3())));
VoxelTerrain::BlockToSave b;
if (with_copy) {
RWLockRead lock(block.get_voxels().get_lock());
b.voxels = make_shared_instance<VoxelBufferInternal>();
block.get_voxels_const().duplicate_to(*b.voxels, true);
} else {
b.voxels = block.get_voxels_shared();
// If a modified block has no voxels, it is equivalent to removing the block from the stream
if (block.has_voxels()) {
if (with_copy) {
RWLockRead lock(block.get_voxels().get_lock());
b.voxels = make_shared_instance<VoxelBufferInternal>();
block.get_voxels_const().duplicate_to(*b.voxels, true);
} else {
b.voxels = block.get_voxels_shared();
}
}
b.position = bpos;
blocks_to_save.push_back(b);
@ -936,8 +939,8 @@ static void request_block_load(uint32_t volume_id, std::shared_ptr<StreamingDepe
init_sparse_grid_priority_dependency(
priority_dependency, block_pos, data_block_size, shared_viewers_data, volume_transform);
LoadBlockDataTask *task = ZN_NEW(LoadBlockDataTask(
volume_id, block_pos, 0, data_block_size, request_instances, stream_dependency, priority_dependency));
LoadBlockDataTask *task = ZN_NEW(LoadBlockDataTask(volume_id, block_pos, 0, data_block_size, request_instances,
stream_dependency, priority_dependency, true));
VoxelServer::get_singleton().push_async_io_task(task);
@ -1471,7 +1474,7 @@ void VoxelTerrain::process_meshing() {
MeshBlockTask *task = ZN_NEW(MeshBlockTask);
task->volume_id = _volume_id;
task->position = mesh_block_pos;
task->lod = 0;
task->lod_index = 0;
task->meshing_dependency = _meshing_dependency;
task->data_block_size = get_data_block_size();
task->collision_hint = _generate_collisions;
@ -1480,7 +1483,7 @@ void VoxelTerrain::process_meshing() {
task->blocks_count = 0;
data_box.for_each_cell_zxy([this, task](Vector3i data_block_pos) {
VoxelDataBlock *data_block = _data_map.get_block(data_block_pos);
if (data_block != nullptr) {
if (data_block != nullptr && data_block->has_voxels()) {
task->blocks[task->blocks_count] = data_block->get_voxels_shared();
}
++task->blocks_count;

View File

@ -106,10 +106,13 @@ struct ScheduleSaveAction {
//print_line(String("Scheduling save for block {0}").format(varray(block->position.to_vec3())));
VoxelLodTerrainUpdateData::BlockToSave b;
b.voxels = make_shared_instance<VoxelBufferInternal>();
{
RWLockRead lock(block.get_voxels().get_lock());
block.get_voxels_const().duplicate_to(*b.voxels, true);
// If a modified block has no voxels, it is equivalent to removing the block from the stream
if (block.has_voxels()) {
b.voxels = make_shared_instance<VoxelBufferInternal>();
{
RWLockRead lock(block.get_voxels().get_lock());
block.get_voxels_const().duplicate_to(*b.voxels, true);
}
}
b.position = bpos;
@ -501,12 +504,16 @@ bool VoxelLodTerrain::is_area_editable(Box3i p_voxel_box) const {
}
inline std::shared_ptr<VoxelBufferInternal> try_get_voxel_buffer_with_lock(
const VoxelDataLodMap::Lod &data_lod, Vector3i block_pos) {
const VoxelDataLodMap::Lod &data_lod, Vector3i block_pos, bool &out_generate) {
RWLockRead rlock(data_lod.map_lock);
const VoxelDataBlock *block = data_lod.map.get_block(block_pos);
if (block == nullptr) {
return nullptr;
}
if (!block->has_voxels()) {
out_generate = true;
return nullptr;
}
return block->get_voxels_shared();
}
@ -528,11 +535,13 @@ VoxelSingleValue VoxelLodTerrain::get_voxel(Vector3i pos, unsigned int channel,
}
Vector3i block_pos = pos >> get_data_block_size_pow2();
bool generate = false;
if (_update_data->settings.full_load_mode) {
const VoxelDataLodMap::Lod &data_lod0 = _data->lods[0];
std::shared_ptr<VoxelBufferInternal> voxels = try_get_voxel_buffer_with_lock(data_lod0, block_pos);
std::shared_ptr<VoxelBufferInternal> voxels = try_get_voxel_buffer_with_lock(data_lod0, block_pos, generate);
if (voxels == nullptr) {
// TODO We should be able to get a value if modifiers are used but not a base generator
if (_generator.is_valid()) {
VoxelSingleValue value = _generator->generate_single(pos, channel);
if (channel == VoxelBufferInternal::CHANNEL_SDF) {
@ -559,10 +568,27 @@ VoxelSingleValue VoxelLodTerrain::get_voxel(Vector3i pos, unsigned int channel,
Vector3i voxel_pos = pos;
for (unsigned int lod_index = 0; lod_index < _update_data->settings.lod_count; ++lod_index) {
const VoxelDataLodMap::Lod &data_lod = _data->lods[lod_index];
std::shared_ptr<VoxelBufferInternal> voxels = try_get_voxel_buffer_with_lock(data_lod, block_pos);
std::shared_ptr<VoxelBufferInternal> voxels = try_get_voxel_buffer_with_lock(data_lod, block_pos, generate);
if (voxels != nullptr) {
return get_voxel_with_lock(*voxels, data_lod.map.to_local(voxel_pos), channel);
} else if (generate) {
// TODO We should be able to get a value if modifiers are used but not a base generator
if (_generator.is_valid()) {
VoxelSingleValue value = _generator->generate_single(pos, channel);
if (channel == VoxelBufferInternal::CHANNEL_SDF) {
float sdf = value.f;
_data->modifiers.apply(sdf, to_vec3(pos));
value.f = sdf;
}
return value;
} else {
return defval;
}
}
// Fallback on lower LOD
block_pos = block_pos >> 1;
voxel_pos = voxel_pos >> 1;
@ -575,9 +601,12 @@ bool VoxelLodTerrain::try_set_voxel_without_update(Vector3i pos, unsigned int ch
const Vector3i block_pos_lod0 = pos >> get_data_block_size_pow2();
VoxelDataLodMap::Lod &data_lod0 = _data->lods[0];
const Vector3i block_pos = data_lod0.map.voxel_to_block(pos);
std::shared_ptr<VoxelBufferInternal> voxels = try_get_voxel_buffer_with_lock(data_lod0, block_pos);
bool can_generate = false;
std::shared_ptr<VoxelBufferInternal> voxels = try_get_voxel_buffer_with_lock(data_lod0, block_pos, can_generate);
if (voxels == nullptr) {
if (!_update_data->settings.full_load_mode) {
if (!_update_data->settings.full_load_mode && !can_generate) {
return false;
}
if (_generator.is_valid()) {
@ -597,6 +626,7 @@ bool VoxelLodTerrain::try_set_voxel_without_update(Vector3i pos, unsigned int ch
// If it turns out to be a problem, use CoW?
RWLockWrite lock(voxels->get_lock());
voxels->set_voxel(value, data_lod0.map.to_local(pos), channel);
// We don't update mips, this must be done by the caller
return true;
}
@ -670,9 +700,12 @@ void VoxelLodTerrain::post_edit_area(Box3i p_box) {
}
}
void VoxelLodTerrain::post_edit_modifiers(Box3i p_box) {
MutexLock lock(_update_data->state.remesh_requests_mutex);
_update_data->state.remesh_requests.push_back(p_box);
void VoxelLodTerrain::post_edit_modifiers(Box3i p_voxel_box) {
clear_cached_blocks_in_voxel_area(*_data, p_voxel_box);
// Not sure if it is worth re-caching these blocks. We may see about that in the future if performance is an issue.
MutexLock lock(_update_data->state.changed_generated_areas_mutex);
_update_data->state.changed_generated_areas.push_back(p_voxel_box);
}
void VoxelLodTerrain::push_async_edit(IThreadedTask *task, Box3i box, std::shared_ptr<AsyncDependencyTracker> tracker) {
@ -851,7 +884,11 @@ void VoxelLodTerrain::reset_maps() {
VoxelLodTerrainUpdateData::State &state = _update_data->state;
// Make a new one, so if threads still reference the old one it will be a different copy
_data = make_shared_instance<VoxelDataLodMap>();
std::shared_ptr<VoxelDataLodMap> new_data = make_shared_instance<VoxelDataLodMap>();
// Keep modifiers, we only reset voxel data
new_data->modifiers = std::move(_data->modifiers);
_data = new_data;
_data->lod_count = lod_count;
for (unsigned int lod_index = 0; lod_index < state.lods.size(); ++lod_index) {
@ -1378,6 +1415,14 @@ void VoxelLodTerrain::apply_data_block_response(VoxelServer::BlockDataOutput &ob
VoxelDataBlock *block = data_lod.map.set_block_buffer(ob.position, ob.voxels, false);
CRASH_COND(block == nullptr);
block->set_edited(ob.type == VoxelServer::BlockDataOutput::TYPE_LOADED);
} else {
// Loading returned an empty block: that means we know the stream does not contain a block here.
// When doing data streaming, we'll generate on the fly if this block is queried.
VoxelDataLodMap::Lod &data_lod = _data->lods[ob.lod];
RWLockWrite wlock(data_lod.map_lock);
VoxelDataBlock *block = data_lod.map.set_empty_block(ob.position, false);
ZN_ASSERT(block != nullptr);
}
{
@ -2223,16 +2268,12 @@ Array VoxelLodTerrain::_b_debug_print_sdf_top_down(Vector3i center, Vector3i ext
const VoxelDataLodMap::Lod &data_lod = _data->lods[lod_index];
world_box.for_each_cell([&data_lod, &buffer, world_box](const Vector3i &world_pos) {
std::shared_ptr<VoxelBufferInternal> voxels =
try_get_voxel_buffer_with_lock(data_lod, data_lod.map.voxel_to_block(world_pos));
if (voxels == nullptr) {
return;
}
const float v =
get_voxel_with_lock(*voxels, data_lod.map.to_local(world_pos), VoxelBufferInternal::CHANNEL_SDF).f;
world_box.for_each_cell([this, world_box, &buffer](const Vector3i &world_pos) {
const Vector3i rpos = world_pos - world_box.pos;
buffer.set_voxel_f(v, rpos.x, rpos.y, rpos.z, VoxelBufferInternal::CHANNEL_SDF);
VoxelSingleValue v;
v.f = 1.f;
v = get_voxel(world_pos, VoxelBufferInternal::CHANNEL_SDF, v);
buffer.set_voxel_f(v.f, rpos.x, rpos.y, rpos.z, VoxelBufferInternal::CHANNEL_SDF);
});
Ref<Image> image = gd::VoxelBuffer::debug_print_sdf_to_image_top_down(buffer);

View File

@ -143,7 +143,7 @@ public:
// These must be called after an edit
void post_edit_area(Box3i p_box);
void post_edit_modifiers(Box3i p_box);
void post_edit_modifiers(Box3i p_voxel_box);
// TODO This still sucks atm cuz the edit will still run on the main thread
void push_async_edit(IThreadedTask *task, Box3i box, std::shared_ptr<AsyncDependencyTracker> tracker);

View File

@ -159,8 +159,9 @@ struct VoxelLodTerrainUpdateData {
BinaryMutex pending_async_edits_mutex;
std::vector<RunningAsyncEdit> running_async_edits;
std::vector<Box3i> remesh_requests;
BinaryMutex remesh_requests_mutex;
// Areas where generated stuff has changed. Similar to an edit, but non-destructive.
std::vector<Box3i> changed_generated_areas;
BinaryMutex changed_generated_areas_mutex;
Stats stats;
};

View File

@ -103,15 +103,18 @@ void VoxelLodTerrainUpdateTask::flush_pending_lod_edits(VoxelLodTerrainUpdateDat
// We should find a way to make it asynchronous, not need mips, or not edit outside viewers area.
std::shared_ptr<VoxelBufferInternal> voxels = make_shared_instance<VoxelBufferInternal>();
voxels->create(Vector3iUtil::create(data_block_size));
VoxelGenerator::VoxelQueryData q{ //
*voxels, //
dst_bpos << (dst_lod_index + data_block_size_po2), //
dst_lod_index
};
if (generator.is_valid()) {
ZN_PROFILE_SCOPE_NAMED("Generate");
VoxelGenerator::VoxelQueryData q{ //
*voxels, //
dst_bpos << (dst_lod_index + data_block_size_po2), //
dst_lod_index
};
generator->generate_block(q);
}
data.modifiers.apply(
q.voxel_buffer, AABB(q.origin_in_voxels, q.voxel_buffer.get_size() << dst_lod_index));
dst_block = dst_data_lod.map.set_block_buffer(dst_bpos, voxels, true);
} else {
@ -121,9 +124,9 @@ void VoxelLodTerrainUpdateTask::flush_pending_lod_edits(VoxelLodTerrainUpdateDat
}
}
// The block and its lower LODs are expected to be available.
// The block and its lower LOD indices are expected to be available.
// Otherwise it means the function was called too late
CRASH_COND(src_block == nullptr);
ZN_ASSERT(src_block != nullptr && src_block->has_voxels());
//CRASH_COND(dst_block == nullptr);
{
@ -177,8 +180,11 @@ struct BeforeUnloadDataAction {
if (save && block.is_modified()) {
//print_line(String("Scheduling save for block {0}").format(varray(block->position.to_vec3())));
VoxelLodTerrainUpdateData::BlockToSave b;
// We don't copy since the block will be unloaded anyways
b.voxels = block.get_voxels_shared();
// We don't copy since the block will be unloaded anyways.
// If a modified block has no voxels, it is equivalent to removing the block from the stream
if (block.has_voxels()) {
b.voxels = block.get_voxels_shared();
}
b.position = bpos;
b.lod = block.get_lod_index();
blocks_to_save.push_back(b);
@ -1053,10 +1059,10 @@ static void init_sparse_octree_priority_dependency(PriorityDependency &dep, Vect
}
static void request_block_generate(uint32_t volume_id, unsigned int data_block_size,
std::shared_ptr<StreamingDependency> &stream_dependency, Vector3i block_pos, int lod,
std::shared_ptr<PriorityDependency::ViewersData> &shared_viewers_data, const Transform3D &volume_transform,
float lod_distance, std::shared_ptr<AsyncDependencyTracker> tracker, bool allow_drop,
BufferedTaskScheduler &task_scheduler) {
std::shared_ptr<StreamingDependency> &stream_dependency, const std::shared_ptr<VoxelDataLodMap> &data,
Vector3i block_pos, int lod, std::shared_ptr<PriorityDependency::ViewersData> &shared_viewers_data,
const Transform3D &volume_transform, float lod_distance, std::shared_ptr<AsyncDependencyTracker> tracker,
bool allow_drop, BufferedTaskScheduler &task_scheduler) {
//
CRASH_COND(data_block_size > 255);
CRASH_COND(stream_dependency == nullptr);
@ -1072,6 +1078,7 @@ static void request_block_generate(uint32_t volume_id, unsigned int data_block_s
task->stream_dependency = stream_dependency;
task->tracker = tracker;
task->drop_beyond_max_distance = allow_drop;
task->data = data;
init_sparse_octree_priority_dependency(task->priority_dependency, block_pos, lod, data_block_size,
shared_viewers_data, volume_transform, lod_distance);
@ -1080,41 +1087,45 @@ static void request_block_generate(uint32_t volume_id, unsigned int data_block_s
}
static void request_block_load(uint32_t volume_id, unsigned int data_block_size,
std::shared_ptr<StreamingDependency> &stream_dependency, Vector3i block_pos, int lod, bool request_instances,
std::shared_ptr<StreamingDependency> &stream_dependency, const std::shared_ptr<VoxelDataLodMap> &data,
Vector3i block_pos, int lod, bool request_instances,
std::shared_ptr<PriorityDependency::ViewersData> &shared_viewers_data, const Transform3D &volume_transform,
float lod_distance, BufferedTaskScheduler &task_scheduler) {
//
CRASH_COND(data_block_size > 255);
CRASH_COND(stream_dependency == nullptr);
// Not an option for now, will wait for it to be really needed
const bool cache_generated_blocks = false;
if (stream_dependency->stream.is_valid()) {
PriorityDependency priority_dependency;
init_sparse_octree_priority_dependency(priority_dependency, block_pos, lod, data_block_size,
shared_viewers_data, volume_transform, lod_distance);
LoadBlockDataTask *task = memnew(LoadBlockDataTask(
volume_id, block_pos, lod, data_block_size, request_instances, stream_dependency, priority_dependency));
LoadBlockDataTask *task = memnew(LoadBlockDataTask(volume_id, block_pos, lod, data_block_size,
request_instances, stream_dependency, priority_dependency, cache_generated_blocks));
task_scheduler.push_io_task(task);
} else {
} else if (cache_generated_blocks) {
// Directly generate the block without checking the stream.
request_block_generate(volume_id, data_block_size, stream_dependency, block_pos, lod, shared_viewers_data,
request_block_generate(volume_id, data_block_size, stream_dependency, data, block_pos, lod, shared_viewers_data,
volume_transform, lod_distance, nullptr, true, task_scheduler);
}
}
static void send_block_data_requests(uint32_t volume_id,
Span<const VoxelLodTerrainUpdateData::BlockLocation> blocks_to_load,
std::shared_ptr<StreamingDependency> &stream_dependency,
std::shared_ptr<StreamingDependency> &stream_dependency, const std::shared_ptr<VoxelDataLodMap> &data,
std::shared_ptr<PriorityDependency::ViewersData> &shared_viewers_data, unsigned int data_block_size,
bool request_instances, const Transform3D &volume_transform, float lod_distance,
BufferedTaskScheduler &task_scheduler) {
//
for (unsigned int i = 0; i < blocks_to_load.size(); ++i) {
const VoxelLodTerrainUpdateData::BlockLocation loc = blocks_to_load[i];
request_block_load(volume_id, data_block_size, stream_dependency, loc.position, loc.lod, request_instances,
shared_viewers_data, volume_transform, lod_distance, task_scheduler);
request_block_load(volume_id, data_block_size, stream_dependency, data, loc.position, loc.lod,
request_instances, shared_viewers_data, volume_transform, lod_distance, task_scheduler);
}
}
@ -1184,7 +1195,7 @@ static void send_mesh_requests(uint32_t volume_id, VoxelLodTerrainUpdateData::St
MeshBlockTask *task = memnew(MeshBlockTask);
task->volume_id = volume_id;
task->position = mesh_block_pos;
task->lod = lod_index;
task->lod_index = lod_index;
task->meshing_dependency = meshing_dependency;
task->data_block_size = data_block_size;
task->data = data_ptr;
@ -1204,13 +1215,13 @@ static void send_mesh_requests(uint32_t volume_id, VoxelLodTerrainUpdateData::St
const VoxelDataBlock *nblock = data_lod.map.get_block(data_block_pos);
// The block can actually be null on some occasions. Not sure yet if it's that bad
//CRASH_COND(nblock == nullptr);
if (nblock != nullptr) {
if (nblock != nullptr && nblock->has_voxels()) {
task->blocks[task->blocks_count] = nblock->get_voxels_shared();
}
++task->blocks_count;
});
init_sparse_octree_priority_dependency(task->priority_dependency, task->position, task->lod,
init_sparse_octree_priority_dependency(task->priority_dependency, task->position, task->lod_index,
mesh_block_size, shared_viewers_data, volume_transform, settings.lod_distance);
task_scheduler.push_main_task(task);
@ -1226,8 +1237,9 @@ static void send_mesh_requests(uint32_t volume_id, VoxelLodTerrainUpdateData::St
// This function schedules one parallel task for every block.
// The returned tracker may be polled to detect when it is complete.
static std::shared_ptr<AsyncDependencyTracker> preload_boxes_async(VoxelLodTerrainUpdateData::State &state,
const VoxelLodTerrainUpdateData::Settings &settings, const VoxelDataLodMap &data, Span<const Box3i> voxel_boxes,
Span<IThreadedTask *> next_tasks, uint32_t volume_id, std::shared_ptr<StreamingDependency> &stream_dependency,
const VoxelLodTerrainUpdateData::Settings &settings, const std::shared_ptr<VoxelDataLodMap> data_ptr,
Span<const Box3i> voxel_boxes, Span<IThreadedTask *> next_tasks, uint32_t volume_id,
std::shared_ptr<StreamingDependency> &stream_dependency,
std::shared_ptr<PriorityDependency::ViewersData> &shared_viewers_data, const Transform3D &volume_transform,
BufferedTaskScheduler &task_scheduler) {
ZN_PROFILE_SCOPE();
@ -1241,6 +1253,8 @@ static std::shared_ptr<AsyncDependencyTracker> preload_boxes_async(VoxelLodTerra
std::vector<TaskArguments> todo;
ZN_ASSERT(data_ptr != nullptr);
VoxelDataLodMap &data = *data_ptr;
const unsigned int data_block_size = data.lods[0].map.get_block_size();
for (unsigned int lod_index = 0; lod_index < settings.lod_count; ++lod_index) {
@ -1288,8 +1302,9 @@ static std::shared_ptr<AsyncDependencyTracker> preload_boxes_async(VoxelLodTerra
for (unsigned int i = 0; i < todo.size(); ++i) {
const TaskArguments args = todo[i];
request_block_generate(volume_id, data_block_size, stream_dependency, args.block_pos, args.lod_index,
shared_viewers_data, volume_transform, settings.lod_distance, tracker, false, task_scheduler);
request_block_generate(volume_id, data_block_size, stream_dependency, data_ptr, args.block_pos,
args.lod_index, shared_viewers_data, volume_transform, settings.lod_distance, tracker, false,
task_scheduler);
}
} else if (next_tasks.size() > 0) {
@ -1301,8 +1316,8 @@ static std::shared_ptr<AsyncDependencyTracker> preload_boxes_async(VoxelLodTerra
}
static void process_async_edits(VoxelLodTerrainUpdateData::State &state,
const VoxelLodTerrainUpdateData::Settings &settings, const VoxelDataLodMap &data, uint32_t volume_id,
std::shared_ptr<StreamingDependency> &stream_dependency,
const VoxelLodTerrainUpdateData::Settings &settings, const std::shared_ptr<VoxelDataLodMap> &data,
uint32_t volume_id, std::shared_ptr<StreamingDependency> &stream_dependency,
std::shared_ptr<PriorityDependency::ViewersData> &shared_viewers_data, const Transform3D &volume_transform,
BufferedTaskScheduler &task_scheduler) {
ZN_PROFILE_SCOPE();
@ -1340,22 +1355,25 @@ static void process_async_edits(VoxelLodTerrainUpdateData::State &state,
}
}
static void process_remesh_requests(
static void process_changed_generated_areas(
VoxelLodTerrainUpdateData::State &state, const VoxelLodTerrainUpdateData::Settings &settings) {
const unsigned int mesh_block_size = 1 << settings.mesh_block_size_po2;
MutexLock lock(state.remesh_requests_mutex);
if (state.remesh_requests.size() == 0) {
MutexLock lock(state.changed_generated_areas_mutex);
if (state.changed_generated_areas.size() == 0) {
return;
}
for (unsigned int lod_index = 0; lod_index < settings.lod_count; ++lod_index) {
VoxelLodTerrainUpdateData::Lod &lod = state.lods[lod_index];
for (auto box_it = state.remesh_requests.begin(); box_it != state.remesh_requests.end(); ++box_it) {
for (auto box_it = state.changed_generated_areas.begin(); box_it != state.changed_generated_areas.end();
++box_it) {
const Box3i &voxel_box = *box_it;
const Box3i bbox = voxel_box.padded(1).downscaled(mesh_block_size << lod_index);
// TODO If there are cached generated blocks, they need to be re-cached or removed
RWLockRead rlock(lod.mesh_map_state.map_lock);
bbox.for_each_cell_zxy([&lod](const Vector3i bpos) {
@ -1367,7 +1385,7 @@ static void process_remesh_requests(
}
}
state.remesh_requests.clear();
state.changed_generated_areas.clear();
}
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
@ -1422,7 +1440,7 @@ void VoxelLodTerrainUpdateTask::run(ThreadedTaskContext ctx) {
flush_pending_lod_edits(state, data, generator, settings.full_load_mode, 1 << settings.mesh_block_size_po2);
// Other mesh updates
process_remesh_requests(state, settings);
process_changed_generated_areas(state, settings);
static thread_local std::vector<VoxelLodTerrainUpdateData::BlockToSave> data_blocks_to_save;
static thread_local std::vector<VoxelLodTerrainUpdateData::BlockLocation> data_blocks_to_load;
@ -1454,7 +1472,7 @@ void VoxelLodTerrainUpdateTask::run(ThreadedTaskContext ctx) {
BufferedTaskScheduler &task_scheduler = BufferedTaskScheduler::get_for_current_thread();
process_async_edits(state, settings, data, _volume_id, _streaming_dependency, _shared_viewers_data,
process_async_edits(state, settings, _data, _volume_id, _streaming_dependency, _shared_viewers_data,
_volume_transform, task_scheduler);
profiling_clock.restart();
@ -1463,7 +1481,7 @@ void VoxelLodTerrainUpdateTask::run(ThreadedTaskContext ctx) {
// It's possible the user didn't set a stream yet, or it is turned off
if (stream_enabled) {
const unsigned int data_block_size = data.lods[0].map.get_block_size();
send_block_data_requests(_volume_id, to_span_const(data_blocks_to_load), _streaming_dependency,
send_block_data_requests(_volume_id, to_span_const(data_blocks_to_load), _streaming_dependency, _data,
_shared_viewers_data, data_block_size, _request_instances, _volume_transform, settings.lod_distance,
task_scheduler);
send_block_save_requests(

View File

@ -10,6 +10,7 @@ int VoxelDataBlockEnterInfo::_b_get_network_peer_id() const {
Ref<gd::VoxelBuffer> VoxelDataBlockEnterInfo::_b_get_voxels() const {
ERR_FAIL_COND_V(voxel_block == nullptr, Ref<gd::VoxelBuffer>());
ERR_FAIL_COND_V(!voxel_block->has_voxels(), Ref<gd::VoxelBuffer>());
std::shared_ptr<VoxelBufferInternal> vbi = voxel_block->get_voxels_shared();
Ref<gd::VoxelBuffer> vb = gd::VoxelBuffer::create_shared(vbi);
return vb;

View File

@ -9,6 +9,7 @@
#endif
#ifdef ZN_DSTACK_ENABLED
// Put this macro on top of each function you want to track in debug stack traces.
#define ZN_DSTACK() zylann::dstack::Scope dstack_scope_##__LINE__(__FILE__, __LINE__, __FUNCTION__)
#else
#define ZN_DSTACK()
@ -37,6 +38,7 @@ struct Frame {
struct Info {
public:
// Constructs a copy of the current stack gathered so far from ZN_DSTACK() calls
Info();
void to_string(FwdMutableStdString s) const;

View File

@ -177,6 +177,7 @@ public:
private:
// Non-static method for scripts because Godot4 does not support binding static methods (it's only
// implemented for primitive types)
// TODO Make it static, it is supported now
String _b_get_simd_level_name(SIMDLevel level);
static void _bind_methods();