Implementing Sparc3D

Using Sparc3D's paper as a reference.

Repository

Introduction

I was excited about the results of Sparc3D (arXiv, github.io) and wanted to break into the world of full 3D mesh reconstruction. However, they had not released their code when I published this, so I set about implementing and understanding it myself (2025-August-20).

Part of their paper details a fast method to obtain Sign distance fields (SDF) on a regular grid of size 1024 x 1024 x 1024 voxels, denoted SDF-1024$^3$. They report under 30s so this was my main target. Later I will analyse the additional components they report.

Goals

SDF-1024$^3$ in under

• 16GiB of VRAM
• 30s

Implementation

I used NVIDIA’s Kaolin library to handle the Unsigned distance field (UDF) calculations and also Marching cubes. However to hit the reported <30 s, I had to resolve the UDF heirarchically (more on this later). To implement the flood-fill algorithm I used scipy.ndimage.label which despite being CPU bound is VERY performant, and I was very happy with this part! Finally, I had to make modifications to FlexiCubes as the original implementation calculates the surface sparsely, but relies on evaluating a dense (D,H,W) tensor of floating point SDFs for querying cube neighbours. While fast, it was not memory efficient and I found myself constantly getting OutOfMemory exceptions. This was the painful part.

Heirarchical resolution of the UDF

Insight: why calculate the UDF-1024$^3$ for all 1025$^3$ grid points when almost all are discarded in Marching cubes.\

The algorithm calculates a coarse UDF (at resolution[0]), then refines only the voxels which have a vertex with a distance at or below the threshold. This can then be repeated an abitrary number of times as long as the next resolution is a multiple of the previous.

args: resolutions [list<int>]

res = Pop(resolutions)

grid = SparseGrid(res, mask=None)
udf = ComputeUDF(grid)
threshold = udf <= sqrt(3) / res

FOR res in resolutions:
    mask = udf < threshold
    grid = SparseGrid(res, mask=mask)
    udf = ComputeUDF(grid)
    threshold = udf <= sqrt(3) / res

return udf

Sparc3D analysis

• Converting the UDF to SDF via the flood-fill labelling is the best contribution in my opinion.
• The high quality results come entirely from the the rendering refinement, not from the initial SDF. This is just a fast initialiser.
• Displacing the vertex with the UDF gradient adds unnecessary complexity.
• The triangle hole patching trick makes me think they had aggressive surface thresholds or had to compensate for the displacement issue

Why is the displacement bad?

1. Memory. The torch.gradient operation in 3D takes more than 16GB for a 1024$^3$ tensor.
2. Introduces a hyperparameter.
3. Gradient calculation is unstable.
4. Not necessary if you are doing rendering refinement.
   See figure 1. The level-set of SDF-512$^3$ with and without $\eta$ of $10^{-2}$.

Left. Mesh resolved from SDF-512$^3$ with no displacement. Right. Mesh resolved from SDF-512$^3$ with displacement of -$10^{-2}\nabla UDF$