Apple Research Method Scales 3D Latent Detail at Inference

Apple researchers have introduced split-and-scale, a method designed to increase the latent capacity of 3D representations during inference without requiring the same expansion during training. The NeurIPS paper targets the loss of fine detail in complex 3D objects while addressing training and memory constraints.
- Apple researchers introduced the split-and-scale method for latent 3D representations.
- The method expands latent capacity at inference while keeping training costs tractable.
- A pretrained tokenizer can be applied to smaller 3D scene sub-regions.
- Apple reports quality competitive with larger models exceeding single-GPU memory limits.
- The method also supports an autoregressive 3D generator.
A way to add capacity during inference
Latent 3D representations provide compact ways to encode 3D scenes and support 3D generative models. Apple’s researchers say existing approaches can struggle with the fine details of complex objects because their latents have limited capacity.
Increasing that capacity during training can be computationally expensive and may be infeasible under memory limits. Simply increasing the number of latents during inference does not necessarily use the extra capacity effectively, according to the paper.
Split-and-scale addresses both issues by expanding latent capacity at inference while keeping training costs tractable. The method is based on representing 3D scenes as continuous functions of 3D coordinates. This allows a single pretrained tokenizer to be applied to arbitrarily small sub-regions of a scene during inference.
Smaller regions, more detail
Applying the tokenizer to smaller spatial regions gives each region the same latent capacity over a narrower area. Apple’s researchers say this can capture substantially more detail than applying the capacity across a larger scene area.
The paper evaluates the approach through extensive ablations examining the trade-offs among compute, memory, latent capacity and reconstruction quality. According to the authors, the inference-time scaling method achieves quality competitive with much larger models that exceed the memory limits of a single GPU.
This approach separates the cost of preparing a model from the capacity used when a scene is reconstructed. The source does not describe split-and-scale as a replacement for training larger models in every setting, but presents it as a way to increase effective capacity at inference without making the training process similarly expensive.
Support for generative 3D modeling
Apple’s researchers also tested split-and-scale for generative modeling. They trained an autoregressive 3D generator and report that it performs competitively with state-of-the-art baselines.
That result extends the method beyond reconstruction quality. The paper’s experiments indicate that the representation can also serve as the basis for a generator that models 3D content, while retaining the inference-time scaling idea.
The work, titled Split-and-Scale Latent 3D Representations, was published by Apple in November 2026 as a NeurIPS paper. Its authors are Dorian Chan, Muhammed Kocabas, Xiaoming Zhao, Oncel Tuzel and Jen-Hao Rick Chang.
Why the research matters
For 3D reconstruction and generation, the method addresses a central tension between detail, model size, computation and memory. The source describes a path to use higher effective latent capacity when needed, without requiring a correspondingly larger model during training.
For users of 3D systems, the reported findings could support higher-detail scene representations under single-GPU memory limits. The paper remains research presented by Apple, and the source does not announce a product, commercial release or general availability.
Apple: Split-and-Scale Latent 3D Representations

