A Foundation Model for Light Fields: Masked Spatial-Angular Transformer Pretraining (LF-MAE)
- Posted
- Server
- Preprints.org
- DOI
- 10.20944/preprints202607.0808.v1
Deep light-field models are commonly trained for a single supervised task such as depth estimation, spatial super-resolution, angular super-resolution, or novel-view synthesis. This narrow training paradigm limits transfer because labeled light-field datasets are relatively small, task-specific, and expensive to acquire. We propose LF-MAE (light field masked autoencoders), a self-supervised masked autoencoding framework that learns reusable 4D spatial-angular representations from unlabeled light fields. LF-MAE divides each sub-aperture image into spatial patches, attaches separable angular and spatial positional embeddings, hides a large subset of tokens, and trains an asymmetric transformer encoder-decoder to reconstruct the missing light-field content. Unlike ordinary image MAE (masked autoencoders) pretraining, LF-MAE introduces light-field-specific masking modes: random spatial-angular patch masking, full angular-view masking, EPI-line masking, disparity-band masking, and refocus-plane masking. These objectives force the model to infer parallax, occlusion, disparity slope, and refocusing cues rather than merely local image texture. The resulting encoder can be fine-tuned for depth, spatial or angular super-resolution, segmentation, material recognition, and active acquisition.