Fourier lightfield multiview stereoscope for large field-of-view 3D imaging in microsurgical settings

We present the Fourier lightfield multiview stereoscope (FiLM-Scope). This imaging device combines concepts from Fourier lightfield microscopy and multiview stereo imaging to capture high-resolution 3D videos over large fields of view. The FiLM-Scope optical hardware consists of a multicamera array, with 48 individual microcameras, placed behind a high-throughput primary lens. This allows the FiLM-Scope to simultaneously capture 48 unique 12.8 megapixel images of a 28×37mm field-of-view, from unique angular perspectives over a 21deg×29deg range, with down to 22μm lateral resolution. We also describe a self-supervised algorithm to reconstruct 3D height maps from these images. Our approach demonstrates height accuracy down to 11μm. To showcase the utility of our system, we perform tool tracking over the surface of an ex vivorat skull and visualize the 3D deformation in stretching human skin, with videos captured at up to 100 frames per second. The FiLM-Scope has the potential to improve 3D visualization in a range of microsurgical settings.