Project Grant 2403122

COLLABORATIVE RESEARCH: CIF: MEDIUM: SNAPSHOT COMPUTATIONAL IMAGING WITH METAOPTICS -LIGHT INTERACTS RICHLY WITH MATERIALS IN THE WORLD ALONG ITS MANY AXES, INCLUDING SPATIAL, TEMPORAL, ANGULAR, SPECTRAL, AND POLARIZATION. MEASURING THESE INTERACTIONS AT A FINE-GRAINED SCALE IS A KEY ENABLING TECHNOLOGY IN NUMEROUS SCIENTIFIC ENDEAVORS, INCLUDING LIFE SCIENCES, REMOTE SENSING, SECURITY AND FORENSICS, AND AUGMENTED AND VIRTUAL REALITY. YET, TRADITIONAL IMAGE SENSORS CAPTURE ONLY TWO-DIMENSIONAL SPATIAL VARIATIONS. THEREFORE, OTHER PROPERTIES OF LIGHT ARE MEASURED BY EITHER EMBEDDING THEM WITHIN THE SPATIAL DIMENSIONS (E.G., BAYER-COLOR OR POLARIZATION-FILTER MOSAICS SPATIALLY SPREAD OVER THE SENSOR) OR CAPTURING THEM SEQUENTIALLY (E.G., ACQUIRING CONSECUTIVE VIDEO FRAMES IN THE TIME DIMENSION OR CONSECUTIVE HYPERSPECTRAL COMPONENTS IN THE SPECTRAL DIMENSION). HOWEVER, SNAPSHOT APPROACHES THAT USE SPATIAL TILING, WHILE SIMPLE, LEAD TO ALIASING ARTIFACTS AND INVARIABLY REQUIRE EXPENSIVE MANUFACTURING TECHNIQUES (BONDING COLOR/POLARIZATION FILTERS TO THE SENSOR ARRAY). ON THE OTHER HAND, SEQUENTIAL MEASUREMENTS ENTAIL MOTION ARTIFACTS AND LOWER FRAME RATES. IN CONTRAST, THIS PROJECT DEVELOPS SNAPSHOT COMPUTATIONAL CAMERAS FOR CAPTURING INFORMATION ALONG LIGHT'S VARIOUS DIMENSIONS BY LEVERAGING RECENT ADVANCES IN METAOPTICS, I.E., OPTICAL DEVICES THAT USE SUB-WAVELENGTH NANO-STRUCTURES TO MANIPULATE LIGHT CHARACTERISTICS - SUCH AS PHASE, WAVELENGTH, AMPLITUDE, OR POLARIZATION - WITH A DEGREE OF CONTROL NOT FEASIBLE IN TRADITIONAL REFRACTIVE OPTICS. THE PROJECT FOCUSES ON DEVELOPING METAOPTICS-BASED IMAGING SYSTEMS WITH FREQUENCY-DOMAIN MULTIPLEXING INSTEAD OF SPATIAL TILING OR SEQUENTIAL IMAGING. SUCH FREQUENCY-MULTIPLEXED TECHNIQUES REQUIRE MINIMAL CHANGES TO EXISTING IMAGING SYSTEMS WHILE ENABLING SNAPSHOT MEASUREMENTS OF MULTIPLE DIMENSIONS WITH MINIMAL ALIASING ARTIFACTS. THE PROJECT WILL FOCUS ON THREE MAIN OBJECTIVES TO ACHIEVE SNAPSHOT COMPUTATIONAL-IMAGING SYSTEMS. THE FIRST OBJECTIVE IS TO BUILD SIMULATORS BASED ON RENDERING ALGORITHMS FOR COMPUTATIONAL CAMERAS, AS WELL AS COMBINATIONS AND DIFFERENTIABLE VERSIONS OF SUCH CAMERAS, TO EFFICIENTLY SIMULATE THE VARIOUS DIMENSIONS OF LIGHT, INCLUDING TIME-OF-FLIGHT, SPECTRUM, AND POLARIZATION. IN TANDEM, A SCALABLE, DIFFERENTIABLE METAOPTICS SIMULATOR WILL BE BUILT THAT CAN HANDLE WAVE EFFECTS AS LIGHT INTERACTS WITH THE METAOPTICAL NANO-STRUCTURES WHICH ARE SMALLER THAN THE WAVELENGTH OF THE LIGHT. THE SECOND OBJECTIVE IS TO DESIGN FREQUENCY-MULTIPLEXED SNAPSHOT CAMERAS BY LEVERAGING THE SIMULATORS DEVELOPED IN THE FIRST OBJECTIVE, LEADING TO METAOPTICS-BASED CAMERAS THAT ENABLE CAPTURING INTENSITY OVER LARGE DEPTHS OF FIELD WITH HIGH NUMERICAL APERTURE, THEREBY ACHIEVING COMPACT IMAGING SYSTEMS WITH LOW OPERATIONAL POWER. THE THIRD OBJECTIVE IS TO DEMONSTRATE THE ADVANTAGES OF SNAPSHOT CAMERAS BY DESIGNING AND BUILDING LAB PROTOTYPES AND COMPARING THEM AGAINST CURRENT STATE-OF-THE-ART IMAGERS. THE OUTCOMES OF THIS PROJECT WILL IMPACT VARIOUS DISCIPLINES, INCLUDING COMPUTER GRAPHICS, OPTICS, COMPUTATIONAL IMAGING, AND BIOMEDICAL IMAGING. THIS AWARD REFLECTS NSF'S STATUTORY MISSION AND HAS BEEN DEEMED WORTHY OF SUPPORT THROUGH EVALUATION USING THE FOUNDATION'S INTELLECTUAL MERIT AND BROADER IMPACTS REVIEW CRITERIA.- SUBAWARDS ARE NOT PLANNED FOR THIS AWARD.