Project Grant 2553936

COLLABORATIVE RESEARCH: CONFIGURABLE SMART SPECTRAL IMAGER FOR DATA-COMPRESSION AND CLASSIFICATION -CONTINUOUS INCREASES IN THE SPECTRAL BANDS AND PIXEL COUNT OF HYPERSPECTRAL IMAGING SENSORS HAVE LED TO PROHIBITIVELY LARGE AMOUNTS OF DATA PRODUCED BY THESE SENSORS THAT MUST BE TRANSMITTED, STORED, AND ANALYZED IN THE CLOUD OR ON POWERFUL LOCAL SERVERS. THIS, IN TURN, HAS RESULTED IN INCREASED COMMUNICATION TRAFFIC, LATENCIES AND POWER CONSUMPTION. AS SUCH, THERE IS STRONG INTEREST IN NOVEL SOFTWARE-HARDWARE CO-DESIGN APPROACHES TO (1) SENSE ONLY THE SPECTRAL BANDS THAT ARE RELEVANT TO A GIVEN APPLICATION AND (2) PERFORM MOST OF THE SENSOR DATA PROCESSING AND ANALYSIS AS CLOSE AS POSSIBLE TO THE SENSOR ARRAY. THESE REQUIREMENTS WILL RENDER A DEVICE THAT TRANSMITS ONLY OUTCOMES (E.G., OCCURRENCES OF EVENTS OF INTEREST), THEREBY SIGNIFICANTLY REDUCING COMMUNICATION TRAFFIC AND POWER CONSUMPTION WHILE IMPROVING SPEED, PRIVACY AND PERFORMANCE. SUCH A SOLUTION WOULD OFFER A MUCH BETTER TRADEOFF BETWEEN THE SIZE OF THE DATA THAT MODERN SPECTRAL IMAGERS GENERATE AND THE EFFICIENCY IN EXTRACTING INFORMATION FROM SUCH BIG DATA WITHIN PRACTICAL TIME CONSTRAINTS. TO THE BEST OF THE AUTHORS? KNOWLEDGE, THERE IS NO PRIOR WORK ON SPECTRAL IMAGING THAT SUCCESSFULLY OVERCOMES THE BIG-DATA BOTTLENECK WITHOUT SACRIFICING THE QUALITY OF DECISION MAKING. THE GOALS OF THE PROJECT ARE TO (1) DEVELOP SMART EVENT-BASED SPECTRAL IMAGER (SEBI) WITH ADVANCED SENSING ALGORITHMS THAT PRODUCE APPLICATION-SPECIFIC DYNAMIC CONTROL-VOLTAGE WAVEFORMS TO INSTRUCT A COMMERCIALLY AVAILABLE ELECTRICALLY-TUNABLE SPECTRAL FILTER TO ONLY SENSE IMPORTANT SPECTRAL AND SPATIAL INFORMATION; (2) DESIGN INTELLIGENT ANALOG READOUT INTEGRATED CIRCUITS (IROICS), WHICH ARE TIGHTLY COUPLED WITH THE TUNABLE SPECTRAL FILTER TO ENABLE ON-CHIP IMPLEMENTATION OF THE SENSING ALGORITHMS; AND (3) DEVELOP NOVEL MACHINE LEARNING (ML) MODELS AND OPTIMIZE THEM FOR DEPLOYMENT ON AN EDGE DEVICE, TIGHTLY COUPLED WITH THE IROIC CIRCUITS AND THE TUNABLE FILTER IT CONTROLS, ENABLING HIGH PERFORMANCE HYPERSPECTRAL OBJECT CLASSIFICATION. THIS PROJECT INTRODUCES NOVEL CONTRIBUTIONS IN THE AREA OF HYPERSPECTRAL IMAGING, WITH SPECIFIC TRANSFORMATIONAL CONTRIBUTIONS ON COMPRESSION AND CLASSIFICATION OF HYPERSPECTRAL IMAGES AND ON PIXEL-LEVEL THEMATIC CLASSIFICATION. IT IS HEAVILY INTERDISCIPLINARY AS IT COMBINES IMAGE PROCESSING, INTEGRATED CIRCUIT DESIGN, ALGORITHM DEVELOPMENT, ML MODEL DEVELOPMENT AND OPTIMIZATION USING EDGE AI, AND SYSTEM INTEGRATION. SUCCESSFUL COMPLETION OF THIS PROJECT WILL POTENTIALLY HAVE A BROAD IMPACT ON A MULTITUDE OF APPLICATION AREAS, INCLUDING MEDICAL DIAGNOSTICS, DEFENSE AND SECURITY, AND PRECISION AGRICULTURE. THE PROJECT ALSO INCLUDES AN EDUCATION AND OUTREACH PLAN. HENCE, ADDITIONAL BROADER IMPACTS INCLUDE DISSEMINATION OF TECHNICAL INFORMATION THROUGH PEER-REVIEWED CONFERENCE AND JOURNAL PUBLICATIONS AND RELEASE OF HARDWARE DESIGN FILES AND SOFTWARE TOOLS. IN ADDITION, HIGH-SCHOOL STUDENTS WILL BE REACHED OUT TO AND INVOLVED IN SUMMER RESEARCH ON TASKS OF THIS PROJECT. EDUCATIONAL MATERIALS RELEVANT TO THE RESEARCH AGENDA WILL BE DEVELOPED AND DEPLOYED IN THE CLASSROOM IN COURSES THAT THE INVESTIGATORS TEACH. THIS AWARD REFLECTS NSF'S STATUTORY MISSION AND HAS BEEN DEEMED WORTHY OF SUPPORT THROUGH EVALUATION USING THE FOUNDATION'S INTELLECTUAL MERIT AND BROADER IMPACTS REVIEW CRITERIA.- SUBAWARDS ARE NOT PLANNED FOR THIS AWARD.