Project Grant R43GM154526

HARNESSING MACHINE LEARNING & OBJECT DETECTION FOR AUTOMATED EVALUATION OF STUDENT-FOLDED PROTEIN MODELS - PROJECT SUMMARY HARNESSING MACHINE LEARNING & OBJECT DETECTION FOR AUTOMATED EVALUATION OF STUDENT-FOLDED PROTEIN MODELS THIS PROJECT WILL IMPACT PUBLIC HEALTH THROUGH IMPROVED BIOSCIENCE EDUCATION. MINI-TOOBERS ARE ONE OF 3D MOLECULAR DESIGNS' (3DMD) MOST INNOVATIVE, POPULAR, AND EFFECTIVE MODELING TOOLS. THIS FREE-FORM MODELING MATERIAL (FOAM- COVERED WIRE) TYPICALLY REPRESENTS THE BACKBONE OF A FOLDED PROTEIN, ALLOWING STUDENTS TO MODEL ACTUAL PROTEIN STRUCTURES. THE PROBLEM IS THAT MINI-TOOBERS LACK AN IMMEDIATE FEEDBACK SYSTEM. AS A FREE-FORM MODELING TOOL, STUDENTS CAN USE MINI-TOOBERS TO MODEL BIOLOGICAL STRUCTURES CORRECTLY JUST AS EASILY AS INCORRECTLY (RIGHT VS. LEFT-HANDED HELICES). 3DMD'S OTHER MODELING TOOLS HAVE FEEDBACK FEATURES LIKE MAGNETS, ARROWS, POST/HOLE CONNECTIONS, ETC. THAT HELP STUDENTS SELF-CORRECT. EDUCATORS OBSERVE STUDENT'S USE OF MODELS TO SEE THEIR UNDERSTANDING OF A CONCEPT; THIS OBSERVATION IS A TYPE OF FORMATIVE ASSESSMENT. HOWEVER, THERE IS OFTEN TOO MUCH TO OBSERVE IN A MINI-TOOBER MODEL (SHAPE, SIDECHAINS, SECONDARY STRUCTURES, ETC.) FOR AN EDUCATOR TO ASSESS ALL THE DATA IN REAL-TIME EFFECTIVELY. OUR SOLUTION IS TO APPLY MACHINE LEARNING AND OBJECT DETECTION TO CREATE DIGITAL APPLICATIONS THAT DELIVER IMMEDIATE FEEDBACK AND PRECISE ASSESSMENT OF STUDENT-FOLDED MINI-TOOBER PROTEIN MODELS. TWO APPLICATIONS WILL BE CREATED: 1) STUDENT TRAINING APP. THE APP'S AUGMENTED REALITY FEATURES WILL GIVE STUDENTS IMMEDIATE FEEDBACK ON THEIR MODELING OF ALPHA HELICES, BETA SHEETS, AND OTHER PROTEIN MOTIFS TO HELP THEM LEARN PROTEIN STRUCTURE AND FUNCTION. 2) ASSESSMENT APP. THIS PASSWORD-PROTECTED APP WILL ASSESS THE MODEL'S ACCURACY, GENERATE A NUMERICAL SCORE, AND IDENTIFY INACCURATE REGIONS OF THE PROTEIN. THIS PHASE I PROPOSAL FOCUSES ON TWO SPECIFIC AIMS NECESSARY TO ESTABLISH THE FEASIBILITY OF ACHIEVING THIS GOAL: 1. DEVELOP AI TECHNOLOGY TO LOCATE STUDENT-FOLDED ZINC FINGER PROTEIN MODELS WITH AN 80% AVERAGE PRECISION RATING. 2. CREATE A DATA GATHERING PLAN TO COLLECT THE 3D COORDINATES OF STUDENT-FOLDED PROTEIN MODELS. WE WILL FOCUS OUR RESEARCH AND DEVELOPMENT EFFORTS ON FEASIBILITY TESTING MACHINE LEARNING FRAMEWORKS, INCLUDING TENSORFLOW, PYTORCH, AND UNITY BARRACUDA; ON COMPUTER VISION LIBRARIES SUCH AS OPENCV; AND ON OBJECT DETECTION AND CONVERSION SYSTEMS, INCLUDING YOLO AND ONNX TO ACHIEVE THESE SPECIFIC AIMS. ONE OF THE MOST POWERFUL ASPECTS OF OBJECT DETECTION IS THE ABILITY TO IDENTIFY VARIOUS SHAPES AND FORMS. THIS PARTICULARLY APPLIES TO 3DMD'S MINI-TOOBER PROTEIN MODELING MATERIALS, AS THEY CAN BE FOLDED INTO VARIOUS STRUCTURES DEPENDING ON HOW A STUDENT INTERACTS WITH THE MATERIAL. WE WILL COLLECT STUDENT-FOLDED ZINC FINGER MODELS FROM OUTREACH PROGRAMS RUN BY OTHER 3DMD STAFF AND PARTNERING ORGANIZATIONS. TO TEST DIFFERENT APPROACHES, WE WILL SCAN THE MODEL WITH THE DEVELOPED APP WHILE RECORDING VIDEO FOR DOCUMENTATION. THE TEAM WILL CONTINUE TO DEVELOP AND REFINE THE APPLICATION UNTIL AT LEAST AN 80% AVERAGE PRECISION RATING IS ACHIEVED. WE WILL THEN CREATE A DATA-COLLECTION PLAN TO DOCUMENT ENOUGH MODELS AT SCIENCE OLYMPIAD COMPETITIONS TO BUILD THE ASSET LIBRARY NEEDED FOR THE NEXT PHASE OF THIS PROJECT.