HR001119S0005.pdf

Broad Agency Announcement Machine Common Sense (MCS)

HR001119S0005

October 19, 2018

Defense Advanced Research Projects Agency Information Innovation Office 675 North Randolph Street Arlington, VA 22203-2114

HR001119S0005 MACHINE COMMON SENSE 2

Table of Contents

I. Funding Opportunity Description

A. Introduction/Background

B. Program Description/Scope

C. Technical Areas (TAs)

TA1: Foundations of Human Common Sense TA2: Test Environment for the Foundations of Human Common Sense TA3: Broad Common Knowledge

D. Schedule/Milestones

E. TA-specific Deliverables

F. Government-furnished Property/Equipment/Information

G. Intellectual Property

H. References

II. Award Information

A. Awards

B. Fundamental Research

C. Disclosure of Information and Compliance with Safeguarding Covered Defense Information Controls

III. Eligibility Information

A. Eligible Applicants

B. Organizational Conflicts of Interest

C. Cost Sharing/Matching

D. Other Eligibility Requirements

IV. Application and Submission Information

A. Address to Request Application Package

B. Content and Form of Application Submission

C. Submission Dates and Times

D. Funding Restrictions

E. Other Submission Requirements

V. Application Review Information

A. Evaluation Criteria

B. Review and Selection Process

VI. Award Administration Information

A. Selection Notices

HR001119S0005 MACHINE COMMON SENSE 3

B. Administrative and National Policy Requirements

C. Reporting

VII. Agency Contacts

VIII. Other Information

A. Frequently Asked Questions (FAQs)

B. Submission Checklist

HR001119S0005 MACHINE COMMON SENSE 4

PART I: OVERVIEW INFORMATION

Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Information Innovation Office (I2O)

Funding Opportunity Title: Machine Common Sense (MCS)

Announcement Type: Initial Announcement

Funding Opportunity Number: HR001119S0005

Catalog of Federal Domestic Assistance Numbers (CFDA):

12.910 Research and Technology Development

Dates o Proposers Day: October 18, 2018 o Posting Date: October 19, 2018 o Abstract Due Date: November 6, 2018, 12:00 noon (ET) o Proposal Due Date: December 18, 2018, 12:00 noon (ET) o BAA Closing Date: December 18, 2018, 12:00 noon (ET)

Anticipated Individual Awards: DARPA anticipates multiple awards for technical areas (TAs) 1 and 3, and a single award for TA2

Types of Instruments that May be Awarded: Procurement contracts, cooperative agreements, or Other Transactions

Agency Contacts o Technical POC: David Gunning, Program Manager, DARPA/I2O o BAA Email: mcs@darpa.mil o BAA Mailing Address:

DARPA/I2O

ATTN: HR001119S0005

675 North Randolph Street Arlington, VA 22203-2114 o I2O Solicitation Website: http://www.darpa.mil/work-with-us/opportunities mailto:mcs@darpa.mil http://www.darpa.mil/work-with-us/opportunities

HR001119S0005 MACHINE COMMON SENSE 5

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

DARPA is soliciting innovative research proposals in the area of machine common sense to enable Artificial Intelligence (AI) applications to understand new situations, monitor the reasonableness of their actions, communicate more effectively with people, and transfer learning to new domains. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

This Broad Agency Announcement (BAA) is being issued, and any resultant selection will be made, using procedures under Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016.

Any negotiations and/or awards will use procedures under FAR 15.4 (or 32 CFR § 200.203 for cooperative agreements). Proposals received as a result of this BAA shall be evaluated in accordance with evaluation criteria specified herein through a scientific review process.

DARPA BAAs are posted on the Federal Business Opportunities (FBO) website (https://www.fbo.gov/) and the Grants.gov website (http://www.grants.gov/).

The following information is for those wishing to respond to this BAA.

A. Introduction/Background

Machine common sense has long been a critical—but missing—component of AI. Recent advances in machine learning have resulted in exciting new capabilities, but machine reasoning remains narrow and highly specialized. Developers must carefully train or program systems for every situation. General machine common sense remains elusive.

Wikipedia defines common sense as, the basic ability to perceive, understand, and judge things that are shared by ("common to") nearly all people and can reasonably be expected of nearly all people without need for debate. It is common sense that helps us quickly answer the question, “can an elephant fit through the doorway?”; or understand the statement, “I saw the Grand Canyon flying to New York.” The vast majority of common sense is typically not expressed by humans because there is no need to state the obvious. We are usually not conscious of the vast sea of commonsense assumptions that underlie every statement and every action. This unstated background knowledge includes: a general understanding of how the physical world works (i.e., intuitive physics); a basic understanding of human motives and behaviors (i.e., intuitive psychology); and knowledge of the common facts that an average adult possesses. Machines lack this basic background knowledge that all humans share. The obscure-but-pervasive nature of common sense makes it difficult to articulate and encode in machines.

The absence of common sense prevents intelligent systems from understanding their world, behaving reasonably in unforeseen situations, communicating naturally with people, and learning from new experiences. Its absence is perhaps the most significant barrier between the narrowly focused AI applications we have today and the more general, human-like AI systems we would like to build in the future. If successful, the MCS program could accelerate the development of https://www.fbo.gov/ http://www.grants.gov/

HR001119S0005 MACHINE COMMON SENSE 6

AI for both defense and commercial applications. Here are four use cases that apply to single AI applications, symbiotic human-machine partnerships, and autonomous systems:

Sensemaking – any AI system that needs to analyze and interpret sensor or data input could benefit from a machine commonsense service to help it understand real-world scenes and situations.

Monitoring the reasonableness of machine actions – a machine commonsense service would provide the ability to monitor and check the reasonableness of any AI system’s actions and decisions, especially in novel situations.

Human-machine collaboration – all human communication and understanding assumes a background of common sense. Machines need to have a basic level of human-like common sense to effectively communicate and collaborate with humans.

Transfer learning (adapting to new situations) – reusable commonsense knowledge would provide a foundation for AI systems to learn new domains and adapt to new situations without voluminous specialized training or programming.

Since the earliest days of AI, researchers have admired the need for, and struggled to achieve, machine common sense. This began with a variety of efforts to develop logic-based approaches to commonsense knowledge and reasoning (e.g., situation calculus, frames, scripts, naïve physics, default reasoning, non-monotonic logics, description logics, qualitative reasoning) and a number of efforts to create logic-based ontologies (e.g., Cyc, WordNet, VerbNet, SUMO, YAGO, DOLCE, and hundreds of smaller ontologies on the Semantic Web). These efforts have produced useful results, but suffer from the brittleness of symbolic logic. Concepts are defined in black or white symbols, which never quite match the subtleties of the human concepts they are intended to represent. Similarly, natural language queries never quite match the precise symbolic concepts. More recently, as machine learning and crowdsourcing have come to dominate AI, those techniques have also been used to extract and collect commonsense knowledge from the Web. While these approaches continue to make significant progress, they generally lack sufficient semantic understanding to enable reasoning beyond simple answer lookup. Overall, all of these approaches have fallen short of producing a widely useful commonsense capability. Machine common sense has remained a difficult and unbounded problem.

However, there has been significant progress in recent years along a number of dimensions that make it possible to address this difficult problem now. Rapid progress in machine learning, especially deep learning, is producing fresh representations and innovative techniques for semi-supervised, self-supervised, and unsupervised learning. This progress has created a resurgence of new researchers who are attempting to take on the common sense problem by learning grounded, semantic representations, such as embeddings; developing commonsense knowledge from the language, vision, and robotic interactions; learning predictive models from experience, by training systems to predict the next event in video sequences; moving beyond supervised classification to create more complete learning systems capable of memory, ‘mental’ simulation, and multi-step reasoning; as well as understanding and modeling the foundations of human cognition from developmental psychology.

B. Program Description/Scope

To focus this new effort, the MCS program will target two diverse strategies for developing and evaluating two different machine commonsense services:

HR001119S0005 MACHINE COMMON SENSE 7

Foundations of Human Common Sense: a service that learns from experience, like a child, to construct computational models that mimic the core domains of cognition for objects (intuitive physics), places (spatial navigation), and agents (intentional actors).

These models will be evaluated against the cognitive development milestones as evidenced in developmental psychology experiments with human children from 0-18 months old.

Broad Common Knowledge: a service that learns from reading the Web, like a research librarian, to construct a commonsense knowledge repository capable of answering natural language and image-based questions about commonsense phenomena. This service will mimic the general knowledge of an average American adult in 2018, and be evaluated against the Allen Institute for Artificial Intelligence (AI2) Common Sense Benchmarks.

Figure 1: Machine Commonsense Services

C. Technical Areas (TAs)

TA1: Foundations of Human Common Sense

The goal of TA1 is to develop computational models that mimic the core cognitive capabilities of children, 0-18 months old. Researchers who study childhood cognition now have years of experimental results that allow them to map out the cognitive capacities of children. The field of cognitive development is at a point where it can provide empirical and theoretical guidance for building intelligent machines that think and learn like children. In particular, developmental psychologists have intensively studied children's knowledge in six domains (Table 1). Some believe that each of these domains constitutes a distinct and relatively autonomous system of knowledge, an idea that has been codified in the Theory of Core Knowledge. Others believe that these domains interact from the beginning of life. Developmental psychologists agree, however, that abilities to reason about objects, agents, places, number, geometry, and the social world, as described in the Theory of Core Knowledge, emerge early and serve as crucial foundations for later learning. See references [1][2][3][4].

HR001119S0005 MACHINE COMMON SENSE 8

Domain Description Objects supports reasoning about objects and the laws of physics Agents supports reasoning about agents that act autonomously to pursue goals Places supports navigation and spatial reasoning around an environment

Number supports reasoning about quantity and how many things are present Geometry supports representation of shapes and their affordances

Social World supports reasoning about Theory of Mind and social interactions

Table 1: Theory of Core Knowledge

These core domains likely form the fundamental building blocks of human intelligence and common sense, especially the core domains of objects (intuitive physics), agents (intentional actors), and places (spatial navigation). For example, the core domain of objects not only provides the fundamental concepts for understanding the physical world but also provides the foundation for understanding causality. The core domain of agents not only provides the fundamental concepts for understanding intentional actors and Theory of Mind (TOM) but also provides the foundation for dealing with the “frame problem” in AI (i.e., knowing that objects in a scene only change if acted on by an agent). The core domain of places not only provides the fundamental concepts for navigation but also provides the foundation for spatial memory and spatial reasoning.

As a result of years of research and new experimental techniques and tools, developmental psychologists are now able to map the cognitive capacities of children. Figure 2 illustrates key stages in the current understanding of the developmental sequence for three core domains of objects, agents, and places for children from 0 to 18 months. This sequence provides an excellent set of target milestones for AI researchers to mimic as a strategy for developing a new foundation for machine common sense. While these milestones are particularly useful, these are just a selection of those the literature suggests. Also, research in development is ongoing and it is helpful to consider Figure 2 as including “error bars” on both the columns (time of acquisition) and rows (the conceptual split and grouping of the abilities and understandings of children).

Note that TA1 computational models are not expected to duplicate the exact developmental sequence of these milestones in children, but they are expected to duplicate the cognitive capabilities of an 18-month old by the end of the program.

HR001119S0005 MACHINE COMMON SENSE 9

Figure 2: Cognitive Development Milestones (0-18 months)

Multiple development teams will be selected to construct these TA1 computational models. The TA1 teams may propose a variety of development strategies, ranging from pre-building initial models to learning everything from scratch using any combination of symbolic, probabilistic, or deep learning techniques. The TA1 development teams are expected to include both AI and developmental psychology expertise to produce both computational models and refined psychological theories of cognition. Although the primary goal of TA1 is to develop computational models, a secondary goal is to consolidate, refine, and extend the psychological theories of child cognition needed to guide model development, and to test through research key predictions made by the computational models. The TA1 teams may also propose optional companion research experiments in developmental psychology to refine their theories of cognition, where needed, to answer critical design questions relevant to their computational models.

TA1 proposals should include a detailed discussion of the technical plan to:

1) Design and develop computational models that mimic the foundations of human common sense for the core domains of objects, agents, and places;

2) Consolidate, refine, and extend the psychological theories of child cognition needed to guide model development; and test key predictions made by the computational models;

3) Sequence the development and evaluation of the computational models over the four-year program, including any optional companion research experiments in developmental psychology to refine relevant theories of cognition;

HR001119S0005 MACHINE COMMON SENSE 10

4) Perform the evaluation tasks described under TA2 below, including the three levels of performance: prediction/expectation, experience learning, and problem solving;

5) Achieve the target milestones and metrics identified in the Schedule/Milestone section below; and

6) Publish, share, and disseminate the results of research and development to the broader AI and Developmental Psychology communities.

Note that, although TA2 will provide sample test problems, each TA1 team is responsible for designing and providing its development strategy, training regimen, and any necessary datasets.

TA2: Test Environment for the Foundations of Human Common Sense

The goal of TA2 is to provide the test and evaluation environment for evaluating the TA1 models against cognitive development milestones as evidenced in developmental psychology research with children from 0 to 18-months old (Figure 3).

Figure 3: Examples of Developmental Psychology Research (0-18 months)

This existing body of research will be used as an initial starting point for the TA2 team to construct the test environment and develop specific test problems for each milestone in order to evaluate the TA1 computational models at three levels of performance:

1) Prediction/expectation: the test environment will present the TA1 models with videos and simulation experiences of the type used to test child cognition for each cognitive milestone. The models will produce a prediction or expectation output that will be used to determine if the model matches human cognitive performance. The models will

HR001119S0005 MACHINE COMMON SENSE 11

provide a measurable Violation of Expectation (VOE) signal when shown a possible next event, for direct comparison to the VOE results observed in children.

2) Experience learning: the test environment will present TA1 models with videos and simulation experiences in which a new object, agent, or place is introduced. The models will be tested to determine that they are able to learn the properties of the newly introduced item in a way that matches human cognitive performance.

3) Problem solving: the test environment will present the TA1 models with videos and simulation experiences in which a problem solving task is introduced. The models will be tested to determine they solve the problem in a way that matches human cognitive performance.

TA2 proposals should include a detailed discussion of the technical plan to:

1. Refine and expand the cognitive milestones shown above for the core domains of objects, agents, and places (including combinations of the three domains);

2. Devise a set of specific test problems for each cognitive milestone to assess computational models at the required three levels of performance (prediction/expectation, experience learning, problem solving);

3. Select, modify, or construct the video and 3D simulation infrastructure needed to conduct the TA1 evaluations;

4. Provide the training and testing infrastructure, with sample test problems, to support TA1 computational model development. TA2 is not expected to provide all of the training data that may be needed by the TA1 teams. TA1 teams are responsible for designing and providing their own development strategy and training regimen.

5. Develop and provide all of the documentation (e.g., user guides, test environment specifications, etc.) and application programming interfaces (APIs) for testing TA1 computational models;

6. Conduct formal evaluations of TA1 computational models every six months; and

7. Provide written test reports that document the performance of the TA1 models for each 6-month evaluation.

TA3: Broad Common Knowledge

The goal of TA3 is to learn/extract/construct a commonsense knowledge repository capable of answering natural language and image-based questions about commonsense phenomena from the AI2 Benchmarks for Common Sense. The AI2 benchmarks are constructed through an extensive crowdsourcing process to represent and measure the broad commonsense knowledge of a typical adult (https://allenai.org/commonsense/).

The AI2 benchmarks consist of a set of commonsense datasets and a leaderboard with an interface for researchers to submit their models for testing. The leaderboard utilizes technologies such as Docker and Kubernetes and operates on top of the highly scalable AI2 Beaker platform.

The leaderboard is designed to streamline the submission workflow and prevent overfitting and leaking (“cheating”). To maximize flexibility, researchers build and train their models on their computational infrastructure. Once researchers wish to create a submission, they encapsulate their model as a Docker container, allowing code in any language or technology. The containers are submitted to the underlying system through a single command and archived for future validation. The simple user interface allows researchers to specify parameters, datasets, etc.

https://allenai.org/commonsense/

HR001119S0005 MACHINE COMMON SENSE 12

Once submitted to the platform, the models can be run on public datasets so researchers can understand their model’s behavior and debug any issues. Once they are satisfied with the performance of their model on the public datasets, they can run it, or any of their previous submissions, on a “blind” evaluation dataset, which is hidden from the public. The underlying system prevents leakage of the blind evaluation dataset by only allowing researchers to view their model’s final score (note that the blind evaluation dataset and logs from the run are kept private). Submissions run on blind evaluation datasets are automatically evaluated by the AI2 system, and the score is then posted to the leaderboard. The underlying system prevents overfitting by limiting the number of runs submitted by each team against the blind evaluation dataset. All submissions are tagged with metadata and a URL, which facilitates seamless sharing and replication by the community.

Initially, five commonsense question datasets will be developed and available for testing of TA3-developed services:

(1) Commonsense Natural Language Inference (NLI): multiple choice, natural language-based questions about commonsense events derived from captions in the ActivityNet Captions and Large Scale Movie Description Challenge (LSMDC) datasets, see reference [38].

(2) Commonsense NLI with Vision: multiple choice, image-based questions about commonsense events selected from the same ActivityNet and LSMDC datasets.

(3) Abductive NLI: questions about inferring the most likely hypothesis for a given set of observations.

(4) Physical Interaction Question Answering (QA): natural language questions (initially) and image-based questions (in later years) about everyday objects and actions.

(5) Social Interaction QA: questions about human social behavior and the causal effects of everyday events.

The development of the first dataset, Commonsense NLI, is completed and is described further in reference [38]. The remaining four datasets are currently in development and will be completed by the start of the program. More information about the AI2 commonsense question datasets and leaderboard will be available at https://leaderboard.allenai.org/ (which requires Chrome).

Multiple development teams will be selected to develop the TA3 commonsense repositories/question answering services. TA3 teams may propose any combination of manual construction, information extraction, machine learning, and crowdsourcing techniques to construct a repository of broad commonsense knowledge. TA3 teams are not required to include personnel with expertise in psychology and are free to use whatever techniques they prefer, whether artificial or biologically inspired. The TA3 teams are expected to submit their system for testing on the blind evaluation datasets (i.e., all five commonsense question datasets identified above, if completed/available) every six months. Additional datasets may be developed over the course of the program, as needed, to align with the evolution of the TA3-developed capabilities. After the first year, TA3 teams may propose their own question datasets for inclusion in the AI2 benchmarks, or propose suggestions for the development of additional datasets by AI2, for testing by all of the TA3 teams.

TA3 proposals should include a detailed discussion of the technical approach to:

(1) Design and develop the broad commonsense knowledge service;

(2) Sequence the development and evaluation of the broad commonsense knowledge service https://leaderboard.allenai.org/

HR001119S0005 MACHINE COMMON SENSE 13

over the four-year program;

(3) Perform the evaluation tasks described above for the AI2 Benchmarks for Common

Sense;

(4) Achieve the target milestones and metrics identified in Schedule/Milestone section below; and

(5) Publish, share, and disseminate the results of research and development to the broader AI community.

D. Schedule/Milestones

DARPA anticipates a June 2019 start date for the MCS program that will run for a duration of 48 months. The following Principal Investigator (PI) Meetings will take place:

o An in-person Kickoff Meeting at program start. For planning purposes, assume a 3-day meeting in Arlington, VA;

o Four (4) web-based PI meetings held at six (6) months into each year of the program to review technical progress. For planning purposes, assume the government as host for these 2-day virtual meetings; and o Four (4) in-person PI meetings held at the end of each year of the program to review technical progress, conduct demonstrations, and provide opportunities for face-to-face collaboration. For planning purposes, assume 3-day meetings, alternating between a west coast and east coast location.

In addition to the PI Meetings above, each team should expect to:

o Host an onsite visit from the Program Manager (and potentially other government personnel) at least once a year; and o Make two additional trips to the Washington, D.C. area in the last two years of the program for possible demonstrations and technology transition meetings.

The target milestones and metrics identified below have been established to assess technical progress over the course of the program. These targets represent the expected pace of technology development. The targets are not “go/no-go” criteria and it is not DARPA’s intention to use these targets as the basis for down-selects or as the primary reason for other funding decisions.

TA1-developed computational models will be assessed for performance against cognitive development milestone capabilities (for the core domains of objects, agents, and places) at increasing levels of performance: prediction/expectation, experience learning, and problem solving. The target milestones and metrics are shown below, where each target metric represents the percentage of cognitive development milestones for which the model has been judged to match human cognitive performance at the designated level.

o Year 1: 30% of prediction/expectation milestones o Year 2: 50% of prediction/expectation and 30% of experience learning milestones o Year 3: 60% of prediction/expectation; 50% of experience learning; 30% of problem solving milestones o Year 4: 80% of prediction/expectation milestones; 80% of experience learning milestones for objects and 50% for agents and places milestones; 50% of problem solving milestones.

HR001119S0005 MACHINE COMMON SENSE 14

TA3-developed services will be assessed for performance on the AI2 Common Sense Benchmark datasets (Commonsense NLI, Commonsense NLI with Vision, Abductive NLI, Physical Interaction QA, and Social Interaction QA). The target metrics are as follows:

o Year 1: 50% correct for all blind evaluation datasets o Year 2: 60% correct for all blind evaluation datasets o Year 3: 70% correct for all blind evaluation datasets o Year 4: 80% correct for all blind evaluation data sets

Assessments of TA1-developed computational models and TA3-developed QA services will be conducted every six (6) months, preceding each PI meeting, in order for results/analyses to be available for review and discussion at the meetings.

E. TA-specific Deliverables

TA1:

o Computational model source code and APIs; and o Any associated data and documentation (including, at a minimum, user manuals and a detailed software design document).

TA2:

o Test environment source code and APIs;

o Any associated data and documentation (including, at a minimum, user manuals and a detailed software design document);

o Test Environment Readiness Assessment Reports; and o Any Test Environment documentation necessary to support TA1 team model assessments

(e.g., standard operating procedures, user guide, etc.).

TA3:

o Repositories, libraries, source code, and APIs; and o Any associated data and documentation (including, at a minimum, user manuals and a detailed software design document).

All TAs (TA1, TA2, and TA3) require the following deliverables:

o Quarterly progress and final reports;

o Presentations (in PowerPoint) for each PI Meeting (total of nine [9]);

o Copies of published papers and presentations at conferences, provided each month; and o Monthly financial status reports, provided within 10 calendar days of the end of each calendar month.

F. Government-furnished Property/Equipment/Information

No Government-furnished equipment is expected to be provided. The test and evaluation infrastructures and environments will be provided by TA2 for TA1, and by AI2 for TA3. The TA3 evaluation datasets will be provided by AI2.

HR001119S0005 MACHINE COMMON SENSE 15

G. Intellectual Property

The program will emphasize creating and leveraging open source technology and architecture.

Intellectual property rights asserted by proposers are strongly encouraged to be aligned with open source regimes. See Section VI.B.1 for more details on intellectual property.

A key goal of the program is to facilitate rapid innovation and advancements in AI by providing foundational capabilities for future users or developers of MCS program technologies and deliverables. Therefore, it is desired that all noncommercial software (including source code), software documentation, hardware designs and documentation, and technical data generated by the program be provided as deliverables to the Government, with a minimum of Government Purpose Rights (GPR), as lesser rights may adversely impact the progress towards the realization of AI systems with machine commonsense knowledge and reasoning capabilities.

H. References

[1] Carey, S., & Spelke, E. (1996). Science and core knowledge. Philosophy of science, 63(4), 515-533.

[2] Spelke, E. S. (2000). Core knowledge. American psychologist, 55(11), 1233.

[3] Spelke, E. S., & Kinzler, K. D. (2007). Core knowledge. Developmental science, 10(1), 89-96.

[4] Spelke - https://www.harvardlds.org/our-labs/spelke-labspelke-lab-members/elizabeth-spelke/

[5] Termine, N., Hrynick, T., Kestenbaum, R., Gleitman, H., & Spelke, E. S. (1987).

Perceptual completion of surfaces in infancy. Journal of Experimental Psychology: Human Perception and Performance, 13, 524-532.

[6] Spelke, E. S., von Hofsten, C., & Kestenbaum, R. (1989). Object perception and object-directed reaching in infancy: Interaction of spatial and kinetic information for object boundaries. Developmental Psychology, 25, 185-196.

[7] Kellman, P. J. & Spelke, E. S. (1983). Perception of partly occluded objects in infancy.

Cognitive Psychology, 15, 483-524.

[8] Ball, W. A. (1973). The perception of causality in the infant. Paper presented at the Society for Research in Child Development, Philadelphia, PA, April.

[9] Johnson, Scott P., Aslin, Richard N. (Sep 1995). Perception of object unity in 2-month-old infants. Developmental Psychology, Vol 31(5), 739-745.

[10] Baillargeon, R., Spelke, E. S., & Wasserman, S. (1985). Object permanence in 5-month-old infants. Cognition, 20, 191-208.

[11] Feigenson, L., & Carey, S. (2003). Tracking individuals via object-files: Evidence from infants’ manual search. Developmental Science, 6, 568–584.

[12] Aguiar, A., & Baillargeon, R. (1999). 2.5-month-old infants' reasoning about when objects should and should not be occluded. Cognitive Psychology, 39, 116-157.

HR001119S0005 MACHINE COMMON SENSE 16

[13] Saxe, R., Tenenbaum, J. B., & Carey, S. (2005). Secret Agents: Inferences about hidden causes by 10- and 12-month-old infants. Psychological Science, 16(12), 995-1001.

[14] Needham - https://www.vanderbilt.edu/psychological_sciences/bio/amy-needham

[15] Kellman, P. J., Gleitman, H., & Spelke, E. S. (1987). Object and observer motion in the perception of objects by infants. Journal of Experimental Psychology: Human Perception & Performance, 13(4), 586-593.

[16] Xu - http://www.babylab.berkeley.edu/publications

[17] Baillargeon - http://labs.psychology.illinois.edu/infantlab/publications.html

[18] Luo - https://psychology.missouri.edu/people/luo

[19] Woodward, A. L. (1999). Infants’ ability to distinguish between purposeful and non-purposeful behaviors. Infant Behavior and Development 22 (2), 145-160.

[20] Csibra, Gergely. (2003). Teleological and referential understanding of action in infancy.

Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

358. 447-58.

[21] Gergely, G. & Csibra, G. Natural pedagogy. In: Banaji MR, Gelman SA, editors.

Navigating the Social World: What Infants, Children, and Other Species Can Teach Us.

Oxford University Press; 2013. p. 127-32.

[22] Liu, S., & Spelke, E. S. (2017). Six-month-old infants expect agents to minimize the cost of their actions. Cognition, 160, 35-42.

[23] Liu, S., Ullman, T. D., Tenenbaum, J. B., & Spelke, E. S. (2017). Ten-month-old infants infer the value of goals from the costs of actions. Science, 358 (6366), 1038-1041.

[24] Leonard, J. A., Lee, Y., & Schulz, L .E. (2017). Infants make more attempts to achieve a goal when they see adults persist. Science 357(6357), 1290-1294.

[25] Hamlin - https://psych.ubc.ca/persons/kiley-hamlin/

[26] Hamlin, J.K., Mahajan, N., Liberman, Z. & Wynn, K. (2013). Not like me = bad: Infants prefer those who harm dissimilar others. Psychological Science, 24(4): 589 – 594.

[27] Song, H., Baillargeon, R., & Fisher, C. (2005). Can infants attribute to an agent a disposition to perform a particular action? Cognition, 98(2), B45–B55.

[28] (Sootsman) Buresh, J. & Woodward, A.L. (2007). Infants track action goals within and across agents. Cognition, 104 2, 287-314.

[29] Poulin-Dubois, D. & Chow, V. (2009). The effect of a looker’s past reliability on infants’ reasoning about beliefs. Developmental Psychology, 45(6), 1576-1582.

[30] Warneken, F. (2016). Insights into the biological foundation of human altruistic sentiments. Current Opinion in Psychology, 7, 51-56.

[31] Gergely - http://publications.ceu.edu/biblio/author/8018

[32] Csibra - http://publications.ceu.edu/biblio/author/985 https://www.vanderbilt.edu/psychological_sciences/bio/amy-needham http://www.babylab.berkeley.edu/publications http://labs.psychology.illinois.edu/infantlab/publications.html https://psychology.missouri.edu/people/luo http://publications.ceu.edu/biblio/author/8018 http://publications.ceu.edu/biblio/author/985

HR001119S0005 MACHINE COMMON SENSE 17

[33] Skerry - https://www.researchgate.net/scientific-contributions/2022834313_Amy_E_Skerry

[34] O’Keefe, J., & Nadel, L. (1978). The Hippocampus as a Cognitive Map (New York:

Oxford University Press).

[35] Spelke, E. S., & Lee, S. A. (2012). Core Systems of Geometry in Animal Minds.

Philosophical Transactions of the Royal Society, B. 367, 2784-93.

[36] Hermer-Vazquez - https://www.semanticscholar.org/author/Linda-Hermer- Vazquez/2032911

[37] Doeller, C. & Burgess, N. (2008). Distinct error-correcting and incidental learning of location relative to landmarks and boundaries. PNAS, 105, 5909-5914.

[38] Zellers, R., et al. (2018) SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference, in publication.

https://www.harvardlds.org/wp-content/uploads/2017/01/Spelke_Lee_2012-1.pdf https://www.semanticscholar.org/author/Linda-Hermer-Vazquez/2032911 https://www.semanticscholar.org/author/Linda-Hermer-Vazquez/2032911

HR001119S0005 MACHINE COMMON SENSE 18

II. Award Information

A. Awards

DARPA anticipates multiple awards for TA1 and TA3, and a single award for TA2. The level of funding for individual awards made under this solicitation has not been predetermined and will depend on the quality of the proposals received and the availability of funds. Awards will be made to proposers whose proposals are determined to be the most advantageous and provide the best value to the Government, all factors considered, including the potential contributions of the proposed work, overall funding strategy, and availability of funding. See Section V for further information.

The Government reserves the right to:

select for negotiation all, some, one, or none of the proposals received in response to this solicitation;

make awards without discussions with proposers;

conduct discussions with proposers if it is later determined to be necessary;

segregate portions of resulting awards into pre-priced options;

accept proposals in their entirety or select only portions of proposals for award;

fund proposals in increments and/or with options for continued work at the end of one or more phases;

request additional documentation once the award instrument has been determined (e.g., representations and certifications); and remove proposers from award consideration should the parties fail to reach agreement on award terms within a reasonable time or the proposer fails to provide requested additional information in a timely manner.

Proposals selected for award negotiation may result in a procurement contract, cooperative agreement, or Other Transaction (OT) depending upon the nature of the work proposed, the required degree of interaction between parties, and other factors. Grants will NOT be awarded under this program.

Proposers looking for innovative, commercial-like contractual arrangements are encouraged to consider requesting Other Transactions. To understand the flexibility and options associated with Other Transactions, consult http://www.darpa.mil/work-with-us/contract-management#OtherTransactions.

In accordance with 10 U.S.C. § 2371b(f), the Government may award a follow-on production contract or Other Transaction (OT) for any OT awarded under this BAA if: (1) that participant in the OT, or a recognized successor in interest to the OT, successfully completed the entire prototype project provided for in the OT, as modified; and (2) the OT provides for the award of a follow-on production contract or OT to the participant, or a recognized successor in interest to the OT.

In all cases, the Government contracting officer shall have sole discretion to select award instrument type, regardless of instrument type proposed, and to negotiate all instrument terms and conditions with selectees. DARPA will apply publication or other restrictions, as necessary, if it determines that the research resulting from the proposed effort will present a high likelihood http://www.darpa.mil/work-with-us/contract-management#OtherTransactions http://www.darpa.mil/work-with-us/contract-management#OtherTransactions

HR001119S0005 MACHINE COMMON SENSE 19

of disclosing performance characteristics of military systems or manufacturing technologies that are unique and critical to defense. Any award resulting from such a determination will include a requirement for DARPA permission before publishing any information or results on the program. For more information on publication restrictions, see the section below on Fundamental Research.

B. Fundamental Research

It is DoD policy that the publication of products of fundamental research will remain unrestricted to the maximum extent possible. National Security Decision Directive (NSDD) 189 defines fundamental research as follows:

‘Fundamental research’ means basic and applied research in science and engineering, the results of which ordinarily are published and shared broadly within the scientific community, as distinguished from proprietary research and from industrial development, design, production, and product utilization, the results of which ordinarily are restricted for proprietary or national security reasons.

As of the date of publication of this BAA, the Government expects that program goals as described herein may be met by proposers intending to perform fundamental research and does not anticipate applying publication restrictions of any kind to individual awards for fundamental research that may result from this BAA. Notwithstanding this statement of expectation, the Government is not prohibited from considering and selecting research proposals that, while perhaps not qualifying as fundamental research under the foregoing definition, still meet the BAA criteria for submissions. If proposals are selected for award that offer other than a fundamental research solution, the Government will either work with the proposer to modify the proposed statement of work to bring the research back into line with fundamental research or else the proposer will agree to restrictions in order to receive an award.

Proposers should indicate in their proposal whether they believe the scope of the research included in their proposal is fundamental or not. While proposers should clearly explain the intended results of their research, the Government shall have sole discretion to select award instrument type and to negotiate all instrument terms and conditions with selectees. Appropriate clauses will be included in resultant awards for non-fundamental research to prescribe publication requirements and other restrictions, as appropriate. This clause can be found at http://www.darpa.mil/work-with-us/additional-baa.

For certain research projects, it may be possible that although the research being performed by the awardee is restricted research, a subawardee may be conducting fundamental research. In those cases, it is the awardee’s responsibility to explain in their proposal why its subawardee’s effort is fundamental research

C. Disclosure of Information and Compliance with Safeguarding Covered Defense Information Controls

The following provisions and clause apply to all solicitations and contracts; however, the definition of “controlled technical information” clearly exempts work considered fundamental research and therefore, even though included in the contract, will not apply if the work is fundamental research.

http://www.darpa.mil/work-with-us/additional-baa

HR001119S0005 MACHINE COMMON SENSE 20

DFARS 252.204-7000, “Disclosure of Information” DFARS 252.204-7008, “Compliance with Safeguarding Covered Defense Information Controls” DFARS 252.204-7012, “Safeguarding Covered Defense Information and Cyber Incident Reporting”

The full text of the above solicitation provision and contract clauses can be found at http://www.darpa.mil/work-with-us/additional-baa#NPRPAC.

Compliance with the above requirements includes the mandate for proposers to implement the security requirements specified by National Institute of Standards and Technology (NIST) Special Publication (SP) 800-171, “Protecting Controlled Unclassified Information in Nonfederal Information Systems and Organizations” (see https://doi.org/10.6028/NIST.SP.800-171r1) that are in effect at the time the BAA is issued.

For awards where the work is considered fundamental research, the contractor will not have to implement the aforementioned requirements and safeguards; however, should the nature of the work change during performance of the award, work not considered fundamental research will be subject to these requirements.

http://www.darpa.mil/work-with-us/additional-baa#NPRPAC https://doi.org/10.6028/NIST.SP.800-171r1

HR001119S0005 MACHINE COMMON SENSE 21

III. Eligibility Information

A. Eligible Applicants

DARPA welcomes engagement from all responsible sources capable of satisfying the Government's needs, including academia (colleges and universities); businesses (large, small, small disadvantaged, etc.); other organizations (including non-profit); other entities (foreign, domestic, and government); FFRDCs; minority institutions; and others.

DARPA welcomes engagement from non-traditional sources in addition to current DARPA performers.

1. Federally Funded Research and Development Centers (FFRDCs) and Government Entities

a. FFRDCs FFRDCs are subject to applicable direct competition limitations and cannot propose to this BAA in any capacity unless they meet the following conditions: (1) FFRDCs must clearly demonstrate that the proposed work is not otherwise available from the private sector. (2) FFRDCs must provide a letter on official letterhead from their sponsoring organization citing the specific authority establishing their eligibility to propose to Government solicitations and compete with industry, and their compliance with the associated FFRDC sponsor agreement’s terms and conditions. This information is required for FFRDCs proposing to be awardees or subawardees.

b. Government Entities Government Entities (e.g., Government/National laboratories, military educational institutions, etc.) are subject to applicable direct competition limitations. Government entities must clearly demonstrate that the work is not otherwise available from the private sector and provide written documentation citing the specific statutory authority and contractual authority, if relevant, establishing their ability to propose to Government solicitations.

c. Authority and Eligibility At the present time, DARPA does not consider 15 U.S.C. § 3710a to be sufficient legal authority to show eligibility. While 10 U.S.C.§ 2539b may be the appropriate statutory starting point for some entities, specific supporting regulatory guidance, together with evidence of agency approval, will still be required to fully establish eligibility. DARPA will consider FFRDC and Government entity eligibility submissions on a case-by-case basis; however, the burden to prove eligibility for all team members rests solely with the proposer.

2. Foreign Participation Non-U.S. organizations and/or individuals may participate to the extent that such participants comply with any necessary nondisclosure agreements, security regulations, export control laws, and other governing statutes applicable under the circumstances.

HR001119S0005 MACHINE COMMON SENSE 22

B. Organizational Conflicts of Interest

FAR 9.5 Requirements In accordance with FAR 9.5, proposers are required to identify and disclose all facts relevant to potential OCIs involving the proposer’s organization and any proposed team member (subawardee, consultant). Under this Section, the proposer is responsible for providing this disclosure with each proposal submitted to the BAA. The disclosure must include the proposer’s, and as applicable, proposed team member’s OCI mitigation plan. The OCI mitigation plan must include a description of the actions the proposer has taken, or intends to take, to prevent the existence of conflicting roles that might bias the proposer’s judgment and to prevent the proposer from having unfair competitive advantage. The OCI mitigation plan will specifically discuss the disclosed OCI in the context of each of the OCI limitations outlined in FAR 9.505-1 through FAR 9.505-4.

Agency Supplemental OCI Policy In addition, DARPA has a supplemental OCI policy that prohibits contractors/performers from concurrently providing Scientific Engineering Technical Assistance (SETA), Advisory and Assistance Services (A&AS) or similar support services and being a technical performer.

Therefore, as part of the FAR 9.5 disclosure requirement above, a proposer must affirm whether the proposer or any proposed team member (subawardee, consultant) is providing SETA, A&AS, or similar support to any DARPA office(s) under: (a) a current award or subaward; or (b) a past award or subaward that ended within one calendar year prior to the proposal’s submission date.

If SETA, A&AS, or similar support is being or was provided to any DARPA office(s), the proposal must include:

The name of the DARPA office receiving the support;

The prime contract number;

Identification of proposed team member (subawardee, consultant) providing the support; and An OCI mitigation plan in accordance with FAR 9.5.

Government Procedures In accordance with FAR 9.503, 9.504 and 9.506, the Government will evaluate OCI mitigation plans to avoid, neutralize or mitigate potential OCI issues before award and to determine whether it is in the Government’s interest to grant a waiver. The Government will only evaluate OCI mitigation plans for proposals that are determined selectable under the BAA evaluation criteria and funding availability.

The Government may require proposers to provide additional information to assist the Government in evaluating the proposer’s OCI mitigation plan.

If the Government determines that a proposer failed to fully disclose an OCI; or failed to provide the affirmation of DARPA support as described above; or failed to reasonably provide additional information requested by the Government to assist in evaluating the proposer’s OCI mitigation plan, the Government may reject the proposal and withdraw it from consideration for award.

HR001119S0005 MACHINE COMMON SENSE 23

C. Cost Sharing/Matching

Cost sharing is not required; however, it will be carefully considered where there is an applicable statutory condition relating to the selected funding instrument (e.g., OTs under the authority of 10 U.S.C. § 2371).

D. Other Eligibility Requirements

Ability to Receive Awards in Multiple Technical Areas - Conflicts of Interest Proposers may submit proposals for all three TAs; however, to avoid Organizational Conflicts of Interest (OCI) situations between the TAs and ensure objective test and evaluation results, the following caveats apply:

Proposers selected for TA1 cannot also be selected for TA2, whether as a prime, subcontractor, or in any other capacity, from an organizational to an individual level.

Similarly, proposers selected for TA2 cannot also be selected for TA1. Exceptions (i.e., selection for both TA1 and TA2) will be considered for proposals that include an OCI Mitigation Strategy as described in Section III.B that sufficiently describes the plans…