Bidders Conference Q A.pdf

Q1) Reference Challenge Question 2: What is the spatial coverage of the BlueGrass data (it does not seem to contain narrow field of view sensors (e.g. soda straws)?

A1) Relatively speaking, there are wider and narrower field of view sensors in the Bluegrass dataset. The EO portion of the BlueGrass data covers a 5x5 km area window. Other surveillance technologies, not included in the BlueGrass dataset cover smaller areas. For example, the unmanned Predator covers a 100x100 m area. In comparison, Ground Moving Target Indicator (GMTI) provides wide area coverage of significantly larger areas.

However, the concept of using cueing from GMTI radar down to a narrower field‐of‐view system like Constant Hawk still applies.

Q2) Are ground‐based, narrow aperture sensors of interest (e.g., video)?

A2) Not particularly, unless it’s a representation of human intelligence inputs.

Q3) Will there be Predator data available?

A3) Not within the BlueGrass dataset.

Q4) Is the entire program related to these two datasets, or are there other datasets available throughout the program?

A4) These are the only datasets available for the program, although we would be interested in other applications (e.g., methods applied to Predator data). The emphasis of the program is on novel methodology, and the datasets are meant to give demonstration context. While the focus of your efforts should be on developing innovative mathematics and methodologies, demonstrations must at least be done on the BlueGrass dataset provided.

Q5) How do you envision adaptive handoff demonstration algorithm? Are we being invited to describe the demo that would demonstrate cueing? Do you envision sensor steering as part of the demonstration?

A5) Yes, bidders are asked to describe how they would do the demo, to include how they would use novel mathematical approaches, and the data, to demonstrate the challenge questions. While the BAA requires that you use a particular collection of resources as part of the deliverable, there is nothing to restrict you from offering additional demonstrations or datasets as well.

Q6) Is it ok to go outside of the scope of the challenge problem (in addition to answering the scope as outlined in the

BAA)?

A6) Yes. The BAA defines the broad challenge problem, and provides datasets intended to be used to develop and test methodology relevant to the challenge problem. The challenge problem is not the datasets themselves, but rather the low‐slow domain. This provides a large area for project focus, and it is not necessary to address the entire domain, but the project should address the domain in a way that is relevant to the data. Remember to keep the goals of the challenge problems in mind.

Q7) Will there be additional challenge problems for the option years two and three?

A7) No. These are the challenge problems. This is a large domain and it does not all have to be addressed within the project scope, or in year one of the project. However, a demonstration at the end of 2011 should show a viable approach.

Q8) Is a solution of the BlueGrass dataset sufficient?

A8) No. We’re looking to attack a problem space, this data is just a tool.

Q9) What are the evaluation criteria? Are you using the ones in the Bidder’s Conference slides? Do the evaluation criteria include additional datasets against which the approach will be measured, in addition to the ones included in the

BAA?

A9) The evaluation criteria in the Bidder’s Conference slides are not comprehensive, but are a list of performance measures of use in the problem space. The BAA has three separate sections on “Evaluation Criteria.” The first two (sections 2.10 and 2.12) are evaluation criteria specific to each challenge problem, the third (section 3.1) is for the overall project. If your project addresses one of these areas of performance in sections 2.10 or 2.12, but not another (e.g., impacts track ambiguity but not velocity accuracy), that is acceptable. We’re looking for dramatic improvement in one or many areas. Solving the entire problem space, while a good thing if it is possible, is not necessary. Solving one portion of the problem space in a way that makes a meaningful difference is also a potentially interesting line of investigation.

Q10) What are the track ID numbers on the high‐fast domain example slide? Does the ID switch from friend to foe when the track changes? What does the track change (change in ID numbers) mean on the high‐fast domain slide?

A10) The ID category does not necessarily change when the track number changes, although that is possible. The single line curves are effectively many small lines, each with a different track ID. The value of the numbers was not significant.

The key point about the track numbers changing is that continuity is lost from the standpoint of a warfighter making a shoot/no‐shoot decision. If you’re on the verge of deciding to shoot the target with track number 1234, and its track number suddenly changes to 5678, you’re decision to shoot will be delayed while you figure out whether the different track number refers to the target you wanted to kill, or whether it’s a different target entirely that you may not want to shoot. A great many things can cause these track number changes. The key point is that we want to clean up the picture so that we get as close to one and only one identifier per target as possible, to minimize the chance of fratricide, and to maximize the effectiveness of our weapon systems.

Q11) How do we use the datasets in the demonstration? How do you perform a demonstration with a static dataset?

A11) As an example relative to Challenge Problem 2, if one is looking at the battlefield and something is happening in a particular area, something that will detect that local event and characterize it as an event. Then you decide to redirect a sensor to that area. Use the data as analogous events. Within the BlueGrass dataset, behavior identifies objects or events of interest (e.g., vehicles). Other factors can be added for your demonstration, such as pseudo‐scheduling, additional data, or mobile robots. However, this does not remove the requirement to do something relevant with the provided data.

Q12) Are there redundant streams in the BlueGrass dataset to provide the ability to do sensor management?

A12) There are repeated events in the BlueGrass dataset under different conditions (e.g., day/night). Note: Optimizing physical placement of sensors is not the focus of this BAA.

Q13) Is the BlueGrass data collected from different trajectories?

A13) No. The surveillance aircraft followed a specific, repeated orbit. The GOTCHA dataset (which is part of the data available on the Sensor Data Management System Public Site mentioned in para 2.14 of the BAA) does have some facets of change in detection method, but we are not sure whether those facets are included in the publically‐available data.

Q14) Is the imagery in the dataset coincidental or cross‐cued?

A14) Refer to the documentation that will come with the Bluegrass data. Most sensors were consistent in their coverage. The SIGINT sensors were used in a more targeted fashion.

Q15) Is there an underlying infrastructure that we have to work under (e.g., time delays, cross‐cueing, etc.)?

A15) There is no defined or assumed infrastructure for the BAA, there is room for that to be approached parametrically.

The more closely a proposed approach reflects the reality of the battlefield, the better.

Q16) For actionable tracks, how “good” does “good” have to be for tracking and target engagement? Are we restricted to the current rules of engagement?

A16) It is important to point out in the proposal by what metrics you’re proposing to measure your results. Refer to the BAA Challenge Questions Evaluation Criteria for Common Tactical Picture (para 2.10 of the BAA) measures. A common tactical picture must support real time operations, versus non‐ or near‐real time operations (e.g. seconds vs. minutes).

Multi sensor fusion for disparate sensors (e.g. radar, optical, acoustic, SIGINT, ELINT, HUMINT, etc) is key to obtaining a continuous and unambiguous Common Tactical Picture. It is possible that ROEs may be modified if necessary.

Q17) How raw is the EO data in BlueGrass (e.g., is there frame‐to‐frame registration)?

A17) Both raw and processed EO data is available. The processed data is stitched (frames from six cameras mosaicked together), contrast enhanced, and geo‐registered. Frame‐to‐frame registration (alignment) is applied to the processed data. Solutions for processing the processed data are preferred as this form of the data is most easily available.

The Blue Grass dataset contains GMTI data (from JSTARS and LSRS sensors) and EO data from Constant Hawk. The geo‐ registration accuracy of these independent data sources is not high.

MIT Lincoln Lab processes the EO data and produces change detection and tracking results. The change detection and tracking results are not highly accurate and are NOT available for the Blue Grass dataset.

Q18) Is there interest to do more accurate tracking on EO data?

A18) Yes. We’re looking for real‐time solutions that don’t require shipping data to an alternate location for analysis.

Q19) Are you looking for integrated solutions for data fusion from JDL architecture?

A19) We’re not concerned with how this maps to the JDL architecture. We’re concerned about a common tactical picture, and how this will integrate into fire control/weapons systems. If someone proposes a good idea that improves the picture, we are not concerned with what JDL level it would be integrated.

Q20) With which system will this platform be integrated?

A20) There is no select system of interest. We want to bring the data and information to the tactical level (lower than 3‐ Star). Proposers should be less concerned with where it fits into the command structure and more about making an impact. Two possible examples: one is to implement the mathematics on multiple platforms (e.g. ships, aircraft, ground radars), another approach could be a single fusion node in an Air Operations Center or Tactical Operations Center. The implementation architecture need not drive proposed solutions.

Q21) Will most of the data be fly‐over? How much three‐dimensional data is included?

A21) All the Blue Grass sensors were flown from aircraft. Airborne LIDAR data was collected and is available. It has 1 meter resolution in x,y,z. It is static (single frame) (no frame rate). Use of ground sensor data is also of interest.

Q22) Can we have access to classified data? What is the level of clearance required for access?

A22) Yes. There is SIGINT data in the Bluegrass data set that is classified, and if your proposal demonstrates a need to know, and you have the facilities to handle the data, that portion of the data could be made available. Methods to submit a classified proposal, if necessary, are outlined in the BAA.

Q23) Do you prefer to fund an integrated solution?

A23) A solution that does a lot for a little cost is clearly good. However, something that does one thing very well, and that makes a big impact, can also be of interest. We don’t necessarily have a preset preference; we’ll look at the ideas that are presented.

Q24) Is the system stand‐alone (i.e., can there be human interaction or should the machine come to the final answer)?

A24) Solutions should be proposed regardless of whether there will be human interaction. We’re not looking for an operational concept. We are looking for innovative mathematics.

Q25) When is implementation anticipated?

A25) As outlined in the BAA, there will be a laboratory demonstration in December 2011. We are interested in the earliest field implementation, but the demonstration need not be equivalent to field implementation.

Q26) Optimal algorithm(s) will probably require engagement and disengagement of systems. Is this practical?

A26) Dynamic engagement is of interest. Demonstration on static data might require creativity.

Q27) What additional data do we get/can we include to help track enemy vs. friendly (e.g., EO, radar, etc.)?

A27) We are looking for an integrated solution that combines multiple data types (HUMINT, SIGINT, ELINT, radar, optical, acoustic, etc.).

Q28) The BAA states that techniques that solve both challenge problems simultaneously are preferred. It’s not clear that such approaches will also comprehensively address one. Which is preferred?

A28) We are looking for revolutionary approaches with orders of magnitude improvement, and are not pre‐disposed to one answer or approach over another.

Q29) What is the limit on the number of quad chart submissions for an organization/PI?

A29) There is no per‐organization limit.

Q30) Will we receive binary feedback, or more detailed feedback about our submissions?

A30) We will provide yes/no feedback.

Q31) What kind of research and modeling has been done on this data?

A31) MIT Lincoln Lab, MITRE and others have conducted research on the Blue Grass data. Much of the research has focused on tracking vehicles in the EO (Constant Hawk data), but also improved geo‐registration, track pattern analysis (using graphs for example) and ways to speed the data handling (storage, streaming, compression). At least two other research programs are using the Blue Grass dataset: 1) OSD/DDR&E/RRTO has a Wide Area Persistent Surveillance (WAPS) BAA with two contractors conducting vehicle tracking and stop/start detection (and one contractor developing compression and IO speed improvements, and 2) the DARPA PerSeas program will also use the Bluegrass dataset in a forensic mode.

Q32) What is the anticipated award amount?

A32) The award amount will not be announced, but is contingent on the quality of proposals received. Innovation doesn’t necessarily require a big team to develop.

Q33) Will feedback be given for the quad chart and white paper in terms of budget?

A33) No.

Q34) Will offers be entertained from other government organizations?

A34) Government organizations interested in competing within this BAA process should see para 2.19.1.3.2 in the BAA, which addresses this.

Question added to the Previously posted Q&A on May 12, 2010.

(Q5) (added 20 May 2010) ‐ Does a person *have* to submit a quad chart, *have* to be requested by the government to submit a white paper, *have* to submit a white paper, and *have* to be requested by the government to submit a proposal, or can they just submit a proposal?

(A5) ‐ A bidder may not skip any of the prior steps and just submit a proposal. They must submit a quad chart, be requested to submit a white paper, submit a white paper, and be requested to submit a proposal before submitting a proposal.