Amendment 0004_Questions and Answers.pdf

Amendment Number 0004 Special Notice Number 11-SN-0004

“Data to Decision”

The purpose of Amendment 04 is to answer technical and cost questions received in response to In-dustry Days held in San Francisco, CA on 28 February 2011 and Boston, MA on 03 March 2011.

Q#1. Will the challenge problems be available very soon (or their topics) so the proposals speak to them? Or should the proposals describe hypothetical challenge problem areas?

A#1. The focus of this program is on developing technology that can be used to solve many cross-

Service challenge problems. The problems will change each year and there are likely to be several ones at once. Your proposal should thus describe your ideas on how to build analytical tools that exploit motion data rather than dwelling on a specific challenge problem. You should feel free to describe how your approaches might solve problems the military face in general, but the goal of this effort is to build a toolbox of motion analytic tools, not to solve an existing challenge problem.

Q#2. What are the challenge problems set by the government? When do you know what they are, and how they are selected by whom?

A#2. The challenge problems will be set by the (red) Challenge team. A primary goal of this pro-gram will be to build motion analytic tools that can be repurposed across several missions; for in-stance, we might build a tracker that can be used on video data of vehicles as well as Ground Moving Target Intelligence data of ships. There are thus likely to be several challenge problems at once.

Q#3. Will the (blue) Develop team get test data for validation during development? When will the selected challenge problems be available to the develop team?

A#3. The (blue) Develop team will get data to use as they develop their algorithms.

Q#4. The challenge problems seem open-ended. Are we bidding a level of effort or to build specific modules?

A#4. Offerors should propose to build specific algorithms, modules, and/or technical approaches to address the challenge problems.

Q#5. Among motion, imagery, and text, how/why did you choose motion to focus on first?

A#5. Recent sensor programs have focused on motion data.

Q#6. Should industry apply to "contract" only?

A#6. It is the offeror’s responsibility to determine the instrument type (contract or grant).

Q#7. Suppose that we have some good ideas on how to build a tracker, what other requirements have for GMTI to be met to bid a proposal? Do we need to form a consortium? Do we have to if it is expandable to other applications such as Wide Area Motion Imagery?

A#7. A functional consortium will be built from proposers whose efforts have been selected. If an offeror has a technology that they believe is sufficiently mature and do not wish to be part of the consortium that offeror should submit a proposal under the ONR Long Range BAA.

Q#8. Can user interface proposals be made under this motion BAA since it will be necessary for a complete build?

A#8. It is not ruled out, even though user interfaces are emphasized in subsequent BAAs.

Q#9. Do you expect a proposal for 4 years covering all three aspects of analytics (motion, text, and image), or only motion analytics?

A#9. The period of performance for projects under this Special Notice may be from one (1) to four

(4) years. This Special Notice is seeking proposals in motion analytics. The other (text and im-agery) will be solicited under separate BAAs.

Q#10. Can you provide example datasets for the proposal development, i.e. examples that are similar to what may be provided by the red team?

A#10. The datasets will be provided after award.

Q#11. Should proposals include specific descriptions of proposed analytic modules, and /or interfac-es to these modules? Can open test data be copied to the consortium member (developer) site?

A#11. Offerors should describe their proposed analytic modules in enough detail for us to understand your approach without divulging any proprietary approaches. Interfaces to these modules are of lesser importance since the ultimate application will likely be an amalgam of many approaches.

Open test data can indeed be copied to your development site.

Q#12. Will the government entertain proposals on Service Oriented Architecture and open architec-ture?

A#12. The government will not entertain proposals on Service Oriented Architecture and open archi-tecture under this Special Notice.

Q#13. What should proposals mention regarding integration with other proposers and with the testbed? Should explicit effort be devoted to such activities?

A#13. The goal of the consortium is to get the “best of breed” across all consortium members into a single application. Proposals that have been integrated by the proposers should indicate the prear-ranged coupling of proposals. You should expect and plan to collaborate with others on this pro-gram.

Q#14. Are all performers expected to contribute new algorithms for each six-month system build?

A#14. Offerors are not expected to contribute new algorithms for each six month system build.

However, offerors are expected to contribute as much as possible.

Q#15. What prior information for an area of interest will be (or could be) available to the analytic algorithms?

A#15. The government does not intend to constrain any approaches. Offerors are encouraged to think about how they can bring in additional information to solve problems.

Q#16. Will this D2D consider the application of mixed-initiative analytics?

A#16. The D2D will consider the application of mixed-initiative analytics.

Q#17. Can you provide information on teaming opportunities to others who want to form teams on the website?

A#17. Teaming opportunities can be found at the http://data-to-decisions.ida.org website.

Q#18. Do you expect performers to form small teams within the consortium to focus on one topic, such as tracking, reacquisition, activity recognition, or anomaly detection?

A#18. It is possible there would be subgroups in the consortium focusing on specific topics.

Q#19. How will the consortium community's cohesion be built? Will there be a series of face-to-face meetings, or group tele/video conferences?

A#19. The consortium cohesion will be built through a series of meetings.

http://data-to-decisions.ida.org/�

Q#20. Will the consortium have input to the build/integration process?

A#20. The consortium will have input to the build/integration process if they have ideas on how to better the build and integration process.

Q#21. Do individual organizations have to be part of a well-defined consortium at proposal time? Or is it possible that the government will create/augment consortiums based on the proposals that are submitted?

A#21.Offerors should not feel that they need to team prior to proposal submission. It is the govern-ment’s responsibility to determine which proposal(s) are most innovative, understand the goals, and illustrate a willingness to collaborate openly.

Q#22. Is there minimum and maximum number in forming a consortium? Does it have to include a team from each segment of industry, academia, national laboratories?

A#22. There is no minimum or maximum number in forming a consortium. Consortiums are open to all who want to contribute to the success of the program.

Q#23. Is it preferred for the bidder to form a consortium? That is, is teaming important to cover the whole spectrum of the problem? Or will the government select the proposer individually?

A#23. Offerors should not feel as if they need to form a consortium. Based on the evaluation of the proposals, the government will form the consortium. Offerors can choose a team member if they feel that strengthens their proposal, but it is not required.

Q#24. You say that the goal of the program is to make quick solutions to real problems, but put in a high barrier for existing intellectual property. How will existing software modules, developed at significant private expense, be evaluated fairly to determine 10dB performance improvement?

Does schedule or budget performance impact decisions on licensing Intellectual Property software modules?

A#24. Existing software modules will be considered. The “10-decibel” is of course, subjective, but the offeror will have to clearly demonstrate that their algorithms are much better than what the consortium could ever develop.

Q#25. Will consortium members be allowed to use Intellectual Property developed in the program for commercial applications?

A#25. Consortium members are encouraged to use the Intellectual Property developed for commer-cial applications using a non-exclusive, royalty-free license.

Q#26. Will the software developed by the consortium be open to the public or just the consortium and government?

A#26. The software will be open only to the consortium and not to the public.

Q#27. Has an operational system been selected for this initial analytics development BAA?

A#27. An operational system has not been selected. The modules are being developed so they can be used by many operational systems.

Q#28. Do you have specific existing commercial solutions for the end-user interface(s) in mind?

Can you give examples of specific end-users of the developed system? And specific data?

A#28. The end-user interface will be developed using results obtained under a separate BAA. Pro-posals should concentrate on motion analytics and should not be tailored to any specific end-user.

Q#29. What is the transition target for this research?

A#29. The research will result in modules that can be cast as services on many operational systems.

Q#30. Will the slides from today's presentations be published?

A#30. The slides will be posted to FedBizOps, grants.gov, and the Office of Naval Research website.

Q#31. Will the hardware testbed include Graphics Processor Units?

A#31. Not at present, but maybe added at a later time.

Q#32. Is the $300K to $750K guidance for one year or for total program cost (all years)?

A#32. As stated on page 6 of the Special Notice, “ONR, on behalf of DDR&E, plans to fund eight

(8) to ten (10) individual awards with a value of $300,000 to $750,000 per year”.

Q#33. Is there a process for a government lab. (e.g. warfare center) to propose, either alone or in co-operation with a company or academic institution given that there is no prime/sub relationship in the Special Notice and government lab cannot get a prime award?

A#33. Government laboratories (e.g. warfare centers) can propose by submitting an unsolicited white paper to the responsible program manager.

Q#34: What will be additional roles of FFRDC/UARCs when government already addresses "ana-lyses, build, and challenge"?

A#34. Federally Funded Research and Development Centers (FFRDCs) and University Affi-liated Research Centers (UARCs) can also participate in the "Develop" part of the program, as described in the Special Notice by submitting unsolicited white paper to the responsible program manager.

Q#35. Will offerors be permitted to augment challenge data with open source or publicly available data? (e.g., adding available satellite, geographic data to government – provided Wide Area Mo-tion Imagery data)?

A#35. Offerors are permitted to augment challenge data with open source or publicly available data.

There are no restrictions on how you can solve the problems.

Q#36. Will contracts be awarded with the consortium or separately with each proposer?

A#36. The contracts will be awarded separately with each offeror.

Q#37. I understand that the objective of Movement Intelligence (MOVINT) data is to provide enough information to build meaning social relationship graph. What would you think it is enough? Which information could be added to complete the graph model? Would you anticipate that expert opinions play a significant role?

A#37. The goal is to build a toolbox of MOVINT modules that allow us to solve many operational problems. MOVINT is only one piece of building social relationship graphs. The goal of this BAA is, however, not to build social relationship graphs. That is only one challenge problem we might address.

Q#38. Do we need to handle text in addition to Wide Area Motion Imagery (WAMI) and Ground Moving Target Intelligence (GMTI) in this Special Notice? Or, we need to wait for the other BAA?

A#38. This Notice is only for Motion.

Q#39. Are small businesses and universities encouraged to bid? Is the take-away that large busi-nesses are not encouraged to bid?

A#39. The BAA is not restricted and all potential offerors are encouraged to bid. The government’s goal is to develop an extensive capability for MOVINT analysis.

Q#40. Has the structure of the testbed already been determined? It seems that the details depend on the specifics of the challenge problem. Will there be a unique testbed for each challenge problem?

A#40. Data 2Decision (D2D) system architecture is evolving. The current testbed is just the starting point.

Q#41. Please clarify for us how you would like for proposers to formulate a technical approach and

Statement of Work (SOW). The BAA has described only in general terms the topics of interest.

Should Contractors propose specific algorithm solutions to be developed? This is somewhat diffi-cult given that the challenge problems are yet to be presented. Moreover, if you expect contrac-tors to propose specific solutions, the IT would likely require leveraging existing work, which is likely to be proprietary. Thus, posing “open source” solutions is difficult given the information available.

A#41. The BAA is for technology, not for solving a specific operational challenge. It will use opera-tional challenges to build relevant technology, but the goal is to build motion analytics that can be used across as many operational challenges as possible. You might, for instance, propose specific algorithmic approaches and discuss how they could be made to work across many different sensor and target types. You might also feel free to discuss new motion analytic tools that you would like to propose as opposed to the ones you already have that are proprietary.

Q#42. For “win-win”, are performers expected to coordinate proposals prior to submission/award?

A#42. Offerors are not expected to coordinate proposals prior to submission/award.

Q#43. We often include consultants on our teams who provide limited (~100 hours/year) consulting.

Many of our consultants hold academic positions. Would the no-prime/sub concept require that our consultants submit their own technical and cost proposals for a small number of hours of work per year?

A#43. The offeror (prime contractor) is responsible for submitting a full proposal package (technical and cost proposal). The technical proposal should describe the research effort and identify who (prime, subcontractor, consultant) will be performing what part of the research effort. The offe-ror’s proposal package should also include a copy of the consultant agreement that should include the number of hours, labor/hourly rate, etc..

Q#44. What is the page limit for the entire technical proposal?

A#44. Please refer to paragraph b. (Full Proposals) under Section IV, entitled “Application and Sub-mission Information”, of the BAA.

Q# 45. Are Offeror’s required to submit White papers?

A#45. Offerors are not required to submit white papers. See Section IV.(Full Proposal Submission and Award Information) of the Special Notice.

Q#46. Are there travel guidelines? (e.g., destinations, # of trips, frequency?)

A#46. Please refer to paragraph b. (Full Proposals) under Section IV, entitled “Application and Sub-mission Information” and paragraph 7. Project Meetings and Reviews” under Section VII entitled “Other Information” of the BAA.

Q#47. Would it be permissible to have a proposal primarily focused on, Motion, but which has ex-ploratory or preliminary efforts, that include data management, software management, and/or user interfaces?

A#47. This research effort is on Motion analytics only and will only fund work that is directly tied to Motion.

Q#48. Is the area of “motion tracking” broadly inclusive of spatial tracking and 3D reconstruction techniques, for example tracking moving objects within buildings or through mountainous terrain?

A#48. Yes.

Q#49. Can analysts address fusion of motion analytic with existing Human Intelligence (HUMINT) & Geospatial Intelligence (GEO-INT)?

A#49. Fusion will not be a key part of this Notice, but we want to make sure there are no restrictions to the type of data you use in your motion analytics.

Q#50. Is it recommended/encouraged for consortium members to provide data for the rest of the con-sortium members to use? If so what recommended data standard/format?

A#50. The consortium members should be willing to share any data that they have that would be beneficial to the consortium. The format can be decided after the fact by the consortium.

Q#51. Will technologies developed under this BAA be in the public domain?

A#51. The technologies will not be in the public domain, but open to the Consortium members.

Q#52. Does “open source” mean all members of the Consortium can get our source code? Or just the Government?

A#52. All members of the consortium will be able to access the source code developed under this program and add new pieces for enhancement.

Q#53. Is it possible for developed software to be considered Government-off-the-Shelf (GOTS), and therefore prevent reuse in future contractor procurements by competitor companies?

A#53. The government will retain Government Purpose Rights to everything developed by the con-sortium.

Q#54. Why is Open Source necessary? Can exchange executable SW to collaborate?

A#54. The goal is to have many organizations contribute and work on an algorithm so it becomes the

“best of breed”. Exchanging executable does not achieve this, nor does it achieve the kind of col-laborations we’re looking for.

Q#55. Who are end users and front-line users?

A#55. The end users and front-line users are Operational and Intelligence Centers.

Q#56. Would the Government provide a representative example of Counter Insurgency (COIN) mis-sion which could have been successful with this program product?

A#56. Forensic tracking is just one example.

Q#57. What sensors and data will be provided for Motion Analytic - Ground Moving Target Intelli-gence (GMTI) or Wide Area Motion Imagery (WAMI)?

A#57. There will be a mix of sensors and sensing type. Proposals submitted in response to this No-tice should not be tied directly to any specific sensor.

Q#58. Are traditional image (or radar) detection and target feature extraction algorithms relevant to the Department of Defense (DoD) effort, or are these assumed “mature”?

A#58. Image detection and feature extraction will be part of a subsequent image analytics.

Q#59. What is the data provided in the motion analytics algorithm development? Electro-

Optics/InfraRed (EO/IR)? Ground Moving Target Intelligence (GMTI) radar? Or combination of all?

A#59. The data combination will be all types.

Q#60. Will government provide unclassified data or can contractors provide some data? What kind of data will government collect?

A#60. Government will provide unclassified data, but offerors are encouraged to provide data of their own if they have it in house. Offerors could propose data collection efforts as an option.

Q#61. Will the Government provide Wide Area Motion Imagery (WAMI) real, simulation; or not?

A#61. Yes, as much as possible, but it could include simulation data.

Q#62. Will the Government provide Ground Moving Target Intelligence (GMTI) in this program?

A#62. Yes.

Q#63. What sources of unrestricted data are being considered? Approximate Ground Sample Dis-tances (GSD)? Black and white versus. Color?

A#63. The (red) Challenge team is looking at that now. The goal is to build modules that are extens-ible over many sensors and many target types. Offerors should not depend on specific attributes or resolutions.

Q#64 Can you give examples showing the difference between structured and unstructured datasets?

A#64. Data in relational databases that can be accessed by Structure Query Language (SQL) would be considered structured, whereas motion, images, and natural language text would be considered unstructured.

Q#65. Are there only one data source or multiple sources of motion data?

A#65. All sources will be considered.

Q#66. Where can we get instructions on requesting access to testbed?

A#66. Selected awardees will receive instructions for accessing the testbed.

Q# 67. Are there existing Department of Defense or other interface standards that prototypes will be expected to comply with?

A#67. No, the consortium will determine interface formats by consensus. The goal is to build the best algorithms and the build team will worry about standards.

Q#68. Are industry-university partnerships desirable?

A#68. See Section III entitled “Eligibility Information” of BAA 11-001.

Q#69. What are the expected technical maturity level for entry into program and the goal for maturity upon completion?

A#69. Applied Research (6.2 funds).

Q#70. Has the Service Oriented Architecture (SOA) architecture already been determined; or will offerors have input into its design?

A#70. The D2D architecture is evolving. The only requirement is the system be a government-owned open architecture consisting of off-the-shelf components where possible.

Q#71. In Special Notice 11-SN-0004, there is a discussion of link analysis trees as the output of tar-get tracking. Can the Government give an example of the link analysis?

A#71. That is one example of the types of products. The goal is not just to build a link analysis tree;

it is to build motion analytic algorithms that could be used in applications like link analysis trees.

Q#72. Will the government actively manage this consortium; or will the consortium have to self manage?

A#72. The government anticipates active management of the consortium.

Q#73. Will the government consider the combined offerings from the offerors (in essence as-sembling the strongest team); or will they select the highest rated individual offerors and then have these modify their efforts to build the right mix

A#73. The government will select those proposals that offer the best value to the government;

and fulfill programmatic objectives.

Q#74. How much detail does the government expect for the out-year proposals? Would it be acceptable to provide a fairly high level of abstraction for this?

A#74. The government expects a highly detailed approach for the first spiral development cycle and a broad outline of the approach for subsequent spirals.

Q#75. How do you want us to propose if we have several separate and independent module ideas? If options, it will be hard to award if you do not want to select the "base" technology.

A#75. Each offeror should be a cohesive and integrated set of ideas and approaches to solving the challenge problem posed in the BAA which should be reflected in the Statement of Work. If the solutions are distinctly separable, they should be submitted as separate proposals.

Q#76. Can a single organization submit multiple proposals?

A#76. A single organization can submit multiple proposals. However, each proposal should be a cohesive and integrated set of ideas and approaches to solving the challenge problem posed in the BAA which should be reflected in the Statement of Work. If the solutions are distinctly separable, they should be submitted as separate proposals.

Q#77. If proposers form a team, should they submit a single proposal for the team (and exceed rec-ommended funding level), or submit separate proposals that point to the other proposals in the team?

A#77. The team members should submit a separate proposal that identify the other team members.

However, if that team member is identified as a subcontractor then that team member should be part of the prime’s proposal. The prime and subcontractor(s) are responsible for ensuring that their proposal package is submitted by the closing date and time.

Q#78. We have existing software libraries that address the motion analytics challenges. These libra-ries are protected with Small Business Innovation Research (SBIR) and International Traffic in Arms Regulation (ITAR) restrictions. Will SBIR rights be protected?

A#78. This BAA is for developers that want to collaborate on new ideas. If you believe that you have already solved the problem using your own independent research and development funding, you should submit your proposal package under the Long Range BAA 11-001.

Q#79. What information should be provided to indicate teaming? For instance industry/academic partner?

A#79. Offerors should provide the point of contact information (organization’s name, point of contact at the organization, address, telephone number, e-mail address, etc.) for their proposed team.

Q#80. If our technical approach includes the work of a subcontractor, should we proceed as if the partnering team is a subcontractor, including their cost in our proposal as well as their technical contribution, realizing that you will pull the cost out as a separate contract?

A#80. Subcontractor technical and cost information should be included in the prime’s proposal.

Subcontractors may submit their detailed cost proposal directly to the point of contact information stated in the Special Notice.

Q#81. What is the submission date and time for Full Proposals? (i.e., Received by March 29th or post marked by March 29th?)

A#81. The submission date and time for full proposals has been extended. Full proposals are due no later than 4:30 PM (Daylight Standard Time) on Thursday, 21 April 2011. Any proposal pack-ages not received by that date will not be included in the funding decisions.

Q#82. Can we tailor the cost proposal template? E.g., using “employee benefits” instead of “fringe benefits”?

A#82. The cost proposal template can be tailored to do so.

Q#83. Depending on the size of a company, it may have multiple technological strengths to bring to bear on D2D – (ex: activity recognition, geo-registration, target-tracking, anomaly-tracking, etc.)

Does the Government except separate proposals for each technical/strength contributed, or a sin-gle proposal with options?

A#83. Offerors should submit a single proposal with options.

Q#84. For large companies with multiple business units, can they submit one bid? Is prime/sub-prime relationships allowed within one company?

A#84. A company with multiple business units can submit one proposal provided that each item in the statement of work is fully described and has its own budget. Prime/subcontractor relationships are allowed.

Q#85. Are there no prime-subcontract relations allowed? Are subcontracts allowed? For example, a small university team without the infrastructure to bid alone?

A#85. Prime/subcontractor relationships are allowed.

Q#86. For $650K> proposal what do you mean by small businesses as subcontracting? How is it different from the no prime/sub format?

A#86. The Office of Naval Research is committed to providing meaningful subcontracting opportun-ities. In accordance with FAR 52.219-9, a subcontracting plan is required for awards to be made in the form of contracts that exceed $650,000. The subcontracting plan requirement is for those offe-rors other than small businesses.

Q#87. Would each offeror need an Audit or Rate Check? Explain.

A#87. The purpose of the audit or rate check is to ensure that the contractor has an acceptable ac-counting system in order to receive a cost type contract. The audit or rate check is also performed to verify the proposed direct and indirect rates.

Q#88. Will ONR anticipate the use of award fee contracts?

A#88. The use of award fee contracts is not anticipated.

Q#89. Are U.S. permanent-resident permitted for Industry?

A#89. Research in the areas that cover export controlled technologies are limited to “U.S. persons”.

Q#90. Is there a simpler contract vehicle that small organizations can use? The inability to subcon-tract plus the substantial DCAA hurdles will surely stifle small companies.

A#90. There is no simpler contract vehicle that is used for small organizations.

Q#91. Are DCAA related expenses acceptable as line items in the proposal?

A#92. DCAA related expenses are not acceptable as line items in the proposal.