2018-NIST-PSIAP-MCVQOE NOFO.pdf

NIST Public Safety Innovation Accelerator Program – MCVQoE

Notice of Funding Opportunity

May 23, 2018

NOTICE OF FUNDING OPPORTUNITY (NOFO)

NIST Public Safety Innovation Accelerator Program – Mission Critical Voice

Quality of Experience

EXECUTIVE SUMMARY

• Federal Agency Name: National Institute of Standards and Technology (NIST), United States Department of Commerce (DoC)

• Funding Opportunity Title: NIST Public Safety Innovation Accelerator Program

– Mission Critical Voice Quality of Experience

• Announcement Type: Initial

• Funding Opportunity Number: 2018-NIST-PSIAP-MCVQOE

• Catalog of Federal Domestic Assistance (CFDA) Number: 11.609, Measurement and Engineering Research and Standards

• Dates: Electronic applications must be received at grants.gov no later than 11:59 p.m. Eastern Time, Friday, August 3, 2018. Applications received after this deadline will not be reviewed or considered. Paper applications will not be accepted.

Applicants should be aware, and factor in their application submission planning, that the Grants.gov system is expected to be closed for routine maintenance from 12:01 a.m. Eastern Time, Saturday, June 16, 2018 until Monday, June 18, 2018 at 6:00 a.m. Eastern Time; and again from 12:01 a.m. Eastern Time, Saturday, July 21, 2018 until Monday, July 23, 2018 at 6:00 a.m. Eastern Time.

Applications cannot be submitted when Grants.gov is closed. NIST expects to complete its review, selection of successful applicants, and award processing by October 2018. NIST expects the earliest start date for awards under this NOFO to be November 1, 2018.

Applicants are strongly urged to read Section IV.2.b. Attachment of Required Application Documents of this NOFO with great attention. Applicants should carefully follow the instructions and recommendations regarding attachments and use the Download Submitted Forms and Applications feature on www.grants.gov to check that all required attachments were contained in their submission.

https://www.grants.gov/

Applications submitted without the required documents will not pass the Initial Administrative Review, described in Section V.3.a. of this NOFO.

When developing the submission timeline, please keep in mind that: (1) all applicants are required to have a current registration in the electronic System for Award Management (SAM.gov); (2) the free annual registration process in the SAM.gov (see Section IV.3. and Section IV.7.a.(1).(b) of this NOFO) often takes between three and five business days and may take as long as two weeks; (3) electronic applicants are required to have a current registration in Grants.gov;

and (4) applicants using Grants.gov will receive e-mail notifications over a period of up to two business days as the application moves through intermediate systems before the applicant learns via a validation or rejection notification whether NIST has received the application. (See grants.gov for full information on application and notification through Grants.gov). Please note that a federal assistance award cannot be issued if the designated recipient’s registration in the System for Award Management (SAM.gov) is not current at the time of the award.

• Application Submission Address: Applications must be submitted using Grants.gov.

• Funding Opportunity Description: The NIST Public Safety Innovation Accelerator Program (PSIAP) is seeking applications from eligible applicants for activities to measure the impact of impairments to mission critical voice (MCV) systems and quantify the quality of experience (QoE) for first responders.

• Anticipated Amounts: In FY2019, NIST anticipates funding several awards for $250,000 to $1,000,000 per year with project performance periods of up to three

(3) years. Funding for the first year of awards is anticipated to total up to approximately $3,000,000.

• Funding Instrument: Grant or cooperative agreement, as appropriate.

• Eligibility: This NOFO is open to all non-Federal entities. Eligible applicants include institutions of higher education, non-profit organizations, for-profit organizations, state and local governments, Indian tribes, hospitals, foreign public entities, and foreign governments. Please note that individuals and unincorporated sole proprietors are not considered “non-Federal entities” and are not eligible to apply under this NOFO. Although Federal entities are not eligible to receive funding under this NOFO, they may participate as unfunded collaborators.

http://grants.gov/

NIST will only consider one application per applicant; however, an applicant entity may be proposed as a sub-recipient, contractor, or unfunded collaborator within applications submitted by other entities.

• Cost Sharing Requirements: Matching funds are not required. However, voluntary committed cost-share (e.g., in-kind contributions from partners) may be included in project budgets and will be considered as part of the selection factors in Section V.2. of this NOFO.

• Public Website, Frequently Asked Questions (FAQs) and Webinar: NIST has a public website (pscr.gov) that provides information pertaining to this Funding Opportunity1. NIST anticipates that a “Frequently Asked Questions” section or other resource materials will be maintained and updated on the website as needed to provide additional guidance and clarifying information that may arise related to this Funding Opportunity. Any amendments to this NOFO will be announced through Grants.gov.

Applicants must submit all questions pertaining to this funding opportunity via e-mail to pscr@nist.gov with ‘MCVQoE’ in the subject line. Questions submitted to NIST may be posted on pscr.gov. Alternatively, applicants may ask questions during the informational public webinar as described in the next paragraph.

NIST will host a webinar to provide general information regarding this NOFO, offer general guidance on preparing applications, and answer questions.

Scheduling details about the webinar will be available at pscr.gov. Proprietary technical discussions about specific project ideas will not be permitted during the webinar and NIST staff will not critique or provide feedback on specific project ideas while they are being developed by an applicant, brought forth during the webinar, or at any time before the deadline for all applications. However, general questions about the PSIAP, eligibility requirements, evaluation and award criteria, selection process, and the general characteristics of a competitive application will be addressed at the webinar and by e-mail to pscr@nist.gov as described in the previous paragraph. There is no cost to attend the webinar, but participants must register in advance. Participation in the webinar is not required, and will not be considered in the application review and selection process.

Additional information on the PSIAP and webinar is available at pscr.gov.

1 Refer to Section VII. of this NOFO, Federal Awarding Agency Contacts, Programmatic and Technical Questions, if this link is no longer working or more information is needed.

http://www.nist.gov/ctl/pscr mailto:pscr@nist.gov http://www.nist.gov/ctl/pscr http://www.nist.gov/ctl/pscr mailto:pscr@nist.gov https://www.nist.gov/ctl/pscr

Table of Contents

I. Program Description II. Federal Award Information III. Eligibility Information IV. Application and Submission Information V. Application Review Information VI. Federal Award Administration Information VII. Federal Awarding Agency Contacts VIII. Other Information

FULL ANNOUNCEMENT TEXT

I. Program Description

The statutory authority for the NIST Public Safety Innovation Accelerator Program is 15 U.S.C. § 3706 and 47 U.S.C. § 1443.

The NIST Public Safety Innovation Accelerator Program (PSIAP) is seeking applications from eligible applicants for activities to measure the impact of impairments to mission critical voice (MCV) systems and quantify the quality of experience (QoE) for first responders as described in this section of the NOFO.

Specifically, PSIAP-Mission Critical Voice Quality of Experience (MCVQoE) is seeking applications that address all five of the following goals: (1) Simulated Land Mobile Radio (LMR) Equipment Development, (2) Development of Test Facilities, Test Scenarios, and Test Protocols for MCV QoE Tests, (3) Development of Public Safety Testing Cadre, (4) Test and Measurement of Public Safety Users’ Performance, and (5) Analysis and Modeling of Public Safety User QoE. These goals, and specific technical areas of research in the PSIAP-MCVQoE program, are discussed in detail in Section I.D. of this

NOFO.

A. Background

The Public Safety Communications Research (PSCR) division, housed within the NIST Communications Technology Laboratory, conducts research for the first responder community. In 2012, NIST was mandated by Congress2, to, among other things, “accelerate the development of mission critical voice … for the nationwide public safety broadband network”. PSCR conducted extensive stakeholder engagement starting in 2013 to address the requirements of the legislation, including hosting two roundtable events in 2016 and 2017 focused on MCV with stakeholders from public safety, 2 47 U.S.C. § 1443(b)(4) – Public safety wireless communications research and development government, and industry. The second roundtable focused exclusively on refining the concept of QoE for public safety MCV. The PSIAP-MCVQoE is one of several current opportunities within the NIST PSCR program. In addition to funding internal research addressing these areas, in 2017 the PSIAP awarded $14M for ten grants to address aspects related to the MCV portfolio goals. More information about the MCV portfolio, PSIAP, and PSCR can be found at pscr.gov.

The public safety community’s use of two-way LMR systems spans over eight decades;

these systems are a critical tool for every first responder on the job today and represent the current solution for MCV. While the specific technological implementation and equipment vary, users consistently employ half-duplex radios and contact other users by pressing a button on a radio when they wish to speak – this feature is typically referred to as push-to-talk (PTT). This ability to speak (and listen) allows them to address both individuals and groups of users, depending on the implementation.

Public safety users have an expectation that these systems will function for them reliably and consistently, particularly when performing difficult job tasks in challenging conditions and stressful situations. Unlike commercial cellular systems, LMR systems have been optimized to operate with a high probability of successful transmission and reception for the users.

Previous research3 at the Institute for Telecommunications Science (ITS) has studied mission critical voice quality and intelligibility; however, there is a significant gap in knowledge concerning how other factors, such as the time it takes to access the network, may affect the overall QoE of users. Additionally, while the work performed previously by ITS attempted to duplicate field conditions to some extent, testing was conducted in a lab environment that could not simulate the real operating environments first responders encounter every day.

B. Research Opportunity

The intent of this NOFO is to generate publicly-released data sets and a QoE model that correlates impairments in the communications systems that first responders currently use across the United States to real task performance. Award recipients will help achieve this by creating test devices and infrastructure to simulate these impairments, develop a test capability/methodology for human testing, and apply these tests to a diverse pool of first responders representing multiple disciplines, regions, and demographics while performing critical tasks in realistic operating conditions. This research data will be key in quantifying the impact on real users of public safety mission critical push-to-talk (MCPTT) systems and their corresponding QoE. The data will in turn be used to compare existing land mobile radio (LMR) performance against emerging

3 Reports can be found at https://www.its.bldrdoc.gov/resources/audio-quality-research/publications-talks.aspx.

http://www.nist.gov/ctl/pscr https://www.its.bldrdoc.gov/resources/audio-quality-research/publications-talks.aspx https://www.its.bldrdoc.gov/resources/audio-quality-research/publications-talks.aspx and future communications technologies (currently 4G/5G broadband) for the entire United States public safety community.

C. Basic Competition Information

Applicants must address all the goals (see Section D.) within a single application.

Applicants are required to develop partnerships with eligible public safety organizations (PSOs). Potential applicants are responsible for contacting the organizations and arranging partnerships. NIST will not assist potential applicants with finding partners.

For purposes of this NOFO, eligible PSOs include U.S. federal, state, and local emergency medical services (EMS), fire services, law enforcement, and public safety communications/911 centers. Federal entities, including federal first responders, are not eligible to receive funding under this NOFO, though they may participate as unfunded collaborators.

Applicants must propose projects that include active and sustained engagement with first responders and PSOs. This engagement ensures that the technical outputs of each PSIAP-MCVQoE project are highly relevant and will have a meaningful impact on the public safety community. Applicants are encouraged to include funding in their proposed budget for first responders and public safety personnel to actively participate in their projects, and to budget significant time and sufficient travel for this interaction.

Applicants should ensure that their projects are tailored to disseminate their ideas, findings, and technology to the public safety stakeholder community. Dissemination may be achieved through publications and technology transfer, including but not limited to:

commercialization, training, or the release of tools, designs, and/or data sets. Further, to assist the PSIAP with generating metrics to understand and demonstrate the long-term value of these outputs, applicants must describe any use of feedback, licensing (e.g., Creative Commons, Open Data Commons, etc.), citations, persistent identifiers, page views, downloads, or other means of measuring research impacts. Applicants may include costs to support the required dissemination of the results and lessons of their PSIAP-MCVQoE research and development (R&D) efforts to the public safety stakeholder community in their proposed budgets.

Recipients will be required to attend the PSCR Public Safety Broadband Stakeholder Meetings each June during the term of the award to meet with stakeholders from public safety, government, industry, and academia and to present key plans and findings of their work to date. This event should be reflected in the project schedule and budget.

D. Technical Program Goals Recipients will rapidly accelerate the objectives of the PSIAP-MCVQoE by proposing innovative R&D projects addressing each of the following five research goals.

1. Goal 1 – LMR Simulation Tools

During the 2017 PSCR MCV Roundtable, public safety users and industry experts were asked to identify expectations and metrics that would enable PSCR to understand, measure, monitor, and predict MCV QoE across LMR, Long-Term Evolution (LTE), and future technologies. identified key factors to consider in a QoE framework for MCV. Four of the key performance indicators (KPIs) that were identified relative to public safety PTT systems were:

a. Mouth to Ear Latency – The time delay between a user speaking on one radio and a user hearing the speaker on another radio in an open radio channel.

b. Access Time – The minimum length of time a user must wait between pressing a PTT button on a radio and starting to speak to ensure that the start of the message is not lost.

c. Link Establishment and Retention – The ability of users of a system to establish and retain communication links with other users.

d. Speech Intelligibility and Background Audio Quality – The ability of the user to understand received voice messages and to identify other transmitted sounds.

To test how each of these KPIs affects public safety job/task performance, applicants must develop a tool that will allow researchers to vary, at a minimum, the four KPIs above, both independently and concurrently, while simultaneously measuring the QoE of public safety practitioners communicating with LMR equipment currently leveraged by the public safety community during realistic scenarios. This simulator should allow the simultaneous recording of KPI values and QoE measurements so that relationships between the two sets of metrics may be studied.

All variables except for audio quality and intelligibility are technology agnostic relative to user perception. Speech intelligibility and background audio quality are technology dependent. Awardees should have the ability to simulate both traditional analog FM narrowband radios and modern digital radios. In the case of digital, awardees should be able to simulate radios that comply with the Association of Public-Safety Communications Officials (APCO) Project 25 (P25) standard using the Digital Voice Systems, Inc. AMBE+2TM full rate (9.6 kbps) vocoder.

2. Goal 2 – Development of Test Facilities, Scenarios, and Protocols

To measure the performance of public safety users with the LMR simulation tools from Goal 1, facilities that mimic real-life operational environments for public safety users must be used or developed. The test facilities that are required for this activity would ideally be existing facilities used by public safety users for operational training.

The test scenarios that mimic the real operations of public safety users must be developed with the experience and expertise of real public safety practitioners. These test scenarios should be applicable to a diverse group of public safety users, both in geography and job function. Test protocols must be developed to quantify task success, task efficiency, and other attributes of the user experience.

3. Goal 3 – Development of Public Safety Testing Cadre

Awardees must develop/recruit a cadre of public safety professionals to serve as test subjects for QoE testing. The testing cadre should consist almost exclusively of full-time practitioners rather than non-operational leadership staff, part-time practitioners, and retirees. This cadre of public safety users must have reasonable representation from the major first responder job groups (police, fire, emergency medical), and could be further diversified in specialties (police tactical units, explosive ordinance disposal, wildland fire, hazardous materials handling) and support functions (dispatch). The group also must also have reasonable diversity within geographic areas (urban, suburban, rural) and public safety agency size. The group of individuals tested must be sufficiently large enough that collected data converges to reasonable, repeatable statistical distributions.

4. Goal 4 – Test and Measurement of Public Safety User Performance

Awardees will use facilities, scenarios, and protocols from Goal 2, the cadre from Goal 3, and the LMR simulation tools from Goal 1 to conduct first responder operational task simulations and collect data on their performance as a function of the variables listed in Goal 1.

Tests and measurements must be conducted in a scientific, repeatable manner.

Baseline performance of the simulated LMR system must be known, and uncertainties associated with the measurements must be defined. Outputs from PSIAP-MCVQoE, e.g., raw data and results from measurements, must be made publicly available (see sections IV.2.a.(7)(a)(ii) and IV.2.a.(12) of this NOFO).

5. Goal 5 – Analysis and Modeling of Public Safety User QoE

Awardees should use the data from Goal 4 to develop mathematical/functional models describing public safety QoE for MCV. These models will describe the measured relationships between independent variables (those listed in Goal 1 and controlled in Goal 4) and the dependent variables (the performance-related QoE measurements obtained in Goal 4). In addition to describing the measured relationships, the models should generalize (to related situations and environments) and scale (to additional numbers of users) to the maximum extent possible.

II. Federal Award Information

1. Funding Instrument The funding instrument used in this program will be grants or cooperative agreements, as appropriate. Where cooperative agreements are used, the nature of NIST’s “substantial involvement” will generally include collaboration with the recipients in the scope of work.

2. Multi-Year Funding Policy When an application for a multi-year award is approved, funding will usually be provided for only the first year of the project. If a project is selected for funding, NIST has no obligation to provide any additional funding in connection with that award. Continuation of an award to increase funding or extend the period of performance is at the sole discretion of NIST. Continued funding will be contingent upon satisfactory performance, continued relevance to the mission and priorities of the PSIAP-MCVQoE, and the availability of funds. Under this NOFO, NIST may elect to fully fund awards or to fund awards in accordance with the Multi-Year Funding policy.

3. Funding Availability In FY2019, NIST anticipates funding several awards for $250,000 to $1,000,000 per year with project performance periods of up to three (3) years. Funding for the first year of awards is anticipated to total up to approximately $3,000,000

III. Eligibility Information

1. Eligible Applicants This NOFO is open to all non-Federal entities. Eligible applicants include institutions of higher education, non-profit organizations, for-profit organizations, state and local governments, Indian tribes, hospitals, foreign public entities, and foreign governments.

Please note that individuals and unincorporated sole proprietors are not considered “non-Federal entities” and are not eligible to apply under this NOFO. Although Federal entities are not eligible to receive funding under this NOFO, they may participate as unfunded collaborators.

NIST will only consider one application per applicant; however, an applicant entity may be proposed as a subrecipient, contractor, or unfunded collaborator within applications submitted by other entities.

2. Cost Sharing or Matching Matching funds are not required. However, voluntary committed cost-share (e.g., in-kind contributions from partners) may be included in project budgets and will be considered as part of the selection factors in Section V.2. of this NOFO.

IV. Application and Submission Information

1. Address to Request Application Package The application package is available at grants.gov under Funding Opportunity Number

2018-NIST-PSIAP-MCVQOE.

2. Content and Form of Application Submission

a. Required Forms and Documents. The Application must contain the following:

(1) SF-424, Application for Federal Assistance. The SF-424 must be signed by an authorized representative of the applicant organization.

• SF-424, Item 8.d Zip/Postal Code field should reflect the zip code + 4 (#####

- ####) format.

• Item 12, should list the NOFO number 2018-NIST-PSIAP-MCVQOE.

• SF-424, Item 18, field a., should list the total Federal budget amount requested for the entire project.

• If the applicant includes voluntary committed cost share, SF-424, Item 18, field b., should list the applicant’s anticipated voluntary committed cost share for the entire project.

• For SF-424, Item 21, the list of certifications and assurances is contained in the SF-424B.

(2) SF-424A, Budget Information - Non-Construction Programs. The budget should reflect anticipated expenses for the project, considering all potential cost increases, including cost of living adjustments.

The Grant Program Function or Activity on Section A, Line 1, Column (a) should be entered as “Public Safety Communications Research Grant Program”. The Catalog of Federal Domestic Assistance Number on Line 1, Column (b) should be entered as “11.609”. Line 1, Column (e) should reflect the anticipated federal expenses for the full term of the project. If the applicant intends to include voluntary committed cost share, that voluntary committed cost share for the full term of the project should be reflected on Section A, Line 1, Column (f).

Use Section B – Budget Categories to reflect the budget breakdown by year.

Column (1) should be used for the first year. Column (2) should be used for the second year. Column (3) should be used for the third year.

Section C, line 8., column (b) of the SF-424A should reflect the applicant’s anticipated voluntary committed cost share for the full term of the project.

Enter a quarterly breakdown of the first year’s budget of federal funds in Section D, line 13.

http://www.grants.gov/

Section E, line 16, column (b) should reflect the second year’s budget of federal funds. Enter the third year’s budget of federal funds in column (c).

(3) SF-424B, Assurances - Non-Construction Programs

(4) CD-511, Certification Regarding Lobbying. Enter “2018-NIST-PSIAP-

MCVQOE” in the Award Number field. Enter the title of the application used in field 15 of the SF-424, or an abbreviation of that title, in the Project Name field.

(5) SF-LLL, Disclosure of Lobbying Activities (if applicable)

(6) Quad Chart. The quad chart must contain a problem statement, the concept of the proposed project, the potential impact of the project, and key milestones and/or deliverables. The quad chart must not exceed one (1) page. Any materials provided beyond the one (1) page limit for the quad chart will be redacted and not provided to the reviewers.

(7) Technical Proposal. The Technical Proposal is a document of no more than twenty-five (25) pages total, which is responsive to the program description (see Section I. of this NOFO) and the evaluation criteria (see Section V.1. of this NOFO). The Technical Proposal must contain the following information:

(a) Project Description. This is a detailed description of the proposed project and potential impacts. This section should include a:

(i) Detailed description of the approach to address the goals and requirements of this NOFO and how the proposed activities will impact and benefit public safety.

(ii) Proposed list of technical outputs, details on how these outputs will be made available to the broader public safety and R&D communities, and how their adoption and impact will be tracked.

This section will be evaluated in accordance with the Technical Merit evaluation criteria: (see Section V.1.a. of this NOFO). The Project Description section must not exceed fifteen (15) single-sided pages. Any material provided beyond this page limit will be redacted and not provided to the reviewers.

(b) Project Execution. This section should provide clear and quantifiable goals, milestones, timelines, and methods to assess progress and performance towards achieving the objectives in the Technical Proposal. In addition, this section should describe the cost effectiveness and value of the proposed project in terms of meeting the stated objectives.

This section will be evaluated in accordance with the Project Execution evaluation criterion (see Section V.1.b. of this NOFO). The Project Execution section must not exceed five (5) single-sided pages. Any material provided beyond this page limit will be redacted and not provided to the reviewers.

(c) Qualifications. This section should provide detailed examples demonstrating:

(i) Qualifications of the individuals, teams, and organizations that will be executing or supporting the proposed project relative to the program goals.

(ii) Demonstrated success executing projects similar in scope and magnitude to the proposed project, including any examples of working with a broad cross-section of public safety practitioners.

(iii) Access to the necessary staff, equipment, facilities, support, and resources to accomplish the proposed objectives.

This section will be evaluated in accordance with the Qualifications evaluation criterion (see Section V.1.c. of this NOFO). The Qualifications section must not exceed five (5) single-sided pages. Any material provided beyond this page limit will be redacted and not provided to the reviewers.

(8) Resume(s). Resumes are required for all key personnel, including the principal investigator/project lead. Awardees are advised that any changes to individuals deemed ‘key personnel’, including disengagement from the project for more than three months or a 25% reduction of effort, will require prior approval, in writing, by the NIST Grants Officer.

This section will be evaluated in accordance with the Qualifications evaluation criterion (see Section V.1.c. of this NOFO). There are no page limits for resumes.

However, any material considered inappropriate for inclusion in a resume (e.g.

supplementary text for another section) will be redacted and not provided to the reviewers.

Resumes do not count against the twenty-five (25) page limit of the Technical Proposal.

(9) Budget Narrative. The Budget Narrative must provide a detailed breakdown of each of the object class categories as reflected on the SF-424A. The budget justification should address all the budget categories (personnel, fringe benefits, equipment, travel, supplies, other direct costs and indirect costs) for which Federal funds are requested. The written justification should include the necessity and the basis for the cost. Proposed funding levels must be consistent with the project scope, and only allowable costs should be included in the budget. Information on cost allowability is available in the Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards at 2 C.F.R. Part 200 (http://go.usa.gov/SBYh), which apply to awards in this program.

Voluntary committed cost share must be included in the appropriate categories.

Information needed for each category is as follows:

(a) Personnel. At a minimum, the budget justification for all personnel should include the following: name, job title, commitment of effort on the proposed project in terms of average number of hours per week or percentage of time, salary rate, total direct charges on the proposed project, description of the role of the individual on the proposed project and the work to be performed.

(b) Fringe Benefits. Fringe benefits should be identified separately from salaries and wages and based on rates determined by organizational policy. The items included in the fringe benefit rate (e.g. health insurance, parking, etc.)

should not be charged under another cost category.

(c) Travel. For travel costs required by the recipient to complete the project, including attendance at any relevant conferences, the budget justification for travel should include the following: destination; names and number of people traveling; dates and/or duration; mode of transportation, lodging and subsistence rates; and description of how the travel is directly related to the proposed project. For travel that is yet to be determined, please provide best estimates based on prior experience. If a destination is not known, an approximate amount may be used with the assumptions given for the location of the meeting.

As described in the Program Description (see Section I. of this NOFO), recipients will be required to attend the PSCR Public Safety Broadband Stakeholder Meetings (PSBSM) to meet with stakeholders and present key plans and findings of their work to date. Recipients must send a minimum of one team member to each event during the term of the grant. The PSCR PSBSMs are held each June in the continental United States and are typically four days in length. The dates and locations of the PSBSM for 2019 – 2022 have not been determined at this time. Applicants should factor in the cost for attending this event in their budget narrative and SF424A form.

(d) Equipment. Equipment is defined as an item of property that has an acquisition cost of $5,000 or more (unless the organization has established lower levels) and an expected service life of more than one year. Any items that do not meet the threshold for equipment can be included under the supplies line item. The budget justification should list each piece of equipment, the cost, and a description of how it will be used and why it is necessary to the successful completion of the proposed project. Please note http://go.usa.gov/SBYh that any general use equipment (computers, etc.) charged directly to the award should be allocated to the award according to expected usage on the project. Applicants should provide at least two (2) quotes, if available, for equipment costing $25,000 or more. If two (2) quotes are not available, please provide a statement as to why two (2) quotes are not available.

(e) Supplies. Supplies are defined as all tangible personal property other than those described in equipment (see Section IV.2.a.(9)(d) of this NOFO).

Provide a list of each supply necessary for the project, and the breakdown of the total costs by quantity or unit of cost. Include the necessity of the cost for the completion of the proposed project.

(f) Contracts/Subawards. Each contract or subaward should be treated as a separate item. Describe the services to be provided and the necessity of the subaward or contract to the successful performance of the proposed project.

Contracts are for obtaining goods and services. Subawardees perform part of the project scope of work. For each subaward, applicants must provide budget detail justifying the cost of the work performed on the project.

(g) Construction. Applications to PSIAP-MCVQoE may not include construction costs as part of their budget.

(h) Other Direct Costs. For costs that do not easily fit into the other cost categories, e.g. publishing fees or software distribution expenses, please list the cost and the breakdown of the total costs by quantity or unit of cost.

Include the necessity of the cost for the completion of the proposed project.

Only allowable costs can be charged to the award.

The Budget Narrative does not count against the twenty-five (25) page limit of the Technical Proposal.

This section will be evaluated in accordance with the Project Execution evaluation criteria (see Section V.1.b. of this NOFO). It will also be reviewed to determine if all costs are reasonable, allocable, and allowable under 2 C.F.R.

Part 200 Subpart E, Cost Principles.

(10) Indirect Cost Rate Agreement. If indirect costs are included in the proposed budget, provide a copy of the approved negotiated agreement if this rate was negotiated with a cognizant Federal audit agency. If the rate was not established by a cognizant Federal audit agency, provide a statement to this effect. If the successful applicant includes indirect costs in the budget and has not established an indirect cost rate with a cognizant Federal audit agency, the applicant will be required to obtain such a rate in accordance with the Department of Commerce Financial Assistance Standard Terms and Conditions (http://go.usa.gov/xXRxK).

http://go.usa.gov/xXRxK

Alternatively, in accordance with 2 C.F.R. § 200.414(f), applicants that have never received a negotiated indirect cost rate may elect to charge indirect costs to an award pursuant to a de minimis rate of 10 percent of modified total direct costs (MTDC), in which case a negotiated indirect cost rate agreement is not required. Applicants proposing a 10 percent de minimis rate pursuant to 2 C.F.R.

§ 200.414(f) should note this election as part of the budget and budget narrative portion of the application.

(11) Letters of Commitment. Letters of Commitment must be submitted by all funded and unfunded entities that will have an active role in executing the activities outlined in the Technical Proposal. Letters of Commitment must address the level of participation, qualifications of the personnel who will be actively involved, and how successful completion of this project would positively impact their profession or community. Letters of Commitment must also specify any voluntary committed cost-share, including the specific services and/or products to be used in the project. Letters of Commitment must be signed by an individual with sufficient authority to legally bind the organization to its commitment.

Letters of Commitment will be evaluated in accordance with the Qualifications evaluation criteria (see Section V.1.c.) of this NOFO). Letters of commitment do not count against the twenty-five (25) page limit of the Technical Proposal.

(a) Public Safety Organizations. All applications must include at least one Letter of Commitment from an eligible PSO, unless the applicant organization is itself an eligible PSO. Letters of commitment from PSO partners must address the importance of the proposed work as well as potential impact for first responders and public safety in general. The letters of commitment must also include a detailed description of how the PSO will be actively engaged throughout the proposed project. For purposes of this NOFO, eligible PSOs include U.S. federal, state, and local EMS, fire services, law enforcement, and public safety communications/911 centers. Please note that Federal entities are not eligible to receive funding under this NOFO, though they may participate as unfunded collaborators.

(12) Data Management Plan. Consistent with NIST Policy 5700.004, Managing Public Access to Results of Federally Funded Research, and NIST Order 5701.005, Managing Public Access to Results of Federally Funded Research, applicants must include a Data Management Plan (DMP).

4http://www.nist.gov/data/upload/Final-P-5700.pdf 5http://www.nist.gov/data/upload/Final-O-5701_0.pdf http://www.nist.gov/data/upload/Final-P-5700.pdf http://www.nist.gov/data/upload/Final-O-5701_0.pdf

The DMP is a supplementary document of not more than two pages that must include, at a minimum, a summary of proposed activities that are expected to generate data, a summary of the types of data expected to be generated by the identified activities, a plan for storage and maintenance of the data expected to be generated by the identified activities, and a plan describing whether and how data generated by the identified activities will be reviewed and made available to the public. As long as the DMP meets these NIST requirements, it may take the form specified by the applicant’s institution or some other entity (e.g., the National Science Foundation6 or the National Institutes of Health7).

All applications for activities that will generate scientific data using NIST funding are required to adhere to a DMP or explain why data sharing and preservation are not within the scope of the project.

For the purposes of the DMP, NIST adopted the definition of “research data” at 2 C.F.R. § 200.315(e)(3) (available at http://go.usa.gov/3sZvQ).

Reasonable costs for data preservation and access may be included in the application.

The sufficiency of the DMP will be considered as part of the administrative review (see Section V.3.a. of this NOFO); however, the DMP will not be evaluated against any evaluation criteria.

b. Attachment of Required Documents

Items IV.2.a.(1) through IV.2.a.(5) above are part of the standard application package in Grants.gov and can be completed through the download application process.

Items IV.2.a.(6) through IV.2.a.(12) must be completed and attached by clicking on “Add Attachments” found in item 15 of the SF-424, Application for Federal Assistance. This will create a zip file that allows for transmittal of the documents electronically via Grants.gov.

Applicants should carefully follow specific Grants.gov instructions at www.grants.gov to ensure the attachments will be accepted by the Grants.gov system. A receipt from Grants.gov indicates only that an application was transferred to a system. It does not provide details concerning whether all attachments (or how many attachments) transferred successfully. Applicants using Grants.gov will receive a series of e-mail messages over a period of up to two business days before learning whether a Federal agency’s electronic system has received its application.

6http://www.nsf.gov/bfa/dias/policy/dmp.jsp 7http://grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm http://go.usa.gov/3sZvQ http://www.grants.gov/ http://www.nsf.gov/bfa/dias/policy/dmp.jsp http://grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm

Applicants are strongly advised to use Grants.gov’s “Download Submitted Forms and Applications” option to check that their application’s required attachments were contained in their submission.

After submitting the application, follow the directions found at this link, https://go.usa.gov/xQqeh in the Grants.gov Online Users Guide.

If any, or all, of the required attachments are absent from the submission, follow the attachment directions found above, resubmit the application, and check again for the presence of the required attachments.

If the directions found at https://go.usa.gov/xQqeh are not effective, please contact the Grants.gov Help Desk immediately. If calling from within the United States or from a U.S. territory, please call 800-518-4726. If calling from a place outside the United States or a U.S. territory, please call 606-545-5035. E-mails should be addressed to support@grants.gov. Assistance from the Grants.gov Help Desk will be available around the clock every day, except for Federal holidays. Help Desk service will resume at 7:00 a.m. Eastern Time the day after Federal holidays.

Applicants can track their submission in the Grants.gov system by following the procedures at the Grants.gov site (http://go.usa.gov/cjamz). It can take up to two business days for an application to fully move through the Grants.gov system to NIST.

NIST uses the Tracking Numbers assigned by Grants.gov, and does not issue Agency Tracking Numbers.

c. Application Format

(1) Paper, E-mail and Facsimile (fax) Submissions. Will not be accepted.

(2) Figures, Graphs, Images, and Pictures. Should be of a size that is easily readable or viewable and may be landscape orientation. Any figures, graphs, images, or pictures will count toward the fifteen (15) page limit for the Project Description.

(3) Font. Easy to read font (12-point minimum). Smaller type may be used in figures and tables but must be clearly legible.

(4) Page Limits. See the table below:

Section Page Limit Technical Proposal Project Description

Technical Proposal Project Execution https://go.usa.gov/xQqeh https://go.usa.gov/xQqeh mailto:support@grants.gov http://go.usa.gov/cjamz

Technical Proposal Qualifications

Quad Chart 1 Data Management Plan 2

Page Limit Excludes: SF-424, Application for Federal Assistance; SF-424A, Budget Information – Non-Construction Programs; SF-424B, Assurances – Non- Construction Programs; CD-511, Certification Regarding Lobbying; SF-LLL, Disclosure of Lobbying Activities; Resumes of key personnel; Budget Narrative;

Indirect Cost Rate Agreement; and Letters of Commitment.

(5) Page size. 21.6 centimeters by 27.9 centimeters (8 ½ inches by 11 inches).

(6) Application language. English.

d. Application Replacement Pages. Applicants may not submit replacement pages and/or missing documents once an application has been submitted. Revisions can only be made by submitting a complete new application that is received by NIST before the submission deadline.

e. Pre-Applications. Pre-applications are not required under this NOFO.

f. Statement of Intent. To assist NIST in gauging interest and planning for the evaluation process all potential applicants are strongly encouraged to send an e-mail to pscr@nist.gov with ‘MCVQoE’ in the subject line indicating intent to apply. The statement of intent will only be used for competition planning purposes; it will not be used as part of the evaluation process or to eliminate any applicants from consideration under this NOFO. An applicant will receive full consideration under this NOFO if they do not submit a statement of intent.

g. Certifications Regarding Federal Felony and Federal Criminal Tax Convictions, Unpaid Federal Tax Assessments and Delinquent Federal Tax Returns. In accordance with Federal appropriations law, an authorized representative of the selected applicant(s) may be required to provide certain pre-award certifications regarding Federal felony and Federal criminal tax convictions, unpaid Federal tax assessments, and delinquent Federal tax returns.

3. Unique Entity Identifier and System for Award Management (SAM) Pursuant to 2 C.F.R. part 25, applicants and recipients (as the case may be) are required to: (i) be registered in SAM before submitting its application; (ii) provide a valid unique entity identifier in its application; and (iii) continue to maintain an active SAM registration with current information at all times during which it has an active Federal award or an application or plan under consideration by a Federal awarding agency, unless otherwise excepted from these requirements pursuant to 2 C.F.R. § 25.110.

NIST will not make a Federal award to an applicant until the applicant has complied with all applicable unique entity identifier and SAM requirements and, if an applicant has not fully complied with the requirements by the time that NIST is ready to make a Federal award pursuant to this NOFO, NIST may determine that the applicant is not qualified to receive a Federal award and use that determination as a basis for making a Federal award to another applicant.

4. Submission Dates and Times

Applications must be received at Grants.gov no later than 11:59 p.m. Eastern Time, Friday, August 3, 2018. NIST will consider the date and time recorded by Grants.gov as the official submission time. Applications received after this deadline will not be reviewed or considered. Paper applications will not be accepted.

Applicants should be aware, and factor in their application submission planning, that the Grants.gov system is expected to be closed for routine maintenance from 12:01 a.m.

Eastern Time, Saturday, June 16, 2018 until Monday, June 18, 2018 at 6:00 a.m.

Eastern Time; and from 12:01 a.m. Eastern Time, Saturday, July 21, 2018 until Monday, July 23, 2018 at 6:00 a.m. Eastern Time. Applications cannot be submitted when Grants.gov is closed.

When developing the submission timeline, please keep in mind that: (1) all applicants are required to have a current registration in the electronic System for Award Management (SAM.gov); (2) the free annual registration process in the SAM.gov (see Sections IV.3. and IV.7.a.(1).(b). of this NOFO) often takes between three and five business days and may take as long as two weeks; (3) applicants are required to have a current registration in Grants.gov; and (4) applicants using Grants.gov will receive e-mail notifications over a period of up to two business days as the application moves through intermediate systems before the applicant learns via a validation or rejection notification whether NIST has received the application. (See grants.gov for full information on application and notification through Grants.gov.). Please note that a Federal assistance award cannot be issued if the designated recipient’s registration in the System for Award Management (SAM.gov) is not current at the time of the award.

5. Intergovernmental Review Applications under this Program are not subject to Executive Order 12372.

6. Funding Restrictions Applications for product development and/or commercialization are not considered responsive to this NOFO. Profit or fee is not an allowable cost.

7. Other Submission Requirements http://www.grants.gov/

a. Applications must be submitted electronically.

(1) Applications must be submitted via Grants.gov at grants.gov.

(a) Applicants should carefully follow specific Grants.gov instructions to ensure that all attachments will be accepted by the Grants.gov system. A receipt from Grants.gov indicating an application is received does not provide information about whether attachments have been received. For further information or questions regarding applying electronically for the 2018-NIST-PSIAP- MCVQOE announcement, contact Christopher Hunton by phone at 301-975- 5718 or by e-mail at grants@nist.gov.

(b) Applicants are strongly encouraged to start early and not wait until the approaching due date before logging on and reviewing the instructions for submitting an application through Grants.gov. The Grants.gov registration process must be completed before a new registrant can apply electronically. If all goes well, the registration process takes three to five business days. If problems are encountered, the registration process can take up to two weeks or more. Applicants must have a valid unique entity identifier number and must maintain a current registration in the Federal government’s primary registrant database, the System for Award Management (https://www.sam.gov/), as explained on the Grants.gov Web site (also see Section IV.4. of this NOFO). After registering, it may take several days or longer from the initial log-on before a new Grants.gov system user can submit an application. Only individuals authorized as organization representatives will be able to submit the application, and the system may need time to process a submitted application. Applicants should save and print the proof of submission they receive from Grants.gov. If problems occur while using Grants.gov, the applicant is advised to (a) print any error message received and (b) call Grants.gov directly for immediate assistance. If calling from within the United States or from a U.S. territory, please call 800-518-4726. If calling from a place other than the United States or a U.S. territory, please call 606- 545-5035. Assistance from the Grants.gov Help Desk will be available around the clock every day, except for Federal holidays. Help Desk service will resume at 7:00 a.m.