DTDP RFP - FINAL - Amendment 02.pdf

1. INTRODUCTION

The First Responder Network Authority (FirstNet Authority) is issuing this competitive Request for Proposal (RFP) in order to enter into a contract to meet the requirements specified in this document. The FirstNet Authority will conduct this acquisition using Federal Acquisition Regulation (FAR) Part 12, Acquisition of Commercial Items, and FAR Part 15, Contracting by Negotiation.

1.1 BACKGROUND

The Middle-Class Tax Relief Act of 2012 established the FirstNet Authority and charged it with the responsibility to ensure the building, deployment, and operation of a Nationwide Public Safety Broadband Network (NPSBN). Designed to meet the information and communications technology needs of first responders and public safety users, the NPSBN is the first interoperable, nationwide, high-speed network dedicated to public safety. The FirstNet Authority is an independent authority within the National Telecommunications and Information Administration (NTIA) under the Department of Commerce.

The FirstNet Authority fulfills a fundamental need of the public safety community and a key recommendation of the 9/11 Commission. Using a nationwide spectrum license, the FirstNet NPSBN is a single platform for daily public safety communications. When natural disasters, threats to the nation’s security, or other emergencies occur anywhere within the 50 States and/or U.S. Territories, the FirstNet NPSBN enables local, state, regional and national emergency responders to communicate

1.2. ANTICIPATED AWARD TYPE

The Government anticipates an award of a contract utilizing both firm fixed price (FFP) and Time-and-Materials (T&M) contract line item numbers (CLINs).

[THE REMAINDER OF THIS PAGE INTENTIONALLY LEFT BLANK]

For purposes of this DTDP, rural geography will be defined using the Office of Management and Budget (OMB) definition of rural and the U.S. Census Bureau 2010 Census Data. OMB designates counties as Metropolitan, Micropolitan, or Neither.

A Metro area contains a core urban area of 50,000 or more population, and a Micro area contains an urban core of at least 10,000, but less than 50,000, population. All counties that are not part of a Metropolitan Statistical Area (MSA) are considered rural. Micropolitan counties are considered non-Metropolitan or rural along with all counties that are not classified as either Metro or Micro.

2. DESCRIPTION AND PERFORMANCE WORK STATEMENT (PWS)

2.1. SCOPE AND OBJECTIVES

The objective of the Drive Test Data Program (DTDP) specifically is to gather, process, analyze and report network coverage and performance data for the NPSBN to prospectively inform FirstNet Authority actions. This data will be utilized to ensure continued coverage compliance and to understand the network coverage experienced by public safety throughout the deployment and growth of the

NPSBN.

The scope of work includes all labor, equipment, data, tools, software, support, storage and materials needed to collect and generate top-quality performance analytics, visualization, and reporting to support the FirstNet Authority’s on-going mission.

The Drive Test Data Program (DTDP) consists of the following three phases as described in the Table below. The requirements and deliverables are specified below for each phase. Each phase of the Drive Test Data Program shall be completed in the specified time interval.

Table 1: Drive Test Data Program (DTDP) phases and scope.

Phase Option Duration Period Scope 1 Base Period 5 months 6/1/21 – 10/31/21 Drive test data collection, post-processing and analysis for 15 states 2 Option Period 1 7 months 11/1/21 – 5/31/22 Drive test data collection, post-processing and analysis for 15 states; additional optional drive test data collection for 100k miles, and optional In-door testing up to 100 buildings

3 Option Period 2 1 full year 6/1/22 – 5/31/23 Drive test data collection, post-processing and analysis for 15 states; additional optional drive test data collection for 200k miles, and optional In-door testing up to a maximum of 200 buildings and optional ad-hoc testing up to a maximum of 104 incidents

3 Option Period 3 1 full year 6/1/23 – 5/31/24 Drive test data collection, post-processing and analysis for 15 states; additional optional drive test data collection for 200k miles, and optional In-door testing up to a maximum of 200 buildings and optional ad-hoc testing up to a maximum of 104 incidents

3 Option Period 4 1 full year 6/1/24 – 5/31/25 Drive test data collection, post-processing and analysis for 15 states; additional optional drive test data collection for 200k miles, and optional In-door testing up to a maximum of 200 buildings and optional ad-hoc testing up to a maximum of 104 incidents

3 Option Period 5 1 full year 6/1/25 – 5/31/26 Drive test data collection, post-processing and analysis for 15 states; additional optional drive test data collection for 200k miles, and optional In-door testing up to a maximum of 200 buildings and optional ad-hoc testing up to a maximum of 104 incidents

For purposes of this DTDP, rural geography will be defined using the Office of Management and Budget (OMB) definition of rural and the U.S. Census Bureau 2010 Census Data. OMB designates counties as Metropolitan, Micropolitan, or Neither.

A Metro area contains a core urban area of 50,000 or more population, and a Micro area contains an urban core of at least 10,000, but less than 50,000, population. All counties that are not part of a Metropolitan Statistical Area (MSA) are considered rural. Micropolitan counties are considered non-Metropolitan or rural along with all counties that are not

2.2. REQUIREMENTS

The FirstNet Authority’s requirements are organized by the following guiding principles. These principles are essential to the overall success of the effort:

• On-Going Breadth and Quality of Data: the ability to understand coverage, performance and the end-to-end user experience is paramount, both in terms of current and future network deployment and technologies. Additionally, the data collection methodology and reporting shall be standards based and/or consistent with telecommunication industry best practices.

• Public Safety Relevance: It is essential that this program is designed to meet the needs of Public Safety from a geographical area and technology perspective. For example, it is important to understand the NPSBN coverage and performance in rural and underserved areas (see footer), which are typically less important for commercial networks.

• Reporting Flexibility: A flexible and intuitive data repository interface is essential for the FirstNet Authority to access post-processed data, and reports.

• Operational Flexibility: Flexibility is important for the success of the program. This includes the ability, for example, to collect more data than originally planned for a given area, collect indoor data, and to include emerging technologies and services.

• Operational Reliability: The implementation of quality assurance processes and best practices for data collection, post-processing and analysis are essential to ensure the integrity and accuracy of the drive test data analysis and reports.

2.2.1. ON-GOING BREADTH AND QUALITY OF DATA

The Contractor’s products and services shall perform the following requirements for Phase 1 (Base Period):

1. The Contractor shall collect, process, and analyze drive test data in the following 15 states:

1 California 6 Kentucky 11 Ohio 2 Connecticut 7 Louisiana 12 Oklahoma 3 Florida 8 Maryland 13 Pennsylvania 4 Georgia 9 New Jersey 14 Texas 5 Illinois 10 New York 15 Virginia

For purposes of this DTDP, rural geography will be defined using the Office of Management and Budget (OMB) definition of rural and the U.S. Census Bureau 2010 Census Data. OMB designates counties as Metropolitan, Micropolitan, or Neither.

A Metro area contains a core urban area of 50,000 or more population, and a Micro area contains an urban core of at least 10,000, but less than 50,000, population. All counties that are not part of a Metropolitan Statistical Area (MSA) are considered rural. Micropolitan counties are considered non-Metropolitan or rural along with all counties that are not

2. The Contractor shall determine and propose the amount of drive test data needed in each of the

15 states to properly characterize the coverage and performance of the NPSBN in each state.

3. At a minimum, the Contractor shall collect 100,000 miles of drive test data which is a sum of all 15 states;

4. The Contractor shall complete the data collection, post-processing, and analysis for all 15 states in the time interval specified for phases in Table 1 (DTDP phases and scope)

5. The Contractor shall provide monthly progress reports indicating states/territories where the data collection, post-processing, and reports are completed including findings and recommendations for changes and updates that may bring clarity to the reporting;

6. The Contractor shall use commercially available handsets and RF scanners for testing;

7. The Contractor and the FirstNet Authority shall review data collection processes;

methodologies and tools to be used. FirstNet will exercise sole discretion on approving changes during performance to: commercial devices to be used for testing, drive test routes, equipment set-up, test procedures, and tools used for data processing and analysis based on input provided from the vendor;

8. The Contractor shall collect, process and analyze data for 4G technologies based on 3GPP standards;

9. The Contractor shall collect, process and analyze layer 1 and layer 3 data, as well as SIB messages, to enable deep-dive analysis;

10. The Contractor shall make recommendations to improve the data gathering methodology and ability to analyze and trend the data

11. At a minimum, the Contractor shall provide the following measurements:

a. Coverage:

i. By operating frequency

ii. Technology Accessed (4G)

iii. Signal Strength (Reference Signal Power (RSRP), Reference Signal Received Quality (RSRQ), Receive Signal Strength Indicator (RSSI))

iv. Signal-to-Noise Ratio (SINR)

v. Depth of Coverage/overlap/number of servers

vi. No Service

vii. Cell Identification (ID)

viii. Public land mobile network ID (PLMNID)

b. Voice:

i. Reliability, Retainability, Accessibility

ii. Voice Quality (MOS score)

iii. Call set-up time

iv. Handoff/Handover performance

c. 4G Data:

i. Uplink and downlink Throughput

ii. Latency

iii. Reliability, Accessibility, Retainability

iv. Bandwidth and frequency utilization

For purposes of this DTDP, rural geography will be defined using the Office of Management and Budget (OMB) definition of rural and the U.S. Census Bureau 2010 Census Data. OMB designates counties as Metropolitan, Micropolitan, or Neither.

A Metro area contains a core urban area of 50,000 or more population, and a Micro area contains an urban core of at least 10,000, but less than 50,000, population. All counties that are not part of a Metropolitan Statistical Area (MSA) are considered rural. Micropolitan counties are considered non-Metropolitan or rural along with all counties that are not

v. Modulation type

vi. Carrier Aggregation (CA) and Multiple Input Multiple Output (MIMO) usage

vii. Handoff/Handover performance

viii. Radio Frequency (RF) and spectrum utilization

d. Video:

i. Reliability

ii. Accessibility

iii. Retainability

iv. Quality video mean opinion score (V-MOS)

e. SMS (Short Message Service):

i. Success rate

ii. time to receive/send

12. The Contractor shall determine required test servers as well as their geographical location to accurately and consistently measure the NPSBN. Test servers shall be deployed as determined by the Contractor. In addition to the contractor test servers, the Contractor may be requested to use AT&T’s test servers in addition to their own capability;

13. The Contractor shall analyze network configurations, and parameters to identify possible gaps in measured data

14. The Contractor shall use industry standards and best practices for data collection, post-processing and analysis

The Contractor’s products and services shall perform the following requirements for Phase 2 (Option Period 1):

1. The Contractor shall collect, process, and analyze drive test data in the following 15 states/territories

1 Alaska 9 Idaho 2 Alabama 10 Indiana 3 Arkansas 11 Kansas 4 Arizona 12 Massachusetts 5 Colorado 13 Maine 6 Delaware 14 Michigan 7 Hawaii 15 Minnesota 8 Iowa

15 states/territories to properly characterize the NPSBN coverage and performance in each state;

3. At a minimum, the Contractor shall collect 100,000 miles of drive test data which is a sum of all 15 states/territories

4. At a minimum, the Contractor shall drive test each of the 15 states/territories once in the time interval specified for phase. An equal number of states/territories shall be driven each quarter or a schedule mutually agreed upon prior to commencement of phase; which plans that the drives will occur throughout the period and not be back end loaded;

5. The Contractor shall complete the data collection, post-processing, and analysis for all states/territories in the time interval specified for phases in Table 1 (DTDP phases and scope)

6. The Contractor shall provide monthly progress reports indicating states/territories where the data collection, post-processing, and reports are completed including findings and recommendations for changes and updates that may bring clarity to the reporting;

7. The Contractor shall use commercially available handsets and RF scanners for testing;

8. The Contractor and the FirstNet Authority shall review collection processes; methodologies and tools to be used prior to the commencement of phase. The FirstNet Authority shall exercise sole discretion on determining changes to: commercial devices to be used for testing, drive test routes, equipment set-up, test procedures, and tools used for data processing and analysis based on input provided from the vendor;

9. The Contractor shall drive test for RF propagation worse-case scenarios, e.g. when foliage is on the trees;

10. The Contractor shall collect, process and analyze data for 4G technologies based on 3GPP standards;

11. The Contractor shall collect, process and analyze layer 1 and layer 3 data, as well as SIB messages, to enable deep-dive analysis;

12. The Contractor shall make recommendations to improve the data gathering methodology and ability to analyze and trend the data collected

13. At a minimum, the Contractor shall provide the following measurements:

a. Coverage:

i. By operating frequency

ii. Technology Accessed (i.e., 4G, Wi-Fi)

iii. Signal Strength (Reference Signal Power (RSRP), Reference Signal Received Quality (RSRQ), Receive Signal Strength Indicator (RSSI))

iv. Signal-to-Noise Ratio (SINR)

v. Depth of Coverage/overlap/number of servers

vi. No Service

vii. Cell Identification (ID)

viii. Public land mobile network ID (PLMNID)

b. Voice:

i. Reliability, Retainability, Accessibility

ii. Voice Quality (MOS score)

ii. Latency

iii. Reliability, Accessibility, Retainability

iv. Bandwidth and frequency utilization

v. Modulation type

vi. Carrier Aggregation (CA) and Multiple Input Multiple Output (MIMO) usage

ii. Accessibility

iii. Retainability

iv. Quality video mean opinion score (V-MOS)

e. SMS (Short Message Service):

14. The Contractor shall determine required test servers as well as their geographical location to by the Contractor. In addition to the contractor test servers, the Contractor may be requested to use AT&T’s test servers in addition to their own capability;

15. The Contractor shall analyze network configurations, and parameters to identify possible gaps in measured data

16. The Contractor shall use industry standards and best practices for data collection, post-

Phase 2 (Option Period 1) Optional Services

1. The Contractor shall collect, process, and analyze drive test data in the following 21

1 Missouri 12 Rhode Island 2 Mississippi 13 South Carolina 3 Montana 14 South Dakota 4 North Carolina 15 Tennessee 5 North Dakota 16 Utah 6 Nebraska 17 Vermont 7 New Hampshire 18 Washington 8 New Mexico 19 Wisconsin 9 Nevada 20 West Virginia 10 Oregon 21 Wyoming 11 Puerto Rico

21 states/territories to properly characterize the NPSBN coverage and performance in each state;

3. At a minimum, the Contractor shall collect 100,000 miles of drive test data which is a sum of all 21 states/territories (See Attachment 5, Price Workbook).

4. At a minimum, the Contractor shall drive test each of the 21 states/territories once in the time interval specified for phase. An equal number of states/territories shall be driven each quarter or a schedule mutually agreed upon prior to commencement of phase; which plans that the drives will occur throughout the period and not be back end loaded;

5. The Contractor shall complete the data collection, post-processing, and analysis for all states/territories in the time interval specified for phases in Table 1 (DTDP phases and scope)

6. The Contractor shall provide monthly progress reports indicating states/territories where the data collection, post-processing, and reports are completed including findings and recommendations for changes and updates that may bring clarity to the reporting;

7. The Contractor shall use commercially available handsets and RF scanners for testing;

8. The Contractor and the FirstNet Authority shall review collection processes; methodologies and tools to be used prior to the commencement of phase. The FirstNet Authority shall exercise sole discretion on determining changes to: commercial devices to be used for testing, drive test routes, equipment set-up, test procedures, and tools used for data processing and analysis based on input provided from the vendor;

9. The Contractor shall drive test for RF propagation worse-case scenarios, e.g. when foliage is on the trees;

10. The Contractor shall collect, process and analyze data for 4G technologies based on 3GPP standards;

11. The Contractor shall collect, process and analyze layer 1 and layer 3 data, as well as SIB messages, to enable deep-dive analysis;

12. The Contractor shall make recommendations to improve the data gathering methodology and ability to analyze and trend the data collected

13. At a minimum, the Contractor shall provide the following measurements:

a. Coverage:

i. By operating frequency

ii. Technology Accessed (i.e., 4G, Wi-Fi)

iii. Signal Strength (Reference Signal Power (RSRP), Reference Signal Received Quality (RSRQ), Receive Signal Strength Indicator (RSSI))

iv. Signal-to-Noise Ratio (SINR)

v. Depth of Coverage/overlap/number of servers

vi. No Service

vii. Cell Identification (ID)

viii. Public land mobile network ID (PLMNID)

b. Voice:

i. Reliability, Retainability, Accessibility

ii. Voice Quality (MOS score)

ii. Latency

iii. Reliability, Accessibility, Retainability

iv. Bandwidth and frequency utilization

v. Modulation type

vi. Carrier Aggregation (CA) and Multiple Input Multiple Output (MIMO) usage

ii. Accessibility

iii. Retainability

iv. Quality video mean opinion score (V-MOS)

e. SMS (Short Message Service):

by the Contractor. In addition to the contractor test servers, the Contractor may be requested to use AT&T’s test servers in addition to their own capability;

15. The Contractor shall analyze network configurations, and parameters to identify possible gaps in measured data

16. The Contractor shall use industry standards and best practices for data collection, post-processing and analysis

17. The Contractor shall perform in-building testing in each of the 21 state/territory. The number of buildings to be tested will be determined by the FirstNet Authority after input from the vendor and prior to the commencement of the Phase drive testing campaign. No more than 100 venues shall be tested in Option Period 1;

18. The Contractor shall secure access to the buildings to be tested in each state/territory;

19. The Contractor shall verify the performance of the NPSBN Z-axis solution whenever building access allows it;

The Contractor’s products and services shall perform the following requirements annually for Phase 3 (Option Periods 2-5):

1. The Contractor shall collect, process, and analyze drive test data in the following 15 states:

1 California 6 Kentucky 11 Ohio 2 Connecticut 7 Louisiana 12 Oklahoma 3 Florida 8 Maryland 13 Pennsylvania 4 Georgia 9 New Jersey 14 Texas 5 Illinois 10 New York 15 Virginia

15 states to properly characterize the coverage and performance of the NPSBN in each state.

3. At a minimum, the Contractor shall collect 100,000 miles of drive test data which is a sum of all

15 states;

4. The Contractor shall complete the data collection, post-processing, and analysis for all 15 states in the time interval specified for phases in Table 1 (DTDP phases and scope)

5. The Contractor shall provide monthly progress reports indicating states/territories where the data collection, post-processing, and reports are completed including findings and recommendations for changes and updates that may bring clarity to the reporting;

6. The Contractor shall use commercially available handsets and RF scanners for testing;

7. The Contractor and the FirstNet Authority shall review data collection processes; methodologies and tools to be used. FirstNet will exercise sole discretion on determining changes to:

commercial devices to be used for testing, drive test routes, equipment set-up, test procedures, and tools used for data processing and analysis based on input provided from the vendor;

8. The Contractor shall collect, process and analyze data for 4G technologies based on 3GPP standards;

9. The Contractor shall collect, process and analyze layer 1 and layer 3 data, as well as SIB messages, to enable deep-dive analysis;

10. The Contractor shall make recommendations to improve the data gathering methodology and ability to analyze and trend the data

11. At a minimum, the Contractor shall provide the following measurements:

a. Coverage:

i. By operating frequency

ii. Technology Accessed (4G)

iii. Signal Strength (Reference Signal Power (RSRP), Reference Signal Received Quality (RSRQ), Receive Signal Strength Indicator (RSSI))

iv. Signal-to-Noise Ratio (SINR)

v. Depth of Coverage/overlap/number of servers

vi. No Service

vii. Cell Identification (ID)

viii. Public land mobile network ID (PLMNID)

b. Voice:

i. Reliability, Retainability, Accessibility

ii. Voice Quality (MOS score)

ii. Latency

iii. Reliability, Accessibility, Retainability

iv. Bandwidth and frequency utilization

v. Modulation type

vi. Carrier Aggregation (CA) and Multiple Input Multiple Output (MIMO) usage

ii. Accessibility

iii. Retainability

iv. Quality video mean opinion score (V-MOS)

e. SMS (Short Message Service):

12. The Contractor shall determine required test servers as well as their geographical location to accurately and consistently measure the NPSBN. Test servers shall be deployed as determined by the Contractor. In addition to the contractor test servers, the Contractor may be requested to use AT&T’s test servers in addition to their own capability;

13. The Contractor shall analyze network configurations, and parameters to identify possible gaps in measured data

14. The Contractor shall use industry standards and best practices for data collection, post-

Phase 3 (Option Periods 2-5) Optional Services

1. The Contractor shall collect, process, and analyze drive test data in the following 36

1 Alaska 10 Indiana 19 North Carolina 28 South Carolina 2 Alabama 11 Kansas 20 North Dakota 29 South Dakota 3 Arkansas 12 Massachusetts 21 Nebraska 30 Tennessee 4 Arizona 13 Maine 22 New Hampshire 31 Utah 5 Colorado 14 Michigan 23 New Mexico 32 Vermont 6 Delaware 15 Minnesota 24 Nevada 33 Washington 7 Hawaii 16 Missouri 25 Oregon 34 Wisconsin 8 Iowa 17 Mississippi 26 Puerto Rico 35 West Virginia 9 Idaho 18 Montana 27 Rhode Island 36 Wyoming

36 states/territories to properly characterize the NPSBN coverage and performance in each state;

3. At a minimum, the Contractor shall collect 200,000 miles of drive test data which is a sum of all 36 states/territories (See Attachment 5, Price Workbook).

4. At a minimum, the Contractor shall drive test each of the 36 states/territories once in the time interval specified for phase. An equal number of states/territories shall be driven each quarter or a schedule mutually agreed upon prior to commencement of phase; which plans that the drives will occur throughout the period and not be back end loaded;

5. The Contractor shall complete the data collection, post-processing, and analysis for all states/territories in the time interval specified for phases in Table 1 (DTDP phases and scope)

6. The Contractor shall provide monthly progress reports indicating states/territories where the data collection, post-processing, and reports are completed including findings and recommendations for changes and updates that may bring clarity to the reporting;

7. The Contractor shall use commercially available handsets and RF scanners for testing;

8. The Contractor and the FirstNet Authority shall review collection processes; methodologies and tools to be used prior to the commencement of phase. The FirstNet Authority shall exercise sole discretion on determining changes to: commercial devices to be used for testing, drive test routes, equipment set-up, test procedures, and tools used for data processing and analysis based on input provided from the vendor;

9. The Contractor shall drive test for RF propagation worse-case scenarios, e.g. when foliage is on the trees;

10. The Contractor shall collect, process and analyze data for 4G technologies based on 3GPP standards;

11. The Contractor shall collect, process and analyze layer 1 and layer 3 data, as well as SIB messages, to enable deep-dive analysis;

12. The Contractor shall make recommendations to improve the data gathering methodology and ability to analyze and trend the data collected

13. At a minimum, the Contractor shall provide the following measurements:

a. Coverage:

i. By operating frequency

ii. Technology Accessed (i.e., 4G, Wi-Fi)

iii. Signal Strength (Reference Signal Power (RSRP), Reference Signal Received Quality (RSRQ), Receive Signal Strength Indicator (RSSI))

iv. Signal-to-Noise Ratio (SINR)

v. Depth of Coverage/overlap/number of servers

vi. No Service

vii. Cell Identification (ID)

viii. Public land mobile network ID (PLMNID)

b. Voice:

i. Reliability, Retainability, Accessibility

ii. Voice Quality (MOS score)

ii. Latency

iii. Reliability, Accessibility, Retainability

iv. Bandwidth and frequency utilization

v. Modulation type

vi. Carrier Aggregation (CA) and Multiple Input Multiple Output (MIMO) usage

ii. Accessibility

iii. Retainability

iv. Quality video mean opinion score (V-MOS)

e. SMS (Short Message Service):

by the Contractor. In addition to the contractor test servers, the Contractor may be requested to use AT&T’s test servers in addition to their own capability;

15. The Contractor shall analyze network configurations, and parameters to identify possible gaps in measured data

16. The Contractor shall use industry standards and best practices for data collection, post-

17. The Contractor shall perform in-building testing in each of the 36 state/territory. The number of buildings to be tested will be determined by the FirstNet Authority after input from the vendor and prior to the commencement of the Phase drive testing campaign. No more than 200 venues shall be tested in each option period;

18. The Contractor shall secure access to the buildings to be tested in each state/territory;

19. The Contractor shall verify the performance of the NPSBN Z-axis solution whenever building access allows it;

20. The Contractor shall perform ad-hoc testing to investigate specific and localized public safety concerns related to network coverage, performance, or specific service or application during normal operation and public safety emergencies. The FirstNet Authority expects to test no more than 104 investigations nationally per year.

2.2.2. PUBLIC SAFETY RELEVANCE

The Contractor’s products and services shall fulfill the following requirements for Phase 1:

1. The Contractor shall collect, process and analyze sufficient drive test data in not only urban areas, but also rural and underserved areas which are typically not the focus of wireless service providers;

2. At a minimum, 75% of the drive test data shall be collected in rural areas per state;

3. The contractor shall collect, process and analyze drive test data for the NPBSN which includes but is not limited to Band 14, AT&T commercial bands and roaming partners that are part of the NPSBN solution offered by AT&T.

4. The Contractor shall analyze and contrast the experience of Public Safety Users when in Band 14 and when on other commercial network frequency bands mentioned in requirement 3 above;

5. The Contractor shall analyze and contrast network performance and user experience of Public Safety and Commercial users on AT&T’s network;

6. The Contractor shall perform in-depth analysis of the drive test data including correlation of differing data sets collected in Section 2 Phase 1 bullet 11 that will be used to identify causes of coverage and performance issues, as well as inform future network investments.

The Contractor’s products and services shall fulfill the following requirements for Phase 2 and Phase 3 unless specifically identified:

1. The Contractor shall collect, process and analyze sufficient drive test data in not only urban areas, but also rural and underserved areas which are typically not the focus of wireless service providers;

2. At a minimum, 75% of the drive test data shall be collected in rural areas per state;

3. The Contractor shall collect, process and analyze drive test data for the NPBSN which includes both Band 14 and AT&T commercial bands.

4. Reserved

5. The Contractor shall analyze and contrast the experience of Public Safety Users when in Band

14 and when on other commercial network frequency bands mentioned in requirement 3 above;

6. The Contractor shall analyze and contrast network performance and user experience of Public

Safety and Commercial users on AT&T’s network;

7. The Contractor shall perform in-depth analysis of the drive test data including correlation of differing data sets collected in Section 2 Phase 2 bullet 13 and Section 2 Phase 3 bullet 14 respectively that will be used to identify causes of coverage and performance issues, as well as inform future network investments.

2.2.3. REPORTING FLEXIBILITY

The Contractor’s products and services shall fulfill the following requirements for Phase 1, Phase 2 and Phase 3 unless specified:

1. The Contractor shall provide a secured data repository that houses the post-processed drive test data, analysis, and reports;

2. The Contractor shall provide the post-processed data in (comma-separated values) format (or another mutually agreed upon format);

3. The Contractor shall provide flexible reporting that enables analysis and reporting at varying geographical levels such as county, state, nationwide, custom polygons and morphologies;

4. The Contractor shall provide summary and engineering level reports for each state/territory that provide in-depth understanding of coverage and performance;

5. The Contractor shall provide trending reports to facilitate each drive as it is trended year over year and state to state analysis and comparisons;

6. The Contractor shall provide key personnel in back office support to proactively and reactively perform in-depth analysis of the drive test data and interpret results as identified in Section 2.2.2 Public Safety Relevance bullet 7. This support should ensure maximum value is obtained from the FirstNet Authority’s drive test program;

7. The Contractor shall conduct monthly meetings to review state/territory and nationwide drive test results, and insights;

8. The Contractor shall conduct quarterly business review meetings to review program deliverables, program status, challenges, and testing upgrades, and to discuss process improvement opportunities.

2.2.4. OPERATIONAL FLEXIBILITY

The Contractor’s products and services shall fulfill the following requirements in Phase 1, Phase 2 and

1. The Contractor’s proposed solution shall be operationally flexible so that the Drive Test Data Program evolves and remains valuable as technology and needs of Public Safety change;

2. The Contractor shall describe how the Drive Test Data Program would evolve to incorporate measurements of mission critical services including push-to-talk and data;

3. The Contractor shall be able to utilize different drive test routes year-over-year to expand the overall test footprint over time;

4. The Contractor is required to submit drive test routes to the FirstNet Authority annually for review and approval by the FirstNet Authority;

5. The Contractor agrees that the FirstNet Authority owns all deliverables and data, including but not limited to the NPSBN data, and will share the reports with AT&T, Public Safety agencies and stakeholders as determined by the FirstNet Authority.

2.2.5. OPERATIONAL RELIABILITY

The Contractor’s products and services shall fulfill the following requirements in Phase 1, Phase 2 and

• The Contractor shall implement technology upgrades as deemed necessary by the FirstNet

Authority

• The Contractor shall implement installation, maintenance, and calibration policies and procedures needed to ensure the integrity of the data;

• The Contractor shall implement day-to-day best practices to ensure the integrity of the data collection and analysis process.

2.3. PROJECT MANAGEMENT

The Contractor shall provide the following reporting designated in the Deliverables table, located in Section 2.4, which include:

2.3.1 Data Repository, which shall include, but not be limited to, an interface that the FirstNet

Authority can use to securely view and download post-processed data, as well as view and download coverage, performance and user experience reports.

2.3.2 Drive Test Campaign Plan (Solution Description), which shall include, but not be limited to, a detailed description of the solution for the drive test campaign including schedules, hardware and software set-up, metrics to be measured, drive test routes for each state/territory, test procedures, and any feature customized for the FirstNet Authority’s drive test program.

2.3.3 Summary Reports & Maps, which shall include summary reports and maps for each state/territory or building in which drive testing and/or in-building testing and ad-hoc testing campaign is performed. The reports shall include, but not be limited to, coverage, performance, voice, data, video, and SMS metrics as identified in Section 2.2 Requirements. Reports shall include a CSV (comma-separated values) file format (or alternate mutually agreed format) containing each of the metrics measured and their associated values for each state/territory.

2.3.4 Deep Dive Engineering Analysis Reports, which shall include in-depth engineering analysis reports and maps for each state/territory. The reports shall include, but not be limited to, coverage and performance analysis, trends, and network configurations, as well as recommendations to improve coverage and performance as identified in Section 2.2 Requirements.

2.3.5 Ad-hoc and Coverage Discrepancy Reports and Maps, which shall include, but not be limited to, either an in-depth analysis report and maps of the area or issue being investigated, as well as possible solutions and improvements (ad-hoc report and map); or, coverage discrepancy statistics and maps showing areas where the measurements differ from the coverage map (Coverage Discrepancy report and maps). Ad-hoc and Coverage discrepancy reports and maps shall be provided when the FirstNet Authority engages the Contractor for either task.

2.3.6 Monthly Review Meetings. Meetings shall be held at FirstNet Authority facilities or remotely as determined by the COR and shall include mutually agreed agenda items including, but not limited to, review of state/territory and nationwide drive test results, network data insights, project status, progress for period, and planned activities.

2.3.7 Quarterly Review Meetings. Meetings shall be held at FirstNet Authority facilities or remotely as determined by the COR and shall include mutually agreed agenda items including, but not limited to, trending of data month over month, and overall program status.

2.3.8 Monthly Status Report of Contract Spend and Upcoming Forecast, status of funds will be provided in agreed upon format to the CORs at least once per 30 business day period. The spreadsheet shall list each charge per Time-and-Materials CLIN and upcoming forecast for each month.

2.3.9 Additional/Future deliverables, Contractor shall add other specific deliverables when needed to enhance existing reports and/or capture new measurements of existing or new network features, services or technology per mutual agreement between parties.

2.4 DELIVERABLES

The Contractor shall provide the deliverables as identified herein and specified in the table below, throughout the life of this contract. All deliverables shall be submitted to both the Contracting Officer (CO) and Contracting Officer’s Representative (COR) or posted in a format specified by the Contractor for online access by the COR or the Contracting Officer (CO) and/or other Government personnel designated by the CO. The CO will provide the Contractor with the names and addresses of any additional distribution and/or online access for these deliverables.

If additional deliverables are determined necessary post-award, they will be mutually agreed upon prior to incorporation into the contract via a formal modification. As stated herein, all deliverables shall be submitted to both the CO and the COR for inspection, review, and acceptance. Formal acceptance occurs upon the CO written acceptance notification. The deliverables become the property of the FirstNet Authority.

Unless otherwise specified, and/or agreed upon, the FirstNet Authority will have fifteen (15) workdays to review the deliverable and provide comments back to the Contractor. The contractor will have five (5) workdays from receipt of comments to incorporate changes and submit the final deliverable to the FirstNet Authority. In the event of rejection of any deliverable, the contractor shall be notified in writing by the FirstNet Authority of the specific reasons why deliverable is unsatisfactory. The contractor shall return a revised document within five (5) workdays to the CO and COR. Deliverables are not considered accepted by the FirstNet Authority until the CO provides specific notification to the Contractor.

For the deliverables with a defined due date to occur on a rolling basis the FirstNet Authority will accept the deliverables that the Contractor will submit upon receipt, and will seek clarification of details and information as needed during the quarterly review meetings.

General Acceptance Criteria - General quality measures, as set forth below, will be applied to each work product received from the contractor under this contract. Absence of any comments by the FirstNet Authority will not relieve the Contractor of the responsibility for complying with the requirements of this contract. The Contractor shall not construe any letter of acknowledgment of receipt material as a waiver of review, or as an acknowledgment that the material is in conformance with this contract. Any approval given during preparation of the documentation, or approval for shipment, shall not guarantee the final acceptance of the completed documentation.

Accuracy - Deliverable shall be accurate in presentation, technical content, and adherence to accepted elements of style.

Clarity - Deliverable shall be clear and concise. Any/all diagrams shall be easy to understand and be relevant to the supporting narrative.

Consistency to Requirements - All Deliverable shall satisfy the requirements of this contract.

File Editing - All text and diagrammatic files shall be editable by the FirstNet Authority.

Format - Deliverable shall be submitted electronically to the CO and COR in a format mutually agreed upon prior to submission, unless otherwise specified herein. For computer software, the Contractor grants to the Government, and others acting on its behalf, a paid-up, nonexclusive, irrevocable, worldwide license in such copyrighted computer software to reproduce, prepare derivative works, and perform publicly and display publicly (but not to distribute copies to the public) by or on behalf of the Government.

Timeliness – Deliverable shall be submitted on or before the due date specified herein or submitted in accordance with a later scheduled date determined by the FirstNet Authority.

Quality Assurance - The COR or designated inspector may review, for completeness, preliminary or draft documentation that the Contractor submits, and may return it to the Contractor for correction.

The Contractor shall prepare and submit a monthly status report and shall include, at a minimum, the following:

Progress for the period: Detailed progress report of findings, key relevant activities and accomplishments during the reporting period

Activities planned, to include any partner activities, for the next reporting period: Updated information on planned activities, as well as the status of any and all deliverables, including planned delivery date(s) and actual and/or anticipated delivery date(s);

Problems encountered: Identification of any problems, issues or delays and recommendations as to their resolution, and any corrective action that was taken to correct identified problems.

Risk Mitigation: Identification of any risks and proactive mitigation actions undertaken by the contractor.

Cost expenditures: Depict the planned T&M versus actual T&M expenditures for the services portion of this contract (labor and all other expenditures to include direct, indirect, and subcontractors). Show data from the previous month, current reporting period, planned for the next reporting period, as well as cumulative contract report to include current contract staffing levels.

Deliverables Table for Phase 1:

Number Reference Deliverable Title Deliverable Description Due Date

2.3.1

Data Repository

An interface portal that the FirstNet Authority can use to securely view and download raw and post-processed data as well as view and download reports.

Within 60 days of award

2.3.2

Drive Test Campaign Plan

A detailed description of the solution for the drive test campaign including schedules, hardware and software set-up, metrics to be measured, drive test routes for each state/territory, test procedures, and any feature customized for the FirstNet’s Authority drive test program

Within 30 days of award

2.3.3

Summary Reports and Maps

Summary reports and maps for each state/territory. The reports shall include, but not be limited to, coverage, performance, voice, data, video and SMS metrics.

Reports shall include a CSV file format (or alternate mutually agreed format) containing each of the metrics measured and their associated values for each state/territory

Rolling two weeks after completion of each data collection per drive and within the timeframe specified for phase

2.3.4

Deep Dive Engineering Analysis Reports

In-depth Engineering analysis reports and maps for each state/territory. The reports shall include, but not be limited to, coverage and performance analysis, trends, network configurations as well as recommendations to the NPSBN’s network and performance

Within 30 calendar days of conclusion of

2.3.6

Monthly Review Meetings

Meetings shall be held at FirstNet Authority facilities or remotely as determined by the COR and shall include mutually agreed agenda items including, but not limited to, review of state/nationwide drive test results, network data insights, project status, progress for period, and planned activities.

Monthly, by the 15th of each month, and for the duration of

2.3.7

Phase 1 Conclusion Review Meeting

Meetings shall be held at FirstNet Authority facilities or remotely as determined by the COR and shall include mutually agreed agenda items including, but not limited to, trending of data month over month, and overall program status.

Within 30 days of the conclusion of phase

2.3.8

Status Report of Contract Spend and upcoming forecast

A status of funds shall be provided in agreed upon format to the CO and COR(s) at least once per 30 business day period.

The spreadsheet shall list each charge per CLIN and upcoming forecast

Monthly, by the 15th of the month

Deliverables Table for Phase 2

Plan Solution Description

A detailed description of the solution for the drive test program including schedules, hardware and software set-up, metrics to be measured, drive test routes for each state/territory, test procedures, and any feature customized for the FirstNet’s Authority drive test program

Within 30 days before the start of the drive testing campaign

Maps

Summary reports and maps for each state/territory or building in which drive testing or in-building testing is performed. The reports shall include, but not be limited to, coverage, performance, voice, data, video and SMS metrics.

Reports shall include a CSV file format (or alternate mutually completion of each data collection per drive and within the time frame agreed format) containing each of the metrics measured and their associated values for each state/territory specified for

Analysis Reports

In-depth Engineering analysis reports and maps for each state/territory. The reports shall include, but not be limited to, coverage and performance analysis, trends, network configurations as well as

Quarterly by the month and at the conclusion of phase

Review Meetings

Meetings shall be held at FirstNet Authority facilities or remotely as determined by the COR and shall include mutually agreed agenda items including, but not limited to, review of state/nationwide drive test results, network data insights, project status, progress for period, and planned activities.

Monthly, by the 15th of each month and for the duration of

Quarterly Review Meetings

Meetings shall be held at FirstNet Authority facilities or remotely as determined by the COR and shall include mutually agreed agenda items including, but not limited to, trending of data month over month, and overall program status.

Quarterly, schedule to be mutually agreed upon and at the conclusion of

Status Report of Contract Spend and upcoming forecast

A status of funds shall be provided in agreed upon format to the CO and CORs at least once per 30 business day period.

The spreadsheet shall list each charge per CLIN and upcoming forecast

Monthly, by the

Deliverables Table for Phase 3

Plan

A detailed description of the solution for the drive test program including schedules, hardware and software set-up, metrics to be measured, drive test routes for each state/territory, test procedures, and any feature customized for the FirstNet’s Authority drive test program

Within 60 days before the start of the drive testing campaign

Maps

Summary reports and maps for each state/territory or building in which drive testing or in-building testing is performed. The reports shall include, but not be limited to, coverage, performance, voice, data, video and SMS metrics.

Reports shall include a CSV file format (or alternate mutually agreed format) containing each of the metrics measured and their associated values for each state/territory completion of each data collection per drive

Analysis Reports

In-depth Engineering analysis reports and maps for each state/territory. The reports shall include, but not be limited to, coverage and performance analysis, trends, network configurations as well as

Quarterly by the

2.3.5

Ad-hoc and Coverage Discrepanc y Reports and Maps

In-depth analysis report and maps of the area or issue being investigated as well as possible solutions and improvements (ad-hoc report and map); or, coverage discrepancy statistics and maps showing areas where the measurements differ from the coverage map (Coverage Discrepancy report and maps). Ad-hoc and Coverage Discrepancy reports and maps shall be provided when the FirstNet Authority

Ten working days after completion of data collection engages the Contractor for either task

Review Meetings

Meetings shall be held at FirstNet Authority facilities or remotely as determined by the COR and shall include mutually agreed agenda items including, but not limited to, review of state/nationwide drive test results, network data insights, project status, progress for period, and planned activities.

Monthly, by the 15th of each month

Quarterly Review Meetings

Meetings shall be held at FirstNet Authority facilities or remotely as determined by the COR and shall include mutually agreed agenda items including, but not limited to, trending of data month over month, and overall program status.

Quarterly, schedule to be mutually agreed upon

Status Report of Contract Spend and upcoming forecast

A status of funds shall be provided in agreed upon format to the CO and COR(s) at least once per 30 business day period.

The spreadsheet shall list each charge per CLIN and upcoming forecast

Monthly, by the

2.3.11

Additional/ Future Deliverables

Contractor to add other specific deliverables when needed to enhance existing reports and/or capture new measurements of existing or new network features, services or technology per mutual agreement between parties.

When requested by the FirstNet Authority or when a new network feature, service or technology is deployed per mutual agreement

2.5 Quality Assurance Surveillance Plan (QASP) Metrics, Performance Standards, and

Acceptable Quality Levels

The table below provides the QASP metrics, which the FirstNet Authority will utilize in order to ensure that the contractor has performed in accordance with the performance metrics/standards. All work under this contract shall be monitored by the COR, who will act as the primary technical representative of the FirstNet Authority Contracting Officer. The COR shall review, for completeness, documentation that the Contractor submits, and may return it to the Contractor for correction.

2.5.1 QASP Monitoring and Reporting

The FirstNet Authority will utilize the following primary methods to assess the contractor’s performance in…