NPSBN_Section_M_Evaluation_Factors_for_Award_AMEND_009.pdf

Solicitation No. D15PS00295 – Section M Evaluation Factors for Award

Page i

Table of Contents

M Evaluation Factors for Award ...................................................................................................... M-1

M.1 General Information ................................................................................................................... M-1

M.2 Evaluation Process ...................................................................................................................... M-1

M.2.1 Phase I – Capability Statements ......................................................................................M-1 M.2.2 Phase II – Solicitation Conformance ...............................................................................M-2 M.2.3 Phase III – Pass/Fail .........................................................................................................M-2 M.2.4 Phase IV – Detailed Evaluation .......................................................................................M-3

M.3 Basis for Award ........................................................................................................................... M-3

M.4 Evaluation Factors ....................................................................................................................... M-4

M.4.1 Volume I – Business Management Factor ......................................................................M-4 M.4.2 Volume II – Coverage and Capacity Factor .....................................................................M-7 M.4.3 Volume II – Products and Architecture Factor ................................................................M-8 M.4.4 Past Performance Factor ............................................................................................. M-20 M.4.5 Volume III – Offeror’s Value Proposition Assessment ................................................. M-21 M.4.6 Risk ............................................................................................................................... M-22

M.5 Competitive Range .................................................................................................................... M-23

M.6 Evaluation Support.................................................................................................................... M-23

Page M-1

M Evaluation Factors for Award

M.1 General Information

Proposals shall be prepared in accordance with and comply with the requirements and instructions contained in this Request for Proposal (RFP). Each proposal will be evaluated against the evaluation factors identified herein. The details on the complete evaluation process are outlined in M.2, Evaluation Process.

M.2 Evaluation Process

In accordance with Federal Acquisition Regulation (FAR) 15.202 and as described herein, a multi-phased approach will be used to determine the overall best value to the Government for this acquisition. Each proposal will be reviewed and evaluated in accordance with its contents; the Government will make no assumptions related to the Offeror’s performance that the Offeror does not specify in its proposal.

In light of the First Responder Network Authority’s (FirstNet) objective-based acquisition approach and the unique nature of the FirstNet program and the Middle Class Tax Relief and Job Creation Act of 2012 (the Act), the Government will consider unique, innovative approaches to achieving an overall best value solution consistent with the objectives as set forth in Section C, Statement of Objectives (SOO) and the requirements and recommendations specified in Section J, Attachment J-3, FCC TAB RMTR. The Government will consider any and all proposed solutions utilizing emerging technologies and/or non-traditional practices or solutions with regard to the overall Nationwide Public Safety Broadband Network (NPSBN) in accordance with the terms and conditions, instructions, and evaluation criteria as stated herein.

Any exceptions or deviations by the Offeror to the terms and conditions stated in the Offeror’s proposal for inclusion in the resulting contract may make the offer unacceptable for award without discussions. If an Offeror proposes exceptions to the terms and conditions of this RFP, the Government may make an award, without discussions, to another Offeror that did not take exception to the terms and conditions, if such Offeror is determined to be the best overall value for this effort.

As part of the multi-phased approach, the Government reserves the right to request Offerors to conduct oral presentations and/or technical demonstrations as a result of this RFP. Those Offerors will be notified and provided any additional information and instructions, as necessary, regarding oral presentations and/or technical demonstrations.

The Day 1 task order evaluations will not be conducted separately from the evaluation of the overall NPSBN proposed solution as they make up specific portions of the NPSBN solution, based on the SOO (Section C) and associated attachments in Section J.

M.2.1 Phase I – Capability Statements

For Phase I of the multi-phased approach, interested parties should demonstrate they are capable of performing the work by providing a capability statement (see FAR Part 15.202 and Section L, Instructions, Conditions, and Notices to Offerors or Respondents, Section L.2.4, Submission of Capability Statements, for instructions). Following review and evaluation of all capability statements received as a result of this phase, those deemed best qualified based on the evaluation criteria stated herein will be

Page M-2 invited to submit a proposal in accordance with the instructions contained in Section L, Instructions, Conditions, and Notices to Offerors or Respondents.

Notifications will be issued to all Offerors that submit a capability statement as a result of this RFP. Each notification will include feedback regarding evaluation of the capability statement that identifies strengths and/or weaknesses that shows whether the company is or is not, based on the capability statement review and evaluations, considered a viable competitor.

Capability statements will be evaluated based on the following criteria, which are of equal importance:

Public safety use and adoption of the NPSBN – Offerors will be evaluated based on their demonstration of their ability to successfully drive adoption and use of the NPSBN by public safety users.

Nationwide coverage and capacity – Offerors will be evaluated based on their demonstration of their ability to provide Band 14 and non-Band 14 coverage and capacity in each of the 56 states and territories, including rural and non-rural areas.

Rural partnerships – Offerors will be evaluated based on their demonstration of their existing and planned partnerships with rural telecommunications providers, including commercial mobile providers, utilizing existing infrastructure to the maximum extent economically desirable to speed deployment in rural areas.

Ability to monetize network capacity – Offerors will be evaluated based on their strategy and demonstration of their ability to monetize network capacity, which may include a secondary user customer base and sales/distribution channels to reach primary and secondary users.

Financial sustainability – Offerors will be evaluated based on their demonstrated approach and financial sustainability. Additionally, Offerors’ financial stability will be evaluated in regard to their ability to develop, implement, sustain, and enhance the NPSBN based on the Initial Operational Capability (IOC)/Final Operational Capability (FOC) milestones set out in Section J, Attachment J-8, IOC/FOC Target Timeline.

M.2.2 Phase II – Solicitation Conformance

During this phase, the Government will conduct an initial review of the proposals received in order to verify conformance and completeness with the RFP instructions, including any/all attachments and exhibits, prior to commencement of evaluations as stated herein in Section M.2.3, Phase III – Pass/Fail, and Section M.2.4, Phase IV – Detailed Evaluation.

This conformity review will consist of verification that all documentation has been provided in accordance with Section L, Instructions, Conditions, and Notices to Offerors or Respondents, Section L.3, Proposal Format and Submission Instructions. The Government will also utilize the Section J, Attachment J-22, Solicitation Conformance Traceability Matrix (SCTM), in order to make a determination regarding completeness.

Failure to submit any information and/or documentation as stated in this RFP and as listed on the SCTM may result in the proposal submission being removed from any further consideration.

M.2.3 Phase III – Pass/Fail

During this phase, the Government will review and evaluate each proposal to ensure it meets the pass/fail factors identified herein. The following pass/fail factors will be evaluated based on the criteria

Page M-3 as stated herein. Failure to pass any of these factors may result in the proposal submission being removed from any further consideration.

M.2.3.1 FirstNet Minimum Payment Thresholds

Offerors must propose payments to FirstNet to be disbursed on an annual basis. The Offeror’s proposed solution must demonstrate its ability to sustain the annual payments to FirstNet for the life of the contract, no less than the minimum payment thresholds described in Section B, Supplies or Services and Prices/Costs, Section B.4.4, FirstNet Operational Sustainability. Payments to FirstNet must be submitted for each of the 56 states and territories. For this phase, the Offeror must demonstrate its ability to meet the minimum payment thresholds. The minimum payment is evaluated based on the sum of the payments proposed for all 56 states and territories for each contract year without regard to the Net Present Value (NPV) calculation (see Section M.4.5.1, Net Present Value of Payments to FirstNet).

M.2.3.2 Rural Partners and Subcontractors

Offerors shall complete Section J, Attachment J-2, Nationwide and Rural Coverage Compliance Checklist, to demonstrate their ability to meet the objective to provide coverage in each of the 56 states and territories and to ensure that rural coverage includes partnerships with rural telecommunications providers. The Offeror’s solution must demonstrate commitment to exercise rural telecommunications provider partnerships for at least 15 percent of the total rural coverage area nationwide at FOC. While Attachment J-2 requests these data by states, the 15 percent coverage factor will be evaluated on a nationwide basis only for this phase.

M.2.4 Phase IV – Detailed Evaluation

Those Offerors whose proposed solutions have been determined to conform to the RFP in Phase II and successfully pass Phase III will move into Phase IV. During this phase, the Government will commence the detailed evaluation of all information and documentation received from the Offerors based on the evaluation factors as stated herein.

The Government may consider, as part of its evaluation, any oral presentations and/or technical demonstrations or other discussions and publicly available materials gathered as part of the Government’s evaluation. All of these materials may be used as part of the evaluation of the Offeror’s ability to meet the objectives stated in Section C, SOO.

M.3 Basis for Award

Contract award shall be made to the responsible Offeror whose offer, in conforming to this RFP, provides the overall best value to the Government, when all evaluation factors are considered. All evaluation factors, when combined, are significantly more important than the value proposition. The Government may conduct a trade-off analysis and make an award to other than the highest technically rated Offeror or other than the Offeror presenting the most favorable value proposition. Due to the unique nature of FirstNet and the NPSBN, rather than conduct a traditional trade-off analysis where the Government typically considers price and non-price factors, the trade-off analysis for this acquisition will utilize the results of a value proposition assessment (see Section M.4.5) and non-price factors to determine the overall best value solution.

The Government reserves the right to not make an award as a result of this competition if, in the opinion of the Government, none of the submissions would provide satisfactory performance that is

Page M-4 considered fair and reasonable and/or economically feasible in the Government’s sole determination.

Additionally, the Government reserves the right to remove an Offeror’s proposed solution from further consideration if it is determined to be unacceptable in any of the evaluation factors and/or sub-factors.

M.4 Evaluation Factors

The evaluation will consist of a determination and analysis of strengths, weaknesses, and risks of each proposed solution. Risk will be included in the evaluation of each factor (and/or sub-factors) and will not be evaluated as a separate factor. The Government may consider all proposal information submitted when assessing risk. The Offeror will be evaluated based on its demonstration and understanding of the objectives, including demonstrated creativity and thoroughness in its proposed solution. The Government may consider all proposal information submitted and presented pertaining to the Offeror’s proposed solution when conducting the evaluation and determining overall best value.

The following evaluation factors will be utilized in order to determine which Offeror provides the best overall value. Evaluation factors are comprised of Business Management (Section M.4.1), Coverage and Capacity (Section M.4.2), Products and Architecture (Section M.4.3), the Offeror’s Past Performance (Section M.4.4), and the Offeror’s Value Proposition Assessment (Section M.4.5).

Evaluation factors are listed in descending order of importance:

Business Management is more important than Coverage and Capacity.

Coverage and Capacity and Products and Architecture are of equal importance.

Business Management, Coverage and Capacity, and Products and Architecture combined are more important than the Volume III – Offeror’s Value Proposition Assessment.

The Volume III – Offeror’s Value Proposition Assessment is more important than Past Performance.

Sub-factors for any of the evaluation factors are of equal importance unless otherwise stated.

M.4.1 Volume I – Business Management Factor

The Offeror will be evaluated based on its understanding of the effort, including innovation, creativity, and thoroughness shown in addressing the objectives (Section C, SOO) and applicable Section J attachments. The Government will evaluate the Offeror’s business management approach to providing effective management of its delivery, operation, and maintenance of the NPSBN. The proposed approach will be evaluated to determine the extent to which it demonstrates a comprehensive, sound, efficient, and realistic approach to managing and ensuring successful contract performance within time and budget constraints.

The Business Management Factor includes evaluation in the following sub-factors:

General

Leadership and program management

Public safety customer acquisition

Customer care and life-cycle sustainment

Offeror financial sustainability

Delivery mechanism for state plans

Quality Assurance Surveillance Plan

Deliverables table

Page M-5

M.4.1.1 Section One – General

The Offeror’s proposed solution will be evaluated based on the following elements:

Small business subcontracting plan – This shall be evaluated pursuant to FAR Part 19.705.

Contractor responsibility information – This shall be evaluated pursuant to FAR 9.104 in order to determine a prospective Contractor’s determination of responsibility.

Past performance – This shall be evaluated in accordance with Section M.4.4, Past Performance Factor.

Offeror’s experience – This shall be evaluated based on any proposed experience with regard to the Offeror’s solution for the NPSBN to include the Offeror’s proposed structure and experience of the subcontractors/teaming partners, the relationship between the Offeror and these organizations, and their combined ability to support the proposed solution with innovative approaches.

M.4.1.2 Section Two – Leadership and Program Management

The Offeror’s solution must demonstrate its ability and proposed approach regarding leadership and program management. This will be evaluated based on the Offeror's proposed management plan and approach to achieving its stated solution, which demonstrates the following elements:

Solution to ensure the services meet objectives in Section C, SOO

Overall staffing plan for the NPSBN, including any proposed teaming arrangements and/or subcontractors

Integrated Master Schedule and Work Breakdown Structure that encompass build-out and transition-to-operations activities with respect to the IOC/FOC milestones described in Section J, Attachment J-8, IOC/FOC Target Timeline

Details of existing 3rd Generation Partnership Project (3GPP) standards-based mobile broadband service capabilities, mobile and fixed broadband infrastructure, and operations controlled or managed by the Offeror or its proposed teaming partners and/or subcontractors

Solution to leverage existing commercial and/or other infrastructure

The Offeror’s proposed solution will also be evaluated based on its approach pertaining to:

Corporate-level organizational structure, including the teaming/subcontracting roles and responsibilities as they relate to the NPSBN

Comprehensive program management reflecting the Offeror’s ability to provide seamless, efficient management of the NPSBN over the life of the contract and any subsequent task orders

Supporting and facilitating FirstNet’s compliance with the Act and other applicable local, state, tribal, and federal legislation (including, for example, the National Environmental Policy Act [NEPA], and the Communications Act)

Organizational structure, staffing plan, and qualifications and experience of key personnel and executive leadership that will support the FirstNet program

M.4.1.3 Section Three – Public Safety Customer Acquisition

The Offeror must propose an approach to satisfy FirstNet’s public safety customer acquisition objectives. This sub-factor will evaluate the Offeror’s proposed public safety device connection targets (connection targets), which represent its anticipated number of public safety device connections

Page M-6

(primary users and extended primary users) by each of the 56 states and territories over the life of the contract, as detailed in Section L, Instructions, Conditions, and Notices to Offerors or Respondents. The Offeror’s proposed connection targets will be evaluated relative to the estimated current demand at a state and territory level for the primary user group, as well as the extended primary user group as identified in Section J, Attachment J-24, Public Safety Device Connections Template.

This sub-factor will also evaluate the Offeror’s proposed strategy to minimize time required to provide broadband services to public safety. This includes the Offeror’s current sales and marketing structure, proposed go-to-market strategy for adoption, and nationwide sales and marketing plan to drive widespread adoption of FirstNet priority- and preemption-capable products and services by public safety users. Preference in the evaluation will be placed on adoption by primary users.

The Offeror’s proposed solution will be evaluated based on its approach pertaining to:

Connection targets

Go-to-market strategy to sell and market services, including priority and preemption, to public safety users, including pricing and incentive programs

Marketing of an applications ecosystem

Description and staffing of the marketing and sales organization(s) tasked to support the NPSBN

Ability to meet current, emerging, and future customer needs and standards, including a device portfolio and estimated price points

M.4.1.4 Section Four – Customer Care and Life-Cycle Sustainment

The Offeror’s proposed approach must demonstrate its ability to satisfy customer care and life-cycle innovation as stated in Section C, SOO. This sub-factor will be evaluated based on the Offeror's proposed ability to demonstrate that its solution will deliver provisioning capabilities, and solution for service and delivery, including all linkages to sales, fulfillment, and customer care. Additionally, the Offeror’s solution will be evaluated based on the proposed billing management capabilities and proposed management approach.

The Offeror’s proposed solution will be evaluated based on its approach pertaining to:

Performance monitoring and reporting for devices/network equipment, services, and customer care, including providing subscribers with activation, repair, technical assistance, replacement devices, and emergency restoration support

Metrics to monitor customer satisfaction, the service levels reported by current customers, and the solution for maintaining or improving these service levels over the 25-year period of performance

Customer care strategy to minimize churn and promote customer retention among public safety users

Current billing support services for broadband and wireless services and any current public safety services throughout the FirstNet service area

M.4.1.5 Section Five – Offeror Financial Sustainability

The Offeror’s proposed approach must demonstrate its ability to satisfy financial sustainability requirements. This section will be evaluated based on the Offeror’s solution regarding its financial robustness to develop, implement, sustain, and enhance the NPSBN within the time frames, duration, and objectives set out in this RFP.

Page M-7

The Offeror’s proposed solution will be evaluated based on the following aspects:

Financial statements of the Offeror in order to demonstrate financial stability and capacity

Details of source funding or financing to support the NPSBN

Terms of any parent company or other guarantees

Financial forecasts and how the Offeror proposes to commercialize the excess network capacity

M.4.1.6 Section Six – Delivery Mechanism for State Plans

In accordance with the Middle Class Tax Relief and Job Creation Act of 2012 (the Act), upon completion of the RFP process, FirstNet must present a plan to the governor of each state and territory that includes, among other things, “… details of the proposed plan for buildout of the nationwide interoperable, broadband network in such State.”

It is FirstNet’s intent to deliver these plans using an online, interactive tool. The Offeror’s proposed solution will be evaluated on the following aspects:

Readability, navigability, organization, and other pertinent aspects of the user interface design

Ability to support the objectives outlined in Section J, Attachment J-18, Delivery Mechanism Objectives for State Plans

Technical design and system adaptability and security

M.4.1.7 Section Seven – Quality Assurance Surveillance Plan

The Offeror will be evaluated on its proposed Quality Assurance Surveillance Plan and the extent to which it accurately monitors and communicates to FirstNet the contractual, operational, financial, and technical performance of the NPSBN.

M.4.1.8 Section Eight – Deliverables Table

The Offeror’s proposed deliverables will be evaluated based on its ability to demonstrate industry best practices and professional expertise as well as the alignment with the proposed performance metrics/standards defined in the Offeror’s Quality Assurance Surveillance Plan (Section J, Attachment J- 9, QASP Surveillance Matrix Template).

M.4.2 Volume II – Coverage and Capacity Factor

The Offeror’s proposed solution will be evaluated based on its ability to provide coverage and capacity solutions as described in the sub-factors below. Coverage propagation and geographic information system tools will be utilized to assist in the evaluation process. These tools include but are not limited to MapInfo 11 and ArcGIS. This factor will be evaluated based on both a quantitative and qualitative perspective. The coverage and capacity factor includes evaluation in the following sub-factors:

Coverage and Capacity Maps and Statistics

Radio Access Network (RAN) Strategy and Solutions

IOC Milestones for Coverage and Capacity

M.4.2.1 Coverage and Capacity Maps and Statistics

The Government will evaluate this sub-factor utilizing a quantitative approach. The Offeror’s proposed solution will be evaluated for each of the 56 states and territories using the information provided by the Offeror through coverage maps as well as network statistics included in Section J, Attachment J-17, Page M-8 Amendment 009

Coverage and Capacity Template. The Government will evaluate the maps and statistics against the coverage objectives specified in Section J, Attachment J-1, Coverage and Capacity Definitions. For the coverage maps, each individual grid block will be assessed for meeting the definition of coverage (see Section J, Attachment J-1, Coverage and Capacity Definitions). Only those grid blocks that have a reasonable amount of coverage will be considered acceptable. As noted in Section L, Instructions, Conditions, and Notices to Offerors or Respondents, Section L.3.2.1.2.2, Network Planning and Design, detailed site locations are not required; however, for evaluation purposes, the Government reserves the right to request detailed site information (to include all site data for up to two counties per state or territory.) If requested, these data are to be supplied using the “Site Summary” tab in Section J, Attachment J-17, Coverage and Capacity Template.

The Government will evaluate the Offeror’s proposed solution using a quantitative approach for each of the following coverage and capacity elements, which are of equal importance:

Non-Band 14 Area Coverage – The amount of land mass that is covered with non-Band 14 coverage solutions

Non-Band 14 Population Coverage – The amount of population that is covered with non-Band 14 coverage solutions

Band 14 Area Coverage – The amount of land mass that is covered with Band 14 coverage solutions

Band 14 Population Coverage – The amount of population that is covered with Band 14 coverage solutions

Band 14 Network Capacity – The amount of designed network capacity for first responders and secondary users

M.4.2.2 RAN Strategy and Solutions

The Offeror’s proposed solution will be evaluated qualitatively based on the proposed RAN design solution, which must include architecture, functionality, design, implementation, roadmap, and operational strategies that effectively use resources, skillsets, an organizational structure, and tools.

M.4.2.3 IOC Milestones for Coverage and Capacity

The Offeror’s proposed solution will be evaluated qualitatively based on the proposed approach to meeting and/or exceeding the deployment schedule in accordance with the coverage and capacity milestones as provided in Section J, Attachment J-8, IOC/FOC Target Timeline.

M.4.3 Volume II – Products and Architecture Factor

The Offeror’s solution will be evaluated based on its proposed approach, including innovation, creativity, and thoroughness, shown in planned execution of the overall objectives as stated in Section C, SOO, and associated Section J attachments as applicable. The products and architecture factor includes evaluation of the Offeror’s proposed solution in the following sub-factors:

Services

Applications

Device Ecosystem

Architecture and Infrastructure

Operations

Security

Test Strategy

Page M-9

M.4.3.1 Services

M.4.3.1.1 Basic Network Services

The Offeror will be evaluated based on its proposed solution to enable basic network services, including but not limited to messaging, streaming services, voice telephony, machine-to-machine, Next Generation 9-1-1, lawful intercept, Wireless Emergency Alerts, and basic data for the NPSBN. The Offeror should outline its approach based on each of the IOC/FOC milestones and compliance with 3GPP or relevant international standards.

M.4.3.1.2 Quality of Service, Priority, and Preemption

The Offeror will be evaluated on its proposed solution for Quality of Service, Priority, and Preemption (QPP) and service readiness in the event of an emergency and/or network congestion. The Offeror will be evaluated on its proposed solution (including systems, interfaces, and settings) pertaining to QPP states (static, dynamic, controlled), Covered Leasing Agreement (CLA) user-states (free range, restricted, pre-empted), emergency states (user type and role), QPP profiles and static user data (default and emergency QPP profiles for different users with different roles), dynamic data (user location, user operational status, incident role, incident identifier, incident location, and incident severity), QPP application profiles, group of application profiles into operational profiles for an agency, dynamic QPP management, and end-to-end Quality of Service and Priority for public safety users. The Offeror will be evaluated on its proposed solution of a dynamic controller.

M.4.3.1.3 Identity, Credential, and Access Management

The Offeror will be evaluated based on its proposed solution pertaining to Identity, Credential, and Access Management (ICAM) within the following elements:

Federated Identity Management – The Offeror’s ability to support federated identity management, allowing users of one agency to access data and services provided by a different agency, including evaluation of the proposed federated identity interfaces that the solution supports and any impacts on an existing agency’s infrastructure and legacy applications that an agency exposes.

Identity Proofing and Onboarding – The Offeror’s ability to support agencies being properly onboarded to leverage the FirstNet ICAM solution, including identifying proofing of users and ensuring the agency is in compliance with security parameters (additional details available in Section J, Attachment J-10, Cybersecurity). The Offeror’s proposed timelines and processes for onboarding and certifying an agency and identity-proofing users will be considered. This will include the proposed process for addressing agencies that lack a compliant identity proofing and onboarding solution/process (i.e., Identity-as-a-Service).

Credential Management – The Offeror’s ability to support credential management, including ensuring credentials are secure and align with the specifications in Section J, Attachment J-4, System and Standard Views, including but not limited to the management of user attributes and access policies that enable interoperability between agencies.

Single Sign On and Authentication – The Offeror’s ability to support effective, efficient, realistic, and secure methods for public safety users to authenticate, including but not limited to authentication into devices, mobile applications, Web applications, and desktop applications.

Authorization – The Offeror’s ability to support dynamic access management and the manner in which its solution will easily enable applications to authorize users before granting access to the

Page M-10 application or data, including but not limited to how static and dynamic access policies are created, managed, and applied to applications, services, and resources.

M.4.3.1.4 Mission-Critical Services

The Offeror will be evaluated based on its proposed approach regarding the design and plans for mission-critical Push-to-Talk, data, voice, proximity services, and location services for each IOC/FOC milestone, including its compliance with relevant international standards.

M.4.3.2 Applications

The Offeror will be evaluated based on its proposed solution to execute an applications ecosystem, as defined in Section M.4.3.2.1, Applications Ecosystem. This will include demonstrating its ability to provide an applications ecosystem that supports the NPSBN with capabilities and services relevant to public safety. This will include its proposed solution to provide an ecosystem that includes, at a minimum, an evolving portfolio of mobile, enterprise, cloud services, and applications; an applications development platform; a vibrant application developer community; a FirstNet applications store; local control of users, subscriptions, services, and applications; federation of identity management, data, applications, and resource sharing across diverse public safety agencies; Core service and application delivery platforms; data and applications security; and privacy compliance across local, state, regional, tribal, and federal users.

In addition, this sub-factor includes evaluation of the Offeror’s proposed solution within the following elements:

Applications ecosystem

Offeror-provided applications

M.4.3.2.1 Applications Ecosystem

The Offeror’s proposed solution for the applications ecosystem will be evaluated based on the service delivery platform, application development platform, hosting and cloud services, FirstNet applications store, application life-cycle management, developer and application certification, and application security described.

M.4.3.2.1.1 Service Delivery Platform

The Offeror’s proposed solution will be evaluated based on its operational capabilities, which includes gateway policy (i.e., authorization, privacy, throttling, and quotas), security, Application Programming Interfaces (APIs) middleware, and transformation to back-end network service platforms. Additionally, this will include the Offeror’s proposed approach regarding its ability to orchestrate one or more network services with an application consuming these APIs, as well as for first responders handling responder emergencies and immediate peril events.

M.4.3.2.1.2 Application Development Platform

The Offeror’s proposed solution will be evaluated based on how it supports, allows, and facilitates rapid and innovative third-party public safety application development. In addition, the Offeror’s roadmap for creating a vibrant application developer community will be evaluated.

Page M 10.1

M.4.3.2.1.3 Hosting and Cloud Services

The Offeror’s proposed solution will be evaluated based on the extent to which its cloud services solution provides the necessary service redundancy, resiliency, and contingency capabilities to ensure service availability. This includes the analysis of proposed offerings for Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS) solutions, as well as data analytics, storage services, and analytics platform.

Page M-11

M.4.3.2.1.4 FirstNet Applications Store

The Offeror will be evaluated based on its solution and demonstrated approach for supporting a FirstNet applications store, including how the applications store provides public safety agencies and users with a cost-effective, easy method to access secure, value-added public safety applications. The evaluation will include assessing a client’s ability to search, browse, sort, select, download, purchase, review, and rate applications. The evaluation will include an assessment of the proposed approach for the application developers’ registration process, ease of application publishing and management, application version control, and application monetization and settlements methods.

M.4.3.2.1.5 Application Life-Cycle Management

The Offeror’s proposed solution will be evaluated based upon its ability to demonstrate knowledge of application life-cycle management processes, including governance, development, deployment, and operations. This will also include an assessment of the proposed approach for how applications are created based on various categories (e.g., mobile, enterprise, desktop), tested, deployed, updated, monitored, and deprecated.

M.4.3.2.1.6 Developer and Application Certification

The Offeror’s proposed solution will be evaluated based on its approach to developing a timely and effective solution for certifying application developers and public safety applications as a means to ensure the applications function as expected and are secure, resilient, scalable, and free from malware and unintended behavior. This will include a detailed analysis of the Offeror’s approach pertaining to the processes and criteria for certifying application developers and certifying applications. Additionally, the evaluation will include an assessment of the proposed approach for both newly developed and existing public safety applications and how they are handled and certified.

M.4.3.2.1.7 Application Security

The Offeror’s proposed solution will be evaluated based on its approach regarding security of the application layer and the data associated with users. This will include an assessment of the proposed solution regarding data protection against unauthorized access and maximizing privacy, data integrity, and data availability in end-to-end scenarios. Additionally, this evaluation will include an assessment of the Offeror’s ability to enable the secure coexistence of certified FirstNet and commercial applications on a device. This will include evaluation pertaining to the approach for data and applications security monitoring, alerting, and mitigation for ongoing security operations.

M.4.3.2.2 Offeror-Provided Applications

The Offeror will be evaluated based on its proposed solution for demonstrating its understanding of the effort, including innovation, creativity, and thoroughness of providing the specific applications described below.

M.4.3.2.2.1 Local Control Application

The Offeror’s proposed solution will be evaluated for providing direct and indirect control of the NPSBN to Public Safety Entities (PSEs) through local control. Local control is a set of features that allows a PSE to affect direct change over the operational and administrative characteristics of the NPSBN for its users as well as a set of business processes that provides the PSE indirect input into operational aspects of the NPSBN that affect its users. Details can be found in Section L, Instructions, Notices, and Conditions to Offerors or Respondents, Section L.3.2.2.2.2.1, Local Control Application.

Page M-12

M.4.3.2.2.2 Public Safety Entity Home Page

The Offeror’s proposed solution for providing a PSE home page will be evaluated. The home page will provide a central point of access to information about the NPSBN as well as to other applications and information, such as the local control application, news and weather information of interest to the PSE, and link(s) to the customer-facing, Web-based portal. Details can be found in Section L, Instructions, Notices, and Conditions to Offerors or Respondents, Section L.3.2.2.2.2.2, Public Safety Entity Home Page.

M.4.3.3 Device Ecosystem

The Offeror’s proposed solution pertaining to the device ecosystem will be evaluated based on a robust device portfolio and its roadmap. This will include the proposed approach to provide the necessary certifications and support.

In addition, this sub-factor includes evaluation of the Offeror’s proposed solution within the following elements:

Management of the Universal Integrated Circuit Card (UICC) SIM types that automatically support all partner and roaming network(s) and the life-cycle support of the profile(s). Devices that can operate across multiple networks with a single UICC are preferable.

Support of a standards-compliant Device Management client.

Device approval process, including Federal Communications Commission (FCC) certification, Personal Communications Services (PCS) Type Certification Review Board (PTCRB) certification (including the waiver request process), FirstNet’s Device Independent Verification and Validation (IV&V), and carrier acceptance for current and future devices.

M.4.3.3.1 Device Portfolio

The Offeror will be evaluated based on its proposed solution for providing a device portfolio and roadmap that supports Section C, SOO, and completion of Section J, Attachment J-11, Device Specification Template. The proposed solution will be evaluated based on the diversity of the portfolio across key device categories, the Offeror’s ability to operate securely on the NPSBN and partner networks, and the ability to be interoperable with the FirstNet applications ecosystem. The proposed solution will also be evaluated based on the device ecosystem as a whole, its life-cycle management, and support for a variety of UICC configurations, accessories, a Bring Your Own Device (BYOD) ownership model, and Mobile Device Management (MDM).

M.4.3.3.2 Band 14 Devices

The Offeror’s solution will be evaluated on the proposed approach to provide a Band 14-capable device portfolio and roadmap(s), including but not limited to portables, modems, in-vehicle routers, Vehicular Network Systems, and Machine to Machine (M2M) or Internet of Things (IoT) configurations. The Offeror will also be evaluated on the management ecosystem strategy for devices, including device management and application management and security, as well as the portfolio approach based on each IOC/FOC milestone and compliance with 3GPP or other relevant international standards.

Page M-13

M.4.3.3.3 Universal Integrated Circuit Card Management

The Offeror will be evaluated on its proposed solution for the management of the UICC and the life-cycle support of the profile(s). Devices that can operate across multiple networks with a single UICC are preferable.

M.4.3.3.4 Device Management Client

The Offeror will be evaluated on its proposed solution for support of a standards-compliant device management client.

M.4.3.3.5 Device Approval Process

The Offeror’s proposed solution will be evaluated based on its demonstrated ability to provide details of its device approval process, including FCC certification, PTCRB certification (including the waiver request process), FirstNet’s Device IV&V, and commercial carrier acceptance for current and future devices.

M.4.3.4 Architecture and Infrastructure

The Offeror will be evaluated on its proposed NPSBN network architecture solution comprising the nationwide Core network architecture, state integration plans, transmission systems, and interconnection and interworking.

M.4.3.4.1 Nationwide Core Network Architecture and State Integration

The Offeror will be evaluated on its proposed NPSBN network architecture solution comprising the following areas for each IOC/FOC milestone:

Logical architecture regarding system and architecture views for all user and control planes for the NPSBN

Integration of secondary users without impacting public safety users and services

Services to public safety users

Key Core network locations used for the NPSBN

Network specifications, design criteria, service quality, and operational metrics of the Offeror’s solution

Session continuity between the NPSBN and other networks

Roaming strategy pertaining to support of roaming with other wireless networks

Integration of roaming partners, ensuring seamless wireless services throughout the coverage footprint for public safety users, to include an outline demonstrating the proposed approach for integration of rural telecommunications providers

Internet Protocol (IP) address assignments in the NPSBN and interworking with other networks

Heterogeneous network integration, including the proposed strategy regarding integration of wireless technologies (e.g., Wi-Fi, small cells, distributed antenna solutions) into the NPSBN to form a seamless network implementation and operation

M.4.3.4.2 Transmission Systems Strategy

The Offeror will be evaluated on its ability to provide a transmission systems approach supporting RAN backhaul, backhaul aggregation, and a nationwide backbone transmission system for each IOC/FOC milestone. Specifically, the Offeror will be evaluated on its proposed solution in the following areas:

RAN backhaul architecture, topology, and synchronization approach for its RAN solution

Page M-14

RAN backhaul system aggregation transport network approach

National transmission network approach

End-to-end security of the transport network

Routing and Diameter Routing Agent (DRA) approach for the user and signaling plane traffic on the NPSBN

Transport service prioritization to ensure the integrity of end-to-end services for public safety, specifically with regard to QPP

M.4.3.4.3 Interconnection and Interworking

The Offeror will be evaluated on its ability to provide a solution for the interconnection and interworking required to support a seamless and interoperable NPSBN. Specifically, the Offeror will be evaluated on its proposed solution in the following areas:

Integration of state-deployed RANs with the NPSBN, including population locations detailing the function for each location at each of the 56 states and territories

Public Safety Enterprise Network (PSEN) and Public Safety Answering Point (PSAP) integration in accordance with the FOC milestones

Public Switched Telephone Network (PSTN), Internet Service Provider (ISP), and peering integration in accordance with each IOC/FOC milestone

Public Land Mobile Network Number (PLMN) and roaming partner integration in accordance with each IOC/FOC milestone

M.4.3.4.4 Public Safety Grade

The Offeror will be evaluated on its proposed solution for ensuring a level of hardening and resiliency within the NPSBN that will be necessary for public safety services, especially in case of operational challenges that are expected during natural and man-made events. Specifically, the Offeror’s proposed solution will be evaluated based on the following areas:

Network reliability related to the Offeror’s network design, and support for demonstrating how its solution consistently performs according to its specifications for each IOC/FOC milestone

Network resiliency in order to maintain the restoration time specified in Section C, SOO, in the face of natural disasters, faults, and other challenges to normal operation for each IOC/FOC milestone

Network redundancy in order to increase service availability through local and geo-redundancy solutions designed into the NPSBN for each IOC/FOC milestone

Mitigation of environmental factors that may have an adverse effect on the NPSBN’s performance

Operational management—including preventative and proactive measures for maintenance, disaster support, and new technologies—which results in a continual improvement of network performance, services, and support of public safety users

M.4.3.4.5 Network Implementation

The Offeror will be evaluated on its network implementation solution comprising network integration strategy, design assumptions, naming and numbering strategies, and implementation approach.

Specifically, the Offeror’s proposed solution will be evaluated based on the following areas:

Network integration with partner service providers at each IOC/FOC milestone

Page M-15

Naming and identifying NPSBN network nodes to facilitate seamless network services implementation and operation at each IOC/FOC milestone

NPSBN design assumptions at each IOC/FOC milestone

Numbering/addressing schema for public safety devices

Program approach and schedule for the implementation of the NPSBN, covering all IOC/FOC milestones

Migration of public safety users from any temporary non-Band 14 network to the NPSBN as soon as the NPSBN is available in a region

Mobile number portability to ensure public safety users do not change their phone numbers when migrating to the NPSBN

M.4.3.5 Operations

The Offeror’s solution will be evaluated based on the proposed design, architecture, and approach regarding an effective and complete operational life-cycle model that is consistent with the Information Technology Infrastructure Library (ITIL®) or other commercial operational and testing plans for all aspects of the NPSBN. This model should include processes needed for day-to-day management of the network, reactive processes around incidents and national events, and proactive processes resulting in continual improvement of service availability. The areas of operations that follow will be evaluated for the Offeror’s proposed solution.

M.4.3.5.1 Network and Service Operations

The Offeror will be evaluated on how its managed services in the following areas will meet the stated service availability objectives for the NPSBN, as identified in Section C, SOO, for all services and applications. These areas contain processes that focus on continuous service delivery and proactive operations of the NPSBN. The Offeror will also be evaluated based on its proposed solution for the following areas:

Systems and processes to identify and resolve service degradation issues

National Incident Management System (NIMS) processes and interfaces

Release management processes

Business continuity/disaster recovery management processes

Availability management processes

Change management processes

Capacity management processes

National and local support structure for network configuration, maintenance, and monitoring

Procedures and protocols to support use of deployable assets for natural disasters and major events

M.4.3.5.2 Business and Operational Support Systems

The Offeror will be evaluated on how its business and operational support systems in the following areas will support meeting the stated service availability objective for the NPSBN, as identified in Section C, SOO, for all billing, operational, and provisioning support systems. Each of these areas should contain processes that focus on support of the user life-cycle on the NPSBN. The Offeror’s proposed solution will be evaluated based on the following areas:

Business and Operational Support Systems (B/OSS) infrastructure

Page M-16

Network and Element Management Systems

End-user and device provisioning and management systems

Configuration management systems

Billing system capability and flexibility in defining new profiles and billing

Device management systems

Subscriber management (customer relationship management) systems

Asset management systems

Trouble ticketing systems

Workflow systems

Data processing

APIs available to government for creation of automated reports

M.4.3.5.3 Services Management Center

The Offeror’s proposed solution will be evaluated based on its ability to provide an NPSBN Services Management Center that is effective in managing the following operational network and service verticals.