Exhibit A - BDC Mobile Technical Requirements Order.pdf

Federal Communications Commission DA 22-241

Before the Federal Communications Commission

Washington, D.C. 20554

In the Matter of

Establishing the Digital Opportunity Data Collection

WC Docket No. 19-195

ORDER

Adopted: March 9, 2022 Released: March 9, 2022

By the Chiefs, Wireless Telecommunications Bureau, Office of Economics and Analytics, Office of Engineering and Technology:

TABLE OF CONTENTS

Heading Paragraph #

I. INTRODUCTION

II. BACKGROUND

III. DISCUSSION

A. Mobile Service Challenge Process

1. Creating a Challenge/Cognizable Challenges

2. Challenge Responses

a. Rebutting Challenges with On-the-Ground Data

b. Rebutting Challenges with Infrastructure Data

c. Other Data

B. Collecting Verification Information from Mobile Providers

1. Area Subject to Verification

2. Sampling Methodology

3. On-the-Ground Test Data

4. Infrastructure Information

5. Transmitter Monitoring Information

C. Collecting Verified Broadband Data from Government Entities and Third Parties D. Crowdsourced Data

1. Tools to Submit Crowdsourced Data

2. Crowdsourced Data Submitted in the Online Portal

3. When Crowdsourced Filings Reach a “Critical Mass”

4. Public Availability of Crowdsourced Data

E. Other Matters

IV. PROCEDURAL MATTERS

V. ORDERING CLAUSES

APPENDIX A: TECHNICAL APPENDIX

APPENDIX B: FINAL RULES

APPENDIX C: SUPPLEMENTAL FINAL REGULATORY FLEXIBILITY ANALYSIS

I. INTRODUCTION

1. In this Order, the Wireless Telecommunications Bureau (WTB), Office of Economics and Analytics (OEA), and Office of Engineering and Technology (OET) (collectively, the Bureau and Offices) adopt the technical requirements to implement the mobile challenge, verification, and crowdsourcing processes set forth in Appendix A (Technical Appendix) and required by the Broadband DATA Act1 as part of the FCC’s ongoing Broadband Data Collection (BDC) effort to improve the Commission’s broadband availability data.2

2. Specifically, we adopt the proposed processes and methodology set forth in the BDC Mobile Technical Requirements Public Notice for collecting challenge process data and for determining when the threshold to create a cognizable challenge has been met.3 Additionally, we adopt detailed processes for mobile providers to respond to challenges, for the Commission to initiate a verification request to a service provider, and for providers to respond to verification requests to confirm broadband coverage in areas they claim have service. We adopt the parameters and metrics set forth in the Technical Appendix that must be collected both for on-the-ground test data to support challenge submissions, rebuttals to cognizable challenges, and responses to verification requests, and for infrastructure information to support challenge rebuttals and responses to verification requests. We require government entities and third parties to submit verified broadband data using the same data specifications we require of mobile service providers. Finally, we find the Commission’s speed test app to be a reliable and efficient method for entities to use in submitting crowdsourced mobile coverage data to the Commission and describe the methodology staff will use in determining when a “critical mass of” crowdsourced filings suggests that a provider has submitted inaccurate or incomplete data. The measures that we adopt in this Order to implement the mobile challenge, verification, and crowdsourcing processes will enable the Commission, Congress, other federal and state policy makers, Tribal entities, consumers, and other third parties to verify and supplement the data collected by the Commission on the status of mobile broadband availability throughout the United States.

II. BACKGROUND

3. Congress and the Commission have taken several steps in recent years to improve the broadband availability data that the Commission collects and publishes on its coverage maps. In March 2020, Congress passed the Broadband DATA Act, which requires the Commission to collect more granular and consistent data from broadband internet access service providers on the availability and quality of broadband service, as well as to establish a challenge process; verify the accuracy and reliability of the broadband coverage data that providers are required to submit; and improve data accuracy through a crowdsourcing process.4 The Broadband DATA Act also requires the Commission to develop “a process through which it can collect verified data for use in the coverage maps from: (1) [s]tate, local, and Tribal governmental entities that are primarily responsible for mapping or tracking broadband internet access service coverage for a [s]tate, unit of local government, or Indian Tribe, as

1 Broadband Deployment Accuracy and Technological Availability Act, Pub. L. No. 116-130, 134 Stat. 228 (2020) (codified at 47 U.S.C. §§ 641-646) (Broadband DATA Act).

2 See, e.g., 47 U.S.C. §§ 642(a)(1)(B)(i), (iii), (iv), (a)(2), (b)(4), (b)(5), 644(b); see also Comment Sought on Technical Requirements for the Mobile Challenge, Verification, and Crowdsource Processes Required under the Broadband DATA Act, WC Docket No. 19-195, Public Notice, DA 21-853, 2021 WL 3057378 (WTB/OEA/OET July 16, 2021) (BDC Mobile Technical Requirements Public Notice).

3 The Commission referred to a “cognizable challenge” as one requiring a provider response and directed OEA, in consultation with WTB, to establish the methodology for determining this threshold. Establishing the Digital Opportunity Data Collection; Modernizing the FCC Form 477 Data Program, WC Docket Nos. 19-195, 11-10, Third Report and Order, 36 FCC Rcd 1126, 1168, para. 105 (2021) (Third Order).

4 See, e.g., 47 U.S.C. §§ 642(a)(1)(B)(i), (iii), (iv), (b)(4), (b)(5), 644(b).

applicable; (2) third parties . . . ; and (3) other Federal agencies.”5 These tools are designed to provide a more accurate data collection that is informed by input from consumers, other federal agencies, state and local governments, Tribal entities, providers, and other entities.

4. In its Second Order in this proceeding, the Commission implemented a number of the Broadband DATA Act’s requirements for collecting and reporting broadband data from providers, developed the framework for the BDC, established a process for verifying the broadband data it receives from providers in their BDC filings, and adopted a basic framework for collecting crowdsourced information.6 In the Third Order, the Commission adopted additional requirements for collecting and verifying provider-submitted data and established the challenge process.7

5. In the Third Order the Commission determined that it should aggregate speed test results received from multiple consumer challenges in the same general area in order to resolve challenges in an efficient manner, mitigate the time and expense involved, and ensure that the mobile coverage maps are reliable and useful.8 The Commission acknowledged that consumers are likely to submit challenges in distinct, localized areas and recognized that providers should not be subject to the undue cost of responding to a large number of challenges in very small areas.9 The Commission directed OEA, in consultation with WTB, to determine the threshold number of mobile consumer challenges within a specified area that will constitute a cognizable challenge triggering a provider’s obligation to respond.10 In connection with that determination, the Commission also directed OEA, in consultation with WTB, to establish: (1) the methodology for determining this threshold;11 and (2) the methodology for determining the boundaries of a geographic area where the threshold for a cognizable challenge has been met.12

6. Consistent with the approach it adopted for consumer challenges, the Commission stated that it would also aggregate speed test evidence received from multiple government and third-party challengers in the same general area.13 The Commission directed OEA, in consultation with WTB, to determine the threshold number of such challenger speed tests within the same general area that constitute

5 47 U.S.C. § 642(a)(2).

6 Establishing the Digital Opportunity Data Collection; Modernizing the FCC Form 477 Data Program, WC Docket Nos. 19-195, 11-10, Second Report and Order and Third Further Notice of Proposed Rulemaking, 35 FCC Rcd 7460 (2020) (Second Order and Third Further Notice). While the challenge process, crowdsourced data, and other Commission efforts will all serve to validate the data submitted by providers, for purposes of this Order, “verification” or “verification process” refers to the internal process the Commission sought comment on in Section IV.D. of the Third Further Notice and adopted in Section III.E. of the Third Order. Id. at 7503-06, paras. 104-09;

Third Order, 36 FCC Rcd at 1146-51, paras. 47-60; see also 47 U.S.C. § 642(b)(4) (instructing the Commission to “verify the accuracy and reliability of the information in accordance with measures established by the Commission”).

7 Third Order, 36 FCC Rcd at 1146-51, 1164-75, paras. 47-60, 97-125.

8 Id. at 1167-68, para. 105. The Commission found that, when the aggregated results reach an appropriate threshold, they will constitute a cognizable challenge requiring a provider response. Id. at 1168, para. 105.

9 Id. at 1167-68, para. 105.

10 Id. at 1167, para. 105; 47 CFR § 1.7006(e)(2).

11 Third Order, 36 FCC Rcd at 1168, para. 105; see 47 CFR § 1.7006(e)(2). The Commission stated that, “[i]n developing this methodology, OEA should consider, inter alia, the number, location, and timing of the tests, variability in test results, and whether the tests were conducted in urban or rural areas.” Third Order, 36 FCC Rcd at 1168, para. 105.

12 Third Order, 36 FCC Rcd at 1168, para. 106; see 47 CFR § 1.7006(e)(2).

13 Third Order, 36 FCC Rcd at 1173, para. 120.

a cognizable challenge and require a provider response.14 Similar to the consumer challenges, the Commission directed OEA, in consultation with WTB, to establish the methodology for determining this threshold and the boundaries of an area where the threshold has been met.15 The Commission also delegated to the Bureau and Offices certain responsibilities to refine the verification process, and to establish a way for governments to submit verified mobile coverage data, among other things.16

7. In July 2021, the Bureau and Offices released the BDC Mobile Technical Requirements Public Notice seeking comment on proposed technical requirements for the mobile challenge, verification, and crowdsourcing processes required under the Broadband DATA Act.17 The Public Notice included proposals to implement processes delegated to the Bureau and Offices, including establishing:

thresholds for a cognizable challenge to mobile wireless broadband availability data; a process for mobile providers to respond to challenges; a process for collecting verified on-the-ground and infrastructure data from mobile providers in response to a verification inquiry from the Commission; a collection of verified broadband data from government and third-party entities; and processes for the Commission to collect and use crowdsourced data.18 Sixteen parties filed comments and/or replies, including service providers, trade associations, state governments, technology providers, and public interest organizations.19

III. DISCUSSION

A. Mobile Service Challenge Process

8. In this Order, the Bureau and Offices adopt the proposals for the mobile challenge process set forth in the BDC Mobile Technical Requirements Public Notice, with certain modifications described below.

9. The Broadband DATA Act requires that the Commission “establish a user-friendly challenge process through which consumers, [s]tate, local, and Tribal governmental entities, and other entities or individuals may submit coverage data to the Commission to challenge the accuracy of – (i) the coverage maps; (ii) any information submitted by a provider regarding the availability of broadband internet access service; or (iii) the information included in the [Broadband Serviceable Location] Fabric.”20 The general requirements and framework for the mobile challenge process predate the BDC Mobile Technical Requirements Public Notice, and were set forth in either the Broadband DATA Act or

14 Id. at 1168, 1173, paras. 105-06, 120.

15 Id.

16 Id. at 1150, 1154, paras. 59, 68.

17 BDC Mobile Technical Requirements Public Notice. This Order uses Westlaw *pagination for the BDC Mobile Technical Requirements Public Notice.

18 BDC Mobile Technical Requirements Public Notice at *4-6, paras. 8-14, *7-11, paras. 15-25, *11-17, paras. 26- 42, *17, paras. 43-45, *18-19, paras. 46-50, *19-22, paras. 51-59.

19 Comments were filed by: Competitive Carriers Association (CCA), CTIA, Enablers, Inc. (Enablers), Kimberly J.

Lippi (filed on behalf of California Public Utilities Commission (CPUC)), Rural Wireless Association (RWA), Precision Agriculture Connectivity and Accuracy Stakeholder Alliance (PAgCASA), Ookla, T-Mobile, Vermont Department of Public Service (Vermont DPS), and Verizon. Reply comments were filed by: AT&T, Comniscient Technologies, Inc. (Comniscient), CCA, CTIA, Garland T. McCoy (filed on behalf of PAgCASA), Next Century Cities, Ookla, Opensignal, Inc., New America Open Technology Institute and Public Knowledge (Public Knowledge/New America), RWA, T-Mobile, and Vermont DPS.

20 47 U.S.C. § 642(b)(5)(A); see id. § 642(a)(1)(B)(iii). “Fabric” is defined as the “Broadband Serviceable Location Fabric” established under section 642(b)(1)(B). 47 U.S.C. § 641(6).

prior Commission orders.21 In the Second Order and Third Further Notice, the Commission proposed a challenge process that “encourages participation to maximize the accuracy of the maps, while also accounting for the variable nature of wireless service.”22 In the Third Order, the Commission adopted its proposals from the Second Order and Third Further Notice, and established a framework for consumers, state, local, and Tribal governments, and other entities to submit data to challenge the mobile broadband coverage maps.23

10. The Commission determined that it should enable stakeholders to challenge mobile coverage data based on both a lack of service and poor service quality (such as slow delivered user speeds).24 Challenges must be based upon on-the-ground speed test data taken outdoors (i.e., from an in-vehicle mobile or outdoor stationary environment).25 The Commission adopted a requirement that consumers use a speed test application (either developed by the FCC or a third-party app approved by OET for use in the challenge process) that automatically collects information and metrics associated with each speed test and allows for submission of information directly to the Commission from a mobile device.26 Consumers will be required to submit certain identifying information to deter frivolous filings.27 Government and other third-party entity challengers (including competing mobile service providers) may use their own software or hardware to collect data for the challenge process so long as the data contain metrics that are substantially the same as those collected by approved speed test applications.28 Moreover, government and other entity challengers are required to conduct on-the-ground tests using a device advertised by the challenged provider as compatible with its network.29

11. The Commission adopted a requirement for providers to either submit a rebuttal to the challenge or concede the challenge within 60 days of being notified of the challenge.30 Rebuttals must consist of either on-the-ground test data or infrastructure data.31 A challenge respondent may also submit supplemental data in support of its rebuttal, either voluntarily or in response to a request for additional information from OEA.32 The Commission directed OEA to develop a methodology and mechanism to determine if the data submitted by a provider constitute a successful rebuttal to all or some of the challenged service area and to establish procedures to notify challengers and providers of the results of a challenge.33 Further, the Commission adopted a requirement that providers that concede or lose a

21 We note that, to the extent commenters ask the Bureau and Offices to eliminate, modify, or otherwise revisit particular requirements established in either the Broadband DATA Act or prior Commission-level orders, we lack the legal authority to do so.

22 Second Order and Third Further Notice, 35 FCC Rcd at 7515, para. 141.

23 See Third Order, 36 FCC Rcd at 1164-68, 1171-73, paras. 98-106, 113-20.

24 Id. at 1164-65, 1171, paras. 98, 113.

25 Id. at 1164, 1165, 1166, 1171, 1172, paras. 99, 102, 116, 118.

26 Id. at 1166-67, paras. 103-04.

27 Id. at 1166, para. 101.

28 Id. at 1172, 1173, paras. 117, 119.

29 Id. at 1172, para. 118.

30 Id. at 1168, 1173, paras. 107, 121.

31 Id. at 1168-69, 1173, paras. 108, 121.

32 Id. at 1168-70, 1173-74, paras. 108-10, 121-22.

33 Id. at 1170, para. 111.

challenge file new coverage data within 30 days depicting the challenged area that has been shown to lack service.34

12. The requirements that we adopt in this Order will enable the Commission to collect sufficient measurements to ensure that the challenge process is statistically valid while remaining “user-friendly.”35 In particular, we establish a methodology for determining a threshold number of mobile speed tests and the geographic boundaries within a specified area. Based on this methodology, a challenge is created by associating the locations of validated speed tests within geographical hexagons defined by the accessible, open-source H3 geospatial indexing system and analyzing those speed tests.36 We also adopt the parameters and metrics that speed tests must meet to be validated and used to meet the challenge thresholds. Importantly, as the Commission specified in the Third Order, the challenge process will remain user-friendly because all of the information a consumer needs to create a challenge will be collected and submitted by the FCC Speed Test app and any third-party mobile speed test apps approved by OET. Governmental and other entity challengers may use these apps or their own software or hardware to collect data for the challenge process.37 Additionally, we implement the Commission’s decision to aggregate speed tests to resolve challenges “in an efficient manner, mitigate the time and expense involved, and ensure that the mobile coverage maps are as reliable and useful as possible,”38 by adopting our proposal to combine speed tests conducted by consumers, governmental agencies, and other entities to determine whether the thresholds for a cognizable challenge have been met. These requirements strike the appropriate balance between ensuring that consumers, state, local, and Tribal governments, and other entities can participate in the challenge process, on the one hand, and protecting providers from being burdened by having to respond to challenges that do not meet the cognizable challenge standard, on the other hand.

1. Creating a Challenge/Cognizable Challenges

13. On-the-Ground Speed Test Data Parameters and Metrics. Challenges must be supported by on-the-ground test data. We have therefore established the required testing parameters and data metrics for speed test submissions. At the outset, we will require the FCC Speed Test app and approved third-party apps to collect the name and email address of the end user and mobile phone number of the device on which the speed test was conducted, to the extent technically feasible.39 The Commission’s rules state that consumer challengers must include “name and contact information (e.g., address, phone number, and/or email address) in their data submissions.”40 We amend these rules to require that app users also submit their email address so that the Commission can notify testers of the status of their speed test(s) and any resulting challenge(s), and we also amend the rules to require app users to submit the mobile phone number of the device on which the speed test was conducted so that we may, if necessary,41

34 Id. at 1170, 1174, paras. 112, 124.

35 47 U.S.C. § 642(b)(5)(A).

36 Isaac Brodsky, H3: Uber’s Hexagonal Hierarchical Spatial Index, (June 27, 2018), https://eng.uber.com/h3/.

37 See Third Order, 36 FCC Rcd at 1172, para. 117.

38 Id. at 1168, para. 105.

39 As discussed in further detail below, iOS devices will not automatically transmit the mobile phone number associated with the device that runs a speed test. We will therefore require testers submitting tests for use in the challenge process to manually submit, through the speed test app, the phone number associated with the device on which the speed test was conducted.

40 47 CFR § 1.7006(e)(1)(i).

41 We anticipate we will only share the phone number of the device on which the speed test was conducted with mobile broadband providers in situations where a challenged provider is unable to identify a subscriber by using the timestamp that test measurement data were transmitted to the app developer’s servers, as well as the source IP

(continued….)

https://eng.uber.com/h3/ share this information with mobile broadband providers for use when responding to challenges. We will not collect the address of an end user for use in the mobile challenge process at this time in order to minimize the amount of personally identifiable information we require from end users, and because a mobile user’s physical address is not currently helpful either to the Bureau and Offices when considering challenges or to providers when responding to challenges. In addition to the testing metrics adopted by the Commission in the Third Order,42 we adopt the testing parameters and updated metrics for challenge speed test data proposed in the BDC Mobile Technical Requirements Public Notice, with the modifications described below.43 With the exception of different considerations pertaining to the submission of speed test data taken on iOS devices44 and the submission of IP address, source port, and timestamp measured by an app developer’s servers by government entities and service providers in some scenarios,45 these parameters and metrics will apply across all testing mechanisms, not only in the challenge process but also for on-the-ground data submitted in response to verification inquiries.

Individual consumer challengers must collect on-the-ground speed test data using mobile devices running either a Commission-developed app (e.g., the FCC Speed Test app) or another speed test app approved by OET for the submission of challenges.46 Third-party and governmental entities may, as specified in the Third Order, collect data using either one of these speed test apps or their own software and hardware that collects broadband availability data, consistent with the parameters and metrics set forth herein.47

14. In the Third Order, the Commission required consumer challengers to use a speed test app approved by OET for use in the challenge process and provided the metrics that approved apps must address and port of the device, as measured by the server, which we will also require to be included in challenge data submitted by the app, as discussed below. See infra para. 18.

42 See Third Order, 36 FCC Rcd at 1166-67, 1172, paras. 103, 117; see also 47 CFR § 1.7006(e)(1)(i)-(v). The Commission’s rules state that consumer challengers must include in their data submissions, “name and contact information (e.g., address, phone number, and/or email address).” 47 CFR § 1.7006(e)(1)(i). We will require the FCC Speed Test app and approved third-party apps to collect the consumer’s name, email address, and phone number of the device on which the speed test was conducted to the extent technically feasible.

43 See BDC Mobile Technical Requirements Public Notice at *6, para. 14.

44 The information we will use in the challenge process that can be collected from Android devices, but not iOS devices, includes the signal strength, signal quality, unique identifier, and other RF metrics of each serving cell, as well as the spectrum bands used for the test and other network characteristics (e.g., whether the device was roaming, as well as the identity of the provider for the connected network).

45 As discussed in greater detail below, we will allow government and other third-party entities to alternatively submit the International Mobile Equipment Identity (IMEI) of the device used to conduct a speed test for use in the challenge process rather than provide the source IP address, source port, and timestamp measured by an app developer’s servers. We will also not require a service provider to submit either the device IMEI or the combination of source IP address, source port, and timestamp measured by an app developer’s servers when submitting speed tests either in response to a challenge or in response to a verification inquiry. See infra para. 18, Section III.B.3.

Collecting Verification Information from Mobile Providers, On-the-Ground Test Data.

46 BDC Mobile Technical Requirements Public Notice at *6, para. 14. The Bureau and Offices will announce the process and procedures for third-party app providers to seek approval for a speed test app to be used in submitting data for use in the challenge process.

47 See Third Order, 36 FCC Rcd at 1172, para. 117; see also, id. (noting that the metrics the Commission adopted for government and other entity challenge data “are substantially the same as the metrics [the Commission] require[s] approved speed test applications to collect for consumer challenges.”). We include “hardware” to capture the professional tools such as laptops, hard drives, or other hardware devices, used to collect on-the-ground data.

The Third Order provided that government and other entity challengers submit a complete description of the methodologies used to collect the data. Id.; 47 CFR § 1.7006(f)(1)(ii). The Bureau and Offices will issue a public notice announcing the process and procedures for such parties to submit the necessary documentation.

collect for each speed test.48 The Commission directed OET, in consultation with OEA and WTB, to update the FCC Speed Test app as necessary or develop a new speed test app to collect the designated metrics, so that challengers may use it in the challenge process.49 For government and third-party entity challengers, the Commission did not require the use of a Commission-approved speed test app but instead set forth the information that all submitted government and third-party challenger speed test data must contain and directed OEA, WTB, and OET to adopt additional testing requirements if they determine it is necessary to do so.50 Our BDC Mobile Technical Requirements Public Notice proposed certain testing parameters and metrics to standardize the on-the-ground test data submitted in the challenge process and to assure more reliable challenges;51 a number of parties agree that such consistency among the apps used for challenges and rebuttals is important.52 This set of standardized parameters and metrics will also ensure that we can make a meaningful comparison of tests run by different entities using different methods (e.g., tests run on a speed test app versus a government’s own hardware and software), and will enable us to easily combine and evaluate speed test data used in the challenge process. Accordingly, we will require that such data meet the following testing parameters set forth in the BDC Mobile Technical Requirements Public Notice: (1) a minimum test length of 5 seconds and a maximum test length of 30 seconds;53 (2) test measurement results that have been averaged over the duration of the test (i.e., total bits received divided by total test time); and (3) a restriction that tests must be conducted between the hours of 6:00 a.m. and 10:00 p.m. local time.54

15. We clarify that the minimum and maximum test length parameters will apply individually to download speed, upload speed, and round-trip latency measurements, and will not include ramp up time.55 We disagree with CCA, Public Knowledge/New America, and Vermont DPS that imposing a maximum test limit places an arbitrary or inferior limitation on testing.56 These timing requirements balance representative measurement over a stable Transmission Control Protocol (TCP) connection, on the one hand, versus data usage considerations, on the other hand—especially for consumers who may have limited data plans. The FCC Speed Test app, for example, first initiates a test server selection process, which typically takes two seconds (and a maximum of 10 seconds if servers fail to respond) then individually runs, including a warm-up time, a maximum of eight seconds for download and eight seconds for upload tests by establishing three concurrent TCP connections and summing the three

48 Third Order, 36 FCC Rcd at 1166-67, paras. 103-04.

49 Id. at 1167, para. 104.

50 Id. at 1172, paras. 117-18.

51 BDC Mobile Technical Requirements Public Notice at *6, para. 14.

52 See, e.g., Ookla Comments at 1 (calling the collection of on-the-ground data a “positive step”).

53 To avoid requiring excessive data usage for tests on particularly fast networks (e.g., 5G-NR using high-band spectrum), we will relax the minimum test duration requirement once a download or upload test measurement has transferred at least 1,000 megabytes of data. See BDC Mobile Technical Requirements Public Notice at *6, para. 14;

see also Ookla Comments at 10-11 (“strong incentives exist to accurately complete a test as quickly and easily as possible.”). Specifically, when a speed test transfers at least 1,000 megabytes of data, we will validate the test if it has a duration value of greater than 0 seconds and less than or equal to 30 seconds. Otherwise, a speed test must have a duration value of greater than or equal to 5 seconds and less than or equal to 30 seconds to be valid.

54 BDC Mobile Technical Requirements Public Notice at *6, para. 14.

55 See Ookla Comments at 4 (requesting clarification for whether the proposed test length combines latency, download, and upload measurements into one test or considers them individually, and advocating for a minimum of 5 measurements and maximum of 20 measurements for the latency metric); Ookla Reply at 10-11; Vermont DPS Comments at 5-6 (recommending that the Commission specify a test sequence of 15 seconds’ duration with five seconds for each of the individual component test measurements (upload, download, and latency)).

56 CCA Comments at 3-6; Public Knowledge/New America Reply at 5-6; Vermont DPS Reply at 4-5.

resulting data rates for each test.57 In addition, the round-trip latency testing runs for a fixed five seconds to transmit up to 200 UDP (User Datagram Protocol) packets (i.e., datagrams) to calculate the average latency of those datagrams.58 Hence, a typical test cycle takes approximately 23 seconds to complete, and a maximum of 31 seconds to complete.

16. We also decline to adopt CCA’s request to exempt continuous network monitoring from the maximum test length.59 Continuous network monitoring software can monitor active users’ speeds at the cell sites and other network parameters over extended periods of time.60 We are not persuaded that deviating from the uniform 30-second per test component maximum testing standard to accommodate continuous network monitoring will yield equal or more accurate test results. We found in the Mobility Fund Phase II challenge process that continuous network monitoring speed tests recorded significant variability within the same area and across a short time span, in some cases recording strong network performance well exceeding the minimum requirement interspersed with short seconds-long drops in performance that may have been the result of normal network conditions (e.g., sector handover or network scheduling).61 The overall performance in these areas indicated that coverage was adequate (i.e., with the average of tests in the same area over 15-20 seconds exceeding the minimum requirement), but because the test results were so variable, we are concerned that allowing the reporting of continuous speed tests could result in inaccurate results that do not reflect the typical on-the-ground customer experience, which as the results showed, may be adequate when averaged, but may not deliver consistent speeds to consumers.62 To the extent challengers choose to use continuous network monitoring to record challenge data, results of the speed tests should report the average speeds over a uniform time period consistent with the minimum and maximum test lengths we adopt above (i.e., a minimum of 5 seconds and a maximum of 30 seconds).

17. We share Ookla’s concern that averaging the number of bits received over the entire duration of a throughput test may negatively affect the accuracy of any calculation, as that may not

57 FCC Office of Engineering and Technology, 2021 FCC Speed Test App Technical Description at 6 (2021), https://www.fcc.gov/sites/default/files/2021_fcc_speed_test_app_technical_description.pdf (2021 FCC Speed Test App Technical Description) (the 2021 FCC Speed Test App Technical Description is accessible via the FCC’s Measuring Broadband America Mobile Data webpage under the heading, “Data set scripts and descriptions,” available at https://www.fcc.gov/reports-research/reports/measuring-broadband-america/measuring-broadband-america-mobile-data).

58 2021 FCC Speed Test App Technical Description at 6.

59 See CCA Comments at 3-6.

60 See, e.g., Netscout, TruCall, https://www.netscout.com/sites/default/files/2019-01/SPDS_003_EN-1901%20- %20TrueCall.pdf (last visited Feb. 3, 2022) (discussing how network operators can continuously measure and collect performance metrics directly from their RAN and Core networks).

61 See Rural Broadband Auctions Task Force Staff Report, Mobility Fund Phase II Coverage Maps Investigation Staff Report at 61-62, Appx. B, para. 6 (2019), https://docs.fcc.gov/public/attachments/DOC-361165A1.pdf (Mobility Fund Phase II Investigation Staff Report) (noting that staff analysis “indicates that a large portion of challenger data include speed tests both above and below 5 Mbps within the same general area” and concluding that the MF-II challenge process algorithm was “less reliable for data where a challenger conducted dozens of continuously recorded drive tests along roads within a grid cell”); id. at 62, Appx. B, para. 6, n.9 (suggesting “a more appropriate framework for processing a large number of speed tests recorded in a short time period over a limited area could be the use of statistical calculations (e.g., 90th percentile) to mitigate noise in the data due to the variability of wireless networks”). The Commission is obligated by statute to consider lessons learned in MF-II when creating the challenge process. 47 U.S.C. § 642(b)(5)(B)(i)(V).

62 Mobility Fund Phase II Investigation Staff Report at 61-62, Appx. B, para. 6.

https://www.fcc.gov/sites/default/files/2021_fcc_speed_test_app_technical_description.pdf https://www.fcc.gov/reports-research/reports/measuring-broadband-america/measuring-broadband-america-mobile-data https://www.fcc.gov/reports-research/reports/measuring-broadband-america/measuring-broadband-america-mobile-data https://www.netscout.com/sites/default/files/2019-01/SPDS_003_EN-1901%20-%20TrueCall.pdf https://www.netscout.com/sites/default/files/2019-01/SPDS_003_EN-1901%20-%20TrueCall.pdf exclude an internet connection’s known and expected “ramp-up time.”63 To account for this, we will apply the following formula: [(total bits received – ramp up bits) divided by (total test time – ramp up time)].64 We find that this approach will sufficiently account for ramp-up time and fully satisfy Ookla’s concern, especially in light of the clarification above that the test time limits apply individually to tests’ upload and download measurements.

18. We require on-the-ground speed test data to include a standardized set of metrics. Each on-the-ground speed test must include the following metrics that were previously adopted by the Commission65 as modified by the updates proposed in the BDC Mobile Technical Requirements Public Notice:66 (1) the timestamp and duration of each test metric; (2) geographic coordinates (i.e., latitude/longitude) measured at the start and end of each test metric with typical Global Positioning System (GPS) Standard Positioning Service accuracy or better, along with the location accuracy;67 (3) the consumer-grade device type(s), brand/model, and operating system used for the test; (4) the name and identity of the service provider being tested; (5) location (e.g., hostname or IP address) of the test server;

(6) signal strength, signal quality, unique identifier, and other radiofrequency (RF) metrics of each serving cell, where available; (7) download speed; (8) upload speed; (9) round-trip latency; (10) for an in-vehicle test, the speed the vehicle was traveling when the test was taken, where available. All on-the-ground speed tests must also include the following metrics previously adopted by the Commission:68 (11) whether the test was taken in an in-vehicle mobile or outdoor, pedestrian stationary environment;69 (12) an indication of whether the test failed to establish a connection with a mobile network at the time and location it was initiated; and (13) the network technology (e.g., 4G LTE, 5G-NR) and spectrum bands used for the test.70 We adopt an additional metric that was proposed in the BDC Mobile Technical Requirements Public Notice: (14) the app name and version.71 We will also require all speed tests to include: (15) the timestamp that test measurement data were transmitted to the app developer’s servers, as well as the source IP address and port of the device, as measured by the server.72 Finally, we require on-

63 Ookla Comments at 6 (defining “ramp-up time” as “the time period in which a congestion control algorithm – such as Transmission Control Protocol (“TCP”) slow start – gradually increases the amount of data transmitted over a connection until the algorithm finds the network’s maximum carrying capacity”).

64 We consider “ramp up bits” to be the initial bits received during the initial warm-up time.

65 Third Order, 36 FCC Rcd at 1166-67, 1172, paras. 101, 103, 117.

66 BDC Mobile Technical Requirements Public Notice at *6, para. 14.

67 “Location accuracy” refers to a metric that GPS-enabled smartphones are able to report on the horizontal accuracy of the geographic coordinates of the location reported.

68 Third Order, 36 FCC Rcd at 1172, para. 117.

69 Government and other third-party entities must also indicate whether an in-vehicle mobile test was conducted with the antenna outside of the vehicle.

70 Both para. 103 and para. 117 of the Third Order required speed tests to include the network technology (e.g., 4G LTE or 5G-NR) and spectrum bands used for the test.

71 BDC Mobile Technical Requirements Public Notice at *6, para. 14.

72 Given concerns that challengers may conduct tests after exceeding data limits, we will collect the timestamp that test measurement data were transmitted to the app developer’s servers, as well as the source IP address and port of the device, as measured by the server, so that a service provider may determine if a challenger’s device is subject to reduced speeds or otherwise lacks full network performance. See Verizon Comments at 11. The source port of the device is an available network port over which the device communicates with the server and is unique to a particular network connection or transmission. The IP address and source port associated with the device used in testing is attainable from devices using both iOS and Android devices. For the same reasons, we will allow government and other third-party entities to alternatively submit the IMEI of the device used to conduct the test rather than provide the source IP address, source port, and timestamp measured by an app developer’s servers since such entities are the-ground challenge test data to include all other metrics required per the most recent specification for mobile test data adopted by OEA and WTB in accordance with 5 U.S.C. § 553.73 Third-party app developers and government or other third parties that use their own hardware or software to conduct speed tests will be required to update their processes in accordance with such updates, including, as stated in the BDC Mobile Technical Requirements Public Notice, revised specifications for mobile test data adopted by the Bureau and Office in accordance with 5 U.S.C. § 553. The modified set of parameters and metrics we adopt aligns more closely with those already required of government and third-party challengers.74

19. We recognize the concerns raised by Vermont DPS, Enablers, and Public Knowledge/New America about excessive data and burdens on consumers and governments and other third-party challengers to assure that their data aligns to these standards,75 but we believe that such parameters and metrics are necessary to provide the Commission with complete and reliable challenge data that accurately reflect on-the-ground conditions in the challenged area and provide the additional context necessary to efficiently and fully adjudicate challenges and thereby assure that more accurate and reliable coverage maps are made available. These data metrics are also substantially similar to those allowed to use their own hardware or software to conduct speed tests. The purpose of collecting either type of data is to allow for the challenged provider to identify characteristics of the device or service plan used to conduct the test, such as whether the device was roaming or was subjected to slower service due to the subscriber’s data plan.

See Verizon Comments at 11. Accordingly, we will not require a service provider to submit either the device IMEI or the combination of source IP address, source port, and timestamp when submitting speed tests (either in response to a challenge or in response to a verification inquiry), as these fields are relevant only for data submitted by challengers.

73 Concurrent with release of this Order, we are publishing the full technical and data specifications for mobile speed test data. Broadband Data Task Force and Office of Economics and Analytics Publish Additional Data Specifications for the Submission of Mobile Speed Test and Infrastructure Data into the Broadband Data Collection, Public Notice, DA 22-242 (BDTF/OEA 2022). The specification for speed test data includes additional fields derived from the high-level metrics defined herein, as well as other identifiers to facilitate management of the submission of such data. These fields include: a unique device installation ID; a unique test ID; the device Type Allocation Code (TAC); the Mobile Country Code (MCC) and Mobile Network Code (MNC) values measured from the network and from the device’s SIM card; flags indicating whether the network is connected, is available, and/or is roaming; total bytes transferred and calculated bytes per second for download and upload tests; jitter and packets sent and received for latency tests; for each connected cell, the measured cell ID, Physical Cell Identity (PCI), cell connection status, Received Signal Strength Indication (RSSI), Reference Signal Received Power (RSRP), Reference Signal Received Quality (RSRQ), Signal to Interference and Noise Ratio (SINR), Channel Quality Indicator (CQI), spectrum band and bandwidth, and Absolute Radio-Frequency Channel Number (ARFCN); and the horizontal accuracy of GPS coordinates and speed accuracy of measured velocity for each location measurement.

74 47 CFR § 1.7006(f); Third Order, 36 FCC Rcd at 1172, para. 117. The Commission delegated authority to the Bureau and Offices to adopt additional testing requirements for government and third-party challengers. Third Order, 36 FCC Rcd at 1172, para. 118. We therefore add certain metrics to those listed in paragraph 117 of the Third Order and § 1.7006(f) of the Commission’s rules and make clear that all challengers must collect these metrics, with the exception that consumers need not indicate whether an in-vehicle mobile test was conducted with the antenna outside of the vehicle.

75 Vermont DPS Comments at 10 (arguing that these data metrics will require unreasonable processing to produce and record an exorbitant number of coordinates in any of the proposed app options, which will result in significant extra data and exponential possibility for data errors and will be administratively burdensome on government entities to adapt their apps to include six sets of coordinates for each test sequence); Enablers Comments at 6 (arguing that the testing parameters “amount to an exceedingly high burden of proof for consumers and other parties challenging mobile operator broadband coverage maps” and “[a]s a result, contrary to the Broadband DATA Act and its own policy goals, the Commission is at risk of overseeing a largely ineffective and meaningless mobile broadband mapping challenge process.”); Public Knowledge/New America Reply at 3 (agreeing with Enablers).

adopted by the Commission in the Third Order,76 and therefore we do not anticipate that they will create any new burdens on consumers or governmental entities and third parties beyond those in place resulting from the previously adopted requirements. Further, the challenge process will remain user-friendly because any challenger can use a readily downloadable mobile app to collect and submit data (including the FCC Speed Test app, which the FCC makes available for download at no cost), and government and third-party entities have the flexibility also to use their own software or hardware. Therefore, government and other third parties will only need to modify their software once, to the extent necessary to conform to the required testing parameters and metrics we discuss above (and subject to our adopting any new metrics in the future). The Commission will also provide technical assistance to consumers and state, local, and Tribal governmental entities with respect to the challenge process,77 which will be a resource for government entities that do not understand some of our data collection requirements. The Bureau and Offices will ensure that the FCC Speed Test app and other apps approved for use in the challenge process collect this information, and government and other third-party challengers will be able to submit challenge data to the Commission through such apps under the procedures adopted for consumer challenges.

20. We understand that certain technical network information and RF metrics that we would otherwise require are not currently available on Apple iOS devices.78 Therefore, until such time as such information and metrics are available on iOS devices, and the Bureau and Offices indicate they will collect such information from iOS devices, government and third-party entity challenges must use a device that is able to interface with drive test software and/or runs the Android operating system.79 To ensure that the challenge process remains user-friendly and encourage public participation, including by consumers who use a device running the iOS operating system, however, we will not extend this restriction to challenges submitted by consumers, and we will still consider speed test data submitted using an iOS device towards challenges.80 Although we may receive limited data from tests run on iOS devices, we do not anticipate that such tests will significantly impede the creation of challenges because, as mentioned, the Commission will aggregate speed tests to create cognizable challenges. iOS speed tests will be considered in combination with other speed tests that fall within the same resolution 8 hexagon.

We therefore anticipate that data submitted by government and other entities, as well as consumer tests run on Android devices, will help fill in any gaps in information about the on-the-ground quality and

76 See 47 CFR § 1.7006(e)(1)-(2), (f); Third Order, 36 FCC Rcd at 1166-67, 1172, paras. 103, 117.

77 Third Order, 36 FCC Rcd at 1185-86, para. 155 (directing OEA and the Consumer and Governmental Affairs Bureau to make detailed webinars available to explain the challenge process and make available the names and contact information of Commission staff who are available to assist consumers, state, local, and Tribal governments with the challenge process); see also 47 U.S.C. § 644(e) (requiring the Commission to “provide technical assistance to consumers and state, local, and Tribal governmental entities with respect to the challenge process . . ., which shall include . . . detailed tutorials and webinars [and] the provision of staff of the Commission to provide assistance, as needed, throughout the entirety of the challenge process”).

78 The information we will use in the challenge process that can be collected from Android devices, but not iOS devices, includes the signal strength, signal quality, unique identifier, and other RF metrics of each serving cell, as well as the spectrum bands used for the test and other network characteristics (e.g., whether the device was roaming, as well as the identity of the provider for the connected network).

79 See BDC Mobile Technical Requirements Public Notice at *6, para. 14. The iOS operating system, which supports iPhone and iPad hardware devices, does not disclose certain technical network information and RF metrics that are essential to the Commission’s challenge and crowdsource processes. This limits the conclusions that we can draw from on-the-ground tests conducted using such devices. OET will update its guidance if future iOS software versions are released that disclose this technical network information and/or RF metrics.

80 Although iOS software does not report the complete metrics we require in this Order (e.g., certain technical network information and RF metrics), the Bureau and Offices will nevertheless use the remaining on-the-ground data we receive from consumers using iOS software in the challenge process.

availability of broadband coverage that…