Performance Measures Testing
The FCC has adopted a new compliance framework to ensure that broadband service subsidized by the Connect America Fund (CAF) meets basic speed and latency performance standards. Starting in 2020 with CAF Phase II Model participants, carriers that receive CAF support to provide fixed-location broadband service will be required to conduct speed and latency testing of their networks and submit the results to USAC as part of the annual compliance process. As established by FCC orders in 2018 and 2019, the performance testing aims to maximize the impact of CAF investments to close the digital divide in rural America and ensure that people living in rural communities have access to the same high-quality networks as those living in urban areas.
The performance measures testing framework will require carriers to conduct one week of speed and latency testing for a random sample of CAF-supported broadband subscriber locations selected by USAC in each quarter of the calendar year. Testing mandates will apply to carriers participating in the following funds:
- Connect America Fund (CAF) Phase II Model
- Connect America Fund (CAF) Phase II Auction, including participants in the New NY Broadband Program
- Alternative Connect America Cost Model (ACAM), Revised ACAM and ACAM II
- CAF-Broadband Loop Support (CAF BLS)
- Rural Broadband Experiments (RBE)
- Alaska Plan
The period to start testing varies by fund, with CAF Phase II Model carriers set to begin testing in the second half of 2020.
Carriers must test speed and latency from the premises of active subscribers to a remote test server located at, or reached by passing through, an FCC-designated Internet exchange point (IXP), which is any building, facility or location housing a public Internet gateway that has an active interface to a qualifying Internet Autonomous System (ASN). Carriers serving areas greater than 500 air miles from an FCC-designated IXP may conduct all required speed and latency testing between the customer premises and the point at which traffic is aggregated for transport to the continental U.S. Carriers must conduct testing between 6 p.m. and 12 a.m. local time.
A speed test is a single measurement of download or upload speed of 10 to 15 seconds duration between a specific consumer location and specific remote server location that meets the FCC designated IXP requirements. Speed requirements vary by fund and the minimum speed requirement is 10 Mbps downstream and 1 Mbps upstream (10/1 Mbps). Carriers must conduct at least one download test and one upload test during each testing hour at each testing location. (See FCC 19-104 at paras. 24-26)
A carrier may report that no test was successfully completed due to “crosstalk” caused by consumer activity if the consumer traffic meets thresholds of 64 Kbps for download tests or 32 Kbps for upload tests, and if the carrier begins attempting speed tests within the first 15 minutes of a testing hour and repeatedly retries and defers tests at one-minute intervals. (See FCC 19-104 at para. 26)
A latency test is a single measurement of latency, often performed using a single User Datagram Protocol (UDP) packet or a group of three Internet Control Message Protocol (ICMP) or UDP packets sent at essentially the same time. Carriers must offer broadband service with latency suitable for real-time applications, including voice over IP (VoIP) calling. Carriers must conduct one discrete latency measurement or observation per minute (60 per hour) for each testing hour at each subscriber test location. (See FCC 19-104 at paras. 27-38)
A carrier may postpone a latency test because of crosstalk if the consumer traffic load exceeds 64Kbps downstream.
Deployment obligations for each fund will determine when carriers participating in that fund must begin network performance testing. Test results will be due by the following July for testing conducted during the prior calendar year.
Before the start of official testing, carriers will be subject to a “pre-testing” period for one week of each quarter of the calendar year. During pre-testing, carriers must conduct performance measures testing at a random sample of locations selected by USAC and submit the results to USAC within one week of the end of the quarter, but will not face withholding of support for failing to meet speed and latency standards.
Schedule for Pre-Testing and Testing
|Program||Pre-testing Start Date||Pre-testing Results Due||Testing Start Date||Test Results Due|
|CAF Phase II||January 1, 2020||First week of 2nd and 3rd Qtr 2020||July 1, 2020||July 2021|
|Rural Broadband Experiments||January 1, 2021||Last week of end of each Qtr 2021||January 1, 2022||July 2023|
|A-CAM I||January 1, 2021||Last week of end of each Qtr 2021||January 1, 2022||July 2023|
|A-CAM I Revised||January 1, 2021||Last week of end of each Qtr 2021||January 1 2022||July 2023|
|ACAM II||January 1, 2022||Last week of end of each Qtr 2022||January 1, 2023||July 2024|
|Legacy Rate of Return||January 1, 2022||Last week of end of each Qtr 2022||January 1, 2023||July 2024|
|CAF II Auction||January 1, 2022||Last week of end of each Qtr 2022||January 1, 2023||July 2024|
|NY Broadband Program||January 1, 2022||Last week of end of each Qtr 2022||January 1, 2023||July 2024|
How Testing Works
USAC is implementing the performance measures testing framework using a new system called the Performance Measures Module (PMM), which will launch soon. The PMM will let carriers: (1) identify locations that were certified as deployed with CAF support; (2) select a random sample of active subscribers from those locations for speed and latency testing; and (3) collect the speed and latency test results.
In order to identify locations with active subscribers for testing purposes, beginning with the testing start dates and biannually thereafter, carriers must download certified locations that they have already filed in the High Cost Universal Broadband Portal (HUBB) using a comma separated variable (CSV) file and add a carrier-generated, alpha-numeric subscriber ID that serves as a unique identifier for locations with active subscribers. Carriers must then upload this new file into the PMM system, which will select a random sample of those locations for speed and latency testing.
Every two yeas, USAC will randomly select locations for testing from among locations with active subscribers in each speed tier. Carriers must test up to 50 locations for each speed tier per state, depending on the number of subscribers a carrier has in a state per funded speed tier. Carriers should use the same subscriber locations for both speed and latency testing and must obtain a new random sample after two years of testing/pre-testing.
Required Test Locations
|Number of Subscribers at CAF-Supported Locations per State and Service Tier Combination||Number of Test Locations|
|50 or fewer||5|
|51-500||10 percent of total subscribers|
Carriers that serve 50 or fewer subscribers in a state and particular service tier and cannot find five active subscribers for the required testing will be subject to verification that more subscribers are not available.
Carriers can choose from among three options for speed and latency testing:
- A carrier may leverage existing Measuring Broadband America (MBA) testing infrastructure by using entities that manage and perform testing for the FCC MBA program to conduct network performance testing. The carrier is responsible for all costs required to implement network testing.
- A carrier may use existing network management systems and tools, ping tests and other commonly available performance measurement and network management tools to implement performance testing.
- A carrier may develop its own self-testing configuration using software installed on subscriber gateways or in equipment attached to subscriber gateways to conduct speed and latency tests.
Carriers will submit speed and latency test results for each state and speed tier combination to USAC on an annual basis every July by uploading test results from the previous calendar year in the PMM using CSV files. Carriers will use one CSV file for speed test results and one CSV file for latency test results.
Carriers that receive CAF support to deploy broadband may face withholding of support, after the pre-test period, for failure to meet performance measures standards. The FCC will consider failure to meet speed and latency requirements as a failure to deploy.
In order to demonstrate compliance with the performance measures standards, at least 80 percent of network speed measurements must be at 80 percent of required speeds and 95 percent of latency measurements must be at or below 100 milliseconds round-trip time.
USAC determines carrier compliance with applicable performance standards based on the certified test results and may withhold support to carriers in the event of non-compliance. Results from speed and latency testing are subject to USAC verification and audit.