Performance Measures Testing
The FCC performance measures testing framework confirms that broadband service subsidized by the Connect America Fund (CAF) meets basic speed and latency performance standards.
Under this new framework, carriers that receive CAF support to provide fixed-location broadband service must conduct speed and latency testing of their networks and submit the results to USAC as part of the annual compliance process. Established by FCC orders in 2018 and 2019, these requirements aim to maximize the impact of CAF investments to close the digital divide in rural America and ensure that people living in rural communities have access to the same high-quality networks as those living in urban areas.
The FCC mandates that at least 80 percent of network speed measurements be at 80 percent of required speeds and 95 percent of latency measurements be at or below 100 milliseconds round-trip time. Carriers that fail to meet the performance measures standards required of their funds may face potential withholding of support.
The performance measures testing framework requires carriers to conduct one week of speed and latency testing using a random sample of CAF-supported broadband locations with active subscribers selected by USAC in each quarter of the calendar year. Testing mandates apply to carriers participating in the following funds:
- Connect America Fund (CAF) Phase II Model
- Connect America Fund (CAF) Phase II Auction, including participants in the New NY Broadband Program
- Alternative Connect America Cost Model (ACAM), Revised ACAM and ACAM II
- CAF-Broadband Loop Support (CAF BLS)
- Rural Broadband Experiments (RBE)
- Alaska Plan
- Rural Digital Opportunity Fund (RDOF)
The windows to start testing vary by fund and began in 2020 with CAF Phase II Model carriers. Testing continues throughout a carrier’s support term. Carriers that participate in multiple funds must meet the separate testing requirements of each of those funds.
Carriers must test speed and latency from the premises of active subscribers to a remote test server located at, or reached by passing through, an FCC-designated Internet exchange point (IXP), which is any building, facility or location housing a public Internet gateway that has an active interface to a qualifying Internet Autonomous System (ASN). More information about acceptable test paths and remote server locations is available here. Carriers serving areas greater than 500 air miles from an FCC-designated IXP may conduct all required speed and latency testing between the customer premises and the point at which traffic is aggregated for transport to the continental U.S.
A speed test is a single measurement of download or upload speed of 10 to 15 seconds duration between a specific consumer location and specific remote server location that meets the FCC designated IXP requirements. Speed requirements vary by fund. Carriers must conduct at least one download test and one upload test during each testing hour at each testing location. (See FCC 19-104 at paras. 24-26)
A carrier may report that no test was successfully completed due to “crosstalk” caused by consumer activity if the consumer traffic meets thresholds of 64 Kbps for download tests or 32 Kbps for upload tests, and if the carrier begins attempting speed tests within the first 15 minutes of a testing hour and repeatedly retries and defers tests at one-minute intervals. (See FCC 19-104 at para. 26)
A latency test is a single measurement of latency, often performed using a single User Datagram Protocol (UDP) packet or a group of three Internet Control Message Protocol (ICMP) or UDP packets sent at essentially the same time. Carriers must offer broadband service with latency suitable for real-time applications, including voice over IP (VoIP) calling. Carriers must conduct one discrete latency measurement or observation per minute (60 per hour) for each testing hour at each subscriber test location. (See FCC 19-104 at paras. 27-38)
A carrier may postpone a latency test because of crosstalk if the consumer traffic load exceeds 64Kbps downstream.
Carriers can choose from among three options for speed and latency testing:
- A carrier may leverage existing Measuring Broadband America (MBA) testing infrastructure by using entities that manage and perform testing for the FCC MBA program to conduct network performance testing. The carrier is responsible for all costs required to implement network testing.
- A carrier may use existing network management systems and tools, ping tests and other commonly available performance measurement and network management tools to implement performance testing. This is a list of vendors that are providing services to assist carriers in conducting speed and latency testing and reporting test results to USAC.
- A carrier may develop its own self-testing configuration using software installed on subscriber gateways or in equipment attached to subscriber gateways to conduct speed and latency tests.
Deployment obligations for each fund determine when carriers participating in that fund must begin network performance testing. Test results are due by the following July for testing conducted during the prior calendar year. USAC encourages carriers to file and certify test results on a quarterly basis after that quarter’s testing is complete.
Before the start of official testing, carriers are subject to a “pre-testing” period for one week of each quarter of the calendar year. During pre-testing, carriers must conduct performance measures testing at a random sample of subscriber locations selected by USAC and submit and certify the results within one week of the end of the quarter. Carriers will not face withholding of support for failing to meet speed and latency standards during pre-testing as long as they submit all test results for the full sample (unless subject to FCC waiver stating otherwise) on a quarterly basis. USAC encourages carriers to file and certify results as soon as pre-testing is complete each quarter.
Schedule for Pre-Testing and Testing
|Program||Pre-testing Start Date||Pre-testing Results Due||Testing Start Date||Test Results Due|
|CAF Phase II||January 1, 2020||Following last week of end of each Qtr 2020||January 1, 2021||July 2022|
|Rural Broadband Experiments||January 1, 2021||Following last week of end of Qtr 2021||January 1, 2022||July 2023|
|Alaska Plan (carriers required to file in the HUBB)||January 1, 2021||Following last week of end of Qtr 2021||January 1, 2022||July 2023|
|A-CAM I||January 1, 2021||Following last week of end of Qtr 2021||January 1, 2022||July 2023|
|A-CAM I Revised||January 1, 2021||Following last week of end of Qtr 2021||January 1 2022||July 2023|
|ACAM II||January 1, 2022||Following last week of end of Qtr 2022||January 1, 2023||July 2024|
|Legacy Rate of Return (CAF BLS)||January 1, 2022||Following last week of end of Qtr 2022||January 1, 2023||July 2024|
|CAF II Auction and
NY Broadband Program
|January 1, 2022||Following last week of end of Qtr 2022||January 1, 2023||July 2024|
For carriers impacted by Hurricane Ian, the FCC has issued an order extending the deadline to report results from third quarter pre-testing of CAF-supported networks in “affected disaster areas” to Jan. 7, 2023, and waiving fourth-quarter pre-testing and official testing requirements for these networks altogether. The FCC defines “affected disaster areas” as areas in Florida, South Carolina and North Carolina that the Federal Emergency Management Agency (FEMA) has designated as eligible for federal disaster relief assistance. The order also states that because Hurricane Ian made landfall at the end of September, the FCC will provide “appropriate relief” for carriers that had pre-testing or official testing of networks in affected areas scheduled for the final week of the quarter.
How Testing Works
USAC has implemented the performance measures testing framework using a new system called the Performance Measures Module (PMM). The PMM: (1) lets carriers identify locations deployed with CAF support and reported in the High Cost Universal Broadband Portal (HUBB) that have active subscribers; (2) generates a random sample of those locations for speed and latency testing and provides the obligated speed tiers to be tested based on the speed tiers reported for those locations in the HUBB compared with the speeds carriers are required to deliver; (3) collects the speed and latency test results from carriers; and (4) calculates compliance with performance measures standards based on certified test results.
In order to identify locations with active subscribers for testing purposes, beginning with the testing start dates and biannually thereafter, carriers must download certified locations that they have already filed in the HUBB using a comma separated variable (CSV) file and add a carrier-generated, alpha-numeric subscriber ID – such as an account number – that serves as a unique identifier for each location with an active subscriber. Carriers should not use any personally identifiable information – such as a customer phone number or social security number – in a subscriber ID. Carriers must then upload this new file into the PMM system, which selects a random sample of those locations for speed and latency testing and provides the obligated speed tiers to be tested.
Please note that for funds with multiple speed tiers, the obligated speed tier to be tested for some locations may not be the same as the speed tier reported for that location in the HUBB. That is because the system uses a cascading logic that may select locations with higher speeds to fill a sample for testing at slower speeds if there are not enough locations with slower speeds reported in the HUBB.
USAC encourages carriers to be sure that their broadband deployment information in the HUBB is as up-to-date, accurate and complete as possible before obtaining their random samples from the PMM. Carriers should first submit any new locations that have not yet been reported to the HUBB and make any necessary edits to latitude and longitude coordinates, address information and speed tier data to reflect network upgrades if they want those updates reflected in their random samples. Note that because some carriers participate in funds that start pre-testing before the fund’s first deployment milestone, there may be some carriers that are supposed to begin pre-testing but have no locations yet in the HUBB. These carriers are not required to conduct pre-testing until they have deployment data in the HUBB, but should plan to begin pre-testing within one quarter after they do.
Carriers must test up to 50 locations for each speed tier they are required to deploy to in each state where they receive support (with sample sizes determined based on the number of active subscribers submitted to the PMM), must conduct testing at all selected locations and must meet separate testing requirements for each fund in which they participate. Carriers must obtain a new random sample after two years, and cannot delete HUBB records or edit or modify the number of units for HUBB records for subscriber locations that have been randomly selected for speed and latency testing during the two years when those locations are part of the testing sample.
Carriers can only request a random sample one time, and cannot alter uploaded subscriber location data or request a new sample once a sample has been generated.
Required Test Locations
|Number of Subscribers at CAF-Supported Locations per State and Service Tier Combination||Number of Test Locations|
|50 or fewer||5|
|51-500||10 percent of total subscribers|
Carriers should use the same subscriber locations selected for both speed and latency testing, and must provide subscribers at these locations with the modem, router or other customer premises equipment needed to conduct testing at no extra cost to the customer. Consumers can find more information about the FCC performance testing program here. The FCC does not require carriers to inform subscribers at locations selected for speed and latency testing because carriers already conduct routine performance measures testing of their networks.
All speed tests for the sample must occur in the same week and all latency tests for the sample must occur in the same week, but speed and latency tests may take place in different weeks. Carriers must conduct testing between 6 p.m. and 12 a.m. local time.
Carriers can request a replacement location if the subscriber at a location selected for testing refuses to participate or allow installation of testing equipment, drops service or subscribes to a slower speed tier than the one being tested – leaving the carrier with fewer testing locations than required. Carriers should also request a replacement location if the subscriber at a location selected for testing does not use the service for a month or more but continues to subscribe (this would be considered “dropped service” in the PMM), or if the subscriber accepted but does not use or stopped using the modem provided for testing (this would be considered “subscriber demands removal of testing equipment” in the PMM).
If a carrier runs out of replacement locations with active subscribers at the required speed tiers, the carrier would need to upgrade service at the locations selected by USAC in order to conduct testing. Carriers that serve 50 or fewer subscribers in a state and particular service tier and cannot find five active subscribers for the required testing will be subject to verification that more subscribers are not available.
USAC developed this quick tips guide to the Performance Measures Module (PMM) to address frequently asked questions about how the system works and help troubleshoot common issues users may encounter.
The FCC issued a limited waiver of sample size testing requirements for Original and Revised ACAM, RBE and Alaska Plan carriers, and sample size pre-testing requirements for ACAM II, CAF II Auction and CAF BLS carriers, in the first two quarters of 2022 in response to difficulties that carriers encountered obtaining necessary customer premises equipment due to the global semiconductor shortage. The FCC allowed these carriers to conduct testing or pre-testing at only 70 percent of the USAC-generated random sample of subscriber locations in the first two quarters of 2022.
During official testing, carriers must submit and certify test results from the previous calendar year for each state and speed tier combination by an annual July filing deadline. Carriers upload results in the PMM using one CSV file for speed test results and one CSV file for latency test results. Carriers must submit all test results for the full sample (unless subject to FCC waiver stating otherwise). Although they are due annually, USAC encourages carriers to file and certify results on a quarterly basis, after each quarter’s testing is complete.
USAC provides quarterly compliance reports to carriers that file and certify test data on a quarterly basis to allow them to track their progress in meeting speed and latency metrics and address any shortfalls before the end of the year. Note that PMM compliance is calculated based only on certified data. Don’t forget to certify.
USAC will not withhold support from carriers that file and certify data on a quarterly basis and fail and to meet speed and latency metrics before evaluating certified data for the full year. USAC only calculates final performance compliance – and withholds support from carriers that fail to meet speed and latency requirements – after carriers submit and certify test data for all four quarters.
During pre-testing, carriers must submit and certify speed and latency test results within one week of the end of the quarter. USAC provides quarterly compliance reports to carriers during pre-testing to allow them to track their progress in meeting speed and latency metrics.
Carriers that receive CAF support to deploy broadband may face withholding of support, after the pre-test period, for failure to meet performance measures standards. The FCC considers failure to meet speed and latency requirements as a failure to deploy.
In order to demonstrate compliance with the performance measures standards, at least 80 percent of network speed measurements must be at 80 percent of required speeds and 95 percent of latency measurements must be at or below 100 milliseconds round-trip time.
USAC calculates carrier compliance with applicable performance standards separately for each state and speed tier – as well as each fund in which a carrier participates – and may withhold support in the event of non-compliance based on FCC guidance. Results from speed and latency testing are subject to USAC verification and audit.
For a detailed explanation of how USAC calculates compliance results, please see PMM Compliance Calculations.