Verizon’s mobile network has once again been named the best in the US by testing firm RootMetrics, and T-Mobile USA finished last among the four major wireless carriers.
While Verizon Wireless bragged about its victory, T-Mobile claimed that the results are meaningless and that its network is the fastest in the US. The RootMetrics reports have been an ongoing problem for T-Mobile, which has repeatedly claimed that the results shouldn’t be trusted.
RootMetrics releases its reports every six months, based on drive tests conducted throughout the country a few months previously. T-Mobile used to claim that the tests are outdated, but that argument has gotten harder to make as T-Mobile keeps losing to Verizon and AT&T in the tests. In March 2014, T-Mobile said it had been the network leader for “months” and that the RootMetrics data was outdated. Six months later, T-Mobile lost again and predicted that it would “win in their studies in the future as [RootMetrics] data catches up to where our network performance is today.” In August 2015, T-Mobile CEO John Legere slammed the RootMetrics reports as “bullshit” and “antiquated.”
The latest report released today covers the second half of 2016, and T-Mobile still hasn’t won. Verizon scored a 93.9 on a 100-point scale in the overall national rating, followed by AT&T at 90.5, Sprint at 84.7, and T-Mobile at 81.2. Verizon won in all categories, including reliability, speed, data performance, call performance, and text performance. T-Mobile finished third in speed and data performance ahead of Sprint, but ranked last in the other categories. The numbers were barely changed from the previous report covering the first half of 2016.
T-Mobile: Testers are just “paid consultants”
Verizon has routinely used RootMetrics data in promotional material, which likely requires making payments to the testing firm. T-Mobile criticized the results as coming from “paid consultants.” But that doesn’t necessarily make the data suspect. If T-Mobile won the tests, it could pay RootMetrics to tout its own results.
RootMetrics, which is owned by analytics firm IHS Markit, says it performed 3.69 million tests while driving nearly 250,000 miles and visiting 4,283 indoor locations. The testing firm claims its tests cover 100 percent of the US population.
“All RootMetrics testing is conducted from the consumer’s point of view,” the report said. “We use off-the-shelf smartphones purchased from operator stores, and tests are conducted during the day and night while walking and driving. We utilize random sampling techniques to ensure our results offer a robust characterization of performance in the places consumers most often use their smartphones, and all testing is focused on the activities for which consumers typically use their smartphones, including data, call, and text usage.”
But T-Mobile claims that crowdsourced data is better and points to reports by companies such as OpenSignal, whose reports rely on users to download an app and perform a test. The latest OpenSignal results show that T-Mobile is “neck and neck” with Verizon in speed and catching up in LTE availability. Like RootMetrics, OpenSignal doesn’t allow promotional use of its test results “without prior written consent,” so T-Mobile is likely paying for the right to boast about its OpenSignal results (as it did earlier this month). We wrote about previous OpenSignal results in February 2016.
Yesterday, T-Mobile CTO Neville Ray wrote a blog post that said, “Crowd-sourced data is from real people on their real devices. There are more than 500 different device types on our network alone! Guess how many phones Verizon’s paid consultants usually test. One. That’s right. Just one, and it’s not even the most popular phone in the country. #SMH.”
“Verizon will throw millions more into promoting their study on TV, on radio, in your social feeds and everywhere else you go,” Ray also wrote. But crowdsourced tests are “independent” and provide “info on how people actually use their devices, not a consultant’s interpretation of actual usage.” The tests show that the T-Mobile network “reach[es] 99 percent as many people as Verizon with a faster LTE network,” he wrote.
The T-Mobile blog post did not repeat T-Mobile’s former claim that the RootMetrics data is too old and that the carrier will win in future reports. The RootMetrics report said that while T-Mobile’s network has improved, so have those of other carriers, and T-Mobile’s overall ranking hasn’t changed.
“T-Mobile’s United States RootScore rankings in second-half 2016 testing remained identical to those from the first half of 2016,” RootMetrics wrote. “Indeed, T-Mobile’s rankings in all six test categories have remained the same in each of our last four test periods, dating back to the first half of 2015.”
T-Mobile performs well in metro areas
Once again, T-Mobile performed much better in metro areas than in the overall national rankings and in individual state rankings, RootMetrics said. T-Mobile finished first or tied for first in 25 out of the 125 most populous rural areas; Verizon won or tied in 116, AT&T in 55, and Sprint in nine. But when measuring entire states, T-Mobile and Sprint did not win or tie for first in any of the 50 states.
The rural measurements apparently helped push T-Mobile’s overall ranking down below its rivals. That might explain why RootMetrics and OpenSignal report such different results; crowdsourced tests depend on individual consumers to test themselves and may not measure rural performance as extensively as drive tests.
The RootMetrics results could eventually change, however.
“As AT&T, Sprint, and T-Mobile continue to expand their LTE footprints beyond metropolitan markets, Verizon could face stronger competition in our testing of the US moving forward,” RootMetrics said.
But as long as Verizon keeps its lead, it will keep touting the results. Verizon issued a press release this morning boasting about its seventh consecutive win in the RootMetrics report.
“Verizon repeated its industry leadership from the first half of 2016 into the second half of the year, with wins or first place ties in 48 of 50 states in overall network performance,” Verizon said. “The two discount networks [Sprint and T-Mobile] did not win or tie a single state, contributing to their total of zero wins in overall network performance since state RootScore Report studies began in the second half of 2013.”
RootMetrics also issued reports on state-level results and metro results. You can dig further into results for each state, metro area, and airports with the RootMetrics map.