Updated: May 16, 2017
Timeline:
06.09.2017
Facebook is Criticized for Claiming to Reach More People than Census Shows in US
Facebook is being criticized by a financial analyst for showing a substantially larger audience (10-15 million more) for specific age groups in the US in its ad targeting tools than the US census shows actually live in the US. That's obviously not great given the context of Facebook's long-running issues with faulty ad metrics, but it's also not a massive issue. Advertisers typically don't pay based on potential reach but on actual reach, and no advertiser that really wants to reach a broad demographic like this would even pay attention to Facebook's estimated audience size. The bigger issue is why Facebook would be overestimating the numbers by so large a margin, and I suspect the issue is twofold: many people likely have two accounts, and given Facebook's age restrictions, I'm guessing some of those claiming to be in the 18-24 bracket might well be younger in reality, having created accounts when under 13 and thereby artificially inflating those numbers. In short, it's not a great lo
16.05.2017
Facebook Announces Yet Another Measurement Screwup
After a hiatus of several months, Facebook has just announced yet another measurement screwup on its platform, and this is the first to have led to overcharging of advertisers. It only affected one category of ads (video carousel ads) on one of Facebook's platforms (its mobile website), so the total percentage of impressions affected was 0.04%, so this was fairly marginal. If applied to Facebook's ad revenue over the past year, that would come out to around $12 million, so even though it's refunding those advertisers in full, this is hardly material for Facebook financially. More embarrassing is the fact that this is the latest in a long string of similar incidents since September last year, which you can see in the timeline on the Facebook's Bad Metrics narrative. Advertisers are already clamoring for more third party audits of ad metrics, and although Facebook has complied with some of those requests,
10.02.2017
Facebook Agrees to Audit of its Metrics Following Data Controversy – WSJ
This is yet another bit of damage control by Facebook in the wake of its metrics problems in late 2016, and the MRC partnership has been in the works for some time (see the full timeline on the "Facebook's Bad Metrics" narrative page). It sounds like marketers are reassured by some of these moves, which combine better third party auditing with some new video ad buying options.
via WSJ (Facebook's own post here)
31.01.2017
Facebook Improves Transparency and Deepens Partnerships Around Metrics
The other bit of news from Facebook today addresses the recent problems it's had with unreliable metrics for advertisers and publishers. Some of this is just about providing more metrics for measuring performance on Facebook across various channels (Facebook, Instagram, and Audience Network within its own products can be compared with TV and/or print data from wider campaigns), but there's also news on the third party verification front, which advertisers have been asking for. It now has deeper partnerships with Nielsen and ComScore, and is deepening its viewability measurement tools, as well as adding some additional partnerships. There's lots here, the detail of which won't be all that interesting unless you're directly involved in this stuff, but Facebook is showing some promising willingness to open up more to outside measurement platforms its partners trust as a wa
23.12.2016
MRC in Talks With Facebook About Auditing Its Metrics | Digital – AdAge
Facebook said all the way back in November that it intended to form a measurement council to improve external oversight of its metrics and reporting. This is one of the first concrete signs that it's moving towards better outside auditing, though it's not an announced deal yet.
via MRC in Talks With Facebook About Auditing Its Metrics | Digital - AdAge
16.12.2016
Facebook Discloses Another Metrics Mishap Affecting Publishers – WSJ
The latest incident from Facebook relates to Comscore tracking of iPhone usage, and comes a week after Facebook's last disclosure of errors. All this continues to pile pressure on Facebook to engage more outside auditors in order to regain confidence in its metrics.
via Facebook Discloses Another Metrics Mishap Affecting Publishers - WSJ
09.12.2016
Facebook Says It Has Miscalculated Several More Engagement Metrics – Forbes
This was the third time Facebook had to confess to misstating certain engagement metrics, with at least one bug still unresolved at the time of the announcement, and fixes for the rest coming a week or so later. In November, Facebook announced that it planned to create a measurement council to offer more third-party verification, but those plans aren't concrete yet.
via Facebook Says It Has Miscalculated Several More Engagement Metrics
07.12.2016
How Facebook’s measurement errors have eroded marketers’ trust – Marketing Land
All of Facebook's metrics-tracking mishaps have unsurprisingly made marketers and publishers increasingly wary of trusting its reporting, and in some cases those marketers are reconsidering the money they spend on Facebook. Until now, this just looked like a PR black eye for Facebook, but the potential financial impact is now becoming clearer. This is something Facebook will need to address in its next earnings report.
via How Facebook’s measurement errors have eroded marketers’ trust
30.09.2016
ANA Pushes Facebook for Greater Measurement Transparency – WSJ
Following Facebook's first metrics mishap in September, the Association of National Advertisers issued the first call in what's since become a steady drumbeat of requests for Facebook to open itself up to outside auditing of its metrics for advertisers and publishers.
via ANA Pushes Facebook for Greater Measurement Transparency - WSJ
22.09.2016
Facebook Overestimated Key Video Metric for Two Years – WSJ
This was the first of a number of stories in late 2016 relating to Facebook's metrics for advertisers and publishers. At the time, it looked like an isolated incident, but it was bad enough to attract attention even so – Facebook vastly overestimated average viewing time for videos for two full years, and only disclosed this fact through an article on its Advertiser Help Center. Once the story broke, Facebook publicly apologized.
via Facebook Overestimated Key Video Metric for Two Years - WSJ
This narrative is of fairly recent vintage – it was really only in the second half of 2016 that Facebook started to be plagued by reports of faulty metrics relating to the performance of ads and published content on the site. Two things created this narrative: firstly, four separate incidents have now occurred, and secondly, advertisers and publishers are starting to express concern about a pattern and what it implies about whether Facebook can be trusted on this point. That, in turn, has led to calls for independent auditing of Facebook’s various metrics.
All of this also taps into a related narrative about Facebook’s Power, in that none of this would matter if Facebook weren’t the major filter through which people increasingly see the world and consume news and other content. These publishers and advertisers increasingly have to use Facebook if they want to reach their viewers, readers, and potential customers.
It appears that these issues are the result of a lack of appropriate focus at Facebook on metrics – relative to all the other things that people at Facebook are working on, this one is pretty humdrum. And yet getting it right is critical for Facebook to retain (or regain) the trust of content owners and advertisers as they use the platform. Facebook does now seem to be taking all this much more seriously, and its recent internal review of its metrics was what threw up some of the later findings, which suggests that process is working. But this is an area where Facebook will not now have the trust of its partners by default, and as Facebook’s power grows we’ll continue to see calls for greater transparency and external auditing.