As a former newspaperman, I’m sensitive to complaints that “you missed the story.” But today’s New York Times article on audience measurement discrepancies in interactive advertising did skirt by some of the more important developments in our industry during the past year.
The most significant development: The major audience measurement companies are now working with the IAB to get to the bottom of discrepancies, and we are committed to working together to identify, implement, and educate the marketing and media industries on best and emerging practices in audience measurement.
I’m not a Polyanna – these matters can be very complex, and (let’s face it) disputes between media companies and audience measurement firms are as old as audience measurement itself. But I am confident that, as an industry, we are on our way to reducing measurement discrepancies as an issue.
Discrepancies Matter
I don’t want to dump on The Times story – if you think it’s so easy to summarize a history of the world in 750 words, try it someday. (Then try it five days a week, as some of us had to do in our youth.) But today’s piece did read like a rehash of the complaints and counter-complaints we’ve been hearing and voicing for years: The counts offered by comScore and Nielsen//Netratings differ from publishers’ counts by 30-40-50%… The comScore and Nielsen counts differ from each other… Cogent explanations for the differences can’t be easily had… People at work are undersampled… High-income Web users are undersampled…
For my taste, there was too much emphasis in the piece on at-work populations, and not enough notice of other populations that are chronically undersampled in audience research, notably ethnic and racial minorities and young men. And there was no notice of one of the ways current audience measurement techniques subvert a primary promise of the Web: the ability to match niche buyers and niche sellers through the perfect media vehicle. Hundreds of thousands, perhaps millions, of small media sites – the Mom & Pop grocery stores of the Web – likely don’t show up in samples because of their audience size or traffic patterns, thereby robbing advertisers and agencies of potentially valuable matchmaking knowledge (but building a business case for the ad networks and search engines which do deliver ads to and eyeballs from such sites).
IAB’s Historic Summit
These complaints had been festering in the interactive industry for years, but they had resisted resolution. Why? My guess is that publishers, measurement firms, and interactive agencies were too busy inventing a new civilization to take the time to perfect that civilization’s culture. When you’re spending all your time building mud huts and foraging for food, it’s a bit hard to worry about table manners.
But as the funds flow into interactive media began to increase strikingly, the impact of these discrepancies on publishers’ sales, pricing, and competitiveness was taking an ever-larger toll. The IAB’s management and board determined it was time to act, and to act aggressively. Last April, we issued an open letter to comScore and Nielsen//Netratings. We sought two things: a “summit meeting” at which we could agree together on a process by which we could resolve discrepancies; and agreement by these vitally important companies to undergo third-party audits and accreditation of their processes.
That story has been well-covered elsewhere (although alas, not by The Times) so I won’t repeat it. (For a full history and continued updates, go here on the IAB Web site.) What never got covered were the opening remarks I made in kicking of that historic summit in May. To the assembled comScore and Nielsen executives, I said, “Your companies are among the heroes of the interactive marketing and media ecosystem. Starting years ago, and continuing through some very dark days, you took risks based on strong beliefs about where our industries are heading. Because you took those risks, you have built vital, important companies. You are important because you are excellent. You are excellent because you practice superior science, lead superior operations, and deliver superior service. And your strength is our strength. We thank you for the work you’ve done to build interactive media.”
But, noting the persistence of the measurement discrepancies amid the transformation of interactive media into major components in marketing plans, I added, “This transformation requires us to find new ways to collaborate. Yes, there will always be tensions among buyers, seller, and intermediaries. And yes, let us be fierce competitors within our individual industries for revenues and share. But let us also do what our various industries have long done, to greater or lesser degrees: work together to serve our customers and delight our consumers at the same time.”
Audits Are Happening
The important points are these: comScore and Nielsen DID join the summit meeting, along with representatives of the AAAA, ANA, MPA, OPA, ARF, and executives from numerous agencies, publishers, and marketers. And the two companies DID agree to undergo audits and accreditation supervised by the Media Rating Council. And the two companies ARE now undergoing these audits – indeed, they will increasingly compete against each other on the basis of the quality of their numbers, and of the insights that can be derived from these numbers.
The audience measurement companies, in turn, had their requests of the IAB. They need agreed-upon standards against which they can measure, particularly definitions of unique visitors, page views and time spent. We agreed to intensify our Audience Measurement Committees’ efforts to derive those standards. We also agreed that we would work to understand more concretely the impact of cookie deletion, international traffic, spiders and bots and other potential factors in audience-size discrepancies.
(By the way, one of our summit meeting conclusions appears to have gotten lost in the shuffle. While IAB and the measurement firms disagree on the significance of cookie deletion as a systematic factor in measurement discrepancies, we all agreed that more rapid and frequent reporting on site traffic would certainly minimize whatever significance it has. Thus we concluded that a system of “overnight” ratings, similar to television’s overnights, would buttress the current system that emphasizes average monthly visitors – a somewhat jejune construct, anyway, in the interactive space. I still would like to see movement in this direction.)
We also agreed that IAB would lead a market-education campaign on the frontiers of audience research. I’m happy to report that our launch is only a month away. On November 29, IAB inaugurates its Audience Measurement Leadership Forum. comScore and Nielsen will be presenting – as will Quantcast, M:Metrics, Visible Measures, Millward Brown, CNET, Conde Nast, CNN, and many other thought leaders and practice leaders from the research, media, and marketing industries. (Might I suggest booking soon? There are fewer than 300 spaces, and as you might expect, it’s been filling up fast.)
Quest for Insights
Without minimizing the painful financial impact measurement discrepancies have today on many interactive publishers, in a very real sense audience measurement is a side show. The main tent is called Insights, and in it are three rings: Transparency, Accountability, and Predictability — what’s happening, what did happen, and what will happen. Our goal must be – as we said in our May summit meeting – to achieve “convergent validity.”
“Convergent validity” is a phrase coined by Peter Daboll, Yahoo!’s chief of insights and head of global market research, and a former president of comScore’s Media Metrix division. It emphasizes finding the best data sources for different ends, and bringing them together to create a mosaic of the audience, its needs and desires, and its consumption patterns that is richer and more useful than the thousand snapshots we currently assay separately.
Put another way, server logs and panel-based measurement are far more valuable together than they are apart. Server logs – a census counting technique – may be exact, and they should help agency media planners assemble a crank, but they cannot provide us and our advertisers crucial information we need about audience composition. Panels may never, by definition, be exact, but they can provide textural detail of immense value. Bringing these and other techniques together will get us closer to the real insights our customers want.
This is the path we’re now on. I wish The Times had that story. But who needs The Times when you’ve got the Clog?
