The Metrics Vacuum

A few weeks ago I met up with a Grand Media Fromage to chat over the state of the social web.  "So where's the money?'" he began.  "Compared with what?" I replied defaulting to my standard line.  However, Mr GMF then took an unexpected tack by saying he thought Social might be at a similar stage to Search (aka Google) when it was 'pre-revenue' (aka skint).  And that it could easily go the same way as the Big G, snaffling many media executives' luncheons along the way.  This was an observation that I've been mulling over ever since.  I've long held the view that we are measuring tomorrow's world with yesterday's tools.  It seems we are now waiting for a social operator (aka Facebook) to find a market mechanism that can unleash the vast latent demand within the media industry for techniques in keeping with modern customers' P2P lifestyles.  Nick Gonzalez gets it just right for me: "There’s currently a metrics vacuum in the social media space and it’s being filled with traditional metrics pulled from search and display. The only problem is that these metrics don’t make sense for brand campaigns in social media."  Which essentially makes it impossible for brands or anyone else to invest at scale.  Gozalez goes onto say: "Branding campaigns don’t have a conversion event, like search advertising. Click through rates and impression volume used to gauge display campaigns are radically different on social media properties. Blogs average a little over one page view per visit, while social networking properties average in the tens or hundreds. It’s not apples to apples. While impressions do adjust behaviour, clicks and conversions also carry a whole different meaning within social media. People click to send messages, make friends, comment, and share."  Of course, the answer could come from anywhere but Facebook's vastness means it bears the weight of expectation.  Which it has tried to meet through Beacon and more recent innovations.  Maybe Zuckerburg & Co. will just follow Google's lead and keep an eye out for smaller entrepreneurs and *ahem* take their lead.  Either way the stage is set for a new marketing sliderule and it will be fascinating to see if and how one develops.  But in the meantime the vacuum grows...

Comments

[eaon]

'being filled with traditional metrics pulled from search and display.'
thats the problem for sure. I'm struggling with this every day - engagement metrics that clients can buy. its a conundrum..

[James Cherkoff]

Thanks Eaon, yes it's the $64 gazillion question isn't it. A 'bespoke' approach is the only one that works right now. That said, the 'vacuum' forces everyone to look at the business issues and apply the social tools to them, rather than falling back on a lazy preset metrics system. Which may in fact be a more intelligent, productive approach.

[Rory MacDonald]

"Ahhh, metrics. Give me more metrics." James, you have been hanging around with too many fromages.

I fear return on marketing spend may be the best metric you get. And that never works because you can never really view the effect of one programme in isolation. Anything else is going to be so contrived as to be meaningless.

With social media we are talking about people interacting with people, most efforts to put metric on this (in whatever discipline) end up as little more that pseudo science.

Do you think Nike are worried about the metrics on Nike Plus?

[James Cherkoff]

Well said Rory. Here's Nike's view of their mega-community: http://tinyurl.com/dxml6e

The company basically says they've reached out to a whole new market and their metric is sales. Imagine that. Community and other social media being measured by its ability to shift products!

The example shows that the metrics discussion may have to take on a whole new view of the world. One that is more about actions than impressions, views or clicks.

[Charles Frith]

Agreed James. Although I'm pretty sure there's some equity in talking about the different engagement experiences. Tricky to describe and is a post long overdue because I haven't quite reached my conclusions but let's just say that when I'm in reading Newspaper online mode, it's a completely different engagement to social media. Where it gets interesting is the transition of content to interaction.

This probably doesn't make sense but I'm qualitatively interested in the different types of engagement. Even if we just colour code it and develop a revenue model that pays for the fun that way. Just my thoughts James.

Great post.

[James Cherkoff]

And your thoughts are always very much appreciated Charles.

I like the idea of 'different engagement experiences' - very close to something I am working on at the moment.

Look forward to your post... ;-)

[Kevin]

I wonder, compare offline 'media engagement experiences' to online... an ad in the FT carries the FT 'frame' around it and communicates sthg qualitatively different from the same ad in a colour supplement magazine (project Fugitive a dozen yonks ago, could elucidate ...zzz) .. likewise, the 'mode' of reading eg Guardian online is different to the mode of reading FB and I reckon I relate to ads on G (ho hum) differently to FB ads (piss off)

[James Cherkoff]

Hi Kevin, thanks for stopping by. There's something about the 'reading' mode and the 'sharing' mode that seems to me to be at the heart of it. And the effects that result. How could we go about exploring that?

[Nick Buckley]

Hi James - this is a crucial topic and I feel phrases like 'transform the whole of marketing' coming on... but I'll try to desist. This question is like a little fulcrum, posing as a question about the applicability or replacement of 'old' metrics, which ends up supplying the leverage to make the whole marketing paradigm at least wobble (I hope!)

The pursuit of metrics confirms a 'cause-and-effect' model and some notion that impact (on the perceiver) and subsequent action or change in attitude (by the perceiver) are quantum. I don't just mean quantifiable I mean quantum - in the sense of moving between discrete states or degrees (suspiciously similar to moving from one point to another on a market researcher's five point scale). So marketing is seen as action at a distance, implanting behavioural quanta in people, or disposing them to re-transmit same. The end of that chain is, of course, the action of purchase/choice - even if deferred in the form of a shift in brand loyalty etc.

Ergo you want metrics in order to tell whether a particular combination of medium and strategy efficiently a) propagates those little 'causes' to your audience and b) effects enough quantum shifts on arrival.

However you dress that up, or use concepts such as 'affinity' as a buffer store, this is at bottom a Skinnerian, behaviourist approach - and it presumes a publish/transmit model. Even conversations will tend to be measured as re-transmissions or helpful free rides for "THE MESSAGE" or at least "THE CAUSE".

[See where I'm going?] If you have my kind of beliefs about/aspirations for social media, and my preferred way of looking at 'consumers', then the entire medium is antithetical to the model I just described. Is the Seth Godin perspective, you no longer control the message (or you soon won't anyway), compatible in any way with metrics about causal behavioural quanta?

So where does that leave us? I guess it makes us feel better about the fact that this seems hard. But as a first shot I suggest three things:

1. Sales are a pretty good metric. We have to get even more used to the fact that they are the result of multiple variables and then add the idea that social media not only add new ones, they differentially amplify some of the old ones.

2. Get used to making subjective judgements, 'reading' [literally and metaphorically] what's going on out there, rather than leaning so heavily on the comfort of objective empirical data [which in any case we've fretted about the provenance of for as long as we've been collecting it]. Part of that subjectivity, much amplified by the ubiquity of social media and its direspect for corporate boundaries, is about accepting that we are ourselves within a feedback loop - think Hendrix listening for the behaviour of his guitar/amp and drawing on past experience, as opposed to someone at a mixing table analysing digital readings... or even waveforms.

3. if that leaves you with itchy fingers in the research department - I would advocate exploration of the 'engagement experiences' suggested in a couple of places above... I concept I found really helpful. Trying to identify different types of engagement, different types of participant [rather than trying to flood the environment with universal 'causes'] is probably a good way of enhancing your judgement about what works... be that in artifically created market research environments, or by harvesting spontaneously volunteered testimonies within social media itself. I'd use both - and be aware of the strengths and weaknesses of both, and of the methods of analysis applied to them. I think we're mostly talking Qual ! At least for now.

As an idealist I can't resist ruining all the above by switching to soapbox mode ;0) We should stop trying to measure our ability to control and transmit the message - since the ultimate implication of this is that the message has to depart from 'the truth' about our products and services, in order to compensate for the gap between 'the truth' and what we desire people to think and do. [OK - I know there are infinitely more complex versions, e.g. where we have to compensate for the actions of the 'bad guys' who are less principled than us, and so on...] Instead - assuming that social media, and related developments, create ever more transparency and scrutiny of our claims and our performance we should; be open and honest, make good things, provide good services, and *listen* as if our livelihoods depended upon it. [Which they do].

Sorry - long comment... I've broached just about every norm there is about commenting on a blog. But it's your fault James! You made me do it by asking *the* trigger question. I beg your indulgence

[James Cherkoff]

Many thanks Nick, that's really great and plenty of food for thought....

[Charles]

Here it is James :)

http://bit.ly/HhP44

[James Cherkoff]

Thanks Charles, as Blackadder would say media metrics in today's social world is, 'a riddle wrapped in a mystery inside an enigma...'