Campaign measurement: How to
In our continuing series of blogs about Social Media Measurement on the occasion of AMEC Measurement Week, today we are looking at campaign measurement and evaluation through social media. Assessing the success of campaigns is probably the most obvious application of social media measurement and insights. On the most basic level, social media allows brand owners to understand the impact of their social media campaigns in real time, through the analytics available on each of the channels they use as well as through many other metrics. However, what most brands seem to lack when analysing campaigns is context or benchmarks. If I was asked to give my opinion on whether a volume of 3,000 mentions for a specific campaign is good or bad, my response would be a number of questions:
- How does it compare with previous campaigns using the same mechanics?
- How much of uplift does this volume represent on our usual volumes?
- How does that compare to our main competitor’s last campaign?
So benchmarks are absolutely key in any evaluation. This is not confined to social media though as traditional research agencies such as Millward Brown, who specialise in the tracking and measurement of advertising effectiveness, have spent years building benchmarking databases to be able to tell their clients whether their campaigns have been successful or not. But the application of social media measurement is not confined to the evaluation of campaigns on its different channels. Social media is also an excellent way of evaluating the impact of campaigns through either on and off-line media. I recently contributed to to the #IPASocialWorks ‘Measuring Not Counting’ project and guest-blogged for NewMR on this topic and shared some best practice around campaign analysis using social media as follows.
- Run a benchmark analysis prior to the campaign. This will be key to measuring any shifts in levels of conversation about the brand, but also existing attitudes and perceptions. This will also be a useful exercise to determine which metrics the campaign will be measured on. Using a 3-month time frame before the campaign is likely to smooth out any spikes driven by other events or campaigns.
- Build an intelligent search query. Using the campaign strapline or title will not be enough to gather relevant content. Use key words which relate to key elements of the campaign e.g. central character and premise but also key words associated with the themes or topic broached. This will ensure that the range of content gathered is in consumers’ own words.
- Apply sampling principles. The social media data set is vast and generally cannot be analysed in its entirety without a significant resource investment. Intelligent sampling is therefore essential. Sampling can be done across the body of mentions I.e. across all social media channels using random sampling principles or be restricted to one or all of the main consumer channels (Facebook, Twitter, YouTube).
- Remember that volumes and share of voice hide rich insights. While volumetrics are sometimes useful, they are not the be-all and end-all of social media analysis. The exercise is about measuring and not counting. This is why human analysis is important in this context.
Why is applying some of these principles so important? We recently evaluated the Boy and Carrot campaign run by Waitrose in April 2014. In order to evaluate the impact of the campaign, we took three different approaches:
- We analysed social media mentions of the campaign to understand which aspects of the campaign itself and which messages had resonated with the audience.
- We analysed social media mentions in an average week pre and post the campaign to understand whether the campaign had shifted perceptions of the brand.
- We looked at impact of the campaign on overall mentions on social media to understand how impactful the campaign had been overall.
We found that campaign mentions were comparatively few on social media (just over 1% of all Waitrose mentions) indicating potentially lower levels of awareness of the main campaign line than initially appeared, certainly indicating low levels of awareness of the campaign as a whole among social media audiences. However, there were indications on social media of the high awareness levels for the advert via traditional channels, as attested by the spike in mentions during the broadcast of the ad on Channel 4 during ‘Made in Chelsea’. The concept of co-ownership (the key message of the campaign) was not prominent in mentions of Waitrose on social media before the campaign, and the campaign did not appear to trigger any rise in mentions, perhaps indicating that the campaign message did not necessarily resonate on social media despite a significant push on YouTube. What the YouTube push drove though was a marked increase in the number of YouTube subscribers for the Waitrose channel, indicating a positive impact on the reach of the channel. The advert itself was associated with very positive emotions of pride, nostalgia, and caring. The choice of music, production values, the storyline and the characters were also praised, indicating that those who commented on the advert on social media related to it, which will have contributed to increasing positive perceptions of the brand.
So without looking through these different lenses, we would not have been able to provide the necessary context to explain some of what key metrics were showing. Each campaign analysis will vary based on its nature and will require a specific approach but these basic principles if applied will at least provide some rigour to minimise the bias in results. Andrew Nelson @AIFNelson & Gaelle Bertrand @gaelleanneclair Client Directors, Precise Brand Insight.