This post follows on from Part 2 and assumes the reader is familiar with it. Also a warning this post is a little longer than the rest but I hope it is worth it
When measuring PR IMHO too much emphasis seems to be placed on proving *absolute* causality. A piece of PR, results in coverage, which provokes a demonstrable response, which leads to a required outcome. At each stage proof is required. The reality as I mentioned in Part 1 is that this is often unlikely to be possible to do and as I covered in Part 2 lack of proof hasn’t stopped accountants making predictions.
What is achievable though is demonstrating that causality was likely. Econometric modelling (or Regression analysis) can tell you whether outcomes are likely to be correlated with particular observable activities and they can also tell you the likelihood that these correlations didn’t occur by chance.
So how do we go about building models that demonstrate value? I will cover one here and more in the final part of this series.
Share Price based
Quoted companies should, in theory, be the easiest organisations to build a value based approach around as there is a constant real time assessment being made of the value of these organisations – their share price.
First we track the impact of PR as we would normally, but we do so in as close to real time as we can. Online this can be done as we often know the exact time when something is published, whereas offline this is much more problematic. We then qualify the activities that occur, seeking to focus in on those that are most likely to have been influential. Finally we look for evidence of indicators of influence arising from or within these activities e.g. positive sentiment in coverage about the company.
Next we use econometric modelling to estimate how much of the movement in the company’s share price over a period of time is explained by these indicators of PR activity and how much is explained by other factors.
Sounds a bit tricky? Likely to be very expensive? Well perhaps not. I recently had a demo of a new piece of software called Fin-Buzz that seeks to help PR/IR professionals do this for UK FTSE 100 companies (Note this was a complete coincidence I only found out about it when researching for these posts and neither I nor RealWire have any link to the company that produces it).
In my opinion the software could be improved through looking at additional sources of coverage, it currently tracks around 100 I believe that they view as key. I would also suggest including the ability to actually build and run your own econometric models that then produce actual values â€“ to save me the time doing the analysis below! At the moment the software only provides evidence that causality may exist, an example of which I have used as the basis of my analysis below.
But it is still a good start and I will be interested to see how it develops.
(Note: There is no particular rationale behind choosing this example other than I was tracking Centrica at the time as we had expected to meet them at the Communications Directors Forum).
Acquisition of 20% stake in British Energy by Centrica plc
10th May 2009 – Rumours broke in the evening that the acquisition of a stake, thought to be 25% at the time, was going to be announced. http://news.bbc.co.uk/1/hi/business/8042544.stm
11th May 2009 – Announcement made in the morning that stake would actually be 20% http://www.centrica.com/index.asp?pageid=29&newsid=1783 http://news.bbc.co.uk/1/hi/business/8043191.stm
A graph of the change in share price from the day before announcement until two days after looks like this:
The graph shows that the share price of Centrica rose approximately 6% on the day of the announcement representing a change in valuation of approximately£700m.
In order to model how much of the change in share price was potentially explained by PR, and in particular the reaction to the announcement of the deal, the model needs to include detailed data on factors that could explain movements in the share price. For the purpose of this example I have looked at:
– market data FTSE100 used (did companies in general experience similar changes in prices)
– comparable company (same sector) data weighted value of the three FT comparables above used (did companies in this sector experience similar changes in prices)
– sentiment measured in media coverage by Fin Buzz
I have then run regression analysis based on the movements in these variables during the month of May. It should be noted that it is a while since I studied Econometrics at University so the experts out there might pick holes in my analysis
Two models have been produced. One that models all three variables and one that just looks at the extent to which the movements in the share price can be predicted by sentiment alone.
The graph below shows the movement in the actual share price and the share prices that would have been predicted by each of these models (you might need to load the page itself to see clearly):
You can see that both predictions are highly correlated to the actual share price. In fact the statistical analysis says that the vast majority of the explanation for the movements in the price relates to the sentiment (R²s of 79% and 84% respectively for those who know about these things).
In addition further statistical analysis (t tests and F tests for the statistical experts) shows that there is a greater than 95% chance that movements in sentiment are important in explaining the movement in the share price and that there is only a tiny chance that this relationship is only by chance.
N.B If anyone would like to see the statistical details then just let me know.
The sentiment measured by Fin Buzz across their sources explains the vast majority of the fluctuations in the Centrica share price over this period and hence the change in market value of £700m.
The value of good PR to Centrica in this situation was therefore potentially worth millions. We now enter chicken and egg territory. Was there positive sentiment towards the deal because of how it was communicated or because of the deal itself? The answer is probably both.
Some of the value is likely to be in the deal, but only to the extent that the reasoning, the strategy and the implications were communicated well. One person could have heard the announcement and thought “well its a decent deal but not really convinced” whereas another who had been better communicated with and therefore understood the thinking better might respond “this is a great deal actually”. The impact on value in each case could be very different.
Finally even if PR only influenced/resulted in say 10% of the positive sentiment this would still have apparently resulted in the creation of an extra £70m of value.
But perhaps it isn’t necessary to reach a firm conclusion on this to demonstrate the likely value added by PR in this situation. Let’s face it if you had something worth £700m wouldn’t you want to entrust it to the experts?
2 thoughts on “The Value of PR Measurement – Part 3”
Very interesting post Adam and good to see another advocate of using statistical techniques to show how PR coverage drives real world outcomes.
I agree with you that while these techniques can prove correlations there is then a leap of faith in establishing causality.
Much of our work involves showing correlations between editorial coverage and resulting effects such as increased sales or website traffic in the following days and weeks. In these situations it often makes sense to hypothesise that one causes the other 1) because of the time lag, 2) because the former is more likely to cause the latter rather than vice versa and 3) we have worked hard to disaggregate effects from other causes.
My worry with share price is that these first two points become blurred. You mentioned â€˜chicken and eggâ€™ in relation to communications vs the deal itself but there is another example of this. Sentiment can drive share price, but share price can also drive sentiment â€“ after all news of share price increases tends to be positive. Share price can also driveâ€¦share price, since many traders have automatic systems that are programmed to buy or sell at a given price. It also affects psychology â€“ people are more likely to want to invest in an asset that is gaining in value, particularly if â€˜groupthinkâ€™ kicks in and people around us perceive the same. This is of course why we have asset bubbles and why we are all wise after they burst â€“ â€œwell it was obvious the housing market was overheated!â€
All of this shows that markets are complex non linear systems with many feedback loops that are difficult to model using linear regression. The finest financial and statistical minds have real problems building predictive models. I know this because I was recently at a conference with some of them which certainly put a (somewhat sobering) perspective on what the rest of us can hope to achieve!
Paul I completely agree with all your points and thanks for taking the time to write such a detailed comment. When writing this post I was concerned that if I went into too much detail with some of the statistical, systematic and behavioural issues you raise that I might have been writing a thesis
However despite this I still think that analysis such as this has value for the reasons I outlined in Part 2 i.e. when it comes to valuing companies you can get into all sorts of complex analysis such as CAPM in order to reach a valuation but you are still guessing as no one can predict the future. I am not claiming that PR definitely led to the increase in share price, but merely pointing out that there are statistical, logical and common sense arguments that would imply that it can play a part. Consequently organisations should consider that PR’s potential value is likely to vastly exceed numbers generated through measurments such as AVE. Just ask Gerald Ratner the impact on your share price of bad PR