until an observation provides support for it. It adopts a threshold approach to

measuring the probability that a difference or relationship between observa-

tions re¬‚ects a genuine difference. A con¬dence level of 95% (or 99%) is set

as the standard which a measure of difference, or relationship, has to meet to

be considered statistically signi¬cant. If the observation does not meet these,

the test has failed and the null hypothesis “ that the theory is wrong “ is

af¬rmed. If the observation passes the test, the null hypothesis is rejected and

the theory is validated.

• Bayesian statistics: here, by contrast, no hypothesis is ever null. There

is always a probability that a hypothesis is correct “ that our theory is a

correct description of our data. Bayes™ theorem, named after the 18th Century

clergyman, Thomas Bayes, who proved it, is a logical corollary of the basic

rules of probability. A straightforward expression of this formula is:

p(Hi | y) ∝ p(y | Hi ) p(Hi )

where p = probability, Hi = one of a series of hypotheses (one of which is

assumed to be true), and y = the data. The formula in effect says that the

probability of the hypothesis, given the data we are looking at “ p(Hi | y) the

posterior probability “ is proportional to (∝) the probability that we would

see this same data pattern if the hypothesis were true “ p(y | Hi ), what is

known as the likelihood or conditional probability of the data “ multiplied by

the prior probability of the hypothesis, p(Hi ), the probability that we would

have attached to the hypothesis prior to seeing the data.

The theorem licenses “ insists upon “ the use of prior knowledge when inter-

preting data. The theorem makes it clear, for example, that our prior knowledge

about a phenomenon is relevant to our interpretation of subsequent observations.

Rather than the theory being entirely subject to observation, as in the classical

tradition, in Bayesian statistics, the theory is allowed to have an impact on the

probability of the evidence. Thus, if we attached a fairly high prior probability to

a theory, observations which appear to contradict it would reduce the probability

of the theory being true. That is, result in a lower posterior probability. Yet it

would not reduce the probability of it being correct to zero, in the way demanded

by the hypothetico-deductive method.

25

The two schools of statistics

Another important feature of the formula is the conditional probability or

˜likelihood™ function, i.e. p(y|Hi ). This enables us to consider any data or obser-

vations as evidence relative to any particular hypothesis we choose, and to make

allowance for the strength or weakness of the data as evidence for that hypoth-

esis. (Hypotheses can be suggested by the data, provided the prior probability

attributed to them is kept independent of the data. That is, provided the prior

probability of the hypotheses is evaluated on grounds other than those provided

by the data being considered. This requires some mental discipline, but is not

logically impossible.)

Thus, the Bayesian approach facilitates the development of a range of hypothe-

ses, or theories, to explain an observation and provides the means for deciding

between them. In other words, Bayesian thinking provides a formal theoretical

support for the abductive method of reasoning.

We accept that Bayesian methods, although commonplace in many aca-

demic and technical research circles, are considered by most commercial market

researchers to be far too inaccessible to be developed into everyday practice.

However, we make no apology for giving the Bayesian approach prominence

in this book. This is because the concepts that underpin Bayes provide a robust

defence of the holistic school of data analysis. So, if you are nervous of Bayesian

statistics, just stay with the concepts and principles behind the approach. Bayesian

thinking will pay dividends in powering up your analysis and consumer data.

Having reviewed the theoretical underpinning for the holistic approach to

the analysis of market research evidence, in the next chapter we look brie¬‚y

at the theoretical underpinning for the idea of combining management ˜intu-

ition™ with the formal analysis of data. Our aim is to give the analyst the

con¬dence to work with both the data and management ˜hunch™. We will pro-

vide frameworks “ checks and balances “ that will help the analyst differentiate

between unsubstantiated whims, and insightful nuggets of wisdom, that can be

constructively dovetailed into the hard consumer evidence.

Data-rich intuitive analysis

3

Summary

• Central to the concept of holistic data analysis is the notion of factoring

management prior knowledge and intuition into the formal analysis

of the consumer evidence. So here, we review what we know about

intuition: knowing without knowing why.

• We look at exactly what seems to characterize intuitive thought, noting

the way that many commentators argue that intuition is a set of thought

processes that can be cultivated and developed.

• But we also point out that intuition can be plain wrong, stressing the

importance of not allowing intuition to overpower the data.

• This allows us to arrive at the conclusion that powerful analysis often

involves operating in both data and intuitive modes.

• This chapter will give the reader the ability to spot when an argument

is being advanced based on ¬‚awed logic or when it simply re¬‚ects a

biased viewpoint. The aim is to give the reader the ability to differentiate

between ˜informed™ intuition that is compatible with other sources of

evidence, and wilder, speculative claims that need to be challenged.

Factoring intuition into the data analysis process

The tension between the formal analysis of data and the intuitive insights of

entrepreneurs has always been a source of some concern. Anita Roddick once

famously described market research as ˜the view through a car™s rear-view mirror™:

suggesting that market research only tells you where you have been, not where

you are likely to be going. And many will be familiar with the claims made

by senior decision-makers about the apparent shortcomings of market research.

Notably, these include the view that research is often:

• Inconclusive and/or the researcher seems to lack total conviction about their

results: the research seems to pose more questions than it answers, particularly

when predicting future developments.

28 Data-rich intuitive analysis

• Accompanied by a great deal of quali¬cation: all of which appears to be at

odds with the economy and clarity expected of a decision support system.

The above observations lead some decision-makers to conclude that there

is little, or no, point in researching markets, and that all that is required for

successful marketing is the ˜entrepreneur™s vision™, or the marketing person™s

hunch, and the collective will of a well-organized business.

We agree with this, up to a point: market research data, and the conclusions

made, are not suf¬cient, on their own, to drive correct marketing decisions. But

we also know that management intuition alone “ uninformed by research “ is

likely to lead a business astray. Examples of businesses actually succeeding by

ignoring the ¬ndings of research are well known. Everyone has a story (the

favourite one being the Sony Walkman: a ˜tape recorder™ that cannot record

is never going to catch on said the research). Far more typical, though, but

much less well publicized, are the examples where businesses made catastrophic

decisions because they ignored what proved to be very sensible messages coming

from the research. The Ford Edsel and the Sinclair C5 head a long, ignominious

list. So critical to the goal of building a market research discipline that is more

engaged with the decision-making process than before, is the task of integrating

the analysis of market research evidence with the more intuitive contributions

from management.

The value of rigorous data-rich thinking

The way forward for informed business decision-making is clearly to develop

frameworks that allow intuition to ¬‚ourish, but which still keeps ill-informed

intuitive thinking under check. Thus, as indicated earlier, our view on this can

best be summed up as follows: ˜intuition without research is blind, and research

without intuition is dumb™. Marketers need to believe in their own understanding

of the market, and their wider business experience. However, they stand far

more chance of making a good decision if they base this ˜insight™ on appropriate

market research data. So, here we advocate data-rich, as opposed to data-poor,

thinking: this is the basis of ˜informed intuition™.

The power of informed intuition

The notion of ˜intuition™ “ knowing without knowing why “ has been around for

centuries. For instance, Spinoza aligned intuitive thought with the road to truth.

And everyone will be aware of the way famous entrepreneurs extol the power of

that ˜tingle™ of intuition when making key decisions. And the stories continue to

abound. For example, Musto, the sailing clothing company, now have a story to

tell about how they ended up sponsoring Ellen MacArthur “ Britain™s around the