Inf1 CogSci 2013: Lecture 17: Putting the 'science' in Cognitive Science

1. Introduction: The scientific method

Back on day 1 we said that Cognitive Science was distinguished by its methodology

The word 'science' is itself the name for a methodology

• An approach to increasing knowledge
• By formulating explanations
• And checking them experimentally

We call it the scientific method

Historically, an attempt to integrate experimentation into 'natural philosophy'

• By relating hypothesis, prediction and experiment

2. Models and theories

A model predicts—A theory explains. [Amos Rapoport]

There is an enormous literature on exactly what this kind of statement means

• What constitutes a theory? a model? an explanation?
• We're not going to go there
• See the Stanford Encyclopedia of Philosophy article on scientific explanation for a good overview

Two names to recognise

Karl Popper

• Theories must make testable predictions
• Theories cannot be proved, only disproved
• Progress consists in theory refinement/replacement as a result of disproof

Thomas Kuhn

• Normal science is essentially evolutionary, within a methodological paradigm
• More rarely, scientific revolution happens when an old paradigm is abandoned in favour of a new one

3. An example from astronomy

Copernicus proposed a heliocentric model of the solar system

• In which the planets travelled in a circle around the Sun
• This made predictions about where the planets would be seen
• They weren't quite right
• So epicycles were added
  • That is, circles around circling centres
  • As the moon does

Kepler had access to more detailed observations (from Tycho Brahe)

• But was unable to make any number of epicycles fit the data satisfactorily
• Eventually he hit on an elliptical model
• Which fit the data much better

Finally Newton produced an explanation

• With his laws of motion and theory of gravitation
• This provides a mechanism which would cause elliptical orbits

4. An example from biology

Mendel modelled the inheritance of visible properties of pea plants

• For which two pure-bred strains could be observed
  • E.g. tall vs. dwarf pea plants
• Which if cross-bred produced all of one variety
• But in the second generation the other variety would reappear
  • Only 25% of the time

His model was composed of pairs of hidden agents (we call them genes)

• Each pair controlled some visible property
• Each gene came in two versions
• One of which was dominant
• 
  No copying, reproduction or re-use allowed
  Cannot find original creator, reproduced without permission
• The model successfully generalised to predicting the results when multiple properties were involved

An explanation for this didn't come for nearly 100 years

And depended on the unification of theories of cell division, genetics and morphogenesis

5. The role of experiments

Often observation and experiments come first

• As a prelude to model-building or theorising

Once we have a model, we can use it to make predictions

• That is, testable hypotheses
• And then design experiments to test them

Contrast, for example, the Michaelson-Morley experiment with the Einstein-Eddington one

• Michaelson and Morley attempted to measure the effect of the earth's movement through the "luminiferous ether" on the speed of light
  • 
    
    Courtesy of Cronholm144
  • But failed to find an effect
• Einstein's theory of general relativity predicted a gravitational deflection of light twice that predicted by Newton's theory
  • Eddington measured the deflection of light from stars near the sun during a total eclipse in 1919
  • 
    Original is public domain
    From Dyson, Eddington and Davidson 1920
  • Einstein's prediction was confirmed

6. An example from cognition

Some models of lexical memory involve some notion of association

• That is, our participantive experience that words may be associated in our minds is directly reflected in the model
• And furthermore that how this is done makes predictions about priming

Priming is the name for a wide range of reaction time effects

• That is, effects on the speed with which we can respond in certain situations

A typical priming experiment involves a lexical decision task

• A visual stimulus is presented, a sequence of letters which either form a word, or do not
• The experimental participants press one of two buttons to record their judgement as to whether it is a word or not

Word frequency and number of meanings have a measurable effect on how fast a word is judged to actually be a word

So does priming

• If the participant is shown two stimuli
• And asked to judge the second
• They respond 10% faster if the first word is associated with the second
• So for example butter is judged to be a word faster when preceded by bread than when preceded by broken

7. Cognitive science, operationalization and reaction times

It seems that everywhere you look in the experimental cognitive science literature, you see reaction times

There's a good reason for this

• Operationalisation
• That is, the projection of aspects of a model into experimentally measurable phenomena
• What property of a model corresponds to what aspect of an experiment?

8. Operationalising runtime

A key value of computational models is that they invite one particularly easy operationalisation

• Runtime operationalises as reaction time

That can't be right: runtime on how fast a computer?

• So, relative runtime operationalises as relative reaction time

So, the graph everyone wants:

• 
  
• When the model predicts the same times for 'apple' and 'orange' in condition one
• But longer times for 'apple' than 'orange' in condition two

9. Operationalisation, cont'd

Another way to think about operationalisation is to ask

• What aspects of a model make claims on the world?
• What kind of measurements should be compared?
  • Measurements in the model?
  • Measurements in the world?

A good old-fashioned map is a kind of model

• 
  
  Courtesy of OpenStreetMap

What kind of claims does it make?

• Orientation in the world per N-S-E-W corresponds to orientation on the map T-B-R-L
• Colour of items on the map correspond to real-world properties, per a key

And what doesn't it claim?

• Distance in the world is not proportional to distance on the map
  • At least, not in detail
  • Because this map, like most maps, is a plan, that is, it takes no account of elevation changes
  • And when zoomed out, the width of roads is not to scale
    • 
      
      Courtesy of OpenStreetMap
    • If line width were taken as making properly scaled claims on this map
    • They would be claiming that the M8 was half-a-kilometre wide!

10. What is a prediction?

OK, so we have a model, and we've done some operationalisation

• What do we do next?

We're looking for explanations

• Usually that means causal mechanisms
• And whereever there's causation, there's dependency
  • The effects depend on the causes
  • The acceleration depends on the masses and the distance
  • The reaction time decrease depends on the degree of association

So we're looking for one or more aspects of the model we can manipulate

• And, in the world, whose operationalisation(s) we can manipulate

And another aspect we expect to change as a result

• So we measure the operationalisation of its change in the model

And see if that change corresponds to what the model says it should be

• Increase the mass of one of the bodies, and see if the other accelerates faster by an amount linearly proportionate to the increase
• Decrease the association between the prime and the target, and see if the priming effect decreases

11. Dependent and independent variables

In an experiment based on a model, the things we manipulate or measure are called variables

• mass and acceleration
• association and reaction time

The ones we're trying to explain

• The ones our model treats as causal effects
• Are called the dependent variables
• acceleration and reaction time

The ones we manipulate

• The ones our model treats as causes
• Are called the independent variables
• mass and association

So when someone says "We found a significant effect of obesity on morbidity" they mean

• They took obesity as an independent variable
• Operationalised it as, say, BMI
• Then looked at morbidity as a dependent variable
• Operationalised that as, say, numbers of visits to surgeries
• Partitioned a sample population by BMI
• Evaluated the prediction that the higher BMI group would have more surgery visits

12. Correlation is not causation

Where was the model in that imaginary experiment?

• Suppose the prediction was upheld
• We might say "High BMI correlates with more frequent visits to surgeries"
• Can we conclude that obesity causes morbidity?
• No!
• Maybe morbidity causes obesity
• Maybe some third factor causes both
  • This raises the question of control
  • As in "Did you control for age/gender/social class?"

Without a model, correlation cannot prove causation

• After all, 99% of all alcoholics started on milk