Inf1 CG, VMA lecture 3: The frog's eye

1. Review: Building representations

As signals pass ‘upwards’ through the visual system

• They converge as they go
• The higher, the more so

As we go ‘up’ the layers

• The stimulus each neuron responds to becomes more complex
  • The neuron is more specialised
• The information each neuron represents becomes richer

Simple symbols such as light spots

• are combined into larger units such as lines or edges
• based on configurations of excitatory/inhibitory connections

Consider how this relates to Marr’s point about the purpose of visual processing

• The symbols and connections constitute a representation of the world that is useful (in an evolutionary sense) for a particular organism.
• Other animals have different receptive fields to provide them with the type of environmental information that is relevant for them
  • but might be irrelevant to us

2. Today’s vision specialisation

You are a frog living at the edge of a pond

• You need to find lunch
  • and avoid being lunch for someone else
• You rely almost entirely on vision to tell you about the world
  • (as opposed to smell, sound, etc.)

From lecture 1:

Vision should produce descriptions that are “useful to the viewer and not cluttered with irrelevant information.” (Marr)

Consider the following questions:

• What are the most useful things for you to perceive?
  • That is, as part of your conscious experience of vision
• What other kinds of information is present in the external world
  • but not useful to you?

3. Review: Selective responding

Kuffler discovered that retinal ganglion cells are sensitive to spots of light

Hubel and Wiesel (1959) discovered that neurons in a higher layer respond to patterns of light and dark

• such as edges and lines

Neurons will usually fire strongly for a particular type of stimulus

• such as a line with a particular orientation
• This is its preferred stimulus
• The same neuron will also fire for similar stimuli
  • Less strongly, the less similar

Neurons in many layers prefer stimuli that are moving.

• Some may fire for a stimulus moving in any direction
• Others are only interested in stimuli moving in a certain direction

A goal of single neuron recording

• to find out the preferred stimulus for a particular neuron

4. Method: Single-neuron recording

Hubel and Wiesel used Single-neuron recording

• Inserting an electrode into a single neuron and ‘listening’ to that neuron’s electrical activity.
• Provides information about very specific low-level (implementation level) brain activity.

Why do we use it?

• If each neuron in your brain was a person
  • Recording brain activity as a whole would be like listening to a crowd of approximately 100 billion people ( 10^11)
• Hard to understand what is going on if almost all are talking!
• Single-neuron recording is like listening to only one person at a time

The downside:

• Not very representative of all the brain activity in response to an event
• Chance plays a large role

5. Introduction: What the frog’s eye tells the frog’s brain

Collaborators Lettvin, Maturana, McCulloch, and Pitts published several papers

• Recording from single retinal neurons or small groups of neurons
• In the frog retina
• Published around the same time as Hubel and Wiesel
  • using similar methods
• Frogs make good subjects
  • because they have much simpler visual systems than cats or humans

Their goal was to link neurology more closely to behavior than had previous work

• Spots of light are not very naturalistic stimuli
• Used dark spots and lines on a full-color pond-like background

6. Frog findings

The authors discovered five distinct types of cells

• They called them feature detectors

Each type is “interested” in a separate aspect of the environment

• Contrast detector   sustained light/dark edges in a small area
• Convexity detector   dark objects moving on background
• Changing contrast detector   moving edges in an area of the visual field
• Dimming detector   dimming light from edge or centre of visual field
• Dark detector   overall light intensity described in Maturana (1960) only

Next week’s tutorial will focus on these feature detectors

• and how they help the frog create useful descriptions of the world

Today will focus on convexity detectors

• and what they tell the frog’s brain

7. Frogs are convexity detection experts

Lettvin et al (1959) write that

“The convexity detector informs us…whether or not the object has a curved boundary, if it is darker than the background and moving on it, it remembers the object when it has stopped…it shows most activity if the enclosed object moves intermittently with respect to the background.”

This sounds familiar…

• Small dark object
• Intermittently moving on background

Authors go so far as to acknowledge that

• these convexity detector cells appear to be specialised as
• “bug detectors”

8. Data and interpretation

How do we know what is a preferred stimulus?

The following graphics from Maturana et al (1960) show the response of a single neuron

• identified as a convex edge detector

B: Stimulus is a small dark moving object (1 degree of visual angle)

• Note the intense response to the moving object

C: Stimulus is a stationary object of the same size as in B

• Note response to the stationary object starts strong
  • but tapers off

This response pattern tells us that convexity detectors are very happy with stimuli like these

9. Data continued

Compare these responses to those for a dispreferred stimulus:

• E top shows same small, moving stimulus (as in B)
• E bottom shows complete absence of response to a moving bar
  • Wiggly baseline indicates general darkening of visual field

This response pattern tells us that convexity detectors are not interested in stimuli like these

• although other kinds of cells would be very interested

10. Fly spy with my little eye…

Note that three out of five types of frog feature detectors are interested in moving stimuli

• They may still fire to stationary stimuli, but these are less preferred
• The bug detectors even prefer jerky, erratic motions to smooth motions!

The authors note that this as a key criterion for the frog to locate and catch its prey

• among all other environmental objects

Why is it so useful to perceive motion?

• Movement helps organise perception in complex, cluttered scenes
  • Emphasises objects
    • I.e. segments figure from ground
  • This is one reason why an animal may freeze if it thinks it has been seen
• Not dependent on the overall level of illumination

11. Why is it useful to perceive motion?

There are several types of motion that can be perceived:

• Real movement
  • in which an object travels across the visual field
  • what we usually think of as motion
• Apparent motion
  • When two stationary stimuli are presented one after another close together in space
  • Neither of the stimuli actually moves, we only perceive the motion
  • This is the principle that allows flip books and television to work

The frog is only interested in real movement

• An object moves
• but the frog is stationary

There are other types of real movement, see Goldstein

12. How not to starve your pet frog

Back to our riddle: How could you easily starve a pet frog?

Answer: Give it food on the ground or in a dish

Maturana and colleagues write that

“for them, a form deprived of movement seems to be behaviorally meaningless.”

The frog has the capability to perceive many types of motion

• but it's not equipped to detect a stationary source of food
• unless it was previously seen to be moving and has just stopped

The frog can detect light reflecting off objects in the environment, but

• It is physically incapable of perceiving a small still object as possible food
  • E.g. a dead fly on the ground

It's tempting to say that a frog

• Lacks the capability to represent that dead fly
• It has inadequate primitive symbols and rules

13. Conclusions

This work (and related work on other species near the same time) provided compelling evidence

• that analysis of the visual world begins in the retina
• Previously unclear if retina and early visual system transmitted information verbatim to higher brain areas for processing there
• Now clear that even the frog’s earliest cells analyse the incoming image on several dimensions
  • Local variations in light intensity, overall illumination
  • Moving edges, curved objects (convexity), contrast

Lettvin et al (1959) point out that these constitute “complex abstractions from the visual image.”

• They are not just primitives.

In our language of representations

• the frog builds a nearly complete representation of things existing in the world
• in far fewer steps than it takes a human

14. Other types of representation?

So far we have discussed representations

• that are built up through layers of successively abstract processing
• beginning with very primitive elements

Some theories suggest that single neurons might be able to code for complex stimuli

• These are so-called grandmother cells
  • They would respond only to specific objects or people
  • For example, a cell that fires only when you see your grandmother
• This would mean the neuron responds to a concept
• Rather than low-level visual features

The idea was introduced rather as a reductio ad absurdum

• But a recent single-neuron recording study by Quiroga et al (2005) suggests that there may be some truth to this theory!

15. The ‘Jennifer Aniston’ Neuron

Quiroga found neurons that responded consistently to the same celebrity’s face (Jennifer Aniston)

• even from different viewpoints
• also responded to the person’s written name

The neuron did not respond to pictures of other famous people, landmarks, or common objects

Another neuron responded exclusively to Halle Berry

• And a third to the Sydney Opera House

A few methodological notes about these findings:

• Recordings were from neurons in the hippocampus
  • This is a brain structure associated with long term memory, not vision
• The authors presented many stimuli, but were there enough?
  • It is possible that a larger sample set may have found other things to which these same neurons responded

16. Coming up next

Vision tutorial revisits some lecture content

• Marr and the purposes of vision
• Neurons and firing
• Discussion of the frog’s eye and the frog’s brain
  • What do these various receptive fields look like?
  • How a frog’s vision is adapted to the environment

Next week’s lab looks in-depth at an optical illusion (the Hermann Grid)

• What can it tell us about inhibitory connections between neurons?
• Competing explanations for this illusion