“All our knowledge has its origins in our perceptions…”
“Study the science of art… …the art of science. Develop your senses- especially learn how to see. Realize that everything connects to everything else.”
“There are three classes of people; those who see, those who see when they are shown, [and] those who do not see….
“[An] average human looks without seeing, listens without hearing, touches without feeling, eats without tasting, moves without physical awareness, inhales without awareness of odour or fragrance, and talks without thinking.”
“Why does the eye see more clearly when asleep than the imagination when awake?”
“I awoke only to find that the rest of the world was still asleep.”
[Leonardo da Vinci; Notebooks]
Look around you.
As you do this, you may become aware that your gaze focuses more on certain objects. Perhaps they have significance for you. Your mind is using visual information to build a ‘map’ of the space around you, placing you into it and coding your relationship to its contents.
Are you really sure that a floor can’t also be a ceiling?
Our eyes scan the scene before us by jumping from point to point, and focusing on details. This harvests a sequence of two-dimensional ‘snapshots’ which our mind then assembles into a coherent three-dimensional picture. This spatial construction relies upon information from our peripheral vision – the ‘fuzzy’ zone around each focussed ‘photo’. Our visual senses take in only a tiny fraction of the information available, but our conscious awareness experiences a full picture.
Our eyes and brain process visual information using learned ‘rules’. These govern how we interpret line and colour, and hence understand form, depth and motion. We use these rules to interpret the outer world in a way which ‘makes sense’ of what we see. This creation of a three-dimensional reality is so routine that it appears effortless.
We use this information to orientate ourselves in space. Our brain infers depth by computing the difference between what is seen by our two eyes.
We also deduce additional information from our memories and learnings about the nature of objects. When we look for example at a table, we see only one side, and, based on our understanding of tables, assume that the rest of it is there. This enables us to estimate our distance from objects and the relationship between them, without needing to verify every part of the information.
Eyes are energetically expensive organs. Selection has optimized our visual system to the energetically lowest cost solution that still allows us to survive and reproduce within our ecological role. We use minimal amounts of data to build the scene and put ourselves into it. This shows us what we need to know, rather than what is there. Our mind fills in the missing information about what we see. This is why optical illusions can so easily trick us.
Look at the shapes in image ‘C’.
When we focus on one corner of each shape, we sense the object as oriented in a certain way in space. If we look at a different corner, that perspective shifts.
Notice how well your brain builds these simple lines into three dimensional forms.
Seeing, then, is an active process involving many unconscious decisions. What we choose to see, both physically and metaphorically, enables us to build a picture of our world, put ourselves into it, and survive our interactions with it. We share this mechanism with other mammals and some other animals.
‘Making sense’ of what we see requires an interpretation using the memories, emotions and beliefs associated with what we think we are looking at. The way we do this, however, and the meanings we attach to what we see, are unique to us both as humans and also as individuals.
What do we need to see? Why does our mind need rules to interpret what we look for? And what influences these perceptions?
What have we evolved to look for?
All our knowledge has its origins in our perceptions…
[Leonardo da Vinci, Notebooks]
Our visual systems inform our perceptions of depth, distance and movement. This allows us balance and move in relation to the contents of our view.
Colours, facial patterns and subtle rhythmical movements easily ‘catch our eye’, and so are given more attention. We may then visually ‘lock onto’ an area of interest and make a more detailed assessment. Bats and dolphins use sonar in a similar way to target their prey. Our eyes are particularly sensitive to rhythmical patterns, sufficiently so that we can even infer movement in those patterns.
Why does this happen? Lynne Isbell believes that our aptitude for colour and pattern recognition results from selection for a visual system allowing us to avoid snakes as well as find ripe, red, sugar-rich fruits in dense foliage.
Snakes have rhythmical patterns on their bodies, and move in distinctive ways. Isbell proposes that tree-dwelling snakes became a major hazard of our ancestors at an evolutionary stage when they became predominantly fruit-eaters.
The main natural hazards affecting modern chimpanzees are snakes, spiders, some large mammals, and other angry male chimpanzees. In this last case it is important for these primates to read the emotional state of others in the tribe in order to avoid confrontation.
How do we achieve this? Tiny and often rhythmical movements in our head and neck muscles control our facial expressions. Observing faces enables us to infer information about the emotional state of others. The eyes are an important social signalling system in humans, primates and other mammals. In dogs for example, direct eye contact communicates aggression, and averting the gaze shows submission.
Developing a visual system attuned to the small movements of stealthy predators in dense undergrowth may have also allowed our shared ancestors with other primates to detect subtle social cues in other faces. Understanding these signals is an advantage for primates who cooperate for foraging.
How do we ‘put ourselves in the picture’?
Shut your eyes for a few moments and visualize a familiar object, for instance the handle of a door to your home.
Now think about where in space you visualized this object in relation to your body. Was it, for example, close enough to grab?
We represent external things in our internal space in relation to our physical selves. This is how we code all of our experiences, including abstract ideas. Our emotional response tells us how we relate to the contents of the scene. Even though the picture does not change, our experience of it is changing constantly because our internal emotional experience is in constant dynamic flux.
Our camera-like eyes focus onto three-dimensional objects from a slightly different angle. Each eye however captures only a two-dimensional image. The mind uses the small differences in perspective between these two flat images, and also infers information about the object as seen from other angles, to judge depth and distance.
Representational art, like a photograph, provides a two-dimensional ‘index’ of a three-dimensional view. Our eyes scan a painting as they do with any visual field, compiling a montage of snapshots. Our brain uses this information to construct a three dimensional understanding of depth and distance, and relate us physically to the contents of the scene.
Art is a lie!
In Pablo Picasso’s Girl with Mandolin, we appear to be given at least some of these fragments. Despite being able to recognize the subject of the composition from these details, we cannot build the picture into a coherent image of the girl and her instrument. Instead we see a chaotic representation of what the artist implies is our unprocessed sensory experience.
How do we ‘make sense’ of what we see?
As with other primates, our visual senses ‘orientate’ us socially as well as spatially. Our neural visual processing circuits are hard-wired to make social cues a priority, even when we look at a painting. These two parallel functions of our visual system underpin the magic of Leonardo da Vinci’s Mona Lisa.
Study the science of art… …the art of science. Develop your senses- especially learn how to see. Realize that everything connects to everything else.
[Leonardo Da Vinci]
Our eyes scan this painting like any other image, using eye movements to fixate and gather focused ‘snapshots’ of detail. Like any primate, our unconscious mind preferentially directs us to capture social cues from others’ faces and gestures. As a result, our eyes mostly play over the woman’s face and hands. Meanwhile the neural circuits informing our depth perception and sense of balance are active, constructing spatial relationships from information in our peripheral vision, and placing us physically into the scene.
The landscape behind the figure has a different perspective on either side of the figure. We are not consciously aware of these differences because we focus on her face. Meanwhile our unconscious mind is struggling to reconcile this ‘mismatch’ with our understanding of space.
Whilst we look from one of her eyes to the other, our spatial processing circuits ‘make sense’ of where we are in relation to the scene. As our vision moves between her eyes then, our spatial circuits put us in a different location. Our unconscious mind struggles to reconcile this with our expectations of how space is arranged.
Something similar happens when our train is sat in the station whilst a train on the adjacent track starts to move. Our visual processing circuits are so fast that we may initially experience the sensation that it is our train which is in motion. The balance organs in our ears then inform us that our train is the one standing still. Our mind consequently alters its interpretation of the scene into a coherent explanation, placing the movement ‘out there’ into the other train.
So as we look at the Mona Lisa, our unconscious mind reconciles the conflicting spatial information by also putting the movement ‘out there’. In this way it appears to us, as we gaze into her eyes, that the beginnings of a smile start to play around her lips…
Why do we need rules to internally construct our visual world?
Our eyes capture only flat images, like a camera, so we construct all of the depth we see. We do this every single time we open our eyes.
This presents us with a fundamental problem in judging depth; flat images are ambiguous. There are potentially countless ways of interpreting them in three dimensions. Evaluating each of these in our neural processing circuits would be slow and highly energetically costly. We therefore have evolved ‘rules’ which allow us to filter out and quickly ‘delete’ most of these options.
Psychologist Herta Kopfermann first discovered some of these rules in the 1930’s. Her subjects interpreted diagrams representing ‘real’ views of a wire cube using consistent assumptions. We all apply these rules. For instance, we assume that straight lines in an image are also straight in reality, and where the lines in an image coincide, they likewise coincide in three dimensions.
Selection favours systems that deliver the greatest fitness at the minimum cost. Using rules as neural information filters has allowed our vision to be energetically optimized. Our visual system therefore shows us only what is needed to guide our behaviour in a way which ensures that we survive long enough to reproduce.
By using rules and assumptions we can minimize the energy required to process sensory information. By assuming that the images we see are ‘real’, we do not need to evaluate alternative interpretations. Applying these rules allows us to quickly infer depth and form, and to respond rapidly to our external world. This maximises our ‘fitness’. It does not show us the ‘truth’ about what is there.
Donald Hoffman suggests that the way we assemble our inner copy of the outer world follows language-like principles. As with language, our ‘grammar’ of vision applies these visual rules and assumptions as principles. The mind also ‘bends’ these rules in circumstances where they do not ‘work’ for us.
Turning line diagrams into three-dimensional shapes gives ‘meaning’ to these drawings. Likewise, we use grammatical rules to construct meaningful phrases out of words. Our visual circuits are highly active when we speak. Visual information processing circuits may have been coopted and adapted to supply the neural capacity underpinning our speech and language ability.
Do other animals see in the same way as us?
Beyond the vertebrates, cephalopods such as the octopus have independently evolved a camera-type eye, which anatomically is near-identical to our own.
Octopus eyes capture a visually resolved, detailed image. Although they register no colour information, the way they respond to shapes and objects shows that they see patterns and textures, and also understand depth.
The similarities of their eye apparatus with ours suggests that that we process visual information in a similar way. Tests indeed imply that these remarkable invertebrates construct their perceptions with the assistance of learning and memory. The respective differences in our vision lie in what we and the octopus understand from what we see.
When humans or other primates observe making a purposeful gesture such as grabbing for food, this generates activity in the neural pathways involving mirror neurons. Mirror circuits fire in the same way when executing a meaningful movement or when observing others do the same.
For humans all movements are meaningful. We understand a mimed gesture as symbolising a ‘real’ action. If chimpanzees witness a mimed grabbing movement, their mirror reflex however, does not activate.
Similarly, we relate to drawings of objects as ‘being’ those objects, even though we know they are representations, and so are not ‘real’. Other animals do not do this. This highlights how our symbolic understanding codes our visual field with meaning.
Putting ourselves ‘into the picture’ involves creating an idea of ourselves, and combining this ‘self’ with the ‘other’ of the outer world. Putting one idea into another is a creative process, involving a thought pattern known as ‘recursion’. We use this thinking process whenever we tag our phrases together into sentences. Noam Chomsky and many other linguists consider recursive thinking as a defining characteristic of human language.
Memories are neurologically coded combinations of information. We tag an emotional response to an idea of something. The word sound which symbolizes something also provokes in us the emotional response that informs us of how we ‘relate’ to this object or idea. Learning and memory then are also forms of recursion.
Animals learn by having direct experiences and tagging them with an emotional response. For instance the rustling of undergrowth alerts a potential prey animal to what could be the presence of a predator. They access these memories by subsequently encountering similar external stimuli.
The scene before us similarly triggers an emotional response. We, however, can also choose to make associations between thoughts and experiences consciously. We are able to retrieve our memories by intention, without an external trigger.
How do we see ourselves?
There are three classes of people; those who see, those who see when they are shown, [and] those who do not see….
[Leonardo da Vinci]
Emotions process information at an unconscious, instinctual level. Speech activates our emotional neural circuits, producing the physiological responses we associate with the subjects symbolized by our words. This coding is accumulated through experience and is also inherited from our culture.
Cultural influences are adaptations that shift our perceptions. They are another form of neurological ‘rules’, and imprint behavioural responses to certain stimuli. These direct our attention towards what is important for survival in our ecological setting. This is energetically efficient, and consequently increases our ‘fitness’.
This affects not only the way we perceive things, but also may influence what we can see. The linguist Daniel Everett documents a striking example of this during his fieldwork with the Pirahã; a remote Amazonian tribe. Everett recounts an occasion where he and his family were awakened one morning by voices.
“Look! There he is, Xigagai, the spirit”
“Yes, I can see him. He is threatening us.”
“Everybody, come see Xigagai. Quickly! He is on the beach!
They followed the villagers towards the river, but once there, failed to understand what was happening.
I turned to Kóhoi, my principal language teacher, and asked “What’s up?” He was standing to my right, his strong, brown, lean body tensed from what he was looking at.
“Don’t you see him over there?” he asked impatiently. “Xigagai, one of the beings that lives above the clouds, is standing on the beach yelling at us, telling us he will kill us if we go into the jungle!”
“Where?” I asked. “I don’t see him.”
“Right there!” Kóhoi snapped, looking intently towards the middle of the apparently empty beach.
“In the jungle behind the beach?”
“No! There on the beach. Look!” he replied with exasperation.
Everett notes that often in the jungle, the Pirahãs could see snakes, spiders and other hazards, where he just saw forest.
“…but this was different. Even I could tell that there was nothing on that white, sandy beach no more than one hundred yards away. And yet as certain as I was about this, the Pirahãs were equally certain that there was something there. Maybe there had been something that I just missed seeing, but they insisted that what they were seeing, Xigagai, was still there.
Everyone continued to look toward the beach. I heard Kristene, my six-year-old daughter, at my side.
“What are they looking at, daddy?”
“I don’t know. I can’t see anything.”
[extracts in italics from Daniel Everett; Don’t sleep; there are snakes, ppxv-xvii]
We are peculiar amongst animals in how we symbolically assign meaning to our perceptions. This is perhaps most obvious in the highly biased way we choose to see ourselves.
As we look into a mirror, we see an image which is made by our unconscious mind in the same way as our mind constructs any other image. Our reflection provokes however an often substantial emotional response, which depends largely upon how we perceive our ‘self’. Our sense of self is a recursive creation. The way we feel in this moment therefore has relatively little to do with what we look like.
We choose how we respond to the circumstances of our lives. We also choose how we respond to the person returning our gaze. Are we honest with ourselves? Are we kind? These questions offer us the opportunity to choose who we become. Our self-aware ‘self’ can choose what it reflects back, and what it projects. In this way, when others see us, we offer them a very different vision.
“The eyes are the windows to your soul” [William Shakespeare]
“The face is a picture of the mind as the eyes are its interpreter.” [Cicero]
“The eye is the lamp of the body. If your eyes are healthy, your whole body will be full of light.” [Gospel of Matthew]
- Vision is a perceptive tool which we use to build our understanding of the world around us, and our relationship with it.
- Our eyes sample the view before us and compile a set of ‘snapshots’ which our mind then assembles into a coherent picture. This enables us to ‘make sense’ of our situation.
- We infer information about depth from the two-dimensional images captured by our eyes, combined with previously learned experience of the world. Since our eyes prioritise looking for social cues in the faces of others, we gather much of our spatial information from our out-of-focus peripheral vision.
- We construct this understanding of depth, and consequently of movements in space, by applying ‘rules’ as to how we interpret this visual data. This is efficient, allowing us to quickly access an energetically economical interpretation. However it can also mislead us.
- Like most other primates, we are highly attuned to colours and patterns. This acuity may reflect a time in which selection favoured the survival of our ancestors who were better able to find high-quality coloured fruits and to spot snakes and other predators in dense jungle habitats.
- Our more recent evolution has attuned our visual system to social cues. As a result our unconscious mind preferentially directs our eye movements to capture information from the faces and gestures of others.
- Other animals such as the octopus have evolved a camera-type eye. They seem to use them to harvest data in much the same way as we do, although our interpretations are unique to us.
- Our culture, beliefs and expectations about the nature of reality influence how we assemble our understanding of the world we see. This may mean that at times we either do not ‘see’ what is in front of us, or we see something that is not physically there.
- Building our image of the world is a creative process, tagging our current vision with information from our past. This affects how we understand, and ultimately how we ‘feel’ about what we are looking at.
- The face that gazes back at us from a mirror is whatever we make of it…
“I awoke only to find that the rest of the world was still asleep.” [Leonardo Da Vinci; Notebooks]
Text copyright © 2015 Mags Leighton. All rights reved.
Bassett, E.A. and Wallace, V.A. (2012) Cell fate determination in the vertebrate retina. Trends in Neurosciences 35, 565-573.
Behrens, R.R. (1998) Art, design and Gestalt theory. Leonardo 31, 299-303.
Davey, G.C.L. (1992) Classical conditioning and the acquisition of human fears and phobias: a review and synthesis of the literature. Advances in Behaviour Research and Therapy 14, 29-66.
Droit-Volet, S. and Meck, W.H. (2007) How emotions colour our perception of time. Trends in Cognitive Sciences 11, 504-513.
Everett, D. L. (2009) Don't sleep, there are snakes: Life and language in the Amazonian jungle. Vintage.
Fain, G.L. et al. (2010) Phototransduction and the evolution of photoreceptors. Current Biology 20, R114-R124.
Fernald, R.D. (2006) Casting a genetic light on the evolution of eyes. Science 313, 1914-1918.
Hisatomi, O. and Tokunaga, F. (2002) Molecular evolution of proteins involved in vertebrate phototransduction. Comparative Biochemistry and Physiology, B 133, 509-522.
Hochner, B. et al. (2006) The octopus: a model for a comparative analysis of the evolution of learning and memory mechanisms. Biological Bulletin 210, 308-317.
Hoffman, D. D. (2000). Visual intelligence: How we create what we see. Norton
Hoffman, D.D. (2010) Human vision as a reality engine. In Psychology Reader Foundation for the Advancement of Behavioral and Brain Sciences. Washington, DC
Hoffman, D.D. (2012) The construction of visual reality In Hallucinations (J.D. Blom and I. Sommer, eds), pp. 7-15. Springer Verlag.
Isbell, L. A. (2009). The fruit, the tree, and the serpent. Harvard University Press.
Kawamura, S. and Tachibanaki, S. (2008) Rod and cone photoreceptors: molecular basis of the difference in their physiology. Comparative Biochemistry and Physiology, A 150, 369-377.
Kirk, E.C. (2006) Effects of activity pattern on eye size and orbital aperture size in primates. Journal of Human Evolution 51, 159-170.
Lapate, R.C. et al. (2014) Non-conscious emotional activation colors first impressions: a regulatory role for conscious awareness. Psychological Science 25, 349-357.
Mitchell, J.F. et al. (2014) Active vision in marmosets: a model system for visual neuroscience. Journal of Neuroscience 34, 1183-1194.
Nakashima, R. and Shioiri, S. (2014) Why do we move our head to look at an object in our peripheral region? Lateral viewing interferes with attentive search. PLoS ONE 9, e92284.
Nilsson, D-E. (2009) The evolution of eyes and visually guided behaviour. Philosophical Transactions of the Royal Society, B 364, 2833-2847.
Nilsson, D-E. (2013) Eye evolution and its functional basis. Visual Neuroscience 30, 5-20.
Ogura, A. et al. (2004) Comparative analysis of gene expression for convergent evolution of camera eye between octopus and human. Genome Research 14, 1555-1561.
Pizlo, Z. et al. (2014) Making a machine that sees like us. Oxford University Press.
Pyles, J.A. et al. (2007) Visual perception and neural correlates of novel ‘biological motion’. Vision Research 47, 2786-2797.
Rachman, S. (1977) The conditioning theory of fear-acquisition: a critical examination. Behaviour Research and Therapy 15, 375-387.
Rees. B.E. (2011) Development of the retina and optic pathway. Vision Research 51, 613-632.
Reich, E.S. (2012) War of words over tribal tongue. Nature 485, 155-156.
Rolls, E.T. and Webb, T.J. (2014) Finding and recognizing objects in natural scenes: complementary computations in the dorsal and ventral visual systems. Frontiers in Computational Neuroscience 8, e85.
Ross, C.F. and Kirk, E.C. (2007) Evolution of eye size and shape in primates. Journal of Human Evolution 52, 294-313.
Seligman, M.E.P. (1971) Phobias and preparedness. Behavior Therapy 2, 307-320.
Shichida, Y. and Matsuyama, T. (2009) Evolution of opsins and phototransduction. Philosophical Transactions of the Royal Society, B 364, 2881-2895.
Siebold, A. and Donk, M. (2014) Reinstating salience effects over time: the influence of stimulus changes on visual selection behavior over a sequence of eye movements. Attention, Perception, & Psychophysics 76, 1655-1670.
Yokoyama, S. (2000) Molecular evolution of vertebrate visual pigments. Progress in Retinal and Eye Research 19, 385-419.
Zeki, S. & Nash, J. (1999) Inner vision: An exploration of art and the brain Oxford University Press.
Zeki, S. (1980) The representation of colours in the cerebral cortex. Nature 284, 412-418