Uncovering the laws of human perception

Our journey started with observation on invariance (symmetry) regarding some aspects of the world. We are inspired by the phenomena under the umbrella term of perceptual constancy, where some physical quantities are always perceived consistently regardless of time, space or some other transformations.

We follow Noether’s formulation: every invariance in nature is related to its conservation law through the least action principle. By doing so, we hope to uncover the underlying law of human perception, to be able to write equations to describe and predict perceptions the same way we describe physical systems in Physics.

Perception is the mental representation of physical properties. Lift up your eyes to look straight ahead of you. If you are in your favorite reading spot, you probably see a familiar visual scene that looks all too normal. However, we may say that there are two copies of the world here: a copy of the world out there, the reality; and the copy of the world within, your subjective imagery. The copy of the world within is what we called “perception”. Note that the results of perception, called the percepts, are not the exact copy of the physical world, and may undergo some modifications (that is why an illusion occurs). Common synonyms of perception include: the perceived world, mental representation or imagery, subjective experience, …

In cognitive science, perception is categorized as a low level cognitive function, as it acts similarly to an modified sensor, by and large representing information that is available out there. We may therefore describe differnt types of perception based on their sensory modalities, such as visual perception, auditory perception, haptic perception, olfactory perception, and taste. A high level cognitive function integrates perceptions from multiple sensory modalities in order to exert some effect onto the world. Some examples of high level cognitive function includes decision making, problem solving, and self control. High level cognitive function therefore critically depends on the arrived percepts.

The goal of this project is to understand human perception, and to describe the rules and equations that generated perception. What are the equations that allow us to predict the subjective perception given the physical world? Given the noisy and complex input, what are the absolute fundamentals indispensable for perception? Can this rule describe the fast and accurate perception in our everyday life?

To be able to carry out meaningful research, it is important to pick insight-provoking phenomena. To draw a parallel, in Physics, an object by itself does not offer much incentive to be studied. In other words, if a system of interest is a ball, just by looking at the ball itself does not offer enough perspective for physicists to come up with brilliant guesses on the underlying laws of nature that are acting upon the ball. It is when the ball is set into motion that a perspective opened up, which led to the proposal of characterizing dynamics or motion of systems by energy and momentum. This may be why the field of mechanics is one of the oldest branches in physics.

Similarly, we need to take on more perspectives in order to study perception. Can we play with perception as we would play with a ball? By manipulating perception, we hope to gain much insight into the workings of perception, and to uncover the underlying laws. This is because once we set perception "into motion," we can compare two percepts. Specifically, we would like to detect a pattern, some similarities between the two percepts. We believe that these similarities are not merely empirical regularities, they hint at a deeper origin of some laws of nature.

"In the beginning was the symmetry," -- Werner Heisenberg (Winner of The Nobel Prize in Physics 1932)

For example, dropping a ball in New York should have some similarity with dropping a ball in France, even though many aspects may look different. The similarity is due to an underlying law of conservation of energy and momentum. Moreover, there are a set of fundamental variables attached to the law. The two fundamental variables for the conservation of energy are the kinetic and potential energies of the ball; while the fundamental variables for the conservation of momentum are the mass and velocity of the ball. The sceneries in New York or France, although beautiful, are not fundamental to the pattern of ball drops in both locations, and thus are allowed to differ. Therefore, identifying similarities is the first key to discover the laws of nature. Other terms to describe "similarities" include "invariances", "constancies" or "symmetries" (in a technical sense, symmetry under a certain transformation).

Perceptual constancy offers the best setup to study the intrinsic laws of perception. As we have discussed under section Motivation, if two percepts share similarities after some manipulations of the world, we are on our way to identifying the underlying laws and the corresponding fundamental ingredients that constitute our percepts.

This notion of starting with an invariance (preserved similarity even though a transformation has taken place) to derive the underlying law of nature is elegantly summarized by Noether’s theorem (Refer to the page Noether’s theorem for full details). Noether’s theorem has been widely used in physics to reveal the hidden laws of nature such as the conservation laws of energy and momentum. To apply Noether’s theorem in our research, we need to define similarities or invariances or constancies which we observe in our daily lives; and perceptual constancy is exactly the observations of similarities.

An short list of perceptual constancy in everyday life (with a focus on visual perceptual constancy) is included below. We focus on visual perception since it is the most dominant sensory perception in most cases for humans, and that we have the deepest understanding of vision compared to other sensory modalities.

• Shape constancy: the ability to label the shape as rectangular whether it is shown upright on a computer screen or lying flat on a table, slanted in depth
• Size constancy: the ability to attribute consistent size to the book regardless is placed on a table 2m away or held in hands 0.5m away
• Lightness constancy: the ability to distinguish a white object placed in a dark room as white while a black object placed under sunlight is black although the absolute light reflectance may not indicate so
• Color constancy: the ability to interpret a car as having the blue color whether it is parked under sunlight or moonlight
• Object constancy: the ability to tell that the car you saw in the side mirror is now the car in front of you

To summarize, perceptual constancy can be viewed as some transformations that humans are robust to, such that the arrived percept remains constant or invariant before and after some transformation. The more we can expand this list to include more perceptual constancies, the wider perspective we will be able to take on.

Out of the many perceptual constancies listed above, we chose to initiate our investigation by dissecting human 3D shape constancy. Humans see the world in 3D. Three dimensional perception is so natural and prevailing that we may not notice its complexity. Human retina only represents information in 2D. To arrive at a unique perception at a higher dimension (3D) scene given a lower dimension (2D) retinal image, human visual algorithm needs to pinpoint out of an infinitely many possibilities, the one perception to arrive at. Not only so, we agree that we see things the same way, suggesting that all humans picked the same perceptual solution out of the infinitely many choices. There must be a law in nature that governs this process.

The goal of this project is to identify that underlying law which enabled 3D shape perception, a law general enough to guide all humans to arrive at the same percept. We believe we have the best chance of studying this law by following Noether’s formulation: every invariance in nature is related to its conservation law through the least action principle. Refer to the Attempts page for details of our proposal, as we outline the direction of research towards to identify the conservation law for visual perception.