The Visual System

Of the seven main brain networks, the visual system’s name best describes its function as “the observer.”

The visual system controls sight and visual processing, making the system simple to understand even though it handles complicated tasks.

It is a highly complex series of areas that work toward the common goal of turning light into something recognizable. Though at first it appears to be a single region of the brain, the visual system is made up of multiple, neighboring functional areas that work collectively for cohesive visual processing.

Together, they:

• • Engage in visual processing
  • Analyzing motion
  • Recognize patterns, textures, and faces
  • Identify the location of objects, buildings, or people in space
  • Determining the function and permanence of objects
  • Aid in problem solving
  • Reinforce visually learned behaviors

Today, modern neuroscience continues to explore how the visual system makes sense of the world around us.

The visual system can quickly identify visual changes, consistencies, or inconsistencies that aid in problem solving and evaluating the world around us.

To break this down, take a look at the following pattern-based question. Which image comes next in the sequence?

This problem could be solved mathematically by assigning each vertex of the pentagon a numerical value and determining the relationship between vertex 1 and 4, 2 and 5, 3 and 1, etc.

Without mathematics, the problem can be visually solved by recognizing that the arrow moves one vertex clockwise in each sequential image. This indicates that the next image would be image B.

The visual system is responsible for making sense of what we see, but it can also be tricked based on what it thinks it perceives — giving us optical illusions. 

This optical illusion plays with the visual system’s ability to process light, dark, and linear elements. 

At first glance, the grey lines in this optical illusion appear to be slanted, but when you look more closely, you can see that the grey lines are parallel and the squares are equal size. (If your eyes won’t stop playing tricks on you, cover all but the top row with a piece of paper and compare the size of the squares.)

With the squares shifted off-center to the left or right, they give a warped perception that the grey lines are changing in perspective or growing in size. This is due to our visual system’s ability to process depth perception¹ using size and perspective cues.² In these examples, and in most of our daily visual processing, the visual system analyzes images and movement to make sense of what we see. This can be applied to learning movement behaviors, understanding maps, reading written text,³ and the recognition of facial expressions.⁴ Each of these functions is based on patterns that can be identified in a single moment or over the course of an individual's learned experience.

The visual system’s pattern recognition capabilities have also become a key basis for creating computer-based neural networks. Developers model machine learning algorithms after human visual processes to design computer-based image recognition or object identification (more on this later).

To understand how the visual system can be leveraged for computer technology, it’s important to first look at how the system transfers information from the eyes throughout the occipital lobe.

The visual system is one of the older brain networks, having been studied since before the turn of the 20th century. German physiologist, Hermann Munk, identified the occipital cortex as central to vision capabilities in 1881 through experiments on monkeys and dogs,⁵ and he first determined a connection between the eyes and the rear of the brain.

The Russo-Japanese War and World War I later led to increased awareness of the visual cortex as military doctors studied visual impairments — including blindness — in soldiers who had suffered rear-brain injuries.⁶ Japanese physician, Tatsuji Inouye, was the first to develop retinotopic maps of the visual cortex,⁷ which were reinforced by subsequent wartime studies.^8,9

Using this foundation, neuroscience has continued to map the relationship between the eyes, the visual pathway, and the visual cortex. The visual pathway extends from the retina through the optic nerves¹⁰ and the lateral geniculate nuclei (LGN) to the visual cortex. Within the visual cortex, information is then processed through a series of closely connected — but functionally selective¹¹ — nodes in the brain.

Each of these nodes are functionally and structurally connected (see Connectomics) as distributed networks that make up the dorsal and ventral pathways.

The functional nodes of the visual system have a hierarchical organization for how they distribute and process information through the dorsal and ventral visual pathways.

Upon receiving visual information from the retina, imagery is initially processed in the primary node of the visual cortex, V1. It is then passed through the connected nodes V2 and V3, and through the dorsal or ventral visual pathways where task segregation occurs in highly specialized areas¹² of the visual cortex.

The dorsal visual pathway is responsible for the perception of objects,¹³ global motion,¹⁴ and processing visual stimuli¹⁵ for a coordinated response. At the same time, the ventral visual pathway is thought to be involved in the recognition of faces, places,¹⁶ bodies,¹⁷ shapes, letters,¹⁸ and words.¹⁹

The following chart outlines how visual information can be initially processed in the primary nodes of the visual cortex, before being deployed through the dorsal and ventral pathways to highly specialized areas of the visual system.¹²

As an example, when you look at a person’s face, both the dorsal and ventral visual pathways are thought to be involved in the initial stages of processing.²¹ The specialized recognition of facial features through spatial relationships then occurs later in the dorsal visual pathway with strong functional connectivity to the fusiform face area (FFA).²²

The hierarchy of the visual system shows us that visual neural networks are closely connected, highly specialized, and synchronized.

Yet, just because the visual cortex appears relatively self-contained anatomically — unlike the default mode network (DMN), for example, which can be found in multiple separate areas in each hemisphere — it still interacts frequently with other networks.

The visual system works closely with many of the brain’s networks:

• • Dorsal attention network (DAN) for reading²³ and general visual processing²⁴
  • Sensorimotor network for early development²⁵ and imitation learning²⁶
  • Limbic system for visual memory and processing threat-based stimuli noticed in an individual’s peripheral vision⁴
  • Salience network for self-control when presented with tempting imagery,²⁷ as well as emotional empathy²⁸

These are just some of the ways the visual system primarily partners with other networks to carry out complex functions. However, there are also many occasions where this coordination breaks down causing visual impairments or mental health issues.

Many individuals find it difficult to stay focused while reading, and this may correlate with a decoupling of the visual system and the default mode network.²⁹ It has been suggested that stronger connectivity with the DMN is linked to better text processing,²⁹ otherwise it is easy for our minds to wander when we are reading.

Likewise, increased asynchrony between the visual system and the sensorimotor network has been shown to correlate with mild to severe impairment in children with autism spectrum disorder (ASD).³⁰ This impairment likely impacts motor and social skill acquisition. Evaluating brain biomarkers based on the overconnectivity or underconnectivity³¹ of networks may present an opportunity for identifying and diagnosing ASD.³²

Decreased connectivity in the visual system can also be present in individuals with schizophrenia. Diminished connectivity has been shown to correlate with the severity of psychotic symptoms,³³ and it may lead to impaired visual object perception.³³

Studying changes in functional connectivity of the visual system is also relevant to vision disorders. In patients with primary open angle glaucoma (POAG) which causes blindness, there are not only structural abnormalities³⁴ within the eyes. Atrophy also occurs in the visual cortex³² as well distant gray matter areas of the brain — including decreased functional connectivity in working memory and the dorsal attention network (DAN).³⁴ This degradation suggests that POAG could be considered a neurodegenerative condition.

While studies of the visual system are providing ongoing insight into mental illnesses and neurological conditions, understanding the system’s functional connectivity has also led to advances in computer technology.

Moving beyond our understanding of the brain, the visual system’s functional connectivity has become a model for computer-based neural networks. A neural network is a series of algorithms that can be trained to look for visual patterns in large datasets.³⁵ Using computer algorithms, we can recreate cognitive processes³⁶ like visual perception because neural networks mimic the hierarchical organization that occurs in the visual pathways.

Furthermore, the dorsal and ventral pathways present a useful model for close-proximity, small-world networks.³⁷ When developing computer-based information processing systems, the anatomically localized connections between the primary visual nodes and the highly specialized areas of the visual system are a key example for short-range connections in neural networks.

Deep learning, neural networks can then be leveraged as predictive models for image recognition³⁸ by comparing objects, faces, places, and other key tasks. Models can also be further trained to recognize patterns and movement.³⁹

Computationally modeling neural networks poses two unique opportunities:

1. It gives the ability to use machine learning algorithms to process information in a similar method as a human viewer
2. It offers better possibilities for understanding the human brain.

If we can correctly model the brain’s intricate functions, we can use computer technology to dive deeper into how the brain operates when it is healthy versus when it experiences functional problems. As neuroscience continues to study the abnormalities resulting in physical or mental health disorders, we are also paving the way for discovering possible brain-focused treatments.

The visual system has led to incredible technological advances, but other networks have also posed new opportunities for the field of neuroscience.