
Our visual system receives on our retinas a pair of 2D projections of a 3D environment. Notwithstanding specific theoretical stances and underlying neurophysiological correlates, these approaches focus on three-dimensional (3D) or two-dimensional (2D) representations simply because our environment is 3D and its projections are 2D.

Neurophysiological studies revealed multiple representations of space in various brain areas, including ego-centric ( Gross and Graziano, 1995 Grieves and Jeffery, 2017) and exo-centric representations ( Olson, 2003). The progression from sensorimotor stage to formal operational stage takes place as ego-centric “subjective” sensorimotor representations, i.e., representations whose reference frames are based on the subject, such as head-centered representations, become coordinated across initially independent sensory spaces (e.g., ego-centric visual space, ego-centric tactile space) to give rise to exo-centric (i.e., reference-frames) based outside of the subject “objective” representations (e.g., object-centered representations) ( Piaget, 1936, 1950). According to the Piagetian theory, higher cognitive functions are built upon sensorimotor schema, which represent the operation of sensory and motor systems in space and time ( Piaget, 1936, 1950). She suggested that these systems lack the power of abstract generalization however, language and symbolic representations allow children to combine productively their core knowledge to attain abstract thoughts that form the foundations of abstract mathematical geometry ( Spelke et al., 2011). (2011) concluded that two systems of human core knowledge (i.e., ancient, innate, and universal systems) reflect the primary properties of Euclidian plane geometry. For example, based on children’s spatial reorientation and navigation behavior, Spelke et al. Given this background, it goes without saying that the concept of space also plays a major role in psychology and neuroscience. The concepts of space and time and their relationship shape theories in modern physics. Questions such as whether space is a priori or learned, whether it is objective or subjective have been debated. Indeed, the concept of space has been central to the thinking of many philosophers such as Plato, AlHazen, Descartes, Hume, and Kant to name a few ( Huggett, 1999).

In addition to being a fundamental aspect of sensorimotor behavior, the concept of space plays an important role in our understanding of higher cognitive functions. Species operate in space and their behavioral success depends on how well they process, represent, store, and recall spatial information. Here, we review research related to inherent capabilities and limitations of brain plasticity in terms of its spatial representations and discuss whether with appropriate training, humans can build perceptual and sensorimotor representations of spatial 4D environments, and how the presence or lack of ability of a solid and direct 4D representation can reveal underlying neural representations of space. A fundamental question is whether our brains are inherently limited to 3D representations of the environment because we are living in a 3D world, or alternatively, our brains may have the inherent capability and plasticity of representing arbitrary dimensions however, 3D representations emerge from the fact that our development and learning take place in a 3D world. This adaptation continues in adulthood and is quite general to successfully deal with joint-space changes (longer arms due to growth), skull and eye size changes (and still being able of accurate eye movements), etc. Three-dimensional perceptual and sensorimotor capabilities emerge during development: the physiology of the growing baby changes hence necessitating an ongoing re-adaptation of the mapping between 3D sensory representations and the motor coordinates. These 3D representations underlie our 3D perceptions of the world and are mapped into our motor systems to generate accurate sensorimotor behaviors. By using multiple cues, such as disparity, motion parallax, perspective, our brains can construct 3D representations of the world from the 2D projections on our retinas. We live in a three-dimensional (3D) spatial world however, our retinas receive a pair of 2D projections of the 3D environment.
