MLPs ignore data structure

🔴 Problem: MLPs ignore the structure of the data

Warning

Traditional neural networks (such as MLPs) require the input to be a flat vector (1D).
If the input data has dimensionality $\ge 2D$, this implies a flattening operation ($\ge 2D\to 1D$).

The flattening step leads to the loss of information encoded in the structure of the input data (prior knowledge).

❗ MLPs completely ignore the data’s structure.

$2D$ case: Images 🖼️

In the case of images, the value of a pixel is typically correlated with the values of neighboring pixels.
This local correlation underlies a spatial structure that MLPs, due to input flattening, are unable to capture.

• ❗Local correlations between pixels are lost → the spatial coherence inherent in the image disappears
• ❗The spatial prior must be learned again: the layers downstream of the input layer have to (re)learn the structure of the data
• ❗MLPs cannot learn spatial patterns, which are characteristic of structured data such as images, audio, and time series

✅ Solution

Local Receptive Field

The introduction of the Local Receptive Field concept makes it possible to preserve the spatial structure by observing sub-regions of the input.