What are tensors?

Tensors play a pivotal role in AI and deep learning systems, and share a common heritage with both physics and advanced mathematics.

All of which makes it extremely difficult to lock down a definitive meaning of the word.

At their most basic level, tensors are a data roadmap, but one that is multi-dimensional - this ability to define, store and process data in precise multi-dimensional terms make tensors a key part of the AI landscape. Without tensors it would not be possible for neural networks to function.

Tensors represent multi-dimensional data points, for example as one-dimensional, two-dimensional, three-dimensional or more.

In practical terms, tensors give neural network specialists a way to represent and compute data on multiple dimensions.

This is of crucial importance, because without this ability AI models would be unable to reference the real world environment in their computations.

The versatility of tensors is one of their most valuable features.

They can represent individual data points as 1D arrays, manage matrices as 2D arrays for image recognition or text processing, and even handle higher-dimensional data like video or time-series data.

Practical applications

Even something as simple as a video clip needs to be understood in multidimensional terms. And not just three dimensions either, when you include time in the equation.

Training a neural network to generate and manipulate material like video requires a solid understanding of the context of dimensions.

This ability to understand dimensions becomes even more important when AI transitions fully into the robotic world.

This is where it's going to be absolutely crucial for models to understand the world context in which they operate. Otherwise they will never be able to accomplish tasks efficiently and competently.

Tensors provide a consistent map of real world elements that the neural network can understand.

For instance, in image recognition, tensors are used to represent pixel data as matrices, which are then processed by neural networks to classify images.

Similarly, in natural language processing, tensors can represent word embeddings or sequences of text.

The ability to work with tensors is fundamental in order to build models that can analyze and make decisions on large datasets.

Two of the most popular tools for this purpose are PyTorch which was developed by Facebook, and TensorFlow which emerged from the labs at Google.

These frameworks, among others, provide a quick and ready way for data scientists and machine learning specialists to manage the creation and manipulation of multi-dimensional data in neural networks.

A question of definition

It might be worth mentioning at this point that tensors have different definitions depending upon your school of science.

That's because of the aforementioned connection with physics, mathematics and computer science.

For the computer scientist, and especially the AI specialist, tensors are a way to plot dimensionality in a neural network.

For the mathematician, a tensor is a very complicated formula evolving vectors, matrices and scalars.

But for experts in physics and engineering a tensor is something that transforms like a tensor. Which sounds very confusing, and probably is.

One interesting thing to note. The reason AI is so heavily dependent on powerful graphical processing units (GPU) is because these boards are designed specifically to manipulate data in very large 3D matrices (e.g. complex, intricate high resolution scenery in a racing game), which is perfect for the kind of work that an AI neural network will typically have to process in the real world.

So in summary we can say that a tensor, in AI terms, is just a container for numerical data. This mapped data is then used by the AI model to analyze, differentiate, generate or just generally process various inputs from the user.

And if this all sounds rather impenetrable, you're in good company. For those of us without a doctorate in mathematics or particle physics, understanding tensors is rather like a field mouse understanding Netflix. Tricky.