Understanding PyTorch Tensors

PyTorch tensors are the fundamental building blocks of deep learning models. They are multi-dimensional arrays that can reside on CPUs or GPUs and support automatic differentiation.

Creating Tensors

import torch

# Create a simple tensor
x = torch.tensor([1, 2, 3])
print(x.shape)  # Output: torch.Size([3])

# Create tensors with specific properties
zeros = torch.zeros(3, 3)
ones = torch.ones(2, 4)
random = torch.randn(3, 3)  # Normal distribution

Tensor Operations

# Mathematical operations
y = x * 2
z = x + y
w = torch.matmul(x, y)

# Moving to GPU
if torch.cuda.is_available():
    x_gpu = x.cuda()
    # or
    x_gpu = x.to('cuda')

Mathematical Concepts

A basic tensor operation follows the form y = Wx + b, where W is the weight matrix, x is the input tensor, and b is the bias term.

Automatic Differentiation

Tensors support automatic differentiation when requires_grad=True:

x = torch.tensor([2.0], requires_grad=True)
y = x ** 2
y.backward()
print(x.grad)  # Output: tensor([4.])

This automatic differentiation capability is what makes PyTorch so powerful for building and training neural networks.