← Back to DevBytes
Developer Tutorial August 11, 2026 9 min read

TensorFlow from Scratch: Hands-On Tutorial:

Introduction to TensorFlow

TensorFlow is an open-source machine learning framework developed by Google that has become one of the most popular tools for building and deploying machine learning models. Whether you are working on deep neural networks, linear regression, or production-grade AI systems, TensorFlow provides a flexible ecosystem of tools, libraries, and community resources that make it easier to build and scale ML-powered applications.

Originally released in 2015, TensorFlow has evolved significantly. With the introduction of TensorFlow 2.x, the framework adopted eager execution by default, integrated the high-level Keras API, and simplified many workflows that were previously cumbersome. This tutorial will walk you through the fundamentals of TensorFlow from scratch, with hands-on examples you can run immediately.

Why TensorFlow Matters

Before diving into code, it is important to understand why TensorFlow has earned its place as a cornerstone of modern machine learning development. The framework matters for several key reasons:

Setting Up Your Environment

To follow along with this tutorial, you need a working Python environment with TensorFlow installed. The easiest way to get started is using pip within a virtual environment. This isolates your dependencies and prevents conflicts with other projects.

# Create a virtual environment
python -m venv tf-env

# Activate it (Linux/macOS)
source tf-env/bin/activate

# Activate it (Windows)
tf-env\Scripts\activate

# Install TensorFlow
pip install tensorflow

# Verify the installation
python -c "import tensorflow as tf; print(tf.__version__)"

If the installation was successful, the last command will print the installed TensorFlow version. For this tutorial, we assume TensorFlow 2.x, which is the default version installed via pip today.

Understanding Tensors

At the heart of TensorFlow is the tensor, a multi-dimensional array that flows through the computational graph. If you have used NumPy, tensors will feel familiar. The main difference is that TensorFlow tensors can run on accelerators like GPUs and can participate in automatic differentiation for training neural networks.

Let us start by creating some basic tensors and performing operations on them.

import tensorflow as tf

# Create a constant tensor
a = tf.constant([[1, 2], [3, 4]])
b = tf.constant([[5, 6], [7, 8]])

print("Tensor a:")
print(a)
print("Shape:", a.shape)
print("Dtype:", a.dtype)

# Basic operations
c = tf.add(a, b)
d = tf.multiply(a, b)
e = tf.matmul(a, b)

print("Addition result:")
print(c)
print("Element-wise multiplication:")
print(d)
print("Matrix multiplication:")
print(e)

Notice that we did not need to run a session or build a graph explicitly. TensorFlow 2.x uses eager execution by default, meaning operations are evaluated immediately, just like regular Python code. This makes debugging and experimentation much easier.

Variables in TensorFlow

While constants are immutable, variables are mutable and are typically used to store model parameters such as weights and biases. Variables persist across function calls and are optimized during training.

# Create a variable
weights = tf.Variable(tf.random.normal([3, 2]), name="weights")
bias = tf.Variable(tf.zeros([2]), name="bias")

print("Weights:")
print(weights)
print("Bias:")
print(bias)

# Update a variable
weights.assign([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
print("Updated weights:")
print(weights)

Building Your First Neural Network

Now that you understand tensors and variables, let us build a neural network. TensorFlow includes the Keras API, which provides a high-level interface for defining, compiling, and training models. We will use the famous MNIST dataset of handwritten digits to train a model that can classify images.

Loading and Preparing Data

Data preparation is a critical step in any machine learning pipeline. For MNIST, we need to load the dataset, normalize pixel values to the range [0, 1], and reshape the data appropriately.

import tensorflow as tf

# Load the MNIST dataset
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()

# Normalize pixel values to [0, 1]
x_train, x_test = x_train / 255.0, x_test / 255.0

print("Training data shape:", x_train.shape)
print("Training labels shape:", y_train.shape)
print("Test data shape:", x_test.shape)
print("Number of classes:", len(set(y_train)))

The training set contains 60,000 images of size 28x28 pixels, and the test set contains 10,000 images. Each pixel value is an integer between 0 and 255, which we scale to a float between 0 and 1 for better training stability.

Defining the Model

With Keras, defining a model is straightforward. We use the Sequential API to stack layers. For this example, we will flatten the input images, pass them through a dense hidden layer with ReLU activation, and output probabilities across 10 classes using softmax.

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation='softmax')
])

# Display the model architecture
model.summary()

The Flatten layer reshapes each 28x28 image into a 784-dimensional vector. The Dense layer with 128 units learns representations of the input. Dropout randomly disables 20% of neurons during training to reduce overfitting. The final Dense layer outputs a probability distribution over the 10 digit classes.

Compiling and Training the Model

Before training, we compile the model by specifying the optimizer, loss function, and metrics we want to track. Then we call the fit method to train the model on the training data.

model.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

# Train the model
history = model.fit(
    x_train, y_train,
    epochs=5,
    validation_split=0.1,
    batch_size=32
)

The Adam optimizer is a popular choice because it adapts the learning rate during training. Sparse categorical crossentropy is used because our labels are integers (0 through 9) rather than one-hot encoded vectors. Training for 5 epochs is enough for this simple dataset to achieve high accuracy.

Evaluating the Model

After training, we evaluate the model on the test set to measure its performance on unseen data.

test_loss, test_accuracy = model.evaluate(x_test, y_test, verbose=2)
print(f"Test accuracy: {test_accuracy:.4f}")
print(f"Test loss: {test_loss:.4f}")

You should expect a test accuracy of around 97% to 98%. This is a strong result for a simple feedforward network, though convolutional networks can achieve even higher accuracy on MNIST.

Making Predictions

Once the model is trained, you can use it to make predictions on new data.

import numpy as np

# Get predictions for the first 5 test images
predictions = model.predict(x_test[:5])
predicted_classes = np.argmax(predictions, axis=1)
actual_classes = y_test[:5]

print("Predicted classes:", predicted_classes)
print("Actual classes:   ", actual_classes)

Custom Training with GradientTape

While the Keras fit method is convenient, sometimes you need more control over the training loop. TensorFlow provides tf.GradientTape for automatic differentiation, allowing you to write custom training loops. This is useful for research, complex architectures, or specialized training procedures.

import tensorflow as tf

# Define a simple model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(64, activation='relu', input_shape=(4,)),
    tf.keras.layers.Dense(3, activation='softmax')
])

# Loss and optimizer
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy()
optimizer = tf.keras.optimizers.Adam()

# Dummy training data
x_batch = tf.random.normal((32, 4))
y_batch = tf.random.uniform((32,), minval=0, maxval=3, dtype=tf.int32)

# Custom training step
def train_step(x, y):
    with tf.GradientTape() as tape:
        predictions = model(x)
        loss = loss_fn(y, predictions)
    gradients = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(gradients, model.trainable_variables))
    return loss

# Run a few training steps
for step in range(10):
    loss = train_step(x_batch, y_batch)
    print(f"Step {step}, Loss: {loss.numpy():.4f}")

The GradientTape records operations for automatic differentiation. When we call tape.gradient, TensorFlow computes the gradients of the loss with respect to the model's trainable variables. The optimizer then applies these gradients to update the weights. This pattern gives you full control over every aspect of training.

Saving and Loading Models

Saving your trained models is essential for deployment and for resuming training later. TensorFlow provides several formats for saving models. The recommended approach is to save the entire model, including architecture, weights, and optimizer state.

# Save the entire model
model.save('my_model.keras')

# Load the model
loaded_model = tf.keras.models.load_model('my_model.keras')

# Verify it works
result = loaded_model.predict(x_test[:1])
print("Prediction from loaded model:", np.argmax(result))

You can also save just the weights if you prefer to manage the architecture separately. This is useful when you want to experiment with different architectures while reusing trained weights.

# Save only weights
model.save_weights('my_weights.weights.h5')

# Create a new model with the same architecture
new_model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation='softmax')
])

# Load the weights into the new model
new_model.load_weights('my_weights.weights.h5')

Using TensorBoard for Visualization

TensorBoard is a powerful visualization tool that helps you understand and debug your models. It can display metrics like loss and accuracy over time, visualize the model graph, and show histograms of weights and biases.

import tensorflow as tf
import datetime

# Set up a log directory
log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(
    log_dir=log_dir,
    histogram_freq=1
)

# Train with TensorBoard callback
model.fit(
    x_train, y_train,
    epochs=5,
    validation_split=0.1,
    callbacks=[tensorboard_callback]
)

To view the TensorBoard dashboard, run the following command in your terminal and navigate to the provided URL in your browser:

tensorboard --logdir logs/fit

Best Practices

As you become more comfortable with TensorFlow, following best practices will help you write cleaner, more efficient, and more maintainable code. Here are some key recommendations:

Here is an example of using tf.data and callbacks together for a more robust training setup:

# Create a tf.data pipeline
train_ds = tf.data.Dataset.from_tensor_slices((x_train, y_train))
train_ds = train_ds.shuffle(10000).batch(32).prefetch(tf.data.AUTOTUNE)

test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test))
test_ds = test_ds.batch(32).prefetch(tf.data.AUTOTUNE)

# Define callbacks
callbacks = [
    tf.keras.callbacks.EarlyStopping(
        patience=3,
        restore_best_weights=True
    ),
    tf.keras.callbacks.ReduceLROnPlateau(
        factor=0.5,
        patience=2
    )
]

# Train with the data pipeline and callbacks
model.fit(
    train_ds,
    epochs=20,
    validation_data=test_ds,
    callbacks=callbacks
)

Conclusion

TensorFlow is a powerful and versatile framework that can take you from simple tensor operations all the way to production-scale machine learning systems. In this tutorial, you learned the fundamentals of tensors and variables, built and trained a neural network using the Keras API, wrote a custom training loop with GradientTape, saved and loaded models, visualized training with TensorBoard, and explored best practices for building robust ML workflows. The best way to deepen your understanding is to apply these concepts to your own projects, experiment with different architectures and datasets, and gradually explore more advanced topics like convolutional networks, recurrent networks, and deployment with TensorFlow Serving or TensorFlow Lite. With the foundation you now have, you are well equipped to continue your journey into the world of machine learning with TensorFlow.

— Ad —

Google AdSense will appear here after approval

← Back to all articles