Skip to main content

Chapter 12.4 - Complete Weight Loading Pipeline

Overview

We have the smart weights, we have the translation book, and we have our empty PyTorch model. It's time to plug the smart brain into our empty model so it instantly becomes a genius!

🎯 Why we do it​

Rationale

This is the final step of loading pretrained weights. Once this script runs, our completely empty, random model will suddenly know how to speak perfect English, write poetry, and code, all thanks to the weights we injected into it.

🛠️ How we do it​

Methodology

We loop through all the translated weights, grab the numbers, and copy them directly into our PyTorch model's parameters. Boom! Instant genius.

import torch
import torch.nn as nn

# Our empty layer
our_layer = nn.Linear(3, 3)

# The smart weights we downloaded from OpenAI
smart_weights = torch.tensor([[1.0, 2.0, 3.0], 
                              [4.0, 5.0, 6.0], 
                              [7.0, 8.0, 9.0]])

# Injecting the smartness!
with torch.no_grad():
    our_layer.weight.copy_(smart_weights)
    
print("Our model is now super smart!")