Neural scaling law

High school (ages 12+)
Science & Technology

Name:

Date:

Step 1 · Read the article

Open the article and read it at the level above. Then answer the questions below in your own words.

https://smallwhale.org/articles/neural-scaling-law

Write what each word means, using the article.

Do not copy the article. Write two things you learned, the way you would explain them to a friend.

Answers below are taken from the article’s stored facts, so they match what the student read.

1. What is the mathematical relationship?Often described by power-law relationships, indicating consistent performance gains as factors scale
2. Why do bigger neural networks usually work better?Bigger neural networks can learn more detailed patterns, like a camera with more pixels shows a clearer picture. With more parts, the network can model more complex ideas, so it makes fewer mistakes.
3. How does the size of the training data affect a neural network?More training data shows the network many different examples, so it learns to handle new situations better and doesn’t just copy what it saw once. This reduces over‑fitting and improves how well it works on new data.

Neural Network Architectures: The specific design and arrangement of layers and nodes within a neural network model.
Training Cost: The computational resources (time, energy, processing power) required to train a neural network.
Inference Compute: The computational resources used by a trained neural network to make predictions on new data.
Emergent Capabilities: Abilities of large neural networks that are not explicitly programmed but appear as a result of scale.