Big Numbers and Powers of Ten

Scaling Laws, Made Simple · Part 1 · 5 min read

Before we can understand scaling laws, we must be comfortable with one small thing: counting in jumps of 10.

This sounds too simple to be a blog. But almost every confusing sentence in the scaling laws paper becomes easy once this one idea is solid. So let’s build it properly.

Counting in jumps of 10

Normally, we count by adding 1: 1, 2, 3, 4, 5…

But when numbers get huge, adding 1 is useless. Going from 1,000,000 to 1,000,001 changes nothing. So scientists count huge things differently: they multiply by 10 instead of adding 1.

1
10
100
1,000
10,000
100,000
1,000,000
10,000,000

Each step is one multiplication by 10. That is the whole trick.

The shorthand: 10^7

Writing 10,000,000 again and again is painful. So we use a shorthand:

10^7 means: start from 1 and multiply by 10, seven times.

10^7 = 10 × 10 × 10 × 10 × 10 × 10 × 10 = 10,000,000

An easy way to read it: the small number on top tells us how many zeros to write after the 1.

10^1 = 10 (one zero)
10^2 = 100 (two zeros)
10^3 = 1,000 (three zeros)
10^6 = 1,000,000 (six zeros - one million)
10^7 = 10,000,000 (seven zeros - ten million)
10^9 = 1,000,000,000 (nine zeros - one billion)

What is an “order of magnitude”?

Here is the phrase that appears everywhere in the scaling laws paper:

One order of magnitude = one multiplication by 10.

That’s it. It is just a fancy name for “one 10x jump”.

So when someone says “B is two orders of magnitude bigger than A”, they mean B is 10 × 10 = 100 times bigger than A. Not 2 times bigger. This is the most common mistake, so let’s say it clearly:

1 order of magnitude bigger = 10x bigger
2 orders of magnitude bigger = 100x bigger
3 orders of magnitude bigger = 1,000x bigger
7 orders of magnitude bigger = 10,000,000x bigger (ten million times)

Let’s feel how big 10^7 really is

Numbers with many zeros stop feeling real. So let’s make 10^7 (seven orders of magnitude) physical.

Money. Start with 1 rupee. Multiply by 10 seven times: 1 → 10 → 100 → 1,000 → 10,000 → 1 lakh → 10 lakh → 1 crore. Seven orders of magnitude is the gap between 1 rupee and 1 crore rupees.

Time. Start with 1 second. 10^7 seconds is about 4 months. The gap between one second and four months - that is seven orders of magnitude.

Weight. A 5-gram coin versus a 50,000-kg truck. That is seven orders of magnitude.

So when the scaling laws paper says its results hold “across seven orders of magnitude”, it is saying: the same rule worked for the coin AND for the truck. Keep this feeling - we will use it in the next blog.

One careful clarification: 10^7 vs 10^-7

A very common confusion: is 10^7 the tiny number 0.0000007?

No. 0.0000007 is 10 raised to a negative power (it is 7 × 10^-7). A negative power means we divide by 10 that many times, so the number gets smaller:

10^-1 = 0.1
10^-2 = 0.01
10^-3 = 0.001

Remember it like this:

Positive power = big number (multiply by 10 again and again)
Negative power = small number (divide by 10 again and again)

The paper uses both. Model sizes are big numbers like 10^9. The exponents in the laws are small numbers like 0.076. Both will make sense soon.

Quick recap

Scientists count huge things in jumps of 10, not steps of 1.
10^7 means seven multiplications by 10 = 10,000,000 (a 1 with seven zeros).
One order of magnitude = one 10x jump. Seven orders of magnitude = ten million times.
“Two orders of magnitude bigger” means 100x, not 2x.
Positive powers make big numbers; negative powers make tiny numbers.

Next, we will see what it means when a rule keeps working across seven orders of magnitude - and why that is so rare and special.