What Is a Log-Log Plot?

Open the scaling laws paper and you will see its famous figures: dots falling on perfect straight lines. Every one of those figures is a log-log plot. In this blog, we will understand what that is and why straight lines on it are such a big deal.

First: what is a logarithm?

Do not let the word scare you. Here is the plain-words definition:

The logarithm (base 10) of a number is: how many 10x jumps it takes to get there from 1.

• To reach 100 from 1: two jumps (1 → 10 → 100). So log(100) = 2.
• To reach 1,000: three jumps. So log(1,000) = 3.
• To reach 10,000,000: seven jumps. So log(10,000,000) = 7.

The logarithm just counts the zeros. It converts “how big is this number” into “how many 10x jumps is this number”. That is all. Notice it is the exact reverse of the powers of ten from Blog 1: 10^3 = 1,000, and log(1,000) = 3.

The problem with normal plots

Why do we need this? Let’s try to draw the paper’s data on a normal graph.

We want to plot models of size 1,000 and size 1,000,000,000 (one billion) on the same x-axis. Suppose our x-axis is 10 cm wide and the billion sits at the far right end.

Where does the 1,000-parameter model sit? At 1,000 / 1,000,000,000 of the width = one millionth of 10 cm = 0.00001 mm from the left edge. Invisible. In fact, every model except the very biggest ones gets crushed into the first hair’s width of the plot.

Normal axes cannot show data that spans many orders of magnitude. The big values eat all the space.

The fix: the log axis

A log axis spaces numbers by their logarithm - by their number of 10x jumps - instead of by their raw size.

On a log axis, the tick marks read: 1, 10, 100, 1,000, 10,000… and each tick is the same distance apart. Every centimeter of the axis means “one more 10x jump”, one more order of magnitude.

Now our data breathes. The 1,000-parameter model sits at jump 3. The billion-parameter model sits at jump 9. They are 6 cm apart on a 10 cm axis, and every model in between gets its own visible place.

A log-log plot simply uses log axes for BOTH x and y. In the paper: x = model size (spanning many 10x jumps), y = error score (which also spans a big range). Both need the log treatment.

The magic: power laws become straight lines

Here is why scientists love log-log plots. Recall the signature property of a power law from the previous blog:

Every 10x jump in x multiplies y by the same fixed factor.

Now translate that into log-axis language:

• “Every 10x jump in x” = every 1 step to the right on the x-axis.
• “y is multiplied by the same fixed factor” = y moves the same fixed distance on the y-axis (because on a log axis, equal multiplications are equal distances!).

So: every equal step right causes the same equal step up (or down). Step, step, step - always the same. What shape does that trace out?

A perfectly straight line.

That is the whole magic:

Power law data on a log-log plot is a straight line. And any straight line on a log-log plot means a power law.

The straight line is the fingerprint of a power law. You cannot fake it and you cannot miss it.

Reading the line: the slope is the exponent

Two more reading skills and we can read the paper’s figures like a pro.

The slope of the line = the exponent of the power law. If each step right (10x in x) moves us down by 0.076 “multiplication units” on the y-axis, the law is y = x^(-0.076). Steeper line = bigger exponent = scaling helps faster. The paper’s lines are shallow, because its exponents are tiny - improvement is real but slow.

Straightness = the rule holds. If the dots follow the straight line across the whole plot, the same law is working at every scale shown. If the law were breaking down at some size, the dots would start bending away from the line there. In the paper’s figures, the dots stay on the line all the way to the right edge - the law had not broken yet at any size they could afford to test. That is the visual meaning of “trends spanning seven orders of magnitude with no deviation.”

Putting it all together

Now we can decode the paper’s headline picture in one breath:

Dots = trained models of many sizes. Axes = log-log. Dots fall on a straight line = performance follows a power law. Shallow slope = small exponent, so improvement per 10x is modest. Straight all the way = the law holds across the entire tested range. Extend the line past the data = a prediction for models nobody has built yet.

One straight line, and it says all of that.

Quick recap

• log(number) = how many 10x jumps from 1. It just counts the zeros.
• Normal axes crush data that spans orders of magnitude; log axes give every 10x jump equal space.
• A log-log plot uses log axes for both x and y.
• Power law ⇔ straight line on a log-log plot. The slope of the line is the exponent.
• Straight dots across the whole plot = the same law holds at every tested scale.

Next, we finally turn to language models themselves and define the thing all these laws predict: the loss.