How Autofocus Works on the Sensor

A grey heron stands on a mudbank 20 m away, and the Nikkor Z 180-600mm f/5.6-6.3 VR is at 600 mm and wide open at f/6.3. By the arithmetic later in this chapter, the slice of the scene that will look sharp is about 41 cm deep. When the heron flies towards you, the camera has to keep that slice on the bird while the slice gets thinner. At 10 m it is about 10 cm.

Autofocus (AF) is the camera moving the lens's focusing elements until the subject is sharp, and the Z6III does it by reading light that has already landed on the image sensor. Page numbers in parentheses point to Nikon's Z6III Reference Guide, edition 09, which matches firmware 2.00.

01.What Focusing Moves

Where the Light Converges

Light leaves a point on the subject, such as the catchlight in a dancer's eye (the tiny reflection of a light source), in every direction. The lens gathers a cone of it and bends the cone back to a point behind the lens, and where that point lands depends on how far away the subject is. The thin-lens equation describes the relationship:

$$\frac{1}{f} = \frac{1}{u} + \frac{1}{v}$$

Here $f$ is the focal length, $u$ the distance from lens to subject, and $v$ the image distance, from the lens to the point where the light converges, all in millimetres. The equation treats the lens as one sheet of glass with no thickness, so read its answers as how far the point of convergence moves, not how far any piece of glass travels. Rearranged:

$$v = \frac{f\,u}{u - f}$$

For the Nikkor Z 85mm f/1.8 S, $f = 85$ mm. With the subject at infinity, $v = 85$ mm. With a dancer at 10 m, $v = (85 \times 10000)/9915 = 85.729$ mm, and at 2 m, $v = (85 \times 2000)/1915 = 88.773$ mm. Refocusing from 10 m to 2 m pushes the point of convergence back by $88.773 - 85.729 = 3.044$ mm. The sensor is fixed, so the lens must bring that point onto it, often by moving one small internal group.

The long zoom has a bigger job. At 600 mm, a heron at 50 m converges at 607.287 mm and one at 20 m at 618.557 mm, a difference of 11.27 mm.

How Close Is Close Enough

If the light converges slightly in front of or behind the sensor, the point is recorded as a disc. Write $D$ for the diameter of the lens opening and $\delta$ (delta) for the gap between the convergence point and the sensor. Similar triangles give the blur disc $B$:

$$B = \frac{D\,\delta}{v} \approx \frac{\delta}{N}$$

where $N = f/D$ is the f-number, and the second step uses $v \approx f$, true when the subject is many focal lengths away.

Photographers accept a disc up to some diameter as sharp, called the circle of confusion and written $c$. For full frame, 0.03 mm is a common convention for viewing a whole print, not a figure Nikon publishes, and anyone inspecting files at full magnification will want less. Setting $B = c$ gives the depth of focus, how far the convergence point may sit either side of the sensor:

$$\delta_{\max} = \pm N c$$

At f/1.8 that is $\pm 1.8 \times 0.03 = \pm 0.054$ mm, and at f/6.3 it is $\pm 0.189$ mm. With the 85 mm wide open, the lens carries the convergence point 3.044 mm and must stop within 0.054 mm of the sensor, under 2 percent of the trip, since $0.054 / 3.044 = 0.018$.

Depth of Field at Long Focal Lengths

Depth of field is the subject-side counterpart: the range of distances that look sharp. When the subject is many focal lengths away but well short of the distance at which everything to infinity turns sharp, a good approximation is

$$\text{depth of field} \approx \frac{2 N c\, u^2}{f^2}$$

Doubling the distance gives four times the depth, doubling the focal length gives a quarter, and the depth grows in proportion to the f-number. For the heron, $2 \times 6.3 \times 0.03 \times 20000^2 / 600^2 = 0.378 \times 400{,}000{,}000 / 360{,}000 = 420$ mm. The full thin-lens calculation gives sharpness from 19.80 m to 20.21 m, about 41 cm.

The portrait at 2 m has less depth than the gap from nose to ear, and a long lens close to a moving subject is the hardest work autofocus does.

02.Phase Detection

Two Views Through One Lens

Hold a finger at arm's length and close each eye in turn. The finger jumps against the background, because each eye sees from a different place. A lens is a viewpoint too, and a wide one: the 85 mm at f/1.8 has an opening $85 / 1.8 = 47.2$ mm across.

Phase detection exploits that. Some pixels are built to collect light mainly from one side of the lens opening and others from the opposite side, usually by shading half of each pixel or by splitting it in two under one tiny lens. The Reference Guide calls the Z6III a hybrid of phase detection and contrast detection (p. 1002) and says nothing about its focus pixels, so what follows is the general principle, not Nikon's circuit.

Why a Defocused Point Appears Twice

When the catchlight is in focus, every ray from it meets at one spot on the sensor, whichever half of the lens it came through, so both sets of pixels see it in the same place. When it is out of focus, the rays reach the sensor before they have met or after they have crossed. Light from one half of the lens lands on one side of the blur disc and light from the other half on the other side, so each set of pixels sees its own copy, and the gap grows with the focus error.

The order of the copies gives the direction. A subject nearer than the set distance would converge behind the sensor, so its two bundles arrive uncrossed. A farther subject has already converged and crossed, so the copies arrive swapped. Which copy sits on which side tells the camera whether to drive nearer or farther.

Reading Direction and Distance

Model each half of a round opening as a half-disc. The balance point of a half-disc of radius $R$ sits $4R/(3\pi)$ from its straight edge, so the two balance points are separated by

$$b = 2 \times \frac{4R}{3\pi} = \frac{4D}{3\pi} \approx 0.42\,D$$

This baseline $b$ plays the part of the distance between your eyes. The same similar triangles give the separation $s$ of the two copies:

$$s = \frac{b\,\delta}{v}$$

Because $s$ is proportional to $\delta$, the size of the shift says how far focus is off and its sign says which way.

Take the dancer. The 85 mm at f/1.8 is set for 10 m and she steps to 2 m. Then $D = 47.22$ mm and $b = 0.4244 \times 47.22 = 20.04$ mm. Her light converges $\delta = 3.044$ mm behind the sensor at $v = 88.773$ mm, so

$$s = \frac{20.04 \times 3.044}{88.773} = 0.687 \text{ mm}$$

The sensor is 35.9 mm wide (p. 997) and a full-frame picture 6048 pixels wide (p. 998), so pixels sit about $35.9 / 6048 = 0.00594$ mm apart and the shift is about 116 pixels. That spacing is only a ruler, as Nikon does not publish its focus-pixel layout.

That is the great advantage of phase detection. One reading gives a direction and a distance estimate, so the lens can be sent most of the way at once instead of searching.

Precision follows from the same model. At the edge of acceptable focus $\delta = Nc$, so for the 85 mm at f/1.8, $s = 20.04 \times 0.054 / 88.773 = 0.0122$ mm, about 2 pixels. For the 180-600mm at 600 mm, f/6.3 and 20 m, $D = 95.24$ mm, $b = 40.42$ mm and $s = 40.42 \times 0.189 / 618.557 = 0.0124$ mm, again about 2 pixels. In this idealised model, where each pixel sees exactly half the opening, the shift is always about 0.42 of the blur disc, so a slower lens with a narrower baseline also tolerates more error, and the camera must resolve roughly the same two pixels either way.

What Phase Detection Cannot See

The camera finds the shift by sliding one brightness strip along the other until they line up. A clear sky or plain wall lines up at every shift. A row of identical railings lines up at several, so the answer is ambiguous. A line running in the same direction as the slide looks identical wherever it is slid. The Reference Guide warns that the camera may fail on lines that run parallel to the frame's long edge (p. 122). Nikon gives no reason. My reading is that the Z6III compares its views along the long edge, so a horizontal line in a level frame offers nothing to measure.

03.Contrast Detection

Climbing to the Sharpest Setting

The older method measures sharpness itself. Neighbouring pixels differ strongly at a sharp edge and less as focus drifts. Contrast detection moves the lens a step, scores the sharpness, and keeps going while the score rises, like finding a hilltop in fog by feel. A single reading gives no direction, so the first step may go the wrong way, and the camera only knows it has passed the peak once the score falls. The back-and-forth that results is called hunting. In return, the method measures sharpness on the very pixels that form the picture. This is how contrast detection works in general, not a description of Nikon's code.

What Nikon Says About the Hybrid

The specification describes a hybrid phase-detection and contrast system "with AF assist" (p. 1002) but does not define that last phrase, and I take it to mean the built-in AF-assist illuminator (p. 643). Nikon's launch announcement adds that the partially stacked sensor and the EXPEED 7 processor, Nikon's image-processing chip, give "shorter AF calculation cycles", and that the low end of the detection range reaches −10 EV (exposure value, a brightness scale explained under Light below), which Nikon says exceeds the Z9 and Z8.

Nikon does not say when the Z6III uses phase detection, when it uses contrast, or how it blends them. The general case for combining them is that phase detection supplies speed and direction while contrast detection confirms sharpness on the picture's own pixels and can use some detail that phase detection cannot. That is engineering reasoning, not a published description of this camera.

Predicting a Moving Subject

The lens servo specification lists "predictive focus tracking" (p. 1002) without explaining it. The idea behind any such system is to estimate from recent readings where the subject will be when the shutter opens, and drive there rather than to where it was.

The arithmetic shows why. The convergence point moves about $m^2$ times as fast as the subject, where $m = f/(u - f)$ is the magnification. Suppose the heron flies straight at you at 10 m/s, a round number chosen for illustration. At 20 m, $m = 600 / 19400 = 0.0309$ and $m^2 = 0.000957$, so the convergence point moves at $0.000957 \times 10000 = 9.57$ mm/s. The whole depth of focus at f/6.3 is $2 \times 0.189 = 0.378$ mm, crossed in $0.378 / 9.57 = 0.040$ s. At 10 m, $m = 600 / 9400 = 0.0638$ and $m^2 = 0.00407$, so the point moves at 40.7 mm/s, and the crossing takes $0.378 / 40.7 = 0.0093$ s. A lag of 9 ms between measuring and exposing would lose the bird if the camera aimed where it had been. Continuous focusing belongs to Focus Modes.

04.What Limits Autofocus

Light

The specification gives an autofocus detection range of −10 to +19 EV (p. 1002). On the EV scale one step doubles or halves the light, and at a sensitivity of ISO 100, $EV = \log_2(N^2/t)$, with $t$ the shutter time in seconds, as Exposure Modes derives.

To feel how dark −10 EV is, solve for $t$ with an f/1.2 lens at ISO 100: $t = N^2 \times 2^{10} = 1.44 \times 1024 \approx 1475$ s, about 24.6 minutes. At ISO 25600, eight stops more sensitive, it is still $1475 / 256 \approx 5.8$ s. At the top, +19 EV at f/16 and ISO 100 means $t = 256 / 2^{19} = 1/2048$ s. The sunny-16 rule, f/16 at 1/125 s and ISO 100, puts sunlight at $\log_2(256 \times 125) \approx 15$ EV, so the top of the range is about four stops above full sun.

Nikon attaches five conditions to the figure.

• Photo mode. No figure is given for video.
• ISO 100 equivalent. I read this as fixing the scale of the EV numbers, not as an instruction to shoot at ISO 100.
• 20 °C (68 °F). Nikon does not say how heat or cold changes the range.
• Single-servo AF (AF-S). No figure is given for continuous-servo AF (AF-C), so do not assume it reaches −10 EV.
• A lens with an f/1.2 maximum aperture. None of the author's lenses opens that wide.

The metering range, −4 to +17 EV, assumes an f/2.0 lens (p. 1001), so the two ranges do not compare directly. Fewer photons make noisier brightness strips, which makes the line-up less certain.

Aperture

A slower lens costs light. Against f/1.2, f/1.8 loses $2\log_2(1.8/1.2) = 1.17$ stops, f/2.8 loses 2.44 and f/6.3 loses 4.78. If the low-light limit depended only on light reaching the sensor, it would sit near −8.8 EV with the 85 mm, −7.6 EV with the Tamron and −5.2 EV with the 180-600mm at 600 mm. Nikon publishes no such figures. They are my estimate on that stated assumption.

The narrower baseline costs little, since the smaller shift comes with a larger tolerance, so in the idealised model f/6.3 is paid for mainly in light.

Contrast and Detail

Both methods need edges, one to line up and the other to produce a clear peak. A grey heron against grey mud under cloud offers little of either, and a subject short of contrast is on the Reference Guide's list of things that can defeat autofocus (p. 122). The eye, the bill, or the line of the neck against the sky usually gives the camera a usable edge at the same distance.

The Scenes Nikon Warns About

The Reference Guide lists eight situations in which the camera may be unable to focus (p. 122).

The same page adds that the display may brighten or darken while focusing, that focusing may take longer in poor light, and that points at the display's edge can struggle, perhaps because the lens barrel clips one half of the opening more there, though Nikon gives no reason. Most important, the focus point may turn green when the camera failed to focus.

Focus Points and Coverage

The specification gives 273 focus points with single-point AF and 299 with auto-area AF, the mode in which the camera picks the point itself, both counted in photo mode with the FX image area, Nikon's name for full frame (p. 1002). It gives no autofocus coverage figure, meaning no statement of how much of the frame the points span, so I give none. Wide-area AF (C1) and Wide-area AF (C2) let you set the focus area's size in focus points, up to 21x13 in FX photo shooting (p. 125). Nikon does not say which number is the width, but the video maximum of 21x11 shrinks only the second number for the wider 16:9 frame, so I read 21 as points across. As $21 \times 13 = 273$, I infer a grid 21 points wide and 13 high, which Nikon does not state. Custom Setting a4 Focus points used can cut the selectable points to a quarter, leaving Pinpoint AF, a point smaller than the usual single point, unaffected (p. 634), as The Focus Custom Settings explains.

05.What to Do When Autofocus Struggles

The first fix is to move the point onto a textured edge, not parallel to the long side, at the subject's distance. Point size belongs to AF-Area Modes, and finding eyes to Subject Detection.

When autofocus has trouble, Nikon recommends focus lock (p. 134). Nikon's steps are to focus on the subject, lock, recompose and shoot. If the subject itself offers nothing to grip, my own workaround is to focus and lock on something else at the same distance. In AF-S a half-press holds focus, and pressing the centre of the sub-selector, the small joystick on the back, locks it in AF-S or AF-C (pp. 134-135). Locking with the sub-selector also locks exposure, and Nikon rules out Auto-area AF for focus lock (p. 134).

Nikon suggests manual focus when autofocus does not give the result you want (p. 136). The electronic rangefinder shows whether focus is on the subject, in front of it or behind it (pp. 136-137), the same kind of directional answer phase detection gives, though Nikon does not say what drives it. With subjects unsuited to autofocus the in-focus indicator can appear when the subject is not sharp, so zoom in on the view to check, and use a tripod (p. 137). The Reference Guide suggests Custom Setting d11 Starlight view (photo Lv) set to ON for better focus in low light, in photo mode only (p. 122). Lv means live view, and the setting brightens the display, which may then move jerkily and loses its histogram (p. 675). The Built-in AF-assist illuminator (Custom Setting a11), set to ON, lights a lamp on the body as needed in poor light, in photo mode with AF-S (p. 643). Its reach is about 1 to 3 m, lens hoods should come off, and some lenses block it (p. 643). Nikon advises OFF when focusing on animals or birds, because the light may harm their eyes (p. 129).

06.Recommended Settings

For birds with the 180-600mm, depth collapses to about 10 cm at 10 m, so I would rather start tracking while the bird is distant than ask the camera to find it close.

For stage dance with the Tamron at f/2.8, spotlights and LED (light-emitting diode) fixtures that may pulse match three of Nikon's warnings at once: changing light, hard bright and dark areas, and flicker. Aim at a costume edge or a face rather than a bright patch.

For portraits with the 85 mm at f/1.8, about 6 cm of depth at 2 m leaves little room, so let Auto-area AF with People find the eyes and switch to the near one rather than recomposing.

07.Beginner Mistakes

Aiming at a horizontal line. A wire, horizon or stage edge can give the camera nothing to measure (p. 122). Use an edge that crosses the long side.

Using a focus point bigger than the subject. A small bird in branches loses to the branches (p. 122). Shrink the area.

Trusting a green point. It can show green when focus failed (p. 122), and the manual focus indicator can mislead on hard subjects (p. 137). Zoom in and look.

Expecting −10 EV from a slow zoom. That figure needs an f/1.2 lens, AF-S and photo mode (p. 1002).

Leaving the AF-assist illuminator on around animals. Nikon advises turning it off for animals and birds (p. 129), and its 3 m reach would not help a distant subject anyway (p. 643).

Recomposing after the distance changed. Focus lock holds a distance, not a subject. Once the gap between you and the subject changes, let go of the lock and refocus (p. 134).

08.Related Chapters

• Focus Modes: single, continuous and full-time AF, and manual focus.
• AF-Area Modes: how big the focus area is and how it moves.
• Subject Detection: letting the camera find people, animals and vehicles.
• Back-Button Focus: focusing with a button other than the shutter release.
• The Focus Custom Settings: the autofocus Custom Settings.
• Focus Limiter, Saved Positions and Manual Focus: narrowing the search and focusing by hand.
• Choosing an Autofocus Setup: combinations for each kind of shooting.
• What a Partially Stacked Sensor Changes: the sensor Nikon credits, with EXPEED 7, for shorter AF calculation cycles.
• Shutter Types, Rolling Shutter and Flicker: why pulsing lamps make bands.