# Snapshot tests pin bugs

> A test that records what the code outputs, not what is right, protects the bug. The fix then looks like a regression.

Building block · Getting the numbers right · found 2026-09-26 · by John Kingola

## What I was looking for

Why correct fixes kept failing the upstream test suites they were sent to.

## What I found

Maintainers trust a green suite. When the pinned value is wrong, the suite argues against the fix, and a reviewer reasonably asks why the expected numbers changed.

My skfolio fix had to change four pinned values that encoded the start-is-a-peak bug. In ffn, a test added a month earlier pinned the unannualised Ulcer Performance Index, so the fix rewrote two tests to land.

## The move

When a fix breaks a test, I ask where the expected value came from before I assume my fix is wrong.

Web version: https://kingola.dev/notes/snapshot-tests-pin-bugs/