Texas Sharpshooter
Also known as: tss, sharpshooter
Finding a pattern in the data first, then presenting it as the thing you set out to test.
Share: also:
In plain terms
The name comes from a marksman who empties his rifle into the side of a barn, walks over, finds the tightest cluster of holes, and paints a bullseye around it. The shooting was random. The target was drawn afterwards. Anyone arriving later sees a bullseye full of holes and concludes he can shoot.
Data works the same way. Look at enough numbers and some of them will cluster, spike, or line up beautifully by chance alone. If you go looking for a pattern, find one, and then describe it as the pattern you predicted, you have painted the target after firing.
The tell is the order of operations. Not "did they find something interesting?" but "was the claim in place before they looked?"
Why it matters
Chance produces clusters. Real randomness is lumpy, not evenly spread, so a map of anything will always have hot spots somewhere. Scatter 3,000 towns across a country and some will show elevated rates of a rare illness this decade, no cause required. That's why suspected disease clusters are so hard to evaluate: the cluster is real, and it is also exactly what chance predicts, and telling those apart takes evidence from outside the cluster that suggested the question.
The formal version is the multiple comparisons problem. Test one hypothesis at the usual 5% threshold and you accept a 1-in-20 chance of a false alarm. Test twenty and you should expect roughly one false alarm even if nothing is going on. The published finding is the barn wall with a circle painted on it, and the nineteen other tests never get mentioned.
When the searching happens deliberately, it's called p-hacking: slicing the data, dropping outliers, swapping the outcome measure, adding covariates, until something clears the threshold. It rarely feels like cheating from the inside. It feels like being thorough.
Canonical example
"We tested the redesigned checkout page. Conversions among returning mobile users in Canada rose 12%, and the result is statistically significant. The redesign works."
The redesign may well work, but this result is not evidence of it. "Returning mobile users in Canada" is a slice nobody named in advance. It was picked because it won.
Suppose the team tracked twenty such slices: new versus returning, desktop versus mobile, five regions, a few traffic sources. The 5% threshold means that if the redesign did nothing at all, each slice still has about a 1-in-20 chance of clearing the bar by luck. Across twenty slices, the chance that at least one clears it is about 64%. A coin flip's worth of odds, dressed up as a finding.
The honest version reports the outcome the team committed to before launch, usually the overall conversion rate, and labels the Canadian mobile result as something to test next time.
Counter-example (not a fallacy)
"Last quarter's exploratory sweep flagged one segment. We wrote down the prediction, ran a fresh test on new traffic this quarter, and it held."
This is the same finding, and it is not the fallacy, because the target was painted before the second volley. Noticing a pattern is not the error. Noticing a pattern and then treating the observation that produced it as the confirmation of it is.
That's what preregistration is for: state the hypothesis, the outcome measure, and the analysis before seeing the results, so there is a record of where the bullseye was. Exploratory work stays legitimate as long as it is labelled exploratory. The output is a hypothesis, not a conclusion.
The line: was the target drawn before the shots or after them? Fresh data redraws it honestly. The same data cannot.
How to fix it
If you've been linked here, say how many things you looked at. That one sentence does most of the work: "we tested twelve variants and this was the winner" is a completely different claim from "we tested this variant and it won," and readers can judge it correctly once they know which one they're getting. If the pattern turned up during exploration, call it a lead and go get new data to test it against. If new data isn't available, keep the claim proportional: something interesting showed up, and it might be nothing.
If you're on the receiving end, skip the accusation and ask the process question: "How many comparisons did you run?" or "Was that the outcome you picked in advance?" Both are ordinary methodology questions rather than charges of bad faith, and the answer usually settles it either way.