Are vintage charts right?

Every wine region has its legendary years and its washouts, and prices follow them, especially for the most famous names. So it is worth asking whether the reputations are deserved, and how much a good year actually changes what is in the glass.

How I tested it

The obvious comparison is misleading. In a famous year, more of the best estates get reviewed and their wines cost more, so a region's average score rises even if nothing changed in the glass. So I compare each producer only with itself: the same winery, the same grape and the same critic, across at least two vintages. A vintage's score is how much better or worse that producer's wine came out than in its other years, averaged across every producer in the region.

That covers 45,096 reviews across 43 regions and the vintages 1997 to 2016. The reputations below are the widely held views of each year, described in my own words rather than taken from any published chart.

The producer matters far more than the year

Before the vintages themselves, the headline. Within a region, producers differ from one another by about 1.8 points on average. Vintages differ by about 0.4. The name on the label tells you roughly 5 times more than the year on it.

Famous vintages, tested

For each famous year, the dot shows how the region's producers scored against their own other vintages, with a line for the range the data is confident about. Right of zero is better than the producer's usual; left is worse.

Where the charts are right. Most of the famous years do stand out. Bordeaux 2009 and 2010 come in at +0.5 and +0.5 points, Champagne 2008 at +0.7, Burgundy 2005 and 2015 at +0.6 and +0.6, Brunello 2006 at +0.7. The weak years show too: Bordeaux 2013 is −0.9, Napa's cool 2011 −0.4.

But the differences are small. Across all 17 years with a great reputation, the same producers' wines came out on average just +0.3 points out of 100 better than usual. That is real, but it is a small difference in the glass for what can be a large difference in price.

Where the charts are wrong. Two reputations are not borne out. Red Burgundy 2010, much admired, scores −0.7 against the same producers' other years, and Barolo 2004 −0.3. Both are known as wines that needed time, and most of these reviews were written when the wines were young, so the reputation may have been earned later in bottle. Still, on release, neither stood out.

The quiet over-achiever. Bordeaux 2014, usually seen as good but overshadowed by its famous neighbours, scores +0.6: as well as 2009 and 2010. Years like that are where the value is, because the price follows the reputation, not the wine.

2003: when the heat goes too far

The clearest signal in the whole dataset is not a famous good year but a famous hot one. The summer of 2003 brought a record heatwave across Europe, and almost everywhere it touched, the same producers made weaker wine than usual.

Brunello, Barolo, Bordeaux and Champagne all had a 2003 season roughly 1°C warmer than normal, and all came in clearly below their usual level. The Veneto is the exception. Napa's 2003 season was normal, so its poor showing has another cause: not every bad year is a hot one. Across all 43 regions, a little extra warmth tends to help, by about +0.1 points per degree, and the data hints, without proving it, that the most extreme heat tips the other way. It fits what I found in Do hot vintages really taste riper?. As seasons warm, years like 2003 stop being rare.

What it means if you are buying

• Choose the producer first, the year second. A good grower in an ordinary year will usually beat an ordinary grower in a great one.
• Be wary of paying the vintage premium. A celebrated year is, on average, less than half a point better in the same producer's hands.
• Look for the overshadowed years. Vintages like Bordeaux 2014 score close to their famous neighbours without the reputation, or the price, to match.
• Take extreme heat seriously. In this data, the most consistently disappointing vintage was the hottest one.

What it doesn't show

• Scores on release, from one publication. These are Wine Enthusiast reviews, mostly written when the wines were young. Some vintages, especially structured ones, earn their reputation over decades in bottle, and that is not captured here.
• Regions are broad. “Bordeaux” mixes the Left and Right Banks and “Burgundy” the Côte de Nuits with the Côte de Beaune, which often had different vintages.
• Small samples for some years. Where the range is wide, as for Rioja 2001, the data cannot confirm or deny the reputation, so I say so.

Conclusion

Vintage charts are mostly right about which years were good, but they overstate how much it matters. The difference between a great year and an ordinary one is usually less than half a point in the same producer's hands, while the difference between producers is several times larger. The year is worth knowing; the name of the grower is worth knowing more. And with summers like 2003 becoming more common, the most useful vintage warning may soon be not “too cold and wet” but “too hot”.