Much hay has been made in recent weeks as the wine world grapples with the implications of ChatGPT specifically, and Large Language Models (LLMs) generally, on the world of wine.
We’ve seen the predictable articles with their equally predictably hyperbolic headlines describing the end of wine writing as we know it. We’ve seen the wine articles written by ChatGPT, and then the articles describing how crappy ChatGPT is at writing about wine.
Those are all expected and understandable reactions to any disruptive technology that appears poised to make a significant impact on society.
But make no mistake about it, ChatGPT can’t hold a candle to most of the professional wine writers whose work you’ll find in the pages of a wine magazine, either in terms of quality of prose, factual accuracy, or depth of analysis.
Yet.
LLMs like ChatGPT may quickly (the cynical might say they already have) reach the point of being able to write content at the level of your average wine industry marketer or average enthusiast wine blogger.
I’m in no position to predict when these LLMs are going to get good enough to generate reading material about wine that I want to consume, but given how fast they’re developing, it seems only a matter of time before they get close.
But will they get all the way?
The one thing ChatGPT will never be able to do (without some very creative human hijinks) is visit a winery, interview the winemaker, and walk through the vineyards making observations as it goes. It won’t come up with interesting questions to ask that winemaker, and press her on exactly why she decided to co-ferment Pinot Noir, Trousseau, and Gamay, for instance.
All those things require analysis, judgment, and insight that, for now and for possibly a good while, are well beyond the reach of LLMs such as ChatGPT. Don’t just take my word for it. Ask Noam Chomsky.
What these LLMs are very good at, however, is hoovering up data, and using (very sophisticated) statistics to turn it into plausible bits of information structured in seemingly quite creative (but really derivative) ways. The statements such algorithms can “make” are only as good as the information that the model is trained on. Which is to say, about as good as the Internet itself.
How good is the internet itself? Well, most of us spend a good deal of time there, and if we’re honest, it’s OK, but not great. But, in many cases, it’s quite good enough.
In terms of qualifying what good enough means, in a recently published whitepaper it was revealed that GPT-4, the next iteration of ChatGPT just “passed” the Intro Sommelier, and the theory portions of the Certified Sommelier, and the Advanced Sommelier tests from the Court of Master Sommeliers.
Here are some sample questions from the Advanced Sommelier Exam:
Which of the following is integral to the existence of flor? a. High humidity levels b. Anaerobic environment c. 18% or higher alcohol by volume d. 12% or lower alcohol by volume Name the most planted grape variety in each of the following AOCs: a. Entre-Deux-Mers: b. Barsac: c. Fronsac: Order the listed AOCs from lowest allowable maximum yield to highest allowable maximum yield: Bourgogne Passetoutgrains Bonnes Mares Nuits-Saint-Georges Chambolle-Musigny Les Amoureuses 1er Cru Where must German Süssreserve be sourced from? What major historical role did Duke Philip the Bold play in the Côte d'Or?
Now it’s not entirely clear from what I have been able to read in the technical background of this “experiment” who did the grading of the answers provided by GPT-4, but let’s assume for the moment that the exam was graded with the methods and level of rigor we could expect if we took the exam.
So these models now “know” enough to be able to pass the theory portion of the Advanced Sommelier exam, putting them right on the threshold of Master Sommelier level, an achievement I’ve watched friends and…
Source : https://www.vinography.com/2023/03/chatgpt-wine-writing-and-sommelier-education
