GPT-2 Generated Ceramic Recipes

Having been on the GPT-3 API waitlist for months, I finally just decided to fine-tune the older and smaller GPT-2 355M model using over 10,000 public Glazy recipes as training data. Although the loss was still going down after 6,000 steps, I stopped training at that point as I noticed more duplicates being generated and I was afraid of overfitting my relatively small dataset.

As always, Gwern has a wonderful article, GPT-2 Neural Network Poetry. For training I followed Max Woolf's article How To Make Custom AI-Generated Text With GPT-2 and used Max's wonderful Colab Notebook

Sample recipe from training data:

  RECIPE: Winokur Yellow
  INGREDIENT: 53.2000	Potash Feldspar
  INGREDIENT: 22.9000	Kaolin
  INGREDIENT: 19.4000	Dolomite
  INGREDIENT: 4.5000	Whiting
  INGREDIENT: 16.9000	Zircopax
  INGREDIENT: 3.5000	Tin Oxide
  INGREDIENT: 1.4000	Red Iron Oxide

I generated two sets of recipes, one with GPT-2 temperature set at 0.7 and one at 0.9. The results for both sets were surprisingly good: At first glance the recipes seemed "real" with proportional mixes of feldspars, clays, silica, fluxes and colorants/opacifiers. Even the total ingredient amounts added up to a reasonable number, usually in the range of 90-110%. The duplication rate was about 5% for t=0.7 and 4.5% for t=0.9.

Sample generated recipes (removing "RECIPE:" and "INGREDIENT:" tags):

Crawly Elsie's Matte-04
38.0000	EP Kaolin
28.0000	Gerstley Borate
19.0000	G-200 Feldspar
9.0000	Lepidolite
6.0000	Soda Ash
4.0000	Wollastonite

Ame-Sosa-Wenkel
38.0000	Nepheline Syenite
29.0000	Silica
12.0000	Colemanite
8.0000	Whiting
6.0000	Dolomite
5.0000	Barium Carbonate
2.0000	Bentonite
1.0000	Rutile
0.7500	Copper Carbonate

Amber Celadon
34.0000	Albany slip
20.0000	Custer Feldspar
13.0000	Silica
13.0000	Wollastonite
6.0000	Whiting
3.0000	EP Kaolin
3.0000	Gerstley Borate
3.0000	Rutile
2.0000	Red Iron Oxide

Craters
30.0000	Lithium Carbonate
30.0000	Silica
15.0000	Borax
10.0000	Zircopax
10.0000	Kaolin
5.0000	Bentonite
3.0000	Copper Carbonate

Downloads:
Temperature 0.7 Generated Recipes
Temperature 0.9 Generated Recipes

The next step was to load the GPT-2 generated recipes into Glazy in order to see their resulting analyses and visualize them on the Stull Chart. I was surprised to find that, as with "real" glaze recipes, most of these generated recipes fell comfortably within the major Stull regions of Bright, Matte, and Semi-Matte. The set with temperature setting of 0.9 generated more variation.

Generated recipes displayed in the Stull Chart.

Training against a subset of recipes, using only the cone 6 glazes, gave results with lower Silica and Alumina, as well as higher Boron.

Cone 6 generated recipes showing lower Silica & Alumina as well as higher levels of boron.

The obvious next step was to fire some of these "fake" recipes and create real glazes. Rather than randomly testing, I selected a few recipes that looked like they would fire to maturity at my chosen temperature and atmosphere, Orton cone 6 in Oxidation. The results were all quite good, and I uploaded a couple to Glazy:

GPT-2 Yellow Textured

GPT-2 Three's Company

Selected generated recipes fired at Orton cone 6 in Oxidation.

Using Glazy's public data (https://github.com/derekphilipau/glazy-data) I attempted to associate glaze metadata (type, surface, transparency and temperature) with analyses.

Input sample:

TYPE: Iron - Celadon
SURFACE: Glossy
TRANSPARENCY: Transparent
CONE: 9
SiO2: 66.12, Al2O3: 12.08, K2O: 3.13, Na2O: 0.84, MgO: 0.06, CaO: 12.67, Fe2O3: 0.16, CuO: 0.01, ZrO2: 0.26

TYPE: Clear
SURFACE: Glossy
TRANSPARENCY: Transparent
CONE: 8
SiO2: 53.49, Al2O3: 9.29, B2O3: 2.08, Li2O: 1.01, K2O: 0.29, Na2O: 1.11, MgO: 0.03, CaO: 0.12, BaO: 21.06, ZnO: 3.98, Fe2O3: 0.08, CuO: 0.01, ZrO2: 0.93

Output sample:

TYPE: Glaze
SURFACE: N/A
TRANSPARENCY: N/A
CONE: 10
SiO2: 61.7, Al2O3: 8.74, B2O3: 4.26, K2O: 4.05, Na2O: 1.87, MgO: 0.52, CaO: 8.46, P2O5: 0.01, Fe2O3: 0.24, TiO2: 0.17

TYPE: Iron - Celadon - Blue
SURFACE: Glossy
TRANSPARENCY: Translucent
CONE: 9
SiO2: 61.75, Al2O3: 12.13, K2O: 3.16, Na2O: 0.98, MgO: 0.09, CaO: 10.90, Fe2O3: 0.72

TYPE: Blue - Cobalt
SURFACE: Glossy
TRANSPARENCY: Translucent
CONE: 9
SiO2: 61.44, Al2O3: 12.06, K2O: 2.75, Na2O: 0.93, MgO: 0.07, CaO: 11.31, Fe2O3: 0.72, CoO: 1.83, TiO2: 0.31

TYPE: Blue - Cobalt
SURFACE: Glossy
TRANSPARENCY: Translucent
CONE: 9
SiO2: 61.44, Al2O3: 12.06, K2O: 2.75, Na2O: 0.93, MgO: 0.07, CaO: 11.31, Fe2O3: 0.72, CoO: 2.33, TiO2: 0.31

TYPE: Specialty - Crawling
SURFACE: Glossy
TRANSPARENCY: Opaque
CONE: 6
SiO2: 51.63, Al2O3: 16.32, B2O3: 3.20, K2O: 2.63, Na2O: 2.25, MgO: 3.71, CaO: 2.96, P2O5: 0.10, Fe2O3: 0.60, CoO: 0.82, TiO2: 2.82

TYPE: Iron - Kaki, Tomato Red
SURFACE: Matte - Smooth
TRANSPARENCY: Opaque
CONE: 6
SiO2: 44.64, Al2O3: 12.17, B2O3: 3.06, K2O: 0.07, Na2O: 4.76, MgO: 3.96, CaO: 7.55, P2O5: 2.08, Fe2O3: 8.72, TiO2: 0.06

Downloads: Generated "fake" analyses