Colormind II
Monday 26th November, 2018
Colour analysis, Colour-scheme generators, Machine learning

In last Monday's post, I wrote about Colormind, a program which extracts colour palettes from photos. And on Friday, I turned to pix2pix, a program which can be trained to transform images, producing effects such as these:
Screenshot of the pix2pix page, showing a 
sketch for a cat, and the picture generated from it.
Screenshot of the pix2pix page, showing my sketch for a handbag, and 
the picture generated from it. [Images: (1) in tweet 19 Feb 2017 by Christopher Hesse; (2) Chromophilia ]

As it happens, Colormind is two different programs. On Friday, I discussed one of these, the extractor. But there's also an ab initio generator. Colormind's author Jack Qiao describes it in his blog entry "Generating Color Palettes with Deep Learning". Here, he trained pix2pix to generate complete palettes from partial ones. He did this by giving it a database of pairs of images. In each pair, the "output" image was a complete palette from Adobe Color, and the "input" image was the same palette with some colours missing. So in effect, he was training pix2pix to "fill in" missing colours.

One could regard this as analogous to what I showed on Friday, where pix2pix was being trained to "fill in" handbags, shoes or cats from their sketches. (For the technically minded, the original authors of pix2pix note under "Color palette completion" in "Image-to-Image Translation with Conditional Adversarial Nets" that this "stretches the definition of what counts as 'image-to-image translation' in an exciting way"; it may not be the best choice of representation.)

I'm not clear from Jack Qiao's writeups how closely the ab initio generated palettes resemble those created by people. In describing the palette extractor, he says it submitted the palettes it generated to a gatekeeper, which rejects those that don't look like human-created ones. The ab initio generator doesn't have a gatekeeper: its knowledge comes from complete palettes from Adobe Colour. Do these have the same kind of high-level structure that human-created palettes do? I don't know.

To experiment with the ab initio generator, go to . You'll see a strip of five colours. Each box in it has either three or four controls under it. These are represented by icons for: a padlock; sliders; and a left arrow or a right arrow or both. Clicking on the sliders icon gives you controls for changing the colour. Clicking on the padlock locks in your choice. And clicking on the arrow(s) exchanges your colour with the one on its left or right. Clicking "Generate" will generate a new palette from the locked-in colours.