Ship-Street Salvatore
? 19th July 2019
Style transfer

In "Pages from The Oxford Book of Tourists", I wrote about the continuing lack of colour amongst Oxford's visitors:
Group of drab tourists in Ship Street, Oxford.

I'd once thought that it would be a lovely idea if some philanthropic company would do for clothing what Dulux did for buildings in their "Let's Colour" campaign, donating paint around the world in a mission to add colour to people's lives. Imagine Salvatore Ferragamo donating a design from his Milan spring/summer 2013 men's collection to every one of Oxford's 155,000 inhabitants. I mention this particular show because the vigour and purity of its colours still impresses me.

When I thought that, I tried recolouring a few photos to show how much more joyful it would make our streets. But the image processing was horrible, because I needed to cut the people to be coloured from their backgrounds. The "intelligent scissors" in my Gimp image-processing tool are as stupid as the proverbial bag of hammers, and none of the online cutting tools are any better.

But technology advances. Last year, I wrote about style transfer: re-painting one image in the style of another. The researchers who made such an advance in this work — Leon Gatys, Alexander Ecker and Matthias Bethge — have now, with their colleagues Łukasz Kidziński and Michał Warchoł, created a website at http://deepart.io. Go to the Create Your Own page, and you can upload an image to repaint, and an image to give the new style. Repainting takes only a few minutes. The site is painless to use, except that when I tried to register for my own account, it never mailed me the verification link, so the account was left in limbo, un-log-innable-to. Mailing the administrator at admin@deepart.io was no help, as he or she never replied. However, you don't seem to need to register: I presume it's just easier to keep track of your images if you do.

Here's a screenshot of some results. I found four photos of Ferragamo's Milan spring/summer 2013 men's show. I then asked DeepArt to repaint my drab-tourists image in the style of each. Here's a screenshot of its results page:
Group of drab tourists in Ship Street Oxford, recoloured with deepart.io to approximate the colours of Salvatore Ferragamo's Milan 2013 spring/summer men's collections.

And here are the results individually. Click on any one to see it enlarged.
Group of drab tourists in Ship Street Oxford, recoloured with deepart.io to approximate the colours of Salvatore Ferragamo's Milan 2013 spring/summer men's collections.
Group of drab tourists in Ship Street Oxford, recoloured with deepart.io to approximate the colours of Salvatore Ferragamo's Milan 2013 spring/summer men's collections.
Group of drab tourists in Ship Street Oxford, recoloured with deepart.io to approximate the colours of Salvatore Ferragamo's Milan 2013 spring/summer men's collections.
Group of drab tourists in Ship Street Oxford, recoloured with deepart.io to approximate the colours of Salvatore Ferragamo's Milan 2013 spring/summer men's collections.

The results aren't perfect: there is cross-coupling between style and content. This is very noticeable in the final image, where the walls show definite signs of jacket. It probably hasn't helped that in this run, the people in the styling image were much bigger than those in the content.

And there is leakage from one part of an object to another. For example, the top recoloured image has a number of spotty areas such as the legs of the person with the backpack on the left. DeepArt must have derived these from the cardigan on the right of the complete styling image, shown here. (It's from a report by Gildas Le Roux in The Times of Malta for July 1 2012.):
Screenshot of the 'Times of Malta's report on the Milan Male Spring-Summer 2013 collections, showing Salvatore Ferragamo's Milan 2013 spring/summer men's collections.

So DeepArt is a quick and fun way to recolour images, although not perfectly. But one can tweak the styling image to reduce unwanted effects on the content. More on this later.