At the tail end of a post about training a network of sensory devices and learning machine programs to
read sign language, I muse about the state of algorithmic art and how come I can't find any Alex Grey-trained Deep Dream programs to make meta-Grey artwork, and low and behold discover that neural style transfer has been a thing already for a year or so (although I have yet to find any Alex Grey training programs!).
People talk about technological mysticism, but this sh** gave me a temporary lobotomy when I first saw it.
It was a bit like hearing the sampler-sequencer on your phone for making electronic music (as a person with a real, tangible 707), or the robot landing on Mars. Up until a few years ago, you could probably "put together two paintings" - a program that slices and dices a picture, not into pixelbits, but features like edges, curves, or gradients, and then splices together those features to make a hybrid picture. But you could not do it "in the style of Mondrian" or Miro, or Dali. This was a task relegated to the supreme conscious entities of this earth. No more.
And how did we get here? First you thought dreaming was a uniquely human thing, or at least a sentient creature thing (unless you're
PKD), and then Deep Mind shows us that computers can in fact be set to dream just like us.*
But that wasn't enough. The technique I'm referring to as neural style transfer is an extension of the Deep Dream project, where a deep neural net is turned on, but with nothing to do, and so it starts to get creative, and just puts things where it would like them to be, and seeing the things it knows about the world, even when they're not there.
Style transfer then takes this a step further. In a move than seems cruel as I write it, the "Dreamer" is given not a broad set of images of the world-at-large, as a contemporary, living, breathing human would see, but limits that Dreamers' world to one artist. Might be Dali, might be Mondrian (that's the cruel part).
Imagine having your entire brain erased, all but your memories of Mondrian's paintings. Being that we do not perceive objective reality apart from the interface of our previous experiences and current expectations, everything presented to us from that point on would look like a Mondrian painting. (Until we start incorporating the new non-Mondrian images we see into our corpus...). It's like that friend who is obsessed with comic books and superheroes, and everything that ever happens is a scene from Batman. You feel kind of bad, like he should go to a concert or take a walk in the park or something, don't you?
That's what we have here. We have created a thing capable of "seeing" the world, and then limited it's diet to only a bagful of paintings by a single artist.
And then we show it new things that it has never seen, and ask - "What do you see?"
Example - feed its dematerialized android eyes on a steady diet of Kandinsky and only Kandinsky. Then show it Neil Degrasse Tyson.
 |
Neil Degrasse Tyson x Kandinsky
(content image + style image = generated image)
|
I found a pretty accessible description of what's going on:
Now remember- while doing style transfer, we are not training a neural network. Rather, what we're doing is?—?we start from a blank image composed of random pixel values, and we optimize a cost function by changing the pixel values of the image. In simple terms, we start with a blank canvas and a cost function. Then we iteratively modify each pixel so as to minimize our cost function. To put it in another way, while training neural networks we update our weights and biases, but in style transfer, we keep the weights and biases constant, and instead, update our image.
-Towards Data Science
Lastly, a bit about terminology. What I'm referring to here as
algorithmic art is probably better termed AI art, to correctly co-locate it closer to fractal art. The distinguishing factor being the level of human agency. From a purist perspective, if you turn on an algorithmic art algorithm and leave the room, there is nothing on the computer when you return. Deep Dream is an algorithm that does the work while you wait, overnight while you sleep, while you both sleep, as it were. There's a new problem however in neural style transfer work - although much of what you're looking at is fully automated, the small role of the artist is disproportionately relevant to the overall impact of the image. I mean come on, a computer isn't meta-Escherizing itself! There's an artist at work there.
*Sure it is a prodigious act of artistic license to say that Deep Dreamers are dreaming "just like us," but can we not say that our dreams are a default, empty state filled with our life's worth of training data to run amok?
Notes:
Here's a place that will do it for you, but they have their own pre-designed style templates (called pastiches) -
Deep Art.
There's also a few apps that do it,
Prisma is one.
And here is a great
survey of all the slight variations that you can get, like multiple transfer, masked transfer, color interpolation, and probably my favorite, texture transfer.
Vice article on
Algorithmic Artist Dextro, 2014
Where's My Thought Translator
Network Address, 2018
Here's a few (what I imagine will become) classic style transfer artifacts, as well as a couple of the (already) style transfer classics, albeit from early 1900's abstract artists.
The Macro Image Series and the Dematerialization of Artifact
Network Address, 2013
Here's a presentation about the propagation of cultural artifacts in a digital medium.
Post Script:
I like to give Piet Mondrian credit for his prescience in foretelling bitmaps and pixelation, despite the fact that in his day, the primaries were still red-yellow-blue instead of our current day cyan-magenta-yellow.
Here, a painting of Napoleon, trained on Raphael's School of Athens:
 |
Raphael's School of Athens
|
 |
Jacques-Louis David's Napoleon Crossing the Alps
|
Post Post Script:
Same thing here but for music; I'm still waiting for the meta cover band that covers other bands as if they were already covering another band, for example, a cover band that does Radiohead doing the Doors, or if the Squirrel Nut Zippers covered Metallica, what would that sound like? Any tips would be much appreciated.
