Art x Science

AI art is moving fast and breaking things (sorry Getty Images Holdings). But the part that really blows my mind is that it makes a perfect composition, every single time.

The picture above was trained on billions of images. Some of them -- but certainly not all of them -- are actual artwork or at least professional commercially-designed imagery, and yet, after digesting basically every image on the internet, this is what comes out. 

It's perfect: balance, contrast, rhythm, movement, emphasis, pattern, unity; the seven principles of design, they're all there. 

If you mix every paint color together, you get brown. If you took a program that learned how to drive by watching all the cars out on the road, the result would be a car accident. Yet, when we digest every picture on the internet, this is what comes out. The prompt was "complementary color scheme of insect eyes", but that doesn't really matter. You could literally write "shitty amateur artist garbage" and it would produce an image that would win a local art contest.

Anyway, here's some insight into what our eyes want: 

Color composition preferences in art paintings are determined by color statistics

Oct 2022, phys.org

https://phys.org/news/2022-09-composition-art-statistics.html

Color composition preferences of 31,353 participants, for 1,200 paintings with artificially manipulated color compositions where the hue angle was rotated by 90, 180, and 270 degrees.

Participants always preferred the original. But not for the reasons expected -- the "matching-to-nature" hypothesis.

Instead, these statistical color properties predicted preference: 

• asymmetric red–green distribution
• correlation between lightness and blue–yellow distribution
• correlation between red–green and blue–yellow 

It was revealed for the first time that color compositions found in paintings do not simply imitate natural scenes, but have their own unique features, which is linked to preferences of observers and the attractiveness of certain color schemes.

via Toyohashi University of Technology: Shigeki Nakauchi et al, Regularity of colour statistics in explaining colour composition preferences in art paintings, Scientific Reports (2022). DOI: 10.1038/s41598-022-18847-9

https://dx.doi.org/10.1038/s41598-022-18847-9

Scraping the Bottom of the Barrel

Getty is the Equifax of images. They own everything. And if they don't own it, they will buy it, and then own it. Try searching for something, anything. Grass-stained wiffle ball lying on the ground? Beads of rainwater on an umbrella? Two red foxes sniffing each other in the woods in autumn. Guaranteed among the first five images, one will belong to Getty, and it will be the best one. 

This has been written about before on Network Address. Right about the time that Google bought Pinterest (2018ish) and then rearranged their image search algorithm to show nothing but images hosted on Pinterest, the site that requires you to sign-up and log-in to see the images. So now even if I wanted to, I can't source the image I just right-clicked, saved, and posted to my weblog, no matter who it belongs to or who made it.

Then enter Getty -- they forced Google to change their image search engine to force you to go to the website where the image came from in order to see beyond the thumbnail. This way, Getty can get money from advertisers on the websites where their images are found: Anger at Google image search peace deal, Feb 2018, BBC.

Sorry if you forgot, but the "com" in .com stands for commerce.

But things have changed a bit. When Stable Diffusion was released to the public as an open source program that can run an image generating neural net using only the GPU's already in your own computer, it pretty much destroyed the business prospects of any company like Getty. That's why we're looking at the screenshot of their stock price circa August 15 2022, when Stable Diffusion was released. 

Why Getty Images gambled on a SPAC to go public after a 14-year absence from public markets

Sep 2022, Fortune Magazine

https://fortune.com/2022/08/16/getty-images-spac-go-public-markets/

I asked Leyden if A.I. image generation could be a potential threat to Getty Images’ business. “We’re watching that space, like everyone else is, pretty closely,” she told me. “We don’t think about it as competition. We think of it as further evidence that the demand for visual content is just growing.”

Post Script:

It's hard to keep track, but "In 2021, Getty Images acquired Unsplash, which offers “widely accessed,” free creative photography; its website is ad-supported with over 17,000 users of its API. “That’s a big focus for us and really monetizing that creator economy that Unsplash taps into,” Leyden says.

Artificial Artists

Note: This piece of news got overshadowed pretty quickly; the open-sourced, LAION-powered Stable Diffusion was release mid-August. The WikiArt dataset (see mention below) has 40,000 images, whereas the LAION has somewhere between 400 million and 5 billion (sorry I just can't tell anymore because I see both numbers in a bunch of different places). 

A model to generate artistic images based on text descriptions

Jun 2022, phys.org

https://techxplore.com/news/2022-06-artistic-images-based-text-descriptions.html

Dynamic memory generative adversarial network (DM-GAN) can automatically generate unique artistic images based on text descriptions.

"Due to the lack of datasets with paired text description and artistic images, it is hard to directly train an algorithm which can create art based on text input," the researchers explained in their paper.

"To address this issue, we split our task into three steps."

First use the DM-GAN model to generate a realistic image that represents a text description.

Subsequently use neural network ResNet to classify the image produced by the DM-GAN into one of the genre categories outlined by the WikiArt dataset, which contains more than 40,000 artistic paintings produced by 195 artists. Then use deep style transfer, and done. 

via University of Waterloo and New York University Courant Institute: Qinghe Tian, Jean-Claude Franchitti, Text to artistic image generation. arXiv:2205.02439v1 [cs.CV],

arxiv.org/abs/2205.02439https://arxiv.org/abs/2205.02439

Image credit: Tian & Franchitti

Post Script:

Your brain is better at busting deepfakes than you are

Jul 2022, phys.org

https://medicalxpress.com/news/2022-07-brain-deepfakes.html

When looking at participants' brain activity, the University of Sydney researchers found deepfakes could be identified 54% of the time. However, when participants were asked to verbally identify the deepfakes, they could only do this 37% of the time.

"That tells us the brain can spot the difference between deepfakes and authentic images."

(Interpretation: In the future you will need an EEG just to communicate with your own brain.)

via University of Sydney: Michoel L. Moshel et al, Are you for real? Decoding realistic AI-generated faces from neural activity, Vision Research (2022). DOI: 10.1016/j.visres.2022.108079

https://dx.doi.org/10.1016/j.visres.2022.108079