Generative Models, the Good, the Bad, and the Ugly!


(T) Most machine learning models are discriminative. Given a number of features (input data), the model learns how to generate the labels (output data). On the other hand, generative models learn to infer the input data from the output data. Basically, generative models estimate the density distribution of the output data and generate new sample input data, called synthetic data, from the same distribution.

Generative models can leverage either an explicit density estimation that explicitly defines and solves the model or an implicit density estimation that learns the model and can provide the distribution without explicitly defining it.

Introduced by Ian Goodfellow and a team of researchers from the University of Montreal, Generative Adversarial Networks (GANs) for deep learning computer vision models, are certainly the most famous type of generative models.

Introducing Generative Adversarial Networks (GANs)

GANs implements two neural network models: one model generates candidate images (the generative model) and the other evaluates them (the discriminative model) in a zero-sum game. The objective of the training of the generative model is to fool the discriminative model by producing novel synthesized image instances that appear to have come from the true data distribution from which the discriminative model is trained to recognize the images! In game theory, the GAN model converges when the discriminator and the generator reach a Nash equilibrium, which is not so easy to find and makes the training of the GAN quite challenging. The generator is typically a deconvolutional neural network, while the discriminator is a convolutional neural network. Following is a simple representation of a GAN from O’Reilly Media:


Stanford CS231n class on convolutional neural networks for visual recognition has a great lecture on generative models:

Google, Facebook, Apple, and Amazon, among others, have a key asset, a huge amount of data from their customers. And with data, you can do so much! But what if you are a start-up, and you want to develop a new service? How can you do it without data?

The answer is generative models and synthetic data. That is a good thing!

Synthetic data (a good thing)

TechCrunch has published an interesting article “Deep learning with synthetic data will democratize the tech industry” from Evan Nisselson an investor at LDV Capital that describes how start-ups are using synthetic data to create and launch new products.

But while generative models can be good, they are not immune to certain threats called adversarial examples. In that case, generative models can become bad…

Adversarial examples (a bad thing)

Adversarial examples introduce small changes to an image that leads the model to mis-classify the input image. Instead of recognizing a cat, the model will recognize a dog even if it was successfully trained to recognize cats. They can be implemented either by only accessing the input data of the model, or by accessing the trained model itself including the training data, model architecture, hyper-parameters, numbers of layers, activation functions, or model weights.

Adversarial examples have been initially defined in a blog post from OpenAI: “Attacking Machine Learning with Adversarial Examples.”

Adversarial examples can have serious consequences in particular for certain applications such as autonomous vehicles. Imagine what would happen if your Telsa misrecognizes a stop sign?

Adversarial examples fooling humans (an ugly thing)

But while adversarial examples can fool computer systems, they could also fool humans. That is ugly!

In a recent paper “Adversarial Examples that Fool both Computer Vision and Time-Limited Humans“, Ian and other teams members of Googe Brain described the risks of adversarial examples to humans:

“Adversarial examples provide one more way in which machine learning might plausibly be used to subtly manipulate humans. For instance, an ensemble of deep models might be trained on human ratings of face trustworthiness. It might then be possible to generate adversarial perturbations which enhance or reduce human impressions of trustworthiness, and those perturbed images might be used in news reports or political advertising.”

“More speculative risks involve the possibility of crafting sensory stimuli that hack the brain into a more diverse set of ways, and with larger effect. As one example, many animals have been observed to be susceptible to supernormal stimuli. For instance, cuckoo chicks generate begging calls and an associated visual display that causes birds of other species to prefer to feed the cuckoo chick over their own offspring. Adversarial examples can be seen as a form of supernormal stimuli for neural networks. A worrying possibility is that supernormal stimuli designed to influence human behavior or emotions, rather than merely the perceived class label of an image, might also transfer from machines to humans.”

Adversarial training

Mitigation of the effects of adversarial examples is still in its infancy but one key area of research is adversarial training.

Adversarial training ensures that the neural network is trained with adversarial examples. It is currently based on regularization and semi-supervised learning techniques.

To deeper dive into adversarial examples and adversarial training, again a lecture from Ian Goodfellow from Stanford CS231n class:


Note: The picture above is “Coucher du Soleil dan la Cruese” from Armand Guillaumin.