(T) Google, Facebook, Apple, and Amazon have a key asset: a huge amount of data from their customers. And with data, you can do so much! But what if you are a start-up, and you want to develop a new service? How can you do it without data?

The answer is generative models and synthetic data.

Most machine learning models are discriminative. Given a number of features (input data), the model learns how to generate the labels (output data). On the other hand, generative models learn to infer the input data from the output data. Basically, generative models estimate the density distribution of the output data, and generate new sample input data, called synthetic data, from the same distribution.

Generative models can leverage either an explicit density estimation that explicitly defines, and solves the model or an implicit density estimation that learns the model and can provide the distribution without explicitly defining it.

Introduced by Ian Goodfellow and a team of researchers from the University of Montreal, Generative Adversarial Networks (GANs) for deep learning computer vision models, are certainly the most famous type of generative models.

GANs implements two neural network models: one model generates candidate images (the generative model) and the other evaluates them (the discriminative model) in a zero-sum game. The objective of the training of the generative model is to fool the discriminative model by producing novel synthesized image instances that appear to have come from the true data distribution from which the discriminative model is trained to recognize the images!

In game theory, the GAN model converges when the discriminator and the generator reach a Nash equilibrium, which is not so easy to find and makes the training of the GAN quite challenging. The generator is typically a deconvolutional neural network, while the discriminator is a convolutional neural network. Following is a simple representation of a GAN from O’Reilly Media:

Stanford University’s CS231n class on convolutional neural networks for visual recognition has a great lecture on generative models:

TechCrunch has published an interesting article “Deep learning with synthetic data will democratize the tech industry” from Evan Nisselson an investor at LDV Capital that describes how start-ups are using synthetic data to create and launch new products. An example of such a start-up is AiFi:

“AiFi is an early-stage startup building a more efficient checkout-free solution to both mom-and-pop convenience stores and major retailers. They are building a checkout-free store solution similar to Amazon Go. Through AiFi’s system, shoppers will be able to come into a retail store and pick up items without having to use cash, a card, or scan barcodes. AiFi had the typical cold start challenge with a lack of visual data from real-world situations to start training their computers, versus Amazon, which likely gathered real-life data to train its algorithms while Amazon Go was in stealth mode. [To address that challenge, AiFi created] avatars that helped train AiFi shopping algorithms. AiFi’s solution of creating synthetic data has also become one of their defensible, and differentiated technology advantages.”

Note: The picture above is “Coucher du Soleil dan la Cruese” from Armand Guillaumin.

Copyright © 2005-2018 by Serge-Paul Carrasco. All rights reserved.
Contact Us: asvinsider at gmail dot com.