33

I hear from some sources that Generative adversarial networks are unsupervised ML, but i dont get it. Are Generative adversarial networks not in fact supervised?

1) 2-class case Real-against-Fake

Indeed one has to supply training data to the discriminator and this has to be "real" data, meaning data which i would label with f.e. 1. Even though one doesnt label the data explicit, one does so implicitly by presenting the discriminator in the first steps with training data, which you tell the discriminator is authentic. In that way you somehow tell the discriminator a labeling of the training data. And on the contrary a labeling of the noise data that is generated at the first steps of the generator, which the generator knows to be unauthentic.

2) Multi-class case

But it gets really strange in the multi class case. One has to supply descriptions in the training data. The obvious contradiction is that one supplies a response to an unsupervised ML algorithm.

4
  • The input to the GAN is unlabeled real data. The algorithm introduces "fake" data that it distinguishes internally from the real, but no human labeling ("supervision") is required. In that way it serves the same role as other unsupervised methods for which no human labeling is needed / which can be applied to as large an unlabeled dataset as you can gather.
    – Alex Lew
    Commented Jun 8, 2017 at 21:24
  • A human beeing still has to select the training data and therefore implicitly label it. Can't produce cat images with training data which only shows monuments. Therefore the need for supervision in the preparation, when someone wants to produce a specific kind of data.
    – scrimau
    Commented Jun 8, 2017 at 22:16
  • See ai.stackexchange.com/a/22815/2444.
    – nbro
    Commented Aug 2, 2020 at 0:18
  • @scrimau There is nothing that says that you have to limit the training data to cat images, though. Commented Mar 26 at 15:48

2 Answers 2

56

GANs are unsupervised learning algorithms that use a supervised loss as part of the training. The later appears to be where you are getting hung-up.

When we talk about supervised learning, we are usually talking about learning to predict a label associated with the data. The goal is for the model to generalize to new data.

In the GAN case, you don't have either of these components. The data comes in with no labels, and we are not trying to generalize any kind of prediction to new data. The goal is for the GAN to model what the data looks like (i.e., density estimation), and be able to generate new examples of what it has learned.

The GAN sets up a supervised learning problem in order to do unsupervised learning, generates fake / random looking data, and tries to determine if a sample is generated fake data or real data. This is a supervised component, yes. But it is not the goal of the GAN, and the labels are trivial.

The idea of using a supervised component for an unsupervised task is not particularly new. Random Forests have done this for a long time for outlier detection (also trained on random data vs real data), and the One-Class SVM for outlier detection is technically trained in supervised fashion with the original data being the real class and a single point at the origin of the space (i.e., the zero vector) treated as the outlier class.

9
  • 5
    Therefore, GANs have both a supervised step and an unsupervised one. Saying that GANs are unsupervised is wrong.
    – nbro
    Commented Aug 27, 2018 at 16:03
  • 2
    As I elaborated in my answer, GANs do have a supervised component. However, the data comes in unlabeled and uncategorized. This is the crux of what makes it an unsupervised algorithm, no labeling is needed or provided to the GAN algorithm. If we called any algo that had a component normally used in supervised algorithms, a supervised algorithm, then there would be almost no "unsupervised" algorithms by that definition. Its a function of what information needs to come with the data (i.e., supervision in the form of labels), not how the mechanics operate. Commented Aug 28, 2018 at 17:40
  • 1
    "However, the data comes in unlabeled and uncategorized.", to be precise, you should say "the data comes in unlabeled and uncategorized to the trained GAN", because an untrained GAN still needs a labeled dataset.
    – nbro
    Commented Aug 28, 2018 at 17:50
  • 2
    An untrained GAN does not need a labeled dataset. That is not correct. The training occurs by generating a classification problem between the two networks (hence the "Adversarial" in GAN). The labels of this problem are not provided with the data, but a trivial consequence of the training process. Images "Generated" by one part of the network have the trivial label of fake, and the training data the trivial label of real. These are not provided with the data. No person at any point needs to label the data before training. Hence the process as a whole is unsupervised. Commented Aug 28, 2018 at 18:04
  • 4
    Because its well accepted that GANs are unsupervised. The original paper directly implies it by explaining how a GAN could be used for semi-supervised learning by using the GAN to learn the feature representation on unlabeled data. If we used your logic, auto-encoders & PCA would also be supervised because they use a supervised loss, even though its only to predict the input. I've explained multiple different ways how supervised vs unsupervised is a function of the labels coming with the data, not a function of the mechanism of learning. I'll be stoping this conversation at this point. Commented Aug 29, 2018 at 13:45
-13

neither. roughly, the hierarchy looks like the following:

               machine learning methodology
                             +
                             |
                             |
                             v
    +-----------------------------------------------+
    |                        |                      |
    |                        |                      |
    v                        v                      v
supervised              unsupervised           reinforcement

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.