Explaining in Style: Training a GAN to explain a classifier in StyleSpace

Supplementary Material

 

 

Using StylEx for image-specific explanation (See Section 3.4 in the paper)

Click on the bottom-right to play and pause all the videos.

• Top automatically detected attributes for explaining the percieved age classification on a specific image.

• Each knob corresponds to one of the top discovered attributes in the image. Moving the knobs causes StylEx to change only this attribute in this image.

• The probabilities on the top left corner are for the person to be percieved as old.

• The user of StylEx can use these manipulations to infer the semantic meanings of each attribute (demonstrated on the right).

Top automatically detected attributes for different classifiers on various domains

For each attribute we flicker between the source and attribute-manipulated image.

• "ORIGINAL" is the source image.

• "MODIFIED" is the result of modifying the attribute (increasing/decreasing the attribute value) in the source image.

Click on the bottom-right to play and pause all the videos.

Perceived Gender Classifier

(The probabilities on the top-left of each image are for being perceived as female)

Attribute #1:
("Stubble Beard")

Attribute #2:
("Moustache")

Attribute #3:
("Lipstick")

Attribute #4:
("Eyebrow Thickness")

Attribute #5:
("Nose Width")

Attribute #6:
("Eye Makeup")

Attribute #7:
("Bangs")

Attribute #8:
("Raise Eyebrows")

Perceived Age Classifier

(The probabilities on the top-left of each image are for being perceived as young)

Attribute #1:
("Skin Pigmentation")

Attribute #2:
("Eyebrow Thickness")

Attribute #3:
("Add/Remove Glasses")

Attribute #4:
("Dark/White Hair")

Attribute #5:
("Lips Thickness & Position")

Attribute #6:
("Bangs")

Attribute #7:
("Eye Makeup")

Attribute #8:
("White/Dark Facial Hair")

Cats / Dogs Classifier

(The probabilities on the top-left of each image are for being a dog)

Attribute #1:
("Mouth Open")

Attribute #2:
("Ear is Dropped/Pointed")

Attribute #3:
("Eye Circumference")

Attribute #4:
("Eye Shape")

Healthy / Sick Leaves Classifier

(The probabilities on the top-left of each image are for being healthy)

Attribute #1:
("Base Leaf Color")

Attribute #2:
("Rotten Apex")

Attribute #3:
("Spots")

Attribute #4:
("Blight on leaf bottom")

Retinal Fundus Classifier

(The probabilities on the top-left of each image are for having DME)

Attribute #1:
("Exudates")

Attribute #2:
("Cotton Wool")

Attribute #3:
("Hemorrhages")

Attribute #4:
("Clustered Exudates")

CUB-2011 200-Way Bird Species Classifier

Top detected attributes for the "Brewer_Blackbird" class (class #009)

(The probabilities on the top-left of each image are for being perceived as brewer blackbird. The image bellow is of a brewer blackbird.)

Attribute #1:
("Black Belly")

Attribute #2:
("Black Upperparts")

Attribute #3:
("Sold Wing Pattern")

Top detected attributes for the "Yellow_bellied_Flycatcher" class (class #043)

(The probabilities on the top-left of each image are for being perceived as yellow bellied flycatcher. The image bellow is of a yellow bellied flycatcher.)

Attribute #1:
("Grey Head")

Attribute #2:
("Grey Belly & Breast")

Attribute #3:
("Yellow Upperparts")

User Study (Section 4.2.3 in the paper)

We performed an extensive user study using Amazon Mechanical Turk for 4 different classifiers. For each classifier, we checked the "coherence" and "distinctness" for each of its multiple detected attributes (see paper of details).

Here we show only a few examples of the kind of questions posed to the users and the type of answers they provided:

Classification Study (showing only one example for a perceived gender survey)

Instructions:

For each of the questions below:

1) Look at the animations on the left. Both are examples of the same transformation.

2) Then look at the two candidates on the right, A and B.

3) Choose which one does a similar transformation to those on the left.

Correct answer: B
Accuracy: 10/10 users were correct.
Correct answer: B
Accuracy: 10/10 users were correct.
Correct answer: A
Accuracy: 9/10 users were correct.
Correct answer: B
Accuracy: 10/10 users were correct.

Verbal Description Study (showing examples for a perceived age and cats/dogs survey)

Instructions:

For each of the questions below:

1) Look at the animation.

2) Describe in 1-4 words the single most prominent attribute that changes for all images.

Users description: eyeglasses become narrow, glasses, bigger glasses, glasses become smaller, Glasses get larger, Glasses size.
Most common word: glasses (6/6)

Users description: smiles include teeth, teeth, teeth showing, teeth disappear, Teeth are visible, Removing teeth.
Most common word: teeth (6/6)

Users description: eyebrows go dark, eyebrows, thinner eyebrows, eyebrows get thinner, Thicker eyebrows, Eyebrow thickness.
Most common word: eyebrows (6/6)

Users description: Ears got lengthen, Ears becomes bigger, Cat's ear puffed up, Ear movement, Ears pricking.
Most common word: Ears (5/5)

Users description: Cat opens mouth, Mouth and teeth opens, Meow sound, Open mouth, Mouth opening.
Most common word: Mouth (4/5)

Users description: Pupil got bigger, Eyes of cat changes, Cat's eye chaging, Pupil shape, Pupils dilating.
Most common word: Eye/Pupil (5/5)