Who Do I Look Like?Determining Parent-Offspring Resemblance via Gated

Autoencoders

Afshin Dehghan

Enrique G. Ortiz

Ruben Villegas

Mubarak Shah

Outline

Introduction

Autoencoder

Gated Autoencoder

Experiments

Conclusions and Future Work

Introduction

“Who does he look like more, the father or the mother?”

In this paper, we aim to bridge the gap between findings in the social sciences and computer vision to answer the age-old question, “Who do I look like?”

Introduction

Introduction

The studies have corroborated that offspring do in fact resemble parents more than random strangers and at different ages may resemble a particular parent more.

Using a new generative and discriminative model based on the gated autoencoder.

This paper ‘s discovery :

1. Optimal features

2. Metrics relating a parent and offspring via gated autoencoders.

3. Enhances the relationship of a parent-offspring pair converging on a more discriminative function.

Introduction

We aim to answer two key questions from the perspective of a computer:

1. Do offspring resemble their parents?

2. Do offspring resemble one parent more than the other?

Given the answers to these questions, we can conclude why computer vision discover which facial features lead to the best performance in parent-offspring recognition.

We believe familial resemblance can aid in reuniting parents with their missing children.

Autoencoder

This deep learning architecture keeps the most important information.

This property allows us to learn the most discriminative features that we refer to as ‘genetic features’.

Ny represent the dimension of the image patch.

Nm represent number of hidden units.

Gated Autoencoder(1)

Why use Gated Autoencoder?

We are interested in encoding the relationship between a pair of images.

The final output of our system is a relatedness or resemblance statistic, which we can use for classification.

Gated Autoencoder(2)- Generative Training

Weights zk as mapping units.

Gated Autoencoder(3)- Generative Training

Nx, Ny and Nz are the dimension of x, y and z.

F is the number of hidden units.

Minimize the loss function

Gated Autoencoder(4)- Discriminative Training

Ground-truth labels:

1. 0 - not same family

2. 1 - same family

Discriminative objective function:

Final hybrid model:

The best is 0.4.

Experiments(1)

Explore our two main questions:

1. Do offspring resemble their parents?

2. Do offspring resemble one parent more than the other?

For all experiments involving the gated autoencoder method.

Extract 8x8 patches from an RGB image of size 64x64.

Set the number of filters to F = 160 and the number of mapping units to Nz=40.

The parameter is found through cross validation, which is 0.4.

Using SVMs for classification.

Using the RBF kernel with parameters selected via 4-fold cross validation.

Do offspring resemble their parents?

Experiments(2) - Dataset

Family 101[11] dataset

1. 206 nuclear families

2. 101 unique family trees

3. 14,816 images

We select 101 unique, nuclear families for our experiments.

We split the set into 50 training families and 51 testing families for a total of 11,300 images.

[11] R. Fang, A. C. Gallagher, T. Chen, and A. Loui. Kinship Classification by Modeling Facial Feature Heredity. IEEE ICIP, 2013.

Experiments(3)

Apply the discriminative feature learning technique on the training relationships between all possible.

1. mother-daughter (MD)

2. mother-son (MS)

3. father-daughter(FD)

4. father-son (FS)

Experiments(4)

Do offspring resemble one parent more than the other?

Experiments(5)

Daughters resemble their mother more.

Sons resemble their fathers more.

The conclusion is aligned with anthropological studies.

Experiments(6) – Genetic Features

We examine our method with respect to three factors:

1. How our discovered features compare to those from anthropological studies.

2. How well our genetic features outscore the state-of-the-art in metric learning.

3. How well the feature models generalize.

Experiments(7) - Computer vs. Anthropology

Experiments(8) - Face Verification

How well our method performs against existing metric learning techniques.

Determine whether fusing the findings from anthropological studies with our method improves performance.

KinFaceW [19] dataset, which is comprised of two sets.

1. KinFaceW-I with 533 parent-offspring pairs from different images.

2. KinFaceW-II with 1,000 parent-offspring pairs from the same image.

KinFaceW-I KinFaceW-II [19] J. Lu, X. Zhou, Y.-P. Tan, Y. Shang, and J. Zhou. Neighborhood Repulsed Metric Learning for Kinship Verification. IEEE TPAMI, 2013.

Experiments(9) - KinFaceW-I

Metric learners:

1. Information-Theoretic Metric Learning (ITML)

2. Neighborhood Repulsed Metric Learning (NRML)

Experiments(10) - KinFaceW-II

Experiments(11) - 5-fold crossvalidation

Conclusions and Future Work

Using this method, we uncover three key insights that bridge the gap between anthropological studies and computer vision.

1. Offspring resemble their parents with a probability higher than chance.

2. Female offspring resemble their mothers more often than their fathers, while a male offspring only slightly favor the father.

3. The algorithm discovers features similar to those found in anthropological studies.

References

Who's Your Daddy?

KinFaceW