..

face morphing

original kanye face, final morphed result, original thomas face

overview

the aim of this project is to morph my face to another person’s face - in this case, kanye west. this will require two main parts: warping the image shape and cross-dissolve of the image colors. the cross-dissolve is the easy part, but warping the different faces to the shape requires a little more work. warping utilizes corresponding points on the features of both faces, triangulates it, and warps/morphs the individual triangles to a control shape depending on a constant [0,1].

part 1: defining correspondences

first, i labeled all of the points using this tool. the triangulations for both images using the delaunay triangulation function.

168 triangles formed by delaunay triangulation

part 2: computing the mid-way face

morphing two different dogs into the average shape

i used inverse warping to warp both images to an average shape so that we can cross-dissolve both images to a mid-way face. here are the steps for computing the mid-way face:

  1. compute the average shape of both faces
  2. warping both faces into that shape
  3. average colors together

to perform inverse warping, we need to find the affine transformation matrix for each triangle for both images.

affine transformation matrix

we can solve for T from the equation above im' = im * T. then, we can use inverse T to warp the faces to the average shape.

both faces warped into the average shape

you can see that the triangulation in both images are the same.

cross-dissolve

cross-dissolving two average-shaped dog
warped face of kanye, cross-dissolved mid-way face, warped face of me

you can see that after warping the faces to the average shape then performing cross-dissolve on both faces, the mid-way face comes out pretty well.

part 3: the morph sequence

all 45 frames of the morph sequence

you can see that in the morph sequence the first frame is alpha = 0.00 and alpha = 1.00 for the last frame. alpha 0 and 1 are both the original faces (images).

part 4: the mean face of a population

the images i used for this part are from the FEI face database

the dataset: 100 smiling men from th FEI face database

performing what we did in part 2 but for multiple multiple faces instead of two faces, we can get the average face shape of the 100 men. this is what it looks like:

the mean face

after that, i warped all of the faces to the mean shape. here are some examples:

original vs warped faces

you can see that the most noticeable differences are in the eyes.

i then warped my face to the average:

which looks… really weird.

similarly, i warped the average face to my face shape:

part 5: caricatures - extrapolating from the mean

below are caricatures of my face by extrapolating from the mean from part 4. i chose alpha values that are outside the range [0,1] to emphasize certain features of my face. here is the equation i used:

caricature_points = alpha * (points_thomas - points_avg) + points_avg
alpha = 1.25, alpha = 2, alpha = -1

bells and whistles

(!) watch the video below in fullscreen to watch the evolution of music!