Summary
Highlights
The video introduces a complex project on live face swapping using OpenCV and MediaPipe. The presenter demonstrates how his face is swapped with famous personalities like Bill Gates, Steve Jobs, Donald Trump, and Dwayne Johnson using a live webcam, highlighting the accuracy and minimal distortion.
The initial setup in Visual Studio Code is explained, including the creation of 'app.py' and 'media_utils.py' files, and an 'images' folder containing celebrity faces. Essential libraries like cv2 and NumPy are imported, webcam capture is configured, and a list of source image paths is prepared for face swapping.
The video delves into 'media_utils.py', importing modules like MediaPipe and sys. It initializes MediaPipe's drawing utilities and face mesh solution, which detects 468 facial landmarks. A 'get_landmark_points' function is created to detect these landmarks and convert them into pixel coordinates, handling cases of no or multiple faces detected.
A 'get_triangles' function is introduced, which uses Delaunay triangulation to divide the face into small triangles. The importance of Delaunay triangulation for accurate face swapping is explained, emphasizing its role in preventing distortion and maintaining smooth transitions by converting landmark points into a consistent set of triangle indices.
The 'triangulation' function is detailed, which is crucial for breaking the face into small, manageable triangles for accurate reshaping, warping, and blending. It extracts triangle landmark points, converts them to pixel coordinates, creates a bounding rectangle, and generates a mask for the triangle area.
The 'wrap_triangle' function is developed, which takes a source triangle and precisely reshapes and moves it to fit the corresponding destination triangle. This involves computing an affine transformation matrix and applying a bitwise AND operation with a mask to keep only the triangular area, ensuring extra pixels are removed.
The 'add_piece_of_new_face' function is created to blend each wrapped triangle into the new face image without overlapping. It extracts rectangle coordinates, crops the target region, converts it to grayscale, creates an inverse mask, and uses bitwise operations to combine the existing and new face pixel data.
The 'swap_new_face' function is built to integrate the new face onto the destination image seamlessly. This involves creating a face mask, filling the face area, inverting the mask, and removing the original face. Finally, OpenCV's 'seamlessClone' function is used to blend the new face naturally, ensuring realistic lighting and texture.
Back in 'app.py', the 'set_source_image' function is completed, initializing global variables for the source image, mask, landmarks, and triangle indices. The main application loop starts, continuously reading webcam frames, resizing them, converting to grayscale, and detecting facial landmarks using the functions from 'media_utils.py'.
Inside the main loop, triangulation is applied to both the source and destination faces. The 'wrap_triangle' function is used to perform affine transformation, aligning the source triangles with the destination. Then, 'add_piece_of_new_face' reconstructs the face triangle by triangle, and 'swap_new_face' performs the final seamless blending (cloning) to produce the swapped face.
Keyboard input is implemented to allow users to dynamically change the source celebrity image during runtime by pressing number keys. Additionally, code is added to automatically resize source images while maintaining their aspect ratio to prevent distortion during swapping, leading to a dynamic and visually corrected face-swapping experience.