Navigation
3D Reconstruction from 2D Human Images
Gulraiz Khan's PhD project at the University of Hull began in January 2023 under the primary supervision of Professor Kevin Pimbblet. I became his secondary supervisor in July 2023.
In the first part of his project, Gulraiz developed a deep learning framework to predict facial landmarks, age, and gender from human face images. The framework employs ResNet-18 as a backbone model for feature extraction. The extracted convolutional feature maps are shared across three lightweight neural network branches. These branches predict facial landmarks, age, and gender respectively.
In the second part, he built a modular deep learning framework for real-time, identity-aware, and volume-aware 3D reconstruction called SARS. Two 3D morphable models representing an individual's face and body are generated by two separate modules. Prior to generation, the embedding of the facial image is combined with the facial landmarks, age, and gender predicted by the extraction module for the individual. The face and body models are merged by an integration module.
In the final stage of the project, he improved the body module. He found the relationship between the shape coefficients (SMPL-X parameters) and the dimensions of various body parts (e.g. leg thickness). This allowed him to develop a visual language model for natural language–guided body shape control. From a 2D image and a text prompt, the model autoregressively predicts SMPL-X parameters as quantised position tokens. Parameterised by it, the body module supports two inference modes: standard body reconstruction and natural language–guided shape editing.