This work addresses the issue of ground truth annotation by measuring head pose using a motion capture system and eye gaze using mobile eyetracking glasses and applies semantic image inpainting to the area covered by the glasses to bridge the gap between training and testing images by removing the obtrusiveness of the glasses.