DeepHand: Robust Hand Pose Estimation by Completing a Matrix Imputed with Deep Features

by | Jun 21, 2016

Authors: Ayan Sinha*, Chiho Choi*, and Karthik Ramani (* denotes equal contribution)
In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016, Las Vegas, USA.


We propose DeepHand to estimate the 3D pose of a hand using depth data from commercial 3D sensors. We discriminatively train convolutional neural networks to output a low dimensional activation feature given a depth map. This activation feature vector is representative of the global or local joint angle parameters of a hand pose. We efficiently identify ‘spatial’ nearest neighbors to the activation feature, from a database of features corresponding to synthetic depth maps, and store some ‘temporal’ neighbors from previous frames. Our matrix completion algorithm uses these ‘spatio-temporal’ activation features and the corresponding known pose parameter values to to estimate the unknown pose parameters of the input feature vector. Our database of activation features supplements large viewpoint coverage and our hierarchical estimation of pose parameters is robust to occlusions. We show that our approach compares favorably to state-of-the-art methods while achieving real time performance (32 FPS) on a standard computer.

Please click here to download our paper.

Supplementary material is also available, click here to download.

Chiho Choi

Chiho Choi

Chiho Choi is currently working at Honda Research Laboratory, Palo Alto, CA. He received his degree of Ph.D. in the school of Electrical and Computer Engineering at Purdue University. Dr. Choi received a B.S. from Hanyang University, Korea in 2011, and a M.S. from University of Southern California, in 2013. His research interest lies at the intersection of machine learning and computer vision. He broadly builds machine learning algorithms in a practical way for computer vision systems. Currently,Choi is working as a research assistant in the Computational Design and Innovation Lab led by Professor Karthik Ramani, focusing on human shape interaction. [Personal Website][LinkedIn]