In , Wang et al. conducted a comprehensive survey aiming at helping readers to gain a thorough understanding of the Face Hallucination (FH) research landscape. In , Autee et al. studied the various approaches and methodologies used for FH.
Considering the critical role of image interpretation in modern intelligent systems, FH has attracted growing attention in recent years. FH fuels a wide spectrum of applications in authentication,surveillance monitoring, law enforcement, security control, entertainment, bio-metrics, digital entertainment, rendered services for a legitimate user only, age synthesis and estimation, forensic art, electronic customer relationship management, and cosmetology [1, 2].
In most surveillance imaging framework, the camera and the objects of interest are at the large interest in the scene that results in normally low resolution of objects that are taken. In automatic face recognition and identification methods, it is very much required to improve the resolution of the faces. FH or face super resolution as proposed by Baker and Kanade serves this purpose .
FH refers to both face super-resolution (FSR) and face sketch-photo synthesis (FSPS) because they share the similar intrinsic mathematical model; that is, they infer an image lying in an image space from its corresponding counterpart lying in another space .
Although face super-resolution and FSPS share the similar framework, this does not mean that methods which work well for FSR also work well on face sketch-photo synthesis and vice versa. This indicates that applying face super-resolution techniques directly to FSPS may not always achieve good performance and vice versa. This may be due to the fact that though down-sampling and blurring effect are the main factors of difference between low-resolution and high-resolution images, they have the similar texture or intensity expressions. However, sketches and photos are in quite different texture expressions .
In cases where low-resolution face images are acquired by live surveillance cameras at a distance, FH techniques can be used to enhance low-resolution images and transform sketches to photos and photos to sketches for the subsequent utilization .
1. Sanity constraint: the target HR image should be very close to the input LR image when smoothed and down-sampled.
2. Global constraint: the target HR image should have the common characteristics of human faces, e.g., possessing a mouth and a nose, being symmetrical, etc.
3. Local constraint: the target HR image should have the specific characteristics of the original LR face image, with photorealistic local features.
A number of related super-resolution and face hallucination algorithms have been proposed, which can be grouped into three types; Interpolation- , Reconstruction-, and Learning-based methods .
Interpolation-based algorithms (e.g. Bilinear, Cubic B-Spline) suffer from severe blurring problem especially when the resolution of the input is very low .
Reconstruction-based methods [6, 7], which try to model the process of image formulation to build the relationship between LRI and HRI based on reconstruction constraints and smoothness constraints, are quite limited by the number of input LRIs and usually cannot work well in single-image super-resolution problem .
Learning-based methods explore mapping relations between high- and low-resolution image pairs to infer high- resolution images from their low-resolution counterparts . Recently, learning-based methods become very popular. Usually, the unknown HRI is inferred by making use of some training set directly or indirectly. Compared with other methods, learning-based method can achieve higher magnification factor and output better results especially for single-image super-resolution problem .
Existing Learning-based methods can be grouped into four categories: Bayesian inference approaches, sub-space learning approaches, a combination of Bayesian inference and subspace learning approaches, and sparse representation-based approaches .
By means of a comprehensive analysis and comparison of these methods, Bayesian inference methods was found to have the disadvantage of high computation cost and heavy memory load, although neighbor compatibility reduces the boundary noise (except for the gradient-based prior for data modeling-based methods) .
Subspace learning-based methods were also found to make strict assumptions about the geometric structure of two image spaces and low computation cost. Thus, The combination of these two frameworks may result in a more accurate method (except for the LLE-based methods in this category) .
Although different from subspace learning-based methods, sparse representation-based methods also assume that two image spaces share a similar geometric structure; however, this assumption is constrained on two sparse spaces. This relaxes the original, much more restrictive assumption to some extent .
1. Nannan Wang, Dacheng Tao, Xinbo Gao, Xuelong Li & Jie Li. A Comprehensive Survey to Face Hallucination. International Journal of Computer Vision, Volume 106 Issue 1, January 2014.
2. Prachi Autee, Samyak Mehta, Sampada Desai, Vinaya Sawant& Anuja Nagare. A Review of Various Approaches To Face Hallucination. International Conference on Advanced Computing Technologies and Applications, (ICACTA-2015).
3. Simon Baker & Takeo Kanade. Hallucinating Faces. Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000, Page 83.
4. Ce Liu, Heung-Yeung Shum & William T. Freeman. Face Hallucination: Theory and Practice. International Journal of Computer Vision, Volume 75 Issue 1, October 2007, Pages 115-134.
5. Wei Zhang & Wai-Kuen Cham. Hallucinating Face in the DCT Domain. IEEE Transactions on Image Processing, Volume 20 Issue 10, October 2011, Page 2769-2779.
6. B. S. Morse and D. Schwartzwald. Image magnification using level-set reconstruction. In Proc. of CVPR, pp.333-340, 2001.
7. Z. Lin and H. Y. Shum. Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation. IEEE Trans. PAMI, Vol. 26, No. 1, pp. 83-97, 2004.