Face Hallucination

Promising Future Directions and Tasks for Face Hallucination

Finally, several promising future directions and tasks were proposed by [Wang et al. 2013].

an automatic objective image quality assessment metric is
essential in evaluating the performance of the FH algorithm.
Classical full reference metrics such as PSNR, MSE, and
RMSE are holistic and cannot yet reflect the detailed infor-
mation that is needed to assess image quality. This point
is discussed in detail by Wang and Bovik (2009).

Several metrics such as
UIQI, SSIM, VIF (Sheikh and Bovik 2006), and FSIM
(Zhang et al. 2011b) have been proposed; however, none
of them is specialized for hallucinated face images, which
have their own unique characteristics due to both the struc-
ture of the face and the property of the hallucinated image.
Hence, synthesized face image quality assessment may be a
promising and helpful research direction.

Recently, sparse representation has achieved great
progress in computer vision (Wright et al. 2010) and data
analysis (Zhou and Tao 2013). In particular, methods have
been proposed for image reconstruction and state-of-the-art
results have been obtained (Mairal et al. 2008a,b). Yang
et al. (2008b) applied the idea of the sparse representation
model with a coupled learning process to face image super-
resolution and achieved good results. Yang et al.’s method
(2008b) is not the end of the application of sparse represen-
tation to FH, since the method considers less prior knowledge
of the face image than the face images provide, and the effec-
tive exploration of the sparsity of face images is therefore an
interesting problem to resolve.

Most image databases used
for face image super-resolution were not sampled from sur-
veillance camera videos, since the main application of face
image super-resolution is face recognition or face retrieval
from a monitor. Therefore, an image database extracted from
surveillance videos should be constructed that incorporates pose, illumination, expression, and view variant images.
Although the CUFS database has been constructed, there is
only one sketch with neutral expression and front view corre-
sponding to each photo in the database for face sketch-photo
synthesis; therefore, constructing a database containing sev-
eral sketches corresponding to each photo across multiple
modalities is essential. Furthermore, these two databases will
stimulate the progress of study on multi-modality FH and

Though FSR and FSPS share a similar mathematical form,
they are intrinsically different. The first difference comes
from how much the face alignment precision affects the hal-
lucination. Face alignment is a critical preprocessing phase
before FH, because imprecise localization of the facial fea-
tures (landmarks) degrades the subsequent processes. Exper-
iments (Liu et al. 2007a; Jia and Gong 2008; Luo et al. 2012)
indicate accurate face alignment is more important for FSR
than for FSPS. Because face sketches and corresponding
photo counterparts are generally in high or moderate resolu-
tion, their alignment is relatively easier. Even a small amount
of misalignment can dramatically degenerate the FSR perfor-
mance. Low-resolution images usually have blurring effect
and contain limited structure information, and so many ambi-
guities exist for facial landmark localization which raises the
alignment of low-resolution face images a challenging prob-

Another difference lies in whether they need to handle
the problem of shape exaggeration. Artists usually exagger-
ate some distinctive facial features when they draw sketches,
which results in some deformation. Wang and Tang (2009)
explained that “if a face has a big nose in a photo, the nose
drawn in the sketch will be even bigger”. Consequently, in
contrast to FSR, FSPS needs to handle the problem of shape
Besides learning-based face sketch synthesis methods sur-
veyed in this paper, some sketch synthesis algorithms are
not learning-based (Kang et al. 2005; Wen et al. 2006).
Whatever these methods are, they are applicable to general
images. However, they can hardly handle the styles by dif-
ferent artists. This is because different artists may have dif-
ferent representation and exaggeration styles for many parts of a face. For example, different artists may render the nose,
eye, mouth and other parts of a face differently. It may be
even more difficult to model these different artistic styles than
model the shape exaggeration. To learn these different styles,
some discriminative information among them may favor the
synthesis process since it can assist to choose a sketch part
(here face part can be a face patch or a holistic face) from
sketches of desired styles.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s