Face Hallucination

Face Hallucination Using Bayesian Global Estimation and Local Basis Selection

Hsu et al.[1] proposed a maximum a posteriori (MAP) estimator to estimate the optimum set of coefficients in the low-dimensional coefficient domain.

In the local refinement step, the facial parts (i.e., eyes, nose and mouth) are further refined using a basis selection method based on overcomplete nonnegative matrix factorization (ONMF).

Example-based face hallucination schemes can be classified into two major kinds of approaches: prototype-face-based approaches [3][6] and model-based approaches [2][8][10].

The prototype-faces-based methods proposed in [3] decomposes an input LR face into a set of prototype faces (e.g., eigenfaces) as prior models using principle components analysis (PCA).

The model-based approaches generally utilize probabilistic models to find the relationship between HR images and their corresponding LR versions.

Recently, the manifold learning techniques have been used in face hallucination methods and more details of the hallucinated face image can be preserved since manifold learning can preserve the neighboring structures in the pixel domain.

Besides, PCA, nonnegative matrix factorization (NMF) has also been used for basis decomposition for face recognition [9] and face hallucination [4].

The NMF method, like PCA, represents a facial image as a linear combination of basis images, whereas the NMF method can better learn localized parts-based representation for face images. For example, NMF can yield a decomposition of human faces into parts reminiscent of features such as lips, eyes, and nose.

Compared to NMF, the reconstruction results of PCA are not that intuitive and hard to interpret as PCA allows subtractive combinations of the basis images [9]. Moreover, the NMF-based face recognition schemes have been shown to achieve better performance than PCA-based schemes do [9], [14].

However, the NMF basis decomposition function used in [4] is an incomplete basis since the number of the bases is restricted by its definition. Usually, an overcomplete bases set provides better performance in term of the quality of the reconstructed images [13], especially in local facial features. For example, an overcomplete NMF (ONMF) method was proposed in [14] which can represnt the local facial features well.

The ill-posedness problem

Estimating the corresponding HR details from a LR face is in nature an ill-posed inverse problem. Therefore, priors such as spatio-temporal consistency, sparsity of signal representation, and structures of faces, are used to relax the ill-posedness of the HR face reconstruction. Recently, it was shown in [4][5] that learning the statistical structures of faces (e.g., the prototype faces) as priors is an effective way to address the ill-posedness problem.

Hsu et al.[1] method is built on top of the prototype-faces-based framework proposed in [3][4]. This method exploits a whole LR face image, rather than dividing the face into small patches like in [5], to reconstruct the HR face.

After computing the coefficients of the input LR face image corresponding to a set of LR prototype faces, the HR face image can be hallucinated via a linear combination of HR prototype faces weighted by the coefficients as illustrated in Fig. 1.

The coefficients obtained from decomposing the LR face image, however, are typically not very accurate for synthesizing the HR counterpart. Therefore, in the reconstruction phase, an iterative method based on back projection was proposed in [3] to refine the estimated the coefficients corresponding to the HR prototype faces for face hallucination.

As shown in Fig. 2, the proposed face hallucination method consists of two phases: the training phase and the hallucination phase.


[1]C.C. Hsu, C.W. Lin, C.T. Hsu, and H.Y. Mark Liao, “Face hallucination using Bayesian global estimation and local basis selection,” in Proc. IEEE Workshop Multimedia Signal Processing (MMSP), Saint-Malo, France, Oct. 2010.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s