healthmedicinet

Quantification and averaging methods

The main purpose of this project is to create an average face from a random set of
faces. We are interested in finding point-to-point correspondences across a large,
potentially highly variable population of human face models. Using surface mapping,
this project will refine and apply tools developed in neuroimaging research to create
surface-based maps of the human face. On a broad level, the project can be divided
into five parts, summarized below, and described in more detail in this section:

1. Collection of sample 3D Face models

2. Surface topology correction and spherical mapping

a. Non-manifold polygon correction

b. Boundary closure and Smoothing

c. Spherical Mapping

3. Shape registration

a. Initial Spherical Matching

b. Texture Matching

c. Geometry Matching and Registration Model

4. Average and distance map creation

a. Procrustes alignment/ Tensor Based Morphometry

b. Average and distance map creation, shape statistics analysis

5. Pilot study

a. Evaluation of individual shape morphology compared to averaged face

Collection of 3D face models

The novel nature of this investigation requires the formulation of a standard protocol
for consistent image acquisition using the 3dMD facial imaging system. Our goal was
to create an ideal environment and maintain consistent image acquisition for individual
subjects over the duration of comprehensive treatment time. Natural head posture (NHP)
was adopted for this study because it has been shown to be clinically reproducible
8]–10]. IRB approval was acquired for this project.

Surface topology correction and spherical mapping

In non-technical terms, “topology” essentially refers to the number of handles, islands
and boundaries of the surface. Since no well-defined correspondence between surfaces
of different topologies is theoretically possible, one must perform a topological
correction of each facial model before computing a dense correspondence. The simplest
and most common approach is to make each surface topologically equivalent to a sphere.

Non-manifold polygon correction

Removing triangles (“faces”) and vertices of non-manifold nature from polygon models
is a fairly common problem in 3D modeling.

Boundary closure and smoothing

We propose a boundary closure and surface extrapolation procedure similar to our previous
work with shape correspondence 11]. Each boundary is initially “sewn together” with a new set of triangle faces, and
the surface area of the new surface patch is minimized using standard linear optimization
techniques with boundary conditions 12]. The triangulation of the patch is then subdivided into more faces, and the process
is repeated iteratively, until the reduction in surface area is sufficiently small.
This procedure is guaranteed to produce face models of spherical topology.

Spherical mapping

To enable efficient correspondence search across a dataset of faces, it is necessary
to create an intermediate mapping to a common canonical space, where the final registration
may be performed. A correspondence search on a sphere was performed, where all points
moved around freely matching geometry appropriately, and matching extraneous tissue
in some models to filler regions in others, as appropriate.

Shape registration

Several shape registration techniques exist for genus zero shapes (shapes of spherical
topology), including those based on spherical parameterization. Among these are the
rigid spherical cross-correlation 13], spherical demons 14], Laplace-Beltrami Eigen-function registration 15], just to name a few. The unique challenge for 3D face models, not addressed by existing
methods, lies in the need to combine texture information from the coloring of the
face and face geometry. Our proposed method would find dense correspondence across
a set of faces using both texture and geometry information, while maintaining sufficient
flexibility to deal with non-face regions of the model.

Initial spherical matching

To ensure a robust initial map, we used a curve-matching algorithm. A simple set of
10 curves was manually drawn on each face model using the BrainSuite 14 software,
taking roughly 5 minutes per model by a trained operator. The correspondence between
landmark points on the facial surface was determined automatically via the arclength
map. We attain spherical displacements between corresponding curve points projected
onto the sphere. This initial map suffices for further local refinement described
below.

Texture matching

To minimize the mismatch between texture maps, we choose to use multi-channel Mutual
Information criteria, often used to match 2D and 3D medical images from different
modalities 16]. This choice is motivated by the fact that facial texture correspondence is characterized
by complex relationships between intensities of different color channels, without
a straight-forward transfer function. For example, different individuals may have
entirely different color composition of their eyes and skin.

Geometry matching and registration model

In addition to texture, geometry mismatch will be simultaneously minimized following
14], 17] based on position- and orientation-invariant features such as mean and Gaussian curvature.
We choose the fluid spherical registration model, which we developed recently in LONI’s
past study in mapping the hippocampus in Alzheimer’s patients 17], because it is maximally flexible and completely agnostic with respect to the mismatch
function, unlike previous methods. 13], 14] We believe that matching texture and geometry simultaneously will lead to the best
most accurate mapping.

Average and distance map creation

Procrustes alignment/tensor-based morphometry

To compute the average face, we must first align the shape models in their original
(not parametric) space, based on the computed dense correspondence. We will use the
7-parameter Procrustes method for this 18], excluding filler patches from the mismatch cost. An alternative measure of face
morphometry, called Tensor Based Morphometry, has gained popularity in recent years
19]. Unlike distance-based features above, the TBM features invariant to the position
and orientation of the shapes, making accurate Procrustes alignment a non-issue.

Distance and statistical maps

Having computed the average shape, we will compute the distance from the average to
each shape at each point. Distance-to-average maps are displayed as colorized surface
maps. Looking to the future, given a discrete, or continuous biological variable,
such as whether the subject carries a certain gene, or some clinical measure of severity
of a particular deformity, it is it possible to create statistical parametric maps
based on distance to the average. These typically involve parametric or non-parametric
(e.g. permutation) statistical tests done at each point 20], which localize the effect of the biological variable on the face surface.

Each single 3dMD individual image consists of roughly 32,000 vertices which represent
an x,y,z coordinate on a Cartesian coordinate scale. Each averaged facial image retains
this 32,000 vertex mesh with each single vertex possessing a specific variance. Therefore,
superimposition of two average samples essentially represents a statistical p-map
representing deviation from the norm.

Proof of methods/pilot study

A critical aspect in our analysis of the human face is to demonstrate that these methods
can be used to average faces with variable morphology. In order to demonstrate our
proof of methods, we have subjected our pipeline to several processes to demonstrate
the function, accuracy, and potential applications for our pipeline.

A) A random non-homogenous sample of 10 subjects was selected with no exclusions on
gender, age, and ethnicity. These subjects’ images were plugged into our averaging
pipeline to create a true 3D average.

B) Furthermore, 3D distance?to?average maps displayed as colorized surface maps which
will show individual deviation from our normative 10 sample average were created.
This is a key feature that will allow for comparison of individual facial morphology
to age, gender and race specific normative models for specific populations.

C) In order to demonstrate viability of the pipeline to average samples accurately
over multiple time points, the T0 (initial) average of our 10 non-homogenous samples
to their corresponding T4 (4 week) average was generated. This will illustrate the
significance and accuracy of our average T0 to average T4 superimpositions and serve
as a clear proof of methods.