Signal-specialized parameterization for piecewise linear reconstruction

We propose a metric for surface parameterization specialized to its signal that can be used to create more efficient, high-quality texture maps. Derived from Taylor expansion of signal error, our metric predicts the signal approximation error - the difference between the original surface signal and its reconstruction from the sampled texture. Unlike previous methods, our metric assumes piecewise-linear reconstruction, and thus makes a good approximation to bilinear reconstruction employed in graphics hardware. We achieve significant savings in texture area for a desired signal accuracy compared to the signal-specialized parameterization metric proposed by Sander et al. in the 2002 Eurographics Workshop on Rendering.


Introduction
Texture mapping features in graphics hardware are being increasingly used in real-time rendering.Surface signals play an important role in achieving a number of rendering effects.While these rendering effects can also be stored per vertex on a high resolution mesh, using a coarser mesh with textures is generally more efficient [COM98].
Texture mapping requires a surface to be parameterized onto a texture domain by assigning texture coordinates to its vertices.Given this parameterization, the surface signal is sampled into a texture image of a given resolution.Texture memory can become a scarce resource in complex scenes with many textured objects.In this paper we examine how to construct a parameterization to best represent a given surface signal using textures as compact as possible.We construct such a parameterization as an off-line, automatic preprocess.
The majority of surface parameterization schemes assume no a priori knowledge of the signal and instead minimize for various geometric distortion measures such as preservation of area and angles.Sander et al. build a surface parameterization optimized for a specific signal by trying to reduce the signal approximation error -the difference between the re-constructed signal and the original signal [SGSH02].While their approach already produces more efficient texture maps than signal-independent parameterizations, it is based on the assumption that the reconstruction is piecewise constant.The integrated metric tensors in their signal-specialized metric distinguish between constant and linear signals but do not differentiate between linear and higher-order signals.Given that graphics hardware employs bilinear interpolation, their metric could lead to over or undersampling, depending upon the surface signal.
In this paper, we examine minimizing the signal approximation error with the assumption that the reconstruction is piecewise linear.Based on Taylor polynomial expansion of signal error our metric consists of a weighted sum of squares of second derivatives of the mapping h from the texture domain (s,t) to an n dimensional surface signal.These second derivatives allow our metric to distinguish between linear and higher order signals and thus provide greater sensitivity to signal detail.This results in reduced signal approximation error for a given texture size (see Fig. 7).
The specific contributions of this paper are: • A novel error metric that integrates signal approximation c The Eurographics Association 2004.
error over the surface.It is derived using Taylor expansion of signal error with the assumption that the reconstruction is piecewise linear.
• A method to compute Hessians, second derivatives of the mapping h between the texture domain and the surface signal, using a least squares fitting method in order to evaluate the terms of the metric.These terms are precomputed and integrated over each face.• An affine transformation rule for efficient evaluation of the metric during the parameterization process.
We implement an efficient algorithm that minimizes the signal error over the parameterization, while maintaining an embedding.Our optimization algorithm incorporates a multiresolution hierarchy [SGSH02] to propagate metric information from fine-to-coarse and creates the parameterization in a coarse-to-fine manner.In Section 6 we compare the signal approximation error of texture maps created using our metric with that of Sander et al., and show that our metric achieves significant savings in texture area for a desired signal accuracy.

Previous Work
Signal Independent Parameterizations The problem of minimizing distortion while flattening a surface chart into 2D has been studied in great detail since 1995 as noted in the survey by Floater and Hormann [FH04].
Pinkall and Polthier first introduce the notion of using cotangent weights as a discrete measurement of Dirichlet energy with the aim of computing minimal surfaces [PP93].Eck et al. propose the use of Dirichlet energy minimization to parameterizing a mesh using harmonic maps [EDD * 95].The texture coordinates for boundary vertices, however, must be fixed a priori and harmonic maps may contain face flips (adjacent faces in texture space with opposite orientation) which violate the bijectivity of a parameterization.Duchamp et al. investigate multiresolution methods for computing harmonic maps [DCDA97].Based on earlier work by Tutte [Tut60], Floater [Flo97] proposes a different set of weights for the edge spring model that guarantees an embedding if the texture coordinates of the boundary are fixed to a convex polygon.Desbrun et al. define a space of measures spanned by a discrete version of the Dirichlet energy, and a discrete authalic energy [DMA02].
Hormann and Greiner propose the MIPS parameterization [HG00], which maximizes the conformality of the piecewise linear mapping without demanding the mesh boundary to be mapped onto a fixed shape.Another approach to minimize angular distortion is proposed by Sheffer and de Sturler [SdS01].They define a non-linear energy in terms of the corner angles of the mesh in texture space.Levy et al. formulate the discrete conformality problem as a quadratic minimization problem and prove the uniqueness and existence of its solution [LPRM02].Using standard numerical conjugate gradient solver they are able to compute least squares approximations to continuous conformal maps very efficiently without requiring fixed boundary texture coordinates.
There are some methods to compute parameterizations over a non-planar domain.Haker [SSGH01].To optimize for a uniform sampling, Sorkine et al. minimize the maximum of the maximum singular value and the inverse of the minimal singular value, which penalizes both under-and oversampling [SCOGL02].
Signal-specialized parameterizations Until recently, there has been little emphasis on exploiting knowledge of the surface signal in optimizing the parameterization.
Given an existing parameterization, Sloan et al. warp the texture domain onto itself to more evenly distribute a scalar importance field [SWB98].Unlike importance, our metric is derived directly from signal approximation error, and is integrated over the surface.Terzopoulos and Vasilescu approximate a 2D image using a warped grid of sample values [TV91].The warping is achieved using a dynamic simulation where grid edge weights are set according to local image content.We consider signals mapped onto surfaces in 3D, define the parameterization on a coarser, irregular mesh, and store the signal in a texture image mapped onto this mesh.Sander et al. build a signal-specialized parameterization in a mutigrid hierarchy and minimize the signal approximation error, the error between the original surface signal and its reconstruction from the sampled texture with the assumption that the reconstruction is piecewise constant [SGSH02].In contrast, our metric is derived assuming that our reconstruction method is piecewise linear.This more closely matches texture reconstruction hardware which does bilinear interpolation.This also holds true if the parameterization is used for remeshing since the output remesh is typically made up c The Eurographics Association 2004. of triangles which linearly interpolate the vertices.In Section 6 we demonstrate significant improvement in the quality of texture maps obtained for a desired level of signal accuracy over a variety of different models.
Balmelli et al. create space-efficient texture maps by distributing frequency content uniformly across an image using a wavelet packet decomposition technique and denoising filter [BBT02].The frequency map captures the relative importance of different regions in the image, and causes the image to be stretched in high frequency areas, and contracted in low frequency areas.There are two main differences between our approach and that adopted by Balmelli et al.First, our metric is derived specifically to reduce reconstruction errors.It is able to differentiate directional differences in the variability of the signal, and it agrees with optimality results from approximation theory.Second, given some signal over a mesh, our method simply solves for a parametrization of the mesh, and then samples and stores that signal as a single texture.In contrast, Balmelli et al. begin with the signal represented as an input texture and with associated texture coordinates.They warp the image in order to more efficiently use the texture space and subsequently update the texture coordinates to account for the warping.However, since the warping is performed on a different grid than the triangulation of the mesh, this necessarily creates some slipping of the texture over the surface.

Signals over Meshes
Let the surface signal be denoted by the function g : S → Q, where the signal-space Q can be vector-valued (e.g.RGB color is a 3-vector in Q).There are a number of ways a surface signal may be defined over the mesh.For example, a signal may result from the evaluation of a procedural texture computation.In this context, we wish to parameterize S and sample this signal over the domain D. One also may begin with a signal defined by a high-resolution image-texture and a given mapping onto S. In this case we may wish to parameterize S, and resample the original signal at some lower resolution.

¢¡ ¤£ ¦¥ ¨ § © ¥ ! " § $# %© ¢¥ ¨&
One can also define a signal over a low resolution mesh S, using a high resolution mesh of the same model.If attributes (e.g.RGB color, or normals) are specified at each vertex of this high resolution mesh, linear interpolation then defines a signal at each point on the high resolution mesh.Normal shooting [SGG * 00] can then be used to create a correspondance between S and the high resolution mesh, thus defining a signal over S. All of our examples are of this type.
Normal shooting involves interpolating surface normals of the face on which a subvertex lies on the low resolution mesh S, and shooting rays in the direction of these interpolated normals to the high resolution mesh.During rayshooting, a ray may fail to hit the high resolution model, in which case we use the nearest high resolution mesh vertex to the subvertex to estimate its signal.

Metric Derivation
To find the surface parameterization f , we examine how well the function h = g • f (from the texture domain D to the signal-space Q) is approximated when reconstructed from a discrete sampling over D (see Fig. 1).
In this section we derive a metric for signal approximation error, E h (s,t), defined as the difference between h and its reconstruction ĥ from a discrete sampling with spacing δ in D with the assumptions: (1) ĥ is a piecewise linear reconstruction, and (2) the sampling is asymptotically dense.
We assume that the domain D contains a regular grid of sample points (s i ,t j ), spaced 2δ apart on each axis as illustrated in Fig. 2. Let ( ŝ, t) ∈ [−δ, +δ]×[−δ, +δ] be a local coordinate system within the grid square i j about each sample, such that (s,t) = (s i + ŝ,t j + t) ∈ i j.Given that texture  each sample is given by a function with a constant and linear term in it: ĥi j (s,t) = h(s i ,t j )+[hs(s i ,t j ), ht (s i ,t j )] ŝ t .
With this reconstruction function the pointwise squared error can be expressed as: Using a Taylor expansion about (s i ,t j ), this can be written as where the error term from the squared Taylor expansion gives rise to terms of order O(δ 5 ), which are insignificant compared to the first term of order δ 4 as δ → 0. The first term, corresponding to the third term of the Taylor expansion, is defined via To find the sum of squared error over all components of a vector function, we obtain where n is the dimensionality of the signal function h (for instance n = 3 for RGB color), and h k is the k-th component of h, and where is the integrated H of the signal function h, and welsum (weighted element sum) is defined via Reducing this integral to a sum over domain triangles, T , we obtain where With very similar analysis it is possible to derive a measure for bilinear reconstruction.For the sake of simplicity, we chose to implement the metric for piecewise-linear reconstruction.

Properties of our Metric
The approximation theory/finite element literature provides various asymptotic results on optimal triangulations of the plane.In particular, suppose one wishes to approximate c The Eurographics Association 2004.some bivariate scalar function g(x, y) using linear interpolation with a given number of triangles over the (x, y) plane.If approximation error is measured in the L 2 sense then, according to Nadler, as the number of triangles goes to infinity, an optimal triangle's orientation is given by the eigenvectors of the Hessian of g and its aspect ratio by where λmax and λ min are eigenvalues of the Hessian of the approximation function [Nad86].It has also been shown that a mesh best approximates a smooth surface if the anisotropy of the mesh follows (in non-hyperbolic regions) the eigenvalues and eigenvectors of the curvature tensor of the smooth surface regions [Sim94].It can be shown that, at least locally, our metric agrees with Nadler's result.Assume that our surface S is the unit square on the x, y plane, and that the signal is some quadratic function g(x, y) = Ax 2 + By 2 + F for which |A| > |B|.This considers quadratics in normal form; a general quadratic can always be reduced to normal form by a translation and rotation of the x, y domain.It is easy to see that 2A and 2B are eigenvalues of the Hessian of g 2A 0 0 2B In this case, Nadler's result implies that the optimal sampling on a uniform grid will be axis aligned and have aspect ratio of Now, assume that we parameterize S over the (s,t) texture domain, using a single affine map from S to a rectangle R of width w, height 1 w , and orientation θ (see Fig. 3).Over these two degrees of freedom, (w, θ), we minimized our definition of signal error in closed form and verified that the following are indeed local minima as predicted by [Nad86]: θ an integer multiple of π 2 , and w = | B A | 1 4 .Thus our metric also obtains the optimal orientation (axis-aligned) and optimal aspect ratio (w/h = w Nadler further identifies a one degree of freedom family of optimal solutions for the case in which the Hessian has a negative eigenvalue (i.e, the surface is locally hyperbolic) [Nad86].It remains for future work to show that this entire family minimizes our metric, and that our metric admits no minima outside Nadler's set.

Affine Transformation Rule for H
During optimization, we need to repeatedly modify the parameterizations of mesh vertices and compute the resulting change in the signal error.Recomputing H matrices based on the modified parameterization is undesirable since this involves expensive numerical integration as well as recomputing second derivatives of h.
Since modification of the parameterization is simply an affine transformation of each mesh triangle, we can exactly compute H of a transformed triangle from its original H without having to do the computation explicitly.Expressing this as a linear system in the untransformed second derivatives: In the case of a vector function h, we can absorb the sum over components inside the H term. Integrating, we obtain: where Q is a function of the Jacobian as defined above, and δ i is the ith triangle over the surface.Since the Jacobian is constant within each triangle δ i , we can factor out c The Eurographics Association 2004.
the multiplication by Q in the above integral yielding a transformation rule for H(S) welsum( H (S))

Numerical Computation of H
We described how to apply the affine transformation rule on the H matrices to quickly evaluate the signal error metric.However, in order to use the affine transformation rule during the parameterization process, we need to have initial H matrices stored on the original mesh faces.
At the outset, we do not have a parameterization of the original mesh that we could use to compute the initial H matrices.Thus, for each face, we create a canonical parameterization based on a local isometric flattening of that face and its three neighboring faces.As described below, we will also need to sample these neighboring faces in order to accurately compute the H matrices.
During the optimization process, the affine transformation rule is used to transform the H matrices from their canonical parametrization to the current parameterization that is being evaluated.In this section we describe how to pre-compute H matrices on the original mesh faces with respect to canonical face parameterizations.
We assume that a continuous signal (e.g.normal map as shown in Fig. 4) exists over the surface.Our method of computing H( i ), is based on a numerical integration approach.Specifically we apply a number of regular 1-to-4 subdivisions to a face and obtain subfaces and subvertices.We compute H at all the subfaces, sum all of these up, and multiply by the geometric area of the face, in order to obtain H( i ).
The level of subdivision depends on the detail of the mesh.For all examples in this paper, we subdivided each face into 64 subfaces (3 subdivisions) to compute H.To compute H at a subface, we sample the signal at some number of neighboring subvertices.In our examples, we selected the 15 closest subvertices after the 3 regular 1-to-4 subdivisions.We then estimate the second derivatives of the mapping h by finding the best fitting quadratic signal to these samples (the coefficients A, B, C, D, E and F of the quadratic f = As 2 + Bt 2 + Cst + Ds + Et + F).We then use the second derivatives of this quadratic approximation.
It is possible for our signals to vary linearly within two adjacent mesh faces but have dramatically different derivatives across the shared edge.Unless the subvertices used to compute the second derivatives of h over a face include subvertices from adjacent faces, the second derivatives estimated by our Least Squares solver along the boundary of the face will be inaccurate.This could lead to high signal approximation error.In order to prevent this problem, as described above, we also include subvertices from the three neighboring faces using the face's local isometric parameterization.During numerical integration, when selecting the 15 closest subvertices, we also allow these subvertices to come from these neighboring faces.

Chart parameterization algorithm
The goal of the optimization procedure is to minimize signal error over the parameterizations of the mesh vertices, while maintaining an embedding.Our optimization algorithm is based on that of Sander et al. [SSGH01].For completeness we summarize this algorithm.
After obtaining an initial parameterization, we minimize the signal error by repeatedly perturbing each vertex within the kernel of the polygon formed by its neighboring vertices.To improve the speed and result of the optimizations we adopt the multiresolution optimization scheme adapted by Sander et al. [SGSH02].We use a multiresolution progressive mesh [Hop96] sequence to propagate H fine-to-coarse (FTC) from the original mesh to all coarser meshes and apply a coarse-to-fine (CTF) parameterization algorithm that uses these Hs.During FTC we redistribute the H according to Sander et al.'s scheme of redistributing integrated metric tensors (IMTs) [SGSH02].We use a CTF optimization with the geometric stretch metric [SSGH01] to obtain an initial parameterization in order to transform the Hs on the finest level mesh, and thus bootstrap the iterative optimization process.
This parameterization algorithm allows chart boundary vertices to move in the texture domain at all levels of the CTF optimization algorithm.Our error metric is not scale invariant, and thus the signal error could go to zero as the chart becomes infinitely large.We achieve scale-invariance by multiplying signal error by the square of the total chart area.
The high-level algorithm can be summarized as: c The Eurographics Association 2004.

function OptimizeChartParameterization
Pre-compute canonical Hs on fine mesh faces Construct progressive mesh of chart //Create an initial parameterization Do CTF using geometric-stretch.//Iteratively optimize using error metric: for n iterations Transform fine mesh Hs using current param.FTC propagate Hs to all PM meshes.CTF optimize signal error using Hs.

Results
We have created signal-specialized parameterizations for several models and compared them both quantitatively and qualitatively against the signal-specialized parameterization metric of Sander et al.The comparisons are shown in Fig. 7-Fig 11 .All models originated from 3D scanning.The signals on the parasaur, fandisk, and cat are per-vertex normals on a high resolution mesh, mapped to a low resolution mesh using normal-shooting.The signal on the face in Fig. 11 is defined using per vertex color data on a high resolution mesh, mapped to a low resolution mesh using normal-shooting.For the fandisk model in Fig. 9 we manually partitioned the mesh into 4 charts.Table 1 shows a comparison between the running time of our metric with that of Sander et al. on several models.Our parameterization scheme takes a few minutes to run per model.
To quantify parameterization quality, we measure signal approximation error (SAE) as rms difference on a dense set of surface points, distributed uniformly according to surface area.For each point we compute the difference between the original signal and the bilinear interpolation of the four adjacent texture samples.For vector-valued signals we use the L 2 norm.In Fig. 5-6 we compare the reduction in SAE by our metric with that of Sander et al.We refer to the former as signal error with piecewise linear reconstruction, and the latter as signal error with piecewise constant reconstruction.The graphs show the signal error as a function of the number of texture samples for two parameterizations.
To prevent the formation of degenerate triangles when the signal is locally constant on a region of the surface, Sander  et al. add a tiny fraction of geometric stretch to their energy functional [SGSH02].For our metric we find it necessary to do this as well on models such as the fandisk.Empirical analysis shows that the fraction of geometric stretch can have a significant impact in reducing the SAE of the parameterization.Furthermore, it seems that the amount of geometric stretch needed is dependent upon the scale of the error functional and varies from model to model.Therefore, in order to ensure fairness in our experimental comparisons in Fig. 6, we only add sufficient geometric stretch to allow the parameterization algorithm to converge to an optimal solution for both metrics.Empirical analysis also shows that for each model there is some value of the fraction of geometric stretch that produces a parameterization with the lowest SAE.For experiments with the models in Fig. 10-Fig.11 we compare parameterizations created by our metric and the metric by Sander et al. combined with their optimal geometric stretch value that is predetermined experimentally.
Both graphs in Fig. 5-6 show a significant reduction in error from the signal error with piecewise constant reconstrucc The Eurographics Association 2004.tion to the signal error with piecewise linear reconstruction.In particular Fig. 5 shows that using our metric, a given signal approximation error can be obtained with up to a factor of 4 savings in texture size.Fig. 7 illustrates the parameterizations of the two metrics for the parasaur model used in Fig. 5 for specific texture resolutions.

Summary and future work
Motivated by the need to reduce the size of texture maps used in real-time rendering systems, we introduced a parameterization metric that is derived from a Taylor expansion of signal approximation error.Unlike former methods, ours assumes piecewise linear reconstruction and is thus a good approximation to bilinear reconstruction employed in graphics hardware.In order to empirically evaluate our metric, we have implemented a multiresolution parameterization algorithm that minimizes the signal error defined by our metric.
Our signal-specialized parameterization metric allocates more texture samples to mesh regions with greater signal variation.In particular, the integrated H term in our metric allows it to distinguish between linear and higher order signals and allocate more texels to the latter than the former.As shown in Fig. 5-6, a factor of 4 of savings in texture space is possible using our signal-specialized parameterization method.
There are several areas of related work that could be explored in the future.Our metric does not place any restriction on the dimensionality of the signal.It is possible to specialize a parameterization to a combination of signals, such as normals and colors.Another area for future work is to explore perceptual measures in addition to signal approximation error and propagate these measures through the rendering process.

Figure 1 :
Figure 1: We examine the mapping h = g • f .

Figure 3 :
Figure 3: A rectangle R of width w rotated by θ.
su rfa c e m e sh c h a rt (1 3 1 6 v e rtic e s) n o rm a l-fie ld sig n a l (R G B = n x ,n y ,n z ) sh a d e d su rfa c e

Figure 4 :
Figure 4: Our input consists of surface mesh and an associated surface signal.Here the input is a single chart of the fandisk mesh where the normal (nx, ny, nz) is the signal.

Figure 7 :
Figure 7: Our parameterization algorithm can reduce texture size by more than a factor of 4.
Let e : D → D : (s ,t ) → (s,t) be a local affine transform from the new triangle parameterization to the old, resulting in the new map h = h • e.Let J(s,t) be the Jacobian of the mapping e =p(s,t) q(s,t) r(s,t) s(s,t)where e 1 denotes the first coordinate of e and e 2 the second and where the coordinates (s,t) now parameterize the domain of e and thus h .Thus the transformed second derivatives of a scalar function h are given by

Table 1 :
A comparison of running time in seconds.