Polyhedral Geometry and the Two-plane Parameterization

Recently the light-ﬁeld and lumigraph systems have been proposed as general methods of representing the visual information present in a scene. These methods represent this information as a 4D function of light over the domain of directed lines. These systems use the intersection points of the lines on two planes to parameterize the lines in space. This paper explores the structure of the two-plane parameterization in detail. In particular we analyze the association between the geometry of the scene and subsets of the 4D data. The answers to these questions are essential to understanding the relationship between a lumigraph, and the geometry that it attempts to represent. This knowledge is potentially important for a variety of applications such as extracting shape from lumi-graph data, and lumigraph compression.


Introduction
Recently the light-field and lumigraph systems have been proposed as general methods of representing the visual information present in a scene [7,9].These methods represent this information as a 4D function of light over the domain of directed lines.This information can be captured using video cameras, without requiring the solution to a stereo vision problem.Given this representation one can generate novel views of the scene quickly by extracting the relevant data from the representation.
Generally, when representing the visual information about a scene, one needs to store the five dimensional radiance function (3 degrees of freedom for position and 2 for direction).Since in "free space", radiance is constant along a line, the complete appearance of an object from outside its convex hull, or conversely, the appearance of scene viewed from within an empty convex region, is fully represented by the radiance along all directed lines intersection the convex region.The space of lines has only 4 degrees of freedom, which makes a lumigraph a 4D representation.There are many possible ways to parameterize lines in 3 space.The light-field/lumigraph systems use the lines' intersection with two planes to parameterize the lines.The motivation for this choice is the simplicity of the representation, and the speed at which the data needed to generate a novel image can be extracted.In particular, the new image can be constructed using texture mapping operations [7].
Typically, the light leaving some geometric point is a smooth function over directions, e.g.constant in the case of a diffuse surface.Therefore the lumigraph function will be correspondingly smooth over some associated lumigraph domain region.Thus we may wish to understand the association between the geometry of the scene and subsets of the 4D data.This paper explores the structure of the two-plane parameterization in detail.This knowledge is both interesting theoretically and is potentially important for a variety of applications, for example, lumigraph compression, deriving geometry from lumigraphs, and creating more accurate renderings from lumigraphs.We do not address the specific issues surrounding each application here, but rather discuss the more basic theoretical issues.
There are many questions we seek to answer.The simplest such question is to determine what "the set of all lines that pass through some fixed point in space" corresponds to in a lumigraph.This set of lines has two continuous degrees of freedom, and so the associated lumigraph subset is a two dimensional subset.Moreover we will show that this 2D subset is an affine manifold; one could call this a 2D plane in a 4D space.We are also interested in "converse" questions; suppose we choose some arbitrary 2D affine subset of a lumigraph, does this correspond to the set of lines through some fixed point?The answer to this question is necessarily negative.There are only three degrees of freedom in choosing a point in space, while there are six degrees of freedom in choosing a 2D affine subset of a 4D space. 1 Therefore there must be 2D affine subsets that do not correspond to the set of lines through a fixed point.In this paper, we will explore this question fully, and characterize all 2D affine lumigraph subsets.
In this paper, we will also address questions such as: what is the algebraic structure of the subset of the lumigraph that corresponds to all lines that pass through some triangle in space?Because a triangle is bounded by line segments, it becomes relevant to ask: what is the subset corresponding to all lines through some line segment in space?
The knowledge gained by answering the above questions may prove useful for a variety of practical applications.One such application is extracting geometric information from real world lumigraph data.Bolles et al. use a 3D data structure comprised of images taken as a camera follows a linear path [3].They then analyze the structure of the "epipolar plane image" (EPI) slices of the 3D data, in order to extract geometric information about the scene.In particular, a feature point in scene corresponds to a 1D affine subset of an EPI.Instead of solving the standard stereo correspondence problem, Bolles et al. simply search for lines in the EPI's.This allows them to robustly deduce the depth of scene features.Unfortunately, in order to use a 3D data structure, the camera must follow a linear path; in practice this is a very restrictive assumption.On the other hand, as described in Gortler et al. [7], even when one allows the camera to move freely, one can create a 4D lumigraph data structure using a process called rebinning.Thus by looking for certain features in the lumigraph data, one may be able to robustly deduce geometric information about the scene, much like can be done from an EPI.For example, by identifying certain 2D affine subsets of lumigraph data one can extract the 3D scene locations of the associated points in space.Pursuing such a program requires us to clearly understand the the relationship between scene features and lumigraph subsets.
Another important application is lumigraph compression since these data sets are large, but highly redundant.Levoy and Hanrahan discuss a low dimensional vector quantization method for compressing the 4D datasets that arise from a lumigraph setting, achieving 120:1 lossy compression rates [9].Their method uses no knowledge about the structure of the data.It seems plausible that much higher compression ratios may be achieved if structural knowledge is taken into account.By knowing what to expect in a lumigraph, we hope to gain insight as to how to most efficiently represent them.
2D Slice of 4D Space ray space in 2D In this paper, we will use two parallel planes to parameterize lines in 3 space, and will refer to this parameterization as 2PP.Without loss of generality, the "front plane" will lie at z = 0 with points of the form (s t 0).The "back plane" is z = ;1, with points of the form (u v ;1).A (non-parallel) line will intersect both planes exactly once, and is identified with the 4D parameters (s t u v).Words such as point, line, segment, and plane will describe sets in 3D geometric space.Words such as 1D, 2D and 3D affine subsets will be used to refer to "lines", "planes" and "hyperplanes" in the 2PP parameterization.

Flatland
In order to build up some intuition for the problem, we will start by reviewing a lumigraph in flatland.In this case all geometry resides in the (x z) plane.A line is parameterized by where it crosses two canonical parallel lines.Thus a line in (x z) space is parameterized as (s u) in the lumigraph (see Figure 1).Because lines and points are projective duals in 2 space, flatland is easy to understand.This kind of analysis is found in [3].We first discuss the set of all lines that pass through some point (x 0 z 0 ).This pencil corresponds to a 1D subset of the (s u) domain.Using a simple similar triangles argument, it is clear that as one moves a constant distance ds, in order for the line to still pass through (x 0 z 0 ), one must move some distance du which is linear in ds (see Figure 2).Thus, this set of lines is a one dimensional affine subset of the 2D (s u) domain.For example, if the scene geometry consists of a single light emitting point, the 2D lumigraph function will only be non zero over the support of a single 1D affine subset.

Lines Through a Point
The same analysis is true for all lines that pass through a single camera's pin-hole.When synthesizing an image with a pin hole camera, one measures the radiance along all lines that pass through the pin-hole.Thus, the lumigraph data needed to synthesize an image lies along a 1D affine subset in (s u) space.The 1D subset is entirely determined by the (x z) position of the camera pin hole.The orientation, and intrinsic parameters of the camera have no effect the choice of the subset.They just determine the projective mapping between the data on that 1D subset, and the camera's "film line".
The converse statement is equally as simple.Any affine 1D subset of the (s u) domain corresponds to the set of lines that pass through a single point (x 0 z 0 ) in space.

Lines Through a Segment
In a flatland environment, one may approximate all of the visible "surfaces" using line segments.Thus it is important to consider the set of all lines that pass through some line segment in the (x z) domain.Let us identify the line l 0 on which this segment lies; this line has some particular parameter value (s 0 u 0 ).Let us specify points on the segment with the single parameter ; each fixes some point (x 0 z 0 ) on l 0 .All lines through that point are associated with a single 1D affine subset in (s u).This 1D subset must include (s 0 u 0 ), because the line l 0 passes through all points on the segment.As a result, all lines passing through a segment, correspond in the lumigraph to a pencil of 1D subsets through (s 0 u 0 ).The entire pencil spans all of the 2D (s u) space.When approximating geometry, we are considering a segment with a boundary.The corresponding (s u) subset consists of a "wedge" of the complete 360 degree pencil through (s 0 u 0 ) (see Figure 3).

3D Geometry
In 3 dimensions, lines and points are not projective duals, and the association becomes more complicated.It is our goal to understand this case as well as the flatland case.

Lines Through a Point
The most basic question to ask is what 2PP subset corresponds to all lines through a point (p x p y p z ) in space?Points along the line passing through some point (s t 0) and the point (p x p y p z ) can be parameterized by w and expressed as: " s t Thus, the set of lines passing through the point describes a 2 dimensional affine subset of the 2PP.If one parameterizes the 2 dimensional subset with parameters (a b), then the corresponding

Lines Through a Segment
In general it is common to approximate the 3D geometry of a surface using a set of triangles.A triangle is defined by 3 segments.Thus our next question will be what is the 2PP subset corresponding to a segment in space.Clearly, we have added a new degree of freedom and the corresponding subset is a three dimensional subset.
Call the line of the segment l 0 , it is defined by a point (p x p y p z ) and a direction (d x d y d z ).l 0 corresponds to some (s 0 t 0 u 0 v 0 ) of the 2PP domain.Define one degree of freedom to move along the line l 0 which specifies some point (x 0 y 0 z 0 ).The lines through each point (x 0 y 0 z 0 ) on l 0 make up one 2D affine 2PP subset.Also note that the set of lines that passes through (x 0 y 0 z 0 ) includes l 0 itself.Thus, the 2D affine subset contains (s 0 t 0 u 0 v 0 ).In this sense the three dimensional subset corresponding to all lines through l 0 is a "pencil" of 2D affine subsets that all share the single point (s 0 t 0 u 0 v 0 ).If we consider a bounded line segment, then the associated subset is a wedge of the "pencil".
The geometry of this subset can be determined by fixing s, t, and .Points along If the line is not parallel to the z = 0 plane, we can specify l 0 with the direction (d x d y 1) and the point along the line that crosses the z = 0 plane, (p x p y 0).Thus the coordinates (s 0 t 0 u 0 v 0 ) of l 0 are (p x p y p x ;d x p y ;d y ).
This surface is bilinear in the variables (a b ).Because the parameter is not directly measurable from lumigraph data, it is best to eliminate this hidden parameter.When we do so, the subset is expressed as the solution to the following implicit bilinear equation Another natural representation is to eliminate and express the (s t u v) subset parametrically as subset(a b c) = ( a b c (b ; t 0 )(c ; u 0 ) + v 0 a ; s 0 ) If we hold s t constant, then v is linear in u.This is simply the image of the line as seen from a camera at s t.If we hold s u constant, then v is linear in t.This is exactly what is observed in a single EPI slice.See figure 4 for such a lumigraph decomposition of the fruitbowl lumigraph used in [7].If we hold t u constant, then v is linear rational in s.See figure 5 for such a lumigraph decomposition.

Lines Through a Triangle
The set of all lines through a triangle is a 4D manifold of the 2PP domain with a boundary.The boundary of this manifold is defined by the 3D subsets associated with the set of all lines that pass through the three line segments defining the triangle.Thus the subset of lines passing through a single triangle can be fully described by the equations above.

Occlusion
In an actual scene, a triangle may be obscured by other geometry.As such, not all of the light rays emanating from some triangle are represented in the lumigraph of the scene.If a triangle is partially obscured by another triangle, then only the rays that emanate from the first triangle and do not pass through the blocking triangle will be represented in the lumigraph.In this case, the set of lines seen from the first triangle will make up a more complicated 2PP subset.The three dimensional geometry of these "critical" sets of lines is well understood [6].The relationship between critical manifolds of various dimensions is also well understood [4].We wish to understand the algebraic nature of these critical sets when expressed in the 2PP parmeterization.

Apparent vertices: Lines Through 2 Lines
When a triangle is obscured, then the observed set of rays can terminate at apparent vertices.These are created when edges from two polygons appear to cross from the receiver's point of view [1].The set of rays that pass through apparent vertices can be expressed as "the set of lines that pass through 2 lines".Equation 3gives us the implicit equation for all lines that pass through one line.Thus the sets of lines passing through two lines is the solution to the two equations This is the intersection of the solution set of a quadratic equation, and a 3D affine subset.The result is a 2D quadric subset of the 2PP domain.
Thus in a partially obscured triangle, the corresponding 2PP subset is a 4D manifold bounded by a 3D piecewise quadric manifold (corresponding to the apperant edges).The 3D quadratic pieces meet at 2D affine manifolds, (corresponding to actual vertices), and at 2D quadric manifolds (corresponding to apparent vertices).

Lines Through Three Lines
In a scene with multiple triangles, there exist so-called "critical" locations; these are places where the number of apparent vertices changes.These locations make up the contours of the aspect graph of the scene [6,13,8].Such critical locations occur at "eee" events, when three edges of the scene appear to intersect from the point of view of the receiver.A degenerate "eee" case occurs when one polygon edge and one polygon vertex appear from the point of view of the receiver to coincide.Such cases are called "ve" events.At these "eee" and "ev" locations, the irradiance undergoes a discontinuity in its second derivatives [1,8].

Lines Through One Point and One Line
A "ve" event is a special case of an "eee" event.In this case, it can be shown that the quadratic elements of the equations can be eliminated leaving three linear constraints.Thus the corresponding subset is a 1D affine subset.
Thus for a partially obscured triangle, three of the 3D quadratic manifolds comprising its boundary can meet either in a 1D conic section, or degenerately in a 1D affine subset.

Converse Questions
In order to better understand the lumigraph structure, it is useful to ask converse questions such as: what is the set of lines corresponding to a 2D affine 2PP subset.This will help us geometrically interpret observed lumigraph structure.
We said above that the subset associated with all lines that pass through a point is a 2D affine subset.As mentioned in the introduction, a simple counting argument shows that there must be 2D affine subsets that do not correspond to lines through a single point.In particular it takes only 3 numbers to specify a point in space, whereas there are 6 degrees of freedom in choosing a 2D affine subset of 4D space [11].What are the other 2D affine subsets?In order to understand these questions we first must introduce the concept of a shallow segment.

Lines Through a Shallow Segment
We define a shallow line to be a line that lies in a plane of constant z.A shallow segment is a segment of a shallow line.As we will see, shallow lines are important in characterizing affine 2PP subsets.
What 2PP subset corresponds to all lines through a shallow line l defined by a point (0 p y p z ) and a direction (1 d y 0) in space? 2 This subset can be computed starting from equation 2, and setting d z to zero.Thus we conclude that for a shallow line, the corresponding 3D lumigraph subset is affine.

2D Affine Lumigraph Subsets
In general a 2D affine subset of a 4D space results from the intersection of two 3D affine subsets.As a result, the set of lines that pass through two shallow lines l 1 , l 2 , must be a 2D affine subset.There are 3 degrees of freedom in specifying each shallow line, giving us a total of 6 degrees of freedom.It can be shown that these 6 dofs are indendent and can thus specify any 2D affine lumigraph subset.
Thus we conclude that any 2D affine subset corresponds to the set of lines through two shallow lines.The set of lines passing through a single point, is simply a degenerate case where two shallow lines intersect in a point.

Relationship to Pl ücker Coordinates
One classic way to represent lines in 3D space is using 6 coordinates called Plücker coordinates.Plücker coordinates express elements of the space P (G 2 (R 4 )) from the Graßman-Cayley algebra [10,5,2,13,11,12].This 5D projective space consists of lines in 3D and "linear combinations of lines in 3D" up to an arbitrary scale.Given the 6 Plücker coordinates of an element of this space, one can test if it represents a single line (and not a non-decomposable linear combination of multiple lines) by seeing if the coordinates satisfy a certain quadratic constraint.This leaves 5 degrees of freedom.Because the scale is arbitrary, we are left with 4 independent degrees of freedom to describe a line.
For Lumigraph purposes, the 2PP parameterization has certain advantages over Plücker coordinates.The 2PP domain is a four dimensional linear space, while lines in Plücker coordinates live on a quadric four dimensional manifold in a five dimensional projective space.This can have a dramatic impact on the space complexity of the representation as well as the time required to compute the coordinates of the lines needed to extract an image.But because the algebra and geometry of Plücker coordinates is well understood, we wish to understand the relationship between 2PP coordinates and Plücker coordinates.
Given a line (s 0 t 0 u 0 v 0 ) in the two plane parameterization, one identifies two points on the line, (s 0 t 0 0) and (u 0 v 0 ;1) and computes the normalized Plücker coordinates as (s 0 v 0 ; t 0 u 0 ;s 0 s 0 ; u 0 ;t 0 1 v 0 ; t 0 ) The formula for the first Plücker coordinate is non-linear and so in general this mapping is non-linear.Interestingly, the Plücker subsets corresponding to many of the geometric features discussed in this paper still have the same algebraic complexity as they do in the two plane parameterization.For example, to compute the Plücker coordinates of the lines that pass through a single point (p x p y p z ) we start with equation 1 and map them to the Plücker coordinates: subset(a b) = ( ap y =p z ; bp x =p z ;a ;a=p z + p x =p z ;b 1 b = p z ; p y =p z ) The non-linear parts of the mapping cancel out.As a result we see that the set of lines passing through a single geometric point is a 2D affine subset of normalized Plücker coordinates.The same result is obtained in a coordinate free setting in [5,Prop 2.4].
The set of lines that pass through two lines is associated with a quadric in (s t u v), and the set of lines that pass through three lines is associated with a conic in (s t u v).
Once again, the same is true of the associated subsets of Plücker coordinates [13].

Conclusion
In this paper we have investigated the association between sets of lines through certain features in 3D geometric space, and subsets of the two plane parametric domain.This gives us insight into the structure of lumigraph functions, and may prove very useful in extracting geometric information from lumigraphs as well as lumigraph compression.We plan to pursue these avenues as future work.

Fig. 2 .
Fig. 2. Lines through a point are associated with a line

Fig. 3 .
Fig. 3. Lines through a segment are associated with a wedge subset(a b ) = ( a b a + a=p z ; =p z b + b=p z ; p y =p z ; d y =p z ) Once again, we eliminate resulting in subset(a b c) = ( a b c (1 + 1=p z )b ; d y ( 1 + 1 =p z )a + d y c ; p y =p z ) resulting line is are written as The line crosses the z = ;1 plane when w = ;1=(p z + d z ) at the point " s + s=(p z + d z ) ; p x =(p z + d z ) ; d x =(p z + d z ) t + t=(p z + d z ) ; p y =(p z + d z ) ; d y =(p z + d z )