In general, 3D modeling of scenes may allow handling of cases that per-frame projective transformations do not allow, such as registering frames having different viewpoints over a non-flat or specular scene, or recovery of depth from frames having different viewpoints. ALE's implementation of 3D scene reconstruction is considered experimental in versions prior to 0.8.3.
The method of modeling used in 0.8.2 and later is spatial subdivision based on regions of interest, followed by iterative update of the estimated occupancy and color of each region of interest. (For a general discussion of occupancy, see Hans Moravec's work on depth recovery.) In 0.8.1, the method of modeling is decomposition and deformation of a surface. In earlier versions, a per-frame, per-pixel model of depth is used.
The color and depth output options provide a reconstruction of the scene as
viewed from a particular frame's vantage point, or from an arbitrary camera
view [0.8.2 and later]. In 0.8.2 and later, normalization of output values by
accumulated occupancy can optionally be disabled for unfiltered views. Version
0.8.3 adds support for options --et and --3dpx, which can be useful for 3D
cropping applications. --et specifies the minimum accumulated weight required
for a pixel to be defined, allowing refinements of crops based on clip planes
(--rc and --fc) established prior to scene
reconstruction, whereas --3dpx specifies a region of space to be removed
after scene reconstruction. 0.8.3 uses depth image pixels to determine such spatial regions,
leaving the pixels undefined, whereas 0.8.4 removes scene geometry prior to
rendering, allowing rendering of background objects after the foreground has been removed.
--3dv <n> <o> Render, to file <o>, colors as viewed from frame <n>.
--3dd <n> <o> Render, to file <o>, depths as viewed from frame <n>.
--3dvp <args> <o> Render, to file <o>, colors viewed with projective <args>. [0.8.2 and later]
--3ddp <args> <o> Render, to file <o>, depths viewed with projective <args>. [0.8.2 and later]
<args> are W H V x y z P Y R:
W image width.
H image height.
V camera view angle.
x translation x component.
y translation y component.
z translation z component.
P rotation around x-axis.
Y rotation around y-axis.
R rotation around z-axis.
--occ-norm Normalize output with accumulated occupancy. [default] [0.8.2 and later]
--occ-nonorm Don't normalize output with accumulated occupancy. [0.8.2 and later]
--et <x> Set encounter threshold <x> for defined pixels. [default is 0] [0.8.3 and later]
--3dpx <args> Exclude a specified spatial volume following full-scene [0.8.3]
reconstruction, assuming a single depth at each point
in the output. <args> are:
<xmin> <xmax> <ymin> <ymax> <zmin> <zmax>
--3dpx <args> Exclude a specified spatial volume following full-scene [0.8.4 and later]
reconstruction. <args> are:
<xmin> <xmax> <ymin> <ymax> <zmin> <zmax>
Version 0.8.4 adds support for filtering color values from the inputs to produce final color renderings. This is now the default. Determining surface depth and gradient for filtering the inputs draws from a set of depth pixels within a disc whose radius can be specified with the --3d-dmr and --3d-fmr options.
Filtering color from views distributed on a disc (about the camera origin) of specified aperture allows simulation of focus effects; such effects can be controlled with the --focus option.
--3d-filter Use filtering for 3D color output. [default]
--3d-nofilter Don't use filtering for 3D color output.
--3d-dmr <x> Set radius for filtering median depth to <x> [default 0]
--3d-fmr <x> Set radius for filtering median diff to <x> [default 0]
--focus <ft> <op> Create focus region with type <ft> and options <op>:
Focus type:
d <d> focus at distance <d>
p <x> <y> focus at point (x, y)
Space-separated options may include zero or more of:
ci=<ci> camera index [default 0]
fr=<fr> focal range [default 0]
ht=<ht> horizontal tilt gradient [default 0]
vt=<vt> vertical tilt gradient [default 0]
ap=<ap> aperture diameter [default 3]
sc=<sc> sample count [default 3]
sx=<sx> start x coordinate [default -Inf]
ex=<ex> end x coordinate [default Inf]
sy=<sy> start y coordinate [default -Inf]
ey=<ey> end y coordinate [default Inf]
sd=<sd> start depth [default 0]
ed=<ed> end depth [default Inf]
sr=<sr> view sample randomization:
aperture per aperture [default]
pixel per pixel
fs=<fs> focal statistic, one of:
mean mean of views [default]
median median of views
In versions 0.8.1 and later, camera parameters can be adjusted according to control points to decrease the error between points in different frames. Perturbation bounds for adjustment can be specified, including an additional upper bound on view angle perturbation, and a stereo threshold can be set for discarding control points after adjustment is complete. An initial camera view angle can also be specified. Versions 0.8.2 and later allow calculation of error by median rather than mean, and also allow disabling of adjustment of orientation and/or position.
--view-angle <x> Set the initial diagonal view angle to <x> degrees.
(Default is 43.7 degrees or device-specific.)
--cpp-upper=<x> Set upper bound <x> for camera parameter perturbation, [0.8.1 and later]
in pixels or degrees [default is 32]
--cpp-lower=<x> Set lower bound <x> for camera parameter perturbation, [0.8.1 and later]
in pixels or degrees [default is 0.125]
--va-upper=<x> View-angle perturbation upper bound in degrees [default is 32] [0.8.1 and later]
--st <x> Control point stereo threshold is <x> pixels. [default is 4] [0.8.1 and later]
--cpp-err-mean Use RMS error to determine camera parameters. [default] [0.8.2 and later]
--cpp-err-median Use median error to determine camera parameters. [0.8.2 and later]
--vp-adjust Adjust the view point [default] [0.8.2 and later]
--vp-noadjust Do not adjust the view point [0.8.2 and later]
--vo-adjust Adjust the view orientation [default] [0.8.2 and later]
--vo-noadjust Do not adjust the view orientation [0.8.2 and later]
Transformation data file options allow reading and writing of camera position, orientation, and view angle information.
--3d-trans-load=x Load initial transformation settings from file x [0.8.2 and later] --3d-trans-save=x Save final transformation data in file x [0.8.2 and later]
The transformation data file uses the following format, starting with a version command, and then describing view angle and euclidean transformation of each frame in turn:
W 0 V <view angle> E <width> <height> <X> <Y> <Z> <P> <Y> <R> V <view angle> E <width> <height> <X> <Y> <Z> <P> <Y> <R> . . . V <view angle> E <width> <height> <X> <Y> <Z> <P> <Y> <R>
After the cameras are adjusted, a 2D surface is decomposed and deformed to reduce local cost measures, calculated from angles between neighboring triangles, edge lengths, and differences in color between different views. Multipliers for edge length and angle costs can be specified on the command line.
--ecm <x> Set edge length cost multiplier <x>. [default is 0.001] [0.8.1 only] --acm <x> Set angle cost multiplier <x>. [default is 0.001] [0.8.1 only]
In versions 0.8.2 and later, clip planes can be specified, and a color falloff
exponent for pairwise occupancy estimation can be specified. Larger exponents
are more forgiving of color differences between pixels in different frames.
In versions 0.8.3 and later, error contributions from cameras outside of the
generating pair can be assigned non-zero multipliers (--tcem), the
number of occupancy and color update iterations can be specified
(--oui), the number of points in space generated for each pixel
pair can be specified (--pa), and the type of frame pairings can
be specified (--pc).
--di-upper <x> Decimate primary input resolution by at most 2^x [default is 0] [0.8.4 and later]
--di-lower <x> Decimate input resolutions by at least 2^x [default is 0] [0.8.4 and later]
--do-try <x> Decimate output resolution by 2^x if possible [default is 0] [0.8.4 and later]
--oc Clip scene to output regions. [0.8.4 and later]
--no-oc Do not clip scene to output regions. [default] [0.8.4 and later]
--fc <x> Set front-clip to <x> (0 < x < 1) [default is 0] [0.8.2 and later]
--rc <x> Set rear-clip to <x> (1 < x < inf) [default is inf] [0.8.2 and later]
--fx <x> Set falloff exponent to <x> [default is 0] [0.8.2 and later]
--tcem <x> Set third-camera error multiplier to <x> [default is 0] [0.8.3 and later]
--oui <x> Set occupancy update iterations to <x> [default is 10] [0.8.3 and later]
--pa <x> Set pairwise ambiguity to <x> [default is 3] [0.8.3 and later]
--pc <type> Set the type of pairwise comparisons: [0.8.3 and later]
auto Determine comparisons automatically. [default]
all Perform all comparisons.