Unsupervised joint alignment of complex images
http://vis-www.cs.umass.edu/code/congealingcomplex/

All code provided under a BSD-style license.  Terms of license can be
found at the top of each source file.


README contents:
--------------------------------
1. Overview
2. Quick Guide
3. Cars
4. Full Details
5. Additional Notes


1. Overview
--------------------------------

The source code is given for congealReal.cpp and funnelReal.cpp,
producing binaries congealReal and funnelReal.  Both are written in
C++ and require the OpenCV library (http://opencvlibrary.sourceforge.net).  

On a Linux machine, the following Makefile commands will produce the
binaries, following the conventions from
http://opencvlibrary.sourceforge.net/CompileOpenCVUsingLinux (refer to
the first URL for other environments).

IFLAGS = `pkg-config --cflags opencv` -O2
LFLAGS = `pkg-config --libs opencv`

all: congealReal funnelReal

congealReal: congealReal.cpp
	gcc $(IFLAGS) $(LFLAGS) -o congealReal congealReal.cpp

funnelReal: funnelReal.cpp
	gcc $(IFLAGS) $(LFLAGS) -o funnelReal funnelReal.cpp

Depending on the settings congealReal is run with, it may bring up
images in new windows.  Press any key to continue with the program.


2. Quick Guide
--------------------------------

congealReal images.list images.model

This command will read in a list of image filenames, one per line,
from images.list, perform congealing, and save the sequence of
distribution fields to the file images.model.  

funnelReal images.list images.model images_aligned.list

This command will read in a list of image filenames, one per line,
from images.list, align each image by funneling it according to the
sequence of distribution fields in images.model, then save the aligned
images using the filenames specified in images_aligned.list (which
should be in the same order as images.list).


3. Cars
--------------------------------

The set of car images used in the ICCV paper is provided to try with
the source code.

After uncompressing cars.tgz, one can run the following commands from
the cars/ directory.

congealReal carsFn.txt cars.train -o animSeq.txt -d display -v visualize -g carsOutFn.txt -outer 176 132 -inner 120 76 -nonrand -verbose

This will display the resulting images in new windows and save these
to the directory display, produce visualizations of the final
distribution field entropy and highest posterior probability cluster
representatives to visualize, generate aligned images and save them to
the directory final, use a 176x132 outer image and 120x76 inner image,
compute the entropy at all points within the inner window, and output
the entropy at each iteration.

congealReal carsFn.txt cars.train -a animSeq.txt animations -outer 176 132 -inner 120 76

This will produce frames for animation of the congealing done by the
previous command, saving the images grouped in 5x5 panels and entropy
of the distribution field, at each iteration of congealing, to the
directory animations.

funnelReal carsFn.txt cars.train carsOut.fn -outer 176 132 -inner 120 76 -o params.txt

This will align the car images using the funnel learned from the
congealing and save the aligned images to the directory final and the
transformation parameters used to align each image to params.txt.  

In the case of the cars, the funneling is duplicating the result of
congealing with the -g option, so this is just for illustration.
Funneling would be used in instances where it is not feasible to
congeal all images in a data set at once, or when congealing has been
done on an initial set of images and subsequently new images are found
that need to be aligned.


4. Full Details
--------------------------------

congealReal.cpp : congealing for complex, realistic images
                  using soft clusters of SIFT descriptors

usage : congealReal <list of image filenames> <model output file> ...
           [options]
        
        <list of image filenames> is a list of filenames of the images
           to process
        <model output file> is the filename to which the sequence of
           distribution fields should be written to (for use later in
           funneling)

        options :

           -o filename 
              output the transformations at each iteration to the
              specified file, in order to create an animation later

           -a filename directory
              create a frame (for animation) using the transformations
              given in the specified file, and write the result to the
              specified directory (must be used alone, and no
              congealing will be done)

           -v directory
              create visualizations of highest probability patches for
              each cluster and of entropy of final distribution field,
              writing images to the specified directory

           -g directory or list of filenames
              generate the final aligned images.  if the argument is a
              directory name, the images will be written to the
              specified directory using the original filenames (this
              assumes the original filenames were relative filenames,
              and appends them to the specified directory).
              otherwise, it is assumed the argument is the name of a
              file containing a list of filenames to use for the
              aligned images

           -d directory
              display the final transformations in 5x5 panels and
              write images to specified directory (press ESC to skip
              display of panels)

           -outer w h
              resize images to w by h for congealing computations
              (default 150x150)

           -inner w h
              use an inner window of size w by h, within which to
              calculate likelihood for congealing (must be smaller
              than outer dimensions by at least the size of the window
              for which SIFT descriptor is calculated over) (default
              100x100)

           -loc n
              sample n pixel locations at which to calculate
              likelihood for congealing (default 6,000)

           -nonrand
              use all points within inner window rather than sampling
              (will ignore -loc if provided)

           -clusters k
              use k clusters of SIFT descriptors (default 12)

           -verbose
              print out entropy for each iteration of congealing


funnelReal.cpp : funneling for complex, realistic images
                 using sequence of distribution fields learned from congealReal

usage : funnelReal <list of image filenames> ...
        <model file from congealing> ...
        <output directory or list of output filenames> [options]

        <list of image filenames> is a list of filenames of the images
           to process
        <model file from congealing> is the file containing the
           sequence of distribution fields from congealing
        <output directory or list of output filenames> if this is the
           name of a directory, the aligned images will be written to
           this directory (making the assumption to the filenames
           provided in the first argument are relative.  If it is not
           the name of the directory, then it should be the name of a
           file containing the filenames to use for the aligned
           images, in order corresponding to the first argument

        options :

           -o filename
              output the final parameter values used to generate
              aligned images

           -outer w h
              resize images to w by h for funneling computations
              (default 150x150).  this must match the values used in
              congealing

           -inner w h
              use an inner window of size w by h, within which to
              calculate likelihood for congealing (must be smaller
              than outer dimensions by at least the size of the window
              for which SIFT descriptor is calculated over) (default
              100x100).  this must match the values used in congealing


5. Additional Notes
--------------------------------

Currently, the SIFT descriptor computation remains as described in the
ICCV paper.  Important points are that it is computed over 8x8 patches
split into 4x4 subregions, yielding a 32 dimensional vector, and the
patches are not re-oriented to the dominent edge orientation of the
patch.

congealReal.cpp contains a constant variable maxIters = 100, and will
terminate if the number of iterations of congealing exceeds this
number.  In practice, congealing takes approximately 20 to 30
iterations.  Of course, this will vary depending on your particular
data, so you may wish to increase this number of necessary.

congealReal.cpp also contains a constant variable maxFrameIndex = 5.
This number determines how many sets of 5x5 images it will create when
making frames for animation.

funnelReal.cpp contains a constant variable maxProcessAtOnce = 600,
and will attempt to simultaneously funnel at most maxProcessAtOnce
images together.  This number should be set based on memory
constraints, though there should not be any significant slowdown if a
smaller number of images are funneled together in one round.