Fixation-based Segmentation code w/o fixation strategy
Version 1.2
- activeSegCode_ver_1_2.zip (For Unix/Linux) (Updated on June 18, 2012)
- activeSegCode_win_ver_1_2.zip (For windows) (Updated on June 18, 2012).
Version 1.1
- activeSegCode_ver_1_1.zip (For Unix/Linux)
- activeSegCode_win_ver_1_1.zip (For windows)
Version 1.0
- activeSegSourceCode.zip (For Unix/Linux) (Released on Jan 21, 2011).
- activeSegSourceCode_win.zip (For windows) (Released on Apr 03, 2011).
Changes in version 1.2
- Canny edge map is added to the probabilistic map to bridge gaps
- In the Windows version, a static library for 64-bit architecture compiled using MS Visual C++ 2010 Express has been included.
What's new in version 1.1
- Fixed the color gradient error -- pixel values out of range [0,1]
- Added the option to save the output (closed contours) into an xml file
- Added the function to save edge orientation information
- Added the option to select multiple fixations interactively
Saving the closed contours into an XML file. Why?
Since the segmentation strategy outputs a closed contour for each fixation point, the easiest way to store them is to save the interior of the closed contours in separate binary masks. But this means storing as many binary images as the number of fixation points, causing problems with managing all those files. Can we do better?
Yes. We can now store the closed contours together in an XML file, which can be read using the matlab function readCCFrmXML.m provided with the code. The matlab function reads the closed contours from the XML file and returns a cell array whose each element is a Nx2 matrix containing (x,y) coordinates of N points on a segmented closed contour.
Saving the edge orientation. Why?
Two reasons.
First, to solve the speed problem. Since the edge detection takes substantial amount of
time, let's precompute the edge gradient and orientation
and save them as gray scale images using member functions like saveEdgeGrad and saveEdgeOri in the
segLayer class. Later, instead of recomputing the edge map,
we can simply read both the edge map and orientation from the stored images. This is particularly
important for those who want to experiment with their fixation
strategies while not getting stuck with computing edge maps everytime they run the code.
Second, to be able to use a different gradient map in the segmentation process, if you intend to.
The code to calculate and save the edge gradient and orientation is
as follows:
int main(int argc, char* argv[]){ if (argc != 2){ fprintf(stderr, "usage: %s <image> ",argv[0]); exit(1); } class segLayer frame1; frame1.readImage(argv[1]); // Edge detection! frame1.edgeCGTG(); frame1.generatePbBoundary(); //save edge gradient frame1.saveEdgeGrad(argv[1]) //save edge orientation frame1.saveEdgeOri(argv[1]); return 0; }
The names of the save gradient and orientation maps are created by adding _gradient.pgm and _orientation.pgm to the original image name respectively. The gradient value is rescaled from [0,1] to [0,255] before storing it as a grayscale image. The orientation information is also scaled by 255/pi before being saved as a gray scale image.
To read these edge information directly from the stored files.
int main(int argc, char* argv[]){ class segLayer frame1; frame1.readImage(argv[1]); //Reading gradient image frame1.setEdgeGrad(argv[2]); //Reading orientation image frame1.setEdgeOri(argv[3]); //generate Pb Boundary frame1.generatePbBoundary(); ... ... return 0; }
EXAMPLES
Example 1: Segmenting objects in a single Image
int main(int argc, char* argv[]){ if (argc != 2){ fprintf(stderr, "usage: %s <image> ",argv[0]); exit(1); } class segLayer frame1; frame1.readImage(argv[1]); // Edge detection! frame1.edgeCGTG(); frame1.generatePbBoundary(); frame1.displayPbBoundary(-1); //select fixation point! int num = 2; frame1.selectFixPt_interactive(num); //segment frame1.allocateMemForContours(); frame1.segmentCurrFix(); //display! frame1.displayCurrSegs(-1); //release memory! frame1.deallocateMemForContours(); return 0; }
Explanation
Reads the image of the scene.
class segLayer frame1; frame1.readImage(argv[1]);
Finds all possible boundary locations as the pixels with positive color/texture gradient.
frame1.edgeCGTG();
Since there is no additional cue to differentiate between an internal and a boundary edge pixel, the gradient magnitude is used as the probabilistic estimate of each edge pixel to be at an object boundary. This might even suffice for the objects with smooth interior.
frame1.generatePbBoundary();
To select 2 fixation points inside the object of interest interactively. If you already have an attention module to select fixation points, you can use the member function such as assignFixPt(int, int) or readFixPts() to input those locations into the system.
int num = 2; frame1.selectFixPt_interactive(num);
Allocate the memory to store the closed contours computed by the algorithm. segmentCurrFix() segments for the most recent fixation point. To do the batch processing, use segmentAllFixs().
frame1.allocateMemForContours(); frame1.segmentCurrFix();
Displays the segmented contours. But you could use the other member functions to save your results.
frame1.displayCurrSegs(-1);
Example 2: Segmenting moving objects in a video.
int main(int argc, char* argv[]){ if (argc != 3){ fprintf(stderr, "usage: %s <image_1> <flowMap/disparity> ",argv[0]); exit(1); } class segLayer frame1; frame1.readImage(argv[1]); // Edge detection! frame1.edgeCGTG(); //Biasing with optic flow! frame1.readFlow_flo(argv[2]); frame1.generatePbBoundary(); frame1.displayPbBoundary(-1); ..... .....
From the single image case above, the only change is the introduction of optical flow map. In this case, the possible boundary locations are still found using color and texture cues. However, the probabilistic estimate for those locations to be at an object boundary is computed using optical flow gradient instead of color or texture gradient.