Fixation-based Segmentation code w/o fixation strategy

Version 1.2

Version 1.1

Version 1.0

Changes in version 1.2

What's new in version 1.1

Saving the closed contours into an XML file. Why?

Since the segmentation strategy outputs a closed contour for each fixation point, the easiest way to store them is to save the interior of the closed contours in separate binary masks. But this means storing as many binary images as the number of fixation points, causing problems with managing all those files. Can we do better?

Yes. We can now store the closed contours together in an XML file, which can be read using the matlab function readCCFrmXML.m provided with the code. The matlab function reads the closed contours from the XML file and returns a cell array whose each element is a Nx2 matrix containing (x,y) coordinates of N points on a segmented closed contour.

Saving the edge orientation. Why?

Two reasons.
First, to solve the speed problem. Since the edge detection takes substantial amount of time, let's precompute the edge gradient and orientation and save them as gray scale images using member functions like saveEdgeGrad and saveEdgeOri in the segLayer class. Later, instead of recomputing the edge map, we can simply read both the edge map and orientation from the stored images. This is particularly important for those who want to experiment with their fixation strategies while not getting stuck with computing edge maps everytime they run the code.
Second, to be able to use a different gradient map in the segmentation process, if you intend to.
The code to calculate and save the edge gradient and orientation is as follows:

int main(int argc, char* argv[]){
	if (argc != 2){
		fprintf(stderr, "usage: %s <image> ",argv[0]);
		exit(1);
	}

	class segLayer frame1; 
  	frame1.readImage(argv[1]);

	// Edge detection!
  	frame1.edgeCGTG();
	frame1.generatePbBoundary();

	//save edge gradient
	frame1.saveEdgeGrad(argv[1])
	
	//save edge orientation
	frame1.saveEdgeOri(argv[1]);
	
	return 0;
	}

The names of the save gradient and orientation maps are created by adding _gradient.pgm and _orientation.pgm to the original image name respectively. The gradient value is rescaled from [0,1] to [0,255] before storing it as a grayscale image. The orientation information is also scaled by 255/pi before being saved as a gray scale image.

To read these edge information directly from the stored files.

int main(int argc, char* argv[]){

	class segLayer frame1; 
  	frame1.readImage(argv[1]);

	//Reading gradient image
  	frame1.setEdgeGrad(argv[2]);
	
	//Reading orientation image
  	frame1.setEdgeOri(argv[3]);
	
	//generate Pb Boundary
	frame1.generatePbBoundary();

	...
	...	
	return 0;
	}

EXAMPLES

Example 1: Segmenting objects in a single Image 


int main(int argc, char* argv[]){
	if (argc != 2){
		fprintf(stderr, "usage: %s <image> ",argv[0]);
		exit(1);
	}

	class segLayer frame1; 
  	frame1.readImage(argv[1]);

	// Edge detection!
  	frame1.edgeCGTG();
	frame1.generatePbBoundary();
	frame1.displayPbBoundary(-1);

	//select fixation point!
	int num = 2;
	frame1.selectFixPt_interactive(num);

	//segment
	frame1.allocateMemForContours();
	frame1.segmentCurrFix();

	//display!
	frame1.displayCurrSegs(-1);

	//release memory!
	frame1.deallocateMemForContours();
  	return 0;
	
}

Explanation

Reads the image of the scene. 

class segLayer frame1; 
frame1.readImage(argv[1]);

Finds all possible boundary locations as the pixels with positive color/texture gradient. 

 frame1.edgeCGTG();

Since there is no additional cue to differentiate between an internal and a boundary edge pixel, the gradient magnitude is used as the probabilistic estimate of each edge pixel to be at an object boundary. This might even suffice for the objects with smooth interior. 

frame1.generatePbBoundary();

To select 2 fixation points inside the object of interest interactively. If you already have an attention module to select fixation points, you can use the member function such as assignFixPt(int, int) or readFixPts() to input those locations into the system. 

int num = 2;
frame1.selectFixPt_interactive(num);

Allocate the memory to store the closed contours computed by the algorithm. segmentCurrFix() segments for the most recent fixation point. To do the batch processing, use segmentAllFixs(). 

frame1.allocateMemForContours();
frame1.segmentCurrFix();

Displays the segmented contours. But you could use the other member functions to save your results. 

frame1.displayCurrSegs(-1);

Example 2: Segmenting moving objects in a video.


int main(int argc, char* argv[]){
	if (argc != 3){
		fprintf(stderr, "usage: %s <image_1> <flowMap/disparity> ",argv[0]);
		exit(1);
	}
	class segLayer frame1; 
  	frame1.readImage(argv[1]);

	// Edge detection!
  	frame1.edgeCGTG();

	//Biasing with optic flow!
	frame1.readFlow_flo(argv[2]);
	frame1.generatePbBoundary();
	frame1.displayPbBoundary(-1);
	.....
	.....

From the single image case above, the only change is the introduction of optical flow map. In this case, the possible boundary locations are still found using color and texture cues. However, the probabilistic estimate for those locations to be at an object boundary is computed using optical flow gradient instead of color or texture gradient.