|
|
|
|
|
The main task in stereo reconstruction is the matching of points in one image to the corresponding points in the other image. Normally this is done via correlation using epipolar geometry and other matching constraints to speed the process and maximise the reliability of matching.
For simplicity in this lab exercise we will do the matching of points manually.
Write a MATLAB function to the following specification:
% STEREO % % Usage: pt3D = stereo(im1, im2, C1, C2) % % Where: im1 and im2 are the two stereo images % C1 and C2 are the calibration matrices % for these two images respectively % % The function will prompt you to digitise some points in the first image % (finishing by clicking the right button). The function then % prompts you to digitise the equivalent points (which you must digitise % in exactly the same sequence) in the second image. % The function then solves the stereo equations % and returns the 3D coordinates of the points in pt3D. function pt3D = stereo(im1, im2, C1, C2)
You can use the disp or fprintf functions to print out messages on the screen and the digipts function to do the digitising. For example:
>> disp('Digitise some points in figure 1');
>> figure(1), imshow(im1);
>> [u1,v1] = digipts;
Note that digipts behaves similarly to MATLAB's
impixel but uses the cross-hair cursor from ginput
which I find easier to use.
If you have an N × 3 array of 3D points
p3D = [. . .
. . .
. . .
. . .];
and you want to draw a line that, say, passes through points 2, 3, 5,
3, and 1 from the p3D array you can do the following:
% Extract the points you want pts = [p3D(2,:); p3D(3,:); p3D(5,:); p3D(3,:); p3D(1,:)]; % Alternatively (note that it is optional to separate the numbers % inside the square brackets by commas), pts = p3D([2 3 5 3 1],:); % Extract X, Y and Z components and pass to the Matlab's line function: line(pts(:,1), pts(:,2), pts(:,3)); % To draw a closed polygon through the points 2, 3, 4, 1: indices = [2 3 4 1 2]; % repeat the 1st point to close the polygon line(p3D(indices,1), p3D(indices,2), p3D(indices,3));
When plotting your 3D reconstruction of the stereo view of the scene, use the following
>> axis equal % Keeps the aspect ratio of the objects regular.
>> box on % Overlays a box which helps in the 3D visualisation.
>> rotate3D on % Drag the mouse in the image to rotate the view.
>> grid on % You may find this useful too.
An example of the 3D reconstruction of a similar stereo pair
is given below:
Write a MATLAB function to the following specification:
% HOMOGRAPHY % % Usage: H = homography(pts1, pts2) % % Where: pts1 and pts2 are both N x 2 matrices % containing the set of points related by % a plane-to-plane homography transformation. % H should be the 3 x 3 matrix that describes % the homography. function H = homography(pts1, pts2)The minimum number of pairs of points required for homography is 4. Your Matlab code should check whether this condition is satisfied and output an appropriate error message accordingly.
Given in painting.jpg is an image of a painting. The picture inside the wooden frame is 63cm (width) × 41cm (height). Your task for this part of the lab sheet is to select four corner points (for instance, using the Matlab function ginput) just inside the wooden frame and rectify the picture bounded by the corners to get a perfect frontal view of it. The output image of the picture should be 410 rows × 630 columns in dimension. Thus, the four corner points that you collect from ginput should be mapped by a homography H to the following (x,y)-coordinates: [1,1] (top-left), [630,1] (top-right), [640,410] (bottom-right), and [1,410] (bottom-left). Do not forget to specify axis equal to ensure the aspect ratio of the picture is retained. Your output image should contain the rectified picture bounded by the 4 corners and nothing else.
You would probably notice that edges of the picture frame are not straight. This is due to the distortion of the camera lens. This distortion can be corrected and a pre-processing step on the image is often applied before rectification. In this lab sheet, we temporarily ignore lens distortion. So when you run ginput, try to position the mouse a couple of pixels away from the edge of the picture frame so that they would fall outside your final output image. Note that when a point (x,y,1)T is mapped to a new point (x',y',1)T by H, both x' and y' are usually not whole numbers. To get the colour of a pixel at non-integer pixel coordinates, you will need to adopt a bilinear or bicubic interpolation on the four surrounding pixels. You will find the Matlab function maketform and imtransform useful for this exercise. An example about using these two functions can be found in the lecture note.
Your Matlab functions/scripts will be run and tested. So, do not forget to include all other Matlab functions (excluding the Matlab library functions but including Dr Peter Kovesi's) that are needed by your Matlab functions/scripts.
You might find it more tidy to save your MATLAB code and input/output test image files into subdirectories away from your HTML file. Ensure that all the links in your HTML file are correct. You should also provide some explanation in your HTML file to demonstrate your understanding about the problem and the work that you carried out.
