UWA logo

 
School of Computer Science
& Software Engineering
Computer Vision CITS4240

Laboratory 7: Camera Calibration

Camera calibration involves finding the relationship between 3D world coordinates and their 2D projected positions in the image.

Two images, stereo1.jpg and stereo2.jpg have been placed in the Images directory. These images are two views of a calibration target. The XYZ coordinates of the target points are available in calibpts.m. These points are defined with respect to a coordinate frame having its origin at the base of the calibration target as shown below. This image also shows the numbering of the points.

Write a Matlab function to the following specification 

% CALIBRATE
%
% Function to perform camera calibration
%
% Usage:   C = calibrate(im, XYZ, uv)
%
%   Where:   im  - is the image of the calibration target.
%            XYZ - is a N x 3 array of  XYZ coordinates
%                  of the calibration target points. 
%            uv  - is a N x 2 array of the image coordinates
%                  of the calibration target points.
%            C   - is the 3 x 4 camera calibration matrix.
%  The variable N should be an integer greater than or equal to 6.
%
%  This function plots the uv coordinates onto the image of
%  the calibration target.  It also projects the XYZ coordinates
%  back into image coordinates using the  calibration matrix
%  and plots these points too as a visual check on the accuracy of
%  the calibration process.  
%  Lines from the origin to the vanishing points in the X, Y and 
%  Z directions are overlaid on the image.
%  The mean squared error between the  positions of the uv coordinates 
%  and the projected XYZ coordinates is also reported.
%
%  The function should also report the error in satisfying the 
%  camera calibration matrix constraints.

function C = calibrate(im, XYZ, uv)
The error that your function needs to report is described further down the page.

The XYZ coordinates can be obtained by simply running the function calibpts.m. The uv coordinates can be obtained using the MATLAB function ginput. If one invokes ginput as follows: 

>> uv = ginput(12)   % digitise 12 points in the image
and digitises a series of points by clicking with the left mouse button, then uv will be a matrix containing the column and row coordinates of the points that you digitised. You can also use MATLAB's impixel function; however, you might find the cursor used by this function makes it very difficult to digitise things accurately.

After the above operation, the variable uv should be a 12 × 2 matrix, each row of which should contain the coordinates of one image point.

Refer to the lecture on camera calibration to set up the matrix equation that needs to be solved. You will end up setting up an equation of the form: 

         A q  =  b
where:
A is a 2N × 11 matrix of constraint coefficients (in this case, N=12).
q is an 11 element column vector of calibration matrix coefficients to be solved, and
b is a 2N column vector of the image coordinates of the target points. This is the vector that is composed of the coordinates stored in the uv matrix.

This over-constrained set of equations can be solved (in a least squares sense) in Matlab using the expression 

         q = A \ b;
In this case where the matrix equation is over-constrained Matlab will perform a QR decomposition on the matrix A in order to solve the least squares solution. This technique overcomes many of the numerical problems that arise if one attempts to solve least squares problems via classical equations. (do a help on mldivide).

Hint
In writing your code you will want to 'reshape' a 2D vector into a 1D vector. You can use the Matlab function reshape to reshape a matrix to arbitrary dimensions. You can also use the colon operator as follows: 

>> a = [1 2
        3 4
        5 6]
 
>> b = a(:)
 
b =
 
     1
     3
     5
     2
     4
     6
Note however that Matlab extracts data from the source matrix by column (its Fortran heritage). To extract data from a matrix by row simply transpose the matrix first. 
>> b = a'
 
b =
 
     1     3     5
     2     4     6
 
>> b = b(:)
 
b =
 
     1
     2
     3
     4
     5
     6

The main measure of the error in satisfying the camera matrix constraints is the normalised value of (q1 × q3).(q2 × q3).

With the arbitrary scale of the camera calibration matrix, the quantity (q1 × q3).(q2 × q3) represents the scaled cosine of the angle between the vectors (q1 × q3) and (q2 × q3), which should be perpendicular. To get a value that represents a pure (unscaled) cosine it should be divided by the magnitude of ||q1 × q3|| and the magnitude of ||q2 × q3|| in order to get the dot product between two unit vectors, and hence the cosine. This is the error in satisfying the camera calibration matrix constraints that your function should report.

Note that you will need to use these calibration matrices for next weeks lab exercise. You can save your data using Matlab's save command. For example the command 

     >> save mydata im1 im2 calibPts uv1 uv2 C1 C2  % (do not use commas between variable names)
will save your variables im1 im2, calibPts, uv1, uv2, C1, and C2 in a file called mydata.mat. At a later date you can reload this data with the command: 
     >> load mydata
The variables im1 im2, calibPts, uv1, uv2, C1, and C2 will then be restored in your workspace.

Portfolio Requirements

You should include the following in your portfolio:

Your Matlab functions/scripts will be run and tested. So, do not forget to include all other Matlab functions (excluding the Matlab library functions but including Dr Peter Kovesi's) that are needed by your Matlab functions/scripts.

You might find it more tidy to save your MATLAB code and input/output test image files into subdirectories away from your HTML file. Ensure that all the links in your HTML file are correct. You should also provide some explanation in your HTML file to demonstrate your understanding about the problem and the work that you carried out.

Return to Computer Vision 412