1. What is Camera Calibration?
Camera calibration estimates the parameters that connect a camera's image coordinates to the geometry of the observed scene. This section explains why calibration is required, which parameters are estimated, and how the results are used in mono and stereo NRV DVS systems.
A camera converts light from a three-dimensional scene into two-dimensional image coordinates. The conversion is not perfectly ideal: the lens bends incoming light, the sensor has a specific size and pixel layout, and the optical center may not align exactly with the center of the image. Calibration identifies these camera-specific characteristics so that a coordinate in an image can be interpreted with greater geometric accuracy.
Calibration is therefore a foundation for measurement, not simply an image enhancement step. Once the camera model is known, software can correct predictable geometric errors and relate observations from one camera—or several cameras—to a consistent coordinate system.
1.1 Basic Concept of Camera Calibration
Every camera has a unique combination of lens and sensor characteristics. Even two units of the same model can have small differences caused by manufacturing tolerances, lens mounting, or sensor alignment. As a result, the captured image may not be a perfectly scaled representation of the real scene.
Camera calibration estimates a set of parameters that describes how that individual camera forms an image. A known geometric pattern is typically observed from multiple positions so that the expected pattern geometry can be compared with its measured image coordinates. The estimated parameters then provide a reliable mapping between ideal image geometry and the coordinates reported by the camera.
1.2 Why Camera Calibration is Needed
Lens distortion is one of the most visible reasons for calibration. Straight structures can appear curved, especially near the image boundary, and an object can appear slightly different depending on where it is located in the field of view. These effects introduce systematic position errors rather than random noise.
This position-dependent distortion also changes the apparent shape of an object. The same object may look close to its normal shape near the image center but stretched, compressed, or curved near the boundary. As a result, an object detector may respond differently to the same object at different image locations, reducing detection consistency and accuracy.
Without correction, those errors propagate into later processing. The center of a detected object may be displaced, corresponding points in a stereo pair may not align, and a depth estimate may become less accurate. Calibration is therefore required whenever a system depends on precise object detection, position estimation, stereo matching, dimensional measurement, or depth calculation.
Corresponding points must lie on the same epipolar line
Placing two cameras side by side does not guarantee perfect image alignment. Differences in the two cameras' intrinsic characteristics and small mounting errors in height, pitch, or roll can cause the same scene point to appear at different vertical positions in the left and right images.
Stereo calibration estimates the relationship between the two cameras. Epipolar rectification then removes this misalignment so that corresponding points lie on the same horizontal epipolar line. Stereo matching can therefore search primarily along the horizontal direction, which improves disparity and depth estimation.
1.3 Parameters Estimated in Mono and Stereo Calibration
Calibration does not produce only a corrected image. It first estimates numerical parameters that describe the camera system, and those parameters are saved for later correction. The required output set depends on whether the system uses one camera or a stereo pair.
Estimates how a single camera converts incoming light into image or event coordinates.
Camera matrix K
Focal lengths fx, fy and principal point cx, cy.
Distortion vector D
Radial coefficients k1, k2, k3… and tangential coefficients p1, p2.
Undistorting one image or correcting individual pixel and event coordinates.
Left and right camera models
KL, DL and KR, DR.
Relative pose R, T
Rotation and translation from one camera coordinate system to the other.
Baseline ‖T‖
Physical distance between the two camera centers, in the same unit used for the calibration pattern.
Derived stereo geometry
Essential matrix E, fundamental matrix F, and rectification transforms.
Epipolar alignment, stereo matching, disparity, and depth estimation.
1.4 Applying Calibration Parameters: Undistortion and Rectification
Calibration and rectification are closely connected, but they are not the same step. Calibration estimates the camera parameters from calibration data. After those parameters are available, they can be applied to actual image, measurement, or event coordinates to produce corrected coordinates.
Calibration calculates the parameters; rectification applies them.
- Calibration
- Estimates the camera matrix, distortion coefficients, and, for stereo systems, the relative pose between cameras. This is usually performed offline or whenever the lens, focus, sensor, or camera mounting changes. Its direct output is a set of parameters.
- Undistortion / rectification
- Uses the estimated parameters on real input data. For a mono camera, this mainly removes lens distortion. For a stereo pair, epipolar rectification also remaps the left and right views so corresponding points lie on the same horizontal epipolar line.
In an online perception system, calibration itself does not have to run for every frame. The correction or rectification mapping is the part applied repeatedly to incoming data and may need to run in real time.
During image undistortion, pixels are resampled so that structures expected to be straight are represented more consistently across the field of view. The corrected or rectified result is a geometrically more useful view for analysis. Depending on the selected output view, correction may crop a small boundary region or create empty areas where no source pixel exists.
Calibration does not recover detail that the sensor never captured. It provides the known geometric mapping, and undistortion or rectification applies that mapping so that later algorithms operate on more accurate coordinates.
1.5 Calibration for Mono and Stereo NRV DVS Systems
NRV DVS units are used in either mono or stereo configurations. Here, NRV DVS identifies the sensing device, while “mono” and “stereo” describe the camera configuration. NRV DVS calibration is therefore not a third option separate from mono and stereo. Mono calibration is performed with one NRV DVS, while stereo calibration is performed with two NRV DVS units.
A mono NRV DVS system estimates the camera model needed to correct the coordinates produced by a single lens and sensor. A stereo NRV DVS system estimates the camera model for the left and right NRV DVS units separately, then additionally estimates their relative pose and baseline. Those stereo parameters are used for epipolar rectification so that corresponding structures from the two cameras are aligned on the same horizontal epipolar line before stereo matching and depth calculation.