Vision is Hard

Vision is Hard

• Vision is Hard

• Camera Model, Lens, Problems and Corrections

• OpenCV

• OpenCV Tour

What is it?

• What is it?

• Turning sensor readings into perception.

• Why is it hard?

• It’s just numbers.

Distortions are corrected mathematically

• Distortions are corrected mathematically

• We use a calibration pattern
• We find where the points ended up
• We know where the points hould be

• OpenCV 2.2 Function:

• double calibrateCamera(
• const vector>& objectPoints,
• const vector>& imagePoints,
• Size imageSize,
• Mat& cameraMatrix,
• Mat& distCoeffs,
• vector& rvecs,
• vector& tvecs,
• int flags=0);

Vision is Hard

• Vision is Hard

• Camera Model, Lens, Problems and Corrections

• OpenCV

• OpenCV Tour

Google Maps, Google street view, Google Earth, Books

• Google Maps, Google street view, Google Earth, Books

• Academic and Industry Research

• Safety monitoring (Dam sites, mines, swimming pools)

• Security systems

• Image retrieval

• Video search

• Structure from motion in movies

• Machine vision factory production inspection systems

• Robotics

Calib3d

• Calib3d

• Calibration, stereo, homography, rectify, projection, solvePNP

• Contrib

• Octree, self-similar feature, sparse L-M, bundle adj, chamfer match

• Core

• Data structures, access, matrix ops, basic image operations

• features2D

• Feature detectors, descriptors and matchers in one architecture

• Flann (Fast library for approximate nearest neighbors)

• Gpu – CUDA speedups

• Highgui

• Gui to read, write, draw, print and interact with images

• Imgproc – image processing functions

• Ml – statistical machine learning, boosting, clustering

• Objdetect – PASCAL VOC latent SVM and data reading

• Traincascade – boosted rejection cascade

Works on:

• Works on:

• Linux, Windows, Mac OS (+ Android since open CV 2.2)

• Languages:

• C++, Python, C

• Online documentation:

• Online reference manuals: C++, C and Python.

Vision is Hard

• Vision is Hard

• Camera Model, Lens, Problems and Corrections

• OpenCV

• OpenCV Tour

Sobel has been the traditional 3x3 gradient finder.

• Sobel has been the traditional 3x3 gradient finder.

• Use the 3x3 Scharr operator instead since it is just as fast but has more accurate response on diagonals.

Graph Cut based segmentation

• Graph Cut based segmentation

Detectors available

• Detectors available

Maps one plane to another

• Maps one plane to another

• In our case: A plane in the world to the camera plane
• Great notes on this: Robert Collins CSE486
• http://www.cse.psu.edu/~rcollins/CSE486/lecture16.pdf
• Derivation details: Learning OpenCV 384-387

We often use the chessboard detector to find 4 non-colinear points

• We often use the chessboard detector to find 4 non-colinear points

• (X,Y * 4 = 8 constraints)
• To solve for the 8 homography parmeters.

• Code: Once again, OpenCV makes this easy

• findHomography(…) or:
• getPerspectiveTransform(…)

Involved topic, here we will just skim the basic geometry.

• Involved topic, here we will just skim the basic geometry.

• Imagine two perfectly aligned image planes:

In aligned stereo, depth is from similar triangles:

• In aligned stereo, depth is from similar triangles:

• Problem: Cameras are almost impossible to align

• Solution: Mathematically align them:

Algorithm steps are shown at right:

• Algorithm steps are shown at right:

• Goal:

• Each row of the image contains the same world points
• “Epipolar constraint”

Brief_match_test

• Brief_match_test

• Use of fast det., brief descrp. ORB will replace. See video_homography.cpp

• Calibration (single camera)

• Chamfer (2D edge matching)

• Connected_components

• Using contours to clean up regions in images.

• Contours2 (finding and drawing)

• Convexhull (finding in 2D)

• Cout_mat – (print out Mat)

• Demhist using calcHist()

• histograms and histogram normalization

• Descriptor_extractor_matcher

• Use of features 2D detector descriptor
• Also see matcher_simple.cpp

• Distrans

• Use of the distanceTransform on edge images and voroni tessel.

• Edge (Canny edge detection)