i image formation 1
1 geometric camera models 3
1.1 image formation 4
1.1.1 pinhole perspective 4
1.1.2 weak perspective 6
1.1.3 cameras with lenses 8
1.1.4 the human eye 12
1.2 intrinsic and extrinsic parameters 14
1.2.1 rigid transformations and homogeneous coordinates 14
1.2.2 intrinsic parameters 16
1.2.3 extrinsic parameters 18
1.2.4 perspective projection matrices 19
1.2.5 weak-perspective projection matrices 20
1.3 geometric camera calibration 22
1.3.1 alinear approach to camera calibration 23
1.3.2 anonlinear approach to camera calibration 27
1.4 notes 29
2 light and shading 32
2.1 modelling pixel brightness 32
2.1.1 reflection at surfaces 33
2.1.2 sources and their effects 34
2.1.3 the lambertian specular model 36
2.1.4 area sources 36
2.2 inference from shading 37
2.2.1 radiometric calibration and high dynamic range images 38
2.2.2 the shape of specularities 40
2.2.3 inferring lightness and illumination 43
2.2.4 photometric stereo: shape from multiple shaded images 46
2.3 modelling interreflection 52
2.3.1 the illumination at a patch due to an area source 52
2.3.2 radiosity and exitance 54
2.3.3 an interreflection model 55
2.3.4 qualitative properties of interreflections 56
2.4 shape from one shaded image 59
2.5 notes 61
3 color 68
3.1 human color perception 68
3.1.1 color matching 68
3.1.2 color receptors 71
3.2 the physics of color 73
3.2.1 the color of light sources 73
3.2.2 the color of surfaces 76
3.3 representing color 77
3.3.1 linear color spaces 77
3.3.2 non-linear color spaces 83
3.4 amodel of image color 86
3.4.1 the diffuse term 88
3.4.2 the specular term 90
3.5 inference from color 90
3.5.1 finding specularities using color 90
3.5.2 shadow removal using color 92
3.5.3 color constancy: surface color from image color 95
3.6 notes 99
ii early vision: just one image 105
4 linear filters 107
4.1 linear filters and convolution 107
4.1.1 convolution 107
4.2 shift invariant linear systems 112
4.2.1 discrete convolution 113
4.2.2 continuous convolution 115
4.2.3 edge effects in discrete convolutions 118
4.3 spatial frequency and fourier transforms 118
4.3.1 fourier transforms 119
4.4 sampling and aliasing 121
4.4.1 sampling 122
4.4.2 aliasing 125
4.4.3 smoothing and resampling 126
4.5 filters as templates 131
4.5.1 convolution as a dot product 131
4.5.2 changing basis 132
4.6 technique: normalized correlation and finding patterns 132
4.6.1 controlling the television by finding hands by normalized
correlation 133
4.7 technique: scale and image pyramids 134
4.7.1 the gaussian pyramid 135
4.7.2 applications of scaled representations 136
4.8 notes 137
5 local image features 141
5.1 computing the image gradient 141
5.1.1 derivative of gaussian filters 142
5.2 representing the image gradient 144
5.2.1 gradient-based edge detectors 145
5.2.2 orientations 147
5.3 finding corners and building neighborhoods 148
5.3.1 finding corners 149
5.3.2 using scale and orientation to build a neighborhood 151
5.4 describing neighborhoods with sift and hog features 155
5.4.1 sift features 157
5.4.2 hog features 159
5.5 computing local features in practice 160
5.6 notes 160
6 texture 164
6.1 local texture representations using filters 166
6.1.1 spots and bars 167
6.1.2 from filter outputs to texture representation 168
6.1.3 local texture representations in practice 170
6.2 pooled texture representations by discovering textons 171
6.2.1 vector quantization and textons 172
6.2.2 k-means clustering for vector quantization 172
6.3 synthesizing textures and filling holes in images 176
6.3.1 synthesis by sampling local models 176
6.3.2 filling in holes in images 179
6.4 image denoising 182
6.4.1 non-local means 183
6.4.2 block matching 3d (bm3d) 183
6.4.3 learned sparse coding 184
6.4.4 results 186
6.5 shape from texture 187
6.5.1 shape from texture for planes 187
6.5.2 shape from texture for curved surfaces 190
6.6 notes 191
iii early vision: multiple images 195
7 stereopsis 197
7.1 binocular camera geometry and the epipolar constraint 198
7.1.1 epipolar geometry 198
7.1.2 the essential matrix 200
7.1.3 the fundamental matrix 201
7.2 binocular reconstruction 201
7.2.1 image rectification 202
7.3 human stereopsis 203
7.4 local methods for binocular fusion 205
7.4.1 correlation 205
7.4.2 multi-scale edge matching 207
7.5 global methods for binocular fusion 210
7.5.1 ordering constraints and dynamic programming 210
7.5.2 smoothness and graphs 211
7.6 using more cameras 214
7.7 application: robot navigation 215
7.8 notes 216
8 structure from motion 221
8.1 internally calibrated perspective cameras 221
8.1.1 natural ambiguity of the problem 223
8.1.2 euclidean structure and motion from two images 224
8.1.3 euclidean structure and motion from multiple images 228
8.2 uncalibrated weak-perspective cameras 230
8.2.1 natural ambiguity of the problem 231
8.2.2 affine structure and motion from two images 233
8.2.3 affine structure and motion from multiple images 237
8.2.4 from affine to euclidean shape 238
8.3 uncalibrated perspective cameras 240
8.3.1 natural ambiguity of the problem 241
8.3.2 projective structure and motion from two images 242
8.3.3 projective structure and motion from multiple images 244
8.3.4 from projective to euclidean shape 246
8.4 notes 248
iv mid-level vision 253
9 segmentation by clustering 255
9.1 human vision: grouping and gestalt 256
9.2 important applications 261
9.2.1 background subtraction 261
9.2.2 shot boundary detection 264
9.2.3 interactive segmentation 265
9.2.4 forming image regions 266
9.3 image segmentation by clustering pixels 268
9.3.1 basic clustering methods 269
9.3.2 the watershed algorithm 271
9.3.3 segmentation using k-means 272
9.3.4 mean shift: finding local modes in data 273
9.3.5 clustering and segmentation with mean shift 275
9.4 segmentation, clustering, and graphs 277
9.4.1 terminology and facts for graphs 277
9.4.2 agglomerative clustering with a graph 279
9.4.3 divisive clustering with a graph 281
9.4.4 normalized cuts 284
9.5 image segmentation in practice 285
9.5.1 evaluating segmenters 286
9.6 notes 287
10 grouping and model fitting 290
10.1 the hough transform 290
10.1.1 fitting lines with the hough transform 290
10.1.2 using the hough transform 292
10.2 fitting lines and planes 293
10.2.1 fitting a single line 294
10.2.2 fitting planes 295
10.2.3 fitting multiple lines 296
10.3 fitting curved structures 297
10.4 robustness 299
10.4.1 m-estimators 300
10.4.2 ransac: searching for good points 302
10.5 fitting using probabilistic models 306
10.5.1 missing data problems 307
10.5.2 mixture models and hidden variables 309
10.5.3 the em algorithm for mixture models 310
10.5.4 difficulties with the em algorithm 312
10.6 motion segmentation by parameter estimation 313
10.6.1 optical flow and motion 315
10.6.2 flow models 316
10.6.3 motion segmentation with layers 317
10.7 model selection: which model is the best fit? 319
10.7.1 model selection using cross-validation 322
10.8 notes 322
11 tracking 326
11.1 simple tracking strategies 327
11.1.1 tracking by detection 327
11.1.2 tracking translations by matching 330
11.1.3 using affine transformations to confirm a match 332
11.2 tracking using matching 334
11.2.1 matching summary representations 335
1