Computer Vision COSC 6373
Popular in Course
Popular in Chemistry
This 51 page Class Notes was uploaded by Lowell Harris on Saturday September 19, 2015. The Class Notes belongs to COSC 6373 at University of Houston taught by Shishir Shah in Fall. Since its upload, it has received 103 views. For similar materials see /class/208166/cosc-6373-university-of-houston in Chemistry at University of Houston.
Reviews for Computer Vision
Report this Material
What is Karma?
Karma is the currency of StudySoup.
You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!
Date Created: 09/19/15
6056 6373 Computer Vision Motion Estimation Acknowledgement Notes by Profs R Szeliski S Seitz L Shapiro and S Shah Why estimate motion IWe live in a 4D world IWide applications Object Tracking Camera Stabilization Image Mosaics 3D Shape Reconstruction SFM Special Effects Match Move Frame from an ARDA Sam le Vid 0 Change detection for surveillance vid oifF mies i F1 FZTF j quoti Objects appear move disappear Background pixels remain the same simple case How do you detect the moving objects 0 Simple answer pixelwise subtraction Example Person detected entering room Pixel changes detected as difference components 0 Regions are 1 person 2 opened door and 3 computer monitor 0 System can know about the door and monitor Only the person region is unexpected Change Detection via Image Subtraction for each pixel 7 if lll rc 12rc gt threshold then Ioutrc 1 else Ioutrc 0 Perform connected components on Iout Remove small regions Perform a closing with a small disk for merging close neighbors Compute and return the bounding boxes B of each remaining region What assumption does this make about the changes Known regions are ignored and system attends to the unexpected region of change Region has bounding box similar to that of a person System might then zoom in on head area and attempt face recognition Optical flow Problem definition optical quot2 2quot x o 0 I I How to estimate pixel motion from image H to image I Solve pixel correspondence problem given a pixel in H look fopixels of the in l Key assumptions color constancy a point in H looks the same in l For grayscale images this is brightness constancy small motion points do not move very far This is called the optical flow problem Optical flow constraints grgscale Images iv 9 displacement uv ltx39uyv H 13 y I 13 y I Let s look at these constraints more closely brightness constancy Q what s the equation HX y IXu yv small motion u and v are less than 1 pixel suppose we take the Taylor series expansion of l 33u yv 513 yug vhigher order terms Ixyug v Optical flow e a l Combining these two equations 0uyyv Hxy shorthand 362 x7y Ix va HQ Thexcoinponentof the gradlent vector e Ion y Hoe y Ixu on s It mu va sItVIu 1 What is It The time derivative of the image at Xy How do we calculate it Optical flow e nation A d onvrmm fl I Q how many unknowns and equations per pixel 1 equation but 2 unknowns u and V Intuitively what does this constraint mean The component of the flow in the gradient direction is determined The component of the flow parallel to an edge is unknown Aperture rolglem Aperture problem V Solving the aperture problem I Basic idea assume motion field is smooth I Lukas amp Kanade assume locally constant motion I pretend the pixel s neighbors have the same uv I If we use a 5X5 window that gives us 25 equations per pixel O ItPi VIpi u v I Many other methods exist Here s an overview I Barron JL Fleet DJ and Beauchemin S Performance of optical flow techniques International Journal of Computer Vision 1214377 1994 LukasKannada flow I How to get more equations for a pixel I Basic idea impose additional constraints most common is to assume that the flow field is smooth locally I one method pretend the pixel s neighbors have the same uv o If we use a 5x5 window that gives us 25 equations per pixel O ItPi Wm u 12 IaP1 IyP1 25131 10P2 I M102 U Iti gt2 Ix1i25 Iy139325 It 25 RGvsn I How to get more equations for a pixel I Basic idea impose additional constraints most common is to assume that the flow field is smooth locally I one method pretend the pixel s neighbors have the same uv o If we use a 5x5 window that gives us 253 equations per pixel O ItPiOa 1 2 Vlpi0 1 2 u v Iap10 IyP10 ItP10 IxP11 IyP11 ItP11 IaP12 IyP12 u ItP12 Iap25l0 Iyp25lOl quot Itltp25ioi IxP251 IyP251 ItP251 ImP252 IyP252 ItP252 LukasKanadeflow e l Prob we have more equations than unknOWns A d b minimize Ad b2 Solution solve least squares problem minimum least squares solution given by solution in d of ATA d ATb EAGLE 2ny u 21x17 21ny 21ny 1 ZIyIt The summations are over all pixels in the K x K window This technique was first proposed by Lukas amp Kanade for stereo matching 1981 Conditions forsolvability I Optimal u v satisfies LucasKanade 21361 1ny u 21x1 2 1ny z 1ny v z 9 When is This Solvable ATA should be invertible ATA should not be too small due to noise eigenvalues l1 and I2 of ATA should not be too small ATA should be wellconditioned I1 2 should not be too large I1 larger eigenvalue Eds auel rolms Z VIVIT large gradients all the same largel1 small I2 L0 feature ons on w Z VIVIT gradients have small magnitude smalll1 small I2 High textured 9 ionWork best N 8 quot IJ39H J r K I 1 JL I ZVIVIT 6 gradients are different large magnitudes 12 o 2 largel1 large I2 I What are the potential causes of errors in this procedure I Suppose ATA is easily invertible I Suppose there is not much noise in the image When our assumptions are violated Brightness constancy is not satisfied The motion is not small A point does not move like its neighbors window size is too large what is the ideal window size Revisiting the small motion a umtion W I t quotT quot1 into o l v a I Is this motion small enough I Probably not it s much larger than one pixel 2quot 1l order terms dominate I How might we solve this problem Iterative Refinement a I Estimate velocity at each pixel using one iteration of Lucas and Kanade estimation I Warp one image toward the other using the estimated flow field easier said than done I Refine estimate by repeating the process 6056 6373 Computer Vision Image Alignment Acknowledgement Notes by Profs R Szeliski S Seitz S Lazebnik and S Shah Image alignment ImagelinimnE 9tiati9n i if r Recognition of object instances ImagelinnmnE hale Occlusion clutter Image alignment at 0 Two broad approaches I Direct pixelbased alignment I Search for alignment where most pixels agree I Featurebased alignment I Search for alignment where extracted features agree I Can be verified using pixelbased alignment Alignment as fitting Last lecture fitting a ni dcTel to Features in one image M Find model M that minimizes 2 residualx M Alignment as fitting Last lecture fitting a model to features in one image M Find model M that minimizes 2 residualx M 0 Alignment fitting a model to a transformation between pairs of features matches in two images x Find transformation T 0 T I39 that minimizes o gt 939 O 0 Q 39 Zresidua1Txx 39 i FeatureI9an alrinment oulimat Extract features Featurebased alig nment outl me al nment ogtl39ne Featurbsed Extract features Compute putative matches Extract features Compute putative matches Loop I Hypothesize transformation Tsma group of putative matches that are related by T Featuirbse ajinment outline Extract features 0 Compute putative matches Loop I Hypothesize transformation Tsma group of putative matches that are related by T I Verify transformation search for other matches consistent with T Featu rbse algnment outline Extract features Compute putative matches Loop I Hypothesize transformation Tsma group of putative matches that are related by T I Verify transformation search for other matches consistent with T 2D transformation models Similarity translation I 39l I W scale rotation I gt I gt I Affine o Projective homography Let s start with affine transformations Simple fitting procedure linear least sqiiares o Approximates viewpoint changes for roughly planar objects and roughly orthographic cameras Can be used to initialize fitting for more complex models Fitting an affine tfansformation gt 0 Assume weiknowith eicbrFespoh dences how do we get the transformation xiy xiayi to K 0 0 90 of r 39 m1 on c x m1 m2 xi t1 xi yi 0 0 1 0 m3 x I I yi m3 m4 yi t2 0 0 xi yi 0 1 m4 yi t1 t2 Fitting an affine transformatiO xyi0010m3 x1 i Ooxiyi01m4 yi Linear system with six unknowns Each match gives us two linearly independent equations need at least three to solve for the transformation parameters What if we dor t know the correspondences What if we don t know the correspondenoes feature feature descriptor descriptor Need to compare feature descriptors of local patches surrounding interest points Feature descriptors Assuming the patches are already normalized ie the local 39 effect of the geometric transformation is factored out how do we compute their similarity Want invariance to intensity changes noise perceptually insignificant changes of the pixel pattern Feature descriptors 7 1 SinTplestdescriEtor vecquot or o raw intensity values 0 How to compare two such vectors I Sum of squared differences SSD SSDuv 2111 vi2 539 I Not invariant to intensity change I Normalized correlation 2104 l7Vz 17 uj z72 102 if I Invariant to affine intensity change puv Feature desclfiptors Disadvantage ofgpatches as descriptors I Small shifts can affect matchini score a lot Solution h I E Feature descriptors SIFT Descriptor computation I Divide patch into 4x4 subpatches I Compute histogram of gradient orientations 8 reference angles inside each subpatch I Resulting descriptor 4X4x8 128 dimensions 144 92 a K 3x I39 4 an quot n WWW David G Lowe quotDistinctive image features from scaleinvariant kevpoints IJCV6O 2 pp 91110 2004 Feature descriptors SIFT Descriptorcomputation 7 I Divide patch into 4X4 subpatches I Compute histogram of gradient orientations 8 reference angles inside each subpatch I Resulting descriptor 4x4x8 128 dimensions 0 Advantage over raw vectors of pixel values I Gradients less sensitive to illumination change I Subdivide and disorderquot strategy achieves robustness to small shifts but still preserves some spatial information David G Lowe quotDistinctive image features from scaleinvariant kevpoints IJCV6O 2 pp 91110 2004 Feature matching Generating pUtatve matches for each patch in one image find a shont list of patches in the other image that could mat
Are you sure you want to buy this material for
You're already Subscribed!
Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'