Computer Vision CS 332
Popular in Course
Popular in ComputerScienence
This 88 page Class Notes was uploaded by Jordon Hermiston on Thursday October 29, 2015. The Class Notes belongs to CS 332 at Wellesley College taught by Staff in Fall. Since its upload, it has received 9 views. For similar materials see /class/230939/cs-332-wellesley-college in ComputerScienence at Wellesley College.
Reviews for Computer Vision
Report this Material
What is Karma?
Karma is the currency of StudySoup.
You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!
Date Created: 10/29/15
Analysis of Motion Recovering observer motion 65332 Visual Processing Department of Computer Science Wellesley College Recovering 3D observer mo rion amp layouT Applica rion Au roma red driving sys rems Challenge Observer mo rion problem From image moTion compu re o Observer Transla rion TX T2 0 Observer roTa rion R R R2 0 DepTh of every loca rion Zgtlty Human percep rion of heading Warren amp colleagues Human accuracy 1 2 visual arc bird39s eye view Observer heading To The Ief r or righT o Targe r on horizon 175 Observer Jus r Transla res Toward FOE heading paint r l A e 391 I I 39 x I DIrecTIons of I veloci ry vec rors inTersec r a r FOE Buf simple s rra regy doesn39T work if observer also ro ra res Observer Translation Rotation Display simulates observer translation Observer rotates l their eyes i mm way b g s 393 Display simulates Q translation rotation Still recover heading with high accuracy Observer motion problem revisited From image matian compute o Observer translation TX Ty T2 0 Observer ratatian RX R R2 0 Depth at every lacatian Zgtlty Observer undergoes both translation rotation meimiuu roli ion Equations of observer mo rion Translation Rotation Depth TX Tyw T2 Ry Ry R2 ZX Y VX Tquot xTzZ Rxxy lyx21 Rzy Vy Ty yTzZ Rxyz1 Ryxy RzX Translational Rotational Component Component Translational componen r of veloci ry V 39T 92 Where is The FOE VV Tyl yT Z X Example 1 TX Ty O TZ 1 Z 10 everywhere Vx Vy SkeTch The veloci ry field Example 2 TX Ty 2 TZ 1 Z 10 everywhere Vx Vy LonguetHiggins Si Prazdny 0 Along a depth discontinuity veacry differences depend only on observer translation 0 Velocity differences point to the focus of expansion Rieger Si Lawton39s algorithm 0 At each image location compute distribution of velocity differences within neighborhood Appearance of sample distributions 6 0 Find points with stroneg oriented distribution compute dominant direction 0 Compute focus of expansion from intersection of dominant directions Analysis of Mo rion Measuring motion in biological vision systems 65332 Visual Processing artmenf of Compufer Science Wellesley College Smoo rl mess assump rion CompuTe a veloci ry field ThaT 1 is consis ren r wi rh local measuremen rs of image mo rion perpendicular componen rs 2 has The leasf amounf of vararia possible Pure Translation 7 true amp smoothest inilial motion velocity field easummems Compu ring The smoo rhes r veloci ry field in71 VYH in W inm VH1 inUxi VYi L39Yi VJ39i mo rion componen rs change in velociTy in139inv VYi139VYi Find in Vyi Tha r minimize ZVgtltiL39gtlti VYiUYi 39 Vii2 AVgtlti139vgtlti2 VYi139VYi2 When is The smoo rhes r veloci l y field correct Rotation ofrigid objects in 20 and 31 m m When is if wrong 3 RM My mic X1 slnnulics iniixi motion vciueiiy eld mamman V X E a me amp smonmsi mum mum kiuc c dcpm enecl vciocil eld mciiwmmcnls mum 4 O39Connell mo rion illusions Twosfage moTion me03uremen r mo rion componen rs 0 2D image mo rion Movshon Adelson Gizzi Si Newsome V1 high 70 of cells selec139ive for direcfion of mofion especially in layer Thaf projecfs To MT MT high 70 of cells selec139ive for direction and speed of mo rion lesions in MTO behavioral defici rs in mofion Tas s TesTing wiTh sinewave quotplaidsquot Illquotl m moving plaid Movshon ef al recorded W E JAULCAV b responses of neurons in area MT To moving plaids wi rh differen r componenf gra rings The logic behind The experimenfs f 239 39 gt f n 1 2 Componenf cells measure perpendicular componenfs of mofion 3 eg selecfive for verfical feafures moving righ r 2 3 Paffern cells in regra re mofion componenfs predicted responses 1 eg selecfive for righ rward mofion of paH39ern 2 3 predicted responses 1 Movshon ef al observafions 39 Cor rical area V1 all neurons behaved like componen r cells 39 Cor rical area MT EVIdence for rwos rage mo rion measuremen 39 Percep rually rwo componen rs are no r in regra red if layers 4 Si 6 componen r cells layers 235 pa r rern cells large difference in spa rial frequency large difference in speed componen rs have differen r s rereo dispari ry Im egr39afing mofion over39 The image in regr39a rion along confour s vs over 2D ar eas Nakayama Xx Silver39man true mafion c perceived ar39ea feafur es moflon mus r be close confour Feafur39es can be far39 away Recovering 3D sTr39ucTur39e from mofion f fur amp smoothest inilial motion kinuli dcplh t c alocily licld ems Wanna amp O Connell Ambigui ry of 3D recovery 1 birds39 eye y x 1 ML We need addifiana cansfrainf to recover 3D structure uniquely quotrigidity constraintquot Image projec rions orThographic perspec rive projec rion projec rion Z Z imageplane X image X plane X Y Z O X Y X Z XZ Z 3D 2D V O W 3D 2D Binocular Stereo Vision Stereo viewing geometry and the stereo correspondence problem 65332 Visual Processing Department of Computer Science Wellesley College Stereo viewing geometry LEFT RIGHT l II positive disparity 9 in front of point of fixation negative disparity 9 behind fixation point LEFT RIGHT 172 Stereo viewing geometry ZED RIGHT J larger dispari l39y 9 further away from fixa l39ion poin l39 zero dispari l39y 1 poin I39s on The horop l39er have zero dispari l39y ResuITs of sTereo processing A 17 n Davi Geiger STereo process eXTracT feaTures from The lefT and righT images whose dispariTy we wanT To measure maTch The lefT and righT image feaTures and measure Their dispariTy in posiTion sTereo correspondence problemquot use sTereo dispariTy To compuTe depTh RandomdoT sTereograms o Bela Julesz 1971 o sTereo sysTem can funcTion independenle 0 we can maTch quotsimplequot feaTures o highlighT The ambyUfyof The maTching process ConsTrainTs on sTereo correspondence 0 Uniqueness each feaTure in The lefT image maTches wiTh only one feaTure in The righT and vice versa 0 Simi IariTy maTching feaTures appear quotsimilarquot in The Two images 0 ConTinuiTy nearby image feaTures have similar dispariTies o Epipolar consTrainT simple version maTching feaTures have similar verTicaI posiTions bu Epipolar consTrainT Left view Right view Possible maTching candidaTes for pL lie along a line in The righT image The epipolar line MarrPoggio cooperaTive sTereo algoriThm I maTches corresponding black amp whiTe doTs in The lefT amp righT images sTereo dispariTies compuTed by a parallel neTwork of simple compuTing elemenTs i ifera ve sTaTe of neTwork changes wiTh each iTeraTion To resolve maTching ambiguiTy dispariTy map emerges over Time InpuT epipolar consfr ainf gm lef r 111 1 if Lxy Rxdy O ofher wise Cleld Cleld similari ry consfr ainf ldlv unno4aa 3D cooperative ne rwor39k ini rial s ra re Cxyd 112 Ngm UpdaTing The neTwor39k For each iTer aTion The sTaTe of every elemenT in The 3D neTwork is updaTed 1 5mad Cgtltyd E eIN supporT for exciTaTion inhibm m disparify d GT currenT sTaTe neigh borhood evidence for locaTion x y of quotafwork SUPPOFT for What d sparr es I O or 1 dispariTy d 0T 0 loca on le locaTion Xy 2c x d 1 if 5x d z 1 Y Y 1 Threshold o if Sxydltr new updaTed sTaTe aT Time T1 3D cooper aTive neTwor39k r ighT Enforcing The com inuify consTr39aim x 1 1 1 O 1 1 O 1 O O 1 00000000 0 O O O O O O O Sxyd Cxyd e I dispari ry 2 E sum of 139s in a neighborhood around xy 3D cooperaTive neTwor39k Cxyd AN Qgt NA D 4 Enfor cing The uniqueness consTr39aim 3 2 I To ral number of 1 al rer na rive 0 dispari ries 1 2 I 3 sxyd Cxyd E c1332 Visual Procemirg in Computer and 51012ng Vman Systemx Edge Detection in Computer Vision Systems changes or images that will be discussed or class General Paints image rthey provide the first hmt about the structure othe scehe being Viewed Step h in NW ima mtehsrty at each location The indices othe rows and columns of the array contammg this image are le column 4 halzmund A dark backgomd upper left 3924 25 2a 27 2s 2s 3o 31 32 33 34 35 24 15 2o 23 15 22 1a 14 17 13 2o 14 14 25 21 13 14 15 1s 14 17 23 21 17 a7 a1 2a 25 as a3 27 24 a1 as 2s 13 53 as 2s 13 as a2 3 14 as aa 31 24 5a 5a 32 2a 54 5a 33 1s 53 57 34 22 a7 57 35 1s 55 an IMAGE The purpose of the smoothing operation is to remove minor uctuations of intensity in the image that are due to noise in the sensors and to allow the detection of changes of intensity that take place at different scales In the image of a handiwipe cloth shown later for example there are changes of intensity taking place at a ne scale that correspond to the individual holes of the cloth If the image is smoothed by a small amount the variations of intensity due to the holes are preserved At a coarser scale there are intensity changes that follow the pattern of colored stripes on the cloth If the image is smoothed by a greater amount the intensity uctuations due to the holes of the cloth can be smoothed out leaving only the larger variations due to the stripes The purpose of the differentiation operation is to transform the image into a representation that makes it easier to identify the locations of significant intensity changes If a simple step change of intensity is smoothed out the first derivative of this smoothed intensity function has a peak at the location of the original step change and the second derivative crosses zero at this location The peaks and zerocrossings of a function are easy to detect and indicate where the significant changes occur in the original intensity function The final step of feature extraction refers to the detection of the peaks or zerocrossings in the result of smoothing and differentiating the image There are properties of these features that give a rough indication of the sharpness and contrast of the original intensity changes Convolution The smoothing and differentiation steps can be performed using the operation of convolution This section provides a simplified description of the convolution operation that is adequate for how we will use convolution in practice This operation involves calculating a weighted sum of the intensities within a neighborhood of each location of the image The weights are stored in a separate array that is usually much smaller than the original image We will refer to the pattern of weights as the convolution operator For simplicity we will assume that the convolution operator has an odd number of rows and columns so that it can easily be centered on each image location To calculate the result of convolution at a particular location in the image we center the convolution operator on that location and compute the product between the image intensity and convolution operator value at every location where the image and convolution operator overlap We then calculate the sum of all of these products over the neighborhood In the example at the top of the next page a portion of an image is shown on the left the array indices are again shown along the top row and left column and a simple 3x3 convolution operator is shown on the right The weights in the convolution operator can be arbitrary values In this example they are integers from 1 to 9 The result of the convolution at location 33 in the image is calculated as follows C33 14 29 33 45 58 67 76 81 99 264 In practice we usually only compute convolution values at locations where the convolution operator can be centered and covers image locations that are all within the bounds of the image array For the image shown below convolution values would not be calculated along rows 1 and 5 and columns 1 and 5 IMAGE CONVOLUTION OPERATOR The Convolutlon opemtloh IS denoted by the it symbol In one dlmmsloh let 1x denote the mtehslty luhctloh 0x denote the com olutloh operator and Cx denote the result ofthe Convolutlon We can then erte Cx 0x 1x In two dlmehslohs each othese functlohs depends on x and y Cxyy 0xyy1xy and show how Smumhing l an n W at r alarger smoothng For example we could calculate the avaage othe mtehsltles wthln a regloh around a a Rl l t al Tn n dlmmslon Lhe Gausslan functlon canbe deflned as follows This function has a maximum value at X 0 In two dimensions the Gaussian function can be de ned as u GAy This function has a maximum value at X y 0 and is circularly symmetric The value of 039 controls the spread of the Gaussian 7 as 039 is increased the Gaussian spreads over alarger area A convolution operator constructed from the Gaussian function erforms more smoothing as 039 is increased For both of the above de nitions of the Gaussian function the height of the central peak decreases as 039 increases but this is not important in practice Differentiation We noted earlier that if a simple step change of intensity is smoothed out the rst derivative of this smoothed intensity function has a peak at the location of the original step change and the second derivative crosses zero at this location In one dimension we can calculate the rst or second derivative with respect to X Let D39X andD X denote the results of the rst and second derivative computations respectively If 1X is also smoothed with the Gaussian function then the combination of the smoothing and differentiation steps can be written as follows D39X ddX GX IX DquotX 12dx2 GX IX Thej 39 quot Jconvolution r quot quot39 sothe L asfollows D39X ddX GX 1X DquotX 12dx2 GX 1X be rewritten This means that the smoothing and derivative operations can be performed in a single step by convolving the intensity signal 1x with either the rst or second derivative of a Gaussian inction shown below the size of the second derivative is scaled in the vertical direction The value of 5 39 L 4 these f quot is increased A 39 quot fthe Gaussian are again r spread over a larger area x1 d X if d2 1 X2 if 70X 736252 GX 3 7 1526 dx 6 dX2 039 o 151 derivative v 239quotl derivative gt The gure on the next page illustrates the smoothing and differentiation operations for a one dimensional intensity pro le Part a shows the original intensity inction Parts and c show the results 0 smoothing the image pro le with Gaussian inctions that have a small and large value for 5 Part d shows the rst derivative of the smoothed intensity pro le in c and part e shows the second derivative of c The vertical dashed lines show the relationship between intensity es in the smoothed intensity pro le peaks in the rst derivative and zerocrossings in the second derivative In two dimensions intensity changes can occur along any orientation in the image and there more choices regarding how to compute the derivative One option is to compute directional derivatives that is to compute the derivatives in particular 2D directions To detect intensity anges at all orientations in the image it is necessary to compute derivatives in at least two directions such as the horizontal and vertical directions Either a rst or second derivative can be calculated in each direction It is possible to rst smooth the image with one convolution step and the derivative computation by a second convolution with operators that implement a derivative Alternatively these two steps can be combined into a single convolution step WWWNWWWW W ALJJWWMVL c NVfo WArvfww A M A M An w A A A A W U W WWW MM AVA M U U U m u r o a ls denvauve ofsmoothedmtensxty e z denvanve ofsmoothedmtensxty The dashedlmes 15 m denvanve and zerorcrossmg m the 2 denvauve We are going to consider in detail an alternative approach that involves computing a single nondirectional second derivative referred to as the Laplacian This approach is motivated by knowledge of the processing that takes place in the retina in biological vision systems The Laplacian operator is the sum of the second partial derivatives in the X and y directions and is denoted as follows V21 i321r 7 x2 6216y2 Although the Laplacian is de ned in terms of partial derivatives in two directions it is a non directional derivative in the following sense Suppose the Laplacian is calculated at a particular location Xy in the image The 2D image can then be rotated by any angle around the location X and the value of the Laplacian does not change this is not true for the directional derivatives mentioned earlier Let LXy denote the result of calculating the Laplacian of an image that has been smoothed with a 2D Gaussian function Then LXy can be written as follows LXy V2GXy 1XY V2GXYl 1Xy This means that the Laplacian and smoothing operations can be performed in a single step by convolving the image with a function whose shape is the Laplacian of a Gaussian This function is de ned as follows 2 2 1 2 727 2 VG 23 rX2f This function is circularly symmetric and is shaped like an upsidedown meXican hat the gure above shows this function with its sign reversed It has a minimum value at the location X y 0 Because the Laplacian is a second derivative the features in the result of the convolution of the image with this function which correspond to the locations of intensity changes in the original image are the contours along which this result crosses zero The space constant c again controls the spread of this function As cs is increased the Laplacian of a Gaussian function spreads over a larger area A convolution operator constructed from this function performs more smoothing of the image as c is increased and the resulting zero crossing contours capture intensity changes that take place at a coarser scale The diameter of the central negative portion of the Laplacian of a Gaussian function which we denote by w is related to c as shown below We will often refer to the size of the Laplacian of a Gaussian mction by the value of w W 2x50quot n practice we can scale all the values of the Laplacian of a Gaussian mction by a constant factor and ip its sign without changing the locations of the resulting zerocrossing contours To construct a convolution operator for this mction it can be sampled at evenly spaced values of X and y and the resulting samples can be placed in a 2D array On the next page we illustrate a convolution operator that was constructed in this way This operator was derived by computing samples of the following function which differs from the Laplacian of a Gaussian mction shown earlier by a constant scale factor and a change in sign 2 ii VZG25 2 2 6262 G In the example shown a xE so that the diameter w of the central region of the function which is now positive is 4 picture elements pixels The array has llxll elements and the center element location 55 represents the origin where x y 0 The maximum positive value of the operator occurs at this location Note that the convolution operator is circularly symmetric CONVOLUTION OPERATOR wnhthxs 1 u m border CONVOLUTION RE SULT Feature Extractinn quotmum L vauve n r y r contours T m u 10 This array could contain l s at the locations of peaks or zerocrossings and 0 s elsewhere The gure below illustrates the results of convolving the image in part a with the Laplacian of a Gaussian function The convolution result is displayed in part b with large positive values shown in white and large negative values shown in black In part c all of the positive values in the convolution result are shown as white and negative values are black The zerocrossings in this result are locationed along the borders between positive and negative values shown in part d These zerocrossing contours indicate where the intensity changes occur in the original image lfthe image is convolved with multiple operators of different size each convolution result can be searched for peaks or zerocrossings that correspond to intensity changes occurring at different scales The next page shows edge contours derived from processing an image of the handi wipe cloth with convolution operators of different size LaplacianofGaussian operators with w 4 and w 12 pixels were used to generate the two results The result of the smaller operator preserves the changes of intensity due to the holes in the cloth while the result of the larger operator captures the intensity changes due to the pattern of colored stripes Some computer vision systems attempt to combine these multiple representations of intensity changes into a single representation of all of the intensity changes in the image Other systems keep these representations separate for use by later stages of processing that might for example compute stereo disparity or image motion i573quot xse to shallowerzero crossm contrast sharpness In the result of convolving a 2D image with the Laplacian of a Gaussian the rate of change of the convolution result as it crosses zero which we will call the slope of the zerocrossing can be used as a rough measure of the contrast and sharpness of the intensity changes Intensity changes can occur at any orientation in the image At the location of a zerocrossing contour the slope of the convolution result is always steepest in the direction perpendicular to the contour The gradient of a 2D function is a vector de ned at each location of the function that poinm in the direction of steepest increase of the function This vector can be constructed by measuring the derivative of the function in the X and y directions The horizontal and vertical components of the gradient vector are the derivatives in the X and y direction grad dXdy The magnitude of this gradient vector indicates the rate of change of the function at each location If we again let LXy denote the result of convolving the image with the Laplacian of a Gaussian function then at the location of a zero crossing the slope of the zerocrossing can be defined as follows slaps The derivatives of LXy in the X and y directions can be calculated by subtracting the convolutionvalues in adjacent locations of the convolution array For example in the convolution result shown on page 9 suppose we want to calculate the slope of the zerocrossing that occurs between location 29 28 ie row 29 and column 28 and location 29 29 The location 29 29 contains the value 231 The derivative in the X direction can be calculated by subtracting the value at location 29 28 yielding 237 and the derivative in the y direction can be calculated by subtracting the value at location 28 29 yielding 270 The result of the slope calculation is then 359 In practice only the relative values of the slopes at different zerocrossings are important so the slopes can be scaled to smaller values The image below was first convolved with the Laplacian of a Gaussian The zerocrossing contours were detected and the slopes of the zerocrossings were calculated as described above Below the image the zerocrossings are displayed with the darkness of the contour proportional to this measure of slope Higher contrast edges in the original image such as the outlines of the coins give rise to darker more steeply sloped zerocrossing contours More MATLAB gtgtnurns13 7 8 4 nums 1 3 7 8 4 gtgt vaIZ 3 8 0 1 0 2 valZ 3 8 4 1 O 2 gtgt sizenums ans 2 5 gtgt sizeval2 ans 2 2 3 gtgt nums 2 nums 3 nums 59172311 gtgt nums3 ans 2 3 gtgt va22 1 ans 2 1 gtgt nums4 10 nums 1 3 7 gtgt va21 3 6 va 2 3 8 1 0 gtgt vaIZAZ 104 6 2 Error using gt mpower Ma rrix mus r be square gtgtva ans 9 64 36 1 O 4 Creating large images gtgt imagel zeros200200 double numbers gtgt image2 ones200200 39 320000 by res gtgt imagel uin1398ones200200 gt 401000 byfeS in regers 0255 Exercise Wri re one s ra remen r ro crea re an ini rial 200x200 image of 10039s There are of leas r 4 differen r ways To do This Colon no39l39a39l39ion nums 18 quotW s defaul l39 step is 1 1 2 3 4 5 6 7 8 vols 2321 vals 2 5 8 11 14 17 20 ex l39ends The sequence vals 051560 15 for as Possibleu 05000 10000 25000 40000 55000 nums24 0 nums 10005678 gtgt index 4 numsindex2index2 use variables In colon no l39a l39lon ans Colon noTaTion amp images gtgt image uin r8zeros200200 gtgt image4080 302120 100 gtgt image60150 1002180 200 gtgt imshowimage Figure1 File all View mm Yals Deskkav Windaw HElD 91quot 591 lt7 145 SE 5 E no l39e pixel coordina res are reversed Defining new functions function ltoutputsgt functionname ltinputsgt ltstatements comprising the body of the functiongt m function distance x1y1 x2y2 distance sqrtx2x y2y1 2 gtgt dist getDistance1184 is 76158 circleInfom function area perimeter circleInforadius area pi radius 2 perimeter 2 pi radius gtgt areal periml circleInfo100 3141593 periml 628319 Conditionals if statements if num lt 0 I If val gt 10 Sum 39 absnum39 result 10 else result 10val nums 3 7 2 9 gal o if val quot result sumnums elseif Val 1 l Val 2 if num 0 result prodnums result 10num elseif val gt 2 amp Val 10 else result minnums result 0 end se result 0 end Loops for statements for ltvariabIenamegt ltvauesgt ltcommands to execute for each value of variablegt end sum1 O for n 110 sum1 sum1 nAZ en numbers715923648 evens O for num numbers if remnum 20 evens evens 1 an end prodl 1 for val 100 15 40 prodl prodl va nd image uint850rand100100 t 0 un for X 1 100 for y 100 if imagegtlty gt 25 count count 1 end end end 3D Shape from Shading Image formation and the shape from shading problem 65332 Visual Processing Department of Computer Science Wellesley College 3D shape from shading Image infensi ries depend on a 3 D surface shape a Surface reflectance proper ries Compu re explicile General a Illumination in The scene 0 Viewing geome rry assump rions From Woodham 1984 images courfesy of Merle Norman Cosmefics Viewing geometry SOURCE i incident angle e emergent angle g phase angle NORMAL VIEWER lt5 lg What fraction of the incident light w is reflected toward the viewer irradiance r ildicmce Reflectance Function ilelg surface radiance surface irradiance Reflectance functions mirroralelg 1 i6 amp ieg 0 otherwise ma eieg p cosi if ilt 90 0 otherwise p39 albedo More r eflecfance functions moonOelg P COS 90 C05 6 0 o rher wise SEMieg 1cos e Represen ring surface orien ra rion using sfer39eogr39aphic projec rion sphere has radius 1 lt quot tR s 5 w v s QIIIII amp I n x 5 r M x g g 039 2 g 2 0 viewer Given surface brightness can we determine surface orientation Surface Normal Lambertian Sphere Reflectance Map Rfg Given 1 light source 2 viewer position 3 surface reflectance properties Rfg relates brightness to surface orientation Image Irradiance Equation Igtlty Io pgtlty R039 Ikeuchi amp Horn shapefromshading algori rhm Three sources of constraint 1 image in i39ensi i39y Igtlty known viewer direction light source direction I x I x R fl surfacerefleciance y o p y 9 Properties 2 surface smoofhness eg minimize To fal variation in surface 3 points of known surface orienfa fion eg occluding boundaries shadow boundaries Boundaries influence shape percep rion Ramachandran 1988 Analysis of Mo rion Computing the velocity field 65332 Visual Processing Deparfmenf of Compufer Science Wellesley College aper rur e problemquot quotlocalquot mofion defectors only v gt 39 2 velocify field measure component afmo an perpendicular To moving edge 2D velocify field no defermined uniquely from The changing image need add39fbna consfranf To compute a unique velocity field Assume pure fransa on or consfanf Veocfy Error in inifial mofion measuremenfs In Prac ce o Velocifies no consfam39 locally 0 Image feafures wifh small range of orierrtafions Smoo rhness assump rion CompuTe a veloci ry field Tha r 1 is consis ren r wi rh local measuremen rs of image mo rion perpendicular componen rs 2 has The eas397 amaLm of var2177017 possible Pure Translation mu amp smomhest inilial motion velocity eld measummcn When is The smoo rhes r veloci i y field curred Rotation ofrigid objects in 21 and 3D When is if wrong 0 In amp sman ics inilial moiion veinciiy eld mmummunis iltgtgt Lrui amp smomiiasl vciuclry min minnlmminn kmwhcv iupihc eci mamman Iiuamp039Cumicii m of i o n Musans Measuring mo rion in one dimension Ix Vx veloci ry in X direc rion o rightward movement VX gt 0 o leftward movement VX lt 0 0 speed IVXI o pixelsTime step Measuremznf of mo fion componznfs in 2D 19radieri1ufimcigeimensiiy VI BI3x al 2 time derivative 31m 3 veiuoiiy niung gradient vi movement in giieciim af giooieni VA gt n movement appaii e amt ion af gimieni VA n v1 GIw2 HI602 Q n 2D veloci ries VXVY consis ren r wiTh VJ39 vx vy Vx Vy vi gtlt All vx vy such that the component of vx vy in the dineotion of the gradient is iii ux uy unit vector in dineotion of gradient Use the dofproducf vx vy ux uy vi TimeouT exercise DeTails 39 solve for VX V 4 aIax 10 aIay 10 BIaf 30 r b Vx Vy aIax 10 aIay 10 aIar 30 r For eacljl component 1 Ux any 3vi 4 VXUX Vyuy 2 VJ In pracfice Previously vy New strategy Find VX V rha r besf f fs ail mo rion x componen rs Toge rher Find Vx Vy Tha r minimizes ZquX Vy Liy vi2 CompuTing The smooThesT velociTy field in1 Vm in Vyi J mel Vym inuxi Vyi in V i mo rion components change in veloci ry VXM39VXiI VYi139Vyi Find in VYi Tha r minimize zVXiUXi VYiUYi 39 Vida 7 in139vxi2 VYi139VYi2 HighLevel Vision Objec r Recognition 65332 Visual Processing Department of Computer Science Wellesley College Recogni rion from geomeTric shape Objec rs can be recognized easily from The shape of Their image confours Very young children can easily recognize a wide vorie ry of common objec rs OTher39 recogniTion cues characteris c mo l39ion Tex ru r39e confex r Why is recogniTion difficulf Face recogniTion AR Face Da l abase WhaT is a chair Approaches to recognition differ in how reguar es are used to constrain the interpretation of the viewed object Three main approaches 0 invariant properties 0 parts decomposition o alignment Invariant properties Every instance of each object class exhibits certain properties 1 measure properties of viewed object 2 apply decision procedure Properties I 3 n 0 gt 1 ratio perimeterarea 7 39 2 brightness I 39 39 Feature I ll 7 7 Space classify cells in culture corn pact elongated Parts decomposition Every instance of each object class shares certain parts arranged in a certain way 1 find object parts 2 recognize objects by presence of parts with proper relationship Structural Descriptionquot 1 9 Biederman39s Geons Face recognition by parts decomposition Training Tempiaies MIT Media Lab Vision Si Modeling Group Fea rur e hierarchies Human 1 E3 in i Objec rcen rer39ed r epr esen ra rion Marr amp Nishihar a MenTal r o ra rion Time needed To determine whe rher pair of objec rs are 15 same is propor rional To angle of r o ra rion be rween pair 112 Viewercentered object representation Tarr 95 After learning to recognize a set of 3D objects from a small set of specific 2D views of these objects the time novel view is proportional to the 3D an le between the new view and closest learned view The debate continues Viewpoint invariant VEWPEIHT depenseflll object representations 0 Jec represen a Ions Biederman AlignmenT meThods Find an objecT model and geomeTric TransformaTion ThaT besf mafch The viewed image V viewed objecT image objecT models T allowable TransformaTions beTween viewed objecT and models F measure of fiT beTween V and The expecTed appearance of model Mi under The TransformaTion TiJ GOAL Find a combinaTion of Mi and TiJ ThaT maximizes The TH F AlignmenT meThod recogniTion process 1 Find besT TransformaTion Ti for each model Mi opTimizing over possible views 2Find Mi whose besT TiJ gives The besT maTch To image V Aligning picToriaI models 4 TriangulaTed model Tm TSformed m9de superimposed on image When The model doesn39T fiT Transformed Transformed model 0 el superimposed on image Analysis of Mo rion Recovering 3D structure from motion 65332 Visual Processing Department of Compu rer39 Science Wellesley College Recovering 3D s rr39uc rur39e from mo rion true amp smoothest initial motion kinetic depth effect velocity eld measurements Wallach amp O Connell Ambigui ry of 3D recovery I l quotl l l I ll views We need addirional consfrainr ro recover 3D structure uniquely rigidity constraintquot Image projec rions Perspec ve projecTio orThographic Z projec on image X plane X Y Z 9 XZ YZ only scaled dep lh image plane X Y Z 9 X Y only rela live dep rh requires Translation of requires objec l ro la rion observer rela live To scene Using The rigidiTy consTrainT Ullman Given 3 dis rinc r or rhographic views of 4 noncoplanar poin rs in mo rion if There exis rs a rigid 3D s rruc rure consis ren r wi rh These views fhen fhis sfrucfure is unique view1 view 2 view 3 se rof swe 3D s rruc rure o oin rs 2D posi rions of poin rs in 3 views caveat dep rh reversals Wha r is needed To compu re a unique rigid 3D s rruc rure orfhographic perspec rive Sample reSUI rs projec on projec rion 4 I 139 7 39 1 correspondence Pom S Pom S 3 views 2 Views velocify field 5 P ims 5 POinTS 112 views 1 view Ullman Rigidi ry cons rrain r alone is sufficien r ro compu re 3D s rruc rure from mo rion BUT Human recovery of 3D s rruc rure from mo rion 39 Needs ex rended Time To ob rain an accura re 3D s rruc rure and deriva rion is no r allornone 39 Can cope wi rh significan r devia rions from rigidi ry 39 In regra res mul riple sources of informa rion These fac rors mo riva red The design of Ullman39s incremenfa rigid1y scheme IncremenTal Rigidify Scheme X x1 y1 21 dep rh Z x 2 Q 323 3 39 ini rially ZO 39 X1 Y1 9 of all poin rs Xal Y339 7 7 Find new 3D model X2 ya 22 rha r maximizes rigid1y le yzl u IIII Iquot Compu re new Z values rha r minimize image change in 3D s rruc rure Birds eye view ZA 1 quot lJ 0 C 4 I Q 1 Q 39J I 390 O I z 1 0 u A n v V image 0 cur39r39enT model Find new Zi ThaT minimize Z 0 new image xVV Incremen ral r39igidi ry scheme r39esul rs bird39s eye views new 3D model afTer ever39y 10 r39oTaTion Tr39ue sTr39ucTur39e 39 39 39 39 39 compuTed 3D model builds 3D model incrementally over39 exTended Time 110 Spon roneous dep rh reversal True sTrucTure compuTed 3D model 111 Human recovery of 3D s rruc rure from mo rion 23 points sufficienT 1 i N m i o JOHANSSON 0 requires exTended Time To build up occuroTe percepT of 3D sTrucTure w quotVa 39 o 539 quota quot50 u 0 P a an 0 J x0 I 5 o 0 o u 39a 39 39 9 39 quot K r i 39 i n I I o 3quot IA 4 39 39 quot2n quotLaue 5 Anderson amp colleagues 112 Edge Defec rion in Compu rer Vision Analyzing in rensi ry changes in digi ral images 65332 Visual Processing szar39fmznf of Compufzr39 Science Wellzslzy College Defec on of image in l39ensi I39y changes Defeating in l39ensi I39y changes Smooth the image intensi es 0 reduces effect ofnoise 0 sets resolution or scale of analysis Differentiate the smoothed intensities transforms image into a representation that facilitates detection of intensity changes Detect and describe features in the transformed image eg peaks or zerocrossings Smoo l39hing The image in l39ensi I39y q Jam intensity smoothing JVLAM MNL smoothing Deriva rives of The smoo rhed in rensi ry h d rA p A A M smoot e I I r f i I intensity kw W mVV 1 L L y 1 quotpeaksquot rst I 0 Wu 7 M A m A A A derivative V V c V V V 1 1 V A V 39 i 259 0 a i Mi f M er1va 1ve V V V j zeroc rossingsquot image after smoothing and second derivative black I l negative 5 0 I u 7 32 Zerocrossmgs white pv cg w posmve 0 1 371 Smoothing The image intensifies Strategy 1 compute the average intensity in a neighborhood around each image position Strategy 2 compute a weighted average of the intensity values in a neighborhood around each image position 0 use a smooth function that weighs nearby intensities more heavily 0 Gaussian function works well in one dimension 2 Ill 1 H39 4 small 6 Gx e 2quot 0 k large 6 Convolution in one dimension 2U gt I1llallmllmlmlmlwlmlmllmlBll1 1539 I10103910101010202020202020 T 1 1 1 1 2 3 4 5 E 7 E 9 1U 11 l 1ntens1ty m convolution I00 5 Operator 2 3 A 5 E 7 E a 1D 11 1 convolution result I O I O I1803918039190220320350360360 O I O I 6X IX 1 8 The deriva rive of a convolu rion F7 Xp3954 2 Fm 9mm immm gq jw WW igsw m 72me m Convolu rion in Two dimensions Illll BEE Illll convolution result convolution operator image SmooThing a 2D image To smooth a 2D image Ixy we convolve With a 2D Gaussian new 202 iii result of convolution Gxy May 1mage Differentiation in 2D To differentiate the smoothed image we Will use the Laplacian operator 2 31 31 V g a We can again combine the smoothing and derivative operations ZZGxy 1xy V2Gxy Km 2 1 r2 26 r22x2y2 quot27 displayed with sign reversed De recTing in rensi ry changes at mul riple scales small 6 large 6 zerocrossings of convolu rions of image wi rh V26 oper39a ror39s Compu ring The con rr39asT of in rensi ry changes Binocular S rereo Vision Properties of human sTereo processing 65332 Visual Processing Department of Computer Science Wellesley College ProperTies of human sTereo processing Use features for S lereo ma lching whose posi rion and dispari ry can be measured veryprecsey Sfer39eoacufy is only a few seconds of visual angle difference in depth m 001 cm at a viewing distance of 30 cm Stereoacuity is the difference between the two angles ProperTies of human sTereo processing MaTching feaTures musT appear s mar in The lefT and righT images w For example we can39T fuse a lefT sTereo image wiTh a negaTive of The righT image ProperTies of human sTereo processing Only quotfusequot objecTs wiThin a unzsgg s zrgrgssnm limiTed range of depTh around The fixaTion disTance Vergence eye mo vemenfs are needed To fuse objecTs over larger range of depThs Double Images Crossed Disparity Convergence for a far target Convergence ver a near target V Len Eye Proper ries of human stereo vision We can only Tolera re small amounts of verfca disparfya r a single eye posi rion VerTical eye movements are needed To handle large verTical dispari ries In The early stages of visual processing The image is analyzed aT mufpe spafa scaes STereo information at mul riple scales can be processed independenle Neural mechanisms for stereo processing o WWmammals zero disparity at fixation distance near in front of point of fixation far behind point of fixation G Poggio Kl colleagues complex cells in area V1 of primate visual cortex are selective for stereo disparity neurons that are selective for a larger disparity range have larger receptive fields In Summary some key points 0 Image features used for matching simple precise locations multiple scales similar between leftright images 0 At single fixation position match features over a limited range of horizontal Si Vertical disparity 0 larger range of disparity O ranges of stereo disparity Eye movements used to match features over Neural mechanisms selective for particular Ear ly Processing in Biological Vision Analyzing in rensi ry changes in The re rinal image 65332 Visual Processing Deparfmenf of Compufer Science Wellesley College The human eye Sclera cillary body The s rruc rure of a neuron Major parts of a neuron l dendrites 2 cell body axon 4 terminals and synapses L V m 1011 neurons in The brain Projec rion from The re rina Laaral genlcuiale body E Ophcuact omicmmauon E quot Opln chiasm Ophc nerve 1339 cortical stage of visual processing primaryvisuol cortex or area Vsual oonax Refinal ganglion cells receptive elds exhibit a centersurround structure Whose crosssection is the difference of two Gaussians Neural Raspcnse RecepliveFleld RespanMPw le Cenlar 5 Surmum u E an on Hanznnvaansmon A ONCENTER OFFSURROUND CELLS 7cm Sunoum E 4 E ailGenie 0quot 0quot Horizanmnsmun Processing wi rhin The r e rina Hun01ml lhpolvn lls un 0 Rm pm Analyzing a 2D image black negative white positive 73 9quot f x s i image after smoothing and second derivative zerocrossings David Hubel amp Thor39S I39on Wiesel Singlecell recording from visual cortex Hubel amp Wiesel identified 3 basic ce Types simple complex hypercomplex cells Some simple cells respond bes r To lines A Light Line Detector B Dark Line Detector Firing Rate Horizontal Position Horizontal Position of a parficular con iras i sign orien ia iion posi iion Some simple cells respond besi To edges 0 DarktoIight Edge Detector D Lighttodark Edge Detector 7 Horizontal Position Horizontal Position Again of a par iicular conir asi sign or ienia rion posi rion large recepiive fields 9 course spaiial sir uciur e small recepiive fields 9 fine spaiial sir uciur e Analysis of Motion Measuring image mo l ion 65332 Visual Processing Department of Computer Science Wellesley College Analysis of visual mo rion Represen ra rions of image mo rion Frame 2 Frame 3 Frame 1 Human visual sys rem 1 shor r range mo rion process 2 long range mo rion process Aper rure problem quotlocalquot mo rion deTec rors provide only one componenT of mo rion in The direc rion perpendicular To a moving edge To make ma er39s wor39se 2D veloci ry field is no r de rer mined uniquely from The changing image We need add39 ona consfrahf To compute a unique velocity field Assume pure fransafon mystery motion measurement strategy Practical considerations for methods based on pure translation 0 Error in initial motion measurements 0 Velocities not constant locally 0 Local image features may have small range of orientations But such strategies are good for detecting sudden movements tracking detecting boundaries Smoo rhness assump rion CompuTe a veloci ry field Tha r 1 is consis ren r wi rh local measuremen rs of image mo rion perpendicular componen rs 2 has The eas397 amaLm of var2177017 possible Pure Translation WM moolhesl quotTm rle tn velocity eld initial motion measurements When is The smoo rhes r veloci fy field correct Rotation of rigid objects in 2D and 3D n c amp Smco msl inilial moliorl vclncity Field measureman true amp smamhesl iniiml mollon velocity eld measuremenls kmcili dcplh cflecl Wallach amp o cqmn When is The smoo rhes r veloci l y field wrong barberpole illusion g frm 9 3 m i 5 lt1 h 3 7 Lrue smoolhest white egg illusion w Spiral 0 We 5mm with ui me bu r so are we
Are you sure you want to buy this material for
You're already Subscribed!
Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'