1、Computer Vision(机器视觉)Image by Todays Talk What is Computer Vision?Why Study Computer Vision?How Vision is Used Now?Overview of Computer Vision Algorithm Challenges of Computer Vision Questions2What is computer vision?Terminator 23Terminator 5Every picture tells a story4Goal of computer vision is to
2、write computer programs that can interpret imagesCan computers match(or beat)human vision?5What is Computer Vision?Automatic understanding of images and video1.Computing properties of the 3D world from visual data(measurement)6 1.Vision for measurementReal-time stereoStructure from motionNASA Mars R
3、overPollefeys et al.Multi-view stereo forcommunity photo collectionsGoesele et al.Slide credit:L.Lazebnik7What is Computer Vision?Automatic understanding of images and video1.Computing properties of the 3D world from visual data(measurement)2.Algorithms and representations to allow a machine to reco
4、gnize objects,people,scenes,and activities.(perception and interpretation)82.Vision for perception,interpretationskywaterFerris wheelamusement parkCedar Point12 Etreetreetreecarouseldeckpeople waiting in linerideriderideumbrellaspedestriansmaxairbenchtreeLake Eriepeople sitting on rideObjectsActivit
5、iesScenesLocationsText/writingFacesGesturesMotionsEmotionsThe Wicked Twister9What is Computer Vision?Automatic understanding of images and video1.Computing properties of the 3D world from visual data(measurement)2.Algorithms and representations to allow a machine to recognize objects,people,scenes,a
6、nd activities.(perception and interpretation)3.Algorithms to mine,search,and interact with visual data(search and organization)103.Vision for search and organization11Components of a computer vision systemLightingSceneCameraComputer Scene InterpretationSrinivasa Narasimhans slide12Computer vision vs
7、 human visionWhat we seeWhat a computer sees13Vision is really hardVision is an amazing feat of natural intelligence Visual cortex occupies about 50%of brain More human brain devoted to vision than anything elseIs that a queen or a bishop?14Vision is multidisciplinary From wikiComputer GraphicsHCI15
8、Why computer vision mattersSafetyHealthSecurityComfortAccessFun16A little story about Computer VisionIn 1966,Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to“spend the summer linking a camera to acomputer and getting the computer to describe what it saw”.We now know that th
9、e problem is slightly more difficult than that.(Szeliski 2009,Computer Vision)17Ridiculously brief history of computer vision1966:Minsky assigns computer vision as an undergraduate summer project1960s:interpretation of synthetic worlds1970s:some progress on interpreting selected images1980s:ANNs com
10、e and go;shift toward geometry and increased mathematical rigor1990s:face recognition;statistical analysis in vogue2000s:broader recognition;large annotated datasets available;video processing starts2030s:robot uprising?Guzman 68Ohta Kanade 78Turk and Pentland 9119 Why study computer vision?Millions
11、 of images being captured all the timeLots of useful applicationsThe next slides show the current state of the artSource:S.Lazebnik Flickr01E+092E+093E+094E+095E+096E+092003/12/152004/12/152005/12/152006/12/152007/12/152008/12/152009/12/151 billion2 billion3 billion4 billion5 billion6 billion Other
12、photo sharing sites10 billion20 billion50 billion2003/12/152004/6/152004/12/152005/6/152005/12/152006/6/152006/12/152007/6/152007/12/152008/6/152008/12/152009/6/1530 billion40 billion and growingFlickr:1.7 million photos/dayFacebook:100 million photos/dayYouTube:35 hours of video every minute 57 bil
13、lion photos will be taken(US)in 2010http:/ of November 2010)(compare with 17 billion negatives exposed in 1996)(as of February 2010)How vision is used nowExamples of state-of-the-art241.Optical character recognition(OCR)Digit recognition,AT&T labshttp:/ to convert scanned docs to text If you have a
14、scanner,it probably came with OCR softwareLicense plate readershttp:/en.wikipedia.org/wiki/Automatic_number_plate_recognition252.Face detectionMany new digital cameras now detect facesCanon,Sony,Fuji,263.Smile detectionSony Cyber-shot T70 Digital Still Camera 274.3D from thousands of imagesBuilding
15、Rome in a Day:Agarwal et al.200928The old city of Dubrovnik,4,619 images,3,485,717 points5.Object recognition(in supermarkets)LaneHawk by EvolutionRobotics“A smart camera is flush-mounted in the checkout lane,continuously watching for items.When an item is detected and recognized,the cashier verifie
16、s the quantity of items that were found under the basket,and continues to close the transaction.The item can remain under the basket,and with LaneHawk,you are assured to get paid for it“296.Vision-based biometrics“How the Afghan Girl was Identified by Her Iris Patterns”National Geographic307.Forensi
17、csSource:Nayar and Nishino,“Eyes for Relighting”Source:Nayar and Nishino,“Eyes for Relighting”Source:Nayar and Nishino,“Eyes for Relighting”8.Login without a passwordFingerprint scanners on many new laptops,other devicesFace recognition systems now beginning to appear more widelyhttp:/ recognition(i
18、n mobile phones)Point&Find,NokiaGoogle Goggles3510.Vision in spaceVision systems(JPL)used for several tasks Panorama stitching 3D terrain modeling Obstacle detection,position tracking For more,read“Computer Vision on Mars”by Matthies et al.NASAS Mars Exploration Rover Spirit captured this westward v
19、iew from atop a low plateau where Spirit spent the closing months of 2007.3611.Industrial robotsVision-guided robots position nut runners on wheels3712.Mobile robotshttp:/www.robocup.org/NASAs Mars Spirit Roverhttp:/en.wikipedia.org/wiki/Spirit_roverSaxena et al.2008STAIR at Stanford38THANK YOUSUCCE
20、SS2022-10-29可编辑13.Medical imagingImage guided surgeryGrimson et al.,MIT3D imagingMRI,CT4014.Digital cosmetics4115.InpaintingBertalmio et al.SIGGRAPH 004216.DebluringFergus et al.SIGGRAPH 064317.SportsSportvision first down lineNice explanation on http:/ carsMobileye Vision systems currently in high-
21、end BMW,GM,Volvo models By 2010:70%of car manufacturers.4519.Google carsOct 9,2010.Google Cars Drive Themselves,in Traffic.The New York Times.John MarkoffJune 24,2011.Nevada state law paves the way for driverless cars.Financial Post.Christine DobbyAug 9,2011,Human error blamed after Googles driverle
22、ss car sparks five-vehicle crash.The Star(Toronto)4620.Interactive Games:KinectObject Recognition:http:/ Matrix movies,ESC Entertainment,XYZRGB,NRC21.Special effects:shape capture48Pirates of the Carribean,Industrial Light and Magic22.Special effects:motion capture49Computer Vision and Nearby Fields
23、l Computer Graphics:Models to Imagesl Comp.Photography:Images to Imagesl Computer Vision:Images to Models50Overview of Computer Vision Algorithm51So what do humans care about?Verification:is that a bus?slide by Fei Fei,Fergus&Torralba 52Detection:are there cars?slide by Fei Fei,Fergus&Torralba 53Ide
24、ntification:is that a picture of Mao?slide by Fei Fei,Fergus&Torralba 54Object categorizationskybuildingflagwallbannerbuscarsbusfacestreet lampslide by Fei Fei,Fergus&Torralba 55Scene and context categorization outdoor city traffic slide by Fei Fei,Fergus&Torralba 56Rough 3D layout,depth ordering57O
25、verview of Computer Vision Algorithml Image formationl Features l Grouping&fittingl Multi-view geometryl Recognition&learningl Motion&tracking581.Image formationHow does light in 3d world project to form 2d images?592.Features and filtersTransforming and describing images;textures,colors,edges603.Gr
26、ouping&fittingfig from Shi et alClustering,segmentation,fitting;what parts belong together?614.Multiple viewsHartley and ZissermanMulti-view geometry,matching,invariant features,stereo visionFei-Fei Li625.Recognition and learningRecognizing objects and categories,learning techniques636.Motion and tr
27、ackingTracking objects,video analysis,low level motion,optical flow64Challenges 1:view point variationMichelangelo 1475-156465Challenges 2:illuminationslide credit:S.Ullman66Challenges 3:occlusionMagritte,1957 67Challenges 4:scaleslide by Fei Fei,Fergus&Torralba 68Challenges 5:deformationXu,Beihong
28、194369Challenges 6:background clutterKlimt,191370Challenges 7:object intra-class variationslide by Fei-Fei,Fergus&Torralba 71Challenges 8:local ambiguityslide by Fei-Fei,Fergus&Torralba 72Challenges 9:the world behind the image 73Challenges 10:complexityThousands to millions of pixels in an image3,0
29、00-30,000 human recognizable object categories30+degrees of freedom in the pose of articulated objects(humans)Billions of images indexed by Google Image Search18 billion+prints produced from digital camera images in 2004295.5 million camera phones sold in 200574Keep Moving Ok,clearly the vision problem is deep and challenging time to give up?Active research area with exciting progress!75THANK YOUSUCCESS2022-10-29可编辑
