行人再识别研究前沿：评测数据构建与视觉表示学习-对外.pptx

1、Person Re-Identification:Datasets and Representation LearningPerson Re-Identification Associate the same individuals across multiple cameras= Multi-camera tracking2Person Re-Identification Associate the tracks for a single camera One person re-appears after a time Target lost due to occlusion3Applic

2、ationsImagedatabasereturnQueryTime: 9:00amPlace: 2ndBuildingPerson searchEvent/action detectionPerson counting by trackingBehavior analysis4Hot Topics Datasets Limited data scale/diversity Representations Variable images: Low quality, part misalignment, Matching Graph matching, Euclidean distance, A

3、ttributes Mid-level information: Long sleeve, short hair, Loss Pair wise loss, triplet loss, classification loss, 5Hot Topics Datasets Limited data scale/diversity Representations Variable images: Low quality, part misalignment, Matching Graph matching, Euclidean distance, Attributes Mid-level infor

4、mation: Long sleeve, short hair, Loss Pair wise loss, triplet loss, classification loss, 6Large-Scale Re-Identification DatasetsReal-World Setting Large scale Coverage/diverse Weather/season Scene Time8Image-Based ReIDCamera 2Camera 1Camera 3Camera 39Existing Image DatasetsPRID450SPRID2011DatasetsRA

5、iDCUHK03 CUHK02 CUHK01 WARDCAVIAR GRIDi-LIDSVIPeRtime#id2014450900020144320141,36013,164020131,8167,264020129711,9420201270201120011340201172200925050077582009119476020076321,2640#boxes#distractors#cam6,92004,7860610024225HandCMC232222labelHandCMCHandCMCDPMCMCHandCMCHandCMChandCMCHandCMCHandCMCHandC

6、MCHandCMCevaluationSmall scale10Market-1501 DPM detector, multi-queries multi-groundtruths for each person 3368 queries, 14.8 cross camera groundtruths11500K Distractor Images12Image DatasetsMarket-1501PRID450SPRID2011DatasetsRAiDCUHK03 CUHK02 CUHK01 WARDCAVIAR GRIDi-LIDSVIPeRtime#id20151,5012014450

7、900020144320141,36013,164020131,8167,264020129711,9420201270201120011340201172200925050077582009119476020076321,2640#boxes#distractors#cam32,6682,793+500k66,92004,7860610024225HandCMC232222labelDPMHandCMCHandCMCDPMCMCHandCMCHandCMChandCMCHandCMCHandCMCHandCMCHandCMCevaluationmAP+CMCEnlarge the datas

8、et13Image DatasetsDukeMTMC-ReIDMarket-1501PRID450SPRID2011DatasetsRAiDCUHK03 CUHK02 CUHK01 WARDCAVIAR GRIDi-LIDSVIPeRtime#id2017140420151,5012014450900020144320141,36013,164020131,8167,264020129711,9420201270201120011340201172200925050077582009119476020076321,2640#boxes#distractors#cam36,00340832,66

9、82,793+500k66,92004,78606100824225HandCMC232222labelHandmAP+CMCDPMHandCMCHandCMCDPMCMCHandCMCHandCMChandCMCHandCMCHandCMCHandCMCHandCMCevaluationmAP+CMCEnlarge the dataset14Video-Based ReIDCamera 2Camera 1Camera 3Camera 315Existing Video DatasetsDatasetsyeariLIDS-VID2014300PRID-201120112003DPES20112

10、00ETHZ20071461468,5800#id#tracklets#Boxes#distractors#cam6004001,000200k043,800040,00002281LabelHandCMCHandCMCHandCMCHandCMCEvaluationSmall scale16MARS Extension of Market-1501; 2009 queries, 3.7 groundgruths17Video DatasetsDatasetsyearMARS2016iLIDS-VID2014300PRID-201120112003DPES2011200ETHZ20071461

11、468,5800#id1,261#tracklets#Boxes#distractors#cam20,7156004001,000200k01,067,5163,24843,800040,000062281LabelDPM+GMMCPmAP+CMCHandCMCHandCMCHandCMCHandCMCEvaluationEnlarge the dataset18Leaderboard500+ citations19Leaderboard170+ citations20Real-World Setting Large scale #Identities #Box for each person

12、 #Cameras Coverage/diverse Weather/season Scene Time21Par t-Aligned Representation LearningSolutions Biometric Face Frontal? Enough resolution?Solutions Biometric Face Silhouette extraction is not Solutions Appearance Clothing color, texture Deep learningCNNGlobalGlobalGlobalUpper body &bottom segme

13、ntationStripesGridGlobalSpatially localGlobalUpper body &bottom segmentationStripesGridGlobalSpatially localGlobalGlobalPart segmentationPart detectionSemantically localGlobalGlobalPart segmentationPart detectionSemantically localOur Motivation Comparison on the same spatial position is not reliable

14、31Our Motivation Comparison on the same spatial position is not reliable Human re-identifies persons by comparing body par ts E.g., arm, bottom, top, 32Deeply-Learned Par t-Aligned Representations An end-to-end solution to Learn the discriminative parts (attention maps) w/o part annotation Weak supe

15、rvision: person matching as the target33PipelinePar t featureextractorAttentionPar t featureextractorAttentionCNN34PipelinePar t featureextractorAttentionPar t featureextractorAttentionAttentionCNNxPar t Optimization Loss Rank loss: Triplet hinge loss+ x Par ts are Well Aligned38Par ts are Well Alig

16、ned39The Effect of #Par ts41The Effect of Attention42Comparison to Spatial Par tition43Stable Improvement over Different Base Nets44Weakly-supervised part-aligned representation learningleads to accurate matching between person images45Pose-Guided Par t-Aligned RepresentationAttention maps Attention

17、 Weak-supervision w/o poseChi Su, Jianing Li, Shiliang Zhang, Junliang Xing, Wen Gao, Qi Tian: Pose-Driven Deep Convolutional Model for Person Re-identification.ICCV 2017: 3980-3989Yumin Suh, Jingdong Wang, Siyu Tang, Tao Mei, Kyoung MuLee: Part-Aligned Bilinear Representations for Person Re-Longhui

18、 Wei, Shiliang Zhang, Hantao Yao, Wen Gao, Qi Tian: GLAD: Global-Local-Alignment Descriptor for Pedestrian Retrieval. ACMMultimedia 2017: 420-428identification. ECCV 201849Liang Zheng, Yujia Huang, Huchuan Lu, Yi Yang: Pose Invariant Embedding for Deep Person Re-identification. CoRR abs/1701.07732 (

19、2017)Pose-Guided Par t-Aligned RepresentationAttention maps Attention Weak-supervision w/o pose Pose-guidedPar t boxesChi Su, Jianing Li, Shiliang Zhang, Junliang Xing, Wen Gao, Qi Tian: Pose-Driven Deep Convolutional Model for Person Re-identification.ICCV 2017: 3980-3989Yumin Suh, Jingdong Wang, S

20、iyu Tang, Tao Mei, Kyoung MuLee: Part-Aligned Bilinear Representations for Person Re-Longhui Wei, Shiliang Zhang, Hantao Yao, Wen Gao, Qi Tian: GLAD: Global-Local-Alignment Descriptor for Pedestrian Retrieval. ACMMultimedia 2017: 420-428identification. ECCV 201850Liang Zheng, Yujia Huang, Huchuan Lu

21、, Yi Yang: Pose Invariant Embedding for Deep Person Re-identification. CoRR abs/1701.07732 (2017)Pose-Guided Par t-Aligned RepresentationAttention maps Attention Weak-supervision w/o pose Pose-guided par t mapsPar t boxesOurs: Par t mapsChi Su, Jianing Li, Shiliang Zhang, Junliang Xing, Wen Gao, Qi

22、Tian: Pose-Driven Deep Convolutional Model for Person Re-identification.ICCV 2017: 3980-3989Yumin Suh, Jingdong Wang, Siyu Tang, Tao Mei, Kyoung MuLee: Part-Aligned Bilinear Representations for Person Re-Longhui Wei, Shiliang Zhang, Hantao Yao, Wen Gao, Qi Tian: GLAD: Global-Local-Alignment Descript

23、or for Pedestrian Retrieval. ACMMultimedia 2017: 420-428identification. ECCV 201851Liang Zheng, Yujia Huang, Huchuan Lu, Yi Yang: Pose Invariant Embedding for Deep Person Re-identification. CoRR abs/1701.07732 (2017)Par t featureextractorAttentionAttentionPar t featureextractorCNNxPar t featureextra

24、ctorAttention53A tionAttentionAttentionCNN54A tionAttentionAttentionCNNx55PoseAppearance (CNN)xPose estimator pretrained on COCO, and re-trained only with the re-id lossAppearance Descriptors Clustered by Colors57Par t Descriptors Clustered by Body Par ts58Appearance Descriptors are Reliable59Par t

25、Descriptors are Reliable60Effectiveness of Par t Descriptors on Various datasets61Effectiveness of Par t Descriptors on Various Networks62Market-150163Market-1501 + 500K64CUHK65DukeMTMC66Video: Mars67Weakly-supervised pose-guided part-alignedrepresentation learning leads to accurate matchingbetween

26、person images68Summar yLargeLargew/o posew/ poseScan QR codes to download datasets and Collaborators70Full Publications1 Learning Correspondence Structures for Person Re-identification. Weiyao Lin, Yang Shen, Junchi Yan, Mingliang Xu, Jianxin Wu, Jingdong Wang, andKe Lu. IEEE Transactions on Image P

27、rocessing (TIP).2 Exemplar-Guided Similarity Learning on Polynomial Feature Map for Person Re-Identification. Dapeng Chen, Zejian Yuan, Jingdong Wang, Gang Hua,and Nanning Zheng. International Journal of Computer Vision (IJCV).3 Part-Aligned Bilinear Representations for Person Re-identification. Yum

28、in Suh, Jingdong Wang, Siyu Tang, Ta o Mei, and Kyoung Mu Lee. ECCV 2018.4 Group Re-Identification: Leveraging and Integrating Multi-Grain Information. Hao Xiao, Weiyao Lin, Bin Sheng, Ke Lu, Junchi Yan, Jingdong Wang,Errui Ding, Yihao Zhang, and Hongkai Xiong. ACM MM 2018.5 Deeply-Learned Part-Alig

29、ned Representations for Person Re-Identification. Liming Zhao, Xi Li, Yueting Zhuang, and Jingdong Wang. ICCV 2017.6 MARS: A Video Benchmark for Large-Scale Person Re-identification. Liang Zheng, Zhi Bie, Yifan Sun, Jingdong Wang, Chi Su, Shengjin Wang, and QiTian. ECCV 2016.7 Scalable Person Re-ide

30、ntification: A Benchmark. Liang Zheng, Liyue Sheng, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. ICCV 2015.8 Person Re-identification with Correspondence Structure Learning. Yang Shen, Weiyao Lin, Junchi Yan, Mingliang Xu, Jianxin Wu, and Jingdong Wang.The Fourteenth IEEE International Confer

31、ence on Computer Vision ICCV 2015.9 Similarity Learning on an Explicit Polynomial Kernel Feature Map for Person Re-Identification. Dapeng Chen, Zejian Yuan, Gang Hua, Nanning Zheng,and Jingdong Wang. IEEE Conference on Computer Vision and Pattern Recognition CVPR15.71Highlighted Projects and Codes72Thanks!QA74