走进数据科学-英文版-课件.pptx

上传人(卖家):ziliao2023 文档编号:5672534 上传时间:2023-05-01 格式:PPTX 页数:92 大小:8.90MB
下载 相关 举报
走进数据科学-英文版-课件.pptx_第1页
第1页 / 共92页
走进数据科学-英文版-课件.pptx_第2页
第2页 / 共92页
走进数据科学-英文版-课件.pptx_第3页
第3页 / 共92页
走进数据科学-英文版-课件.pptx_第4页
第4页 / 共92页
走进数据科学-英文版-课件.pptx_第5页
第5页 / 共92页
点击查看更多>>
资源描述

1、Data Mining:Theory&AlgorithmsMining?Warehousing?5Technology Advancement6Technology Advancement7The World of Data8Data Rich,Information Poor910Learning Resources11International Conference on Data MiningInternational Conference on Data EngineeringInternational Conference on Machine LearningInternation

2、al Joint Conference on Artificial IntelligencePacific-Asia Conference on Knowledge Discovery and Data MiningACM SIGKDD Conference on Knowledge Discovery and Data MiningLearning Resources12Learning Resources13Xindong WuZhihua ZhouJiawei HanJian PeiQiang YangChih-Jen LinPhilip S.Yu Changshui ZhangLear

3、ning Resources14Interdisciplinary15Data MiningMachine LearningPattern RecognitionStatisticsArtificial IntelligenceUbiquitous16Data MiningBusiness IntelligenceData AnalyticsBig DataDecision SupportCustomer Relationship ManagementComprehensive Learning17Class Teaching Thinking DiscussionReading Materi

4、als Extension InspirationPractice Techniques ApplicationsLearning Listening1820DatavDefinition“Data are pieces of information that represent the qualitative or quantitative attributes of a variable or set of variables.Data are often viewed as the lowest level of abstraction from which information an

5、d knowledge are derived.”vData TypesContinuous,BinaryDiscrete,StringSymbolicvStoragePhysicalLogicalvMajor IssuesTransformationErrors and Corruption 21What is Big Data?v“Big data is high-volume,high-velocity and high-variety information assets that demand cost-effective,innovative forms of informatio

6、n processing for enhanced insight and decision making.”Gartnerv“Big data refers to datasets whose size is beyond the ability of typical database software tools to capture,store,manage,and analyze.”Mckinsey&Company22Big Data23Public Security24Health Care Application25Effectiveness ResearchPersonalize

7、d MedicineLocation Data:Urban Planning26Location Data:Mobile User27Location Data:Shopper28Retail Data:Targeted Marketing29Retail Data:Sentiment Analysis30Social Networks31Sports32Attractiveness Mining3334Open Datav Technically Open:available in a machine-readable standard format,which means it can b

8、e retrieved and meaningfully processed by a computer application.v Legally Open:explicitly licensed in a way that permits commercial and non-commercial use without restrictions.35Where to find data?36Open Government Data37Data Miningv People have been analysing and investigating data for centuries.v

9、 StatisticsMean,Variance,Correlation,Distribution v In modern days,data are often far beyond human comprehension.Diversity,Volume,Dimensionalityv DefinitionData Mining is the process of automatically extracting interesting and useful hidden patterns from usually massive,incomplete and noisy data.v N

10、ot a fully automatic processHuman interventions are often inevitable.Domain KnowledgeData Collection and Pre-processingv Synonym:Knowledge Discovery38v“If you are looking for a career where your services will be in high demand,you should find something where you provide a scarce,complementary servic

11、e to something that is getting ubiquitous and cheap.So whats getting ubiquitous and cheap?Data.And what is complementary to data?Analysis.So my recommendation is to take lots of courses about how to manipulate and analyze data:databases,machine learning,econometrics,statistics,visualization,and so o

12、n.”An interview with Google Chief Economist Hal Varian from the New York TimesIs DM really important?3940Business Intelligence41From Data To Intelligence42Decision ModelsData MiningPreprocessingDatabaseDecision SupportKnowledgeInformationDataData Integration&Analysis43The Process of Data Mining44454

13、647DM Techniques-Classificationv“Classification is a procedure in which individual items are placed into groups based on quantitative information on one or more characteristics(referred to as variables)and based on a training set of previously labeled items.”v Given a training set:(x1,y1),(xn,yn),pr

14、oduce a classifier(function)that maps any unknown object xi to its class label yi.v AlgorithmsDecision TreesK-Nearest NeighboursNeural NetworksSupport Vector Machinesv ApplicationsChurn PredictionMedical Diagnosis48XYClassification Boundaries49?XYIncomeSavingsLow RiskHigh RiskOverfitting Classificat

15、ion50Cross Validation51DataTraining SetTest SetEvaluationGenerated ModelsConfusion Matrix52Confusion Matrix Actual ValuePositiveNegativeTotalPredictedValuePositiveTruePositiveFalsePositivePNegativeFalseNegativeTrueNegativeNTotalPNTPR=TP/(TP+FN)TNR=TN/(TN+FP)Accuracy=(TP+TN)/(P+N)Receiver Operating C

16、haracteristic53Very small thresholdVery large thresholdRandom guessCost Sensitive Learning 54Lift Analysis 5556DM Techniques-Clusteringv“Clustering is the assignment of a set of observations into subsets(called clusters)so that observations in the same cluster are similar in some sense.”vDistance Me

17、tricsEuclidean DistanceManhattan DistanceMahalanobis DistancevAlgorithmsK-MeansSequential LeaderAffinity PropagationvApplicationsMarket ResearchImage SegmentationSocial Network Analysis57What is the difference between classification and clustering?Hierarchical Clustering58DM Techniques Association R

18、ule59Association Rule60Transaction IDMilkBreadButterBeer1110020110300014111050100ButterBread Milk,DM Techniques Regression61,XfY kkzkkxxzeyxxyxxyxy110110221010,11Overfitting Regression62yxSeeing is Knowing63Performance Dashboard6465Data PreprocessingvReal data are often surprisingly dirty.A Major Ch

19、allenge for Data MiningvTypical IssuesMissing Attribute ValuesDifferent Coding/Naming SchemesInfeasible ValuesInconsistent DataOutliersvData QualityAccuracyCompletenessConsistencyInterpretabilityCredibilityTimeliness66Data PreprocessingvData CleaningFill in missing values.Correct inconsistent data.I

20、dentify outliers and noisy data.vData IntegrationCombine data from different sources.vData TransformationNormalizationAggregationType ConversionvData ReductionFeature SelectionSampling6768Internet Privacy6970Privacy Protectionv Data:A Double-Edged Sword People can benefit greatly from data analysis.

21、The consequence of information leakage can be catastrophic.v People may be reluctant to give sensitive information due to privacy concerns.Drug,Tax,Sexuality v How to find out the percentage of people with a certain attribute?The interviewer should not know the true answer of each respondent.v Rando

22、mized Response Used in structured survey research.Can maintain the confidentiality of respondents.71Privacy Protectionv Two questions are presented:Q1:I have the attribute A.Q2:I do not have the attribute A.v The respondent uses a random device to:Answer Q1 with probability p.Answer Q2 with probabil

23、ity 1-p.The interviewer has no idea about which question is answered.72121)()()()1()()(*ppTruePTruePFalsePpTruePpTrueP5.0pCloud Computing73Cloud ComputingvPay As You GovSoftware as a Service vPlatform as a ServicevInfrastructure as a Service74Parallel Computing75Parallel Computing7677Mobile Supercom

24、puting78Intel MIC79The Big Picture80v Why bother so many different algorithms?v No algorithm is always superior to others.v No parameter setting is optimal over all problems.v Look for the best match between problem and algorithm.Experience Trial and Errorv Factors to consider:Applicability Computat

25、ional Complexity Interpretabilityv Always start with simple ones.No Free Lunch8182Just in Case Someone Asks 83Just in Case Someone Asks 84Grouping85XYGroup AGroup BViolent Crime vs.Video Game86Tricky?v Is there correlation between height and business success?v Average American Male is 59.v Only 3.9%adult American men are taller than 62”.v Around 30%CEOs of Fortune 500 are taller than 62.87Survivorship Bias88这是真的这是真的吗?吗?89时间去时间去哪儿了?哪儿了?90感谢您的聆听!

展开阅读全文
相关资源
猜你喜欢
相关搜索

当前位置:首页 > 办公、行业 > 各类PPT课件(模板)
版权提示 | 免责声明

1,本文(走进数据科学-英文版-课件.pptx)为本站会员(ziliao2023)主动上传,163文库仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对上载内容本身不做任何修改或编辑。
2,用户下载本文档,所消耗的文币(积分)将全额增加到上传者的账号。
3, 若此文所含内容侵犯了您的版权或隐私,请立即通知163文库(发送邮件至3464097650@qq.com或直接QQ联系客服),我们立即给予删除!


侵权处理QQ:3464097650--上传资料QQ:3464097650

【声明】本站为“文档C2C交易模式”,即用户上传的文档直接卖给(下载)用户,本站只是网络空间服务平台,本站所有原创文档下载所得归上传人所有,如您发现上传作品侵犯了您的版权,请立刻联系我们并提供证据,我们将在3个工作日内予以改正。


163文库-Www.163Wenku.Com |网站地图|