1、ArcSight CorrelationFabian LibeauSuperpan 翻译翻译QQ:28797575ArcSight ESMArcSight ESM作为一款应对安全风险、合规要求和内部威胁的企业安全管理系统，ArcSight ESM（Enterprise Security Management）能够集中展示企业信息安全各方面的概况，同时还提供有实时监视和事件关联、风险分析、深入调查功能、报告、通知以及其他安全管理功能，可在企业范围内全面管理、审计安全事务。2005 ArcSight Confidential2ArcSight ESM强大的事件收集能力和跨设备的事件分类能力强大的事
2、件收集能力和跨设备的事件分类能力 ArcSight ESM实现了实时数据格式标准化，超过260种默认支持的设备，对每一种事件都进行了详尽的分类，以帮助管理员理解事件的含义，并进行跨设备的分析。最为智能和灵活的关联分析最为智能和灵活的关联分析ArcSight ESM提供实时的、内存内(In-Memory)关联分析，具有106种预置关联规则，图形化规则编辑，支持资产分类、漏洞状态与企业策略与风险管理目标的关联。直观的调查分析和合规性报表直观的调查分析和合规性报表ArcSight ESM具有169个可重用、图形化数据监视模块，自由定义的仪表板（预置41个），灵活的报表格式，提供图形化报表编辑器，提供
3、预先打包的合规解决方案。2005 ArcSight Confidential3ArcSight ESM完善的自动安全响应能力完善的自动安全响应能力ArcSight ESM可与安全设备共同协作来关闭威胁通信，以阻止正在进行的攻击，提供威胁升级和工单处理功能。智能存储智能存储ArcSight ESM集成了数据监控、备份脚本、分区管理等等一系列的数据库维护工具，提供综合安全生命周期信息管理（SLIM）策略，利用自动的高度压缩、存档和恢复系统以减少存储长期安全事件所需费用。2005 ArcSight Confidential4ArcSight ESM 2005 ArcSight Confidentia
4、l5SOC中日志关联分析的核心技术中日志关联分析的核心技术SIM/SEM/SIEM/SOC的日志关联分析核心技术主要集中在：日志收集、格式化、事件映射、关联四个方面。日志收集：一个SIM产品是否有优势，就要看日志收集能否支持更多的设备日志类型，能否容易扩展，自动识别支持未知设备日志。例如需要支持的协议有syslog、snmp trap、windows log、checkpoint opsec、database、file、xml、soap等等。格式化：日志收集来了，需要格式化统一标准，为后面的关联，事件映射做准备，如果格式化不够标准，后面也不好做。事件映射：将日志需要统一映射成一个标准，提供统一
5、的解决方案，这个难度也比较大，各个厂家设备的日志名称，类型，含义都不相同，如果统一映射，是个难题。关联分析：这个是SIM的核心部分，例如ArcSight提供了简单的事件关联、上下文关联、攻击场景关联、低慢攻击关联、位置关联、身份关联、角色关联等等。关联分析还有脆弱性信息关联、因果关联、推理关联等等。关键问题是如何利用这些技术，给用户提供一个很好的SIM/SEM/SIEM/SOC系统，也是一个难题。2005 ArcSight Confidential6 2005 ArcSight Confidential7AgendaArchitectural Overview概述ArcSight Risk P
6、rioritization风险的优先顺序ArcSight different ways of correlating information不同的关联分析方法 Rule based correlation基于规则 Statistical correlation统计相关性分析 Pattern discovery(advanced predictive DataMining)模式发现（先进的预测数据挖掘）ArcSight Key Concepts 2005 ArcSight Confidential8VulnerabilityAssessment漏洞评估Architectural Overview
7、架构概述架构概述ConsoleDatabaseArcSightManagerAsset Management资产管理资产管理XMLWindowsWindows SystemsUnix/Linux/AIXUnix/Linux/AIX/Solaris/SolarisSecuritySecurityDeviceDevice安全设备安全设备SecuritySecurityDeviceDeviceDatabaseManagementSystemsSyslogConcentratorConcentrator集中器集中器MainframeMainframe&Apps&Apps主机和应用主机和应用Securi
8、tySecurityDeviceDeviceData FlowsData Flows数据流数据流 2005 ArcSight Confidential9ArcSight SmartAgent Overview智能代理智能代理Largest number of supported devices 150+100%Data CaptureIntelligent Event Capture智能事件捕获 Normalization One format规范化-统一格式化 Categorization Grouping similar events分类-分组类似事件 Aggregation Event
9、redundancy(50-80%for firewalls and routers)聚集-事件冗余（50-80的防火墙和路由器）Filtering Transfer and store only what you need过滤转移和存储您所需要的 Secure,configurable and governed安全，配置和管辖的FlexAgents new SmartAgents in hours在几个小时FlexAgents 新CounterAct Agents automated remediation抵制代理-自动修复Flexible Data Collection Centraliz
10、ed or Distributed灵活的数据收集-集中式或分布式Flexible Collection灵活采集灵活采集CounterActSmartAgentFlexAgent 2005 ArcSight Confidential10ArcSight SmartAgent-Event Normalization and Categorization事件规范化和分类事件规范化和分类Jun 01 2005 00:00:12:%PIX-3-106011:Deny inbound(No xlate)udp src outside:10.50.215.97/6346 dst outside:204.11
11、0.228.254/6346Jun 01 2005 00:00:12:%PIX-6-305011:Built dynamic TCP translation from isp:10.50.107.51/1967 to outside:204.110.228.254/62013Jun 01 2005 00:00:12:%PIX-6-302013:Built outbound TCP connection 2044303174 for outside:213.189.13.17/80(213.189.13.17/80)to isp:10.50.107.51/1967(204.110.228.254
12、/62013)Jun 02 2005 12:16:03:%PIX-6-106015:Deny TCP(no connection)from 10.50.215.102/15605 to 204.110.227.16/443 flags FIN ACK on interface outsideSample Raw Pix Events:Jun 02 2005 12:16:03:%PIX-6-106015:Deny TCP(no connection)from 10.50.215.102/15605 to 204.110.227.16/443 flags FIN ACK on interface
13、outsideArcsight Categorization:Arcsight Normalization:2005 ArcSight Confidential11ArcSight SmartAgent Guaranteed Delivery智能代理智能代理保证交付保证交付AnalystArcSightManagerPort 8443Cache缓存缓存FailoverManager(optional)故障转移管理器（可选）ArcSightEventArcSightEventCompressedEventSSLContentUpdates 2005 ArcSight Confidential12
14、The ArcSight Manager-OverviewReal-Time,In-Memory Correlation实时内存关联Real-time Dashboards实时仪表盘Anomaly Detection异常检测 Correlation Rules-known behaviors关联规则 已知行为 Pattern Discovery undiscovered patterns模式发现-未被发现的模式 Flow Rates deviations from the norm流量速率-标准差 基线偏差Asset Linkage资产联动Priority Scoring优先评分 Vulner
15、ability漏洞 Asset Value资产价值 Severity严重性Alerts,among other configurable actions其他配置的行动警告Scalability and High Availability Options可扩展性和高可用性选项Intelligent Processing智能处理ManagerLINUX,Windows,UNIX,Macintosh 2005 ArcSight Confidential13AgendaArchitectural OverviewArcSight Risk Prioritization风险的优先ArcSight dif
16、ferent ways of correlating information Rule based correlation Statistical correlation Pattern discovery(advanced predictive DataMining)ArcSight Key Concepts 2005 ArcSight Confidential14ArcSight Risk Correlation风险相关性风险相关性EventsScansCorrelationDevicesPrioritizationWhats happening?Whatstargeted?Whatmat
17、ters?Whats vulnerable?漏洞、漏洞、脆弱脆弱=False Alarm or Normal虚假报警或普通事件=Prioritized Red Alarm优先红色警报Dynamic Threat Severity Index动态威胁的严重程度指数Profiled Asset异常资产异常资产Confirmed Vulnerability已确认的漏洞Weighting Algorithms加权算法+Detected Event检测事件检测事件ArcSight fuses all key event sources and related inputs to rank event s
18、ignificance on multiple variables 所有关键的事件源和多变量等级事件 2005 ArcSight Confidential15Asset Linkage and Priority Scoring-Overview资产联动和优先评分资产联动和优先评分-概述概述Windows SystemsUnix/Linux/AIX/SolarisSecurityDeviceSecurityDeviceMainframe&AppsSecurityDevicePrioritization and Imported Scanned Assets资产的优先顺序和导入扫描的资产Smart
19、AgentsArcSightEventArcSightManagerTMArcSight Prioritized Event事件优先权事件优先权VulnerabilityScanner漏洞扫描漏洞扫描SmartAgentsAsset Information建模的程度（信心）建模的程度（信心）Model ConfidenceHas asset been scanned for open ports and vulnerabilities?关联RelevanceAre ports open on asset?Is it vulnerable?Severity严重性Is there a histor
20、y withthis attacker or target(active lists)?资产重要性资产重要性Asset CriticalityHow important is thisasset to the business?代理严重性Agent SeverityMapping of reportingdevice severity toArcSight severity 2005 ArcSight Confidential16Asset Linkage and Priority Scoring Information Flow资产联动和优先评分资产联动和优先评分-信息流信息流Vulnera
21、bility Assessment漏洞评估 Three dimensional correlation of assets,events and vulnerabilities Allows organizations to apply SIM to risk management Minimizes dead end investigations Information seamlessly linked within the ArcSight system三二维相关的资产，事件和漏洞 允许企业申请SIM卡风险管理 最大限度地减少死胡同调查 无缝链接的信息系统内的ArcSightArcSig
22、ht ManagerAssets-Compliance Requirement-Business Role-Application-Operating System-Data roleCriticality资产重要资产重要性性Vulnerabilities-Zones区ArcSightEventEvent CVEEvent Severity事件等级事件等级Priority Score Relevance 2005 ArcSight Confidential17Threat Priority Variables Considered威胁优先威胁优先 多种关系组合考虑多种关系组合考虑Model C
23、onfidence:How well does ArcSight know this asset?Has it been scanned?Options:0=Asset is not modeled没有建模 4=Asset has not been scanned for open ports or vulnerabilities 没有扫描端口或漏洞 8=Asset has been scanned for open ports or vulnerabilities,but not for both扫描端口或漏洞其一10=Asset is scanned for both open ports
24、 and vulnerabilities扫描端口和漏洞Relevance:Is the port open,and has a vulnerability been exploited利用?Options:5=Assets target port is open.5=Event will exploit a know asset vulnerabilitySeverity:Is there a history with this attacker or target(Active Lists)?Options:5=Hostile List 3=Compromised 3=Suspicious
25、List 1=Reconnaissance List 5=敌对目录 3=不受影响 折中 3=可疑名单 1=侦察名单The Priority of an event is theAgent Severity adjusted by:Model Confidence Relevance、Severity、Asset Criticality一个事件的优先事项是代理严重性调整：模式的信心、关联、严重性、资产重要性Asset Criticality:资产重要性资产重要性How critical have I rated this asset within my organization.Options:
26、10=Very High Criticality Assets非常高 8=High Criticality Assets高 6=Medium Criticality Assets中 4=Low Criticality Assets低 2=Very Low Criticality Assets非常低 0=Unknown Criticality Assets未知Agent Severity:Mapping of reporting device severity to ArcSight severity.代理严重性：报告设备严重性到ArcSight的严重性的映射。2005 ArcSight Con
27、fidential181.Relevance drags down the Agent Severity.相关性相关性Example:If Relevance=0,the Priority=0If Relevance=10,the Priority=Agent Severity2.Model Confidence tempers the effect of relevance on priority.建模程度Example:If Model Confidence=0,Relevance has no effect on PriorityIf Model Confidence=10,Priori
28、ty acts the way specified above3.Formulae for the multiplication factor contributed by Model Confidence(M)and Relevance(R)R =(R+M-R*M/10)4.If Severity(S)=10 it adds up to 30%to Agent Severity to provide Priority:(1+S*3/100)5.Criticality applies a boost to Agent Severity by 20%if =(Very High)10;does
29、nothing if Criticality=(High)8;and applies a decrement/drag if the Criticality is Medium/Low/Unknown(6/4/2):(1+(Criticality-8)/10)Threat Priority The Formula威胁优先级的公式威胁优先级的公式 2005 ArcSight Confidential19Heuristic:Formula-Based启发式：按公式计算启发式：按公式计算Threat level formulaPrioritizes incident investigation an
30、d responseSums up complex information from the network model 威胁级别的公式事故调查和应对的优先顺序汇总了网络模型的复杂信息C:arcsightManagerconfigserverThreatLevelFormula.xml 2005 ArcSight Confidential20Priority Calculation Exercise优先级的计算练习优先级的计算练习Steps Exercise Agent Severity=Low Priority=4 Asset Criticality is 0=20%decrease in
31、priority.Priority=3.2Severity=0,no effect on priority.2005 ArcSight Confidential21Priority is adjusted by Criticality通过重要性调整优先级通过重要性调整优先级Combined factor for model confidence and relevance,lets call it MCR=MCR is calculated using the formula R*10 MCR=(R+M-R*M/10)where R(Relavance)=5,M(Model Confidenc
32、e)=4 MCR=7=30%drop in priority again.New Priority=3.2*0.7=2.24 rounded off gives a 2.The Final Priority is-because of low values for criticality and relevance your final priority of the event came down from 4 to 2.2005 ArcSight Confidential22AgendaArchitectural OverviewArcSight Risk PrioritizationAr
33、cSight different ways of correlating information Rule based correlation基于规则的关联 Statistical correlation Pattern discovery(advanced predictive DataMining)ArcSight Key Concepts 2005 ArcSight Confidential23Rule based correlation基于规则的关联基于规则的关联Fast memory based algorithm,based on RETE 2(http:/ in Correlat
34、ion:整合的相关性 Events事件 Vulnerability Information漏洞信息 Active Lists(dynamic list with e.g.Asset/User information)活动列表（如与动态列表资产/用户信息 Asset Categories(see later slides)资产类别（见稍后幻灯片）Asset Zones(IP ranges)资产区（IP范围）Asset Networks(IP networks/groups of Asset Zones)资产网络（IP网络资产区/组）Results earlier rule based corre
35、lation早期规则为基础的相关性 Results earlier statistical correlation早期统计（静态）为基础的相关性 2005 ArcSight Confidential24Rules Theory规则理论规则理论1.Simple Aggregation Single event type or categoryBasic conditionsDe-duplication简单-聚合单事件类型或类别基本条件重复数据删除 targetspinge.g.,any source repetitively profiling targetsarcsight_category
36、startsWith/recontarget_address inSubnetgroupBy source_address2 or more matching events in 1 minutesource2.Complex Correlation Multi-Event JoinMultiple event types or categoriesBoolean conditionsComplete session or“round trip”复杂的关系-多事件加入多个事件类型或类别布尔条件完整会话或“来回”targetse.g.,any source successfully engagi
37、ng a target arcsight_category startsWith/attack target_address inSubnet groupBy source_address,target_address 1+matching events in 1 minute join events across IDS,firewall,and host3.Complex Long SequenceMultiple sessionsPre-attack probes,attack formation/progression,and attack conclusionHandles long
38、-term memory need using active lists 复杂鈥长序列多个会话、预探测攻击，攻击编队/进程，攻击结束处理长期记忆需要使用活动列表attackFWIDSe.g.,low&slow attack pattern across multiple rules/recon rule records source_address suspicious/attack rule upgrades source_address to hostileand records target_address as compromised Final rule looks for evid
39、ence of successrule1activelistactivelistrule2rule3sourceRule Types By Complexity复杂规则类型Example例子Approach方法途径Catch and accumulate events in real-time in memory-Good for event bursts在内存中捕获和累积事件良好的突发事件Catch and correlate events in real-time in memory until the rule chain is complete-Good for cross-event
40、 matching that occurs in a single session在内存中捕获和累积事件直到完成该规则链-良好的交叉配对活动，在单个会话发生Break up sequences in logical segments and maintain active lists in the database that tie together multiple rules-Good for long elapsed time attack sequences that start and stop across multiple sessions打破序列逻辑段，保持积极的数据库列出了多
41、个规则联系在一起-经过好长的时间序列，开始攻击和跨多个会话停止 2005 ArcSight Confidential25Simple Correlation:Event Aggregation简单的相关性：事件聚集简单的相关性：事件聚集Most basic correlation最基础的关联De-duplicates events (many-to-one)去重Single source,single target单一源单一目标Flatten event bursts压扁事件爆发ArcSight SmartAgents do this too!CorrelationSingle EventMu
42、ltiple Events(same base event)As above plusDistributed attack sources分布攻击源Multiple attack targets多攻击目标Any field or combination of event fields(types of event)人行事件领域（事件类型的组合）Interrelates diverse events不同的事件相互联系CorrelationSingle EventMultiple Events(multiple event types,sources and/or targets)2005 Arc
43、Sight Confidential26Simple Correlation:Event Aggregation简单的相关性：事件聚集简单的相关性：事件聚集Most basic correlation最基础的关联De-duplicates events (many-to-one)去重Single source,single target单一源单一目标Flatten event bursts压扁事件爆发ArcSight SmartAgents do this too!CorrelationSingle EventMultiple Events(same base event)As above p
44、lusDistributed attack sources分布攻击源Multiple attack targets多攻击目标Any field or combination of event fields(types of event)人行事件领域（事件类型的组合）Interrelates diverse events不同的事件相互联系CorrelationSingle EventMultiple Events(multiple event types,sources and/or targets)2005 ArcSight Confidential27Advanced Correlation
45、:Multi-event Joins高级的相关性：多事件加人高级的相关性：多事件加人Inter-relates(joins)diverse events with any combination of common field values e.g.,source IP,target IP,port,protocol,username,domain,location,zone etc分析不同事件的相互联系，with事件通用属性：例如，源IP，目标IP，端口，协议，用户名，域，位置，区域等Compare any event fields using flexible boolean logic(
46、AND,OR,NOT)比较任意事件字段采用比较灵活的布尔逻辑（与，或，非）Good for cross-event matching of complete end-to-end sessions良好的跨事件的完整的端至端会话匹配E.g.correlating an attacker detected by NIDS,crossing the firewall,compromising a host,creating a back connection to steal confidential dataCorrelationSingle EventMultiple Events with C
47、ommon Event Fields(different base events)在事件通用属性上分析多事在事件通用属性上分析多事件件 2005 ArcSight Confidential28Complex Correlation:Attack State Monitoring复杂的相关性：攻击状态监测复杂的相关性：攻击状态监测Inter-relates events across sessions using Active Lists使用活动列表分析跨多会话事件Any field or combination of event fields may be persisted from bas
48、e events任何字段或字段组合的事件可能会从基本事件提炼Long&short-term state machines长期与短期的状态机Good for tracking logical sequences of events良好的跟踪事件的逻辑顺序E.g.Reconnaissance,attack formation,progression&conclusion例如侦察，攻击形成，进展及结论CorrelationEvent Sequence 1(multi-event joins)Record on Active List(state 1)CorrelationEvent Sequence
49、 2Event Sequence 3CorrelationRecord on Active List(state 2)Single Event 2005 ArcSight Confidential29(2)(1)Rule based Cross-Correlation基于规则的交叉关联分析基于规则的交叉关联分析Scenario 1 The attacker is unsuccessful and alarms are false positives方案1-攻击不成功和报警器误报HN-IDSIDS reports WEB-IIS ISAPI.printer access to 209.128.9
50、8.148ArcSight categorizes the signature as/Attack/and recognizes thatthe target is hosting Mission Critical ApplicationsArcSight correlates and fires the 1st rule Yellow Alarm:/Attack Started/Perimeter Alarm/Mission Critical Asset(Warning_Display)The source IP address is quietly recorded as suspicio
