preface


In the past, we have done some research on Threat Intelligence , This paper includes some existing research results ( In case of infringement, please contact to delete it ) And personal opinions . actually , Threat Intelligence was put forward a few years ago , At home and abroad also slowly have a certain development , Many security companies have begun to build their own threat intelligence platforms , For a while, I don't seem to do anything related to threat intelligence , It will become obsolete . Especially from WannaCry After the incident , People began to question the existing defense system , And threat intelligence has been given high hopes . This paper attempts to make a comprehensive overview of threat intelligence , Let more people know about participation .

One , essential information

1.1 definition


At present, there is no unified definition of Threat Intelligence in both industry and academia , Many organizations or papers have described the concept of threat intelligence , At present, it is widely accepted that Gartner stay 2014 Published in 《 Market guide for security threat intelligence services 》(Market
Guide for Security Threat Intelligence Service) The definition proposed in , Namely :

Threat Intelligence is about IT Evidence based knowledge of existing or potential threats to information assets , Including context , mechanism , index , Inferences and feasible suggestions , These knowledge can provide decision-making basis for threat response .

1.2 classification


According to different standards, Threat Intelligence can be classified in many different ways , First, according to the data itself, Threat Intelligence can be divided into HASH value ,IP address , domain name , Network or host characteristics ,TTPs(Tactics,Techniques
& Procedures) These kinds of , It comes from David J. Bianco stay 《The Pyramid of
Pain》 The related index of Threat Intelligence proposed in this paper ( A single piece of information or data is generally not a threat intelligence , Only valuable information that has been analyzed and processed can be called Threat Intelligence ) Pyramid model of .




On the left is information that can be used , On the right is the degree of difficulty the information has caused the attacker . Generally speaking, the lowest value of intelligence is Hash value ,IP Address and domain name ( That is to say, the reputation database ), The second is the network / Host characteristics , Attack tool features , The biggest impact on attackers is TTPs( tactics , Technology and behavior patterns ) Type of Threat Intelligence . Here is a brief introduction :

HASH value
: Generally refers to the sample , Document's HASH value , such as MD5 and SHA series . because HASH Avalanche effect of function , Any slight change in the document , A completely unrelated value will result in a different hash . This makes in many cases , It becomes unworthy of tracking , So it brings the lowest defense effect .

IP address : One of the common indicators , adopt IP Access control can resist many common attacks , But because IP The quantity is too large , Any attacker can attempt to change IP address , To bypass access control .

domain name
: Some types of attacks or tactics are also for hidden purposes , The attacker can connect to the external server through the domain name for indirect communication , Because the domain name needs to be purchased , register , The relative cost of binding to the server IP It is relatively high , The defense effect of domain name control is also good . But for advanced APT A mass attack or gang attack , A large number of alternate domain names will be prepared , So its limiting effect is also limited .

Network or host characteristics
: There are many aspects to be referred to here , For example, the attacker browser's User-Agent, Login user name , Frequency of visits, etc , These features are a description of the attacker , These intelligence data can well extract attack traffic from other traffic , It will produce a better defense effect .

Attack tools : This refers to the acquisition or detection of the tool used by the attacker , This tool based intelligence data can disable a number of attacks , The attacker has to perform kill free or rewrite tools , This achieves the purpose of increasing the attack cost .

TTPs:Tactics,Techniques &
Procedures Abbreviation for , Here is the attack strategy used by the attacker , Technique, etc , With some information, you can understand the specific vulnerabilities exploited by the attacker , Can be targeted deployment , The attacker has to look for new vulnerabilities , So it's also the most valuable intelligence data .

Another classification method is by use scenario , Intelligence can be divided into 3 class : Tactical intelligence based on automatic detection and analysis , Operational intelligence for security response analysis , And strategic intelligence to guide the overall security investment strategy .

Tactical intelligence : The main function of tactical intelligence is to detect threat events and to confirm or prioritize alarms . Common lost detection information (CnC
cutting-edge news , That is, the remote command and control server intelligence used by the attacker to control the victim ),IP Intelligence belongs to this category , They're all machine readable information , It can be directly used by the equipment , Automatic completion of the above safety work .

Operational intelligence
: Operational intelligence is used by security analysts or security incident responders , The purpose is to analyze the known important security events ( Alarm acknowledgement , Scope of attack , Attack chain and attack purpose , Technical and tactical methods, etc ) Or use the known attacker's techniques and tactics to actively find out the attack related clues .

Strategic Intelligence
: Strategic threat intelligence is used by the organization's security managers , such as CSO. It can help decision makers grasp the current security situation , It is more reasonable in safety decision-making . What kind of organizations will attack , What are the possible hazards of an attack , The attacker's tactical ability and control of resources , Of course, specific attack examples will also be included .

1.3 life cycle

Gartner think , Intelligence is the product of process , But not the aggregation of independent data points .Gartner The life cycle of threat intelligence is described :



The specific meanings are as follows :

* directional : Define goals and improve
* collect : Collect data from multiple open or closed sources ; Electronic , Artificial
* handle : If necessary , translate ; Reliability assessment ; Check multiple sources
* analysis : Judge the meaning of this information ; Evaluate the importance of information ; Recommend corresponding measures
* transmit : Pass information on to customers
* feedback : Adjust according to demand
1.4 significance


The traditional defense mechanism is based on the “ experience ” Build defense strategy , Deploy security products , Difficult to deal with unknown attacks ; Even the detection algorithm based on machine learning is still in the past “ experience ”( Training set ) To find the best general expression , To cover all possible scenarios , Realize the detection of unknown attack . But past experience cannot fully express the present and future security situation , And there are many ways to attack , In essence, the development speed of defense technology lags behind that of attack technology . Therefore, we need a way to dynamically adjust the defense strategy according to the past and current network security situation , Threat Intelligence came into being . Through the collection of threat information , Processing can directly distribute the corresponding results to security personnel ( Recognition and reading ) And safety equipment ( Machine readable ), Achieve accurate dynamic defense , achieve “ Defense before attack ” The effect of .

Two , Standards and specifications


With the development of threat information industry , So that its use needs a set of standards and norms for agreement , At present, the mature foreign threat intelligence standards include network observable expression (CyboX), Structured threat information expression (StructuredThreatInformationeXpression,STIX), Trusted automatic exchange of index information (TrustedAutomatedeXchangeofIndicatorInformation,TAXII) And lightweight exchange hosting events (Malware
Attribute Enumeration and
Characterization,MILE) etc. . The United States has done a lot in this area , The design and release of many standards are supported by government departments .

2.1 CyboX

Cyber Observable eXpression (CybOX) <https://cybox.mitre.org/>
  The specification defines a method to represent the dynamics and entities of computer observable objects and networks . Observable objects include files ,HTTP conversation ,X509 certificate , System configuration items, etc .CybOX
The specification provides a standard and extensible syntax , It is used to describe all the things we can observe from the computing system and operation . In some cases , Observable objects can be used as indicators to judge threats , such as Windows Of RegistryKey. This observable object has a certain value , It is often used as an indicator to judge whether the threat exists or not .IP The address is also an observable object , It is usually used as an indicator of malicious attempt .



2.2 STIX

Structured Threat Information eXpression (STIX) <https://stix.mitre.org/>
  Provides standards based XML The method of describing the details of Threat Intelligence and threat content based on the syntax of .STIX Support the use of CybOX Format to describe most of them STIX What grammar itself can describe , of course ,STIX Other formats are also supported . Standardization will greatly improve the efficiency and accuracy of threat information exchange among security researchers , Greatly reduce misunderstanding in communication , It also automates the processing of certain threat intelligence . It has been proved by practice ,STIX The specification can describe various characteristics of threat intelligence , Including threats , Threat activities , Safety accidents, etc . It makes the most of it DHS Specification to specify each STIX The format of the data items contained in the entity .



2.3 TAXII

Trusted Automated eXchange of Indicator Information (TAXII)
<https://taxii.mitre.org/>
  Provide secure transmission and exchange of threat intelligence information . Many articles make people think that TAXII Can only be transmitted TAXII Format data , But in fact, it supports multiple formats to transmit data . The current common practice is to use TAXII To transmit data , use STIX For information description .TAXII The exchange protocol is defined in the terms of standardized service and information exchange , It can support multiple shared models , include hub-and-spoke,peer-to-peer,subscription.TAXII While providing secure transmission , There is no need to consider topological structures , Trust issues , Authorization management and other strategies , Leave it to higher-level agreements and conventions to consider .



2.4 MILE

Managed Incident Lightweight Exchange  <https://maecproject.github.io/>
  Packaging standards cover DHS The contents of the series specifications are roughly the same , especially CybOX,STIX and TAXII.MILE The standard defines a data format for metrics and events . The package also contains
Incident Object Description and Exchange Format 
<https://www.rfc-editor.org/rfc/rfc5070.txt>
( Event object description and exchange format , abbreviation IODEF).IODEF A lot of them have been merged DHS Data format of series specification , It also provides an operational format for those events , And support automatic processing . It also contains IODEF
for Structured Cybersecurity Information( Structured network security information , abbreviation IODEF-SCI) Extension and Realtime
Internetwork Defense( Real time network defense , abbreviation RID), Support automatic sharing of intelligence and events .




So far, there is no standard for threat intelligence that is universally used in the world , But the establishment of relevant standards has undoubtedly promoted the development of threat intelligence , In particular, the threat intelligence products of different manufacturers can be coordinated , Information sharing laid the foundation , We also look forward to the rapid development of domestic standards .

Three , Open platform

At present, many Threat Intelligence open platforms have been established at home and abroad , Domestic, for example

Green Alliance Threat Analysis Center <https://poma.nsfocus.com/>



360 Threat Intelligence Center <https://ti.360.net/>



The best thing to do in China should be micro step online <https://x.threatbook.cn/>
, Micro step is more than just a search function , The corresponding communities have also been established for information sharing , At the same time, the product of threat intelligence is realized .



There are also some abroad , such as Virustotal <https://www.virustotal.com/>



Threatcrowd <http://www.threatcrowd.org/>



Threatcrowd yes alienvault <https://www.alienvault.com/> A threat intelligence search engine , And established the corresponding threat intelligence community
<https://otx.alienvault.com/>



Threat information can be provided and shared by volunteers , Claiming to be “The World’s First Truly Open Threat Intelligence
Community”.

Riskiq <https://community.riskiq.com/>



Riskiq A set of Threat Intelligence driven “Digital Threat Management”, It includes a variety of products and services .


In addition, there are many things that have not been mentioned , The above Threat Intelligence platforms generally have one “ Search Engines ” As entrance , input IP, domain name , file HASH Output the result of the query , Including antivirus engine detection rate ,Whois,PDNS, Relationship map, etc . Some will also be provided API Interface , Or provide information push service , And as a way of making profits .

Four , Usage scenarios

4.1 Attack detection and defense


Based on Threat Intelligence Data , Can be created IDPs perhaps AV Signature of the product , Or generate NFT( Network forensics tools ),SIEM,ETDR( Terminal threat detection and response ) The rules of products such as , For attack detection . such as IP, domain name ,URL As machine-readable information IOC( There are many international standards for machine-readable Threat Intelligence , include :STIX,OpenIOC,IODEF,CIF,OTX etc. ), Direct import device , Access control of import and export flow . Manufacturers that do a better job in this area 是Fireeye,其核心产品都可以使用威胁情报数据来增强检测和防御能力.而其他大部分厂商的产品依然无法直接使用威胁情报,这也是阻碍威胁情报落地的困难之一.




乍一看这跟传统的黑白名单似乎没有区别,但实际上IOC具有更好的时效性,情报厂商不断的产生新的IOC,使用者可以不断地获取与自身相关的情报,使得在防护设备中始终保持一份“最新的热名单”,始终保持着对新型攻击的防护能力.一个比较好的例子:

http://www.chinaz.com/news/2017/0630/766058.shtml
<http://www.chinaz.com/news/2017/0630/766058.shtml>

4.2攻击溯源


安全分析及事件响应中攻击溯源是重要的工作内容之一,同样可以依赖威胁情报来更简单,高效的进行处理.在攻击范围确定中可以利用预测类型的指标,预测已发现攻击线索之前或之后可能的恶意活动,来更快速的明确攻击范围;同时可以将前期的工作成果作为威胁情报,输入SIEM类型的设备,进行历史性索引,更全面的得到可能受影响的资产清单或者其它线索.尤其是借助现有的威胁情报平台可以很好的对一些关键信息紧缩检索,比如在设备日志中发现了一些可以的ip,那么就可以根据ip进行威胁情报搜索,通过ip的历史通信记录,是否与恶意样本存在过关联,PDNS,反向域名解析结果等判断ip的性质,若为恶意ip也可以根据结果顺藤摸瓜进一步查询.

这一部分可以参考elknot大佬的文章:

https://zhuanlan.zhihu.com/p/30105006 <https://zhuanlan.zhihu.com/p/30105006>

https://zhuanlan.zhihu.com/p/30160133 <https://zhuanlan.zhihu.com/p/30160133>

https://zhuanlan.zhihu.com/p/30197024 <https://zhuanlan.zhihu.com/p/30197024>

4.3态势感知


态势感知也是一个说了很多年的比较大的概念,在很多人看来好像也没起到多大的防护效果.这里我们可以换一个更接地气的名字:安全运营.现在一个稍大一点的企业都会招募自己的安全团队吗,建立自己的安全运营中心SOC(Security
Operations
Center).SOC的作用一般负责应急响应,安全监测及制定整体的安全策略等工作,随着威胁情报的兴起,情报驱动的安全运营中心ISOC(Intelligence-Driven
Security Operations Center
)开始被提出来.ISOC具有融合分析大数据的能力,可以产生与企业相关本的自身情报,形成对自身的感知能力(知己);也可以调用外部开源或付费的威胁情报接口,获取最新的外部咨询,形成对外的感知能力(知彼),由此产生一定的态势感知能力.

五,未来


构建威胁情报体系,协同联动,扭转攻防失衡的局面.什么是攻防失衡呢?以下图示例(国外某数据公司发布的攻防时间对比图).第一行是攻击初始化阶段,大部分操作在分钟级就可以完成;第二行是数据获取阶段,在分钟级,时级或者天级可以完成大部分操作;第三行是发现阶段,要在月级才能完成大部分操作;第四行是数据恢复和防御阶段,要在天级以上才能完成.通过对时间的对比就可以看出攻防两端实际上是不对称的,这就是为什么这么多年的安全建设依然无法防御网络上形形色色的威胁.