How to do big data analysis even if the data of Internet of things is not clear ?
Data as value is a highly respected concept in the field of computer science . Data, no matter how much, comes down to big data , Data analysis is becoming more and more popular , Capital is also flocking to companies with big data labels . Data is being evaluated like a flowing digital currency , Pursue and worship .
When the Internet of things began to land and apply in the industry , Because of the speed of data generation , The variety , The huge size of the city will affect the existing cloud technology architecture , Data processing methods bring pressure and challenges beyond imagination .
Faced with such complex data , Legendary “ Data is value ” Will be “ Valuable data ” This rational understanding is broken .
What are the characteristics of Internet of things data ? Is there a reasonable classification ? Do we have the ability to cope with the pressure and challenges brought about by the data of the Internet of things ? Internet of things 1．0 What are the valuable data of the stage ?
Static data and dynamic data
In terms of data changes alone , Internet of things data can be divided into static data and dynamic data , Static data is mostly label class , Address data ,RFID
The data generated are mostly static data , Static data are mostly structured , Relational database storage ; Dynamic data is data with time series , The characteristic of dynamic data of Internet of things is that each data has one-to-one correspondence with time , And this relationship is particularly important in data processing , This kind of data storage is usually stored in time series database .
Static data will increase with the increase of sensors , The number of control equipment increases ; Dynamic data not only depends on the number of devices , The number of sensors increases , It will increase with time .
Whether static data or dynamic data , In the Internet of things 1．0 The growth of stage data is linear , It's not exponential , But because the dynamic data of the Internet of things is continuous , So the amount of data is huge . So the Internet of things 1．0 The pressure of stage data is controllable , It's not as innumerable as it is advertised , Uncontrollable .
Energy ／ Asset attribute class ／ Diagnosis ／ Signal class
In terms of the original characteristics of the data , We can divide the data of Internet of things into energy data , Asset attribute data , Diagnostic data , Signal data .
Energy data ： Energy consumption related , Or the relevant data needed to calculate energy consumption, such as current , Voltage , Power factor , frequency , Harmonics and so on . Energy data is the most critical data type of the Internet of things , One of the ultimate goals of the Internet of things is to save energy , So get energy data , Understanding energy data , Analysis of energy data is a necessary function in the implementation of Internet of things . Energy acquisition equipment is also one of the important devices of the Internet of things .
Asset attribute data ：
Asset data usually refers to hardware asset data, such as equipment specifications , Parameters, etc , Location information of the device , Subordination between devices, etc . Asset data is mainly used for asset management , Asset management is a very important function of industrial Internet of things, and even can be studied as an independent system , Because it can interact with ERP system ,MES system , Logistics and other almost all the system docking .
Diagnostic data ：
Diagnostic data refers to the data that detects the running state of equipment during the operation of equipment , There are two types of diagnostic data ： One is equipment operation parameters , For example, device input ／ Output value , This is usually the traditional industrial automation data OT Technical related data ; The other is peripheral diagnostic data , For example, the surface temperature of the equipment , Equipment noise , Equipment vibration, etc , It is worth mentioning that peripheral diagnosis is
Internet of things technology
It includes new sensor technology and Internet of things communication technology . Peripheral diagnostic data is an important metadata of predictive maintenance , It also provides the basis for depth control model , Therefore, diagnostic data is the data type that we need to focus on .
Signal data ：
Signal data or alarm data is the most popular data in the industrial field , Because it's intuitive , Easy to understand , crux , At the same time, local , Remote simultaneous notification . Signal data is easy to be ignored , But it's what the Internet of things needs , It can also be collected quickly , And provides one of the important reference value data for the Internet of things system .
Correlation between data
The relationship between data is the relationship between different data , The relationship between data has the most direct impact on understanding the operation of the whole system , Sorting out the correct relationship between data is the effective operation of the system , The cornerstone of value .
The correlation between data can be analyzed from the following aspects ：
Time relevance ：
That is, data photography at the same time , Data is generated by the system at the same time , It reflects the state of the system at this moment , From the perspective of data world , This system is the data collection of this moment . Data photography reflects the static display of the system ; Time stamp is a key factor in this kind of data , Therefore, the time stamp of each data acquisition must be the same , A lot of the current time stamp is missing , It is also one of the problems that need to be paid attention to and solved in the implementation of the Internet of things .
Process relevance ： That is, the data of one point will affect the generation of the data of the second point after a certain period of time , It reflects the system dynamic process display . The process relationship between data needs to be provided by the model , And it is revised in the implementation .
Timeliness of data
The timeliness of data refers to the time from data generation to data clearing , Data timeliness is determined by the implementation and deployment of the system . Data can be used multiple times, or it can be cleared after being used once . Generally speaking, whether remote deployment data or edge deployment data affects the timeliness of data , Generally, the data timeliness of edge deployment is short , Long time effectiveness of remote data . The data needed for edge deployment is usually timely , But edge storage , It can't be preserved for a long time because of its weak computing power ; Remote data is usually historical data presentation , Computational analysis , Cloud space at the same time , The calculation is flexible , Therefore, the timeliness of data is long .
The real-time of data is also a part of data timeliness , Real time and where data is deployed , The importance of data and the way it is transmitted are relevant .
Data centric Internet of things <http://iot.ofweek.com/> Design methodology
The purpose of summarizing all kinds of data and characteristics in the Internet of things or abstracting its generality is to better select different Internet of things technologies
Go fast , Effective implementation . Data centric Internet of things solutions and even business models will be the mainstream design ideas of the Internet of things .
The following is how the data is collected , How to preprocess data , How to transfer data and other different angles to analyze how to apply various technologies of the Internet of things .
Design ideas with data flow direction as the core
Design from data flow, including ： data acquisition , Data aggregation , data transmission , Data persistence , Data display , data processing , Data correction , Data consumption .
Data production ／ The core of design is consumption
production ／ The design idea with consumption as the core focuses on cloud business , In particular, the integration of existing decentralized subsystems as the target or the upstream and downstream collaborative operation of the industrial chain , The data of the Internet of things is mainly based on the type of key data indicators or as the starting data role .
1） The following is the design idea of data flow of supply chain .
2） The following is the design of integrated management system
Understanding the Internet of things <http://iot.ofweek.com/> What is the data , And can put forward the corresponding Internet of things technology
, Mastering valuable data is the first step at the technical level of the IOT solution . It is also the final concrete unit of Internet of things technology ——“ Valuable data ”.