Data collection, processing and storage system
- "Micronika" LLC
- telemetry, gas metering, data server, process automation
Intelligent telemetry network optimizes production, distribution and consumption of resources through the introduction of information and communication technologies. In essence, intelligent telemetry networks transmit large amounts of data to information systems:
- new information flows from the resource allocation network;
- from new entrants to the resource distribution network, such as decentralized renewable energy producers;
- information flows from houses with connected communication equipment: smart meters, sensors, remotely controlled network nodes.
One of the features of the implementation of intelligent telemetry systems is a significant increase in the flow of processed data due to the installation of smart meters, various sensors and alarms in the network. For example, a meter with a universal telemetry module can send data about the resource consumed by a user every 15 minutes, so that every million meters will send information 96,000,000 times a day.
Thus, systems for collecting, processing and storing data must be efficient enough to be able to cope with a large flow of data, their storage and processing.
Information systems are key components of intelligent telemetry networks that interact with each other to make them more efficient, flexible and scalable.
System structure
The services of the system for collecting, processing and storing data receive telemetric data from network nodes, and then use them to obtain information on the state of power lines, pipelines, heating mains, equipment, volumes of consumed resources and modes of their consumption.
Components of SSSOHD
The system for collecting, processing and storing data includes the following components:
- A software complex for dispatch control and data acquisition (SCADA), it receives information from the measuring units of the telemetry system. This software package is a reliable and secure system for monitoring and control in the network, has the ability to collect data from any systems. The mission of SCADA is to collect data, exchange data, present data in various forms and control, which entails increasing the level of automation, efficiency and reducing costs. Allows you to optimize networks that distribute resources (gas, water, electricity, heat, etc.), offering more options:
- the use of programmable controls;
- adjustable duration of measuring intervals, etc.
- MDMS;
- operating gateways;
- systems for integrating data into application software platforms.
- Resource Streaming Management Service (OMS). Utilities of the SSOHD information system strive to provide the most comfortable level of use by detecting, localizing and eliminating interruptions in the supply of resources in a very short period of time.
- The geographic information service allows you to have a visualization of the location on the maps of trunk networks and electricity distribution networks, gas pipelines, heating mains, water supply pipelines, locations of consumers and measuring nodes.
- The Client Terminal Service (CIS) allows organizing the exchange of information between network clients (resource consumers) and the information system.
- The service of storing and archiving data in an information system has a decisive role, therefore there is a need to have a developed and scalable mechanism for storing large amounts of data, including:
- distributed file system (DFS), which allows multiple users on different computers to share shared files and resources, and allows each user to have a local copy of stored data. There are many solutions that DFS uses, for example: Googles DFS, Qantcast System, HDFS, Ceph, Luster GlusterFS, etc.
- NoSQL databases is a new approach to databases to overcome the limitations of traditional relational SQL databases in the case of large amounts of data.