Volume 8 - Issue 4
Parallel big data processing system for security monitoring in Internet of Things networks
- Igor Kotenko
St. Petersburg Institute for Informatics and Automation (SPIIRAS) 39, 14-th Liniya, Saint-Petersburg, 199178, Russia, St. Petersburg National Research University of Information Technologies Mechanics and Optics (ITMO University) 49, Kronverkskiy prospekt, Saint-Petersburg, 197101, Russia
ivkote@comsec.spb.ru
- Igor Saenko
St. Petersburg Institute for Informatics and Automation (SPIIRAS) 39, 14-th Liniya, Saint-Petersburg, 199178, Russia, St. Petersburg National Research University of Information Technologies Mechanics and Optics (ITMO University) 49, Kronverkskiy prospekt, Saint-Petersburg, 197101, Russia
ibsaen@comsec.spb.ru
- Alexey Kushnerevich
St. Petersburg Institute for Informatics and Automation (SPIIRAS) 39, 14-th Liniya, Saint-Petersburg, 199178, Russia, St. Petersburg National Research University of Information Technologies Mechanics and Optics (ITMO University) 49, Kronverkskiy prospekt, Saint-Petersburg, 197101, Russia
kushnerevich@comsec.spb.ru
Keywords: complex event processing, Hadoop, Spark, security monitoring.
Abstract
Nowadays, the Internet of Things (IoT) networks are increasingly used in many areas. At the same
time, the approach connected with the implementation of the network security monitoring system is
of particular relevance for the protection of IoT networks from threats. Due to the peculiarities for
construction and operation of IoT networks, the use of traditional protection systems for IoT is difficult
or impossible. One of such features is the need to analyze very large amounts of data in real time
and with minimal computational cost. Given the limited computing capabilities of IoT networks, we
propose the architecture of a big data distributed parallel processing system based on Hadoop and
Spark software platforms. The issues related to the implementation of this system and its main components
are also considered. The results of an experimental evaluation of the system performance
are discussed. They confirm the conclusion about its high efficiency. A comparative evaluation of
the implemented systems on Hadoop and Spark platforms is conducted.