Real-Time Building Management System Visual Anomaly Detection Using Heat Points Motion Analysis Machine Learning Algorithm

: The multiplicity of design, construction, and use of IoT devices in homes has made it crucial to provide secure and manageable building management systems and platforms. Increasing security requires increasing the complexity of the user interface and the access verification steps in the system. Today, multi-step verification methods are used via SMS, call, or e-mail to do this. Another topic mentioned here is physical home security and energy management. Artificial intelligence and machine learning-based tools and algorithms are used to analyze images and data from sensors and security cameras. However, these tools are not always available due to the increase in data volume over time and the need for large processing resources. In this study, a new method is proposed to reduce the usage of process resources and the percentage of system error in anomaly detection by reducing visual data to critical points by using thermal cameras. This method can also be used in energy management using home and ambient temperature and user activity measurements. The statistical results of the visual comparison between the proposed method and the legacy CCTV-based visual and sensory surveillance shown in the results section demonstrate its reliability and accuracy.


INTRODUCTION
Today the Internet of Things (IoT) connected devices are the base of a paradigm that will be the network of smart devices and machines.Through IoT, these connected devices will be able to communicate with each other or the human clients (for monitoring or controlling goals).At the same time, during the growth of these revolutionary technologies being more and more in human daily life, serious issues will arise as to how to secure these systems from the network, software, and environmental attacks and errors [1].There are several types of devices connected through IoT.Controlling these huge amounts of systems is not possible using human resources, therefore there are some technologies designed to act like sub-systems to manage and give access to the devices from users and the internet devices.A Building Management System (BMS) is one of the most important systems which is used in the management of devices connected over the same local network in a building.A BMS monitors, supervises, controls, and reports on smart building technology systems.These systems may include access control, video surveillance, fire alarms, Heating, Ventilation, and Air Conditioning (HVAC) control, programmable lighting, and electric power management.Such a considerable and huge number of IoT-based services most include anomaly detection systems (section) to decrease the risk of software or hardware misfunctioning, also detecting unusual activities to manage and secure the power, heating, ventilation, air-conditioning, physical access control, pumping stations, elevators, and lights [2].Controlling the complexity to manage this huge amount of data, and securing the facilities is so vital.Also, many anomaly detection algorithms are not always available due to the wide variety of input data types and limited processing resources [3].The complexity of anomaly detection tools should not make any delay and their accuracy should be high enough because any non-deterministic latency or false anomaly alert is not tolerable for the control system or security personnel [4].To cover the critical and fundamental information about the building, the novel visual anomaly detection algorithm uses thermographic cameras for home surveillance which uses machine vision and intelligence techniques to process image frames taken from thermographic cameras.
Anomaly detection is referred to the identification of an odd, peculiar, or strange condition, situation, quality, etc., [5].An anomaly is frequently misused with the meaning of only security threat, but the meaning of anomaly is considering so many other factors and the security threats.Anomaly detection in a BMS is to identify odd changes or strange conditions in sensory or image data entries.Examples of anomalies for a building may include human-based actions failure, robbery or breaking into the building without permissions, temperature/lighting control systems fault, and technology malfunctioning.Anomaly detection is such a vast distributed management system using visual and sensory data that may be a challenge for resource management and device communication aspects.
Here the method is focusing on how to estimate the percentage of true anomaly detection using heat points motion analysis.
This study will follow the most recent and common methods for anomaly detection in building management systems using visual content processing.After a brief information about the latest advancement in this field, the proposed method implementation will be discussed.The results of the process will be demonstrated using a comparison between two other algorithms.

MATERIALS AND METHODS
Anomaly detection in camera images is usually comparing two or a sequence of images captured from a unique scene or environment and comparing them with each other.Difference estimation between visual data needs a complex and dynamic model definition for the new data to be compared with the last received one(s) [6,7].The modern machine intelligence or fuzzy systems-based comparison methods are concerned with fast, adaptive, and organizational learning which is along with memorizing information in vast amounts and active interaction and struggling with models and information retrieved from input devices [8].These data transactions and process need more and more resources due to the increasing amount of data and sometimes after increasing more values because of adding new sensory or visual input devices.These kinds of systems are called Evolving Connectionist Systems (ECOSs).In this type of system, the framework assumes its structure and functionality should be evolved based on a new input stream of data [9,10].The difference between the reference image and retrieved images is determined as interference.Using these compare and time series anomaly detection techniques, if the pattern of the receiving new data is significantly distinct from reference data, the algorithm may detect an anomaly.Three approaches are sorting the comparing methods [11].These are Full-Reference (FR), No-Reference (NR), and Reduce-Reference (RR).
Almost comparing algorithms are following the FR approach, because they adopt statically modelled images as a reference to estimate variations from noises and cluttered scenes [12].

Full-Reference (FR)
The fully referenced approach for anomaly detection is to use the first input value as a reference to check if the next input value is changed based on that.Several visual anomaly detection methods use this approach, for instance [13]: • One-class classification (OCSVM, SVDD, and OCCNN).• Image reconstruction.
To find out the abnormal changes in the sequence of entry data is common to use Machine learning algorithms.Prophet algorithm, which was released by the Facebooks Core Data Science team, is an open-source library designed for automatic forecasting of univariate time series data.This is done based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects [14].It works best with time series that have strong seasonal effects and several seasons of historical data.Prophet is robust to missing data and shifts in the trend, and typically handles outliers well [15].
Interference analysis of thermographic images was used before for estimation of error or multivariant distributions of input data in medicine [16,17], geological science [18,19], and some other fields, but it has never been used in building management systems anomaly estimation.

Proposed Method 2.2.1 Model Steps
The currently proposed method contains three thermal image processing steps, a model extraction method using Facebook time-series Prophet prediction, an information retrieval step, and two other steps for finalizing the result.The input data contains sensory and thermal video footage.These data will be going to be processed as live stream data after some optimization steps and time-series-based data specifications classifying.The structural similarity will be estimated for each dual-frame to maintain the differences of each frame.Tab. 1 is a dataset of the Structural Similarity Index (SSIM) frame of data to show how the algorithm is handling mass data and computing the mean variance of the related time-series.

Model Codes and Statistics
A big challenge of implementing such an algorithm which is processing the graphical data directly is to determine the resources consumption, where tuples and data frames came in handy.Also eliminating the cold areas which are not changing during at least two frames, helped to decrease RAM and GPU consumption, and helped to boost the algorithm's data handling speed.The codes of the proposed model are shown as Code-1.Variance estimation is an important step to find out the differences between the changes of every dual-frame during the closest time series.Extracted model based on the maximum variance percentage and time series is demonstrated using box-whisker plotting as in this Fig. 3. Having these data shows the usual high/low changes on the frames in Fig. 3, which will be considered for allowable detected changes.By continuing this step and repeating the other time series the allowable variance of changes in frames will be found during the days of the week.The final segmentation to separate the models for each day of the week makes the models more flexible for holidays and specific events.For instance, during holidays and some special events which occasionally make some changes in variance they will be detectable by related models.

Flowchart of the Proposed Method
The survey method between different parts of the algorithm is drawn as a flowchart to show their relationships.

RESULTS
For instance, the final model extracted from data for Thursdays is demonstrated in Fig. 4 and Fig. 5.This model will be used to find out abnormal activities on the following Thursdays.Also, it is compatible to upgrade the weights and thresholds based on special events and changes in activity patterns.
The frame change threshold based on Structural Differences Indexing (SDIM) and time series are given for Thursday in Fig. 4.These data were obtained for a full day.It usually shows the maximum activities on Thursdays, which are approximately at 09:00, 16:00, and 20:00 in Fig. 5.It means, that if the system sensed activities due to the schedule and allowed situations, there is no need to activate any special alert for an abnormal break or any error.Eq. (1) calculation of TP, TN, FP, FN refers to the number of True Positive, True Negative, False Positive, False Negative respectively also Sensitivity, Specificity, and Accuracy based on them [20].
where: TP -number of true positive specimens; FP -number of false-positive specimens.
The accuracy of alerts on specific real suspicious situations is examined and compared to the other similar algorithms.In addition, it shows the comparison of over 75 data records along with the proposed model results in Tab. 2. The testing videos captured from the analogue camera are digitized by using VGA resolution, 30 frames per second, and encoding with MPEG1.All the testing videos have contained 1800 to 2700 frames and at least 30 seconds of normal surveillance scenarios before the anomaly events happened, which is taking some algorithms that needed background modelling time into consideration.There are total 75 testing videos which are including 57 videos with anomaly events and 18 videos without any camera anomaly.An anomaly video has only one event inside.

CONCLUSIONS
Using received images from thermal cameras effectively gives high accuracy of detection of abnormal factors, such as the entry of people or other unusual events, such as a sudden rise or fall in temperature.Something else, as is mentioned, the very low error rate of the proposed algorithm compared to other existing deep learning-based methods.Another noteworthy point in this algorithm is its speed, which was not mentioned in the benchmark.The proposed method is based on data mining, which significantly increases its speed in obtaining flexible models from the received data.When comparing over 75 data records, it is seen that the proposed algorithm has the advantages of 19.05% Optimal FAR and Best redundancy removal.Facebook Prophet method helps the algorithm to find the key points to identify the patterns and prepare the final models, which itself has considerable speed, accuracy, and flexibility compared to time-series-based forecasting methods.It is common for these systems to be complex because of using deep neural networks for processing the big data entries, eliminating noise on data, and being flexible on human-defined values (not accurate and real-time), which presents new challenges and opportunities for the design and development of anomaly detection methods using visual data.Several characteristics of AI-based anomaly detection systems are discussed, as well as challenges in specifying acceptance criteria for such systems, such as accuracy, sensitivity, specificity, and false alarm rating.There are several challenges associated with testing this kind of system, including the environmental-based changes (day/night-time duration, season visual changes, moving obstacles on point of view, etc.), which make it difficult for testers to determine whether such a method has passed or failed by only a set of data from an environment.In addition to covering the testing of these systems across the life cycle, the researchers tested the algorithm in the outdoor and challenging environments and no significant changes in results were observed because of the advantages of the thermographic imaging methods.

Figure 1
Figure 1 Exported frames of a 7-second thermographic video camera (2 fps)

Figure 3
Figure3The extracted model for normal movements

Figure 3
Figure 3 Flowchart of the proposed method

Figure 4 Figure 5
Figure 4 Extracted threshold of frames changes based on structural differences indexing (SDIM) and time-series Time series-based data modelling is usually applied using two types of model generation techniques, Univariate and Multivariate time series modelling techniques.Autoregression (Including Autoregressive Integrated Moving Average (ARIMA) and Seasonal Autoregressive Integrated Moving Average (SARIMA)) and Facebooks Prophet methods are some instance algorithms.

Table 1
SSIM dataset for day 1 to day n to estimate the most repetitive patterns and make the model

Table 2
Benchmarking over 75 data records