Application of Predictive Models for Natural Gas Needs-Current State and Future Trends Review

Nowadays, in terms of trading on the world scale, to foresee a natural gas consumption represents an essential activity. In the first part, the paper examines the current state of the Serbian natural gas sector and methodology applied for prediction and capacity planning. In addition, the study intends to give a comprehensive assessment of predictive algorithms for natural gas needs involved in the last decade with projections and suggestions for future applications. The primary task is to evaluate used predictive models with an emphasis on the accuracy of the predictions obtained. Additionally, the paper will analyse used parameters, consumption scale, prediction scope, forecast algorithms, and other related information. The main objective of this study is to review the new-fangled information related analyses data from peer-reviewed journals, international conferences, and books.


INTRODUCTION
In today's energy engineering practise, a smart approach to achieve sustainable planning, resources assessment or to deal with the global challenges is essential [1,2]. Predicting consumption and how to provide natural gas supply is an essential activity in the energy sector worldwide. In the case of an incompetent estimation of natural gas needs, the price and penalties due to overestimated/undervalued quantities of the fuel can economically jeopardize the operation of the system and the end customer. The natural gas system consists of the following parts: production, transport, and distribution. In addition, end-users of the system should be mentioned as their response is very important for planning and predicting natural gas needs. Concerning predicting the required quantities of natural gas, each of the aforementioned parts of the system must actively participate. A key-term that necessitates a better accuracy in predicting natural gas need is trade. Subject to inadequate assessment of natural gas needs, the potential price and penalties due to non-adequate quantities can financially weaken the system operation and the end customer.
In order to maintain a historical overview of the scientific papers with selected keywords "natural gas" and "forecasting", the search was performed in the SCOPUS database from 1940 to 2020, where the primary accent was on the literature from the year 2010 and ahead. This scientific database includes the WoS (Web of Science) sources as well as papers from some respectable international conferences. Fig. 1 represents this historical overview.
The conclusion is that the growing scientific consideration of the keywords "natural gas" and "forecasting" was in the last 20 years, with the highest number of articles published during the previous ten years. In the periods from 2001 to 2010 and from 2011 to 2020, numbers of publication were 653 and 1418 respectively. Soldo [3], in his comprehensive review paper, examined predictive natural gas consumption models applied until 2010. After, Šebalj [4] included analysis of forecasting algorithms developed until 2017. The focus of both review papers was on historical development, prediction methods, area and horizons, and used variables without assessment of archived model's accuracy. According to that, the novelty and the key accent of this research review is the assessment of used predictive models with an emphasis on the forecast accuracy obtained. Also, the review is made because of the need to evaluate the advantages and disadvantages of the existing forecasting methods, as well as the worldwide experiences in this field. The study aims to obtain a unique view of demands that scientists deal with, and the investigation presently accomplished on the cutting edge of forecasting natural gas consumption. The focus of this review paper will be the latest period from 2011 until 2020. Therefore, this succinct information source intends to help the forthcoming studies as a starting point of their scientific research.

CURRENT STATE AND PRACTICE IN SERBIA REGARDING THE PREDICTION OF NATURAL GAS CONSUMPTION
The Serbian public company "Srbijagas", as the Operator of the natural gas Transmission System (OTS), in accordance with the Energy Law [5], is obliged to draw up a strategic document for the extension of the natural gas transmission system for a period of at least ten years and to harmonize it with the intention of development of connected systems and connection requirements natural gas underground storage facilities, natural gas producers and end-users to the transportation system and connections to the distribution system [6].
The OTS system planning considers the key national strategic documents [7][8][9], integrated with three crucial criteria: 1. Cost-effectiveness criterion (includes cost-benefit analysis with fundamental financial performance indicators of the investment); 2. Functionality criterion (consists of the aspect of optimal capacity planning, technology, and long-term needs), and 3. Environmental benchmark.
According to applied methodology, it is clear the Republic of Serbia does not use sophisticated and advanced models for natural gas prediction. The primary reason for that is because the telemetric system is not installed on the entire transport and distribution system (83% of the system is covered by telemetry). This is the first necessary step in order to apply advanced models for natural gas prediction.
On the other side, countries with developed natural gas market and trade (such as countries in European Union) use advanced forecast models in order to predict the next day, week, month or year consumption. This is essentially important on each level of natural gas market (production, transport, and distribution.) because the contract price and penalties due to overestimated/undervalued quantities of natural gas are based on prediction accuracy. Poor estimation can significantly jeopardize the operation of the system and the end-user.
Until today, all parties that operate on the Serbian natural gas market are not obliged to precisely predict future consumption or to pay the contract penalties due to poor estimation. The main reason is because currently the Republic of Serbia has a legal agreement with the Russian Federation without applicable penalties. This practice will be changed soon due to natural gas market liberalization, and all levels of the Serbian natural gas market must be prepared to apply one of the advanced predictive models.

REVIEW OF PREDICTIVE MODELS APPLIED IN THE PERIOD FROM 1940 TO 2010
The development of mathematical algorithms to foresee a natural gas needs began in the period from 1949 to 1957 [10][11][12], and continues intensively nowadays. Various mathematical models have been developed directed toward the prediction of natural gas consumption. One of the first models was the Hubbert model from [10] 1949. He explored the life cycle of fossil fuel fields and defined a model for the dependence of the production rate dQ/dt and time t. This model is a fundamental tool for predicting the production and consumption of fossil fuels. On the other side, different statistical models are applied to foresee the depletion of natural gas. One of the first models used to foresee natural gas needs in the commercial and residential sectors was the least-squares method [13]. Also, a regression analysis is very often used to predict natural gas needs. A monthly regression analysis of the industrial demand for natural gas in the USA was used in [14] (1987). In the same year, regression assessment was used to approximate the annual and monthly consumption of users in the residential sector in the same country [15]. At the end of the observed period, artificial algorithms have been increasingly applied to predict the consumption of natural gas. In the study [16], the short-term forecast of natural gas needs is done by the multilayer artificial neural networks. The parameters of the neural network are obtained based on archival data on consumption and weather conditions, training through the Levenberg-Marquardt algorithm. The quality of the proposed neural network is checked with real consumption data from a particular area (The Mean Absolute Percentage Error -MAPE was in the range between 4 and 20%). Similarly, a short-term natural gas forecast consumption on an hourly basis is presented in the [17].
The authors used two algorithms for neural networks, the radial distinctive function network, and the multilayer perception. The best fit was obtained by multilayer perception (MAPE = 9,36%). The number of scientific papers involved in developing new and improving existing mathematical models is steadily increasing. A chronological overview of the development of predictive mathematical algorithms for natural gas consumption (period until 2010) is presented in detail in a scientific review [3].
According to previously reviewed practice, general parameters set up to establish a good standing mathematical algorithm are: -Data on natural gas consumption; -Data on energy consumption; -Weather data; -Financial metrics; -Proved mathematical and engineering calculations; -Validated software simulation data; -All kinds of useful statistical data; -Information about calendar days.

REVIEW OF PREDICTIVE MODELS APPLIED IN THE PERIOD FROM 2011 TO 2020
As it can be seen from the period from 2000 to 2010, the development of algorithms to foresee the consumption of natural gas is increasingly based on the principles of artificial intelligence. Accordingly, most applied methods will be discussed below.

Application of Artificial Neural Networks (ANN)
The inspiration for ANN was found in the neurological activity of the people's brain with an analogy concerning the intellectual operation. Experts in various fields show interest in applying ANN due to the ANN's capacity to find a result to complex, multidimensional or nonlinear datasets [35]. Through these networks, it is possible to analyse nonlinear models as well as incomplete data structures. The technically primary task of ANN is to learn the structure to generalize based on a sample of data. ANN, in a way that simulates biological neural networks, connects a network of data processing elements and thus can "learn" on a limited set of data. ANNs can be classified by: the number of layers (monolayer and multilayer), connections between neurons (layered ones -e.g., with back propagation when outputs of all neurons from the previous layer are supplied to the intake of one neuron, with output which is directed to all neurons inputs in the next layer; cellular -only adjacent neurons are interconnected, but the signals extend to both neurons and out of the neighbourhood due to the indirectness of information dissemination; fully connected -a neuron output leads to the input of all the neurons in the network), the type of training (monitored in which training monitoring algorithm (supervisor) compares output data with expected data and sends the difference to a learning rule that corrects network weights -a perceptron or back propagation algorithm; partially monitored training where the network "learns" on its own with occasional work experience and unsupervised training in to whom the network is independent); the direction of information spread (feed forward -where higher layers do not return information to lower layers, propagation -where a signal is only in one direction from input to output -e.g. back propagation algorithm of multi-layer perceptron with applied; feedback -in which higher layers return information to lower layers), the type of data (almost all ANNs are discrete because analogue data is almost never used).

Multilayer Perceptron (MLP)
To enable learning, a network is created to facilitate coaching with all the data. During the training process, input forms are introduced into the first layer. The activation values of the intake nodes are graded and gathered in every node of the hidden part. A "hidden" refers to layers that are not visible either as inputs or as outputs. These layers are interlayers, and there can be one or more of them. Networks with multiple hidden layers are usually called multilayer perceptions, which can be confusing since they often do not contain perceptron at all. In the end, the weighted sum is transmitted by an adequate transfer action to obtain the node activation value.

Radial Basis Function Neural Network (RBFN)
For the RBMF networks, the input layers serve only as a signal distributor to the invisible layer. Unlike MLP, results in the input layer are passed directly to the invisible layer outside of multiplying by the weighting results. A unit of the invisible layer estimates the range between the load vector and the focus of its radial function and creates results lean on the length. The reference vector is the focus of the radial function. If the distance between the load vector and the reference vector is less, the operation is better at the output of the invisible node. Several features are recommended for use in the hidden layer (Gaussian, Multi-square or Thin-plate function).
The most used is the Gaussian function. Otherwise, in [36], it is shown that the selection of the radial basis function applied in the network does not substantially impact the operation of the network.

Adaptive Neuro-Fuzzy Inference System (ANFIS)
This is one form of ANN which is developed through the fuzzy logic (FL) system of Takagi-Sugeno at the beginning of the 90s [37]. The technique was developed in the early 1990s. This kind of hybrid intelligence system integrates both ANNs and FL principles and has more capability to obtain better prediction than in a single network. The system refers to a set of FL rules (If-Then), which have a learning potential to simplify a nonlinear function (this kind of system can be used as a universal estimator). The best way to use the ANFIS models is to integrate the optimal parameters from the genetic algorithm. The ANFIS structure is not a unique one. It is possible to combine some layers and still get the same results.

Problems with ANNs Application
There are usually two problems with ANNs. The first one is the so-called "overfitting" which means data overmatching when an incorrect model fits the data well because its complexity is large enough for the amount of available information (the model learns ideally to recognize instances from a training set but is unable to identify cases that are even slightly different from the ones discovered). The second one is the so-called "curse of dimensionality" caused by the increasing complexity involved in adding additional dimensions to the neural network.
A common problem [38] is highlighted that in training any neural network there is always a certain level of unpredictability of the results obtained. It is challenging to know even if an individually trained pattern is operating correctly since the inputs are right due to chance.

Machine Learning (ML)
ML means the ability of a software system to: • generalize from previous experience (data set of learning entities), and • use these generalizations to provide answers to questions relating to entities/phenomena not previously encountered.
More recently [4], various types of so-called ML for predicting gas consumption are available. Among them are the so-called Support Vector Regression (SVR) and Support Vector Machine (SVM). SVM is found in the theory of statistical learning. The ANN principle employs the principle of Empirical Risk Minimization (ERM), while the SVM is replaced by the structural risk minimization (SRM) principle. The overriding concept of SRM is to use upper bound minimization to generate a fault instead of minimizing training mistakes. With the introduction of the so-called "Vapnik" ε functions of insensitive loss, SVM is extended to solve SVR regression problems. Necessary steps of the overall ML process are composed by the following: 1. Collecting the data needed to form datasets for training, validation, and testing of the model; 2. Preparation of data (usually referred to as "refining" and transformation of data analysis of the resulting datasets and (possibly) their further refinement by selecting or transforming their properties (attributes); 3. Choice of ML methods (one or more); 4. Training, configuration and evaluation of the models created, and 5. Selecting the model to be used (based on the previous one) and testing it. Research presented in [39], compared ANNs with SVMs and concluded that SVMs produce smaller statistical errors in estimating natural gas consumption needs. In [40], for prediction are used various nonlinear and linear algorithms.
Testing faults acquired by SVR and ANN (nonlinear), it can be concluded that they are slightly bigger correlated to linear ones.
A method that integrates the SVR algorithm with the reconstruction of time series properties and optimizes the original local predictor by removing false neighbours' is tested in [41]. Bai and Li [42], used a structurally calibrated SVR approach.

Prediction Accuracy of Selected Applied Models in the Period from 2011 until 2020
Kaynar [43] stated that in the case of Turkey, the forecast consumption of natural gas is important because of its energy dependence and storage capacity, which is only 5% of total internal use. Prediction accuracy is one of the elements of sectorial investments and agreements related to production as well as sectorial progress. In this paper, Turkey's weekly gas consumption is predicted through 3 approaches: Autoregressive Integrated Moving Average (ARIMA), a classic time sequence analysis method, different ANN models (MLP and RBFN) and ANFIS (Adaptive Neuro-Fuzzy Inference System), and integration of Fuzzy Inference System with ANN. According to the MAPE criteria, the best value (slightly compared to the MLP ANN) has an ANFIS of 5,468% versus 5,477% for the MLP and 6,186% for the RBF and 6,410 for the ARIMA method. The set of data used was obtained from official publications in Turkey.
Demirel [44], has compared Ordinary Least Squares (OLS) model (as a benchmark) against which the effect of more advanced methods, Autoregressive and Moving Average (ARMA) and ANN, were analysed. The Box-Jenkins model selection method was used. Daily gas consumption was initially tested for stationary using Dickey-Fuller and Philips-Perron tests. Both tests decline with a significance level of 1%. An Augmented Dickey-Fuller (ADF) analysis was also used. The backpropagation/gradient descent algorithm was the best according to RMSE and MAPE parameters, but the ARMAX method was the best according to MAD.
The authors showed that of the 16 parameters used, temperature and the natural gas price had the most substantial impact on natural gas consumption at the Istanbul level (compared to various influential factors including seasonal variations, newly erected buildings, new arrangements with other suppliers, revenues and changes in needs of consumer, etc.).
Taspinar [45] used MLP ANN and time-series approach (SARIMAX). Meteorological data for the previous four years were used. According to these authors, the SARIMAX model showed better performance compared to the others in all statistical parameters.
Soldo [40] evaluated the impact of solar gains on two datasets: consumption of natural gas of a house and local distribution company model. Different models were analysed with a 1 -day forecast horizon (gradual regression and auto-regression model) and non-linear patterns (ANN and SVR). The obtained results confirmed the evident positive impact of solar gains on the accuracy of gas prediction. It is recommended to include this parameter as an input variable of forecasting models at the building and city scale levels. Aramesh [46] used the ANFIS algorithm. The results proved better accuracy of ANFIS compared to ANN (primary station MAPE was 5,57%). Also, in the secondary station, this algorithm was used after the adaptation. The MAPE value error was 5,73%.
Bianco [47] showed the impact of GDP (gross per capita social product), gas prices, and HDD (heating degree days). The impact of energy savings measures in the building heating sector has been assessed, and it has been shown that concerning consumption like now Business As Usual (BAU), use can be reduced by approximately ~ 20% if 15% of dwellings and 5% of homes are isolated in appropriate way. The so-called "Climate adjustment" was done by calculating the WAC (weather adjusted coefficient) as a quotient of the averaged HDD for a given period (1990 -2012) and the HDD values for the year under review. The results of this model were compared with the prediction of energy intensity (for previous five years, an entire consumption was calculated by multiplying an average energy intensity per capita and population number) and with the prediction using simple fit.
Szoplik [48] used MLP (multilayer perceptron model) with the different numbers of hidden layers and range of the trained datasets to estimate natural gas consumption at any hour on any given day of the year. For the MLP model, author sets 22 timely based variables. The input dataset was divided into three subsets: set 1 with 8760 included data for 2009, set 2 with 17 520 data for 2009 and 2010. In the hidden layer, neurons number varied from 5 to 40. The best result was shown by the MLP 22-36-1 trained on the dataset between 1 January 2009 and 31 December 2010. On average, MAPE values of 8% were obtained, with lower values in the heating period. The reason for that was higher natural gas consumption. Random sampling was used for training, testing and validation sets. By this approach, it is possible to avoid the impact of data used for training of the neural network. The precision obtained is acceptable given that natural gas is used primarily for individual users who have erratic behaviour (to some degree unpredictable patterns of consumption).
Zhu [49], compared an FNF-SVRLP (False Neighbours Filtered-Support Vector Regression Predictor) and SVRLP (Support Vector Regression Local Predictor) to obtain a short-term forecast of natural gas needs. The authors concluded improved results in the case of the FNF-SVRLP model for daily use. This was achieved in case an AM (advanced model) is based on specific consumer behaviour on different days of the week.
The advantage is shown (according to the statistical parameters applied in this case) of both variants of the SVR approach over ANN and ARMA models, as well as the advanced (AM) model (made taking into account the characteristics of each day of the week) over the standard (SM) model for both the SVRLP and the FNF-SVRLP variants, with better results for the FNF-SVRLP variant of the AM model.
Izadyar [50] compared models through statistical parameters RMSE, R 2 , and r. The least learning error was for the ELM model. The study proved that the ELM model had a much higher learning speed related to conventional feed-forward network learning methods error of 3 to 5%. The ELM models can be applied to create a forecasting strategy for forecasting residential heating needs in district heating systems. The new procedure shows satisfactory performance in most cases and is faster to learn compared to other standard learning algorithms.
Oliver [51] developed a model with an adjusted network degree day for variable climate conditions (NDDCA). Solar radiation has also been shown to affect consumption significantly. Impacts of various effects, starting from those already implemented and into existing standards (outside temperature), through the influence of wind, solar radiation, and annual consumption, are analysed -the so-called standard Peak-week supply used on SME gas markets and for Irish domestic sector. The intention was to reflect the comparative variation in peak daily use prescribed by the rules. The Anderson-Darling "Goodness-of-fit" result was unproved compared to the weekly algorithm. The next year peak gas consumption was estimated at 83.14 GWh, which is approximately 13% lower than the 95,7 GWh determined by the mentioned standard. Panapakidis and Dagoumas [52] have examined the stability of a new hybrid model to predict the consumption one day in advance. The envisioned algorithm includes WT (GA), ANFIS, and FFNN. The Wavelet Transform was applied to disintegrate the original inputs into a group of sub-frames. Optimized ANFIS is used for the forecast of an each sub-frame. The FFNN fed by ANFIS results is used to improve the initial prediction. The algorithm is fully applicable to the country's natural gas network. According to the author's conclusion, the developed algorithm enables correlation of the operation across various consumption patterns while providing insight into the aspects of cityscale or industrial districts, power generation plants, and more. Errors are shown for 41 distribution points for the ANFIS / GA / FFNN model.
Beyca [53] compared MLR (multiple linear regression), ANN, and SVR to foresee natural gas needs for the Turkish province of Istanbul. The proposed Polynomial Cubic Kernel function integrated into an SVR algorithm has shown the best results and is recommended as an essential tool for decision-makers and policymakers as well as regulators for megacities such as Istanbul.
Tab. 1 summarizes the advantages and disadvantages of selected applied methods described in the literature from 2011 to 2019.

CONCLUSIONS
This paper reviewed novel approaches and tools in the sense of predicting natural gas consumption. The main focus was on researches published in the last ten years. We believe that there is a lot of space for improvement since a large number of factors and system capabilities must be involved in model creation.
The selection of an appropriate and accurate predicting tool for natural gas consumption will improve the forecasting needs on each level (home, city, national, regional, world, etc.). Surely, this would have the effect of reducing the price and optimal planning of natural gas needs worldwide. Reviewing the scientific and engineering practice in the last ten years' conclusion is that the development of prediction tools for natural gas consumption will follow the way of having more accurate and complex Artificial Neural Networks algorithms and Machine Learning tools. Future models must include more parameters and measured data depending on the level of application. Reviewed authors, based on different criteria for various regions, time intervals, and forecasting horizons, obtained diverse accuracy rankings. Such a mixture of the results and complexity of applied models may lead to the conclusion that until now is not found a perfect way to predict natural gas consumption. One of the major challenges will be to test the predictive models in the environment of short or mid-term weather forecasts. According to that, our intention for future research is to integrate reliable short-term weather forecasts into an appropriate artificial intelligence prediction algorithm.