Hybrid Automaton Based Vehicle Platoon Modelling and Cooperation Behaviour Profile Prediction

: Autonomous cooperative driving systems require the integration of research activities in the field of embedded systems, robotics, communication, control and artificial intelligence in order to create a secure and intelligent autonomous drivers behaviour patterns in the traffic. Beside autonomous vehicle management, an important research focus is on the cooperation behaviour management. In this paper, we propose hybrid automaton modelling to emulate flexible vehicle Platoon and vehicles cooperation interactions. We introduce novel coding function for Platoon cooperation behaviour profile generation in time, which depends of vehicles number in Platoon and behaviour types. As the behaviour prediction of transportation systems, one of the primarily used methods of artificial intelligence in Intelligent Transport Systems, we propose an approach towards NARX neural network prediction of Platoon cooperation behaviour profile. With incorporation of Platoon manoeuvres dynamic prediction, which is capable of analysing traffic behaviour, this approach would be useful for secure implementation of real autonomous vehicles cooperation.


INTRODUCTION
Transportation systems are experiencing a great advancement and are becoming highly evolving, especially in the last two decades. This trend requires an optimal use of transportation infrastructures, and the new way how transportation system interacts with the environment on a sustainable basis. The goal of Intelligent Transport Systems (ITS) is to make the transportation safer, more efficient, with more "intelligent" coordination, enabling more convenient travel for users. These systems incorporate variety of advanced technologies including wireless communication, navigation and control, sensing, computing, and artificial intelligence. Examples of ITS applications include some basic car navigation and traffic signal control, but also more advanced monitoring systems, which support emergency notification and collision avoidance [1]. ITS may result in improved highway capacity, better safety in transport, enhancements in mobility, increased fuel efficiency, and therefore lower environmental impacts.
While stand-alone applications such as traffic management systems, warning systems available in vehicles represent traditional ITS systems, recently is more research emphasis on cooperative ITS (C-ITS) applications. The cooperative intelligent transport system (C-ITS) is based on two or more vehicles working towards a common goal or benefit, C-ITS infrastructure is based on the interaction and information exchange between system's components such as vehicles, transportation infrastructures, cloud services, etc. [2].
Coordinated behaviour in the C-ITS received only minor attention in research and experimental evaluations. The main reason is that testing in real world environment with autonomous vehicles requires a large-scale infrastructure with important security measures. Mobile robot platforms are often used in experimental verification of Intelligent Transport Systems algorithms, which include intersection management techniques [3], vehicle platoons [4], [5] and vehicle dynamics [6].
The Platoon organization of vehicles is an example of a cooperative intelligent transport system. Vehicle platoon means a group of vehicles driving closely to each other, and being controlled as one unit. The coordination between vehicles in the platoon is based on the frequent exchange of communication messages containing information about vehicle's position, speed, acceleration and other attributes. A set of vehicles in the platoon have a predefined geometrical configuration, while the most widely studied platoon configuration is the column or train configuration, often called Road Train (RT) [7]. The behaviour of vehicles in the Platoon is expressed through the control system and it is based on distance relations between these vehicles [8]. The leader vehicle in a RT coordinates and controls distances, speed, acceleration of other vehicles in Platoon, and in general controls participation in the Platoon. Numerous benefits from Road Train Platoon include increased traffic density, improved safety, reduced fuel consumption, and tiredness of human participants [9].
Platoon approach is a basis for variety of ITS, from simplest systems without communication between the vehicles to advanced, intelligent systems which are based on advanced, wireless communication between all participants in the Platoon. Usually these systems use artificial intelligence technology in order to conduct manoeuvres in the Platoon [5]. The different types of join/leave manoeuvres between the vehicles in Road Platoon are presented in [10][11][12]. A platoon management protocol for CACC vehicles, based on wireless communication through VANET, is presented in [13]. The protocol operation is based on various Finite State Machines (FSM), and it is implemented in an integrated simulation platform, VENTOS, which is developed based on SUMO and OMNET++.
Main advances in ITS-based solutions are due to maturity and applicability of artificial intelligence (AI) field. Contemporary transportation systems are a natural environment to conceive, develop, test and apply AI methodologies solving difficult and complex transport problems, and improving the efficiency, safety and environmental-compatibility of a transport infrastructure. Recent ITS enhancements are based on AI techniques such as support vector machines, neural networks, evolutionary algorithms, case-based reasoning, fuzzy logics, ambient intelligence, ubiquitous computing, intelligent multi-agent systems, but also other patternrecognition and optimization techniques. These AI tools enable new quality in Intelligent Transport Systems, and better results in behaviour prediction based on traffic information, more efficient transportation optimization, advanced control systems and process planning, decision making in real time [14].
Machine learning techniques, as an important component of AI, are being a main tool for building intelligent traffic management models. By collecting hystorical data and analysing traffic behaviour and evolution, machine learning methods enable building decision models which are reasoning in a similar way to an expert traffic controller. The predictive techniques are being accomodated to allow advanced nonlinear models using historical baseline traffic data, such as speed, flow and travel time. For traffic information prediction, between numerous nonparametric prediction approaches most often are used artificial neural networks (ANN), support vector regression (SVR), and the adaptive neurofuzzy system (ANFIS)) [15][16][17][18]. The main advantage of the artificial neural network (ANN) is its ability to model very complex multivariable systems, and the quality of prediction is tuned and improved by parameters of the network such as the number of hidden neurons and learning factor [19]. The neural network prediction of the speed profile in transport systems is presented in [15][16]. Similar approach in modelling is with SVR techniques where the predictive model relies heavily on proper determination of model parameters [17]. ANFIS predictive model has proven to have the quality for prediction of cooperative behaviour profile of the flexible Road Train [18].
Despite the currently available gap between theoretical research results and real world experiments and implementations, there is still a strong demand for new approaches, new solutions, and future improvements and enhanced functionality in intelligent transport systems. These trends will stimulate a wider adoption of machine learning based modelling tools and systems for analysis and control of Platoon vehicles. The key contribution of this work is threefold. First, we propose a hybrid automation modelling to emulate flexible vehicle platoon and interactions between vehicles. Second, this paper introduces the cooperation profile, generated with novel function which depends on vehicles number in Platoon and behaviour types. Finally, an improvement of traffic mobility in Intelligent Transportation Systems is reached through better prediction of Platoon manoeuvres dynamics using NARX neural networks.
The paper is organized as follows: In Section 2 is introduced flexible Platoon hybrid automaton modelling, while Section 3 presents the transformation of behavioural patterns into cooperated behaviour profiles. In Section 4, the NARX neural network prediction of the Platoon manoeuvres dynamics is presented with experimental results and discussion. Finally, in the last section are given main conclusions and directions for future research activities.

FLEXIBLE PLATOON MODELLING
Hybrid systems are systems which demonstrate a combination of discrete and continuous behaviour [20]. Typical hybrid systems are used in many application domains, including embedded systems, robotics systems, avionics, automotive engineering tasks, process control, etc.
The basic mathematical and modelling framework to support description and analysis of hybrid systems is Hybrid Automaton (HA). This framework includes a rigorously defined notion of external behaviour for hybrid I/O automata, which involves their discrete and continuous interactions with their environment. Hybrid automata are intended to be used to model all components of hybrid systems, including sensors, actuators, physical components, controllers, computer software, communication services, and humans that interact with the rest of the system [21].
We model the flexible Platoon as a hybrid system which exhibits a combination of discrete and continuous behaviour [20]. Platoon systems are often modelled as platoons of identical vehicles, not accounting their mass, motor characteristics, brake quality, tire pressure and friction, etc.
The examined platoon of vehicles is made of a leader (L) and two followers (FV) and (BV), Fig. 1. The number of vehicles in the platoon changes as vehicles enter and exit the platoon.

Figure 1 String of Platoon vehicles
We suppose that a vehicle can be involved in only one manoeuvre at a time and that each vehicle has the ability to communicate with other vehicles in order to send and receive data. The autonomous vehicles can use three manoeuvres: -vehicle join at the end of the column (this requires coordination with the leader of a platoon), -vehicle merge in the middle of the column, where the coordination is necessary with all vehicles in the platoon, -vehicle leave at the end or from middle of the Platoon, where the coordination is necessary with all vehicles in the platoon.
We can create more complex platooning scenarios using those basic manoeuvres.
The hybrid model of a flexible Platoon consists of the following modules: Leader Module, Cooperative Module, Control Follower Module 1 and 2 and Cooperative Behaviour Scenario Module, all presented in Fig. 2.
Leader Module consists of model leader's vehicle dynamic. We suppose mathematical model of longitudinal motion of the vehicles [9].
Cooperative Module is realized through the finite state machine (FSM) and together with Control Followers Module 1 and 2 defines a hybrid automaton. Each platooning manoeuvre between the leader and the followers is coordinated through status changing in this module. The progresses through a series of states are in response to various events during the manoeuvres.
As control approach in scope of Control Modules of Followers, the proportional-integral-derivative (PID) approach is used.
Behaviour Scenario Module is realized using finite state machine, and it selects the time varying different behaviours of the vehicles. The input values for Cooperative Module are: parameters of the joining, merging and/or leaving of individual follower vehicles from Behaviour Scenario Module, and speed and distance of the leader and first follower. Based on this data, this module makes the settings of status flag (SF1 and SF2) and relative distance, speed and acceleration of individual followers by specific joining, merging and/or leaving manoeuvers.

Vehicle Dynamic Modelling
For vehicle dynamic modelling, we prefer mathematical model of longitudinal motion of the vehicle [9]. Application of Newton's second law on vehicle moving in the x and z directions, effects of aerodynamic force D A and the rolling resistance force R x , give the next equation: where G = mg is the weight of the vehicle; F x is the tractive force. The subscripts r refer to the rear tirereaction forces. All the vehicles have the same numerical values, Tab. 1.

Control Follower Module
For linear system synthesis, there are numerous control approaches: proportional-integral-derivative (PID) controller, Linear Quadratic Regulator (LQR), adaptive optimal control, fuzzy control, etc. With presumption that the dynamic behaviour of the object with respect to the nominal trajectory is linear, the PID control approach is used in scope of Control Modules for first and second vehicle followers.
We generate reference acceleration a 0 , velocity v 0 and position x 0 of the Road Train leader.
These values go through Cooperation Module and then to the PID controller where they are processed according to [9]: Control force F x is determined by a PID controller, i.e. with Eq. (2). By substituting (2) in (3), it generates the acceleration for the i th vehicle. The following equations represent linear state space model of the i th vehicle in the platoon: where hd i is constant distance between (i-1) th and the i th vehicle, variables x i-1 , v i-1 , a i-1 of the previous ((i-1) th vehicle) are input variables for the i th vehicle. MATLAB model of flexible Platoon is presented in Fig. 3. Leader module and Control Follower module in MATLAB/Simulink are presented in Fig. 4 and Fig. 5 respectively.

Behaviour Scenario Module
The Cooperative Scenario Module generates on/off signals for followers 1 and 2. This is the basis for the functioning of the Cooperative module. Simulation starts with activating of leader vehicle. The leader vehicle maintains a speed of 80 km/h and can send a signal to other vehicles to increase or decrease the speed. During the simulation period, the follower vehicles turned on and off. If first vehicle exits from Platoon, leader sends signal to the back follower to accelerate in order to maintain a constant distance between vehicles.
One example of behaviour scenario in the Platoon is presented in Fig. 7.

PLATOON COOPERATION BEHAVIOUR PROFILE GENERATION
In order to improve the traffic mobility in Intelligent Transport systems, through better prediction of Platoon manoeuvres dynamics, we propose the next coding function: where c 1 and c 2 are constants, n k are vehicles notation for k = 1,…, N vehicles and are behaviours notation for j = 1,…, M behaviours. In this way, the complex behaviour interactions in Platoon are presented with only one variable.
One example of coding transformation more Platoon behaviour patterns in time into generated cooperation profile, is presented in Tab. 2. In this case, c 1 = c 2 = 1 and n 1 = 1, for leader vehicle, n 2 = 2 for first follower, n 3 = 3 for back follower. If we have for example two behaviours (j = 2), then we have the next notations: b 1 = 4 for deceleration and b 2 = 8 for acceleration in time.
The vehicle speed profiles and generated behaviour profile of the flexible Platoon are highlighted in Fig. 8, respectively. Table 2 Example of cooperation profile function generating in time Notation for vehicle number (k=3) n k (t) = (n 1 , n 2 , n 3 ) We assume that two vehicles are moving in a Platoon: (n 1 , n 2 , n 3 ) = (1, 0, 3) and function value is F(t, k) = 4. In next moment, the new vehicle sends a signal to the leader (LV) for merging to the Platoon. After receiving the signal, the leader sends approval of entry to the new vehicle and sends a signal to the existing vehicle for temporary deceleration (b 2 = 4), making place for new vehicle in the middle of the Platoon: (n 1 , n 2 , n 3 ) = (1, 2, 3). The function value is F(t, k) = 10.
With different parameters and function setting, we can choose behaviour profile generation.
If we suppose that Road Platoon consists of more vehicle agents, we can propose another coding scheme, where one part is Harrison-Riddler Series: ( , ) ( 1) ( ), f t n n n b t = − + (9) We introduce behaviour notation: b 1 = 100 for deceleration and b 2 = 200 for acceleration. For example for the leader vehicle and 6 followers and deceleration in Platoon, behaviour profile has the value: f(t, 7) = 142. For 8 vehicles totally and acceleration in Platoon, f(t, 8) = 256.

PREDICTION OF COOPERATION BEHAVIOUR PROFILE
An improvement of traffic mobility in Intelligent Transportation Systems is reached through better prediction models, since it reduces the uncertainty of future traffic states. There is a lot of research in the field of traffic speed, flow and travel time predictions, but none in the area of cooperation vehicles behaviour. In this paper, we propose prediction of Platoon cooperation manoeuvres dynamics using NARX neural networks.

NARX Structure
NARX networks is a methodology which has a good ability to handle successfully the modelling of nonlinear dynamic systems. They have proven to be more effective in modelling than other classical autoregressive predictors, and that is the main reason why they are commonly used in many applications including timeseries modelling. They are primary dynamic neural networks with feedback [22]. The general structure of the NARX network, used in our research, is presented in Fig.  9. Specific window with the number of input values I(t-i), and previously generated outputs O(t-i) represent input to NARX network, while the predicted value for the next time interval O(t) is the output of the network. The relation:

Data Set
The data set we used for cooperative profile prediction in ITS consists of 3201 samples obtained from several simulation scenarios. All samples have both inputs and corresponding output values. Input features of the NARX network include time series for speed values for the leader, first and second follower, while the output of the network is defined as the road cooperation of the Platoon expressed through the hybrid automaton values. Available data set from simulations, is divided into training data set (70% of samples for establishing predictive model), validation data subset (15% of samples for tuning the parameters of the network), and testing set (15% of samples tor testing the quality of the proposed methodology).

Prediction Criteria
In all experiments performed, a one-time-intervalahead prediction is considered. The performance of a trained network is measured by the errors on the training, validation, and test sets, including also the correlation coefficient R between the outputs and predicted value [23]: where Y i , ˆi Y and f i are the experimental, forecast and target values, respectively, and n is the total number of the test data. Starting from the neural network structure presented in Fig. 9, the optimize NARX architecture is obtained by tuning the parameters such as number of hidden nodes and the size of time window (number of inputs I, and lagged outputs O will be used in the prediction model). The optimal number of hidden nodes is 5, while time lag window has 4 elements. Tab. 3 shows the results of MSE and R 2 for three different cooperation scenarios. In each three scenarios R 2 is greater than 0.99, which means that the network made prediction well. MSE and R 2 results for one example of behaviour scenario 1 are presented in Fig. 10 and Fig. 11 respectively. Prediction results of three different behaviour scenarios are presented in Fig. 12. Prediction results of Platoon behaviour scenario 2 for cases without noise and with noises in test data are presented in Fig. 13. This NARX predictive model has proven quality for prediction of cooperative behaviour in Platoon using a very small number of epochs.

CONCLUSION
Problems with the urban traffic require new technology based solutions for improving road capacity and reducing vehicles congestion. The nonlinear autoregressive network (NARX) is used in this research to perform modelling and forecasting synchronized behaviour of vehicles organized in the flexible Platoon. The NARX strategy overcomes the drawback and weaknesses of other autoregressive predictors. After tuning up the network structure by repeated experimental trials, the testing phase shows promising results expressed through the mean-square-error and correlation assessment measures.
We used MATLAB/Simulink/Stateflow software tools for the assessment of the cooperation between the vehicles in the Platoon. Predictive model of a Platoon behaviour based on NARX network and control mechanisms of the vehicles based on the novel hybrid automaton model are a core component of our cooperative intelligent transport system (C-ITS). This integrated approach shows promising results, and we believe that further research may result in successful artificial intelligence based system applicable in real world situations such as urban traffic congestions.   Figure 13 One example of predicted behaviour scenario in Platoon for cases: a) without noise; b) with noised test data We are excited by possibilities for many directions in future work, which comes somewhat naturally since the mechanisms introduced in this article are only initial ideas with some experimental results based on samples from simulations. One of the main research directions should include new experiments using real laboratory vehicles (mobile robots) evaluating the performance of the vehicles' cooperation in a real world environment. Analysis of the proposed methodology with larger number of vehicles' agents will enable discussion about scalability of the proposed approach and directions how to adjust the methodology. In particular, one can envision that the system may help the operator in traffic centres to organize control actions in the Platoon.