Simulation Research on Driving Behaviour of Autonomous Vehicles on Expressway Ramp Under the Background of Vehicle-Road Coordination

: Constructing a risk model with the subject of autonomous vehicles to screen out the vehicles of potential conflicts and analyze their choices under different strategies. Based on the co-simulation of Python and SUMO, establishing a model of on-ramp merge driving behaviour of autonomous vehicles based on non-cooperative static game. Under this model, the experiment results that the average speed in the merging area is increased by 12.7%, the standard deviation of the average speed is reduced by 35.46%, and the number of the vehicles successfully merged before the end of the merging area is 4.86 times that of traditional method, indicate that the model can effectively help the vehicles be merged and improve the traffic efficiency to a certain extent.


INTRODUCTION
With the continuous advancement of science and technology, the number of car ownerships has been increased year by year, the efficiency and safety of driving has also been increased.According to statistics, from 2014 to 2021, the average number of traffic accidents in the country has reached more than 150,000 each year [1].In order to solve these problems, traffic practitioners continue to try and work hard.With the development of computer technology, mobile communication technology, big data, artificial intelligence, deep learning, AIoT and other new technologies have brought new ideas and new methods to solve this type of problems.At present, the road traffic system is undergoing a new transformation, showing the trend of intelligence, networking and collaboration.Vehicle-road collaboration technology has become the hot spot and frontier of international intelligent transportation research and practice.One of the effective methods to improve road traffic capacity and reduce parking delays is speed guidance.The vehicle infrastructure cooperation (VIC) system can guide the vehicle's trajectory according to the current traffic flow and operating state [2].Jianqiang Wang and Huimin Niu proposed a distributed dynamic route guidance system based on simulation by using the latest data collection and communication technology in the vehicle-road coordination system, and verified the practical applicability of the system [3].RL Sakhapov and RV Nikolaeva collect and fuse data on roadside conditions, vehicles and pedestrians by using various sensing means such as radar, cameras, and geomagnetic induction coils, and classify the collected data to ensure the real-time and completeness of the data [4].In order to alleviate the problem of blocking traffic flow at signalized intersections, An Shuke and Xu Liangjie proposed a speed guidance decision method for signalized intersections based on vehicle-road coordination technology.The method can eliminate the sudden change of velocity and has certain advantages [5].Baigen Cai proposed an optimal control method for unsignaled intersections based on vehicle speed guidance and information interaction in a vehicle-road coordination environment.Compared with the traditional driving control, this method can effectively reduce the average delay at the intersection, the number of stops and the queue length [6].In the development process of autonomous vehicles, the environmental perception system has certain limitations in the processing of road traffic scene information, and cannot achieve efficient driving under all road conditions.It needs the assistance of vehicleroad collaborative perception technology to operate more safely and efficiently.Vehicle-road coordination technology has become the current research hotspot and future development direction of the intelligent transportation industry.
According to statistics, the driving efficiency of urban expressways has greatly increased, but the driving efficiency and safety issues have also increased.Among them, the conflict in vehicles competing for the right of way at the ramp entrance is an inevitable problem.At the same time, the entry of vehicles at the ramp will affect the vehicles on the main road within a certain range.It is of the greatest significance to improve the safety and efficiency of the entry of vehicles at the ramp to improve the driving efficiency of urban expressways.One of the main reasons for the congestion of expressways is the merging process of the ramps entering the expressway.Although the collision during the merging process is less serious than the accident caused by the conflict at the intersection, the high frequency of occurrence and the large traffic disturbance have been caused [7].The essential research on autonomous driving behavior simulation behavior training data is crucial to the research of autonomous driving.However, autonomous driving systems learn diverse driving strategies that are limited by complex and diverse traffic scenarios in the real world.The most primary is that autonomous vehicles always take the most conservative and inefficient decisions in dangerous situations [8].In order to alleviate congestion and ensure safety, Ioannis A and Ntousakis studied combined paths and proposed a longitudinal trajectory planning method, which maximized the workload of the engine and the discomfort of passengers [9].Wenjing Cao and Masakazu Mukai took safety as the control goal, and proposed a gentle merging path generation method based on MPC.Under reasonable conditions, vehicles can complete the merging at the optimal point, but the study did not take into account unintended disturbances and disturbances from vehicles in the main lane [10].Clark Letter and Lily Elefteriadou taking time as the control objective, based on the vehicleroad collaborative environment, a longitudinal highway merging control algorithm is proposed, which can provide a safe merging operation in congested traffic conditions [11].Liu Chang and Zhuang Weichao established a vehicle longitudinal dynamics model for the collaborative confluence control at the ramp entrance with the goal of energy efficiency and ride comfort, and verified that the model can effectively improve traffic efficiency and reduce potential safety hazards [12].The current research on onramp merging considers the merging conflict analysis under the influence of drivers at the micro level [13], data collected shows that the set is also dominated by nonautonomous vehicle information [14].Game theory is used to study co-opetition behavior, and it has great applicability to solving the problem of merging between vehicles [15].Kita proposed to use game theory to solve the problem of merging traffic conflict firstly, and he pointed out that the merging traffic is not independent of each other, but it has a relationship of mutual influence, he established a twoplayer non-cooperative game model and used the conflict time as the profit function to finally solve the optimal strategy [16].Non-cooperative games represented by the Stackelberg game are widely used to solve conflict problems in CAV scenarios.Peng Hang and Chao Huang proposed a game decision-making framework considering the safety and efficiency of the vehicle as the control goal, and verified its advantages in improving the efficiency of the transportation system [17].Wang Qiuling, Zhao Xiangmo competed for the priority attributes of special CAV tasks and constructed a two-person cooperative game to determine the optimal confluence sequence, solved the vehicle trajectory control, and verified special CAV vehicles with cooperative control can effectively shorten the merging time and reduce fuel consumption [18].Cheng Ying, in his research on coordinated control of intersection conflicts, establishes a game model, takes speed change as a strategy, and takes driving safety, efficiency and comfort as benefits, using solved Nash Equilibrium as the optimal strategy which shows that the conflict can be better alleviated by verifying the model.However, the state analysis of conflicting vehicles is lacking in terms of driving safety benefits [19].Qu Dayi establishes a noncooperative game model based on the lane-changing behavior of autonomous vehicles in mixed traffic flows, and uses its own driving status as the game income to seek better lanes.Combined with SUMO to conduct simulation experiments, it is verified that the model has higher lane utilization, safety and stability [19].Linghui Xu and Jia Lu developed a cooperative merging strategy based on the vehicle-road collaborative environment with the goal of minimizing the travel time of on-ramp vehicles and maximizing the number of merging vehicles on the ramp, and verified the safe and efficient merging of CACC vehicles [21].Based on the optimization framework of multiplayer cooperative game, Shoucai Jing and Fei Hui coordinated the CAV entry ramp merge, decomposed the global problem into the minimum cost of different strategies to achieve the Pareto efficient solution of the game and determined the optimal sequence, and the verified algorithm with potential real-time execution ability, but the research does not consider the vehicle kinematics and its accuracy has room for improvement [22].Researchers at home and abroad have widely used game theory in the field of transportation, and have achieved fruitful results in alleviating traffic conflicts.In research, less consideration is given to vehicle kinematics, and there is room for improvement in accuracy.
Therefore, the paper establishes a driving strategy control model in the merging area.Firstly, the state of the vehicle is detected within the effective communication range under the background of vehicle-road coordination, and the risk assessment model is established with the speed and distance in the information of the current state of the vehicle as indicators.Then, vehicles with a certain degree of risk are preliminarily screened to improve the control efficiency of the model, and to determine the preference of the strategy selection tendency as the degree of risk increases to enhance the accuracy.Secondly, based on safety benefits and efficiency benefits, combined with vehicle kinematics, the differences in safety benefits under different states of vehicles are specifically analyzed, and a non-cooperative static game model is constructed.Finally, the validity of the model is verified by simulation experiments.The experimental results show that the model has a positive impact on the average vehicle speed in the merging area, which can effectively alleviate the largescale fluctuation of speed, reduce the number of stops during operation, and improve the efficiency of vehicle merging.In the future, it will provide an effective reference for the driving strategy control of autonomous vehicles in merging sections under the background of vehicle-road coordination.

RISKMODEL 2.1 System Structure
The signal ramp intersection control system in the vehicle-road coordination environment is mainly composed of a CACC vehicle equipped with on-board communication equipment and a traffic control center.The CACC vehicle can obtain real-time information such as speed, acceleration, distance with the preceding vehicle, and speed difference during the driving process.According to the collected information, the control strategy is formulated and transmitted to the CACC vehicle.

Model Building
Taking the merging vehicle at the ramp entrance as an example, in the vehicle-road coordination environment, the vehicle can locate other vehicles and its own position, and the vehicle can confirm the speed and acceleration of the surrounding vehicles.For the convenience of the study, it is assumed that there are no other obstacles near the junction.The judgment of the risk degree of two vehicles depends on the current position and motion state of the vehicle, which is embodied as two indicators of relative distance and relative speed for analysis [23].
The expression function of relative distance risk is: , The motion relationship diagram of the two vehicles is Technical Gazette 29, 4(2022), 1402-1412 established as follows: L nm is the relative distance between the conflicting vehicles, D n , D m are the safety distances when driving vehicles n and m in emergency braking.It can be known that it is related to the speed, acceleration and maximum braking acceleration of the current state.The calculation process is divided into three stages, driver response stage, braking coordination stage and deceleration growth stage, when the vehicle brakes in emergency, which should ensure a certain distance d rather than zero distance from the potential conflict target, and the value range of d is generally within 3 -6 m [24], the calculation method of the safety distance is: It can be seen that the smaller the relative distance between the two vehicles, the greater the danger.
The expression function of relative speed hazard is: This function indicates that the higher the current speed, the higher the risk.
W 1 and W 2 are weight coefficient of distance and speed hazard which is the specific value.The higher risk symbolizes the greater potential conflict of vehicles.When the risk is greater than the threshold R, the vehicle is defined as a potential conflict vehicle, and the speed control model for the conflict road segment is activated.Since people's driving strategies will have different preferences as the degree of risk increases, when the perceived risk is small, the drivers tend to pay more attention to their own efficiency gains.When the perceived risk is gradually increasing, the driver's driving strategy tends to deviate in the direction of increasing safety benefits, so risk is also considered as a factor affecting driver decision-making [25].

GAME DRIVING MODEL OF CONVERGING ROADS 3.1 Player Set and Strategy Analysis
Two vehicles in a potential conflict compete for the right of way in the conflicting road section which constitutes a clear competitive relationship, M(M = {1, …, Taking a single game as an example in the research process, the game subject is the vehicle to be merged n, the lagging vehicle m on the main road, the strategy of the vehicle in the merging lane is {merge, wait}, and the strategy of the vehicle on the main road is {avoid, not avoid}.

Analysis of the Degree of Choice Preference
The benefit of a potentially conflicting vehicle is not only related to the running state of the vehicle, but also to the degree of conflict.In the merge conflict, the main road vehicle wants to pass the ramp intersection without interference as much as possible, and the ramp intersection vehicle enters the main road as soon as possible under the premise of ensuring safety.Therefore, whether it can pass safely and the efficiency of passing constitute the main objectives.If there is a potential conflict in the prediction, which is returning to the acceleration of the next frame at the current speed and maintaining the specified position will be dangerous, and the safety benefit at this time will be damaged.It is very necessary to consider the impact of different conflict degrees on the benefit in the research.
When a potential conflict is predicted, the control behavior of the vehicle is affected by the degree of danger, which is the greater the perceived danger, the greater the safety gain, and the proportion of the safety gain will increase at this time.On the contrary, when the perceived danger is smaller, more attention will be paid to the efficiency gain [25], the proportion of efficiency benefits is larger at this time, and the selection preference value is positively related to the risk degree.When the risk degree is greater than the threshold, the preference for safety benefits will show a positive growth trend, and the preference value conforms to the following functional relationship, , 0, is the risk

Analysis of Potential Conflict Points
The potential conflict point is the intersection of the main road vehicle and the merging vehicle's journey during the merging process.Let the coordinates of this point be (x c , y c ), and the coordinates of the vehicle to be merged successfully completed is (x e , y e ).Assuming that the vehicle is driving in the center of the target lane after successfully changing lanes, y e can be obtained directly, and the value of y c can be obtained at this time.
L car is the width of the vehicle, y c , y e are the Y-axis of the potential conflict points and merge completion points.
At present, most of the research on the vehicle lane change trajectory is carried out by using cubic polynomials.However, the acceleration is not continuous.Then a quintic polynomial is used to describe the trajectory of lane changing when the automatic driving merges in this paper [27].The general formula of its motion trajectory when it merges laterally is: x t a a t a t a t a t a t Among them, a i , b i ; i = 1, 2, 3, 4, 5 are all unknowns.In the process of trajectory research, it is assumed that the lateral acceleration, lateral velocity and lateral displacement of the start and end points of the trajectory are zero, after solving and derivation, the automatic driving import trajectory is Among them, , v v are the speed of the vehicle entering the starting state and the ending state, respectively, t e is the time when the vehicle completes the entry, and h is the lateral displacement of the entry process.The coordinates of (x c , y c ) can be obtained from the expected lateral velocity of the initial state and the final state and the expected time for the completion of the merger.After obtaining the coordinates of this point, the distances of the vehicles to be merged and the lagging vehicles from the conflict point can be obtained.
If the trajectory of the vehicle to be merged is a curve, its distance from the potential conflict point is: Since the integral cannot find the original function composed of elementary functions, it cannot be solved directly.Therefore, the numerical integration method is used to solve it.The Simpson method has a simple logic structure and only one difference node is added to the most commonly used trapezoidal formula solution method.However, the algebraic accuracy has been improved by 3 times, so Simpson's method is used to approximate the solution, so that   , then there are: Among them, L m is the distance from the current position of the vehicle to be merged to the potential conflict point, and t c is the time required from the current position to the potential conflict point.
The distance between the lagging vehicle and the potential collision point, Among them, x c and x m are the abscissa of the potential conflict point and the abscissa of the current position of the lagging vehicle on the main road.

Analysis of Acceleration Selection Under Each Strategy Combination
The strategy of the lagging vehicle on the main road is {avoid, not avoid}, and the strategy of the vehicle to be merged is {merge, wait}, so there are the following four strategy combinations {merge, avoid}, {merge, wait}, {wait, avoid}, {wait, no avoidance}, the acceleration that should be taken under the combination of the above four strategies is one of the key issues.On the basis of examining rational decision-making, maximization and optimal decision-making should be pursued.Simon pointed out that due to the limitations of knowledge, skills, time, etc., decision-makers cannot always take all issues into consideration.In the context of vehicle-road collaboration however, due to changes in information, cognition, and unknown, the decision-maker's choice is often not optimal under the existing conditions.Therefore, this paper proposes a satisfactory solution.The acceleration model is studied with the speed of the leader car in the target lane as the desired speed.

The Strategy of Merge and Avoid
Under this combination of strategies, the vehicle to be merged chooses to merge, and the vehicle lagging on the main road chooses to avoid, then, the vehicle on the main road will choose a relatively gentle braking rate to increase the time gap between the two vehicles which the merging vehicle reaches the conflict point, the lagging vehicle will keep a safe distance from the merging vehicle, and the vehicle to be merged will also choose a relatively gentle acceleration to drive into the main road, the merging vehicle will reach its desired speed when it reaches the potential conflict point.
The acceleration of the vehicle to be merged in this area at this time max , a m represents the acceleration of the vehicle to be merged at the current moment, v leader represents the speed of the pilot vehicle of the vehicle to be merged, v m represents the current speed of the vehicle to be merged, T is the time fixed value, which means the time threshold for maintaining a safe state, t m represents the time threshold of the vehicle be merged, that is，the expected completion time for the merged vehicle.The parameters are the same as those repeated later.
The acceleration indicates that the merging vehicle will take the speed of the leading vehicle in the target lane as the desired speed.If the current acceleration cannot meet the speed of reaching the speed within a certain period of time, the acceleration will be updated.
Lagging vehicle acceleration in this area: L n represents the distance from the lagging vehicle to the potential conflict point, and L represents the safety distance that the lagging vehicle needs to reserve.The parameters are the same as those repeated later.The acceleration indicates that the lagging vehicle keeps a certain safety distance behind in order to ensure the smooth merging of the merging vehicles, that is, when the merging vehicle reaches a potential conflict point.

The Strategy of Merge and Not Avoid
Under this combination of strategies, the vehicles to be merged choose to merge, and the lagging vehicles on the main road choose not to avoid them.Then the vehicles to be merged will adopt their maximum acceleration to merge in order to reach the potential conflict point before the lagging vehicle arrives.At this time, the main road lagging vehicle chooses to drive at the current acceleration or pass at a higher acceleration.
The acceleration of the vehicle to be merged: Acceleration of the lagging car: max ,

The Strategy of Wait and Avoid
Under this combination of strategies, the vehicle to be merged chooses to wait and the vehicle lagging behind on the main road chooses to avoid it.After driving for a period of time, the vehicle to be merged will receive a positive signal to invite merging while driving.After this state occurs, the vehicles to be merged will not always choose not to wait for the next opportunity, but a certain efficiency gain will be lost at this time.
The acceleration of the vehicle to be merged: The acceleration of the lagging car is the same as Eq. ( 13).

The Strategy of Wait and Not Avoid
Under this strategy, the vehicles to be merged choose to wait, and the vehicles lagging on the main road choose not to avoid them.The vehicles to be merged will ensure that the acceleration they choose can drive normally in the merging area, and will not be forced to stop or take extreme braking rates.
Then the acceleration of the merging car: m L' represents the distance from the vehicle to be merged to the end of the merging area.Acceleration of the lagging car: max , To sum up, the acceleration selection of the merging vehicle and the main road lagging vehicle under each strategy is shown in the following table.

Analysis of Game Profit Function
On the basis of determining the acceleration, a profit function is established with the safety benefit and efficiency benefit of the vehicle as indicators.The safety benefit is based on the time difference between the two vehicles arriving at the potential conflict point as the main research object [28].The larger the time difference is, the safer it is, and the smaller the time difference is, the more dangerous it is which the two vehicles arrive at the potential conflict point.Efficiency benefits take the speed change before and after the decision as the research object.
For the time analysis of the vehicle reaching the potential conflict point, it is known the speed and acceleration of the current state can get the time to reach the specified distance from The calculation method of m is the same as n, and the time difference between them is: Therefore, the income of the two cars is a function of a n and a m under the selection strategy.
i, j corresponding to different strategy.
The distance from the potential conflict point at the current moment n/m The time to reach the conflict point at the current moment with the current speed and the selected acceleration ΔT n/m The time difference at time t, that is the difference between the time between the two vehicles and the conflict point T M Time setting, indicating the time difference that should be kept in a safe state α Preference value for safety gains The final benefit obtained by moving at the current speed and the acceleration corresponding to the selection strategy at time t The return matrix is as follows:

Decision Analysis
The ultimate goal of the vehicles to be merged and the vehicles lagging on the main road participating in the game is to maximize their own interests within a predictable range.The respective benefits of the two players not only depend on their own strategic choices, but also on the opponent's strategy choice in this game model.It is a typical game problem in which strategy and interests are independent.However, when they are choosing a strategy, even if the actual choice of the other party cannot be known, the impact of the other party's choice on its own income cannot be ignored and must be considered which other party has two possible choices, and under the circumstances that different choices affect their own interests differently, they should make their own best strategic choices.According to this idea, the dash method is used to solve it.However, in a specific game, there may be specific uncertain situations which more than one benefit array with a line under each number.In this case, in order to improve the convergence speed of the game, the strategy of the highest total value in the income portfolio is selected as the optimal strategy.The strategy selection for competing at time is as follows:

SIMULATION ANALYSIS 4.1 Construction of Simulation Platform
This paper adopts SUMO and Python co-simulation.
SUMO is an open source, microscopic, multi-modal traffic simulation software, which can simulate a single vehicle passes through a road by itself to meet the traffic demand, each vehicle has its explicitly modeled, own routes, and moves independently in the network.Using Python to connect to the Traci interface, access the running road traffic simulation scenario, get the values of the simulation objects, send simulation step commands, and advance the simulation to the next step to control the simulation and vehicle behavior.
The simulation construction scene is the entrance of the ramp intersection, the main road section is two-lane, 1.2 kilometers l, and the length of the confluence area is 300 meters, the ramp entrance is a one-way road, a total of 250 meters, and the maximum speed of the main road section is 120 km/h, the maximum speed of the ramp is 50 km/h.According to previous research and experimental tests, the parameter valuesare initialized, and the weights of the relative risk are w 1 , w 2 , w 3 = 1/3, the risk threshold CR = 0.3; the parking safety distance is 3 meters, time Technical Gazette 29, 4(2022), 1402-1412 threshold T = 4 seconds, k is 1, and assume that the total time of the vehicle from the initial position to the completion of the merger is 7 seconds.According to the composition of the traffic flow and the loop model of the vehicle, the simulation is divided into two groups: group A and group B. Group A and group B adopts the CACC model (cooperative adaptive cruise control) commonly used in the current automatic driving research.There are four control modes: speed control mode, a distance control mode, distance proximity control mode and anti-collision control mode.But vehicles with certain risk adopts the game model of this paper in group A. The simulation process is shown in Fig. 5.According to the density range of the main road of 800 -2400 and the density of 400 -1000 on the ramp, 36 sets of simulation tests were carried out on groups A and B.

Comparative Analysis of Speed Curves
Compared with the single-vehicle speed curve controlled only by CACC, the speed curve has obviously less large-scale deceleration behaviors using CACC combined with game model algorithm control.Fig. 6 shows the single vehicle speed curves before and after the algorithm control of 8 main road vehicles and ramp vehicles in the simulation operation.The speed of the main road vehicles shows the operation law of first deceleration and second acceleration.The speed curve decreases to a certain extent under the control of the game algorithm.Vehicles to be merged on the ramp generally have two processes of deceleration, and even cause parking at the first deceleration.The improvement effect is more obvious in the case of extremely low vehicle speed, and the process of the second large deceleration can be basically eliminated.
Fig. 7 and Fig. 8 are the distance-time curves of the main road vehicle and the on-ramp merging vehicle with speed in the merging area.The first one is the curve only controlled by the CACC model, and the second one is the curve controlled by the game algorithm.Since the algorithm in this paper does not strive to control all vehicles to be merged and the main road lagging vehicles, but to control the vehicles to be merged with a certain degree of danger, the figure represents not the distancetime curve of the algorithm-controlled vehicle alone, but all the vehicles in the confluence section within a certain period of time, all vehicles in the merging section during the time period, so it can be seen that the controlled vehicles also have a certain degree of influence on the uncontrolled vehicles.In the curve of group A, the speed of the vehicle is low when entering the merging area even to parking.After using the game algorithm control, the speed fluctuation is significantly reduced, and the vehicle can pass through the merging area in less time, and also has a certain degree of positive effect on uncontrolled vehicles

Analysis of Vehicle Motion Data in the Merging Area
It can be seen from Tab. 1 that the average speed of vehicles in the entire merging area has increased by 12.7%, the standard deviation of the average speed has decreased by 35.46%, the maximum speed is the set maximum speed of 27.78m/s, and the average minimum speed has increased by 50.12% which is consistent with the speed control curve of individual vehicles.There are clear indications that the vehicle speed under the control of the game algorithm is basically maintained at a high level in Fig. 9 and Fig. 10, but with the increase in the density of main roads and ramps, although the average vehicle speed does not reach the ideal vehicle speed, there is still a certain degree of improvement compared to the average vehicle speed using the CACC model, and the effect of optimization is more obvious.Since 400 -2 000 did not produce merging behavior, the merged data of this group was excluded, the average merged position was advanced by 19.37%, and the number of merging completed by the group B controlled by the game algorithm before the end of the merge area was 4.86 times that of group A.
During operation, group A generates a certain number of vehicles parked when the traffic density on the main road is too large, such as 1000 -2000, 800 -2400, 600 -2400.These groups generate a large number of vehicles parking in group A which can be effectively controlled using game control algorithm, and the phenomenon of parking will not occur basically, that is, the traffic flow on

Figure 1
Figure 1 the figure of vehicle movement m}) represents a collection of driving vehicles.  i A i M  represents the driving vehicle i in the system.B i indicates driving vehicle exceeds the risk threshold within the recognition range of the driving vehicle A i .A i action set the time required for n to reach the specified potential conflict point can be obtained.

Figure 6
Figure 6 individual vehicle speed curves comparison

Table 1
Acceleration selection table

Table 2
Model parameter tableSymbol Implication v i , i = n, mn/m speed at the current moment a i , i = n, m n/m When choosing the acceleration corresponding to a certain strategy

Table 4
Strategy selection table