Current State of Dynamic Vehicle Routing Problems Solved by Ant Colony Optimization Algorithm

: Dynamic V ehicle R outing Problem is a more complex version of Vehicle Routing Problem, closer to the present , real - world problems. Heuristic methods are used to solve the problem as Vehicle Routing Problem is NP - hard. Among many different solution methods, the Ant Colony Optimization algorithm is proven to be the efficient solution when dealing with the dynamic version of the problem. Even though this problem is known to the scientific community for decades, the field is extremely active due to technological advancements and the current relevance of the problem. As various sub - types of routing problems and solution methods exist, there is a great number of possible problem - solution combinations and research directions. This paper aims to make a focused review of the current state in the field of Dynamic Vehicle Routing Problems solved by Ant Colony Optimization algorithm, to establish current trends in the field.


INTRODUCTION
Vehicle Routing Problem (VRP) was first introduced in 1959 by [1], its goal is to construct an optimal solution, i.e. find routs with minimal cost, travel time, or environmental impact, for multiple vehicles that are visiting n number of nodes (also called locations or customers). If the problem is reduced to one vehicle, it is called the Traveling Salesmen Problem (TSP). VRP is considered a static problem as all information is known before route planning. In contrast with VRP, in the Dynamic Vehicle Routing Problem (DVRP) all relevant information is not known before the planning process begins, and information can change after initial routes have been planned. Dynamic elements include dynamic demand (new customers, canceling orders, changing node location or time window, etc.), dynamic road time (real-time traffic conditions may change vehicle speed). In [2] DVRP is classified by dynamic elements and their characteristics into three categories. Dynamic Demand based VRP (DDVRP), Real-time traffic information based VRP (RTVRP), and Dynamic demand and real-time traffic information based VRP (DDRTVRP) as the most complex problem of DVRP.
DVRP was first introduced in 1980 in [3], and since then it has been a topic of scientific interest, as it is closer to realworld problems than VRP. Between 2000 and 2015 number of scientific papers in the field of DVRP has rapidly grown due to the development of information and communication technologies. [4] Scientific interest continued to this day, as scientists tackle different aspects of VRP and DVRP. Although it may be that the VRP field was never more active there is still much work to do. Logistics 4.0, supply chain 4.0, as well as the technologies that support them, are rapidly evolving as part of the fourth industrial revolution. Industry 4.0 is defined by digitalization, networking, and a higher level of automation in order to achieve better productivity and efficiency. It is expected that in the near future the level of digitalization will rise even higher and that companies, suppliers, and users will work together in a fully digital environment. [5] The dynamic vehicle routing problem is being taken to a new level, as information can be collected in real-time, processed a decentralized way, and in the cloud. All that makes DVRP a very attractive topic for both academic and industrial circles.
VRP is considered an NP-hard problem, so DVRP is also considered to be an NP-hard problem. Computational problems are grouped into classes how complex they are, or how much computational time is consumed to solve specific problems. If n represents the size of the problem, in VRP case number of nodes, P class of problems can be solved in polynomial time (i.e. n 2 ), and NP class of problem can be solved in nondeterministic polynomial time (i.e. 2 n ). The computational time of the NP problem increases much faster with the size of the problem than the P problem. NP-complete class of problems is the hardest class of NP problems, and NP-hard class of problems are problems that are at least hard as NP-complete problems. Considering that VRP is an NPhard problem, it means that optimal solution cannot be calculated in a time acceptable for practical use by exact algorithms [6]. Because of that, finding an optimal solution for both problems often requires the use of heuristic algorithms. Among lots of different heuristics approaches Ant Colony Optimization (ACO) algorithm or its variants is considered by many authors to be a reliable, efficient, and overall good choice for solving DVRP, due to its ability to adapt to dynamic changes. This is because of its inherent ability to memorize past optimal solutions via its artificial pheromone model. [7] Ant Colony Optimization is a sub-type of Swarm Intelligence (SI). The first ACO algorithm, called Ant System (AS) was introduced by M. Dorigo in 1996., it was applied to classical TSP. AS was inspired by real ants which use pheromone trail as a communication method. [8] ACO or any of its variants, mimics the self-organizing behavior of ants searching for food. Real ants lay pheromones to mark their path from the nest to the food source and back. If there is more than one way to reach food, ants will randomly choose a path. A shorter path will have a stronger pheromone trail due to the fact there was less time for pheromones to evaporate. The next ant, when confronted with the possibility to choose between two different paths, will have a higher chance of choosing a path with a stronger pheromone trail. In ACO, artificial ants can solve complex combinatorial problems with high-quality solutions. ACO algorithm was designed with static combinatorial optimization problems in mind, but it turned out to be well suited for dynamic problems as well, especially because dynamic problems may be considered series of static problems if the working day is sliced into a number of time intervals. [7] As the scientific field is decades old and very active, there is a great number of research directions, possible combinations of dynamic vehicle problems, and solution methods. The very active scientific field is making it inconvenient to accurately overview current advances, trends, and the possibility of further research. Extensive research is needed to determine current trends, research directions, and the best possible method for a given problem. To reduce the complexity of the problem, this paper will focus on one solving method of dynamic routing problems.
The goal of this paper is to make a focused review of the current state in the field of Dynamic Vehicle Routing Problems solved by some variant of Ant Colony Optimization algorithm, to establish current trends and best solution method for given problems. As the field is very active, to establish the latest trends, papers published since 2018 were reviewed and categorized into several categories. Each paper was analyzed to determine what type of DVRP was the subject of the paper, if the authors introduced a new version of the routing problem, what method was used to solve the named problem (i.e. was basic ACO algorithm used or a new variant of the algorithm was developed), was the problem solved by using the ACO algorithm alone or if it was combined with some other method (hybrid approach), were Industry 4.0 (I4.0) technologies integrated into the problem, if the authors considered the environmental impact, and if the authors proposed innovation in the evaluation system for DVRP.
In section 2, relevant literature is reviewed, the first part consists of review articles and in the second part are original scientific articles published since 2018 in chronological order. In section 3, the categorization of reviewed literature is visually represented in table form, and current trends are discussed. In section 4 concluding remarks are presented.

LITERATURE REVIEW
As VRP and DVRP are problems decades old, there was a great number of review articles, some more important are listed below.
In [9] authors classified routing problems, introduce the concept of degree of dynamism, and reviewed applications and solution methods for DVRP. As most researched problems are theoretical, simplified routing problems, socalled Rich Vehicle Routing Problems (RVRP) have emerged. RVRP tends to simulate real-life objectives and constraints, but the true definition of RVRP remains somewhat vague. To tackle that problem, in [10] authors provide generic taxonomy of RVRP literature and propose the definition of RVRP. In [4] authors made a taxonomy of DVRP papers according to the eleven criteria, analyzed the influence of advances in information and communication technology on DVRP. In [7] authors analyzed the behavior of ACO algorithms specifically designed for dynamic combinatorial optimization problems. A case study of using these algorithms on dynamic traveling salesman problems is conducted to investigate the effect of different features in dynamic situations. In [2] authors classified VRPs, (among which is DVRP), a mathematical model for each type is presented as well as an analysis of the solution methods.
To the author's knowledge, a focused review of dynamic vehicle routing problems solved exclusively with a variant of the Ant Colony Optimization algorithm has not yet been conducted.
For finding relevant literature, i.e. original scientific articles published since 2018., Web of Science, Scopus, Springer, and Scient Direct databases was searched as well as general search using Google Scholar. The search was conducted with different combinations of keywords: dynamic, vehicle, routing, problem, the Boolean operator AND, and keywords: ant, colony, optimization, system.
Papers are ordered chronologically by year they are published, but it should be noted that the month of publication was not considered.
In [11] authors solve classic DVRP with the new variant of ACO algorithm called enhanced Ant Colony Optimization (E-ACO). The primary goal of E-ACO is to avoid falling into local search prematurely by improving the degree of randomization. ACO is modified in three ways by combining ACO with K-means, crossover operation, and 2-Opt. First, using the K-means algorithm to divide the whole problem into smaller, compact regions. Then ACO is used to solve each region separately. The initial solution is mutated with a crossover operation, that originates from the Genetic Algorithm (GA), in order to avoid falling in the local optimum. Finally, a 2-Opt local search heuristic is applied to optimize routes and as another way to avoid falling into the local optimum. The second contribution of this paper is the design of a mfair and impartial estimation for DVRP by adding -test to the evaluation system.
In [12] authors solve the Dynamic Multidepot Vehicle Routing Problem (DMDVRP) with Hybrid Ant Colony Optimization (HACO). The goal of HACO is to improve randomization to avoid falling into the local optimum prematurely. ACO is modified in three steps: first by dividing DMDRP with clustering algorithm into smaller scale DVRPs. After the ACO algorithm is used to generate the initial solution, mutation operation (derived from GA) is applied to randomize the solution to avoid falling into the local optimum. The 2-Opt heuristic is finally applied to improve the route. To deal with new customer requests authors split working day into 24 same time intervals. In each time interval, new customers are added to the nearest existing route, after which routs are reoptimized.
In [13] authors introduce a new variant of the routing problem called Dynamic Green Vehicle Routing Problem (DGVRP). In the dynamic environment, authors try to reduce greenhouse gas emissions, produced by a set of vehicles. The problem is solved by the hybrid ACO and Large Neighborhood Search (LNS) algorithm.
In [14] authors present a mathematical model of a new variant of the routing problem the multi-tours DVRP with overtime. The problem is solved with a hybrid approach. After a solution is created by Ant Colony System (ACS), the best solution is optimized with a local search algorithm. Finally, new data sets for testing algorithms are proposed.
In [15] authors solve Asymmetric Dynamic Traveling Salesman Problem with the proposed MAX-MIN Ant System (MMAS) algorithm with a build-in short-term memory. Improvement in the algorithm is used to accelerate the convergence of a best so far solution.
In [16] authors exchanged framework of populationbased Ant Colony Algorithm (P-ACO) with Adaptive Large Neighborhood Search (ALNS) to solve classic Dynamic Traveling Salesman Problem (DTSP). A hybrid algorithm is then tested on the real-life problem of surveying deer with drones.
In [17] authors introduce a more complex variant of DTSP in which existing nodes are replaced with new ones. Also, the authors propose two novel restart strategies, local random restart strategy (LRRS) and local restart strategy (LRS) in combination with hyper populated ant systems to solve the named problem. The new variant of the ACO algorithm has shown increased practical efficiency in solving dynamic problems.
In [18] authors analyze the performance of MAX-MIN Ant System combined with unstringing-stringing (US) local search and 3-opt local search, to solve Asymmetric and Dynamic Travelling Salesman Problem with Moving Vehicle (ADTSPMV). The authors state that MMAS-US is the best algorithm for solving ADTSP, but for moving vehicle variant MMAS-3opt is better.
In [19] authors propose a new demand coverage diversity adaptation method abbreviated ACO-CD because it is based on the ACO framework. A new metaheuristic is used to effectively respond to the new customers in the classic DVRP while keeping low traveling costs. This is achieved by maintaining a diversity of customers which ensures that there is always an existing customer next to the new one without knowing the location of the new customer in advance. The effectiveness of ACO-CD is verified by comparing solution results with the basic version of the ACO algorithm.
In [20] authors take into account steep and slow roads (information about roads is obtained using Google Maps) while searching for an optimal solution via ACO for Dynamic Capacitated Vehicle Routing Problem (DCVRP) in order to reduce CO2 emissions. The routing model is developed primarily with Latin America in mind where the population, the topography of big cities, and height above sea level are higher than average, but model usefulness is not limited to that area only.
In [21] authors consider multi-source multi-destination traffic routing problem (MSMD traffic routing problem) in the Internet of Vehicle (IoV) environment. Specific to this problem is that individual vehicles randomly enter road networks while having their unique source-destination combination. The problem is solved by a new ACO variant where the algorithm is improved by considering path distance, vehicle density on the paths, and travel time of the vehicles.
In [22] authors propose a hybrid algorithm named BSO-DVRP to solve classic DVRP. Brain Storm Optimization in objective space (BSO-OS) is used to decide between ACS and ALNS algorithms to create new solutions. BSO-OS randomly chooses one or two existing solutions. If one solution is chosen, then ALNS is applied to generate a new solution. If two solutions are chosen, they are used by ACS as the initial solution to update pheromones and create a new solution.
In [23] authors propose how to heuristically repair the solution given by MMAS when dynamic change occurs on classic DVRP. Ant System has the inherent ability to memorize past solutions via pheromone trail, but if the dynamic environment drastically changes it could be better to reinitialize the pheromone trail rather than transfer knowledge from the previous optimal solution. After the dynamic change occurs, the unstring-string heuristic is used to repair past optimal solutions to remove affected nodes from the solution and add new nodes.
In [24] authors propose the integration of Game Theory (GT) into the ACO algorithm in order to improve performance when dealing with classic DVRP. After the dynamic problem is reduced into series of static problems by dividing the workday into time slices and acquiring all data related to the nodes, GT and ACS are used to decide which node will be visited next.
In [25] authors make use of the Internet of Things technology to determine which waste collection bins are full and need to be visited. New nodes (waste bins) appear dynamically, and the problem is solved by the classic ACO algorithm.
In [26] authors propose pairwise proximity learningbased ant colony algorithm (PPL-ACO) which predicts the customers' orders to get optimal routes before change happens. This approach needs fewer computing resources and enables solving large-scale DVRPs. The effectiveness of the proposed algorithm is greater on large-scale DVRPs because it cannot achieve high learning accuracy on smallscale problems.
In [27] authors propose and validate a framework to solve DVRP with time windows (DVRPTW). Among seven algorithm variables used to validate the framework, a hybrid version that combines multiple ant colony systems with random variable neighborhood descent is proven to have the best performance based on test results.

DISCUSSION
Reviewed papers were categorized into eight categories defined below. Papers included in categorization are [11] - [27]. All of those are original scientific articles published since 2018, written in the English language, that solves DVRP with some variant of the ACO algorithm. Subject review, books, and dissertations were omitted from categorization as well as papers that deal with a static variant of the problem (VRP). Most authors deal with a known variant of DVRPs, classic DVRP is dominant through papers. Several new variants of the problem were introduced, namely Dynamic Green Vehicle Routing Problem [13], multi-tours DVRP with overtime [14], and the new variant of DTSP which replaces existing nodes with new ones [17]. Because the academic community is often interested in solving theoretical problems, they mostly deal with idealized models. Technological advancement in the last two decades has led to an increased number of papers that deal with RVRP that try to simulate the complexity of real-life problems. [10] The absence of real-life, rich routing problems solved by ACO is noticeable.
Papers published in 2018 and 2019 often use a hybrid approach i.e., combine ACO algorithm with other algorithms and mathematical models like Genetic Algorithm, Large Neighbourhood Search, 2-Opt, 3-Opt, and Game Theory for solving DVRP. After 2019 lots of papers are focused on the development of new variants of ACO algorithms.
Although the fourth industrial revolution is in full swing, and there are mentions of the fifth industrial revolution, only [21] and [25] include possibilities offered by I4.0 technologies in DVRP. I4.0 technologies (i.e. real-time tracking of the vehicles and road conditions, communication between vehicles, and predictive analysis) could enable the creation of more realistic RVRP by integrating previously unavailable information in the problem. Two papers, [13] and [20] are trying to reduce environmental impact using ACO algorithms on DVRP.
The effectiveness of newly proposed solving methods is superior when compared with other methods in the field, but they are usually compared with basic variants of other methods. While some papers are introducing innovation in evaluation systems, [11] is trying to design a more equitable evaluation system for DVRP, while [14] propose new data sets to test the algorithms, but it seems that a universal evaluation system has not yet been crated.
The ACO algorithm has proven to be an efficient solution for many different dynamic vehicle routing problems. Even though the review was focused, due to many different combinations of problems and solutions, and the nonexistence of a universal evaluation system, it is difficult to conclude which variant of the ACO algorithm is best for solving a particular variant of the dynamic problem. Similarly, universally best variant of the ACO algorithm, even if there is one, is still not possible to determine.
Regarding future research authors of reviewed literature mostly suggest further improvements on algorithms, testing other heuristics on the same problem, tackling more complex dynamic problems, and applying proposed algorithms on real-world problems.

CONCLUSION
This paper offers a focused review of the Dynamic Vehicle Routing Problem solved by the Ant Colony Optimization algorithm. Papers published since 2018 have been categorized to determine current trends in this field. Most of the papers deal with classic DVRPs, but some new variants of the problem are being introduced. Hybridization and improvement of the ACO algorithm are very popular, and results are promising when compared with basic versions of other solving methods.
The absence of a universal evaluation system is making it difficult to accurately evaluate and rank solution methods. The best solution method is not possible to determine until there is a universal evaluation system, at least for a single variant of a given problem. Also, there is no comparison of the solution given by the proposed algorithm and the best possible solution.
Environmental impact and integration of industry 4.0 technologies are not discussed in many papers.
Possible future research could be even more focused i.e., finding the best possible solution method for a specific type of dynamic problem. But to accomplish that, a universal (at least for a specific type of problem) evaluation system needs to be established. Integrating new possibilities offered by I4.0 technologies could lead to the creation of more complex DVRPs similar to real-life problems.

Notice
The paper was presented at MOTSP 2021 -12 th International Conference Management of Technology -Step to Sustainable Production, which took place in Poreč/Porenzo, Istria (Croatia), on September 8-10, 2021. The paper will not be published anywhere else.