Improving the Diversity of PSO for an Engineering Inverse Problem using Adaptive Inertia Weight

: Particle swarm optimization is a stochastic optimal search algorithm inspired by observing schools of fishes and flocks of birds. It is prevalent due to its easy implementation and fast convergence. However, PSO has been known to succumb to local optima when dealing with complex and higher dimensional optimization problems. To handle the problem of premutature convergence in PSO, this paper presents a novel adaptive inertia weight strategy and modifies the velocity update equation with the new Sbest term. To maintain the diversity of the population a particular radius r is introduced to impulse cluster particles. To validate the effectiveness of the proposed algorithm, various test functions and typical engineering applications are employed, and the experimental results show that with the changing of the proposed parameter the performance of PSO improves when dealing with these complex and high dimensional problems.


INTRODUCTION
Recently various new types of optimal algorithms have been used for solving complex multimodal and advanced optimization problems such as artificial bee colony, Ant colony, Evolutionary programming, Genetic algorithm, etc.In all of these optimal algorithms the population is randomly initialized; this is the reason all of these techniques are considered as heuristic optimization algorithms.The optimal algorithm has the capability to solve complex and multimodal optimization problems due to the mutation process and the presence of randomization in its steps.Currently, optimization algorithms are applied in different research fields, however among all of these optimization algorithms, particle swarm optimization has become very popular due to its easy implantation and its attractiveness in engineering and other research areas [25,18,24].According to the 'no free lunch' theorem optimal algorithms depend on the optimization problem.Some methods can only find the best solution of a specific problem, but not all problems.The main weakness in traditional PSO is that it cannot reach the region of the global optimum while dealing with more complicated and higher dimensional optimization problems.Because of the loss of diversity as well as the absence of proper balance between local and global searches, particles are attracted to local optima, which are called premature convergence.
To handle the problem of premature convergence in PSO different researchers have made different modifications to PSO to improve its performance.
The inertia matric and the two learning parameters play an imperative role in the searching process.PSO algorithm has shown performance improvement when the values of these three parameters are varied.In the PSO the values of these basic parameters are not properly specified.To achieve balance between the exploration and exploitation searches and aid the particles in escaping from local optima, adjustment of these three control parameters (w, c1, c 2 ) to suitable values is required [1,2].It is possible that someone uses wrong values for the basic parameters and as a result all the PSO converge prematurely.
To solve this issue this paper presents a modified basic PSO that brings a new parameter Sbest to the velocity update equation.The proposed modified PSO keeps balance between the local and global searches and it maintains the particles motion during the search process thereby helping the particles to escape from the local optima.The proposed strategy helps the algorithm to move toward the global optimum space during the process.
The remainder of the paper is explained in the following sections as follows.The second part explains previous research work.The proposed PSO and new modified strategy for inertia weight are explained in the third part.The comparative study of different PSO optimal algorithms and simulation results are discussed in the fourth section.The fifth section provides summary of the research.

PREVIOUS RELATED RESEARCH WORK
To facilitate the understanding of the proposed modification, this section will review some related works of the fellow researchers.
Generally, the constant parameters values are not suitable in the whole optimization process as present in the basic PSO, and adjustments are compulsary from time to time [2].In order to control the global and local search abilities, a new strategy was presented [3,4].A new method was developed for the w [18].According to the new strategy the value of w (inertia weight) is dynamically adjusted based on the absolute value of velocity of the particles which can control velocity from stagnation.The inertia matrix is randomly varied in a dynamic environment [5].To enhance the performances of the traditional PSO, the control matrix (w) is set dynamically according to the objective function value in [6].A nonlinear decreasing control matrix strategy was proposed [7], in order to control the diversity of the swarm in the evolution process, avoiding the premature convergence.To enhance the performance of PSO, the inertia weight of every particle is dynamically varied by euclidean distance between the global best particle and individual [8].To keep a good balance between the global and local searches of the particles a new strategy was proposed in [9].The central idea of the strategy was that inertia weight value is not linearly controlled by using tangent function.In Xiang et Technical Gazette 25, 6(2018), 1631-1637 al. a novel adaptive PSO algorithm with no linear inertia weight and time varying acceleration coefficient was designed in order to solve the logistic distribution centre location problem [10].
However, various scholars were developing the velocity update equation and introducing new terms for the basic PSO process.Jiamain introduced a new parameter 'Ebest' in velocity update equation, to tackle the problem of premature convergences of PSO while dealing with complex and multidimensional problems, 'Ebest is the value randomly chosen in every dimension [11].The basic velocity equation is modifed by new term in order to solve complex and practcial problems [12].In [14] medianoriented PSO was proposed.The new MPSO improves the global mechanism of the search process over the entire local search space in order to discourage the premature convergence in the PSO process.A local search strategy was introduced in the PSO in order to maintain the diversity of particles and escape the particles from local optima [15].
Chaoli designed a new fitness estimation strategy in order to reduce the computation time and to control premature convergence in the PSO process [17].
Qing Jian Ni et al. presented a Logistic dynamic particle swarm optimization (DPSO) in [18].Further, he claims that population topology has a significant influence on the PSO performance.An un-uniform mutation-based and sub gradient method were introduced into the traditional PSO in order to enhance the particles' search abilities and to control the diversity of the swarm [19].In [20] the single stage and multi stage evolutions were proposed as modifications of the traditional PSO.In single stage, the sub swarms share information multistage, the particles search independently, and the two stages control premature convergence in PSO.Hai-bin Ouyang et al. designed a hybrid harmony search particle swarm optimization with global dimension selection (HHSPSO-GDS) [22].The central idea of his mechanism is to modify the basic velocity update equation by introducing new global best particles for improving neighborhood search regions of the current best solution.
Zhaohui Ruan et al. described the relationship between the population density and search capabilities of PSO [21].According to their strategy the particles are divided into two classes of functions.Thus Ruan proposed an approach using multi-function global particle swarm optimization on the basis of his strategy in order to find the global best particle in the evolution process.

PROPOSED APPROACHES
In the current research paper we modify the control parameter (w) and introduce a new parameter into the velocity update equation.

A New Approach to Inertia Weight
A proper inertia matrices value is important for keeping stability between the neighbourhood and overall search abilities during the process.As the control metrics value in the traditional PSO is constant which will facilities the particle to stick in one point and the searching of the region will be stopped in the evolution process.
This results in premature convergence.To overcome this difficulty we propose an adaptive control metrix and adjoin the new parameter Sbest to the velocity update equation using a mutation mechanism.The modification helps the particles to escape the local optima and preserves diversity of the population during the optimization process.The new strategy for the adaptive inertia weight is characterized by the following equation:  ,  ( ) According to previous work, a small value for the inertia matrices enhances exploitation and a large value enhances exploration.In the above strategy, the value for inertia weight during the optimization process is different for every test function; the inertia weight achieves its maximum value when the global best value is larger than the personal best value, and the value of inertia weight is small, or when the personal best fitness value is greater than the global best fitness value.
At the beginning of the optimization process the particles have maximum diversity while at the final stages diversity of the particles is reduced so that balance between the local and global search abilities is achieved.
It is clear that in the search process diversity of the particles is increased at the initial stages, and is reduced at the end of the search process thereby disturbing the balance between the local and global search abilities.The proposed variation in the inertia weight value addresses this issue.Because at every iteration the gbest and p best values are varied, considerable balance is maintained between the exploration and exploitation searches.

A New Approach to the Velocity Update Equation using Mutation Mechanism
As M. Clerc and other researchers have proved, PSO is convergent and, in studying PSO, experiments have shown that the velocity of all particles slowly tends to 0. All of the particles at once converge to a single point which is, generally, a local or global optimum.The cognitive part is mostly 0 when the particles converge to this optimal position.During this stage, the particle's movement is dominated by the social and inertia factors which are generally decided by the previous social factors.Now, to avoid premature convergence while keeping the particle population active, we adjust the relationship between a particle and the global best particle.Repulsion force, i.e. the phenomenon of the distance between a particle and the optimal position being smaller than a particular radius, can be the basis for avoiding premature convergence.This is a strategy for maintaining diversity of the population.The modification uses the following strategy to update the position and velocity.1.When the distance between a particle and global best particle is less than a particular radius r use the basic PSO velocity update equation for the particles.2. Otherwise use the following: S best is the global best fitness value which is chosen randomly from the g best values at every generation during the search process such as shown in Fig. 1.Fig. 1 shows the MPSO process for finding the s best value during the search process.The optimization problem consists of one variable (X 1) .There are six particles (each particle is represented by different colours that show its position, and has reached the kth iteration (current iteration).Like the PSO algorithm, the S best value for the population can be obtained for each iteration to update the next particle position.The given diagram explains the g best value process and how the global best particle values vary in the search process.In this example, at the m th iteration, the population has the G best value which is achieved by the 'blue' particle.Therefore, at the m th iteration, the G best values for each dimension are X 1m-blue , X 2m-blue and X 3m-blue .Also, the G best value is obtained by the "light purple" for the current value of iteration.Thus, the G best value is represented by X jklight purple, where j = 1, 2, 3 (number of dimensions).Further, we proposed a new strategy for c 3 parameter which ranges from 0 to 1.The mathematical equation for c 3 is given by the following equation.
The c 3 attracts the particles in the vicinity of S best , which helps the proposed approach to control the premature convergence of PSO.

EXPERIMENTAL RESULTS
The performance of the proposed approach is verified on the well known test function.The detail description of the test functions is presented below in Tab. 1.Other researchers have already used these test functions for testing the performance of optimal algorithms [13,27,28].All the problems have zero optimal solution.

Parameter Setting
In order to check the performance of our proposed modification we performed extensive experiments in Matlab 2013 version.We compared our modified PSO with ARIWS [5], ANLIWS [24], Adaptive IWS [23], BPSO [1], Global PSO [14] and IPSO [13].All the optimal algorithms are tested and validated on unimodel and multimodal test functions and also for typical engineering problem team workshop 22.During the experiments all the optimal algorithms have set with the same number of swarm size 30, same number of dimension 10, and same number of iteration 2000.The values for the inertia weight, cognitive constant and social constant are considered according to [5,24,13,12,23,1].Every algorithm runs 50 times independently and the average values are listed in Tab. 2.
Result and discussion: If we compared our proposed modified PSO with inertia weight strategies, we observed that our proposed modified PSO is type one for the test functions f1, f 3 , f 5 , f 7 , f 8 and f 11 .
From Table 2 we conclude that ARIWS shows the best performance for the test functions f 3 , f 4 , f 5 , f 6 , f 7 , f 8 as compared to ANLIWS and ADIWS but not outside of class compared with MPSO.The ADIWS has shown the worst performance for most test functions so it is considered as type three optimal algorithms.Similarly results of the optimal algorithms are divided into three categories.The optimal algorithm whose average value is near the optimal solution as compared to other optimal algorithms is considered as type one algorithm.The algorithm shows little significant improvement among the other ones considered as type two.The optimal algorithm that has shown the worst performance as compared to other algorithms is considered as type three algorithms.Viewing the above discussion our new improved PSO is type one algorithm specially for the test functions f3, f 5 , f7, f8 and Technical Gazette 25, 6(2018), 1631-1637 f 11 ,.For the test function f 2 our modified PSO is considered as type three algorithm.IPSO and BPSO show out class performance for the test function f 9 as compared to GPSO but not better than MPSO.As for the test function f 10 the performance of GPSO is good as compared to the IPSO and BPSO but not better as compared to MPSO.So GPSO is considered as type two algorithms for test function f 10.BPSO, IPSO and GPSO placed in the third category for the test functions f 1 , f 2 , f 3 , f 4 , f 5 , f 6 , and f 7 .As from the above discussion it is obvious that proposed modified PSO shows best performance for most test functions.

Convergence Plot Comparison
To verify the performance of the proposed PSO, we compared the convergence graphs of different optimal algorithms.The convergence characteristics of different methods were given in the following Figs.2-6.The following are some typical observations if we compare the convergence plots of our proposed PSO with the BPSO, IPSO and GPSO and other well known inertia weight strategies.From Figs. 2-6, our proposed PSO finds the optimal solution at the beginning of the search process for most test problems as compared to the IPSO, GPSO and BPSO.Observing the convergence plot of the test function f 1 , our PSO exactly converges to the global optimal solution after 200 generations while other optimal algorithms cannot find the optimal solution during the whole process.As for the test function f 2 our improved PSO finds the optimal solution during the initial stages of the evolution process.Also PSO and ANLIWS show good performance while adaptive IW and ARIW inertia weight strategies and other optimal algorithms cannot find the optimal solution during the optimization process.As we observe the convergence plot of the test function f 3 and the convergence characteristics of the test functions f 7 and f 10 we conclude that the modified PSO converges to the global optimal solution at the initial stages of the optimization process while the well-known inertia weight strategies and other optimal algorithms are not capable to find the global solution during the whole search process.
From the convergence plot it is demonstrated that the performance of the new method is the best one as compared to other algorithms.

Application
For the verification of our suggested MPSO for solving the engineering problems, we will have to use it for solving an engineering design problem.The problem which we have chosen is the TEAM workshop problem 22 [27,28].It is an optimization problem having 3 different parameters, about optimization of the configuration of a Superconducting Magnetic Energy Storage (SEMS) as shown in Fig. 7.This problem is commonly used in electrical engineering, especially in computational electromagnetic studies, to test different optimal algorithms.There are three different aims regarding the configuration of the SMES device.1) the energy stored in SMES device must be 180 MJ; 2) there should be no violation of the quench condition by magnetic field; and 3) the system must be able to eradicate the stray field on its maximum.These objectives are modelled as the following   For decent comparison, we fixed equal parameters for the maximum generation of 50, swarm size of 15, and the dimension of 3, and c 1 and c 2 of 2. We ran each algorithm five times and the average objective function values are recorded in Tab. 3. The results show that the performance of the proposed PSO is much better if compared to the other two PSOs.

CONCLUSION
In the current work we proposed a new methodology for the velocity update equation and developed a new method for the control matrices (w).The variation in the control matrices provides local positions and regions in the search process while the mutation mechanism facilitates exploration search.In this way a proper correspondence between the candidates was established.The proposed approach is validated on mathematical test functions as well as on the engineering inverse problem.The numerical results and convergence characteristics curve illustrate that the improved PSO performs well as compared to the other ones.

Figure 1
Figure 1The random selection of S best particle at every iteration

Figure 2
Figure 2The convergence plot comparison of different optimal algorithms for test function f 1 (y-axis)

Figure 3 Figure 4 Figure 5 Figure 6
Figure 3The convergence plot comparison of different optimal algorithms for test function f 3 (y-axis)

1 hFigure 7
Figure 7The schematic diagram of a SMES )in which E is the energy stored in the form of magnetic field inside the SMES device; E ref = 180 MJ; B norm = 3uT; J i is the current density and (B max ) i is the maximum magnetic flux intensity in the ith coil; 2 stray B is estimated along with 22 equal distant points along line a and line B, as shown in Fig.7

Table 1
Mathematical test functions

Table 2
Comparison of different optimal algorithms on 10 dimensional problems

Table 3
Comparison of optimal algorithm on TeamWorkshop 22 problem