IMPROVEMENT OF THE MAINTENANCE MANAGEMENT PROCESS OF COMPLEX TECHNICAL SYSTEMS WHICH DEMAND HIGH RELIABILITY

Preliminary communication The main objective of this research was to develop a procedure for assessing the performance of maintenance functions in organizations that are responsible for maintaining systems that require a high level of reliability; a procedure which, with minor changes, could be applicable in other organizations, as well. The process included consideration of factors that affect the success and performance of maintenance functions. The research was carried out in Aviation Maintenance Organizations in the region of South-eastern Europe.


Introduction
Today, almost everyone is familiar with famous words of W. Edwards Deming: "It is not necessary to change.Survival is not mandatory."But, if one wishes to survive, change or improvement becomes mandatory.The only problem is how to decide what and how to change.Most of the world's leading organizations accepted that there is and always will be space for improvement in each area of their businesses.In this case, a new problem arises: how to identify problems-areas of most beneficial improvement, and decide which problem to tackle and improve.It is impossible to tackle all of the problems at once, because there are not enough resources for that (time, human resources, money...).In this situation, it would be wise to listen to the motivational speaker Brian Tracy and his Law of Forced Efficiency: "There is never enough time to do everything, but there is always enough time to do the most important thing".So, we find ourselves in a situation where we have to develop or adopt a new system of designing, selection and prioritization of possible activities / improvements.
The maintenance function cannot be separated from the rest of the organization, so it should be subject to improvement, as well.There are a lot of different approaches to the maintenance improvement [1][2][3][4].Sometimes, it is a department-wide or even companywide movement, like in TPM -Total Productive Maintenance [5] / Lean Maintenance [6] / Autonomous Maintenance [7,8] (rather extensive, although a little outdated literature review about TPM can be found in [9]).Sometimes, it is about believing in and following the good practice in the industry -for example: adoption of condition based maintenance (CbM) policy.In [10], (among other sources with the same data) it is stated that the cost of reactive maintenance is US$18 per installed Horse Power per year, the cost of preventive maintenance US$13/HP/year, cost of predictive maintenance / CbM US$9/HP/year and cost of Reliability Centred Maintenance US$6/HP/year.So, this data could be the guiding idea for maintenance improvement.But, one has to bear in mind that although CbM is proven to be superior to corrective or preventive maintenance, it is not economically justified for low cost parts/failures, small organizations, and it requires investment in time, data, skilled workers and equipment (evidence for that can be found in [11]).
According to the Allied Consultants Europe report [12], 57% of all European organizations have adopted lean philosophy (and lean maintenance as well) and another 20% plan to do the same, which means that at least 77% of European organizations are focused on a management concept with continuous improvement built in its core (since this report is from the year 2008, it can be expected that present percentages are even higher), so it is wise to think how to find a way to improve maintenance constantly.Not a revolutionary, but an evolutionary change, i.e. a systematic approach including small-step improvements, is something proved to be a safe path to excellence or a safe path to the market survival.
Each systematic improvement approach includes the following four steps: (i) measuring of some important indicators (Key Performance Indicators -KPIs), (ii) determining the gap for those indicators, (iii) designing and implementing the measures for gap closing and (iv) checking whether the gap is closed -whether the measure is effective.This approach is present in ISO 9001 (PDCA cycle or Deming cycle or Shewhart cycle), Six Sigma (DMAIC), Toyota's A3 technique for problem solving, 8D process (pioneered by Ford Motor Company who called it TOPS -Team Oriented Problem Solving), etc.
The first step is the necessary development of KPIs that will enable measuring of each important aspect of maintenance effectiveness (Lord Kelvin once said "If you cannot measure it, you cannot improve it").Maintenance effectiveness measurement is one of the most persistent problems in the industrial engineering, just because it is extremely difficult to define those KPIs that will actually comprehend every important aspect of maintenance effectiveness.In the last 50 years, there have been many attempts to define those KPIs.One of the first attempts (to our knowledge) was presented by Priel [13] -20 maintenance performance ratios (today, we call them KPIs).After that, the European Federation of National Maintenance Societies -EFNMS, presented their system including 29 Maintenance Benchmark indices (GMARI -Global Maintenance and Reliability Indicators).The weakness of this approach is that there exist many indicators and it is almost impossible to say which organization / maintenance is better if the values of their indicators are similar.Also, it is almost impossible to define the direction of improvement based only on the values of those indicators (sometimes, local improvement means global worsening).It would be beneficial if all those values were somehow combined in one final value and compared.One approach to solving this problem was presented in [14] where the Data Envelopment Analysis was used for that purpose.Furthermore, that source presented the possibility of development of a matrix, which combined the KPIs and possible activities for improvement of a particular KPI, as it was said in [15]: Performance measurements, when used properly, should highlight opportunities for improvement, detect problems, and help find solutions.
In [5], Seiichi Nakajima introduced a KPI that is very popular nowadays: OEE (Overall Equipment Efficiency), which measures how an organization takes care of its equipment.Although OEE results in one final value, comparable between different organizations, the problem is that OEE does not include some important factors (e.g. money invested in improvements, staff trainings...).
The idea of determining the complex KPI (inherent health and integrated health of the equipment) of the equipment maintenance care quality, presented in [16] by using data acquired from integrated sensors, is really promising.Unfortunately, at this stage of development, the concept is offering "measure" only for one machine separately, and there is no system for "measuring" the whole production line or the whole factory (hopefully, it will be developed in the near future).
One interesting approach to maintenance optimization is presented in [17], where the combination of the Monte Carlo simulation and genetic algorithms is used.The paper is focused on the optimization of the maintenance costs, reliability, availability and accident costs, while having in control maintenance interventions frequency and the number of maintenance teams.Although the authors were not focused on improvement of a maintenance function as a whole and did not consider the possible quantum leap effect of knowledge based improvement, this approach could be used for that purpose, but it could be that it is too complex to be realized.
One of the recent and promising innovations in maintenance approach is risk-based maintenance [18] where risk management technique is used to "measure" criticality of possible outcomes and, accordingly, the appropriate action taken.Benefits of this approach could be found in [19].
Some continuous improvement initiatives are focused on one specific parameter important for maintenance, reliability, such as those shown in [20][21][22], which are focused on reliability in aviation industry, or even on parameter combination, as presented in [23], in the review of the life-cycle reliability-based optimization field with emphasis on civil and aerospace structures.
One problem with defining the KPIs is that it is sometimes impossible to find data for a specific phenomenon in the organization database.In that case, the only way to "measure" the phenomenon is to rely on the expert's subjective opinion.In that case, it is possible to present KPIs as questions in questionnaire.In Terry Wireman's book World Class Maintenance Management [24], the author presented the first version of his questionnaire for "measuring" maintenance effectiveness.The newer version could be found in [25] (in 2014, the author published the third edition of this book).In the first version, the questionnaire had 8 sections with 10 questions each.Each question was worth maximum 4 points, which means that maximal number of points was 320.The author defined the top 90% as a world class maintenance management.In the newer version of questionnaire, the author incorporated 16 sections with 10 questions, 4 points each, with the same logic about world class.The latest version of the questionnaire is used for this research.
The second step in the systematic approach to improvement is focused on the comparison of achieved results (values for KPIs) with the best-in-class or the best possible.Unfortunately, maintenance function is specific, and it is impossible to establish the best possible result or even the best-in-class.Each maintenance function has some specific circumstances or some areas that are valued more than other maintenance functions and because of that it is impossible to establish a maintenance function as the best-in-class.But, if the blind eye is turned to that, it can be accepted that the imperfect measuring system will give one result as the best-in-class, so the comparison can be done with it.
A widely accepted way of comparison between organizations is Benchmarking [25][26][27], and this approach is used for this research.

Maintenance improvement model
Because of the extensive data and mature maintenance system needed as a basis for developing this model, it was decided to conduct the research among maintenance organizations dealing with complex systems which require high reliability (it is expected that those organizations have mature maintenance management systems).An aircraft is a perfect example of a complex system which requires high reliability.Aircraft maintenance programs are laid down by the manufacturer.Strict regulations for obtaining aircraft maintenance permission, as well as numerous requirements of international aviation authorities, mainly from the EU and the United States (EASA -European Aviation Safety Agency and FAA -Federal Aviation Administration), set very strict criteria for organizations which maintain aircrafts and their components.
By studying the current scientific developments in the field of maintenance, it has been noticed that there is a gap in the area of maintenance function analysis within the Aviation Maintenance Organizations -AMO (maintenance within maintenance).Various authors dealt or are dealing with problems of the aircraft maintenance organization [28], the problem of maintenance planning optimization [29,30], planning, scheduling, staff training, and the impact of human error in aircraft maintenance [31][32][33], needs for spare parts [34][35][36][37] and maintenance of aircraft [38][39][40].Additionally, there are no papers that analyse the performance of maintenance functions within these organizations.
The operation of this function is vital to the work of these organizations, and therefore, the improvement of this function and the work quality is in line with the aircraft maintenance improvement.By improving the efficiency of their maintenance function, AMO provide better quality service in less time and with less expense.
The research subject was the analysis of the key elements in the maintenance of AMO, using the method of self-assessment.In this research, an existing questionnaire was used for the assessment of maintenance functions [25].
During research preparation, the authors provided access to three organizations that deal with aircraft maintenance, from Serbia and Bosnia and Herzegovina.Due to required confidentiality, those organizations will be referred to as Company A, Company B and Company C in this paper.
The first step in the research was the initial selfassessment according to Wireman's questionnaire.The initial results are presented in Tab. 4, in the columns headed by I.It is assumed that difference between achieved and maximal results is the "space" for the improvement.Those differences are presented in the same table, in the column headed by II.
During the initial result analysis, a shortcoming of the Wireman's questionnaire was analysed.The questionnaire treated each section with an equal number of points (40 points), without taking into account the area of operations of the organization, its real need for a piece of equipment, tools, level of training of the staff, including its financial capabilities, etc.
It is this lack of flexibility of the questionnaire to be adapted to the needs and capacities of organizations (without compromising maintenance quality) that led to the need for devising ways to enable organizations to look at their priorities realistically and determine the actual state of maintenance functions.

Determining the significance factors
To determine the significance factors that will be used to determine the "weight" of a particular area, FMEA analysis was chosen, an analysis which has also been used in the automotive and aerospace industry for many years.This analysis has been used for the preventive detection of possible failures and for identification of potential delays in the production process.In the literature sources this method (or any of its forms) has never been mentioned (according to our knowledge) as a tool used in order to improve system performance analysis tool of maintenance in organizations, regardless of their business area.
Standard FMEA analysis is used to try to determine the possible failure modes (product or process), and the possible effects that arise in the event of the failure.For each of these effects, severity, occurrence and detection are determined.Then, for each of the identified failures, the Risk Priority Number (RPN) is calculated.
In this method of using the standard FMEA analysis, negative consequences are observed, i.e. the negative effects caused by non-performance or inadequate performance of certain tasks.In this way, the process improvement is achieved by elimination of possible critical points.
During the development of a model to determine the success of maintenance, the possible positive impact of maintenance function was emphasized, not the negative one.By using such a model, instead of identifying possible failure modes and their negative effects, it is possible to see all the improvements and their effects on the maintenance function.In this way, organizations are able to eliminate current and future negative impacts, but also to project the desired state of maintenance functions in the future.The reason for this approach is that the focus on failures restricts the ability to notice possible improvements for something that is already good (it has been said many times that the worst enemy of great is good).
To achieve this goal, FMEA analysis had to undergo some changes, as well as the recommended FMEA form [41].This amended FMEA analysis is entitled "Improvement Modes and Effects Analysis -IMEA".A detailed explanation of the changes in the form and procedure for improvement determination follows below.
The form used for FMEA analysis has undergone most changes so it could be adapted to the needs of this research.
The main part of the adapted form keeps the basic concept of FMEA analysis with some modifications.This section is presented in tabular form and is divided into 9 columns: 1) No.; 2) Part / Function; 3) Potential improvement; 4) Potential effects of improvement; 5) Maintenance goals affected by improvement; 6) Benefit; 7) Potential obstacles in improvement implementation; 8) Implementation effort; 9) BPN -Benefit Priority Number The column No. represents the number of the monitored part of an organization or process.
In the column Part / Function, the name of the functions of an organization or of its observed parts is to be written, for example Maintenance of some smaller organisational part.The areas defined in the questionnaire were used in this section.
Potential improvement is the improvement that the organization wants to implement in the given function or the part of the organization.In conducting the analysis, the questions from the questionnaire are used as guidelines for the identification of potential improvements.Organizations are given freedom to define the type and number of improvements which they want to implement.
For each potential improvement, one or more of the potential effects of improvements are defined.This part should be thoroughly elaborated because this information is later used to calculate benefits that organizations can have by implementing these improvements.
For each of the potential effects of improvements, the improvement-affected maintenance goals should be determined.The organization should pre-define goals that will be influenced by these effects.These goals can be presented in a table (separately), or in an IMEA form.In the case that the analysis is performed for some other process rather than maintenance, it is necessary to change the line pattern in this field.The maintenance goals that were used in this analysis are: -Better maintenance organization -refers to the better organization and maintenance activities; -High equipment reliability; -Higher equipment availability; -Lower maintenance costs; -More efficient maintenance; -Better use of resources; -Occupational health and safety at work; -Environmental protection, and -Efficacy analysis of the maintenance interventions.
Each improvement can have several different effects; each effect can have a different level of "severity" of benefits for the organization.The improvement and the improvement-generated effect are assessed.Thus, each effect should be given its own score, even if there are several effects of an improvement.If a particular effect has a greater influence on the goals, its benefit to the organization is greater.The defined benefits used in this analysis are given in Tab. 1.

Improvement Rank
Criteria Negligible 1 Negligible side effects, the savings are minimal; the performance of the system is not changed.

Small 2
The user will probably notice the effect, but the effect is slight, small saving.

Moderate 3
The effect is noticeable but does not bring a significant improvement, cost reduction or better system performance.

High 4
The impacts will be noticeable through the operations.Increased performance with gradual further improvement.

Very high 5
Significantly reduced downtime and overall performance improvement, significantly reduced cost.
For each of the effects, potential obstacles in improvement implementation should be taken into consideration.By defining obstacles, the organization enables an easier way to prevent potential negative impacts when implementing improvements and enhances positive effects resulting from the improvement implementation.Any organization can independently define the obstacles that are considered to affect the process of improvements implementation.Below is the list of the obstacles that have been found to impact the improvement establishment in the function of maintenance used in the analysis: -High initial costs; -Employee resistance to change; -Insufficient training of staff; -Lack of knowledge of employees; -Insufficient knowledge of managers; -Labour costs; -Long deployment time for improvement; -Inadequate maintenance information system; -Lack of necessary equipment; -Attitude towards maintenance; -Lack of discipline.
Implementation effort is a necessary effort that organizations must invest to implement improvements.It is left to organizations to adjust the scale according to their needs.The scale for implementation effort criteria used in this analysis is shown in Tab. 2. Improvement is difficult to apply in an organization, great resistance to change, high costs of implementation improvements, long time to introduce improvements.

Difficult 2
Improvement can be applied with greater efforts, higher resistance to change and higher costs of improvement implementation.

Moderate 3
Efforts to establish improvements are present but moderate.Resistance to change is present but weak.The costs of implementing improvements are moderate.

Easy 4
Improvement is implemented with little effort, low resistance to change, while improvement application requires a low cost.

Very easy 5
Effort to implement the improvements does not exist or is insignificant; the resistance is almost non-existent, extremely low cost of implementation of improvements.
BPN -Benefit Priority Number is a number that is calculated for each potential effect by multiplying the Benefit from this effect and Implementation effort.

Calculating significance factors
In order to identify potential improvements in the observed organizations, the form of IMEA analysis was used.These improvements with the calculated BPN numbers will form the basis for defining significance factors.
The idea behind this model is to self-assess and improve the way of defining priorities, so it allows organizations to realistically perceive the quality of their maintenance in relation to the established goals.Some of the organization goals (and therefore the potential improvement) can be defined by regulations that the organization must comply with, or imposed by the equipment manufacturer or competition.Despite these limitations, the organizations are allowed the freedom to set priorities for maintenance, depending on their requirements and circumstances.
However, in this study, it was not left to the organizations to define their own improvement.This was done to ensure that all organizations have the same starting point.Virtually all organizations used the same questionnaire, completed by the method of selfassessment.In the second part of the analysis, determining significance factors, potential improvements were determined for all organizations.The questionnaire was used as a guideline in defining these improvements.
In this way it was not possible for the observed organizations to put emphasis on those areas where they received the largest number of points.The starting point was the same for all.In the practical application of this analysis, an organization conducts self-assessment and makes comparisons with its previous results or with the results of similar organizations.
The completed form represents the basis for obtaining significance factors.
In order to obtain significance factors, it is necessary to calculate the BPN sum for each area alone and the total BPN sum for all areas together.The averages of BPN for improvement are taken into account.If some improvement has more than one effect, the average BPN points for this improvement are calculated.For example, if an improvement has four effects and the points are 15, 20, 16 and 15, the average score is 16.5 BPN.
Significance factors are obtained using the following formula: where: SF s -section significance factor; BPN s -sum of the potential improvements points in a certain section; ƩBPN s -the total sum of potential improvements points; SF q -significance factor defined by questionnaire.
The questionnaire has a total of 640 points, evenly divided into 16 sections.Therefore, the significance factor of the questionnaire is the same for each section: Section significance factor will be larger than the significance factor of the questionnaire if: Example: The first section in the questionnaire is Maintenance Organizations.For this field, its significance factor -SF 1 is calculated: This procedure is repeated for all the sections of the questionnaire, for all of the observed organizations.This way the results represent a new value of certain areas of the proposed questionnaire and, as such, provide the basis for further performance analysis of the maintenance.The obtained values are rounded to whole numbers for easier comparison with the original results.

Research results
Applying the IMEA analysis and calculation of significance factors, new results were obtained that represent the more real state of maintenance functions in the observed organizations (if applied to values in column I) and indicates the area for improvement that will have major impact as well (if applied to values in column IIthose values are presented in the column III).All results were rounded to integer numbers for comparison with the original results.The results are presented in Tab. 4, in the columns headed by III (higher number indicates higher potential benefit of the improvement in concrete area/section).As it can be seen from the presented, the organization has not made even a little progress in total score, compared to the original results (due to rounding to integer numbers).But, if we look at the sections Maintenance Planning and Scheduling, Preventive Maintenance, Predictive Maintenance and Reliability engineering, we can see that this approach suggests focusing on the improvement in those areas (biggest difference between columns II and III for each organization).This also can be concluded by observing significance factors for those sections (1,23550; 1,12573; 1,37744 and 1,35227 respectively).Another way of "reading" this table is focusing just at values in the column III (biggest value = biggest possible benefits of improvement in that section), and that is specific for each organization.Third way of reading this table is comparing / benchmarking each section values between two organizations.

Conclusion
In applying this model, it is primarily necessary for an organization to bear in mind that in assessing the maintenance performance, the goal is not to find "the guilty party" for the current state, but to define the current position of the maintenance relative to another organization, or relative to the defined average value.Organizations must be aware that this procedure is only a first step towards improving maintenance function.A successful maintenance function brings higher reliability and availability of the equipment, lower maintenance costs, and it continues to affect the productivity of the organization and ultimately leads to competitive advantage and increase of profit.
This model should enable organizations to define their current situation, based on their needs, but without affecting the quality of maintenance, and to provide the basis for the definition of improvement within the maintenance function.
The procedure can be divided into four steps: -Filling out questionnaire and analysis; -Identification of potential improvements using IMEA analysis and assessment of the improvements; -Defining significance factors; and -Calculating points based on the significance factors and second analysis.
Filling out questionnaire and analysis -At this stage, an organization conducts evaluation of the current state of maintenance functions based on the questionnaire.Additionally, the organization benchmarks its maintenance with some other (similar) maintenance in a different organization or compared to the average value defined by the questionnaire.
The second step is to define the potential improvements and assess them.The organization can define improvements based on the questionnaire or on their maintenance process.Regardless of the method of defining improvement, the next step is to evaluate these improvements.
At the third stage, the organization, based on the evaluation of potential improvements, defines significance factors.The significance of these factors will help the organization to focus its resources on those stages that are most critical.
At the fourth stage, the organization re-calculates the points from the survey, now based on the significance factors.This way the organization determines whether the current situation in maintenance is better or worse than the original evaluation.This assessment is more realistic because it takes into account the capabilities and needs of the organization.Furthermore, in this way organizations can better define the areas for further improvements of their maintenance functions.
By using this procedure, organizations will enable continuous improvement of the maintenance function, by focusing on the improvements which can bring best ratio of improvement effort and benefits.Reviewing of the procedure in a specified time period, an organization will gain insight into list of possible improvement actions and effects that those actions can bring.By following the "suggestion" of that list, organization will ensure that next improvement action will bring the biggest improvement leap possible, at the moment.
The weakness of this system is that results and offered areas of improvement are absolutely dependant on the Wireman's questionnaire.If some researchers discover a new, improved concept of maintenance (for example [42]), we will have to wait until that area takes its place in Wireman's questionnaire.Only after that, we can expect a suggestion from IMEA for improvement in that direction.
factors are shown in Tab. 3.

Table 2
Implementation effort criteria

Table 3
Significance factors

Table 4
Results of the first and second analyses