INTELLIGENT SYSTEM FOR PREDICTION OF MECHANICAL PROPERTIES OF MATERIAL BASED ON METALLOGRAPHIC IMAGES

Original scientific paper This article presents developed intelligent system for prediction of mechanical properties of material based on metallographic images. The system is composed of two modules. The first module of the system is an algorithm for features extraction from metallographic images. The first algorithm reads metallographic image, which was obtained by microscope, followed by image features extraction with developed algorithm and in the end algorithm calculates proportions of the material microstructure. In this research we need to determine proportions of graphite, ferrite and ausferrite from metallographic images as accurately as possible. The second module of the developed system is a system for prediction of mechanical properties of material. Prediction of mechanical properties of material was performed by feed-forward artificial neural network. As inputs into artificial neural network calculated proportions of graphite, ferrite and ausferrite were used, as targets for training mechanical properties of material were used. Training of artificial neural network was performed on quite small database, but with parameters changing we succeeded. Artificial neural network learned to such extent that the error was acceptable. With the oriented neural network we successfully predicted mechanical properties for excluded sample.


Introduction
In modern engineering modern methods to solve different problems are nowadays more and more appearing and used.We were faced with the problems in which we have a lot of input data, a bunch of overlapping and often conflicting requirements and many outputs, usually used oriented artificial neural networks to solve such problems [1 ÷ 7].Neural networks have a great ability to recognize and identify certain patterns on the basis of which they can be used to predict the mechanical properties of materials.
Malinov et al. [8] developed a model for analysis and prediction of the correlation between processing parameters and mechanical properties in titanium alloys by applying artificial neural network.They used as inputs in artificial neural network the data of alloy composition, heat treatment parameters and work (test) temperature.They are predicting nine most important mechanical properties as follows: ultimate tensile strength, tensile yield strength, elongation, reduction of area, impact strength, hardness, modulus of elasticity, fatigue strength and fracture toughness.Similar research was done by Dobrzański et al. [9].They present the method for predicting the yield point and the ultimate tensile strength for steel.They used artificial neural network for development of models for prediction of these mechanical properties.Sterjovski et al. [10] made three different models of artificial neural network, the first model was for predicting the impact toughness of quenched and tempered pressure vessel steel, the second model for predicting hardness of the simulated heat affected zone in pipeline, the third model for predicting the hot ductility and hot strength of various microalloyed steels for casting process.Bahrami et al. [11] researched the use of artificial neural networks for prediction of mechanical properties based on different morphology and volume fractions of martensite.
Some authors showed that it is possible with use of neural networks models to predict mechanical properties of material based on its chemical composition [12,13].With use of such models we can predict mechanical properties of the materials very accurately and reliably.
Authors dealing with the image processing and rapid and automated determination of individual microstructure from metallographic images, for this purpose use special developed algorithms, and / or for the same purpose special developed artificial neural networks [14 ÷ 17].Peregrina-Barreto et al. [14] report that they had problems with the accurate determination of the microstructures from metallographic images which they were processing.Images were differently illuminated, this causing a lot of noise on the image, low contrasts, badly defined boundaries and so on.
The main goal of our research was to automate the prediction of mechanical properties exclusively on the basis of metallographic images of material.In our case we knew for each metallographic image mechanical properties of material, which are also predicted with oriented artificial neural network.The biggest problem for us was how to extract from metallographic image some useful data or information by which we will be able to successfully predict mechanical properties of material.System has been developed, which can be widely used for feature extraction from metallographic images, not just for one type of material but which can be adapted with small modifications and used for the processing of metallographic images of completely different material.
Presented system allows working faster and more reliably, because it automatically extracts features from metallographic images and predicts mechanical properties of material.With the use of such a tool which has been developed it saves the cost of expensive tests that must be performed to verify and measure certain mechanical properties of material.Preforming such mechanical tests of material and making of test parts is very time consuming and expensive, which is not affordable in time of economic crisis.In times of economic crisis the demand for cost reduction is more and more growing, thus making the system for prediction of mechanical properties of material based on metallographic images more justified.Such system will be a major contribution to the practice of avoiding the need for expensive mechanical tests.
The paper is organized in the following order.In the second section, we briefly introduce the whole system, first the image feature extraction algorithm is described and then the system for prediction of mechanical properties based on obtained information from metallographic image is described.In the third section the algorithm for the feature extraction from image is presented in details.The fourth section presents the system for prediction of the mechanical properties based on the information obtained from the image and oriented artificial neural network.Results are presented in the fifth section.Article ends with a brief conclusion.

Description of the system
The system for prediction of the mechanical properties of the material consists of two modules.The first module of the system is the algorithm for features extraction from metallographic images, which were obtained with the microscope.The second module of the system is the system for prediction of the mechanical properties based on the oriented artificial neural network, based on previously acquired data from images.
Fig. 1 shows block scheme on how the whole system works.The system works by first obtaining an image from microscope.The image is processed by the feature extraction algorithm, algorithm's pseudocode is shown in Fig. 2, with the help of this algorithm we can extract useful information from a metallographic image.After the image processing has been completed, algorithm calculates the proportions of graphite, ferrite and ausferrite on the image.These three values are the inputs into oriented artificial neural network.With the help of feed-forward artificial neural network we can predict mechanical properties of the material.The following mechanical properties were predicted: the ultimate tensile strength of material (σ max ), the yield strength of the material (σ 0,2 ), the factor of phase coherence between the surfaces (δ) and the fracture toughness (K IC ).Material was Dual Phase ADI.Chemical composition of material was 3,35 % C, 3,21 % Si, 0,46 % Mn 0,94 % Cu, 0,04 % Mg, <0,02 % S, <0,02 P, 4,26 % CE.The functioning of the algorithm for processing metallographic image is shown in the block diagram in Fig. 3.
The algorithm for determination of graphite, ferrite and ausferrite from metallographic images first performs reading of the colour image obtained from the microscope in Fig. 4.
Coloured image consists of three different colour components: red, green and blue.The colour of each pixel from image is determined by the combination of colour intensities of the components in each coloured plane of the image.The intensity of each colour component is in the range from 0 to 255 [17,18].Because the images made with the microscope were not of the same brightness and the same colour shades, we first calculated the average value of each colour component of the image.Then correction of each colour component was made, so that the average of the colour components was in the middle of the pixel colour range.This means that the images which were darker become brighter and brighter images become darker.So we got all the pictures that we are processing in the similar brightness and colour shades.Similar brightness of the picture and the similar colour shades of image are very useful in determining microstructural composition of the material on the image.(1) In the Eq. ( 1) S presents a particular grayscale pixel, R, G and B are the colour intensities of the individual coloured component of the pixel.
In our study we need to determine proportions of graphite, ferrite and ausferrite from metallographic image as accurately as possible.On images that we are processing we have the graphite, which is represented by darker shades on most of the images.In Fig. 4 we can see the entire composition of the material.For the exclusion of graphite from the image we used threshold method, because the graphite is the darkest on the image.
Thresholding is one of the most important approaches to image segmentation.With thresholding we separate brighter objects from the dark background [18,19].Segmentation is made so that we scan each pixel of image and compare grey level with the threshold level we have set.If the value of pixel was greater than the threshold, then the pixel was background and had a value of 0, if the grey level value of pixel was smaller than the threshold value, then the pixel represents the object and has value of 1.The threshold level that we use to separate graphite from the background on image was determined by experiments, threshold level had to be established so that the algorithm will work correctly when we process other metallographic images.
In the figure the so-called image noise shown in Fig. 9 remains.We remove noise from the image by replacing with the black colour all the white areas that have surface less than 400 pixels.In that way only graphite stays on the image.Even in areas occupied by graphite remain anomalies, so those areas in graphite are coloured black and are within the range of graphite, they were replaced with white colour.Results of the noise removal from image are shown in Fig. 8.
On the grayscale image created by algorithm in the second step (Fig. 5), and with the help of Fig. 7, mark the areas of graphite on a colour image.We do this by checking each pixel value in Fig. 10, if the value of each pixel is equal to 0, add value 0 on coloured image to the pixel at the same position, but if value of the pixel in Fig. 10 is equal to 1, add value 255 on coloured image to the pixel at the same position, but only in the red colour component.
The next step is determination of the other two components, which remain on the image, ferrite and ausferrite.
We tried to solve the problem by setting the threshold level for all three colour components for the ausferrite.For each pixel on the image of all three colour components values were compared with the set threshold level and if the values were higher than the limit, then write the value of 255 to that position, in green colour component.The limit was set based on tests.Determination of threshold level between ferrite and ausferrite on colour image Fig. 4 was not successful, because the method worked only on the current image.For the rest of the images, it was necessary to manually set threshold levels again with testing.
Because we were developing an algorithm, that will operate independently and will be robust, we cannot afford manual settings and changing the limit values.Therefore, it was necessary to solve ausferrite and ferrite extraction with another method.
On the grayscale image ausferrite and excluded form image were defined by thresholding.The threshold level between the ferrite and ausferrite was determined through testing and has worked on all images processed.
In order to facilitate the determination shares of graphite, ferrite and ausferrite on images, we coloured areas with red that are occupied by graphite, green colour indicates areas of ferrite, and the part of the image which remains unchanged is occupied by ausferrite.At the end, algorithm calculates the proportions of each material component on image.The algorithm performs the calculation of proportions in such a way that it reads the values of R, G and B components for each pixel on image.If the value of red component is 255 and the other two components have value 0, then this pixel represents the graphite and adds one to graphite.If the value of the green component is 255 and the other two components are 0 then this pixel represents ferrite and adds one to ferrite.The rest of the image represents ausferrite.Algorithm according to the size of the image in pixels calculates proportions of each material component and displays them.

Prediction of mechanical properties with feed-forward neural network
Prediction of mechanical properties of material was performed by artificial neural network.Artificial neural network used, was feed-forward neural network and consisted of input layer of neurons, four hidden layers of neurons and of output layer of neurons; topology of used artificial neural network is shown in Fig. 7. Transfer function, which was used in the hidden layers of artificial neural network, was Tan-Sigmoid.Fig. 8 shows the block diagram of the system working for the prediction of mechanical properties with the help of feed-forward neural network.Inputs for training of artificial neural network are extracted from metallographic images, these are the proportions of graphite, ferrite and ausferrite.As targets for training of artificial neural network mechanical properties of material were used, which have been measured using mechanical tests.These are the maximum tensile strength of material (σ max ), the yield strength of the material (σ 0,2 ), the factor of phase coherence between the surfaces (δ), the fracture toughness of material (K IC ).

Results
Results of graphite separation from the other elements in the image are shown in Fig. 9. Fig. 10 shows removed image noise from Fig. 9. Determination of ausferrite was held by threshold and results of thresholding are shown in Fig. 11.The final result of image processing is presented in Fig. 12.For training of artificial neural network we had available little training data, however.The maximum training error of used artificial neural network was 5,381 % and it was within acceptable limits.With the oriented neural network, we predicted mechanical properties for excluded sample.The maximum error in prediction of mechanical properties for excluded sample was 7,3 %, which was also within acceptable limits.Prediction of mechanical properties was performed by feed-forward neural network.Training of artificial neural network was performed on small data base, but with changing parameters we succeeded.Artificial neural network learned to such extent that the error was acceptable.With the help of such oriented artificial neural network we were able to predict the mechanical properties for excluded sample.Prediction of mechanical properties for the sample, which was excluded from the training database, was successful.Of course, to make the system for prediction of mechanical properties with the help of artificial neural network even more efficient, reliable and robust, we will continue to update and add information into the training database.
In the future work we would like to add to the system for prediction of mechanical properties some new features from metallographic images, such as the position and density of individual microstructural composition of material, either the position of individual microstructural composition is on grain boundaries, or it is uniformly distributed through the whole material, because this has great effect on the mechanical properties of material.
This research can be widely used and has a wide range of opportunities for further development and it is highly topical in the world of engineering.

Figure 1
Figure 1 Block diagram of the intelligent system for prediction of mechanical properties of materials based on the microscopic images

Figure 2
Figure 2 Pseudocode of algorithm for processing microscopic images 3 Algorithm for feature extraction from metallographic images

Figure 3 Figure 4
Figure 3 Block diagram of image processing algorithm

Figure 5
Figure 5 Transformation to grayscale image Next step is transformation of colour image to grayscale, which is shown in Fig. 5. Transformation of image is carried out as shown in Eq. (1), so that each colour component of pixel is weighted and then summed. = 0,2989 •  + 0,5870 •  + 0,1140 • .(1)

Figure 7 Figure 8
Figure 7 Topology of used artificial neural network

Figure 9 Figure 10 Figure 11
Figure 9 Eliminated graphite using thresholding for image segmentation

Figure 12
Figure 12 Final result of the image processing