Possibilities of Evaluating the Dimensional Acceptability of Workpieces Using Computer Vision

: This paper discusses the possibilities of an automated solution for determining dimensionally accurate and defective products using a computer vision system. In a real industrial environment, research was conducted on a prototype of a quality control machine, i.e. a machine that, based on product images, evaluates whether the product is accurate or defective using computer vision. Various geometric features are extracted from the obtained images of products, on the basis of which a fuzzy inference system based on Fuzzy C-means clustering features is created. The extracted geometric features represent the input variables, and the output variable has two values - true and false. The root mean square error in the evaluation of the accuracy and defectiveness of products ranges between 0.07 and 0.16. Through this research, valuable findings and conclusions were reached for the future research, since this topic is poorly examined in the most renowned databases.


INTRODUCTION
Possibilities and advantages of implementing a computer vision system or a somewhat less extensive machine vision system, which is a subset of computer vision, represent a long-recognized interdisciplinary technology. The main conditions for the application of technology based on computer vision were related to the level of development of software and hardware products and the speed of processing large data sets. Creating digital systems consisting of one or more cameras and computers for the purpose of processing, analyzing and understanding digital images and extracting meaningful and purposeful insights from the digital input (digital image) is the main objective of computer vision. Computer vision, as a field of artificial intelligence, mostly includes research of replication and imitation of human vision segments, which are later used through the machine vision system to determine various actions such as automated tasks in the industrial or manufacturing environment. Technologies and methods cover a wide range of application areas: automated control (inspection), process control, face recognition, object recognition or detection, image search, object tracking, etc. Since 2010, the development of computers and the availability of large databases has enabled a more significant application of deep learning methods in the sense of applying computer vision, from autonomous driving and robotics to various applications in detecting and recognizing certain areas or zones of interest in digital images [1].
The review of scientific literature below will present the scope and diversity of the application of methods and technologies based on computer vision. In paper [2], the developed machine vision system is used for inspection and evaluation of fruits in terms of detecting external faults since the consumers associate fruit quality with good appearance and the total absence of external faults, which ultimately significantly affects the market price. An important feature of methods and technologies based on computer vision is their non-destructive nature (non-destructive procedures).
Security surveillance in terms of access control and human recognition is the main focus of research conducted in paper [3], where the authors developed a precise method for detecting human faces using a hybrid neural network. Paper [4] presents a computer vision system that conducts plant phenotyping based on features such as plant volume, leaf surface area and stem length. The proposed computer vision system combines the best of the 2D imaging approach with the three-dimensional (3D) reconstruction method. Similar to the aforementioned paper, important information for agriculture is the measurement of surface area and volume in food processing, for which a machine vision system has been developed in paper [5]. The three-dimensional wire-frame model of the object is reconstructed by integrating silhouettes recorded from different viewing angles. In paper [6], dimensional analysis of geometric figures is performed using computer vision, while paper [7] analyzes the use of smartphones as machine vision devices, with the focus on drilling. Using the machine vision method, the measurement of required parameters according to the ISO 10545-2 standard, which refers to the dimensional deviation of ceramic tiles, is performed in paper [8]. The final paper in the first part of the scientific literature review is paper [9], which investigates specular surfaces as one of the significant limitations of computer vision. Specularity of outer surfaces directly caused by ambient lighting makes it difficult to accurately reconstruct the product, especially when detecting faults such as bumps, cracks and scratches present on the product.
Unlike the first part of the scientific literature review, which intends to present a part of the scope and diversity of application, the second part of the scientific literature review refers to the creation of 3D models from a set of images, i.e. 3D reconstruction. This part also includes the previously mentioned papers [4,5] and [9]. In paper [10], machine vision technology is used to evaluate the logarithmic spiral bevel gears. Two cameras at different locations capture images of the gear using correspondence between the feature points of these two images to solve the three-dimensional coordinates of the tooth surface points. Manner of estimating 3D measurement errors at an early stage of optical design is the topic of paper [11]. Computer simulation using optical design software enables the optimization of optical parameters and selection of the most effective mathematical model as well as the equipment necessary for calibration. The issue of specular surfaces of objects is a very common occurrence in methods and technologies based on computer vision. In paper [12], numerous experiments performed on different shapes and sizes of specular surfaces of objects are described, with qualitatively and quantitatively reconstructed 3D profiles of different shapes of specular surfaces presented. A new method of measuring mirror surfaces that directly provides analytical solutions for three-dimensional points on a mirror surface is proposed in paper [13]. The technique of three-dimensional camera calibration based on two 2D camera calibrations was developed in paper [14]. A physical model is used to determine the exact locations of the calibration points. A significant issue in 3D reconstruction or creating 3D models from a set of images is often the need for large memory. In order to solve this issue as well as the issue of low efficiency of the algorithm for recognizing 3D objects, paper [15] proposes an algorithm for that purpose, which is based on enhanced point pair feature. In paper [16], a threedimensional scene reconstruction based on binary space coding and decoding is proposed, with coding accuracy of up to 100% and the reconstruction result for the plane with an error of 0.0993 mm. Obtaining a three-dimensional textured model based on 2D-2D transformation on a distorted (warped) image is shown in paper [17]. Shape reconstruction is based on planar rectification and collation of laser profiles, and not on triangulation. The final paper [18] included in the second part of the scientific literature review proposes a method for reconstructing the two-dimensional profiles of ring-shaped objects using image processing, resulting in a point cloud consisting of outer and inner contours that can be directly used for automated measurement.
The third part of the scientific literature review is related to the application of computer vision or machine vision to problems in mechanical engineering, which is also the focus area of this paper. In paper [19], illumination compensation techniques were used to evaluate the ground surface roughness with regard to statistical texture parameters using machine vision. Three-dimensional surface roughness parameters are compared with texture parameters. Contouring errors of CNC machine tools are investigated in paper [20]. Binocular vision-based 3D method for detecting high dynamic and wide-range contouring errors of CNC machine tools has been successfully proposed. In paper [21], a vision system was developed to recognize turning inserts placed in a tool holder. The subject vision system recognizes 9 different types of inserts based on the insert angles, edge lengths and nose radii of each insert. A dimensional inspection system of shaft parts based on machine vision is proposed in paper [22]. Experimental results have shown that the measurement accuracy reaches 0.015 mm, i.e. 15 µm, which can be promising, but care should be taken about the appropriateness of the application in relation to the object dimensions. Paper [23] describes an improved 3D imaging (vision) system for dimensional quality inspection of long, flat-rolled metal products. Two-dimensional characteristics of rolled products -width and flatness are the focus of measurement. A simple three-dimensional measurement system based on machine vision was developed in paper [24]. In the case of hole diameters, the maximum error was 0.373 mm and the minimum error was 0.053 mm, which can meet one part of the tolerances used in engineering schematics. Tool positioning plays a significant role in the accuracy of manufacturing workpieces using CNC machine tools. The system for precise tool positioning and verification on turning and milling machines based on machine vision is presented in paper [25]. The developed system extracts the difference between the actual and target tool positions from the captured images through image processing and calculates the error, whereby the maximum positioning error observed was +/-206 μm. Paper [26] presents a vision-based system for evaluation of surface roughness as well as quantitative and qualitative evaluation of surface texture. The results obtained using the vision-based system vary between 9% and 11% compared to the stylus-based ones. Detection and selection of bearing diameter based on machine vision system is the topic of paper [27]. After processing the images collected by the CCD camera, the bearing edge contour is obtained, on the basis of which the bearing diameter is detected and determined.
Considering the scientific literature review, it can be concluded that there is not a large number of papers that deal with the evaluation of the dimensions of workpieces in terms of determining their acceptability in view of the requirements set out in the technical drawing. This paper will conduct extensive research in terms of dimensional control of workpieces and list all the existing limitations of computer vision with regard to its application in this task.

PROBLEM FORMULATION AND RESEARCH OBJECTIVE
When considering the accuracy of manufactured components in mechanical engineering in relation to the requirements set out in the technical drawing, the following should be taken into account: absolute precision in component production is not possible due to limitations of regular production processes and/or excessive costs that are not economically justified. For this reason, the components are manufactured within the permitted variations or defined tolerances with respect to the nominal dimension values. The application of such production (within a certain accuracy limit) was first introduced in the manufacture of ammunition and weapons [28]. It was soon noticed that the quality of fits must be proportional to the dimension, i.e. expressed as a percentage of the dimension. For the practical application and proper functioning of machine components, the tolerances of the components in the fits are important, i.e. the components that form a joint through their abutting surfaces. According to the ISO system, the fundamental tolerance is determined, which strictly defines the range of the tolerance field in relation to the nominal value of the observed dimension, i.e. the area of nominal dimensions to which it belongs. After the manufacture of components, quality control is performed, where the accuracy of the manufactured components is evaluated in relation to the requirements set out in the technical drawing. Classic devices for measuring tolerated dimensions include various types of calipers, micrometers, dial indicators and similar devices that involve contact between the device, the manufactured component and the person performing the measurement task. This approach takes up a lot of time, which often entails insufficient competitiveness in todayꞌs market. The focus of this research is on the possibilities of applying computer and/or machine vision in terms of verifying the acceptability of manufactured components.
As soon as computer vision is mentioned, it immediately becomes clear that acquisition and processing of digital images is paramount. Digital image is created through several phases, which are described below and shown in Fig. 1.

Figure 1 Creation of a digital image
After the light passes through the lens, it reaches the photodiodes on the sensor (usually CCD and CMOS sensor) which convert the energy of the incoming photons into the appropriate charge. That electric charge is converted into voltage, which is then amplified by a power amplifier. The amplified voltage is processed by an A/D converter that converts the analog signal into a digital recording. Finally, the processor processes the obtained digital recording using a series of complex operations (interpolation, focusing, noise reduction, compression …), after which the processed image is recorded on a memory card. Since the aim of this paper is to evaluate the dimensions of a workpiece, the camera sensor plays a very important role because it determines the number and quality of pixels in the image, which are the only repetitive elements on which the evaluation is based. The lens also has a very important role since it should ensure adequate projection and zooming of the workpiece in order to occupy as large an area of the digital image as possible. All variables on the basis of which the dimensions, i.e. the acceptability of the workpiece will be evaluated are based on measuring the properties of the image regions of accurate and defective workpieces according to the principles explained in Fig. 2. The left frame of Fig. 2 shows the sensor and edge of a workpiece with black lines and yellow lines that represent the maximum and minimum tolerated dimension of the workpiece. The middle frame contains the ideal appearance of the workpiece shown in the image (green pixels), and the right frame indicates the tolerated pixels when the maximum and minimum values of the workpiece dimensions are taken into account. In the right frame, each pixel on the outside of the yellow lines, if activated directly, means that the workpiece is defective. The research in this paper will be based on stator acceptability evaluations, which are also a part of a project funded by the European Regional Development Fund under code KK.01.2.1.02.0062. The main activity of the project is the development of new innovative products, namely a quality control machine and a palletizing machine in the automotive industry. Various sizes of stators and rotors will be checked for correctness on the quality control machine. In this regard, the input variables described below and extracted from stator digital images will be used to evaluate the acceptability of each stator. Since the aforementioned project (project code: KK.01.2.1.02.0062) has several phases, this paper will present only the first phase of research, with the conclusion containing guidelines for future research.

Description of Input Variables Used to Evaluate Acceptability
The input variables that will be used to evaluate the acceptability of the workpieces (stators) are based on measurements of the properties of the image region or regions. Therefore, the areas (regions) of the image, which in this case include the workpiece or blurs (impurities), have properties such as surface area, center of mass, orientation, bounding box, etc. The first steps in processing the workpiece image are reduced to obtaining a black and white image and removing all possible stains (impurities) that can arise due to various factors. The first four input variables are based on the surface area or pixel count of the region of interest, i.e. the stator. The first input variable Var1 represents the number of pixels from the stator drawing, the second variable Var2 represents the number of pixels of the stator hole, the third variable Var3 represents the number of pixels that make up the outer stator edge, and the fourth variable Var4 represents the number of pixels that make up the inner stator edge, as shown in Fig. 3.
The fifth and sixth variable (Var5 and Var6) represent the maximum width and height (measured in pixels) of the stator. The seventh variable Var7 is defined by the number of pixels that make up the surface area of a solid convex hull (polygon) containing/including a stator. The ratio of the surface area of the region (stator) and the surface area of the solid convex hull (polygon) represents the eighth variable Var8, and the ratio of the surface area of the region (stator) and the total surface area of the bounding box represents the ninth variable Var9.

APPLIED METHOD -FUZZY C-MEANS CLUSTERING
The Fuzzy C-means clustering method will be used to evaluate the dimensional acceptability. The idea is that multidimensional data points consisting of the previously described variables provide sufficient qualitative and quantitative indicators so that the set of all data points can be observed through two clusters. One cluster represents data points that are related to accurate workpieces, while the other cluster represents data points that are related to defective workpieces. A certain peculiarity of Fuzzy C-means clustering is that each data point can belong to a larger number of clusters with a certain membership degree, while the sum of all memberships in different clusters must be equal to 1.
The set of data points X to be clustered can be specified in the form of a set: Each point x i is a vector which has a number of dimensions equal to the number of input variables and represents the quantitative features of the i workpiece.

Fuzzy C-means Clustering Algorithm
Mathematical algorithm of Fuzzy C-means clustering consists of the steps described below. 1. There are n data points set to be clustered xi, where i = 1, 2, 3, … n. 2. Initiate the wanted number of clusters k by meeting the following condition 2 ≤ k ≤ n.
where j represents the cluster, and m represents the algorithm iteration. 6. Calculate the Euclidean distance where i represents the data point, j represents the cluster, and m represents the algorithm iteration. 7. Update the membership matrix ijm U with new membership values by using the following formula: Formula set out above is applied only to data points where ijm D > 0. If ijm D = 0, the membership is full and the value is initiated with 1.0. 8. Repeat steps 5 to 7 until the maximum value of difference of all members of matrix ijm U of the current and previous iteration is higher than the algorithm stopping criterion ɛ or until the maximum set number of iterations m is performed.

DESCRIPTION OF THE EXPERIMENTAL RESEARCH CONDUCTED
Experimental research, i.e. acquisition/recording of digital images was carried out on a machine prototype that is being developed within the aforementioned project under code KK.01.2.1.02.0062. Fig. 4 shows a prototype machine for quality control of finished products of smaller dimensions. Quality control, that is, dimensional acceptability evaluation on this machine was related to stators and rotors.
The machine prototype incorporates the components indicated below that are essential for computer vision and ultimately for the acquisition of digital images of products being controlled. An Allied Vision industrial camera model Alvium 1800 U-507 and a bi-telecentric lens model TC23085 from Opto Engineering are used for the acquisition. Two Alvium 1800 U-507 cameras and two TC23085 lenses are visible on the left, and their detailed view is given on the right side of Fig. 4.

Figure 4 Machine prototype for quality control of finished products
The backlight from model LED1-FL-83×75 manufactured by GETCAMERAS was used to illuminate the workpieces when acquiring digital images. Fig. 5 shows the illumination with a sketch of the entire system that includes the camera and the lens. The basic technical specifications of the camera and lens are given in Tab. 1. Using this machine prototype, an experimental research was conducted on 115 specimens of stators shown in Fig. 3. Using the equipment shown, 115 digital images of all specimens were acquire, of which 12 samples were defective. Using the digital images, 9 different features were quantified that will represent the input variables for the fuzzy inference system of C-means classification. Prior to quantifying features from digital images, all digital images were converted to binary or monochrome format and cropped to a size of 1850 × 1850 pixels. The matrix of each digital image had 1850 rows and columns (Fig. 3). The stator was placed in the center of the image after cropping, and a small area was reserved for the unnecessary surroundings in relation to the maximum height and width of the stator.

RESULTS OBTAINED
The input/output data set consists of 9 input variables and an output variable that can take only two values -1 or 2. A value of 1 represents an accurate specimen, while a value of 2 represents a defective specimen. The accuracy of all specimens used in this study was verified using a measurement matrix. Out of a total of 115 records of input/output experimental data, 98 records were used for training and 17 for testing the fuzzy inference system of Cmeans classification. Random selections of input/output data for training were generated based on the criterion of the smallest root mean square error obtained. The condition for random selection of input and output data for training is that the root mean square error must be less than 0.14 in the training phase or the time spent to detect it must not exceed 4 hours, i.e. 14400 seconds.
where: n -number of specimens (different stators or their digital images), measured(Out) -value of specimen accuracy determined using the measurement matrix, and systematic(Out) -value of specimen accuracy obtained through the fuzzy inference system of C-means classification. Five different fuzzy inference systems of the C-means classification were generated. For the first system the first four input variables and the output variable were taken, in the second system the first six input variables and the output variable were taken and so on up to the fifth fuzzy inference system, where all nine input variables and the output variable were taken. Since the fifth and sixth input variables represent the width and length of the stator, the second system is expanded with two input variables, and all thereafter are expanded with one input variable.
Tab. 2 indicates the root mean square errors in the training phase and the testing phase for all five fuzzy inference systems of the C-means classification. Whichever of the five fuzzy inference systems generated is placed into focus, with graphs indicating the dependencies of the input variables on the output variable, three different dependencies arise. Fig. 6 contains graphs indicating the dependencies of the input variables on the output variable. Even though the generated fuzzy inference systems have more than two input variables, only two input variables with an output variable can be placed on 3D graphs. The figure below contains 3D graphs with combinations of input variables covering the aforementioned dependencies between the input variables and the output variable, where linear or segmentally predominantly linear dependencies between the input variables and the output variable prevail. Most often in the case of the first and second input variables with the output variable a somewhat more complex dependence occurs.   The final significant observation to be presented herein relates to the dependence of certain variables on the stator orientation angle. During the acquisition of digital image, each stator is oriented differently with regard to the position of the grooves located on the outside, with the angles between them amounting to 120°. The following graph indicates a significant correlation of Var3, Var4, Var5 and Var6 with the angle, i.e. orientation of the stator.

CONCLUSION AND DISCUSSION
This research is the first step in finding a satisfactory solution for evaluating dimensional accuracy using computer vision. In this research, a number of limiting factors were discovered that should be taken into account when planning to use computer vision to evaluate the dimensional accuracy of workpieces. The first limiting factor relates to the precision up to which the dimensional accuracy of workpieces can be evaluated with regard to economic justification and technical feasibility. Precision of 0.1 mm, i.e. 100 µm is sufficient to evaluate the dimensional accuracy of the stator that was used as a specimen in this paper. The camera used has a resolution of 5.066 MP and a pixel size of 3.45 × 3.45 µm when the area of the image is equal to the size of the built-in sensor of 2/3, i.e. 8.50 × 7.10 mm. As the digital image almost always shows a larger area, the basic building block of a pixel has much larger dimensions, i.e. the larger the workpiece, the lower the accuracy. In the case of stators with a maximum dimension equal to 60.2 mm, each pixel when recalculated has a size of about 33 × 33 µm, i.e. 3 pixels make 0.1 mm. If one of the more precise sensors is taken into focus (model: Canon LI8020SA CMOS), which has the following features: a resolution of 250 MP (19568h × 12588v) pixels and a sensor size of 29.35 × 18.88 mm, or pixel size of 1.5 × 1.5 µm, it can be concluded that for a workpiece that fits in a square measuring 100 × 100 mm, the size of the recalculated pixel should ideally be about 7.5 × 7.5 µm. From the above it can be concluded that the application of machine vision for evaluating the dimensions of workpieces is very limited to a small range of products and precision that is at least ten times lower than with the conventional hand-held measuring devices. To evaluate the acceptability of the stator dimensions in this paper with respect to the obtained root mean square error, it can be concluded that a camera with a higher quality sensor of much higher resolution is definitely required.
The lens has a significant role in the issues of estimating the acceptability of the workpiece dimensions. The lens must have an orthogonal or parallel projection and the largest possible area that can be projected onto the sensor without large distortions and differences in the projected size displayed by the pixel with respect to the distance between the center of the workpiece and the projected area. Furthermore, the size of the projected area significantly reduces the range of different products that are suitable for the application of machine vision.
After successfully resolving the aforementioned limitations, there are several other limitations related to the processing of the obtained digital image. One of the most significant limitations is indicated in Fig. 7. If digital image is rotated mathematically, the cascaded continuity of the edges is destroyed and the total number of pixels from which the projection of the observed objects is constructed changes. This limitation hinders the overlap of the tested items with the accurate workpieces on the differences of which much better quality input variables would be generated than those of one-dimensional significance presented in this paper. Since the mathematical rotation causes a number of adverse effects, the control of orientation of test specimens becomes a key issue, regardless of whether it is solved by design of a machine vision system or planned generation of a database to be used in the training and testing phase of a model for evaluating dimensional acceptability. Furthermore, another important finding regarding the orientation of the observed workpieces relates to the existence of dependencies between certain variables and orientation angle as shown in Fig. 8. For variables Var5 and Var6, it is intuitively clear why there is dependence because there are three properly spaced grooves on the exterior side, but there is no simple explanation for variables Var3 and Var4. As for the type of dependence of input variables on the output variable, the linear and segmentally predominantly linear dependencies that prevail indicate to a simple and stable correlation that ultimately has a positive effect on mapping the input into the output area. Through the review of the literature, a small number of articles dealing with this issue were found, therefore it is considered that this research can provide good guidelines for future research. This research did not provide a solid solution, but it outlined the issues while defining the main limitations and research directions.
Future research could be conducted by creating more meaningful input variables with one-dimensional as well as two-dimensional correlations expressed through the value of each variable. Solutions related to the control of the orientation of workpieces, whether they are structural or of some other framework, are certainly one large area within which various research can be conducted. An appropriate orientation control solution would open up a series of other possibilities, from creating input variables resulting from differences in the overlap of workpieces, to the creation of databases containing generic workpieces or generic forms that would be used for comparison. Future papers dealing with the aforementioned topics would probably find interested readers.