No Clamp Robotic Assembly with Use of Point Cloud Data from Low-Cost Triangulation Scanner

The paper shows the clamp-less assembly idea as a very important one in the modern assembly. Assembly equipment such as clamps represent a significant group of industrial equipment in manufacturing plants whose number can be effectively reduced. The article presents the concept of using industrial robot equipped with a triangulation scanner in the assembly process in order to minimize the number of clamps that hold the units in a particular position in space. It also shows how the system searches for objects in the point cloud based on multi-step processing algorithm proposed in this work, then picks them up, transports and positions them in the right assembly locations with the use of industrial robot manipulator. The accuracy of the positioning of parts was also examined as well as the impact of the number of iterations of the algorithm searching the models in the point cloud on the accuracy of determining the position of the objects. The tests show that presented system is suitable for assembly of various items as plastic packaging and palletizing of products. Such kind of system is the basis for modern, fully flexible assembly systems.


INTRODUCTION
The assembly process is the final and at the same time the most important stage of the manufacturing process. It determines not only the parameters of the final product, but also its labour intensity and closing manufacturing costs. For this reason, the development of the best technology for joining parts is crucial; it entails considering multiple criteria and thus is highly complicated. The paper presents the concept of a robotic assembly station, characterized by the possibility of assembling unordered parts into a finished product without the use of assembly equipment, such as clamps holding parts in the required position.
The barrier to implementation of sophisticated assembly techniques in small and medium-sized enterprises is usually high cost. The paper proposes costeffective solution that increases investment costs by just a few percents at the same time significantly increasing capabilities of robotic assembly station.
The proposed solution is a system consisting of an industrial robot, a 3D laser scanner, a PC and software. The contribution is the development of methods for processing the point cloud from the laser scanner and generating the trajectory of the robot performing the assembly task.
It is a flexible solution, since any change within the manufactured product will only entail adjustments in the software and possible replacement of the gripper in order to adapt to the new shape of the part.
The proposed robotic assembly station uses a universal vacuum gripper and a belt conveyor that delivers pallets with parts. The system does not require the use of clamps fixing the position of parts, nor setting them by people, which distinguishes it from other assembly stations.

RELATED WORKS
The high costs of manual assembly processes had led to a rapid and significant increase in the share of automated and robotized processes in developed countries. All kinds of flexible assembly systems are gaining importance, particularly where the problem of collection and pre-orientating of parts is frequent. The units are usually selected in a chaotic manner, in contrast to a less flexible automatic assembly [1][2][3][4].
Robotic cells in which movement trajectory of an industrial robot is dynamically generated on the basis of the collected three-dimensional geometric data are still relatively rare. The reason is the difficulty in obtaining adequate accuracy, quick data acquisition and analysis of point clouds. Cheap laser scanners are sometimes used in mobile robots [5]. In industrial applications, 3D acquisition systems allow us to prevent manipulator collision and detect components with sufficient accuracy to perform tasks related to palletizing goods and painting [6].
Considerably more research has been devoted to the application of robotic laser scanners for reverse engineering purposes, as in [7,8], where the main issue is to choose the scanning path to obtain a complete object model. Positioning of objects in the workspace is also important in the context of CNC machine tools, where high precision is required in order to compete with the use of touch probes known so far [9,10]. Extensive research was carried out on an industrial metrology [11][12][13], several studies [14][15][16] centered on the relationship of surface quality and process properties.
Some papers describe methods for determining the position of objects on the basis of 3D sensor data. The most commonly used approach is to calculate the set of attributes for the characteristic points of objects and matching the characteristics of each object model [17]. Such an approach can fully determine six spatial coordinates of the object, even for data in the form of flat images [18]. This approach is highly effective for objects with organic shapes that have many characteristic points, corners, and sudden changes of curvature; however it is less effective for parts that are simple solids consisting of a few planes and cylindrical surfaces and this approach is not recommended. In the latter case RANSAC method (Random Sample Consensus) is recommended, as it is intended to recognize simple solids [19].

CLAMP-LESS ASSEMBLY
Today there is a continuous trend to shorten the time of the products' lifetime, while increasing the number of variants thereof. This presents a significant challenge to manufacturers because the time and cost of retooling production lines to produce a new type of product significantly affect the profitability of the business.
Assembly clamps represent a significant group of industrial equipment in manufacturing plants. Typically, any clamp is designed for specific, well-identified parts.
Thus, assembled parts, their shape, weight, type of surfaces, and many other parameters play a key role in the designing of assembly equipment, such as clamps.
The high level of flexibility of many contemporary robotic assembly systems corresponds largely to the ability to reduce or completely eliminate tooling.
There are several approaches to minimize the costs generated by the production and customization of tooling.

Modular Approach
The first approach is a widely used modular approach. It uses standardized profiles to build the equipment. The equipment components and clamps are produced in easy-to-connect modules.
The supporting structure and combination of elements depends on the manufacturer of the system. The main feature of such equipment is that it is easy to expand. Moreover, it is easy to adjust it to the current product. However, assembly equipment built from modules is flexible only to a certain degree, and a big change in the shape and dimensions of the product enforces a thorough rebuilding of the equipment.

Adjustable Elements
Another way to make the assembly equipment more flexible is to incorporate some adjustable elements. Such elements, driven or passive, can easily adapt to the fixed parts, yet again, the possibility to adapt and change the equipment is limited.

Sensory Systems
The most promising method seems to be refraining from immobilizing the parts using clamps, and using robotized stands equipped with sensory system instead. The parts may be placed in any orientation on a pallet or conveyor belts. The position of parts is determined on the basis of data from machine vision systems. Most often the parts can move in two directions and rotate in one axis on pallete or conveyor surface. The recognition of the type of item and its location can be based on the contour of the object, or extra tags such as bar or matrix codes.Another approach to the reduction of assembly equipment is the use of force sensors. Analysis of the forces acting on the grippers enables to compensate for deviations of the position of objects and eliminate the need to use clamps. The use of this technology and its integration with the assembly process are discussed in [20].
The elimination of the need to keep parts in place with the use of expensive and complex tooling is desirable not only in assembly operations but also in other stages of manufacturing, such as laser cutting [21] or dimensional inspection [22].

OPTICAL SCANNER APPROACH FOR ASSEMBLY
The presented robotic assembly station recognizes shifts and rotations in all three directions, i.e. in 6 degrees of freedom. For this purpose, it uses the entire surface of the part within the reach of camera, not just the outline. The assumptions of the publication reflect the case of real flexible production: − there is a database containing 3D part models and a description of how to combine them in automatic cycle, − production batches are so short that the development of assembly equipment for parts is economically unjustified, − the elements on a pallet are unprinted and can be found at any position, − the gripper and the laser line generator are placed at manipulator arm end using the special adapter ( Fig. 1).

Figure 1 Test stand model
The robot sweeps a laser beam along the workspace measuring the elements located there. The result is a cloud of measured points. A set of algorithms running on a PC recognizes the objects, that is, matches pieces of point cloud to models in the database (Fig. 2). Then the assembly task is carried out for the found components, using data about their current position. The assembly task performed by presented industrial robot station is shown in Fig. 3. Technical Gazette 25, 3(2018), 904-909

System Description
The test stand consists of a serial six-axis industrial robot ABB IRB 140, an industrial camera, a PC the conveyor that delivers the pallets. The adapter to which the vacuum gripper and the laser line generator are attached is fixed on the robot arm. A laser beam with a wavelength of 532 nm and the power of 6mW is diffused by a cylindrical lens and generates a line with an angle of 100°. Above the workspace, the camera with a resolution of 1600 × 1200 and a lens with a fixed focal length of 25 mm and a manually adjustable focus and iris is mounted on a fixed boom. The workspace in the plane of the pallet has dimensions of 350 × 260 mm. The robot laser beam sweeps the objects in the workspace, and the camera records a trace shape of the beam. By analysing the image, and knowing the relative position of the robot and the camera, a three-dimensional cloud of points which is the representation of the object in the workspace can be generated. Reconstruction of the surfaces of the objects is based on the triangulation [19].

Detection of the Models in Point Cloud Data
The cloud of points is an unstructured data source. It is an unordered set of vectors whose coordinates include the position, normal direction and colour of points. The challenge is to analyse this type of data in order to detect high-level dependencies, such as belonging to the object to be assembled. The result of the algorithm should be a list of the object identifiers and generalized transformation matrices that define the current offset and rotation of the object relative to the model stored in the database. The model in the database may be a threedimensional solid CAD model, mesh or point cloud collected with the use of the described system or any other 3D measuring device. The processing algorithm, proposed in this paper is multi-step. The outline of the algorithm is presented in Fig. 4.
The cloud of points obtained directly from the scanner is usually disturbed with points that have no equivalent in physical objects, the so-called outliers. The main reason for the phenomenon is the reflections of laser light from the reflective surfaces of the scanned objects. The effects of this phenomenon can be repaired by a procedure that removes points that are located at a distance from the nearest neighbors greater than the assumed threshold. The next step is down sampling, i.e. the reduction of the resolution of the point cloud. Redundant points whose distance from the nearest neighbour is less than another assumed threshold are removed. Down sampling distance threshold depends on the required speed of processing. Removal of part of the measured points significantly speeds up the clustering and detection, but may make it difficult to identify and precisely determine the location of the objects.

Figure 4
The method of locating objects in the cloud of points RANSAC is an iterative method of detection of models in the point cloud. In each step, a random location of the model in the point cloud is generated. This location is then checked to match the current data. To reduce the space in which the models are placed, first, the cloud of points is divided into groupings (clusters) and then principal component analysis (PCA) is performed. First, the base plane representing the pallet is removed. This is a kind of filtering the point cloud. It consists in removing of points, satisfying the condition z > -ε where ε is an arbitrarily small positive number. This simplified procedure is acceptable because the pallet plane is the basis for the scanner coordinate system defined in the calibration process. The remaining points are divided into N clusters based on the Euclidean distance. Subsequent cluster points are sought in the sphere of a given radius from the currently analysed point. The sphere radius should be smaller than the distance of physical objects treated as separate point clouds. If two or more objects are in contact and are classified as one group, models of all objects will be detected, but the processing time will be longer. It is possible to assign one object to several clusters, for example if a significant part of the surface of the object is occluded. With proper selection of parameters, the detection algorithm processing the object split into two groups should generate two identical model classifications and locations for both groups. PCA is used to calculate the Eigen values, whose physical equivalents are the moments of inertia relative to the determined Eigen vectors [23]. This analysis allows moving and rotating the group so that its centre is located at the origin of the coordinate system and the variance is the largest in the direction of the x-axis, and the smallest in the z-axis. PCA gives a good initial fit of models prepared in the same convention as the groups. The best results of PCA initial fitting are obtained for simple shapes, such as plates, whose surface is almost completely scanned at the assumed camera position. In the case of complex shapes, the share of occluded areas is significant androtation estimation is uncertain, but this does not exclude matching using the RANSAC method. This is a robust way to detect models in the point cloud, even if scanner fails to scan the entire surface. This method, described in [24][25][26], is based on the random generation of hypotheses, so the results are slightly different in every execution.
The RANSAC algorithms group is robust, it detects the objects even in case of poor quality measurement, where a large number of measurement artefacts occurs and only a part of the surface is available. However, obtained results are not very accurate and are characterized by significant volatility. To improve the accuracy of the fit, the Iterative Closest Point (ICP) algorithm can be additionally applied. This is an optimization algorithm, moving a model to minimize the mean square distance between the model and the point cloud. For proper operation of the algorithm, it is necessary to have a good initial fit, which is provided by the RANSAC algorithm. It is necessary to balance the number of iterations of these two algorithms in order to gain assurance of object detection, accuracy of fit and acceptable computation time.
In tests, the system uses the PCA, Euclidean clusters and RANSAC implementations available in the library PCL [27].

EXPERIMENTAL EVALUATION OF SOLUTIONS
To test the feasibility of the use of the presented set of methods in the assembly tasks within a flexible production system, repeatability of gripping objects placed in any manner on a work object pallet was verified. For comparison, the same repeatability measurement technique was also used to check the grasping of objects from the assembly clamp. The evaluation was performed by measuring the position of the planes of the parts using a dial indicator with an accuracy of 0.01 mm. The principle of the measurements is presented in Fig. 5.
In order to determine the element grasping error, the distance and an angle between base plane and the planes of the object are measured. For this purpose, the dial indicator conducts measure in two locations indicated in  Tab. 1 gives standard deviation of distance x and angle α for grasping the object from a fixed clamp and for grasping the object, based on data obtained from the scanner.  The impact of the number of iterations of the algorithm searches in the point cloud models on the accuracy of determining the position of models was also examined. Dependence of accuracy and search time is presented in Fig. 6.
Accuracy is measured as a root-mean-square deviation of model to the measured point cloud. The dependence is shown for different clouds resolutions. The difference in time is due to the use of a different number of iterations of the algorithm and changes in point cloud resolution. With an increasing number of iterations, the fit is better. Clouds at more points can give a better fit, but it is connected with an unacceptably long processing time. The processing time may be reduced, but it requires the use of other hardware configurations. The presented times refer to the searching of one model in a single cluster point cloud using a laptop with Intel® Core ™ 4702MQ. The processor can simultaneously process 8 single clusters point clouds.
For the part shown in Fig. 7, it was determined that it would be appropriate to down sample the cloud to around 4,000 points, which corresponds to the average distance between points equal to 2 mm. In case of objects or parts of other, more complex classes, results may be different.

Figure 7
Object to test the accuracy of the solution

CONCLUSIONS AND FURTHER RESEARCH
Assembly with no clamps is a method of reducing the cost and increasing the flexibility of equipment for fully automatic assembly. It leads to the minimization of the number of product specific clamps and equipment.
The presented system is suitable for assembly of various products such as plastic packaging, and for the palletizing of products.
The tests show that the resulting accuracy of the positioning of parts collected by the six-axis industrial robot manipulator based on data from a simple laser scanner is too small for precision assembly operations. Work on algorithms and hardware improvement in order to increase the precision of the system is planned.
The aim of the scanner is to identify the object in the palette and define its position. The initial location of the object is unknown, because the scan trajectory is always the same and covers the entire pallet plane. This extends the scan time and calculations. The scan field could be limited, if positions of parts on a pallet were predefined. The processing time can be significantly reduced if the number of models that are matched to the cloud points can be reduced. In further research it is planned to include a preliminary analysis of the workspace image. The system with additional classifiers that work on flat images could initially determine the types of objects on the pallet and their location. Analysis of point clouds allows for the precise determination of the position of the elements in the 3D space, which would be impossible on the basis of a single plain image.