Apresentao do PowerPoint

A Robotic System for Welding Groove Mapping:Machine Vision on Metallic SurfacesBQ Leonardo, CR Steffens, SC Silva Fil., JL Mr, V Httner, EA Leivas, VS Rosa and SSC Botelhocristianosteffens@furg.brCenter of Computer Science, Federal University of Rio Grande, Brazil

1

Hello!My name is Cristiano Steffens and we are presenting a Robotic System for Welding Groove Mapping: Using Machine Vision on Metallic Surfaces.I am with the center of Computer Science at the Federal University of Rio Grande, in Brazil1

A Robotic System for Welding Groove Mapping:Machine Vision on Metallic SurfacesWelding is easy! Isnt it?Manual process affects the quality of the weldReworkMaterial wasteWeak and breakable final productReproducibility and regularityThe human sideWelding is unhealthy ergonomy, heat and fumesLaborious and repetitive task

BQ Leonardo, CR Steffens, SC Silva Fil., JL Mr, V Httner, EA Leivas, VS Rosa and SSC BotelhoFederal University of Rio Grande Brazil http://c3.furg.brTypical Setup of a Linear Welding System

Proposed Video-Based Measurement System

Algorithm Structure

Use CaseBUG-O MDS Welding RobotRobust Modular RobotCan be used on a large variety of surfacesAble to make different welding seamsLincoln Flextec 450 Power sourceLincoln wire feeder

ContributionsModular VBM for linear welding robotsEnd-to-end embedded welding system prototypeComputer Vision applied to reflective surfaces, without the need of structured light, polarized lenses or complex optical arrangementsState of the art algorithms offer better cost-benefitDeveloped a complete solution, featuring illumination, image acquisition and processing, robot operation and welding equipment setup

2

2

Welding is easy! Isnt it?Manual process affects the quality of the weldReworkMaterial wasteWeak and breakable final productReproducibility and regularityThe human sideWelding is unhealthy ergonomy, heat and fumesLaborious and repetitive task3

Welding is a fundamental task in the heavy steel industry. Its automation is required in order to keep pace with the demanding and competitive market.Today, many industries still rely on manual welding process for industrial production.

The manual process, however, has a direct impact on the quality of the final product.

First and more visible at the business management level, as the manual process depends on the weldors ability and skill it is susceptible to human error. A moment of inattention is often enough to result in hours of rework and material waste.

Sometimes, even if the visual inspection does not show that the process failed at some point, the final product may be rejected on the quality inspection stage or even reach the final consumer. You see, nobody wants to buy a buy a pig in a poke! (SIC)

Finally, it is very difficult to ensure reproducibility and regularity on a manual process.

Now, if we look at the welding activity from the operator side, we can note that the automation of the process would have an impact on its quality of life. Welding fumes have been associated to many lung diseases. The heat produced during the process, which the operator is subjected during many hours a day has also been associated with male infertility. The position and the repetitive movements are also associated with many occupational diseases.

3

Prior approaches for welding process automationA combination of structured illumination laser and camera, as used in Kawahara (1983), Drews et al. (1986), De Xu (2004), Liu (2010), and Zhang et al. (2014)

A touch sensor based approach as in Kim and Na (2000);

Techniques where the arc current feedback is explored, as in Dilthey and Gollnick (1998) and Halmy (1999);

4

When we talk about linear welding automation, 3 main approaches can be enumerated:The first approaches where based on structured illumination and cameras or specialized sensors. The main idea here is that you can project a pattern on the welding plate surface and it allows you to recognize the geometry.

Latter, a touch and sensor has been proposed. It is a mechanical device that is able to follow or measure a groove.

Some techniques also tried to find the proper settings for each welding plate by starting the process and then adjust the ecquipment parameters based on the electric arc measurement.

4

Typical Setup of a Linear Welding System

Typical linear welding robot installation5

Here we can observe the typical setup for FCAW and GMAW welding used in shipyards. A 2 DoF robot is used to carry the welding gun and execute the weaving.A separate power source is used.The system runs over rails which can be attached to the ship hulk in many different positions.5

Typical Setup of a Linear Welding System

Operating. Source: BUG-O Systems (2013)6

Here we have a Picture of the robot operating.6

The BUG-O MDS Welding RobotRobust Modular RobotRails and CarriagesLinear WeaverPendulum WeaverCan be used on a large variety of surfacesAble to make different welding seamsWeldor adjusts the linear rail and the parameters in runtime7

Proposed Vision-based Measurement System

High-level architecture of the vision-based measurement system8

The main contribution of our work is a VBM module to be used on top of the welding robot structure, assuming the control of the welding process.From a top view input image, we extract the groove edges and find its geometric properties.

An Image Acquition module is implemented used off the shelf componentes and a Altera DE-0 Nano FPGA board. The designing and programming of this system is done using VHDL and Verilog hardware description languages.

The Operations Unit is implemented in a standard PC software, which is developed in C++ taking advantage of many functionalities provided in the OpenCv Library.

8

Prior WorkVBM

Zhang, W. Ke, Q. Ye, and J. Jiao, A novel laser vision sensor for weld line detection on wall-climbing robot, Optics & Laser Technology, vol. 60, pp. 6979, 2014.9

present a cross-structure light (CSL) sensor, that consists a structured light projector anda camera, for weld line detection. The structured light projector projects cross laser beams on theweldment to form cross stripes, which are captured in images by a CCD camera for measurement. We usefeature points, a planar target and a homograph matrix to calibrate the sensor. We also propose aneffective approach to extract laser stripes in images for weld line detection. Experiments show that theCSL sensor can capture 3D information of the weldment with very low measurement error, and the weldline detection approach is effective in wall-climbing robotic platform navigation.9

Prior WorkVBM

Drews, B. Frassek, and K. Willms, Optical sensor systems for automated arc welding, Robotics, vol. 2, no. 1, pp. 3143, 198610

In many cases the automation of arc welding processes cannot be realized because the permissible workpiece tolerances are exceeded. Extensive workpiece preparations are often not practicable because of economic reasons. Therefore appropriate sensing systems for seam tracking and joint recognition have to be developed, which allow an adaptive control of the welding process and guarantee a satisfactory quality of the weld. Some special developments of optical sensing systems for automated arc welding are presented in this article.10

Prior WorkVBM

D. Xu, M. Tan, X. Zhao, and Z. Tu, Seam tracking and visual control for robotic arc welding based on structured light stereo vision, International Journal of Automation and Computing, vol. 1, no. 1, pp.6375, 2004.11

presents a technology about real-time seam tracking, which is necessary to overcome thedeficiencies of the teaching-playback welding robots in seam tracking control during gas tungsten arcwelding (GTAW) process. A set of vision sensor system has been designed for the welding robot, which canacquire clear and steady welding images. By analyzing the features of welding images, a new improvedCanny algorithm has been proposed to detect the edges of seam and pool, and extract the characteristicparameters of welding images. Based on the analysis of the characteristic of the real-time seam tracking,a segmented self-adaptive PID controller is introduced to the system, and some experiments have beendone to testify whether the accuracy of the technology can meet the requirements of quality control ofseam forming

11

Proposed Vision-based Measurement System

Image acquisition setupWelding groove properties

12

On the right, we have a representation of a commonly used V-shapped welging groove. Here, we look for the properties that describe the groove geometry.Gap A is the distance within plates in the top part. Gap B is the distance within the places on the bottom part of the welding plates. Plate thinkness and computed bevel angle complete the description.

On the left, we presente the trigonometrical basis that enable us to compute the geometry knowing only the working distance, lens and sensor properties.12

Overview of the VBM System13

Machine Vision System (HW)

Altera DE0-Nano FPGA.Source: AlteraTerasic D5M CMOS CameraSource: AlteraIllumination14

Machine Vision System (HW)

FPGA + FTDI breakout(PC communication)

Overview of the prototype15

Machine Vision System (HW)Overview of the prototype

16

Algorithm StructureNormalization and Histogram Equalization

Noise reduction (Gaussian, Mean, Median filters)

Edge and line detection (Canny + Hough, PPHT, LSWMS, EDLines)

Heuristics and Non-Maxima Suppression

Pixel to metric unit conversion

17

The algorithm follows a straightforward approach. First we work on contrast enhancement and noise reduction. Then, we use line segment detectors to determine the edges of the welding groove. Once we have the candidate regions we apply Heuristics and NMS to remove any false positives.From the triangulation we can calculate the pixel to metric conversion. The measured groove dimensions are then used as input for the equipment configuration step.17

Algorithm ComparisonGap AGap BAlgorithm Average Std. Dev. Average Std. Dev.Gauss + EDLines + NMS19.992 0.3705.487 0.407Mean + EDLines + NMS20.103 0.3835.863 0.739Median + EDLines + NMS20.214 0.2905.555 0.505Gauss + LSWMS + NMS18.9912.1975.846 0.797Mean + LSWMS + NMS19.643 0.3896.2561.134Median + LSWMS + NMS18.6262.6406.0511.123Gauss + Canny +PPHT + NMS7.98710.3122.0512.657Mean + Canny + PPHT + NMS20.151 0.3925.333 0.417Median + Canny + PPHT + NMS23.1846.3805.5211.304Gauss + Canny + Hough + NMS 0.000 0.000 0.000 0.000Mean + Canny + Hough + NMS1.9536.176 0.5811.838Median + Canny + Hough + NMS4.0178.4701.1282.391Ground Truth19.916 0.2516.558 0.258

18

Results show the EDLines from Akinlar et al. (2011) algorithm provides the best results. Using any of the noise suppression kernels its standard deviation is still lower than the other tested combinations on both Gap A and Gap B measurements.The next best solutions, which minimize error and standard deviation are obtained using the LSWMS algorithm by From Nieto et al (2011).18

Integration

Figure 7 System Integration in Embedded Hardware19

The proposed system is integrated in embedded hardware as shown in the figure. 19

Ongoing: Results of the Measurement System (Best-Case)

Gap B - Plate BottomGap A - Plate TopMean ErrorStd. Dev.Mean ErrorStd. Dev.0.143mm0.084mm0.780mm0.157mm

Measured/position x Ground TruthRepeatabilityTable 1 Gaussian filtering + LSD by Von Gioi (2012)20

Ongoing: Debevecs HDR CompositionHDR Input images

HDR composedLine segment detectionFinal groove modeling21

Ongoing: Method Comparison

Figure 17 Error and Std. Deviation in millimeters for Gap A (smaller is better)22

Video (https://youtu.be/-fONDmtlnpw)

23

Ongoing and Future WorkExplore lighting options, noise suppression algorithms and image composition techniques to improve the systemBilateral and L0 gradient minimization filtering (not trivial to implement)Compare Debevecs multi-exposure composition to other approaches that minimize the computational cost and are hardware-friendlyOnline application - mapping while weldingDeep learning based image restorationProduce a general purpose welding workcell

24

ConclusionModular Vision-Based Measurement for linear welding robotsEnd-to-end embedded welding system prototypeComputer Vision applied to reflective surfaces, without the need of structured light, polarized lenses or complex optical arrangementsState of the art algorithms offer better cost-benefit ratioDeveloped a complete solution, featuring illumination, image acquisition and processing, robot operation and welding equipment setup

25

In conclusion, we buit a modular VBM system for linear welding robots.We show that machine vision can be applied even in an hard contexto such as metallic reflective surfaces and that it can be done avoiding complicated hardware setups.We found that state of the art algorithms offer a better cost benefit ratio.And, finally, we presented a complete solution, featuring illumination, image aquisition and processing, robot operation and welding equipment setup.

25

ContactSilviaCB@furg.brCristianoSteffens@furg.br

http://c3.furg.brhttp://nautec.furg.br/

26