Don Braggins, UK Industrial Vision Association looks at the use of robotics in manufacturing

The use of vision in the automation of manufacturing processes is well documented, but most application areas tend to concentrate on the use of vision as an inspection device for quality control. Linking of such systems into automatic pass/fail selectors or even into statistical process control for the elimination of waste by adjusting the process before it goes out of specification are powerful uses of vision, but the technique also has an important role to play in robot vision applications.

Robot Vision
The combination of robotics and image processing enables the fault free, fast, reliable and economic manufacture and quality assurance of all kinds of products in many areas. There are two distinct ways in which vision and robotics combine in an automation environment. In the first, image processing provides mechanical robots with a new mode of perception by giving them added "vision" which is used to locate parts or guide the robot (pick and place) and in the second, the robot interacts with the vision system either by presenting parts for inspection or transferring parts to the next stage of the process based on the results of a vision inspection.  Both can have huge advantages over manual systems by reliably automating transfers from one process to another or to feed parts into a system. Robot vision can simplify the automation by reducing or eliminating the need for lots of bespoke mechanical handling as well as being able to cope with many different parts within a single system.

Vision For Robotic Engineers
For pick and place applications, it is necessary to locate an item, identify what type of object it is and its orientation and then get accurate co-ordinates to pick the item up. In the past, users were often very reluctant to integrate image processing into their robots. This was because providing an interface to the camera and configuring the whole assembly was very complex, demanding expertise in both robotics and image processing. To overcome this, a number of companies, including UKIVA members, FS Systems, Scorpion Vision and Stemmer Imaging have developed imaging interfaces for robot applications. FS Systems’ RoboVis package has been designed specifically for robotic pick and place tasks and includes a wide range of features to simplify implementation of such systems in manufacturing applications, providing direct communication with the robot controller via Ethernet, RS232 etc. Typically the application is designed to enable minimal tooling for component transport systems. In a different approach, Tordivel  (developers of Scorpion Vision software) has worked with Sony to develop a high-level interface for Sony desktop robots. This interface is designed for production engineers, rather than vision system experts. In addition the software can be used (without any programming) to set up command strings for any robot interface using a broad range of protocols including RS232, TCP/IP and Profibus. Similarly, Stemmer Imaging, KUKA Switzerland and the Interstaatliche Hochschule für Technik (Interstate College for Technology NTB) in Buchs, Switzerland  have developed a robotics software package, V4R, using Stemmer’s Common Vision Blox imaging toolkit. The software runs directly on the controller of the KUKA robot, allowing users with robotics experience to code the vision application in the language they are familiar with. They can teach the system how to recognize objects and translate this into automatic operation. In addition, information about object parameters is transferred to the robot control system using the coordinate system of the robot. With the recent introduction of the GigE Vision standard, a GigE Vision camera can plug directly into the KUKA robot controller and all the program scripting is done inside the KUKA language, with just the vision settings set by the configuration graphical user interface.

3-D Applications
In the examples described above, the image processing systems are taught to recognize various products in situ using sophisticated 2D shape recognition algorithms. However, recent advances in the ability of machine vision to make 3D measurements at speed using laser triangulation methods, has opened up new possibilities for pick and place by introducing a 3D element. This is particularly suitable for applications where the heights of the items to be picked are variable, and where traditionally manual labour is required to inspect, sort and pack. This is expensive and is a potential bottleneck in manufacturing throughput.

Surface Inspection
With or without robotic involvement, vision has an important role to play in surface inspection of raw material, as well as part processed and completed product. In addition to the traditional area scan cameras that look at an area of interest, linescan cameras are frequently used in surface inspection applications. These cameras have a single row (or line) of active pixels, (from 512 to 12K, depending on the resolution requirement) and either the target is moved underneath the camera, or the camera is moved over the target. Either way, an image is created by building up a sequential series of lines from the camera. This technique is particularly useful for imaging continuous materials such as steel sheet, or long components such as pre-formed car bumpers which can be inspected for defects. Sometimes, a combination of area scan and linescan cameras can provide a multiple inspection capability. Such a combination is used in the CapaCam inspection system, developed by the Fraunhofer Institute for Integrated Circuitry (IIS) in Erlangen, and in use at the Daimler AG factory in Hedelfingen, Germany. Using up to 4 area scan cameras and 2 linescan cameras from Stemmer Imaging, CapaCom detects pores, blow holes and other damage to complex cast components and inspects several thousand components in three shifts each day. The converter housings fitted in automatic car transmission systems are made from die-cast aluminium and are then cut, and deburred using a high pressure water jet, before being dried. The quality of the component's machined surfaces are crucial, as this ensures that they function properly as a seal. The most commonly occurring faults in castings are pores and blow holes, tiny holes in the casting which are sometimes exposed after a surface has been machined. In addition, handling the components or using blunt, worn-out tools for machining can damage them. Powerful surface inspection algorithms are at the heart of the application software which can be adapted for use with a single computer, right up to parallel processing using computer clusters, in order to fulfil the cycle times required. Flexible integration into the production line is achieved with a standardised industrial bus.

Selecting Items For Painting
An example from the plastics industry shows how vision and robotics can be linked into a production line painting application, but could readily be applied to other finishing applications. Lear Corporation (now part of the International Automotive Components Group) made plastic car interior parts. The requirement was to establish an automatic system to paint hundreds of plastic parts of many different shapes and sizes. Software from Scorpion Vision was used to identify each part and verify it against the parts list, checking that its measurements match the details held in an external library. Once the part is measured, identified and verified, the information is transmitted to the production system to start the correct robot painting program for the part. Each part hangs on a frame suspended from the ceiling. As the frame passes the camera, the camera takes an image. Due to the lack of space between rear wall and the frame, the camera position typically gives a perspective error of +/- 10% which leads to measurement errors. The main challenges faced by the vision system were:
• 100s of different parts to recognise
• Camera position meant there was an acute angle causing a perspective error
• Wide angle lens had to be used which caused large image distortion

The software has excellent image processing tools which can be used to correct the lens distortion. Figure 1 shows the hanging frame before correction and Figure 2 shows the corrected image.  This correction is applied to every image before the inspection takes place, allowing the measurements to be made and part identification to be performed correctly. Figure 3 shows a frame with the components correctly positioned and aligned for painting, while Figure 4 shows a similar frame with a slight positioning and fixture error. This system has increased throughput whilst at the same time improving quality control. It has also minimised wastage by preventing bad or incorrect parts being sent for painting. Also, the painting program is guaranteed to be the correct one, further reducing wastage. Multiple systems are in continuous operation at the factory with no downtime.