3D vision helps precisely palletize parts

The Spanish robotics specialist Tekniker developed a solution for parts to be automatically gripped from containers and placed in other containers in an orderly manner. The integrated 3D vision software MVTec HALCON ensures precise gripping of parts. Reliable part recognition allows the entire workflow to be optimized and automated, speeding up palletizing, saving costs, and relieving employees.

Fully automated part-picking by robots is a major technical challenge. Components aren’t always delivered to the robot in an orderly fashion, but rather unsorted and even overlapping each other, meaning humans still must intervene frequently. Tekniker has developed an application based on machine vision enabling fully automated bin-picking with two robots. A particular highlight is the 3D vision technology Surface-based Matching recognizing the delivered components and acting as the robots’ eyes and enabling precise gripping.

Relieve employees through process automation

“Until now, the palletizing of parts was carried out purely manually,” explains Kepa Laka, automation and robotics area manager at CIKAUTXO GROUP, a Tekniker customer. “Our goal was to automate this process end-to-end to speed it up, increase productivity, and save costs. In addition, our workers were relieved of monotonous, ergonomically unsound tasks so they could devote themselves to more demanding tasks.”

Instead of implementing a mechatronic system offering low flexibility, low reliability, and generating a lot of noise, Tekniker decided to use a machine vision-based solution.

The system setup consists of two industrial robots from KUKA, a 3D scanner from Photoneo, a depth camera from Intel, a PC running the machine vision software, and a SIMATIC PLC from Siemens to control the cell. A robot removes stamped parts in large quantities from the input bin using a magnetic multifunction gripper and deposits them on two conveyor belts. The removal is stopped as soon as a certain volume of parts is reached on both belts. This is calculated using data provided by the depth camera and a customized HALCON algorithm combining 2D and 3D vision tools.

The machine vision software also continuously processes the data from the 3D scanner and provides the robots with the part position and orientation best suited for gripping on the conveyor belt.

With this data, the second robot begins to pick the parts from the conveyor belts, aligning them so it can sort them into two target bins in an orderly manner with the help of 3D vision. After the first robot stops picking from the input bin, it helps the second: it also picks up parts from the conveyor belt, aligns them, and sorts them into the target container. This works at eight seconds per part. The parts must be inserted into guide bars within the target container with the front facing upward and with a tolerance of 0.5mm. Should the gripped part face downward, it’s placed in a rotating device to pick it up again in the correct position.

Precise localization of disordered parts with HALCON

The challenge in this setup is robots need to precisely recognize the position and orientation of components in 3D space to pick them up and put them down again safely.

“To implement this requirement, we have developed the SMARTPICKING software and integrated it into the overall solution,” explains Jorge Molina, researcher in the smart and autonomous systems department at Tekniker. “The software can accurately identify objects by analyzing 3D data and using the surface model as well as the corresponding geometric information. The machine vision software MVTec HALCON is at the heart of the SMARTPICKING solution. It provides sophisticated algorithms and tools to precisely locate the parts lying chaotically on the conveyor belt and transmit the coordinates of the pick point to the robots.”

Classic 3D-based image processing methods are used including the Surface-based Matching feature integrated in HALCON. The technology uses data from 3D point clouds captured by the 3D scanner. The data is further processed using filters and preprocessing operators to improve the pose estimation of the parts, allowing for a robust positioning of objects – regardless of the surface. Now, robots can recognize the parts and their exact position and grip them unerringly for different parts as they occur in industrial production processes. Reliable position determination is ensured even for deformed or smooth surfaces without distinctive edges showing significant gray-value differences in conventional images. For a safe gripping process, the robots must determine the relative position of the 3D camera in relation to the respective object exactly in their coordinate system. A hand-eye calibration is performed with MVTec HALCON and a calibration plate. This calibration is essential to establish the relationship between camera, robot, and parts so the robot senses exactly where its gripper is in relation to the object to be gripped.

Avoiding collisions during pick-and-place

In addition to detecting and determining the position of the parts to be gripped, possible collisions between the robot arms and individual objects (components, containers, etc.) must be avoided during the entire pick-and-place workflow. The 3D Object Processing technology integrated in HALCON can determine parts lying favorably on the conveyor belt that can be gripped without collision. Based on this initial selection, the software determines the optimal candidate using decision criteria such as the height and orientation of the parts or the position of the gripping point and returns the coordinates to the robot. The robot can safely carry out the next gripping process. The controlled sequence allows the removal of components without collisions and disruptions to the workflow. The software also detects the remaining capacity of the target container. For this purpose, operators precisely calculate its volume. A previously defined threshold value can then be used to determine whether further parts can be added to the container.

Increase process speed by up to 100%

“Thanks to the machine vision capabilities in MVTec HALCON, we can automate and optimize the robotic palletizing process end-to-end. The software offers sophisticated machine vision tools that can precisely locate the parts to be picked up. This ensures safe and smooth pick-and-place processes. In addition, we can categorically exclude collisions of the gripper arms, ensuring the safety and continuity of automated palletizing,” Molina sums up. “As a result of using the system, we can increase process speed by up to 100%, improve productivity accordingly, and relieve our employees of routine tasks,” Laka confirms.

Motors for advanced robotic systems

PHOTOS COURTESY OF RESPECTIVE COMPANIES

MOONS’ Industries offers a range of motors designed for robotic applications. Due to their unique construction, MOONS’ slotless motors offer higher torque density, improved heat dissipation, and reduced cogging torque and torque ripple. Slotless motors are ideal for precise motion control, efficient thermal management, and smooth, cog-free operation.

MOONS’ frameless motors eliminate traditional motor housing, allowing unparalleled torque density and integration flexibility within tight spaces. MOONS’ frameless, brushless BLDC motors consist of only the stator and rotor elements, resulting in an ultra-compact form factor integrated into robotic joint assemblies, end-effectors, and other space-constrained applications. Despite their minimal footprint, these motors are customizable and offer a high torque-to-weight ratio, enabling high-speed operation and dynamic motion control capabilities.

MOONS’ motors can be combined with harmonic drive gearheads and other components to create customizable motion control solutions. Harmonic drive gearheads’ compact design and torque transmission capabilities enhance the agility and accuracy of robotic systems. Integrating with frameless BLDC motors optimizes energy usage and reduces operational costs.

MOONS’ Industries

Software updates fix bugs in robotics software

SprutCAM Tech updates for SprutCAM X and SprutCAM X Robot add support for new robot models, introduce profile mode for lathe operations enhancement, and fix bugs throughout the software.

User experience enhancements include:

• Updated pop-up hints for lathe machining, engraving, pocketing, and chamfer machining operations
• Improved functioning of pinned snapshots when clicking Clear All in project snapshots manager
• Enhanced overall performance of SprutCAM X and SprutCAM X Robot
• New feature allowing users to save tutorials on post processor development as an archive file, enabling offline access
• Added machine setup template for Swiss-type machines
• Increased speed of machining report generation with sketch output

Updated smart hints for lathe part off, 5D by meshes, turn takeover, 3D helical, and scallop operations

SprutCAM Tech

Denmark cobot hub

Universal Robots and MiR celebrated the Grand Opening of their new 215,000 ft² headquarters in Odense, Denmark. Partners from UR and MiR’s global ecosystems attended the event, including NVIDIA and Siemens, with whom Teradyne Robotics and its companies have recently announced several collaborations.

The opening included a panel discussion featuring Deepu Talla, vice president of Robotics and Edge Computing of NVIDIA, Rainer Brehm, CEO of Siemens Factory Automation, and Ujjwal Kumar, group president of Teradyne Robotics. The panelists discussed the use of physical artificial intelligence (AI) across industries, underscoring the transformative power of advanced automation for businesses and workplaces.

MiR

Universal Robotics
IMTS booth #236131

Adaptive robotic solutions

Flexiv announced a strategic cooperation agreement with Siemens, joining the Siemens Xcelerator ecosystem. This collaboration will enable Flexiv and Siemens to co-create and deliver adaptive robotic application solutions to industries within the robotics market.

As a member of the Siemens Xcelerator ecosystem, Flexiv will leverage the Siemens Workflow Canvas platform, an integrated information technology/operational technology (IT/OT) development kit, to create customized adaptive robotic applications. This program creation is further enhanced by the Workflow Canvas, which facilitates rapid digital integration of IT and OT, enabling domain knowledge integration and value extraction. Through a graphical drag-and-drop interface, a standardized common information model, a cross-platform SPIDR executor, and data fusion capabilities from IIH and Data Layer, the Workflow Canvas provides a digitally integrated delivery solution with customizable, open, and agile options for complex industrial scenarios.

Flexiv

Siemens
IMTS booth #133249