ifm – the secret experts in image processing

With more than 7,000 employees worldwide, ifm electronic gmbh is one of the leading names in the sensor industry. Perhaps a lesser known fact outside the industry is that the company has also been extensively and successfully involved in image processing for many years.

The ifm portfolio includes position and process sensors, sensors for motion control, safety technology, industrial communication, condition monitoring systems, connection technology and many other product areas. The Essen-based company is well established on the international market as a leading manufacturer in these segments, but has also been a specialist in industrial image processing for more than ten years now, says Product Manager Mike Gonschior, who is responsible for the vision activities of ifm: “We have been working in this field for a long time, focusing on four product areas: vision sensors, 3D sensors, 3D cameras and illumination units. Our products from these categories are already being used in numerous applications around the globe.”

When talking about vision sensors, Gonschior refers to compact units integrating all the components required for image processing in a robust, industrially compatible housing. This includes lighting, lens, camera chip, evaluation and process connections for the integration in the higher-level controller. “Today’s automation technology would be inconceivable without such vision sensors, as they are an integral part of assembly, production and quality control,” says Gonschior.

At Hannover Messe 2019, ifm will present new multicode readers for 1D and 2D codes. (Image source: ifm electronic gmbh)

A highlight of this ifm segment is the O2D object recognition sensor, which can detect repeatable and defined shapes of objects irrespective of their orientation and is therefore the ideal choice for tasks like presence and position monitoring, quality control, sorting and counting. For applications where the objects vary in shape, size or shade, ifm offers the O2V object inspection sensor. “And at this year’s Hannover Messe, we will be introducing a new generation of vision sensors – the O2i series,” announces the ifm Vision Product Manager. “This multicode reader for 1D and 2D codes is suited both for easy tasks with just one code and for very complex applications with parallel image captures for different codes. The sensor’s operating concept offers the customer three different setting modes: one-button teaching to set up simple applications without the need for a PC, an app for fast teaching of multiple parameters and the user-friendly Vision Assistant software for more complex tasks.” According to Gonschior, other interesting features of the new product include automatic focusing for different reading situations during operation, multi-image teach options for learning several codes simultaneously, and a large USB data storage for storing faulty codes. Various 1D and 2D code readers as well as user-friendly software packages for these products round off the ifm vision sensor range.

3D sensors with time-of-flight technology

With its photoelectric 3D sensors, ifm targets applications such as volume, distance or level detection. The technical basis for this product group is the Time-of-Flight principle (ToF): The sensors illuminate the scene with an internal or external infrared light source and calculate the distance between the sensor and the object surface by means of the travel time of the light reflected from the surface.

“A particular strength of our 3D sensors is that we have a competent partner at our side thanks to our cooperation with the Siegen-based company pmdtechnologies ag, allowing us to develop our own chip designs and thus react with great flexibility to specific market requirements,” emphasises Gonschior. pmdtechnologies ag is an established German manufacturer of 3D ToF sensors and has been a wholly-owned subsidiary of ifm since 2013.

Mike Gonschior: “With its products, ifm offers powerful possibilities for solving image processing applications in a variety of industries.” (Image source: ifm electronic gmbh)

Another sensor based on the patented pmd technology is ifm’s current 3D flagship O3D. It uses a special time-of-flight 3D chip to determine the distance and grey-scale value for each pixel. “The function of this chip is extremely reliable even in poor and changing light conditions,” says Gonschior, providing some application examples of the O3D sensor. “This product is ideally suited for many tasks in the packaging and logistics industry, e. g. to check handling units for underfill or overfill or to detect missing or incorrectly positioned objects as well as reject material in palletising applications. The sensor is completely colour-independent and, being a one-sensor solution, offers outstanding accuracy for object dimensioning.”

With the help of the parameter setting software ifm Vision Assistant, the image data recorded by this product family can be used to solve comprehensive applications. “This software is very easy to use so that even users with little experience in image processing can quickly and easily set up our 3D sensors,” says Gonschior. Typical applications can be found in the area of packaging, storage and materials handling, airport logistics, monitoring of space and persons, and robotics.

Application solutions ifm Vision Assistant (Nov. 2018)

• Flexible automation of robot grippers
• Depalletising of uniform packages
• 3D level measurement and point level detection
• Completeness monitoring for the packaging industry
• Volume determination for storage and conveyor technology

Versatile 3D cameras

ifm’s 3D cameras are based on the same hardware as their 3D sensors and also use the time-of-flight principle based on the 3D ToF sensor chips from pmdtechnologies ag to detect scenes and objects in their spatial dimensions.

There are even more ways to use the O3D cameras: For more complex applications, the recorded point cloud can be sent to a PC or controller via the ifm SDK or further processed for the robot operating system ROS, the free programming library pcl for processing point clouds or the image processing software library Halcon with the help of drivers. This makes it easy to seamlessly integrate the O3D cameras into a variety of automation environments. ifm has also developed versions for use in mobile machines providing enhanced shock and vibration resistance.

Linkliste

• ifm-SDK
• Github (ifm)
• Github (pcl)
• Halcon

A strong partner

It is no coincidence that Gonschior identifies the field of robotics as a particularly important application area for ifm’s image processing components: “In this segment, we are working very closely with a US company that was previously known under the name of Love Park Robotics and is now a wholly-owned subsidiary of ifm.” The Philadelphia-based company develops special software algorithms that allow robots to perceive their environment in 3D and react accordingly. “This works very reliably even in harsh industrial environments. By combining this software with our 3D sensors and cameras as well as suitable robots, we can provide extremely powerful and flexible robot vision solutions for applications in a variety of industries.”

ifm also collaborates with other partners such as the image processing software specialist MVTec from Munich, using their imaging library Halcon to provide powerful software development options for image analysis in combination with their own 3D sensors and cameras. “With these partners and the chip design capabilities of pmd as well as our own image processing expertise, we can offer our customers interesting options in this field,” says Gonschior.

Successful examples

Cleaning robots from Adlatus Robotics, equipped with O3D cameras from ifm, will soon automatically clean the platforms of Deutsche Bahn. (Image source: ifm electronic gmbh)

That this statement is not just wishful thinking is proven by numerous applications which became reality in the industry and in other application areas in the last years based on ifm’s image processing components. For example, the Ulm-based startup company Adlatus Robotics recently signed a two-year contract with Deutsche Bahn concerning the supply and further development of cleaning robots after winning Germany’s first cleaning robot contest. The winning system uses the O3D camera from ifm as its “eye”.

Another example is the “EasyPilot” grape harvester of the French company Grégoire, which replaces about 40 to 60 manual workers. Based on the patented pmd technology and the time-of-flight principle, ifm’s 3D sensor system O3M ensures automatic line guidance of the machine with a precision of 3 cm without needing a GPS position signal, ensuring accurate automated grape harvesting in the vineyard.

In agriculture, many other applications have been successfully implemented based on ifm’s image processing components. For example, the units are used to monitor filling cones during filling of loading areas by combine harvesters to ensure better load distribution. Robust, time-of-flight-based ifm systems help milking robots when it is time for automated milking: They detect the teat position, enable more precise application of care products and ensure efficient and gentle processes.

According to Gonschior, smart homes are another promising area of application for ifm’s image processing solutions: “Interactive camera assistants based on deep learning and artificial intelligence technologies make it possible to increase security in your own four walls. Such systems can identify the inhabitants of a house and even distinguish between adults, children, pets and objects. They can even recognise familiar faces and detect actions such as opening cabinet doors or other movements in the home, notifying home owners in such cases.” These systems are also based on 3D sensors from ifm with the basic sensor technology from pmd.

The 3D sensor system O3M from ifm ensures automatic line guidance of the “EasyPilot” grape harvester with a precision of 3 cm, ensuring accurate automated grape harvesting in the vineyard. (Image source: ifm electronic gmbh)

As Gonschior points out, there are many other successful applications of ifm image processing components. These include collision avoidance systems for robots or automated guided vehicle systems, service robots in the consumer goods sector used to direct customers in stores, stock-taking robots that inspect shelves in supermarkets and notify their human colleagues of shelves to be refilled, robots that automatically deliver orders such as drinks to hotel guests or security robots that detect unauthorised intruders or even weapons in industrial estates or parking lots using thermal imaging.

“The possibilities are virtually endless. With our product portfolio in industrial image processing, our strong partner network and ifm’s broad range of services, we hope to significantly expand our position in this segment in the next years and establish ifm as a high-performance player in the market,” Gonschior describes the goal for his area of responsibility. Given the market potential and ifm’s possibilities in terms of technology and service, the chances of achieving this goal are high.