Machine vision is a branch of computer vision, mainly referring to the visual control or visual applications in the field of industrial automation. It involves extracting relevant information from digital images captured by front-end optical cameras, and then analyzing and processing this information for controlling the production line process or quality, replacing repetitive manual operations. At present, the practical applications of machine vision in the factory production process include: integrity quality inspection of workpieces, recognition, classification, part positioning and orientation, size measurement, code reading and traceability, surface/color difference detection, automated assembly line operations, and so on. It can be said that machine vision has provided a pair of "eyes" for the production line and automated equipment, capable of providing all-weather, high consistency, and high reliability of inspection results, adding to the intelligent level of factory production, and has become an important technology towards intelligent manufacturing.

According to statistical data from relevant institutions, in 2019, the global market size of machine vision technology used in the field of industrial automation was about 5.37 billion US dollars, and it is expected to reach 12.29 billion US dollars by 2023, with a compound annual growth rate of 21%, and the market potential is huge.

From the overall supply chain, manufacturers involved in the development of machine vision systems can be roughly divided into upstream basic developers, including core components and software developers; midstream equipment integration and software service providers engaged in secondary development; and downstream various machine vision application manufacturers. Among them, core components and software suppliers mainly include manufacturers of light sources, lenses, industrial cameras, image acquisition cards, image processing software, etc. In the entire cost structure, the cost of components and software development accounts for about 80% of the proportion, which is the most important. Light source suppliers include OPT Ophtech, Rui Vision Optoelectronics, etc.; the main brands of industrial cameras include Basler, B&R, DALSA, Point Grey, Images Source, etc.; well-known image software packages on the market include Halcon (MVtec), VisionPro (Cognex), free open source library OpenCV, MIL (Matrox), etc.; motion control card suppliers include Gute, Leisite, Advantech, and so on.

Advertisement

At present, machine vision in the field of industrial control mostly belongs to the secondary development level, and the forms mainly include system integration and assembly production automation special machines, etc., requiring developers to master the Know-How technology in different production and manufacturing environments, and design targeted products or integrated solutions. Visual dispensing control, visual screw machine control, visual positioning + flying shooting, visual inspection and other electronic product post-manufacturing processes are all hot spots in the current market development. Most manufacturers adopt the development model of "software platform + vision development kit", and the development kit encapsulates various common image processing algorithms based on the software platform. Software engineers can directly call the encapsulated algorithms to implement various complex image processing and analysis functions, greatly reducing the difficulty and workload of secondary development.

The three major difficulties of visual application

However, in terms of practical application, in the current implementation process of machine vision + motion control solutions, various "thresholds" are still high, and the problems mainly come from three aspects: First, for most technical development engineers, machine vision technology is a new field. Learning a new programming language (such as C++) has a certain difficulty and requires a lot of time and energy. In addition, vision projects are generally non-standard, and the experience of developers is very important. It is not easy for ordinary electrical engineers to quickly master the programming and debugging of machine vision applications.

Secondly, for equipment manufacturers, in order to improve the production efficiency of the factory, the demand for upgrading the manufacturing line is urgent, and the application of machine vision has become a focus. However, the hardware investment cost of machine vision has always been high, which makes them hesitate; in addition to the hardware investment, it is often necessary to hire high-priced vision engineers to develop and maintain the purchased vision system, which undoubtedly increases the labor cost of the factory. How to effectively control the investment cost is their main demand point, such as whether it is possible to add machine vision functions to the existing equipment system in an economical and convenient manner.

In addition, for end users, in the process of using machine vision technology, they will find that machine vision and automated control (process control, process control, etc.) often belong to two different systems, and there may be instability in the entire system due to communication and other factors (such as shutdowns, line stops, etc.). Especially in the current production mode of multi-variety, small batch, and flexibility, the frequent replacement of the production line increases the modification and debugging time of the machine vision system process, thereby greatly affecting the factory's production capacity. Therefore, how to make the equipment more stable in production and more convenient to operate, and meet the needs of the production line for rapid adjustment and switching, is a subject of machine vision technology development.Seamless Integration of Machine Vision and Motion Control

Generally speaking, most hardware architectures for machine vision + motion control applications use a single controller, which is externally connected to the vision system and industrial intelligent cameras and light sources at various points on the production line. The vision signals are transmitted to the controller through cables and serial ports. In the internal software framework, taking common computer vision as an example, the vision program is usually located in the non-real-time user mode of the Windows system for code analysis and algorithm processing; while the PLC process control is mostly in the real-time kernel mode. Due to the task execution mechanism of the non-real-time user mode adopting priority levels, when the CPU load is too high, memory is too large, or processing procedures are too many, there may be a phenomenon of stuttering, making the time required for the machine vision in the non-real-time user mode to complete the same detection task different, which leads to slowing down the production rhythm and affecting production efficiency.

To solve this problem, Beckhoff has developed the TwinCAT Vision machine vision solution for machine vision applications, integrating vision technology into its unified general control technology platform, TwinCAT. The image processing function is added to the general control platform, TwinCAT software, which integrates PLC, motion control, robots, high-end measurement technology, IoT, and HMI, to complete all control tasks in the factory through this unified software platform.

Compared with traditional vision application methods, Beckhoff's TwinCAT Vision is also on a single controller, in addition to integrating motion control, I/O control, and PLC control, the external industrial camera is directly connected to the machine vision module in the controller. In the internal software architecture, TwinCAT Vision transmits the captured image signals to the PLC machine vision and process control system in the real-time kernel mode through the GigE camera driver, running precisely in a "time slice" manner. All control functions related to image processing can be precisely synchronized in real-time in the operating system, thereby eliminating delays and executing image processing algorithms in real-time. The advantages of this solution are that a single controller completes all control tasks, reducing hardware costs and simplifying the system; the real-time system runs vision tasks, ensuring that the response time for each execution is stable enough, speeding up production rhythm, and improving factory capacity; task memory shares data between tasks, and multi-task cooperation (such as vision + motion control, robotic arms/humans, etc.) is more efficient and convenient; on the Beckhoff TwinCAT platform, only IEC programming languages (ST/LD, etc.) are needed, and the original PLC programming skills and habits can be continued, shortening the development cycle, reducing labor costs, and also using the vision function blocks that Beckhoff has already integrated for code development, bringing convenience to technical personnel.

The core of chip technology and algorithm library at the bottom layer

Industrial production has high requirements for the computing speed and accuracy of machine vision, which requires strong underlying technology support, such as the performance of the CPU central processing unit chip, the characteristics of FPGA devices for parallel processing of multi-level pipelines, and professional vision algorithm libraries, to improve the data throughput and processing capabilities of the machine vision system.

In May this year, Xinjie of Wuxi and Intel joined hands to release a new range of products based on Intel chips, including IoT industrial controllers and intelligent cameras. Based on the Intel 14nm Apollo Lake & Kaby Lake platform, the V200|V300 series IoT industrial controllers launched by Xinjie have powerful computing performance and optimized image processing capabilities, offering a variety of configuration options. The S200 series intelligent camera, based on the Intel Apollo Lake platform, adopts the IA+FPGA architecture, integrating image acquisition, image processing, and external communication, supporting 1.3 million to 12 million pixels. Compared with traditional solutions, the S200 series intelligent camera has a maximum surface temperature below 50°C when fully loaded, providing a safety guarantee for intelligent manufacturing. In addition, the new X-SIGHT VISIONSTUDIO software platform has more than 2000 processing operators, including common detection, positioning, calibration, communication, character reading, etc., without the need for writing code, just drag and drop components to complete development, support multi-threaded multi-task processing, and provide a rich set of HMI interaction modules. It is reported that in addition to the products and solutions launched at this conference, Xinjie and Intel will carry out comprehensive cooperation and exchange in the fields of painting, grinding, artificial intelligence recognition, sorting, and other industrial manufacturing.

The development of machine vision based on FPGA devices can conveniently modify its internal logical functions through programming, thereby achieving high-speed hardware calculation and parallel computing, which is a more convenient solution for high-performance embedded vision systems, and the power consumption is also low, which is an important development direction for future computer vision systems.

In terms of vision software, Vision Pro is an excellent computer vision software developed by Cognex, characterized by its simplicity and ease of use. Its Quick Build rapid prototype design environment combines the advanced and flexible nature of advanced programming with ease of development, assisting manufacturers in shortening the development cycle time. With VisionPro, users can access a powerful pattern matching, spot, caliper, line position, image filtering, OCR, and OCV vision tool library, as well as one-dimensional barcode and two-dimensional barcode reading, to perform various functions such as detection, recognition, and measurement. VisionPro software can be fully integrated with a wide range of .NET class libraries and user controls. In addition, by connecting to VisionPro ViDi, a deep learning software specifically designed for industrial image analysis through the API, it can further optimize complex detection, component positioning, classification, and optical character recognition applications.

Integrated machine vision solutionTargeting the "pain points" in the existing production process visual applications, starting from the integration of software and hardware closely connected with machine control, B&R's integrated vision system, which includes lenses, industrial cameras, and software, is another way to implement it.

This integrated vision system, which combines software and hardware, solves the synchronization issues in visual inspection applications. First, it is the synchronization of lighting and imaging. For vision, good lighting design constitutes 70% of the success weight of visual applications. The integrated vision solution is controlled by the same controller, which allows the LED intensity of lighting, stroboscopic, and the camera's focus, exposure, and imaging process to achieve synchronization at extremely high speeds, thus ensuring image quality. Second, it is the synchronization between the vision system and the automation system. The integrated vision solution seamlessly integrates visual applications with machine processes and control, and together with B&R's own control, motion control, and I/O modules, achieves microsecond-level synchronization under the POWERLINK architecture, allowing the overall system to achieve higher precision and speed.

On the other hand, this vision integration solution is very convenient for developers of field integration applications. They no longer need to understand the knowledge of various lenses, cameras, controllers, lighting, and communication interfaces. They only need to use one product to build various visual applications, shortening the project development cycle and greatly reducing engineering time and cost. By using the ready-made software components provided by B&R's mapp technology framework, machine vision applications can be easily created with less programming. It only takes a few clicks to integrate the images captured by the smart camera into the mapp View HMI application, and the camera and lighting parameters, as well as triggering conditions, can be changed at any time, making product changeover and other running time adjustments very easy to implement.

In fact, in various complex industrial production scenarios, the light source is particularly important, as it is directly related to the quality of the collected images, such as whether the image contour is clear, whether the imaging features are perfectly reflected, and so on. For this reason, B&R's integrated vision system also provides a very flexible light source management function, including modular design of background light and light strips, which can be electrically rotated from -40° to +90°, and built-in strobe controllers, support for various colors, high-power, and fast response. These can be configured according to the on-site application and can also be modified in real-time, such as exposure time, angle, color, etc. These functions make machine vision applications more flexible and free.

Conclusion

The combination of machine vision and motion control brings a new vitality to the upgrade of industrial automation production lines and is increasingly applied in non-contact detection and measurement scenarios to improve processing accuracy, discover product defects, and perform automatic analysis and decision-making, thus becoming an important part of advanced manufacturing. At present, although the application of machine vision technology in the industrial automation industry involves multiple disciplines such as optics, electronics, mechanics, and industrial software, the technical difficulty is still relatively high, and the market access barriers in some specific fields are also high; however, it is obvious that the combination of machine vision and motion control technology has promoted the optimization of factory production line automation level, product quality control, and labor cost structure, thereby enhancing the comprehensive competitiveness of enterprises. In the future, machine vision systems that integrate AI technologies such as edge computing and deep learning will truly implant "eyes and brains" into machines, preparing for the era of Industry 4.0.