Computer Vision and Robotics Use Case

Computer Vision Changes the Scope of Industrial Robotics

 

THE CHALLENGE

The first industrial robot was invented in 1954 and was installed seven years later in a General Motors factory for spot welding and die-casting. Since then, robotic technology has been used in industries from manufacturing to agricultural farming as a means to increase efficiencies, lower costs, and increase revenues. These robots are usually designed to work independently, executing pre-scripted tasks in spaces protected from human interference. They have increased factory productivity but have limited capabilities.

Cobots, or collaborative robots, are a new step in industrial robot technology. Unlike most robots, which act as replacements for human workers (and often operate in cages to prevent injury to workers), cobots are designed to work sideby- side with their human counterparts, even collaborating on the same task. How do these robots gain new abilities that can increase their operational value while remaining safe and secure as they operate in a factory near humans?

THE APPROACH

One way to increase robotic abilities in a safe and efficient manner is to use an innovative new technology: computer vision. This technology enables a robot, or computer system, to use a camera or scanner to transform multidimensional inputs into data it can process, “perceiving” its surroundings and mimicking sight. Computer vision coupled with machine learning gives the computer increased technical abilities and the opportunity to perform more complex tasks. Robots accessing computer vision gain abilities beyond scripted tasks and can augment the abilities of their human coworkers by participating in their labors or by using technologies such as infrared imaging to see and report on things invisible to the human eye.

This technology dramatically increases the potential for robotics in industry, creating avenues that would not otherwise be viable. For example, using an AI-enabled cloud, connected robots could recognize objects faster and send collective messages, notifying or warning humans of situations that they could not see. They could also aid in quality control, as they could be able to recognize the condition of products when compared against the expected visual representation.

Similar advantages could pertain to agricultural production. Independent robots using computer vision could differentiate between product quality levels; for example, the robot could use imaging types in both visible and ultraviolet light to detect below-surface discrepancies and extract a higher profit from varying products by identifying food grades. It could even warn for diseases, such as peach leaf curl on trees, that would significantly reduce productivity if not treated.

ROBOTICS CHALLENGES

  • Increase the capabilities and productivity of robots working in industrial, agricul tural, and other fields
  • Increase the safety and security of robotic operations with and near human workers

WIND RIVER SOLUTIONS

  • Wind River Helix Virtualization Platform: A real-time, embedded, Type 1 hypervisor that can manage unmodified guest operating systems running in virtual machines, consolidating robotics control system applications and providing safety, performance, and flexibility for the opera tion of a modern robotics system
  • VxWorks: The world’s leading RTOS, enabling deterministic applications scaling from very small compute packages that provide the real-time performance required for safe and efficient robotic functions
  • OpenCV for VxWorks: A custom modification of the Open Source Computer Vision Library git repository to allow integration with VxWorks 7, with complex algorithms to allow computers to recog nize multidimensional environments
  • Wind River Titanium Cloud: Portfolio of virtualization software products that enable a modern cloud infrastructure for industrial applications (including robotics), reducing OPEX and increasing agility
  • Wind River development tools: Powerful tools, such as compilers, analyzers, tuners, debuggers, and full system simulators, to save developers time and increase quality

Wind River Titanium Cloud

Adding new technologies would enable the machines to do more, but new approaches would be required, including a virtualization system to enable the running of multiple applica tions. Wind River® Titanium Cloud™ is a portfolio of virtualization software products that can meet this need, enabling a cloud infrastructure for robotics applications. Implementing an ultra-low latency system such as Titanium Cloud, coupled with advanced 5G technology, would allow robots to communicate with one another almost instantaneously, sending data throughout the factory and creating a cohesive, connected network to further streamline production.

VxWorks

The VxWorks® real-time, deterministic operating system enables rapid data processing for real-time actions that will allow robots to work reliably and consistently in close quarters with human employees. The robot would be able to rapidly process data, including potentially dangerous changes in its environment, and immediately respond to them. VxWorks supports the ROS 2 operating system framework, a re-architecture of the original ROS framework, enabling use cases such as teams of multiple robots and production environments on a real-time platform.

OpenCV for VxWorks

Implementing computer vision can be a daunting task, as it takes thousands of lines of code to allow computers to process sensory data. However, Wind River offers a modification of the open source computer vision library, OpenCV, optimized for VxWorks. With more than 22,000 commits and nearly 800 contributors, OpenCV for VxWorks includes algorithms to recognize faces, classify human actions, identify and track objects, track camera movements, and more, adding complexity to robotic software without increasing development time.

Wind River Helix Virtualization Platform

Wind River Helix™ Virtualization Platform is a virtualization technology that uses a Type 1 hypervisor. It is based on VxWorks, so it can enable a real-time virtualization platform to run multiple applications in virtual machines for new robotic functions. It can provide a foundation for operating a cloud-based artificial intelligence that can provide data-driven decisions for robotic operations in a factory. Helix Platform allows multiple coding environments to exist in the same system, operating as independent virtual machines on a single compute platform.

Helix Platform in conjunction with Titanium Cloud would create a low latency system with the safety, security, and reliability required in critical infrastructure environments. Moreover, they can work with VxWorks to create a cohesive real-time environment to incorporate computer vision and 5G in a cloud based system, while maintaining elite cybersecurity standards.

THE RESULT

Currently, industrial robots harbor many potential safety dangers, as they have no awareness of their surroundings other than what is provided by sensors; this could cause serious harm to people working nearby or alongside them. However, with the addition of new sensing technologies, robots could be used in closer proximity to humans and in more confined spaces, so that factory workers and robots would be able to safely work in tandem. Both the production capacity and the safety of the factory could increase. Robots could perform more complex tasks, and they could operate in a disordered space by recognizing the objects they should interact with.

Wind River offers solutions that incorporate the latest ROS 2 framework, so developers can focus on application development, leading to more innovative robotics. Compute and partitioning capabilities can protect safety applications while providing the high performance that is important to enhance further collaboration between humans and robots.

To learn more about Titanium Cloud, VxWorks, or Helix Platform, visit the Wind River website or contact our sales inquiry desk.

Download PDF
Return to Resource Center