CAD-Based Object Localization for Robotic Applications

The background for this project lies in the increasing demand for autonomous robotic systems in manufacturing, logistics, and industrial automation, where robots are expected to handle complex tasks such as object recognition and manipulation with high precision. Current state-of-the-art systems often rely on pre-trained models and object datasets for vision tasks but face limitations when dealing with new or uniquely shaped objects. We are looking for 2 Master Students with skills and interest in programming and/or control to develop a system as described below. The scope of the project can be scaled up or down based on the team.

The project will involve the following components/topics in the order of skills required (high to low):

• Programming: Python
• ROS2
• Robotics: UR5e/UR10e setup, integration, control
• Machine Vision/Deep Learning (PyTorch)
• CAD: To design and print parts for experiments
• Docker: Containerized development

The aim of this project is to enable a robotic arm (UR5e/UR10e) to detect and localize novel objects based on given CAD models (STL/OBJ) and perform a robotic task like assembly. Using deep learning for vision, the project will explore how CAD data can be leveraged to teach robots to detect, localize and interact with objects in real-time. Students will implement this solution on an edge-compute device like Nvidia Jetson and integrate the NVIDIA Foundation Pose model (and others) for pose estimation