Hand-Controlled Robotic Arm

Exploring Hand-Controlled Tele-Operation of a Robotic Arm on the Tria Vision AI-KIT 6490

Not a Medium Member ? Refer to my full write-up on Hackster:

• [Hackster] Hand-Controlled Robot Arm

Introduction

This demonstration shows how to use the Tria Vision AI-KIT 6490 for robotics applications. More specifically, it shows how to use a Computer Vision based Hand Controller to control the LeRobot robotic arm.

In the previous demo, where we used a hand controller to control a wheeled vehicle in simulation. This time, we will control a real robotic arm, in addition to the simulated robotic arm.

Distributed ROS2 Application

The demo is a distributed ROS2 application. This means that the demo runs on two different hardware devices:

• Vision AI Kit (QCS6490, running QIRP v1.6)
• Host PC (laptop or workstation, running Ubuntu 24.04)

Hand-Controlled Robotics Demo — Distributed ROS2 System (📷: AlbertaBeef)

Both devices are running ROS2 Jazzy, and interact with each other using ROS2’s DDS protocol over ethernet.