Robotic Generative AI 3-Cup Game

Overview

Developed the vision system for a “3-Cup and Ball” game played by a robot. This project was showcased at Intelligent Asia 2024 in Taipei, within the National Science and Technology Council booth.

The Vision System

The core challenge was tracking a ball hidden under moving cups.

1. Detection: Utilized YOLOv7 to detect the cups and the ball in real-time.
2. Tracking: Implemented the SORT algorithm (Simple Online and Realtime Tracking) to maintain object identity across frames.
3. Logic: Combined visual tracking with a custom algorithm to predict which cup contained the ball after shuffling.

Impact

This system demonstrated effective Human-Robotic Collaboration (HRC) by allowing a robot to interact dynamically with human players in a game of chance and perception.

Publication

• Title: A Case of Cups and a Ball: Utilizing Generative Artificial Intelligence for Human-robotic Collaboration in Task Execution
• Conference: 2024 International Automatic Control Conference (CACS), Taoyuan, Taiwan.
• DOI: 10.1109/CACS63404.2024.10773301