Week 1: Basic Understanding and Assembly of Device Components

About this project

Our project focuses on developing a hand gesture recognition system using a Raspberry Pi AI camera. The system is designed to recognize users' hand gestures and enable interactive gameplay for rock-paper-scissors. 

By leveraging computer vision and machine learning algorithms, the camera captures real-time hand movements, accurately identifies gestures, and determines the game outcome instantly. This project not only demonstrates the potential of AI-powered interaction but also showcases how embedded systems like Raspberry Pi can be used for fun and educational purposes.

What We've Achieved - Week 1
After receiving the equipment, we first familiarized ourselves with each component by examining the packaging and visiting the manufacturers' websites. This helped us understand the model, working principles, dimensions, and assembly methods of the parts. Following this, we proceeded to assemble all the components.

          

After our initial understanding of the camera, we found that The Raspberry Pi AI Camera works differently from traditional AI-based camera image processing systems, as shown in the diagram below. The left side demonstrates the architecture of a traditional AI camera system. In such a system, the camera delivers images to the Raspberry Pi. The Raspberry Pi processes the images and then performs AI inference. Traditional systems may use external AI accelerators (as shown) or rely exclusively on the CPU.
The right side demonstrates the architecture of a system that uses IMX500. The camera module contains a small Image Signal Processor (ISP) which turns the raw camera image data into an input tensor. The camera module sends this tensor directly into the AI accelerator within the camera, which produces output tensors that contain the inferencing results. The AI accelerator sends these tensors to the Raspberry Pi. There is no need for an external accelerator, nor for the Raspberry Pi to run neural network software on the CPU. that




After confirming the connections were correct, we successfully powered on the device. By connecting a keyboard and mouse, we completed the device login and checked for updates. By running the detection program, we briefly confirmed that the camera was functioning properly without any issues, completing this week's tasks.

Preparation for Next Week

Next week, we will attempt to use BrainBuilder to create a custom training dataset for recognizing three hand gestures: rock, paper, and scissors. We need to capture and collect images of hands showing these three different gestures, including variations in shooting angles and lighting conditions. Each of the three team members will take 150 photos of one specific gesture among the three (rock, paper, or scissors) to prepare for next week's AI training.

The image set of the "rock" gesture taken by team members