Build Self Checkout Machine & Virtual Keyboard with OpenCV

Welcome to Building Self Checkout Machine & Virtual Keyboard with OpenCV course. This is a comprehensive project based course where you will learn step by step on how to build a fully automated self checkout system and interactive virtual keyboard using OpenCV, Keras, Convolutional Neural Networks, Media Pipe, and Tkinter. This course is a perfect combination between computer vision and object detection, making it an ideal opportunity to practice your programming skills while improving your technical knowledge in retail automation. In the introduction session, you will learn the basic fundamentals of the self checkout system, such as getting to know its use cases, technologies that will be used, and some technical challenges. Then, in the next section, you will learn how the self checkout machine works. This section will cover data collection, preprocessing, model training, object detection, matching the product to the dataset, displaying product name and price. Afterward, we will create training data which will consist of one folder containing product images and an excel file containing product information like product ID, product name, price, and discount. Once, everything is all set, we will start the first project, firstly, we will train the self checkout model using CNN and Keras, after that we will build simple user interface using Tkinter and we will also embed OpenCV webcam to the interface, once the camera has been activated, the user will be able to scan products and the system will automatically calculate the total price. In addition, we will also create a simple payment simulation where users can enter the payment amount and the system will check if the entered payment amount is more than the total price, if yes, then it will display the change but if the entered payment amount is less than total price, the system will ask the user to enter the right amount. Meanwhile, in the second project section, we will build an interactive virtual keyboard using OpenCV and Media Pipe. This system will be able to recognise hand gestures and provide users with touchless typing experience. After building these two models, we will be conducting testing to make sure these models have been fully functioning and all logics have been implemented correctly. Lastly, at the end of the course, we will integrate the virtual keyboard to a self checkout machine, enabling users to scan products and complete payments by entering the payment amount directly on the virtual keyboard using a hand gesture, ensuring a smooth and efficient checkout experience.

First of all, before getting into the course, we need to ask ourselves this question: why should we build an automated self checkout machine and virtual keyboard? Well, here is my answer, long queues and slow checkout processes in retail can frustrate customers and affect store efficiency. Building an automated self-checkout machine and a virtual keyboard can greatly enhance the retail experience by streamlining transactions and improving customer satisfaction. The self-checkout machine speeds up the checkout process, reduces wait times, and minimizes the need for human labors, leading to increased operational efficiency. Meanwhile, the virtual keyboard offers a touchless input method, enhancing hygiene and convenience in high traffic environments. Moreover, by building these innovative projects, you will gain valuable skills in automation that are transferable across various industries.

Below are things that you can expect to learn from this course:

• Learn about self checkout machine and retail automation, such as getting to know its use cases, technical limitations, and technologies that will be used

• Learn how self checkout machines work. This section will cover training data creation, preprocessing, model training, product scanning, displaying product information, and payment

• Learn about virtual keyboard and how this technology enables users to type in using finger movement without physically touching the keyboard

• Learn how to create training data consisting of product images and products informations like product ID, product name, price, and discount

• Learn how to activate webcam using OpenCV

• Learn how to create function to load product images from training data

• Learn how to train self checkout model using Convolutional Neural Network and Keras

• Learn how to build self checkout machine using OpenCV and Tkinter

• Learn how to create function to detect object and recognize product

• Learn how to create function for payment processing simulation

• Learn how to design custom virtual keyboard layout

• Learn how to integrate hand tracking and detection system to virtual keyboard

• Learn how to build virtual keyboard using OpenCV, Tkinter and MediaPipe

• Learn how to design simple graphical user interface and create button using Tkinter

• Learn how to conduct performance testing on self checkout machine and virtual keyboard

• Learn how to integrate virtual keyboard to self checkout machine