Deep learning based fruit and vegetable recognition for android pos devices

Abstract

With the recent improvements in technology, time and expense-saving products have gained an important trend in marketplaces. Recent increases in shopping trends created a need for faster payment technologies. Even though the barcode system is currently one of the most popular technologies being used, this system is not fault-tolerant and likely to be inefficient for unpackaged products by being human dependent. As an example, fruits and vegetables are two of the most popular unpackaged product categories that are being priced based on the amount bought and cashiers usually enter a product code manually in case of purchase. Along with that, some businesses have installed self-checkout tables where customers can handle their own payments to decrease shopping time in rush hours and save on employee expenses. But self-checkout tables depend on customer trust when it comes to unpackaged products since any customer can select a product of a different price from the list. At the same time, self-checkout tables are very costly for most businesses. As a solution to the aforementioned problems, a system is proposed in this project that aims to benefit in security, expense, and time. The system aims to be advantageous by not creating an additional hardware expense by using the Point of Sale (POS) devices that most businesses already have. Focusing on fruits and vegetables, a recognition system is added using the device camera to prevent human-dependent security problems of the existing systems. Various techniques are experimented with to achieve a real-time system. In this project, a dataset consisting of 14 types of packaged and unpackaged fruits and vegetables is used. Related works usually implemented object classification and object recognition algorithms for similar problems but since objects can exist in different locations and amounts on a frame, an object recognition algorithm is decided to be more suitable for this project. Along with object detection algorithms being more complex than object classification algorithms, having POS devices with limited resources created a risk of the device being insufficient against the high computational needs of the system. For this reason, decreasing the model size and making the model closer to real time by decreasing the computation time became one of the purposes of this project. Therefore, among the object detector methods, it is decided to select a one-shot detector model. "You Only Look Once (YOLO)" is one of the state-of-the-art one-shot algorithms and is a well-known algorithm that has several versions developed over years. In this project, two of the latest versions; YOLOv4 and YOLOv5 are used and compared under several performance metrics and the best results are obtained with YOLOv5 with a mAP score of 98%. Later, several quantization methods are examined and compared for the purpose of creating a model of smaller model size and better performance. Among the quantization methods, best results are achieved with Full Integer Quantization, and model size is decreased by 75%. The proposed detection model is deployed on an android-based 400TR POS device developed by Token Financial Technologies with MT8167A (1.5 GHz) CPU. On the final system, inference time is observed as 1.332 seconds.