Image analysis based symbol recognition in colored maps

Abstract

In the tests of software containing maps, especially military software, the verification of military symbols is mostly done manually. The fact that the symbols are very varied and some of them are similar to each other can make it difficult to remember the meanings of the symbols and lead to incorrect verification. Additionally, performing the tests manually requires verification for each symbol in the image one by one. Repeating the same manual testing process for software whose new versions are constantly released causes a lot of labor consumption. Automating manual visual verification processes in software tests by making them independent of the software test engineer will save time and reduce the possibility of making mistakes. Recognition of symbols on the map will be automated with a symbol recognition method that finds the location of the symbols on the maps and performs the classification process. Symbol recognition has many application areas. There are many different symbols around, from cosmetics to food and beverage packaging, from banners to signs. These symbols give a lot of information. Because there are so many symbols and their number increases rapidly, it becomes difficult to remember the meanings of the symbols. For this reason, symbol recognition methods are used when detecting mathematical symbols, logical circuit symbols, map symbols, musical symbols, traffic signs, and brand logos, etc. It is thought that the symbol recognition methods used can also be used to recognize symbols on the map. Research in this field offers different approaches to symbol recognition with several image analysis techniques. However, there are some difficulties in recognizing symbols placed later on color maps. To briefly summarize these challenges: • The complexity of the map image and the variety of colors the map contains can make it difficult to recognize the symbols on it. • Symbols may be the same shape but in different colors or very similar to each other. In this case, the probability of confusion when detecting symbols increases. • The scale of the symbols may vary depending on the scale of the map. If the symbols are too large in some images and too small in others, this may cause the symbol not to be recognized. • Symbols may not be filled. In this case, symbol recognition may be difficult because the background of the transparent parts of the symbol will constantly change. • Symbols may overlap and one may cover part of the other. The bottom symbol may not be detected. As can be seen from the difficulties mentioned above, symbol detection is a complex process. This problem can be solved to a certain level with correct feature extraction and classification algorithms to obtain correct results. However, it is seen that greater success has been achieved in symbol recognition with advanced artificial intelligence techniques compared to other methods. In this study, a dataset was produced for the problem of symbol detection on maps. While preparing the dataset, the map types used in military systems were researched and it was determined that elevation and satellite map images were mostly used. For this reason, these map types were used in the dataset. Colorful and complex map images from different regions of the world have been selected, resembling maps used in real military systems. Then, the symbols used in military systems were examined and similar symbols were designed. The difficulties mentioned above were taken into consideration when designing the symbols and they were produced to include difficult situations. Afterward, the prepared symbols were placed randomly on the map images with the background as mixed colors as possible. Some symbols were positioned on the map, overlapping each other, and some symbols were positioned at different scales, and label files were created. The results were obtained using image analysis techniques such as Template Matching, HOG (Histogram of Oriented Gradients), SVM (Support Vector Machines), Faster R-CNN and YOLO (You Only Look Once) methods for symbol detection on the colored map were compared. According to the results obtained by testing the methods mentioned in this study, the behavior of the methods against the difficulties defined above is presented in the table. Looking at the results obtained, the most successful methods were deep learning models. With Faster R-CNN and YOLO methods, mAP result was obtained over 90%. In this thesis, the methods used for symbol recognition in the literature were examined and the use of deep learning models was suggested to automatically detect the symbols on the map. Results suggest that in the software testing process, it is predicted that automatic symbol recognition using deep learning models will provide gains such as reducing the number of faulty tests with a high accuracy rate, completing the test processes faster, and reducing the labor spent on repetitive processes.