LEE- Bilgisayar Mühendisliği-Yüksek Lisans

Bu koleksiyon için kalıcı URI

http://hdl.handle.net/11527/19207

Gözat

Yazar "Bilen, Barış" ile LEE- Bilgisayar Mühendisliği-Yüksek Lisans'a göz atma
  • Öge
    Emotional awareness based adaptive social navigation for humanoid robots
    (Graduate School, 2024-07-01) Bilen, Barış ; Köse, Hatice ; 504201510 ; Computer Engineering
    Day by day, robots are becoming more and more involved in our daily lives. They continue to work and become integrated in many service sectors, from hospitals to schools. In order to achieve this integration, it has become a necessity for robots to leave factories and research laboratories and enter our daily lives. Naturally, this situation has required robots to be aware of the people around them, as well as recognising obstacles and objects around them. Beyond this awareness, it has become a necessity for them to recognise social cues and act accordingly. The development of social awareness in robots has paved the way for many research topics. The verbal or non-verbal communication methods that people use in their daily lives have been a motive for further research and further development of social awareness in robots. Robot navigation with social awareness is one of the fields that emerged in line with these needs. It is a field that focuses on navigating robots with social awareness while moving between two points without disturbing the people around it, as if it were a social member of that environment. The emotional states of people have a significant impact on their daily lives. The violent and sharp behaviour of an angry person or the tolerance and goodwill of a happy person are among the best examples of such effects. If the emotional states we are in have such an impact on our daily lives, it is equally important to provide this awareness to robots. The aim of this thesis is to investigate the effects of people's emotional states on their personal and social spaces and to make these outputs a part of social navigation. During this study, a new robot navigation framework with emotional awareness has been developed. The framework has been designed to protect people's physical wellbeing during navigation by expanding and contracting social zone distances according to different emotional states, which many social robots follow today. This thesis is designed to answer the following questions: - Does the emotional state of people affect the size of their social space in their social environment? - Would an adult with negative emotions want a robot to pass by him/her at a distance further than the currently used social zone distances for his/her physical safety? - Would an adult with positive emotions tolerate a robot to navigate by at a distance closer than the currently used social zone distances for the optimization of the robot's path? - If the person the robot is navigating around is a child with positive emotions, would it be appropriate for the robot to pass at a closer distance than the current social space distance? - If the person the robot is navigating around is a child with negative emotions, would it be appropriate for the robot to pass at a further distance than the social space distance currently used? In order to investigate the answers to these questions and to include emotions in social navigation systems, three different surveys are conducted with a simulation environment, a game environment and real robot recordings done by us. In addition, a navigation system that recognises emotion and organises the navigation according to the emotional state of the people has been developed. This navigation system allows the local planner to pass farther away from the angry person or closer to the happy person by making expansion or contracting adjustments in the social zones of the people according to the emotional state that it recognises from the faces of the people. First of all, a two layer control system has been established for the robot to detect and recognise the people around it. The first layer detects the foot coordinates of the people from the laser sensor data, and the second layer confirms these foot coordinates with the YOLO deep learning model through the camera. When the person is detected by the system, the emotional state of the person is labelled as positive, neutral and negative by a deep learning model that performs emotion analysis. A new cost map has also been developed and added to the layered cost map system in order to make the expansion and contraction adjustments in the social zone of the person with the emotional state of the person. In the survey conducted with the simulation environment, the participants are shown videos of the person in three different emotional states (positive, neutral, negative). The participants are asked at what distance the robot should pass according to the emotional states of the person. In addition, NARS questions are asked to the participants and their attitudes towards robots are measured. It is statistically analysed whether the participants had a negative attitude towards robots, and it is observed that they have no negative attitude. The results of the survey showed that a person with angry emotions preferred to be passed farther away, while a person with happy emotions did not prefer to be passed closer during robot navigation. For the second survey, a game in which participants control a robot in a hospital environment is used. In this study, in which both children and adults participated, the participants are asked to perform the tasks in the game and answer the survey questions before and after playing the game. The distances between the robot and the NPCs in different emotional states that the participant encountered in the game are tracked. In these results, it is observed that participants (adults and children) passed farther away from angry NPCs and closer to happy NPCs than neutral NPCs. The validity of these observed data is also analysed statistically. In the pre-game surveys, the attitudes of the participants towards robots are measured, and it is observed that they did not have negative attitudes. In addition, they are asked at what distance they think it is appropriate for the robot moving around an angry or happy individual or a child to pass by the person. It is observed that the participants wanted the robot to pass far away for angry individuals, at an average distance for happy individuals, and at a long distance for children. In the post-game survey, it is measured how realistic the game environment and the way it is played felt to the participants, and it is concluded that the participants found the game environment and the way it is played realistic, and also these results are supported by statistical tests. In the survey study conducted with real robot videos, participants are shown videos of robots passing around people (adults or children) with different emotional states. They are asked which of the distances between the robot and the human they found appropriate, and whether they found the robot behaviours appropriate in scenarios with multiple people. Emojis are used instead of the faces of the people in the videos. In order to measure whether the emotional states in these emojis are perceived correctly by the participants, the participants are asked which emotional state the emojis reflected. At the beginning of the survey, as in other surveys, questions are asked to measure the attitudes of the participants towards the robot, and it is observed and statistically shown that the participants are not in a negative emotion. According to the results of the survey, in scenarios with a single child or adult, if the person in the video is angry, the participants wanted the robot to pass from a distance, while if the person is happy, they did not want the robot to pass from a closer distance. In scenarios with multiple people and emotions, the fact that the robot passes far away from angry people and close to happy people is found appropriate in terms of robot behaviour by the participants. When an adult individual or a child is in a negative state of emotion, it is considered appropriate by the participants that the robot passes at a distance that is definitely farther than the currently used social distance. This situation is supported by the statistical analysis of the survey results. When an adult individual or a child is in a happy state of emotion, mixed results are obtained in the case of the robot passing at a distance closer than the currently used social space distance. Looking at the survey results, the participants did not want the robot to pass closely, while in the game environment, they showed a reverse attitude and preferred to pass close to happy NPCs. Both the survey results and the game data are statistically analysed and verified by tests. Based on the results of the survey conducted with 194 people in total, it has been proved that the emotional states of people affect the social space distances.