ISTANBUL TECHNICAL UNIVERSITY  GRADUATE SCHOOL DESIGNING AN EXPERT SYSTEM FOR NON-EXPERT USERS IN ORAL HEALTH: STORY OF A HYBRID DESIGN RESEARCH M.Sc. THESIS Deniz GÖÇHAN Department of Industrial Design Industrial Design Program JUNE 2023 ISTANBUL TECHNICAL UNIVERSITY  GRADUATE SCHOOL DESIGNING AN EXPERT SYSTEM FOR NON-EXPERT USERS IN ORAL HEALTH: STORY OF A HYBRID DESIGN RESEARCH M.Sc. THESIS Deniz GÖÇHAN (502171903) Department of Industrial Design Industrial Design Program Thesis Advisor: Prof. Dr. H. Hümanur BAĞLI JUNE 2023 AĞIZ SAĞLIĞI ALANINDA UZMAN OLMAYAN KULLANICILAR İÇİN UZMAN SİSTEM TASARLAMAK: HİBRİT TASARIM ARAŞTIRMASI HİKAYESİ YÜKSEK LİSANS TEZİ Deniz GÖÇHAN (502171903) Endüstriyel Tasarım Anabilim Dalı Endüstriyel Tasarım Programı Tez Danışmanı: Prof. Dr. H. Hümanur BAĞLI HAZİRAN 2023 İSTANBUL TEKNİK ÜNİVERSİTESİ  LİSANSÜSTÜ EĞİTİM ENSTİTÜSÜ v Deniz GÖÇHAN, an M.Sc. student of İTU Graduate School student ID 502171903, successfully defended the thesis/dissertation entitled “DESIGNING AN EXPERT SYSTEM FOR NON-EXPERT USERS IN ORAL HEALTH: STORY OF A HYBRID DESIGN RESEARCH”, which he prepared after fulfilling the requirements specified in the associated legislations, before the jury whose signatures are below. Thesis Advisor: Prof. Dr. H. Hümanur BAĞLI .............................. Istanbul Technical University Jury Members: Prof. Dr. Şebnem TİMUR .............................. Istanbul Technical University Jury Members: Assoc. Prof. Dr. A. Selami ÇİFTER .............................. Mimar Sinan Fine Arts University Date of Submission : 24 May 2023 Date of Defense : 23 June 2023 vi vii To Alan Turing, the founding father of artificial intelligence who saved millions of lives, viii ix FOREWORD The master's education and this thesis represent much more than just a degree and research in the field for me; rather a turning point in my life to succeed in studying in this school, which was my childhood dream. Design education has changed my personality and my perspective on life. Besides design, I learned a lot about myself and what I am capable of. I grew up. After years of training in the natural sciences as a dentist, I enjoyed the freedom that social studies give a researcher. I also gained a deeper understanding of the other values that make people human, besides flesh and bone. This education has a great role in the emergence of the project that I have been working on for the past three years, aiming to develop artificial intelligence in pediatric oral health. I would like to thank my teacher, mentor, and dearest friend, Prof. Dr. H. Hümanur BAĞLI. I learned a lot from her about not only design but also friendship. She is the closest witness to the change in me and one of the main characters of this story. I am grateful for her sincerity, love, and company. In 2015, when I knew nothing but wanted to study design, the first person I contacted in the department was Prof. Dr. Şebnem TİMUR. She welcomed me wholeheartedly, and thanks to her guidance, I was able to get accepted to the school. I am also grateful for everything I learned from her. I would like to thank all the professors and friends who taught me something on this path and made me who I am today. I would like to thank TÜBİTAK for the financial support. Thanks to the grant, this project has become my job and changed the course of my life. I would like to express my gratitude towards Nuray Öner GÜCİN and Elçin KURBANOĞLU for their invaluable support. Their presence throughout the entire process was greatly appreciated and meant a lot. I cannot thank my mother, Sevim GÖÇHAN, and my father, Fahri GÖÇHAN enough for their financial and moral support throughout the project. Without them, this project would not have been the subject of this thesis. Sometimes all you need is people who see your potential and believe in you. I am very happy and feel lucky that life has matched me up with these people. So, thank you life… June 2023 Dr. Deniz Göçhan x xi TABLE OF CONTENTS Page FOREWORD ............................................................................................................. ix TABLE OF CONTENTS .......................................................................................... xi ABBREVIATIONS ................................................................................................. xiii LIST OF TABLES ................................................................................................... xv LIST OF FIGURES ............................................................................................... xvii SUMMARY ............................................................................................................. xix ÖZET ........................................................................................................................ xxi 1. INTRODUCTION ................................................................................................. 1 1.1 Reflexive Prelude ............................................................................................... 1 1.2 Structure of the Thesis ........................................................................................ 4 2. LITERATURE REVIEW ..................................................................................... 9 2.1 Human-Computer Interaction ............................................................................ 9 2.1.1 Defining users ............................................................................................ 9 2.1.1.1 Users in the medical device context ................................................... 12 2.1.1.2 Redefining the user in the context of expert systems ......................... 14 2.1.2 Introduction to artificial intelligence and expert systems ........................ 17 2.1.2.1 Knowledge base ................................................................................. 19 2.1.2.2 Database or storage ............................................................................ 23 2.1.2.3 Inference engine ................................................................................. 23 2.1.2.4 Explanation facility ............................................................................ 28 2.1.2.5 User interface ..................................................................................... 28 2.1.3 Designing the interaction ........................................................................ 30 2.1.3.1 The role of semiotics in interaction .................................................... 31 2.1.3.2 Brief introduction to the theory of affordances and its effect on inter- action ................................................................................................ 35 2.1.3.3 Introduction to user modeling ............................................................ 38 2.1.3.4 User profiling ..................................................................................... 42 2.1.3.5 A model proposition for profiling the users of expert systems .......... 45 2.2 Digital Oral Health Applications ...................................................................... 49 2.2.1 Approaches in the literature ..................................................................... 50 2.2.1.1 Clinical approach in mDentistry ........................................................ 50 2.2.1.2 The categorizations of oral health apps published in reviews ............ 57 2.2.1.3 Design approach in the evaluations by health experts ....................... 59 2.2.1.4 The design methods adopted in the development processes of oral health apps ........................................................................................ 61 2.2.1.5 The issues of approaches in the mDentistry literature ....................... 64 2.2.2 Apps as medical devices ........................................................................... 67 2.2.2.1 Governing regulations for medical apps ............................................ 68 2.2.2.2 The role of designers in the medical device development processes . 76 2.2.2.3 Prospects and challenges of regulations ............................................. 78 3. RESEARCH APPROACH .................................................................................. 83 3.1 Research in Design ........................................................................................... 88 xii 3.2 My Approach .................................................................................................... 89 4. MY DESIGN PROCESS ..................................................................................... 95 4.1 Ideation Phase ................................................................................................... 95 4.1.1 The features of the app .............................................................................. 97 4.1.2 How I formulated the features ................................................................... 98 4.1.3 How I planned the TÜBİTAK project: From an idea to a prototype ...... 102 4.2 Prototyping Phase ........................................................................................... 105 4.2.1 The first work package: Developing the SAS algorithm ........................ 105 4.2.2 The second work package: Designing the user interfaces and the wireframes ............................................................................................... 118 4.2.2.1 Research 2: Testing the two symptom selection prototype ............. 124 4.2.2.2 Preparation for the coding process .................................................. 131 4.2.2.3 The coding process .......................................................................... 132 4.3 Testing Phase .................................................................................................. 132 4.3.1 The Third Work Package: Testing the Prototype .................................... 132 4.3.1.1 Research 3: Evaluation of the application with users from 3 different fields of expertise .............................................................................. 134 4.3.1.1.1 The results of research 3 from the RfD perspective ................ 139 4.3.1.1.2 The results of research 3 from the RiD perspective ................ 161 5. DISCUSSION ..................................................................................................... 165 6. CONCLUSIVE REMARKS .............................................................................. 175 REFERENCES ....................................................................................................... 181 APPENDICES ........................................................................................................ 213 CURRICULUM VITAE ........................................................................................ 225 xiii ABBREVIATIONS AAPD : American Academy of Pediatric Dentistry AI : Artificial Intelligence AIC : Acıbadem Incubation Center AIMDD : Active Implantable Medical Device Directive CE : Conformité Européenne DRS : Dental Recording System EG : Emergency Guide EU : European Union FDA : United States Food and Drug Administration GDPR : General Data Protection Regulation HIPAA : Health Insurance Portability and Accountability Act of 1996 HONcode : Health on the Net Foundation Code of Conduct IOS : iPhone Operating System MARS : Mobile App Rating Scale MDD : Medical Device Directive MDR : Medical Device Regulation ML : Machine Learning ND : Nearby Dentists NHS : National Health Service of the United Kingdom NLP : Natural Language Processing OEG : Oral Examination Guide OHPP : Oral Hygiene Product Platform OHTS : Oral Hygiene Tracking System xiv RfD : Research for Design RiD : Research into Design RtD : Research through Design SaMD : Software as a Medical Device SAS : Symptom Assessment System SBOSC : Scientific Basis of Oral Self-care SSV1 : Symptom System Version 1 SSV2 : Symptom System Version 2 TMMDA : Turkish Medicines and Medical Devices Agency TÜBİTAK : Scientific and Technological Research Council of Turkiye UDI : Unique Device Identification UI : User Interface UX : User Experience WHO : World Health Organization xv LIST OF TABLES Page Table 2.1: Classification of end-users of an expert system. ...................................... 17 Table 4.1: Backgrounds of the participants. ............................................................ 108 Table 4.2: Backgrounds of the participants included in Research 3 ....................... 138 Table 4.3: The distribution of the comments .......................................................... 161 xvi xvii LIST OF FIGURES Page Figure 1.1: The writing styles of the sections. ............................................................ 7 Figure 1.2: The map of the four areas related to the thesis and the positions of the sections on this map. ................................................................................. 8 Figure 2.1: The classification of end users (Cifter, 2011: p.25) ............................... 13 Figure 2.2: The components of an expert system. .................................................... 20 Figure 2.3: The rules of the example algorithm that suggests what to wear depending on the three weather conditions. ............................................................. 21 Figure 2.4: The rules of the second algorithm that suggests what to wear depending on the weather conditions and the temperature. ..................................... 23 Figure 2.5: The user flow of a system using the forward chaining inference engine. ................................................................................................................ 25 Figure 2.6: The user flow of a system using the backward chaining inference engine. ................................................................................................................ 28 Figure 2.7: Three axes of determinants in defining a user. ....................................... 46 Figure 2.8: The relation between variables. .............................................................. 47 Figure 2.9: 3D model of user profiling using an expert system. ............................... 49 Figure 2.10: Purpose of mobile dental apps published in the peer review literature (Qari, 2019, p.33).................................................................................... 58 Figure 3.1: Types of design research in my process according to Frayling's classification (1994)................................................................................ 92 Figure 4.1: The ultimate goal of the project............................................................ 100 Figure 4.2: The estimated timeline of the project in the submission ...................... 104 Figure 4.3: Images of amelogenesis imperfecta (a) on the left and trauma-related enamel hypoplasia (b) on the right ....................................................... 110 Figure 4.4: An image of a case of open bite ........................................................... 111 Figure 4.5: An image of a case of deep bite............................................................ 112 Figure 4.6: An image of a case of gingival recession ............................................. 113 Figure 4.7: An image of a case of geographic tongue ............................................ 114 Figure 4.8: An image of a case of anterior crossbite .............................................. 115 Figure 4.9: An image of a case of posterior crossbite ............................................. 116 Figure 4.10: Symptom selection page ..................................................................... 122 Figure 4.11: Symptom selection interfaces of the first prototype ........................... 123 Figure 4.12: Examples of symptom selection interfaces of the second prototype .. 125 Figure 4.13: Comparison of my decision and the common point of view for the opacity of selected, unselected, and inactive buttons. ......................... 129 Figure 4.14: The changes of the icons over interviews........................................... 130 Figure 4.15: Three different poles according to the backgrounds of the participants. .............................................................................................................. 135 Figure 4.16: Ali’s health information given to the participants .............................. 136 Figure 4.17: Three different situations Ali experiences .......................................... 137 Figure 4.18: Screenshots of interfaces (Part 1/3) .................................................... 149 xviii Figure 4.19: Screenshots of interfaces (Part 2/3) .................................................... 150 Figure 4.20: Screenshots of interfaces (Part 3/3) .................................................... 151 Figure 4.21: Number of medical content and UI/UX comments made by three participant groups.. ............................................................................... 162 Figure 5.1: The actual process. ............................................................................... 169 Figure 5.2: The map of dominance in the expert system I designed....................... 173 xix DESIGNING AN EXPERT SYSTEM FOR NON-EXPERT USERS IN ORAL HEALTH: STORY OF A HYBRID DESIGN RESEARCH SUMMARY The subject of this thesis is the research, development, and design process of an expert system that I designed for non-experts in the field of pediatric oral health, which started as an enterprise project and later received the TÜBİTAK 1512 Techno- initiative Capital Support Program grant. In this thesis, you will read about the maturation of my idea that I created as a dentist, its transformation into a mobile application prototype within the scope of the TÜBİTAK project, the testing processes, the difficulties encountered, and how I tried to overcome them. In empirical research, the independence of the researcher and the notion that examines the subject from an outsider’s perspective are very important. I, on the other hand, was the leading actor in the three-year enterprise story that I addressed as a researcher in this thesis. At every stage of the product from an idea to a prototype; I spent a lot of time applying to institutions, developing the algorithm, and designing the interfaces and the system. For these reasons, I did not want to write down the research I conducted in the process as if I were an observer. Because that would be quite wrong. Thus, the complexity of the process and my inseparable relationship with the product led to the need to adopt a hybrid approach to the subject. In my approach, I combined the three forms of design research that Frayling described. The “Research through Design” approach allowed me the freedom to convey what I experienced in the process from my perspective, without skipping any steps. Since it is not possible to generalize about entrepreneurship based on my own story, there could not be a more appropriate method in which I would be involved as a subject. The research I conducted to enable non-expert users to use an expert system was exactly in line with the "Research for Design" approach. Because at the end of the day, my goal was to improve the product. The “Research into design” approach enabled me to document data that emerged independently of the product I designed and could contribute to the literature. I am aware that in this approach I have adopted, a portrayal of dissociative identity emerges. I became a “Research through Design” researcher while describing my process, a designer when researching to improve the product, and sometimes a design researcher who intends to contribute to the literature. A researcher can wear these hats in different periods. I wore all these hats during one study. Therefore, in this thesis, you will witness hybridity just like my process. This thesis touches on more than one area. Therefore, what I have told can be evaluated from many different angles. It can be considered as the process of developing an entrepreneur's product, the place of digital products in the field of oral health, a critique of the way dentists handle these products in the literature, the difficulties in developing a medical device, designing of artificial intelligence and xx human interaction, the effort to bring an expert system closer to the non-expert user, the design’s role in all these aspects or as a different method attempt at design. xxi AĞIZ SAĞLIĞI ALANINDA UZMAN OLMAYAN KULLANICILAR İÇİN UZMAN SİSTEM TASARLAMAK: HİBRİT TASARIM ARAŞTIRMASI HİKAYESİ ÖZET Bu tez, bir girişimcilik fikri olarak başlattığım ve sonrasında “Çocuk Ağız ve Diş Sağlığı Alanında Bulanık Mantık Yönteminin Kullanılmasıyla Uzman Sistemin Ön Prototipinin Geliştirilmesi ve Ticarileşme Amacıyla Mobil Uygulamaya Uyarlanması” isimiyle TÜBİTAK 1512 Tekno-girişim Sermaye Destek Programı hibesini almaya hak kazanan bir inovasyon projesini konu edinmektedir. Bu projede, uzman olmayan kullanıcılara yani ebeveynlere çocuk ağız sağlığı alanında olası tanıyı sunan bir uzman sistem prototipini geliştirmeyi amaçladım. Dolayısıyla tezde, bir diş hekimi olarak ortaya koyduğum fikrin olgunlaşmasını, TÜBİTAK projesi kapsamında bir mobil uygulama prototipine dönüşmesini, Ar-Ge sürecinde yapılan tasarım araştırmalarını, test edilme süreçlerini, tüm bu süreçlerde karşılaşılan zorlukları ve bu zorlukların nasıl üstesinden gelmeye çalıştığımı okuyacaksınız. Girişimcilik/inovasyon sürecimde yaptığım tasarım araştırmalarını tez kapsamında ele almaya karar verdiğimde çeşitli sorunlarla karşılaştım. Bu sorunlardan birincisi bir araştırmacı olarak süreçten ve üründen bağımsız olmamamdı. Ürünü geliştiren ve tasarlayan kişiydim. Araştırmaları geliştirdiğim ürünü iyileştirmek amacıyla yapmıştım. Bu araştırmaları girişimcilik sürecinden bağımsız incelemek de anlamsız olacaktı çünkü içeriği özgün olsa bile yöntemsel açıdan tasarım süreçlerinde sıklıkla yapılan araştırmalardı. Ayrıca bu sürecin tez konum olmasına, sürecin ortasında karar vermiştim. Ampirik araştırmalarda, araştırmacının bağımsızlığı, konuyu dışarıdan bir gözle irdeleyen bakış açısı oldukça önemlidir. Araştırma önden planlanır; bir hipotez ortaya atılır ya da bir soru sorulur ve hipotez test edilir ya da bir cevap aranır. İçinde bulunduğum durumun karmaşıklığı ve bir araştırmacı olarak çoklu pozisyonum bu kuralların hiçbirine uymuyordu. Ben zaten genellenebilir, yeniden üretilebilir bir bilgi üretmiyordum. Sadece girişimcilik hikayemi ve Ar-Ge sürecimi, bir araştırmacı olarak pozisyonumu ve amaçlarımı açıkça belirterek tasarım çerçevesinden anlatmak istiyordum. Dolayısıyla, sürecin karmaşıklığı ve ürün ile ayrılamaz ilişkim, tez için hibrit bir yaklaşımı benimseme ihtiyacını doğurdu. Yaklaşımımda Frayling’in tanımladığı üç tasarım araştırması biçimini birleştirdim. “Tasarım yoluyla araştırma” yaklaşımı bana herhangi bir aşamayı atlamadan kendi gözümden süreci aktarma özgürlüğünü tanıdı. Kendi hikayemden yola çıkarak girişimciliğe dair bir genelleme yapılamayacağı için özne olarak benim de yer alacağım daha uygun bir yöntem olamazdı. Bu şekilde fikrin ortaya çıkmasından, girişimin geldiği son noktaya kadar yapılan tüm işleri ve aldığım kararları ben diliyle yazabildim. Uzman bir sistemi uzman olmayan kullanıcının kullanabilmesi için yaptığım araştırmalar tam olarak “Tasarım için araştırma” yaklaşımına uyuyordu. Çünkü xxii günün sonunda süreç içindeki amacım tasarladığım ürünü iyileştirmekti. Bu amaç doğrultusunda toplam üç tasarım araştırması yaptım. Çocuklarda ağız hastalıklarının olası tanısını koyacak uzman sistemin ilk prototipini tamamladığımda önemli bir engelle karşılaştım. Uzman terimleri uzman olmayan kullanıcılar nasıl anlayacaklardı? Çünkü kullanıcının semptom arama kutusuna ne yazdığını anlayacak “Doğal Dil İşleme” kabiliyeti algoritmada mevcut değildi. Bu nedenle, onların ağız hastalıklarını ve bu hastalıklara ait belirtileri nasıl tanımladığını öğrenmeye karar verdim. Birinci araştırmada, beş uzman olmayan katılımcıya internetten bulduğum kırk ağız hastalığı fotoğrafı gösterdim ve burada gördükleri hastalıkları sanki telefonda bir diş hekimine anlatır gibi tarif etmelerini istedim. Elde edilen veriler ışığında semptom tanımlarını ve bağlantılı olarak sınıflandırmaları ve algoritmayı değiştirdim. Bütün semptomları dokuz alt başlık altında topladım. Yeni semptom sistemine uygun olarak kullanıcıların ilgili semptomları seçtikleri arayüzleri Figma’da tasarladım ve ikinci tasarım araştırması için iki farklı kullanıcı akışına göre iki prototip oluşturdum. İkinci araştırmada, birinci araştırmadaki katılımcıların bu arayüzleri kullanıp kullanamayacaklarını test ettim. Onlara çocuk ağız hastalıklarıyla ilgili toplam sekiz vaka hikayesi okudum ve ağız içi fotoğraflar gösterdim. Katılımcılardan bu fotoğraflarda gördükleri ve hikayede tespit ettikleri belirtileri göz önüne alarak iki farklı prototipi kullanmalarını istedim ve görüşmeleri kaydettim. Kullanıcıların büyük bir kısmı yeni oluşturulan semptom sistemini arayüzler üzerinde kullanabildiler. Sonuçların tatmin edici çıkması ile bir sonraki aşama olan kodlama sürecine geçebildim. Algoritma ve arayüzler, Android Studio kullanılarak yazılımcı Mehmet Demir tarafından mobil uygulama olarak kodlandı. Üçüncü araştırmada mobil uygulama formunda geliştirdiğimiz uzman sistemi üç farklı kullanıcı grubuna, ebeveynler (N:3), tasarımcılar (N:3) ve diş hekimlerine (N:3) test ettirdim. Bu araştırmada katılımcılardan, oluşturduğum bir kurgu vaka hikayesindeki verileri sanki kendi çocuklarının verileriymiş gibi kabul etmelerini, çocuk için bir profil oluşturmalarını, hikayedeki belirtileri sisteme girerek bir değerlendirme yapmalarını ve bu faaliyetler sırasında sesli düşünmelerini istedim. Araştırma esnasında hem ekran görüntüsü hem de ses kaydı aldım ve sonuçları içerik analizi yaparak değerlendirdim. Üçüncü araştırmada ürünün kendisinden bağımsız olarak başka sonuçlar da elde ettim. Genelde kullanılabilirlik araştırmaları son kullanıcının dahil edilmesiyle gerçekleştirilir. Ben ise bu araştırmada, ürünün temel aldığı iki alandaki uzmanları; tasarımcıları ve diş hekimlerini de dahil ettim. Ancak bu farklı katılımcıların değerlendirme esnasında hangi noktaların üzerinde daha çok durduklarını ve bu yeni yöntemin verileri hangi açılardan zenginleştirdiklerini sunmak “Tasarım için araştırma” yaklaşımına uygun değildi. Araştırmada elde edilen aynı sonuçları farklı bir bağlamda değerlendirmek, farklı bir yaklaşım gerektiriyordu. Bu nedenle, bu değerlendirmeyi Frayling’in son sınıfı olan “Tasarım adına araştırma” yaklaşımını benimseyerek yaptım. Benimsediğim bu yaklaşımda ortaya bir kişilik bölünmesi tablosunun çıktığının farkındayım. Sürecimi anlatırken bir eylem araştırmacısı, ürünü iyileştirmek için araştırırken bir tasarımcı ve bazen de literatüre katkı yapmak isteyen bir tasarım araştırmacısı oldum. Bir araştırmacı farklı zaman dilimlerinde bu şapkaları takabilir. Ben ise bu şapkaların hepsini bir çalışma başlığı altında taktım. Çünkü mevcut yöntemlerin hiçbiri bu kompleks süreci anlatmak için yeterli değildi. Sürecin ve bir araştırmacı olarak konumumun melezliği benimsediğim yaklaşıma sirayet etti. Bu xxiii nedenle Frayling’in sınıflandırmasındaki bütün tasarım araştırması türlerini bir araya getirerek sürecimi anlattım. İnovasyon projemdeki tasarım sürecini merkeze koyduğum bu tezde, ürünün temas ettiği dört alanda (İnsan- Bilgisayar Etkileşimi, Uzman Sistemler, Ağız Sağlığı/Mobil Uygulamalar ve Tıbbi Cihazlar) kapsamlı bir literatür incelemesi de gerçekleştirdim. Literatür incelemesini iki çatı altında topladım: İnsan-bilgisayar etkileşimi ve ağız sağlığı mobil uygulamaları/medikal cihazlar. Birinci kısımın ilk bölümünde insanı yani kullanıcıyı mevcut literatürün nasıl sınıflandırdığını anlattım ve bu sınıflandırmaların eksikliklerini ortaya koyarak uzman sistemler ya da bir uzmanlık içeren sistemler bağlamında yeni bir kullanıcı sınıflandırması önerisinde bulundum. Bu sınıflandırmada uzman sistemin içerdiği uzmanlığı referans noktası alarak kullanıcıları uzman ve uzman olmayan kullanıcılar olarak ikiye ayırdım. Bu iki grubu bilgi, deneyim ve kabiliyet olmak üzere üç düzlemde alt gruplara ayırdım İkinci bölümde ise uzman sistemlerin ne olduğunu, hangi komponentlerden oluştuğunu örnekler vererek açıkladım. Üçüncü bölümde etkileşimi etkileyen parametreleri, insan-bilgisayar etkileşimi ve tasarım literatüründe hangi açılardan incelediklerini değerlendirdim. Bu değerlendirme sonucunda, kullanıcı profillemesinde yardımcı olabilecek üç boyutlu bir model önerisinde bulundum. Bu modeli kullanıcı sınıflandırmasıyla ilişkilendirerek açıkladım. İkinci kısımda, ağız sağlığı uygulamalarını klinik ve tasarım açılarından değerlendiren çalışmaları ve geliştirilme aşamalarını anlatan çalışmalarda kullanılan tasarım yöntemlerini ele aldım. Literatürdeki çalışmalarda var olan yöntemsel eksiklere değinerek eleştiri ve önerilerde bulundum. Ağız sağlığı uygulamalarını tıbbi cihaz kapsamında değerlendiren yönetmelikleri, uyum ve sertifika süreçlerini inceledim. Bir tıbbi cihaz üreticisi olarak yönetmeliklerin sektöre getireceği fayda ve riskler üzerine yorumlarda bulundum. Bu tezde, ağız sağlığı alanında uzman olmayan kullanıcılar için bir uzman sistem geliştirdiğim proje sürecini hibrit bir araştırma yöntemi benimseyerek anlattım. Buna ek olarak, ürünün temas ettiği alanlarda kapsamlı bir literatür incelemesi yaptım ve bu alanlara katkı sağlamak amacıyla eleştirilerde ve önerilerde bulundum. Öneri, eleştiri ve yorumlarımın yer aldığı bölümleri, literatür incelemesi ve süreçte elde ettiğim deneyimlerin bir çıktısı ya da bir girişimci, diş hekimi, tıbbi mühendis, tasarımcı ve araştırmacının öz-yansıtması olarak kabul edebilirsiniz. Bu çıktılar çok farklı açılardan, sağlık alanında bir girişimcilik hikayesi, yapay zekâ ve insan etkileşiminin tasarlanması, uzman bir sistemi uzman olmayan kullanıcıya yaklaştırma çabası, tasarımın tüm bu süreçlerdeki rolü ya da tasarımda farklı bir yöntem denemesi olarak ele alınabilir. Bu nedenle bu tezin, mobil uygulamalar, uzman sistemler ya da bir uzmanlık içeren sistemler üzerine çalışan girişimciler, sağlık uzmanları, kural koyucular, tıbbi cihaz üreticileri, mühendisler, tasarımcılar, akademisyenler ve araştırmacılara faydalı olabileceğine inanıyorum. xxiv 1 1. INTRODUCTION 1.1 Reflexive Prelude Since my approach in this thesis is strongly influenced by the principles of action research and “Research through Design” is a part of my hybrid method, I need to briefly introduce myself, my position, and my perspective on life. Thus, you can more easily understand my place and role in this study, the aspects of my criticisms, and comments, and especially how I handle the design process part in which I speak in the first person. As a dentist, it would be right to start by answering the question of how my path crossed with industrial design. A very simple answer that you would often encounter in Turkey: The major I always wanted to study as a kid was industrial design. Since my score in those years was not enough for the industrial design major at Istanbul Technical University (ITU) and Middle East Technical University, and since I was raised in a middle-class family by teacher-parents, I chose dentistry as a profession where I would not have "future anxiety". I graduated from Ege University Faculty of Dentistry in 2008. Although my bond with the profession has been shaken at times, I can say that I mostly enjoyed being a dentist or I never hated it. In 2009, I started my doctorate in the Department of Periodontology at Başkent University and studied it with pleasure as well. My area of interest during my Ph.D. was oral medicine and I made publications in which I generally reported the findings of systemic or genetic diseases in the mouth. Diagnosing oral pathologies was my favorite thing to do because it always felt like solving a puzzle. After completing my doctorate in 2013, I moved to Istanbul and started to work as an oral surgeon at Istanbul Aydın University Dental Hospital. I worked there full-time for about three years. Although my income was good, my job started to not satisfy me after a while. Something was missing in what I was doing for a living and I needed to move to a field where I could use my creativity. 2 While trying to figure out what to do in this quest, I started taking drawing lessons in 2015. In the first lesson, my art teacher, Fatma Demirtaş, asked me if I ever wanted to be an industrial designer because she thought my drawings were very neat. This was quite shocking and hit the bull’s eyes. It reminded me of the fact that I had a childhood dream. During our sessions, she motivated me so hard to take the aptitude test, so I suddenly found myself preparing for the university exam again. However, I knew in my heart that I was not ready to study undergraduate again, and I was also aware that it was not enough to take a drawing course for 6 months to get accepted. However, I took the aptitude test anyway and of course, I did not succeed. Then I decided that doing a master's in design was a more realistic plan. I sent an e-mail to Prof. Şebnem Timur at ITU and asked for an appointment. In our interview, she told me that dentistry should have been on the list of eligible degrees so that I could apply for the master’s. Thanks to her, I wrote a petition to the board of directors for the addition of dentistry to the list and my request was accepted. In the fall of 2016, I took the Design Thinking course given by Prof. Hümanur Bağlı as a guest student to get to know the field and see what I could do. Students who took this master's course were expected to improve the projects of the startups they were matched with by using design methods and to bring a designer perspective to the entrepreneurs. Luckily, I was matched with Erhan Ermek. Erhan was developing a POCT device that measures clotting time. For his project, I did interviews with different healthcare professionals and learned what they cared about in a POCT device. In addition, I evaluated the perceptions of designers and 60 diabetic patients by showing them ten different blood glucose meter POCT device designs. I used several semantic differential scales such as trust, seriousness, professionalism, quality, etc. to understand which design elements came into effect in such parameters. I likened Design Thinking to medicine in terms of diagnosing problems in projects using design methods and making an intervention based on the results obtained. Thanks to this course, it was easy for me to enter the world of design. In the spring term of 2017, I applied for the master's program at ITU but was not accepted. Encouraged by my success in the previous course, I took three more courses as a guest student: Design Semantics, Paradigms of Design, and Theoretical Studies in Design. In the Design Semantics course, I conducted user research that evaluated toothbrush designs semantically. In May, I presented my final paper for the 3 Design Thinking course at the 4T Design and Design History Society's Cross- Abilities Symposium. Meanwhile, Erhan offered to work together because he liked what I had done for his project. When his venture was accepted by Bayer's Grants4Apps program, we went to Berlin in June for investor meetings. It was an incredible experience for me. Over the course of a two-year collaboration with Erhan, I had the opportunity to take responsibility for the application process for numerous incubation centers and pitch events. Within this role, I was tasked with managing both the medical background and design aspects of the project. Through this experience, I gained a wealth of knowledge surrounding the intricacies of entrepreneurship and the many challenges that come along with it. In the following year, I applied again for the master's program and became the first dentist admitted to the department. After a one-term scientific preparation process, I took other post-graduate courses. Particularly, the Ethnographic Methods in User- Centered Design and Cultural Approaches to Design courses made me fall in love with social and cultural studies. For the Cultural Approaches to Design course, I wrote a paper exploring the experience of users from different cultures with different types of bidets. After years of positivist education, design education taught me to look at research forms, methods of acquiring knowledge, and even medicine from a different perspective. For this reason, besides the happiness of fulfilling my childhood dream, I see this process as one of the most significant turning points of my life. My ability to develop artificial intelligence for parents in the field of pediatric oral health has been realized thanks to the comprehensive knowledge about the field and most importantly the scientific notion I gained through dentistry education, as well as the designer perspective, qualitative research methods, production processes, user research I learned through design education. This innovation is the fruit of my 19 years of university education, and I hope it will be of some use to humanity. You can find some of my thoughts related to the entrepreneurial parts of my story in Chapter 3, where I describe my design process. In this thesis, you will encounter many of my identities: a dentist, an inventor, a medical device manufacturer, a medical engineer, a design student, an entrepreneur, and above all, a researcher. 4 1.2 Structure of the Thesis This thesis is the outcome of my entrepreneurship project where I designed an expert system in the field of pediatric dentistry for non-expert users. It was quite challenging to deal with such a multi-layered and multi-disciplinary project within the scope of a master's thesis in design. Because this three-year project is located right at the intersection of four different areas: Oral health & mobile apps -if considered one-, medical devices, expert systems, and human-computer interaction (HCI). Although I tried to review relevant literature on these four fields from a design perspective as much as possible, some parts that were out of the scope of the design remained in the thesis for a better understanding of the technology I have developed. For this reason, sections such as the clinical approach to oral health mobile applications, expert systems, or regulations governing medical devices can be considered subsections that support the main framework. The thesis is divided into 5 chapters: Chapter 1 is an introductory part where I review the literature on four fields in which the product intersects. It consists of two main sub-sections: (1) HCI in expert systems and (2) oral health apps. Under HCI, there are three headings: “Defining users”, “Introduction to AI, and expert systems” and “Designing interaction”. In the “Defining users” section, after reviewing the current definitions in the literature, I define and categorize possible end users of an expert system and their characteristics. The “Introduction to AI and expert systems” briefly introduces the technical aspects of expert systems and their components. “Designing interaction” focuses on the factors that affect the interaction between expert systems and end users. It is a section where I mention semiotics, affordances, user models, and profiling, and I propose a new model for user profiling. The literature in the oral health apps section is divided into two. In the first part, how oral health mobile apps are evaluated in the mDentistry literature is discussed. In the “Clinical approach in mDentistry” section, after a brief introduction to both the health and socio-economic consequences of oral diseases, it is focused on what intervention purposes oral health apps are used for and how their performance is 5 evaluated in the mDentistry literature. “The categorizations of oral health apps in the literature” is a section that focuses on how existing oral health apps on the market are classified in the literature. “The design approach in the evaluations by health experts” reviews how the design and usability of oral health apps are evaluated by health experts. “The design methods adopted in the development processes of oral health apps” emphasizes the design methods adopted by the developers in the development processes of oral health applications. “The issues of approaches in the mDentistry literature” discusses the methodological problems and drawbacks of the studies in the mDentistry literature. In the second part, the unique dynamics of medical devices are addressed such as governing regulations for medical apps, the role of designers in the field, and the prospects and challenges. The “Governing regulations for medical apps” section focuses on the governing rules in FDA and MDR regulations and compliance processes regarding health apps. “The role of designers in the medical device development processes” emphasizes the challenges of medical device development processes, the role of designers, and how the design service is perceived by the manufacturers. The “Prospects and challenges” section discusses the potential benefits and challenges that the newly enacted regulations to which health apps are subject will bring to the field. Chapter 2 is the part where I explain how I approach my design process within the research context. In the beginning, I introduce action research and its principles, then Frayling’s (1994) classification of design research, and finally I elucidate the hybrid research approach I created by combining Frayling's classification. Chapter 3 is devoted to a meta-reflection of the design process of the expert system I developed for pediatric oral health. In this part, the development process is elaborated considering its stages. Research and development activities, especially design research carried out from the idea to the test stage, are explained with an active voice in this section. Chapter 4 is the part where I discuss the research approach that I have adopted in this thesis and my design process. Chapter 5 is the section where I summarize my findings, critiques, proposals, and comments as conclusive remarks. 6 This thesis has some structural differences from the theses written at Istanbul Technical University and even those we encounter frequently in academia. In the four subsections of the literature review (“Redefining the user in the context of expert systems”, “A model proposition for profiling the users of expert systems”, “The issues of approaches in the mDentistry literature” and “Prospects and challenges of regulations”), you will read my suggestions, criticisms, and comments on the relevant subjects. According to the school’s thesis writing style, these should have been included in the discussion. However, it would not be right to place these subsections there where my design process was mostly discussed. Because they were closely related to the sections in the literature review. Placing them in the discussion would cause a break in the flow. This unusual structure carries some risks of misunderstanding. Since the reader will read these sections first, they can naturally expect to see a reflection of my suggestions and criticisms in my design process whereas they have emerged as a result of my experiences in the process and extensive literature review. If I had been able to write the thesis in chronological order, you would have read my design process first and then the literature review and hence my suggestions and criticisms. Unfortunately, the current thesis template of the school did not allow for the eclectic nature of the subject that this thesis was dealing with. Therefore, I would like you to accept the first part, the literature review, both as an introduction to the areas in which the project intersects and as an output that has been put forward based on the experience and data I have obtained at the end of the process. While active voice is used in the sections where the product’s design process is explained, passive voice is preferred in the sections where approaches or relevant studies in the literature are discussed. Both writing styles are used in the sections such as the discussion or research approach where the process and the literature are discussed together. Figure 1.1 shows the writing styles generally adopted in the chapters. 7 Figure 1.1: The writing styles of the sections. Figure 1.2 illustrates the map of the four areas related to the thesis and the positions of some sections in the first chapter on this map. 1.2.1 Research Questions The thesis aims to document the design and development process of an AI-powered diagnostic mobile app for oral health as transparent as possible by looking at it from a design perspective, without losing touch with where I stand as a researcher. I was not an observer, on the contrary, I was an actor, a participant. I was not only a design researcher but also a health expert and an entrepreneur. Therefore, it adopts a hybrid research approach to overcome these conflicts triggered by the complex nature of the project and my multiple identities, just to tell it as it is without pretending. Since this study was not intended to generate scientific knowledge and did not start with a hypothesis, there is no point in asking questions. Instead, it would be more appropriate to state what it deals with. It deals with an innovation project where social studies and natural sciences, design and dentistry, qualitative methods and engineering, and design research and mDentistry meet. It deals with what I learned from the experience and the literature review and what I reflected. 8 Figure 1.2: The map of the four areas related to the thesis and the positions of the sections on this map. 9 2. LITERATURE REVIEW 2.1 Human-Computer Interaction This section consists of three subsections. In the first part, the human being, that is, the user, will be defined, in the second part, a brief introduction to artificial intelligence and its branch, expert systems, will be made. In the third part, the factors affecting the interaction between humans and computers will be discussed and a new model for user profile will be proposed. 2.1.1 Defining users It would be appropriate to start by defining the human, that is, the user, in accordance with the name of the human-computer interaction section. In this section, I will concentrate on the current user definitions in the HCI literature and redefine the user according to various characteristics in the context of expert systems. In the early days of computer science research, the focus was more on machine- related errors, while human-factor-related problems were only superficially addressed (Schneiderman, 1976). The need for a separation between these two issues was understood and some researchers focused more on human users to define and elaborate their characteristics. Miller (1974) indicated the influence of prior experience with procedures on the production of program errors. Similarly, Schneiderman (1976) also addressed the crucial role of users’ previous experiences in performance and categorized users into four divisions according to the number of programming courses they had taken: “naïves” are those who had not attended any programming courses; “novices” are those who were currently enrolled in and completed their first programming course; “intermediates” are those who were currently taking or had just completed their second or third course; and “advanced users” are those who were graduate students and faculty members of a computer science department. 10 After computers began to be widely used, users who do not have expertise in computer science but who nevertheless can benefit from the use of computers in their work or daily lives became increasingly heterogeneous. At this point, it was needed to distinguish between a user who has just started taking computer courses at university and a user who encounters it at the bank, at work, at school, or home but has no experience. Eason (1976), in his study on managers' interaction with computers, defined "naive users" as those who are not computer experts but make direct use of computer systems in the everyday performance of their duties. Coombs and Alty (1980) investigated independent variables affecting the course of interaction in advisory services held in computer centers and found that beginners whom they defined as “inexpert users” have greater difficulty in benefiting from the help given by advisors. Scapin (1981) compared two groups of subjects having different levels of experience with computers in terms of learning and recall of computer commands and found that the “experienced users” were able to recognize commands, but the “naïve/inexperienced users” were not. On the other hand, in their review, Paxton and Turner (1984) made no distinction between the two terms, “novice” and “naïve” and used them interchangeably. Like Scapin (1981), Burgess and Swigger (1986) have also defined a naïve user as someone who has little or no expertise in operating computers. Allwood (1986) emphasized that novice users were not uniformly used in the literature; and noted that such users were not the least experienced users in a research group, instead, it would be more accurate to use the term to represent users with little or no experience in computer tasks. He also preferred to call users with experience “experts” instead of Schneiderman’s term, “advanced users”. He attributed the reason for this change to the terminology used in studies conducted in other fields (Larkin, 1981, 1983; Simon & Simon, 1978) that showed experts’ use of knowledge and how it played an important role in enabling them quickly to tackle the problem they encountered. Later, Fisher (1991) wanted to make a distinction between the novice and naïve and between experienced and expert terms to put an end to this confusion in the literature. He brought a different perspective on this issue by stating “Thus while the terms, novice and naïve may coincide, they carry separable meanings…” (Fisher, 1991, p. 439) and defined “naive users” as those who lack the ability or capacity to 11 analyze and reason in a given situation and who need to make meaningful use of computers for certain tasks but are not interested in other functions; and “novice users” as those who are new or inexperienced in a certain task or situation, in other words, “beginners” or “new users.” By underlining that being experienced does not say anything about the quality of the experience gained, he defined the “experienced user” as someone who has developed skills or knowledge due to participation or exposure. According to Fisher (1991), an experienced user is the opposite of a novice user. Lastly, he defined “expert users” as those who make an effort to gain skills and knowledge and benefit from them to use the system extensively. In Fisher’s terms, being an expert stands for the opposite of being naïve. He stated that even a new user who intends to become an expert would begin as an expert novice1. The efforts in the literature to define the user, sometimes using the same and sometimes different terms, have progressed in parallel with the studies revealing the factors affecting the interaction of the user with the computer, and with the fact that computer use has become popular and consequently, the diversity of the user profile increased. Since the first studies were carried out with computer science students, the definitions were determined based on the experience and the number of courses taken by the students. After a while, as computers started to be used by users who did not have the expertise, confusion started in the definitions of the terms. Of course, the necessity of distinguishing between two novice users emerged when one of them was a computer science student, and the other was not. However, the literature could not reach a consensus on this issue either. As studies have been carried out, it has been seen that making definitions is not that simple. Another issue is that a person's experience with computer systems also depends on that person's access to that technology. Therefore, not all potential users can benefit from these technologies equally. Inequality exists not only between developed and developing countries but also between those who have easy access to information and those who do not know how and where to find it (Goulding, 2001). In this regard, Van Deursen and Van Dijk (2009) underscored that users having access to or owning a computer does not necessarily mean that they have the capacity to operate and use it and they define this 1 Fisher wrote the term in his original paper as "experienced novice". Throughout his article, he distinguished between novice and naive, depending on how prone the user is to develop new skills. So, assuming it's a typo, I change it to expert-novice. 12 divide as “digital skills”. In fact, this is the subject that Fisher (1991) wanted to emphasize in his work by making the distinction between novice vs. naïve and experienced vs. expert. Users may be inexperienced; their experience may increase with computer exposure, but it is not the only parameter. Users’ willingness to use the computer extensively also became prominent. 2.1.1.1 Users in the medical device context Like computers, as medical devices for the use of patients and their relatives at home became widespread, similar discussions have emerged about defining users. While some terms from the HCI were transferred to the field, new terms were also coined. Lay-user is frequently used in the medical device literature. Hogg and Williamson (2001) argued that its description is not clear although it is largely used in the health field. In dictionaries, a layperson is defined as “a person who is not trained, qualified, or experienced in a particular subject or activity” (Collin’s dictionary2) and as “a person without professional or specialized knowledge in a particular subject” (Oxford Dictionaries Online3 as cited in Cifter, 2011). In this sense, the definition is made not according to the positive qualities that these people may have, but according to who they are not and what they do not have (Hogg & Williamson, 2001). In his thesis, Cifter (2011) criticized the implication of specialized knowledge in Oxford’s definition by stating that “…in the medical field, patients can develop their own specialized knowledge of their specific condition or their illness through personal experience, or via other information sources such as the Internet, magazines or books” (p. 22). Similarly, Entwistle et al. (1998) implied the same issue and stated “…We use the term “lay” to mean people who are neither health care professionals nor health services researchers, but who may have specialized knowledge related to health. This includes patients, the general public, and consumer advocates” (p. 463). Cifter (2011) supported Entwistle’s argument and noted that although his definition emphasized the knowledge lay user might also have, it was not clear what it meant 2 https://www.collinsdictionary.com/dictionary/english/lay-person (Last accessed: 01/04/2023) 3 http://oxforddictionaries.com/view/entry/m_en_us1262529?rskey=yqz7F0&result=2#m_en_us 1262529 (last accessed: 18/06/10) 13 by “professional”. According to Hogg and Williamson (2001), the term professional covers a wide range of attributes and the term lay covers even wider. In Cifter’s terms (2011), a lay user is “A user of a product or a system who has not undergone extensive training in the subject field (which enables him/her to be eligible to act as a member of a profession), but uses the system or the product due to his/her special interest or needs” (p. 24). A professional user was also defined by highlighting the knowledge and existence of training (Cifter & Dong, 2009). According to them, professional users are “the users who have gone through extensive training to achieve particular knowledge which is valuable in a social or economical context” (p. 4). Cifter, (2011) classified users according to the definitions above and according to three determinants: Training, knowledge, and experience (Figure 2.1). In this classification, the end-users of a medical device were first divided into professional and lay. It was noted that if the lay user is experienced in using the medical device, they may have limited knowledge compared to professional users. Hogg and Williamson (2001) stated that as laypeople become more knowledgeable about the profession and develop a greater understanding, they lose their amateur status, resulting in the loss of representation of other laypersons who do not have the same knowledge. Figure 2.1: The classification of end users (Cifter, 2011, p. 25). 14 2.1.1.2 Redefining the user in the context of expert systems Since I thought that the definitions made in both HCI and medical device literature were incompatible with my case (the expert system in pediatric oral health that I developed in the project), I decided to redefine the users in the context of expert systems. All definitions of the user that you will encounter in the following sections of the thesis are according to the definitions I made in this section. Before defining the users, it would be more appropriate to define the expert system. They can briefly be defined as algorithms that mimic the decision-making process or thinking of an expert from a specific field. Some expert systems are based on scientific literature, while others also draw on the expert's professional experience or both (Gupta & Nagpal, 2020). Since it is software mimicking human behavior, it is accepted as a branch of artificial intelligence in computer science (Abu-Nasser, 2017). Expert systems, as the name implies, are usually designed to be used by experts. In medicine, they are called “clinical decision support systems” and their use purpose is to help or assist practitioners in decision making such as diagnosing, individualized dosing, evaluating laboratory test results, etc. (Wasylewicz & Scheepers-Hoeks, 2019). In this context, we can simply define an expert user, utilizing Cifter's definition of the lay user (2011, p.24), as a person who has the knowledge or expertise from which the system is derived and has undergone training in the relevant field which enables him/her to act as a member of the profession. Owing to that, being an expert user is highly dependent on the area of the system’s expertise. Like expert users, non-expert users have the same dependency. According to the Cambridge Dictionary4, a non-expert is “a person who does not have a high level of knowledge or skill relating to a particular subject or activity.” As Entwistle et al. (1998, p. 463) and Cifter (2011, p. 22) implied above, even if non- experts are not trained in the relevant field, this does not mean that they do not have the knowledge. As in Cambridge's definition, their level of knowledge is just not as high as experts who have been educated in the relevant field. Therefore, we can describe them as users who have not undergone any training and professional 4 https://dictionary.cambridge.org/dictionary/english/non-expert (Last accessed: 09/04/2023) 15 experience in the field from which the expert system they use is derived. Although the lay user, in Cifter’s terms (2011, p. 24), has a similar meaning to the non-expert user, in the context of expert systems, I believe that defining users through this contrast or antilogy emphasizes the differentiating factor, which is the expertise in the relevant field. For this reason, I prefer to use the non-expert user term instead of lay user on purpose. As mentioned earlier, novice user is a widely used term in HCI to indicate users with no previous experience and knowledge. However, I believe that this label does not fully represent the non-expert users using expert systems. Because being a novice is solely dependent on experience in using the product regardless of the expertise and tends to change over a period of time as the user continues to use the product. Expert users have training in the relevant field, from which the expert system’s knowledge base is derived, so they are familiar with the content. However, being an expert is not always equal to being advanced in using an expert system. Therefore, expert users can also be novices as well as non-experts. It is worthy of note that they are two separate labels that identify two different attributes of the user, and they are not mutually exclusive. For example, being a dentist does not necessarily mean being an advanced user of an expert system in dentistry. They may become advanced faster than non-expert users, but they still start using it as a novice if they have no previous experience with such systems. The same applies to a user who studied computer science. Although she/he may be an expert in computers and software, she/he still uses a medical expert system as a non-expert user. At this point, my classification differs from Cifter's (2011). As Fisher (1991) underscored, some users’ understanding, analyzing, and problem- solving skills can be better than others. Moreover, any impairment in their visual perception, memory, information processing skills (Kaye & Crowley, 2000), or even personality traits such as attention deficit hyperactivity disorder (McKnight, 2010) can negatively affect their ability to use a system. I do not get into the details because the cognitive aspects of skills fall within the scope of psychology and are beyond the scope of this thesis. All I want to emphasize here is the “internal factors” (Osvalder & Ulfvengren, 2009) that users inherently possess and somehow affect the interaction. That is what I call the user’s competence and I define being “naïve” or 16 “adept” through this axis. According to this classification, both expert and non-expert users can be naïve or adept at using expert systems. In my classification, I define the user’s competence without including the user’s willingness or openness for extensive use unlike Fisher (1991). Because I believe that they are two separate attributes but not completely unrelated. While one of them has to do with the physical and psychological state of the user, the other is related to the state of being interested in the subject. Just because someone can do something does not mean they want to do it or the other way around. However, some skills can also be improved, but they cannot exceed the individual's maximum capacity. In summary, I classify end users of an expert system based on three main attributes: expertise, experience, and competence. According to these classes, a user can be either an expert or non-expert depending on the training in the relevant field; a novice or advanced depending on the experience; a naïve or adept depending on the skills (or other internal factors). Table 2.1 illustrates the classification. The types of users can briefly be described as follows: Expert Users: They are people who have the knowledge or expertise from which the system is derived and have received training in the relevant field. Non-expert Users: They are people who do not have any training and professional experience in the field from which the expert system they use is derived. Novice Users: They are the users who have never used the expert system before and have no experience. Advanced Users: They are the users who are experienced in using an expert system or accustomed to similar software. Naïve Users: They are the users who have limited abilities such as reasoning, problem-solving, seeing, reading, etc., that enable users to interact with a product. Adept Users: They are the users who do not experience any difficulties in learning how to use a new product and who have the required competence to make extensive use of the product's features. 17 Table 2.1: Classification of end-users of an expert system. End-users of an Expert System Expert User Non-expert User Training in the relevant field Yes No Knowledge Has expertise in the field None or to a certain extent Experience Novice Advanced Novice Advanced Competence Naive Adept Naive Adept In the interaction section, I will elaborate on these three attributes with a three- dimensional model in the context of an expert system. 2.1.2 Introduction to artificial intelligence and expert systems Intelligence is the ability to acquire knowledge with experience and skills to achieve certain goals. The level of intelligence differs in different species such as humans and animals. From an evolutionary point of view, the ultimate goal of living beings is to survive in an environment and adapt their behaviors to ever-changing conditions, so we observe our surroundings and kins to learn and predict the possible consequences of certain actions. According to Sternberg (1998, 2008), intelligence performs best, when it is “augmented” with wisdom. Therefore, intelligent beings must draw inferences from their own or others' experiences and keep them in their memory consistently to use them later to avoid negative consequences in the future or to achieve certain goals. A simple computer lacks reasoning and common sense because it does not have consciousness and it is not possible for a computer to understand and adapt to new situations unless it is programmed to do so. They are machines that cannot go beyond the functions and physical characteristics determined by the engineers and designers who created them. Humans are conscious beings. They see, hear, smell, taste, and touch. Thanks to these senses, they recognize the objects and living things in the environment and can interact with others. They gain experience and knowledge through their observations and actions, and they store the information they learn. At this point, artificial intelligence (AI) aims to bring all these human features to computers to perform more complex tasks. 18 The term, artificial intelligence, was first coined by McCarthy (2006), a computer and cognitive scientist, at the Dartmouth Conference in 1955. He and his colleagues suggested that a machine could mimic the ability to learn and any other aspects of intelligence that were then reserved only for humans. Later, other definitions were proposed such as “The art of creating machines that perform functions that require intelligence when performed by people” by Kurzweil (1992, as cited in Gupta & Nagpal, 2020, p. 10) and “The branch of computer science that is concerned with the automation of intelligent behavior” by Luger and Stubblefield (1997, as cited in Gupta & Nagpal, 2020, p. 10). It is simply a man-made system that is expected to behave like humans, capable of perceiving and recognizing (speech and image recognition), processing, acting, and reacting. Some intelligent systems are designed for problem-solving, planning, and decision-making, and some are capable of self- learning and -updating, which is called machine learning. The test to evaluate how successfully artificial intelligence can mimic a human was first devised by Alan Turing (1950). The Turing Test named after him, aims to determine whether an intelligent machine can achieve human-level performance in given tasks to sufficiently fool the interrogator. In the Turing Test, there are three terminals: two humans and a machine in a space where they are separated from each other. One of the human terminals, the interrogator, is designated to ask a series of questions and the other two terminals, the human, and the machine respond. After the interrogation, if the interrogator cannot distinguish which answers are given by the machine, the intelligent system passes the test. However, conducting such a test is not as easy as it seems. First, the results largely depend on the type of message. The more diverse messages the interrogator receives such as the audio or visual, the harder it becomes to pass the test. Second, the type of questions is another parameter that affects the results. It would be difficult for the interrogator to identify the machine from the responses to yes/no questions. Another issue to consider is how the questions are formulated because humans are much better than machines when it comes to conversations, on the contrary, machines can easily beat humans in mathematical calculations. According to Gupta and Nagpal (2020, p. 7), there are four capabilities that AI must possess to pass the Turing Test: 1) Natural Language Processing to understand what the interrogator says. 19 2) Storing the information provided before and during the interrogation. 3) Reasoning to answer questions or draw new conclusions leveraging the stored information. 4) Learning to adapt to new circumstances by detecting and extrapolating patterns. Expert systems are programs that are studied in computer science under the roof of the research branch of AI. An expert system is simply an algorithm that mimics an expert’s decision-making ability in a particular field, in other words, it is “an application of AI that embodies human expertise” (Gupta and Nagpal, 2020, p. 71). As Sternberg (1998, 2008) earlier pointed out, intelligence shows high performance with knowledge. Expert systems are also known as knowledge-based programs. In the design process, a knowledge engineer who studies how experts make decisions translates what is known in their expertise into a rule-based system, so that a computer can simulate their decision-making behavior. An expert system contains five main components (Figure 2.2): 1- Knowledge base 2- Database or storage 3- Inference engine 4- Explanation facility 5- User interface 2.1.2.1 Knowledge base The knowledge base is the representation of the knowledge in a particular domain in the system, which contains information about relevant parameters, the rules that describe the relationship between these parameters, and the outcomes. It plays a key role in determining which outputs will be shown under which conditions or after which inputs, the user enters into the system. 20 According to Gupta and Nagpal (2020), three types of knowledge can be applied in an expert system: Procedural, factual, and heuristic. Procedural knowledge-based systems perform all kinds of tasks step by step related to the main task that is aimed to be completed at the end of the process. For example, a robot designed to open a locked door must first pick the right key, insert it into the lock, and turn it. If everything goes well with the sub-tasks, the system manages to complete the main task, which is to open a locked door. The factual knowledge-based systems draw upon the facts of a particular domain that can be found in science textbooks or journals. Expert systems in medicine often use factual knowledge. For this reason, decisions that are critical to human life cannot be left to systems developed with subjective rules, because their harmful consequences can be enormous. The heuristic systems on the other hand are experimental and subjective. The rules in the knowledge base consist of two major parts: conditions (antecedents) and conclusions (consequents) in the form of “IF” and “THEN”. The conditions and conclusions respectively come after “IF” and “THEN”. To avoid conflicting outputs, the knowledge engineer should first understand the expert’s way of thinking, what conditions are necessary to draw a particular conclusion, and especially the distinctive features of each possible outcome. Figure 2.2: The components of an expert system. For example: Let's imagine a simple algorithm that suggests what to wear before going out, depending on the weather. The conditions are sunny, cloudy, and rainy, and the 21 consequents are a t-shirt, a jacket, and a raincoat as shown in Figure 2.3. The rules can be as follows: - IF it is sunny, THEN he/she should wear a T-shirt. - IF it is cloudy, THEN he/she should wear a jacket. - IF it is rainy, THEN he/she should wear a raincoat. The inference above is simple, and the distinctive features of each parameter are very clear. There are three mutually exclusive premises in the algorithm: whether it's sunny, cloudy, or rainy, and depending on each circumstance, there is a related consequence. Figure 2.3: The rules of the example algorithm that suggests what to wear depending on the three weather conditions. Rules can contain multiple antecedents that can be joined with logical connectives such as "AND" (the conjunction connective) or "OR" (the disjunction connective) and become compound statements. For example, when the two antecedents are conjoined with “AND”, the consequent part depends on these two being true at the same time. On the other hand, the consequent in an “IF…THEN” statement depends on either one of the antecedents being true, that is conjoined with “OR”. The 22 relationship between weather conditions in Figure 2.3 is disjunctive, so it is either sunny, cloudy, or rainy. If we add a second antecedent such as temperature to the rules using the conjunction connective “AND”, and diversify the consequents (Figure 2.4), the rules become as follows: - IF it is sunny AND hotter than 22⁰C, THEN he/she should wear a t-shirt. - IF it is cloudy AND hotter than 22⁰C, THEN he/she should wear a t-shirt. - IF it is rainy AND hotter than 22⁰C, THEN he/she should wear a T-shirt and a raincoat. - IF it is sunny AND colder than 22⁰C, THEN he/she should wear a T-shirt and a jacket. - IF it is cloudy AND colder than 22⁰C, THEN he/she should wear a jacket. - IF it is rainy AND colder than 22⁰C, THEN he/she should wear a raincoat. The first two rules can also be stated as follows: - IF it is sunny OR cloudy, AND hotter than 22⁰C, THEN he/she should wear a t-shirt. As shown in the rules, there is always an output for every possibility. If the system cannot find the relevant rule in the knowledge base for the valid set of conditions, it cannot infer, resulting in an error. 23 Figure 2.4: The rules of the second algorithm that suggest what to wear depending on the weather conditions and the temperature. 2.1.2.2 Database or storage A database or storage is a component in an expert system, that acts as a memory, holding user-entered inputs related to the problem being solved, to be processed by the inference engine. The inference engine uses the input from the database, scans the knowledge base to find a matching antecedent part of a rule, and if they match, it draws a conclusion from the existing knowledge. The database stores the relevant inputs, intermediate results, and conclusions. In some cases, it pulls data from external sources. The difference between the database and the knowledge base is that the database holds the data related to the particular problem, whereas the knowledge base contains information from the domain to be used repeatedly for other problems (Gupta and Nagpal, 2020). 2.1.2.3 Inference Engine The inference engine works as an interpreter that compares the inputs the user enters into the system with the antecedent part of the rules. If there is a match, it infers a conclusion drawing the consequent part of the relevant rule. There are two kinds of inference methods: Forward chaining and backward chaining. In the forward chaining inference method, the inputs are first processed by the rules, while in the backward chaining inference method, the goal or consequent is processed initially, so 24 that the system looks for the relevant rules and necessary antecedents to conclude that determined goal. Returning to the same example, a forward-chaining inference engine would query the weather conditions to recommend which clothes the user should better wear. In the backward chaining inference engine, the user must decide first what to wear, and then the system checks what conditions are necessary to achieve the goal. Considering that the examples given in the current literature may be complex for readers unfamiliar with the subject, I will simply explain both methods with examples. The first example: In the forward-chaining inference method, the question order is as follows: 1- Ask about the weather 2- Ask about the temperature The user flow is shown in Figure 2.5 and the algorithm works as follows: STEP-1: Inference Engine → Explanation Facility: Prompt the first question. STEP-2: Explanation System → User Interface: How is the weather? a) Sunny b) Cloudy c) Rainy STEP-3: User’s Input: a → User Interface → Database: Sunny is true. STEP-4: Database → Inference Engine: Sunny is true. STEP-5: Inference Engine → Knowledge Base: If Sunny is true, then… • Wear a T-shirt. (AND if it is hotter than 22⁰C) 25 OR • Wear a T-shirt and a jacket (AND if it is colder than 22⁰C) STEP-6: Inference Engine → Explanation System: Prompt the second question. STEP-7: Explanation System → User Interface: Is it hotter than 22⁰C? • Yes • No STEP-8: User → Input: Yes → User Interface→ Database: Hot is true. STEP-9: Database → Inference Engine: Hot is true. STEP-10: Inference Engine → Knowledge Base: If Sunny AND Hot are true, then… • Wear a T-shirt. STEP-11: Inference Engine → Explanation System → User Interface: Wear a t-shirt. Figure 2.5: The user flow of a system using the forward chaining inference engine. The second example: In the system using the backward chaining inference engine, the question order is as follows: 1- Ask him what he would like to wear today. 2- Ask about the temperature. 26 3- Ask about the weather. The user flow is shown in Figure 2.6 and the algorithm works as follows: STEP-1: Inference Engine → Explanation System: Prompt the first question. STEP-2: Explanation System → User Interface: What do you want to wear today? a) T-shirt b) Jacket c) Raincoat d) T-shirt & Jacket e) T-shirt & Raincoat STEP-3: User’s Input: a → User Interface → Database: T-shirt is true. STEP-4: Database → Inference Engine: T-shirt is true. STEP-5: Inference Engine → Knowledge Base: Then the T-shirt is true, if… • It is hotter than 22⁰C. • It is colder than 22⁰C. STEP-6: Inference Engine → Explanation System: Prompt the second question. STEP-7: Explanation System → User Interface: Is it hotter than 22⁰C? • Yes • No STEP-8: User’s Input: Yes → User Interface → Database: Hot is true. STEP-9: Database → Inference Engine: Hot is true. 27 STEP-10: Inference Engine → Knowledge Base: Then T-shirt is true, if Hot is true AND if it is… • Sunny OR • Cloudy STEP-11: Inference Engine → Explanation System: Prompt the third question. STEP-12: Explanation System → User Interface: How is the weather? a) Sunny b) Cloudy c) Rainy STEP-13: User’s Input: a → User Interface → Database: Sunny is true. STEP-14: Database → Inference Engine: Sunny is true. STEP-15: Inference Engine → Knowledge Base: Then T-shirt is true if Hot and Sunny are true → The rule exists. STEP-16: Inference Engine → Explanation System → User Interface: You can wear a t-shirt. Both methods can be used for different purposes. In the medical field, expert systems with a forward chain inference engine can be used when diagnosing an unknown disease, while those with a backward chaining inference engine can act as diagnostic protocols to confirm the possible diagnosis made by healthcare professionals. 28 Figure 2.6: The user flow of a system using the backward chaining inference engine. 2.1.2.4 Explanation facility As noted in the literature (Gupta & Nagpal 2020), the explanation facility is a component that informs the user or developer of how the expert system achieves certain results by using which rules played a role during the inference process. The explanation facility enables users to backtrack on which data and rules the system has used in the process to get the output, so it increases the system’s accountability (Gaines, 1991). However, an additional function can be defined for this component. The knowledge base contains the outputs to be obtained as a result of the decision process and the rules that determine which premises these outputs depend on. So, in which part of this system do the questions that enable these inputs to be drawn from the user, and the possible answers to these questions take place? Whether the user is an expert or a non-expert and whether she/he understands what the inputs mean plays a key role at this point. If an expert system is designed for use by non-experts, there should be an intermediary component that allows the user to enter the correct input into the algorithm by explaining which input corresponds to what to increase the accuracy of the algorithm. Thus, the explanation facility can take over this function. 2.1.2.5 User interface The user interface is the component that allows the system to communicate with the user, it is where the interaction takes place. They give orders to the system through interfaces or answer the questions asked by the system. Users are generally not concerned with what is inside the program and how it works, but with how it 29 communicates with them. Communication has several dimensions including content, information architecture, navigation, graphics, layout, and the design of interface signs such as labels, affordances, and icons (Bolchini et al, 2009). Therefore, interface design in all its scope plays a critical role in the system’s usability. Expert systems have several advantages and disadvantages when compared to other artificial intelligence methods. The advantages of expert systems: - Reliability: Expert systems are highly reliable systems if the knowledge base is developed without bias and subjectivity (Gupta & Nagpal 2020). - Efficiency: The number of rules or the data volume is not a major concern for the computational resources used by the algorithm (Gupta & Nagpal 2020). - Transparent: It can be easily determined in which situations the system reaches a certain conclusion. The rules, inputs, and outputs are clearly compared to other complex algorithms (Gupta & Nagpal 2020). - Updateability: Developers can change or delete the rules and add new data to the knowledge base over time. - Practicality: Well-designed expert systems are practical and useful. They can act as reminders of forgotten or underrated facts in a field. The disadvantages of expert systems: - Incapable of Learning: Expert system algorithms are incapable of learning new subjects. They have no common sense or reasoning, so they cannot create new rules on their own, which makes them dependent on the developer. - Lack of Creativity: Expert systems cannot adapt themselves to new conditions or respond to inputs differently from the way they were formulated. They strictly adhere to the rules (Gupta & Nagpal 2020). - Limited Inference: Expert systems only infer the results formulated in the knowledge base. The parameters, weight points, calculations, and results 30 are predetermined until the developer makes changes. In this sense, their maintenance is expensive (Gupta & Nagpal 2020). 2.1.3 Designing interaction The interface is a medium where the user interacts with the system or the machine. Although every product has an interface, when it is mentioned, it usually comes to mind as an element of software products. In software, interface design is critical in terms of interaction and usability. HCI is a multidisciplinary field that concerns and studies the factors affecting the human use of interactive computing systems (ACM SIGCHI, 1992) in which computer science, cognitive psychology, human factors, engineering, design, and social and organizational psychology contribute (Liu et al, 2010). For the past decades, a wide range of studies have been conducted on various aspects of HCI such as dialogue techniques, gestural analysis, multimodal interfaces, computer graphics, computational linguistics, spatial cognition, robot navigation and wayfinding, input styles or devices, monitor screens, etc. but the ultimate goal of these studies has always been improving the interaction, usability, and user satisfaction (Liu et al, 2010). During the interaction, various factors arising from the user, the task, the technical system, and the work environment can affect users’ overall performances (Liu et al, 2010). Osvalder and Ulfvengren (2009) classify such factors into three categories: internal factors that are conditions affecting human operators mentally or physically such as age, perceptive and cognitive abilities, personality, and motivation, etc., external factors that refer to operational and environmental issues within the working contexts such as conditions of surroundings, tools and equipment, interfaces, etc., and lastly, stressors that negatively affect the operator’s decision-making process directly or indirectly, such as high workload and pace, pain, exhaustion, etc. Design mostly focuses on external factors, interfaces in particular. Interface design plays a major role in the system’s usability (Islam, 2013). Usability is defined as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of user” (ISO, 1998). According to this definition, if the product's target user can use it effectively and achieve their goals without any problems or setbacks, it can be said that the product is usable. 31 Bolchini et al. (2009) indicate that there are several dimensions of interface design: “content (what is communicated), information architecture (the structuring of information in a meaningful way), navigation (how the information architecture can be used and interacted with), graphics and layout” (p. 67). They consider signs used on the interface as “communication artifacts” that indicate or convey the presence of content, functions, and links (Bolchini et al, 2009, p. 67). Interfaces mostly consist of signs that come into the body in different forms such as links, images, buttons, icons, or animations, and are assigned various functions (Neumuller, 2001). Islam (2011) emphasizes the importance of signs by stating that if these dimensions are the same in two different interfaces, the one with more intuitive and well-used signs can be assumed to be more useful. During the interaction, communication takes place between two human proxies of systems; on one side the designer sends messages to subtly show how the system works or how the goals can be achieved, on the other side, the user receives and interprets these messages through these beacons. In the literature, the communication between these two sides is called “metacommunication” or “computer-mediated human communication” (de Souza, 2005; 2018, p. 45). In this sense, it can be assumed that the friendliness of the design is strongly related to the fact that how well the message is formed through the signs in the interface by the designer, and more critically, whether they are all received and interpreted as intended by the user. The key here is the degree to which the designer speaks the same language as the user. It was noted that semiotics can help designers enhance their power in communication (Barbosa et al, 2001). 2.1.3.1 The role of semiotics in interaction Signs are vehicles to transmit a message or information from someone who knows it and wants others to know it as well (Eco, 1988). Semiotics is the research field that studies “signs, signification, and signifying systems” as well as representation, meaning, and reference (Robert et al, 1992, in cited as Islam, 2013). Eco (1976) states “Semiotics is concerned with everything that can be taken as a sign” and defines it as a method of investigating how sign vehicles may function and a means of understanding how a sign vehicle may be produced and interpreted. 32 There are two major figures in the history of semiotics: Ferdinand de Saussure (1857-1913) and Charles Sanders Peirce (1839-1914). Saussure (1983) developed a model in which a sign is composed of two essences: the “signifier”, the form the sign takes, and the “signified”, the concept or meaning the sign represents. He calls the relationship between the two the “signification.” For example, when the perceiver encounters a “closed” sign hung on the door of a shop, he/she interprets the sign as the shop is closed for customers. What the perceiver sees on the door is considered a “signifier”, while what he/she interprets refers to a “signified” in Saussure’s model. Peirce (1931, p. 58), on the other hand, developed a triadic model that consists of three essences: the “representamen”, the “object” and the “interpretant.” Representamen is the representation of an entity, for example, if the entity is a person, it addresses him/her by creating an equivalent or a more advanced version of that person in the perceiver's mind (Ferreira et al, 2005). The object is the actual entity the sign represents (Peirce, 1931). The interpretant is the meaning of the sign created in the perceiver’s mind or the reaction caused by the object in the perceiver (Andersen & Nowack, 2002, as cited in Ferreira et al, 2005). For example, the diskette icon or the save icon in the form of a diskette is the representamen which represents the object of saving the document and is interpreted as “this sign is for saving my document file” (Islam, 2013, p. 48). Peirce (1931) classified signs into categories and the most acknowledged and fundamental sign classes are icon, index, and symbol. If the representamen resembles or in some way imitates the object, the sign is considered iconic. Ferreira et al. (2005) exemplify iconic signs with the portrait of Charles Sanders Peirce. When a perceiver sees it, he/she can easily recognize it (if she/he knows his appearance) as his portrait, because the representamen resembles the object (the real Peirce) enough (Ferreira et al, 2005). If there is a causal relationship between the representamen and the object, the sign is indexical. In such cases, the representamen creates a link that refers to the object in the perceiver’s mind. For example, a human footprint (representamen) in the snow indicates someone who has walked here (object) rather than the outsole of a shoe. If the relationship between the object and the representamen must be learned before it can be interpreted by the perceiver, the sign is considered symbolic (Ferreira et al, 2005). Symbols can also be defined as objects that create meanings for a particular group of people or culture (Luna et al, 2008). 33 Before making inferences, one must know in advance that the cross is the symbol of Christianity because nothing in the representamen allows the perceiver to interpret the cross in this way. In fact, being identified is a prerequisite for all kinds of signs as Peirce (1931, p. 58) stated “Nothing is a sign unless it is interpreted as a sign.” If a human footprint cannot be identified, it would not be possible for the perceiver to infer the footprint is a sign of human presence. The relationship between the object and the representamen is not like black and white, but rather a spectrum, and the three sign classes are not mutually exclusive, so a sign may contain elements of iconicity, indexicality, and symbolism at the same time to varying degrees (Peirce, 1931). A photograph can be given as a well-known example of a sign carrying all. However, one type of element tends to dominate in the sign, and when it happens, that type becomes prominent (Peirce, 1931). Likewise, the link between the object and the sign is not always monotony; some signs may have a single meaning, and some may have more than one (Islam, 2013). Perceivers often make assumptions about the meaning of the sign according to the context, for example, the C sign can be interpreted as the carbon element in chemistry or a programming language in computer science (Islam, 2013). In this sense, it can be said that meanings are contextual and derived from culture and not always universal, hence, a sign may be interpreted with different meanings by perceivers from different socio-cultural backgrounds (Guillemette & Cossette, 2006). In the complex world of signs with different representations and meanings, designers create interfaces to convey messages to users through them, to make possible the achievement of human goals in the systems (Nadin, 1988). Signs take different forms in the interface as buttons, words, scroll bars, tabs, images, etc. They all contribute to the communication process, given that designing can be considered “a semiotic activity” (Ferreira et al, 2005, p. 3). The purpose of the semiotic approach in design is to enable researchers to investigate the use of language/signs in interfaces and their effects on usability (Islam, 2013). Semiotics can benefit not only in increasing communicability but also in increasing usability and user satisfaction; it can assist designers in designing interfaces and hence, in improving users’ task performance (Islam, 2013). 34 Barr et al. (2002) took the interface elements as signs and analyzed them according to the Peircean model. They noted that representamen corresponds to the forms of the signs, the object corresponds to the underlying functionality, and the interpretant corresponds to what was generated in the mind of the user (Barr et al, 2002). Whether the user can successfully perform the intended task during interaction with the system depends on the congruency between the meaning (interpretant) and the sign’s function (the object) because if the user gets the wrong idea, it means that there has been a failure in the sign use and its representamen (Barr et al, 2002). Icons are commonly used in software systems because people generally recognize pictorial descriptions of the same things more easily than their verbal descriptions, they are more cross-cultural than language and occupy less space in the interface (Apple Computer Inc., 1992). Barr et al. (2002) defined icons in computer interfaces as another type of sign with a specific purpose and evaluated them to elucidate their strengths and weaknesses. They noted that the benefit of iconic and indexical icons is that they have the potential to be perceived and understood in the same fashion by users from all around the world with minimal training because of their resemblance (iconic icons) with the actual objects and their causal link to the functionality (indexical icons) having strong ties with our perception of the physical world (Barr et al, 2002). However, they underscored that the representamen of these types of icons is subject to physical constraints, therefore, choosing the right representation can be challenging for designers (Barr et al, 2002). On the other hand, they also noted that symbolic icons are arbitrary in nature, and they do not have any connection with the physical world (Barr et al, 2002). This implies that the nature of this type of icon not only allows designers to design them more easily but also requires users to learn what they mean beforehand and how to use them (Barr et al, 2002). Ferreira et al. (2005) analyzed the interfaces of three sample software semiotically and redesigned the problematic elements they detected. As a result, they stated that the more symbolic signs are used in the interfaces, this will cause problems in navigation and functionality of the system for novice users because symbolic signs require such users to learn by trial and error how to perform their tasks (Ferreira et al, 2005). They also stated that if almost all the signs in the interface are associated with a function, there is a causal relationship, thus meaning that they can all be considered indexical, but how the function is represented by the sign redefines itself. (Ferreira et 35 al, 2005). For exa