Analysis of visual design principles in art and architecture by computer vision and learning based model

Abstract

Visual design is associated with different uses and organizations of design elements and principles. They are explained in numerous books in art and design disciplines as the bases of visual communication. Those are applied subjectively by the designers in various disciplines for aesthetics and presentation of information. For the constitution of a perceptual framework for visual processing, the logical procedures that use the design elements are called visual design principles (VDP) ; three are selected as the main principles for this study: emphasis, balance, and rhythm. As the examples of these principles were inspected, it was established that the use of the design elements differed and led to sub-visual similarities existing in their compositions, despite following the main organizational rationale. So nine sub-VDP are defined, which have similar visual patterns: color, isolation, shape, symmetric, asymmetric, crystallographic, regular, progressive and flowing. Although numerical analysis of design visuals is considered as hard, it has become possible with emerging artificial intelligence (AI) technologies. Due to the advances in computer vision applications, a deep learning model can identify these underlying common visual patterns in the data. This Ph.D. thesis develops an approach to detect and classify the VDP in a visual composition over different domains, including photography, art (paintings, prints and graphic art) and architecture (building facade visuals) by a neural network model. The AI applications in art, design, and architecture conducted by the disciplines of computer science and design have been found, analyzed and the models, methods, numbers, and types of data used in the studies have been extracted. Next to the compiled knowledge in AI studies in art and architecture, the manual and computational analyzes of the building facade in architecture have been researched. As there was no existing dataset for this problem, three genuine datasets have been created in the given domains for this study. The majority of the examples showing the VDP directly belong to the contemporary era, so the data search has been oriented toward this period. Various websites and online museum databases are used for collecting the data. The amount of data found for the labels of VDP in each domain has been kept as high as possible to achieve high performance from the deep learning model. Multiple experiments are structured for testing the model. Classification results within the domains are evaluated by considering the clarity and the amount of the data. The effect of the labeling procedure in the preparation of the initial datasets is discussed by analyzing multi-class and multi-label classification results. Also, domain adaptation is investigated with instances tested in models trained in other domains. The knowledge of myriads of original designs, captured by the underlying computational patterns, can be used to consolidate the design process by providing an objective evaluation of the visual compositions.