LEE- Hesaplamalı Bilim ve Mühendislik-Yüksek Lisans

Bu koleksiyon için kalıcı URI

http://hdl.handle.net/11527/19296

Gözat

Konu "compressed sensing (telecommunication)" ile LEE- Hesaplamalı Bilim ve Mühendislik-Yüksek Lisans'a göz atma
  • Öge
    Comression of convolutional neural networks via high dimensional model representation
    (ITU Graduate School, 2025) Yılmaz, Berna ; Tuna, Süha ; 702221002 ; Physics Engineering
    In recent years, the growing complexity of deep learning architectures—particularly convolutional neural networks (CNNs)—has introduced substantial challenges related to model size, computational overhead, and real-time deployability on resource-limited platforms. Although architectures such as ResNet and VGG have set new benchmarks in image classification tasks by leveraging deep and expressive structures, their significant number of parameters and high inference latency present obstacles when deploying these models on edge devices, embedded systems, or mobile applications. These challenges underscore the necessity of effective model compression techniques that maintain accuracy while reducing computational and memory demands. This thesis addresses these limitations by systematically exploring tensor-based model compression strategies for CNNs, with a specific focus on Tensor Train (TT) decomposition and High Dimensional Model Representation (HDMR). TT decomposition is a powerful low-rank tensor factorization technique that compresses high-dimensional weight tensors into a sequence of interconnected low-rank cores, significantly reducing the parameter count while preserving the structural hierarchy of convolutional layers. HDMR, in contrast, is a functional decomposition approach that approximates a multivariate function as a hierarchical sum of univariate and multivariate component functions, each capturing individual or interactive effects of input variables. While TT operates on the structural representation of tensors, HDMR operates in the functional domain, enabling interpretability and capturing non-linear interactions. Beyond analyzing these techniques in isolation, the study also introduces and evaluates two hybrid compression strategies: (1) TT->HDMR, where TT decomposition is first applied to CNN weight tensors, followed by HDMR analysis on the resulting TT cores, and (2) HDMR->TT, where HDMR is initially used to functionally decompose the weight tensors, and TT is subsequently applied to compress the resulting components. These methods are integrated into four popular CNN architectures—ResNet-18, ResNet-34, VGG16, and VGG19—and evaluated on the CIFAR-10 dataset. Experimental results reveal that TT->HDMR consistently achieves favorable compression-accuracy trade-offs. This strategy leads to up to 6.31× parameter reduction in certain configurations (e.g., ResNet-34) while preserving or even improving the classification accuracy. In contrast, HDMR->TT often results in increased parameter counts and degraded performance due to the high number of interaction terms generated during HDMR expansion (especially when higher-order terms such as 3rd-order components are included). When HDMR is applied directly, a moderate compression is observed, but this comes at the cost of increased inference time and memory due to the expansion of functional components. These findings highlight the importance of decomposition order and the selection of HDMR truncation order in determining the efficiency and effectiveness of hybrid compression strategies. Furthermore, the thesis shows that TT alone is highly effective in compressing convolutional layers while maintaining a balanced performance profile. It is particularly well-suited for models with redundant parameter structures such as VGG19. HDMR, while not inherently a tensor-based method, introduces new perspectives in interpretable compression, although its direct integration into deep networks must be done cautiously to avoid parameter inflation. In conclusion, this study does not merely promote one optimal solution but instead explores a space of tensor-based and function-based compression methods with the aim of discovering whether more efficient decompositions can be achieved without sacrificing accuracy. The results demonstrate that hybrid methods—when applied with the right sequence and order truncation—can uncover more compact and accurate representations of deep CNNs, suitable for real-world deployment. These findings offer valuable guidance for future research, including potential adaptations of TT and HDMR techniques to transformer-based architectures, federated learning settings, or multi-modal models where compression, communication cost, and interpretability are all critical factors.