Vui lòng dùng định danh này để trích dẫn hoặc liên kết đến tài liệu này: https://elib.vku.udn.vn/handle/123456789/3989
Nhan đề: OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning
Tác giả: Thwal, Chu Myaet
Nguyen, Huu Nhat Minh
Tun, Ye Lin
Kim, Seong Tae
Thai, My T.
Hong, Choong Seon
Từ khoá: Federated learning
machine learning
distributed learning
Vision Transformer
neural networks
widespread application
communication bandwidth
data distributions
training data
local features
depthwise separable convolutions
data heterogeneity
Năm xuất bản: thá-2024
Nhà xuất bản: Elsevier Ltd
Tóm tắt: Federated learning (FL) has emerged as a promising approach to collaboratively train machine learning models across multiple edge devices while preserving privacy. The success of FL hinges on the efficiency of participating models and their ability to handle the unique challenges of distributed learning. While several variants of Vision Transformer (ViT) have shown great potential as alternatives to modern convolutional neural networks (CNNs) for centralized training, the unprecedented size and higher computational demands hinder their deployment on resource-constrained edge devices, challenging their widespread application in FL. Since client devices in FL typically have limited computing resources and communication bandwidth, models intended for such devices must strike a balance between model size, computational efficiency, and the ability to adapt to the diverse and non-IID data distributions encountered in FL. To address these challenges, we propose OnDev-LCT: Lightweight Convolutional Transformers for On-Device vision tasks with limited training data and resources. Our models incorporate image-specific inductive biases through the LCT tokenizer by leveraging efficient depthwise separable convolutions in residual linear bottleneck blocks to extract local features, while the multi-head self-attention (MHSA) mechanism in the LCT encoder implicitly facilitates capturing global representations of images. Extensive experiments on benchmark image datasets indicate that our models outperform existing lightweight vision models while having fewer parameters and lower computational demands, making them suitable for FL scenarios with data heterogeneity and communication bottlenecks.
Mô tả: Neural Networks; Volume 170; pp: 635-649
Định danh: https://doi.org/10.1016/j.neunet.2023.11.044
https://elib.vku.udn.vn/handle/123456789/3989
Bộ sưu tập: NĂM 2024

Các tập tin trong tài liệu này:

 Đăng nhập để xem toàn văn



Khi sử dụng các tài liệu trong Thư viện số phải tuân thủ Luật bản quyền.