Abstract—In response to the growing need for privacy-preserving mobile intelligence, this study introduces a new approach that combines Generative Pre-trained Transformer 4 (GPT-4), a state-of-the-art large language model, with Particle Swarm Optimization (PSO) in a two-step process. This method is designed to find efficient neural network structures in federated learning and address issues like high communication costs and unstable network conditions. Leveraging the prowess of GPT-4 for initial population guidance in the Neural Architecture Search (NAS) process, our approach focuses on optimizing neural network architectures that demand minimal data exchange between clients and servers. This is achieved through a variable-length PSO encoding and decoding mechanism at the upper level, ensuring not only a thorough search for efficient architectures but also their optimization for compactness and effectiveness. Additionally, a standard PSO technique is applied at the lower level to optimize neural network weights, thus boosting model performance with reduced communication load. Our methodology’s superiority is demonstrated via benchmark comparisons with FedAvg and FedPSO on the CIFAR-10 dataset, under both normal and compromised network scenarios.
 
Keywords—Generative Pre-trained Transformer 4 (GPT-4), federated learning, Particle Swarm Optimization (PSO), Neural Architecture Search (NAS), communication cost


Copyright © 2024 by the authors. This is an open access article distributed under the Creative Commons Attribution License (CC BY-NC-ND 4.0), which permits use, distribution and reproduction in any medium, provided that the article is properly cited, the use is non-commercial and no modifications or adaptations are made.