= Applying NVFlare to a real-word case =
TBDIn the above comparison, among the two contenders, NVFlare has demonstrated its superiority in terms of addressing the intricate challenges posed by the modern landscape of FL. While Flower demonstrated efficiency in terms of execution time, NVFlare excelled in other crucial topics in real-world deployments. NVFlare’s robust privacy and security architecture significantly outperformed Flower’s, ensuring that sensitive data and model parameters are adequately protected during the FL process. This advantage is of paramount importance, especially in applications involving confidential or sensitive information. Furthermore, NVFlare’s ease of use and comprehensive support infrastructure were pivotal in its selection as the preferred framework. Its straightforward setup and configuration, along with user-friendly APIs and well-documented resources, make it accessible to a wide range of developers and data scientists. While Flower may have a simpler, more centralized server-side architecture, NVFlare’s user-centric approach ensures that users can efficiently leverage its capabilities for their FL projects. However, it is essential to recognize that real-world application scenarios can be considerably more intricate and multifaceted. To address this aspect, a more intricate simulated scenario involving NVFlare warrants examination. In this extended analysis, a more complex use case will be explored, involving the testing of various algorithms and data heterogeneity to assess the impact of this factor on the results and on different algorithms under consideration. By assessing NVFlare’s performance within a more intricate context, the aim is to provide a holistic view of its capabilities and limitations, contributing to a more comprehensive understanding of its viability in various practical scenarios without forgetting even the scalability of the framework. == Advanced system design ==The design settings of this advanced FL system remain consistent with those utilised in the previous comparison between NVFlare and Flower, referred to as the "Local Environment" in section 4.1.1, unless some changes. In this sce- nario, the same desktop machine was utilised, equipped with an NVidia RTX 3080 Ti GPU. The ML framework, Pytorch, remained consistent, as did the Data Preprocessing involving Dataset selection, Dataset splitting, and Data augmentation. However, a significant alteration was introduced between the initial two design components, elaborated upon in subsection 4.1.4, referred to as "Data heterogeneity". The Model configuration and client-side set- tings also remained unchanged. Minor adjustments were made to the met- rics taken into consideration, focusing exclusively on two: local training loss and server validation accuracy. On the server side, the configuration under- went modifications. While maintaining a count of four clients, the number of communication rounds was elevated to 20 in this particular scenario. == FL Algorithms and Centralised Simulation ==One of the two main changes made to the system design was to simulate a centralised training baseline and to consider two other algorithms in addition to FedAvg. The centralised training was conducted using a single client for 20 local epochs, aiming to simulate a ML environment. This approach served as a reference point for comparison against various instances of FL. The other two FL algorithms employed in this study are Federated Op- timisation (FedProx) [37] and Stochastic Controlled Averaging for FL (Scaf- fold) [52]. Starting with FedProx, this algorithm extends the conventional FedAvg method by introducing a proximal term. The proximal term adds a regulari- sation factor to the optimisation process, enhancing the convergence rate and stability of the model across participating clients. FedProx achieves this by optimising the global model using both local updates and a global proximal term, which balances the contributions of individual clients while preventing divergence. This can be better seen in the Listing 5.1: == Data heterogeneity ==In this advanced project, an additional feature was incorporated involving the integration of classes aimed at performing dataset splitting among the designated clients, which, in this instance, were four in number. In addition to dividing the dataset into four subsets, the possibility of choosing the level of heterogeneity of the data [48] was added by applying the Dirichlet sampling strategy [36]. The Dirichlet distribution is a distribution over vectors x that fulfil the == Results analysis ==A series of seven experiments were conducted. The first experiment involved a centralised simulation, while the remaining six experiments focused on test- ing three different algorithms: FedAvg, FedProx and Scaffold. Specifically, each algorithm was tested twice: the first time with α = 0.1 and the second time with α = 1.0.
= Conclusions and future work =
