Flexibility in ML frameworks is crucial when choosing a FL framework as it allows adapting the system to diverse use cases and data distributions. A flexible framework can support various ML algorithms, models, and data types, accommodating the specific requirements of different scenarios. This adaptability enhances the framework’s applicability in real-world deploy- mentsdeployments, ensuring it can effectively handle the heterogeneity and dynamic na- ture nature of distributed data sources across clients. It is noteworthy to mention that most of the frameworks discussed ear- lierearlier, including NVFlare, FATE, Flower, PySyft, IBM, OpenFL, and FedML, are designed to be agnostic to the underlying ML framework used. This ag- nosticism agnosticism provides users with the flexibility to employ various ML libraries, such as TensorFlow, PyTorch, and scikit-learn, among others, based on their preferences and project requirements. However, it is important to highlight that one exception to this trend is TFF. Unlike the other frameworks, TFF is specifically tailored for the TensorFlow ecosystem. While it harnesses the powerful capabilities of Ten- sorFlowTensorFlow, it inherently limits the utilisation of other ML libraries in the FL context. As a result, users opting for TFF should be aware of this frame- work’s framework’s dependency on TensorFlow for their FL endeavors. For that reason TFF was discarded from the potential frameworks to be considered for comparison.

The choice of a suitable license is of paramount importance for any FL frame- work [19]framework. A well-crafted license provides a legal foundation that governs the usage, distribution, and modification of the framework’s source code and as- sociated associated components. A permissive license, like the MIT License or Apache License, allows users to use, modify, and distribute the framework with relatively few restrictions. This encourages widespread adoption, fosters innovation, and facilitates con- tributions from a broader community of developers and researchers. The per- missiveness of these licenses empowers users to incorporate the framework into their projects, even if they have proprietary components. On the other hand, copyleft licenses, like the GNU GPL, require derived works to be distributed under the same terms, ensuring that any modifica- tions or extensions to the framework remain open-source. While this may be more restrictive, it encourages a collaborative ecosystem where improve-

ments to use, modify, and distribute the framework with relatively few restrictions. This encourages widespread adoption, fosters innovation, and facilitates contributions from a broader community of developers and researchers. The permissiveness of these licenses empowers users to incorporate the framework into their projects, even if hey have proprietary components. On the other hand, copyleft licenses, like the GNU GPL, require derived works to be distributed under the same terms, ensuring that any modifications or extensions to the framework remain open-source. While this may be more restrictive, it encourages a collaborative ecosystem where improvements are shared back with the community. A clear and well-defined license also provides legal protection to both developers and users, helping to mitigate potential legal risks and disputes. It ensures that contributors have granted appropriate rights to their work and helps maintain a healthy and sustainable development environment. Most of the frameworks previously described are under the Apache-2.0 license except one: IBMFL. In fact, it is under an unspecified license that makes the framework not suitable for commercial use. For that reason, IBMFL was discarded from the comparison too.

A clear and well-defined license also provides legal protection to both developers and users, helping to mitigate potential legal risks and disputes. It ensures that contributors have granted appropriate rights to their work and helps maintain a healthy and sustainable development environment. Most of the frameworks previously described are under the Apache-2.0 license except one: IBMFL. In fact, it is under an unspecified license that makes the framework not suitable for commercial use. For that reason, IBMFL was discarded from the comparison too. ==== Repository stars rating and releases ====Stars Ratings in public repositories such as "stars" in a GitHub repository are important because they serve as a measure of popularity and community interest in the project. When a repository receives more starsachieves a good rating, it indicates that more developers and users find the project valuable and relevant. This can lead to several benefits:* Visibility: Repositories with more stars good ratings are likely to appear higher in GitHub platform's search results, making it easier for others to discover and use the project.* Credibility: High-starred rating repositories are often perceived as more trust- worthy and reliable, as they are vetted and endorsed by a larger user base.* Contributions: Popular repositories tend to attract more contribu- tions contributions from developers, leading to a more active and vibrant community around the project.* Feedback: Projects with many stars good ratings are more likely to receive feedback, bug reports, and feature requests, helping the developers improve the software.* Maintenance: Higher stars ratings can also stimulate the maintainers to keep the project updated and actively supported. Other important aspects, which are rating-related to the stars obtained by the framework, aspects are the first and latest releases. Thanks to the latter, it is possible respectively to see the maturity of the framework and also how often it is updated, and thus the support behind it. Obviously, a framework that was born earlier than others is much more likely to have more starsbetter ratings. Having this in mind, at the time of writing this thesis, the ranking in terms of star ratings received stars correlated with the first release for each framework is as follows:** PySyft : 8.9k stars / Jan 19, 2020 2. ** FATE : 5.1k stars / Feb 18, 2019 3. ** FedML : 3.1k stars / Apr 30, 2022 4. ** Flower : 2.8k stars / Nov 11, 2020 5. ** TFF : 2.1k stars / Feb 20, 2019 6. ** OpenFL : 567 stars / Feb 1, 2021 7. ** IBMFL : 438 stars / Aug 28, 2020 8. ** NVFlare 413 stars / Nov 23, 2021 These characteristics, although they certainly have a bearing on the choice of frameworks, were not enough to go so far as to discard any of the selected frameworks.

* High quality documentation and well-crafted tutorials are essential consid- erations considerations when selecting a FL framework. In fact, there are several reasons that are presented here below:* Accessibility and Ease of Use: Comprehensive documentation allows users to understand the framework’s functionalities, APIs, and usage quickly. It enables developers, researchers, and practitioners to get started with the framework efficiently, reducing the learning curve. • * Accelerated Development: Well-structured tutorials and examples demon- strate demonstrate how to use the framework to build practical FL systems. They provide step-by-step guidance on setting up experiments, running code, and interpreting results. This expedites the development process and encourages experimentation with different configurations. • * Error Prevention: Clear documentation and good examples help users avoid common mistakes and errors during implementation. It provides troubleshooting tips and addresses frequently asked questions, reducing frustration and increasing user satisfaction. • * Reliability and Robustness: A well-documented framework indicates that developers have invested time in organising organizing their code and ex- plaining explaining its functionalities. This attention to detail suggests a more reliable and stable framework. • * Maintenance: Higher stars can also stimulate the maintainers to keep the project updated and actively supported.Regarding this aspects, there are a lot of frameworks that still don’t have good documentation and tutorials. Among the latter, there are: PySyft, OpenFL and FedML. PySyft is still under construction, as the official repos-

! align="center" | '''Framework'''! align="center" | '''NVFlare'''! align="center" | '''FATE'''! align="center" | '''Flower'''! align="center" | '''PySyft'''! align="center" | '''IBMFL'''! align="center" | '''OpenFL'''! align="center" | '''TFF'''! align="center" | '''FedML'''