ML-TN-007 — AI at the edge: exploring Federated Learning solutions

Info Box Applies to Machine Learning Applies to Machine Learning Technical Notes

History

Version	Date	Notes
1.0.0	August 2023	First public release

Introduction

According to Wikipedia, Federated Learning (FL) is defined as a machine learning technique that trains an algorithm via multiple independent sessions, each using its own dataset. This approach stands in contrast to traditional centralized machine learning techniques where local datasets are merged into one training session, as well as to approaches that assume that local data samples are identically distributed.

Federated learning enables multiple actors to build a common, robust machine learning model without sharing data, thus addressing critical issues such as data privacy, data security, data access rights and access to heterogeneous data. Its applications engage industries including defense, telecommunications, Internet of Things, and pharmaceuticals. A major open question is when/whether federated learning is preferable to pooled data learning. Another open question concerns the trustworthiness of the devices and the impact of malicious actors on the learned model.

In principle, FL can be an extremely useful technique to address critical issues of industrial IoT (IIoT) applications. As such, it matches perfectly DAVE Embedded Systems' IIoT platform, ToloMEO. This Technical Note (TN) illustrates how DAVE Embedded Systems explored, tested, and characterized some of the most promising open-source FL frameworks available to date. One of these frameworks might equip ToloMEO-compliant products in the future allowing our customers to implement federated learning systems easily. From the point of view of machine learning, therefore, we investigated if typical embedded architectures used today for industrial applications are suited for acting not only as inference platforms — we already dealt with this issue here — but as training platforms as well.

In brief, the work consisted of the following steps:

• Selecting the FL frameworks to test.
• Testing the selected frameworks.
• Comparing the results for isolating the best framework.
• Deep investigation of the best framework.

A detailed dissertation of the work that led to this Technical Note is available here TBD.

Choosing Federated learning frameworks

For selecting the frameworks, several factors were taken into account:

• ML frameworks flexibility: The adaptability of the framework to manage different ML frameworks.
• Licensing: It is mandatory that the framework has an open-source, permissive license to cope with the typical requirements of real-world use cases.
• Repository rating and releases: Rating in a repository is important for a FL framework as it indicates a high level of community interest and support, potentially leading to more contributions and improvements. Meanwhile, the first and latest releases indicate respectively the maturity and the support of the framework and whether it is released or still in a beta version.
• Documentation and tutorials: The provided documentation with related tutorials has to be complete and well-made.
• Readiness for commercial usage: The readiness of the framework to be developed in a real-world scenario. In order to establish the readiness, it was checked the version of the framework and the license.

According to the previous criteria, an initial list including the most promising FL frameworks was completed. It comprised of the following products:

• NVIDIA FL Application Runtime Environment (NVFlare)
• Federated AI Technology Enabler (FATE)
• Flower
• PySyft
• IBM Federated Learning (IBMFL)
• OpenFL
• TensorFlowFederated (TFF)
• FedML.

The limitation in selecting only eighth FL frameworks arises from the evolving nature of the field. As a relatively recent and rapidly evolving technique, FL continues to witness the emergence of various frameworks, each with its unique features and capabilities. In this context, the choice to focus on these frameworks reflects the attempt to capture the current state of the art and provide an analysis of the most prominent and well-established options available. This selection aims to offer valuable insights into the leading frameworks that are currently considered among the best choices in the evolving landscape of FL.

In the next sections, the aforementioned factors will be treated individ-

ually, justifying the reasons behind the discard of some frameworks rather

than others.

Testing the selected frameworks

Flower

Flower is

Flower running on SBC ORCA

# of cores
1
4

NVFlare

NVFlare is

TBD

Comparing test results

TBD

Deep investigation of NVFlare

TBD

Conclusions and future work

TBD

One important issue that was not addressed yet is the labeling of new samples. In other words, it was implicitly assumed that new samples collected by a device are somehow labelled prior to being used for training. This is a strong assumption because implies that

labeling of new samples issue

• https://bdtechtalks.com/2021/08/09/what-is-federated-learning/
• unsupervised learning?
  • https://arxiv.org/pdf/1805.03911.pdf