The goal of UniFed is to systematically evaluate the existing open-source FL frameworks, and therefore make recommendations for proper FL frameworks, algorithms, and configurations automatically based on user requests. In addition, with 15 evaluation scenarios, we present both qualitative and quantitative evaluation results of 11 popular open-sourced FL frameworks, from the perspectives of functionality, usability, and system performance for comparison. Please find more details in our paper here.
Based on our performance, functionality, and usability evaluation and survey, we built a decision tree to help users choose the best FL framework for their scenarios, which can be easily accessed through our FL recommendation framework. We also provide an Auto-Run function to automatically generate preferred FL configurations, allowing users to easily use any integrated FL framework and corresponding settings based on their use case.
|
â‘ From the functionality and usability survey, we built a decision tree to help users choose the best FL framework for their scenarios. This can be more easily accessed through our recommendation system.
â‘¡ After finding potential matching frameworks, the user can use the Auto-Run to generate the configuration file for testing scenarios.
â‘¢ Given the configuration file as input, UniFed initiates a distributed training task using the user-specified framework and data source.
â‘£ UniFed outputs the trained model, as well as the distributed execution log in a consistent format, which can be directly used for framework comparison. One can further investigate the log by connecting with other data analysis platforms integrated with UniFed.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|