Many popular web-based services and data mining applications nowadays leverage the power of machine learning (ML) and artificial intelligence (AI) to ensure effective performance. All of these are made possible because of the huge volume of data constantly generated on various devices, such as PCs/laptops and mobile smartphones.

    Centralized ML and AI pose significant challenges due to regulatory and privacy concerns in real-world use cases. Privacy has been traditionally viewed as an essential human right. There have been increasing legislation endeavors on data privacy protection, e.g. European Union General Data Protection Regulation and California Consumer Privacy Act.

    Federated learning (FL) is a new paradigm in machine learning that was first introduced by Google in 2017. It aims to address the challenges above by training a global model using distributed data, without the need for the data to be shared nor transferred to any central facility. Despite the clear advantages, there are still many technical challenges waiting to be solved, such as fairness issues, data statistical heterogeneity, communication efficiency and network robustness.

    The workshop is targeted on the above and other relevant issues, aiming to create a platform for people from academia and industry to communicate their insights and recent results.

    Topics of interest include, but are not limited to, the following:

• FL algorithm related issues, e.g. adversarial attack, communication compression, algorithm explainability/interpretability, data/device heterogeneity, optimization algorithm advances, personalization, fairness, resource efficiency, and so on;
• FL and collaborative ML applications, like advertising, query analysis and processing, web healthcare, search engine, log mining, recommender system, blockchain，social network, and others;
• Other data privacy preservation techniques, such as differential privacy, secure multi-party computing, data/model distillation, data anonymization, etc;
• Social, operational challenges and legislation issues about privacy in web search and data mining;
• Datasets and open-source tools for federated and privacy-preserving web search and data mining.

Yang Liu, Tsinghua University

Towards Private, Efficient and Robust Cross-Silo Federated Learning

In this talk, I will mainly introduce the concept and advances for cross-silo Federated learning. Cross-silo federated learning (FL) enables organizations (e.g., financial, or medical) to collaboratively train a machine learning model without sharing privacy-sensitive data. Applying cross-silo Federated Learning to real-world systems still faces major challenges, including privacy protection, model complexity and performance, computation and communication efficiency in heterogeneous environments. In this talk, we discuss new advances in research for tackling these challenges. To this end, we introduce a new paradigm to take advantage of a powerful server model to boost the privacy, robustness and efficiency in a unified FL framework, where knowledge instead of model parameters/gradients are communicated and accumulated.

Ameet Talwalkar, Carnegie Mellon University

Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing

Tuning hyperparameters is a crucial but arduous part of the machine learning pipeline. Hyperparameter optimization is even more challenging in federated learning, where models are learned over a distributed network of heterogeneous devices; here, the need to keep data on device and perform local training makes it difficult to efficiently train and evaluate configurations. In this work, we investigate the problem of federated hyperparameter tuning. We first identify key challenges and show how standard approaches may be adapted to form baselines for the federated setting. Then, by making a novel connection to the neural architecture search technique of weight-sharing, we introduce a new method, FedEx, to accelerate federated hyperparameter tuning that is applicable to widely-used federated optimization methods such as FedAvg and recent variants. Theoretically, we show that a FedEx variant correctly tunes the on-device learning rate in the setting of online convex optimization across devices. Empirically, we show that FedEx can outperform natural baselines for federated hyperparameter tuning by several percentage points on the Shakespeare, FEMNIST, and CIFAR-10 benchmarks, obtaining higher accuracy using the same training budget.

Salman Avestimehr and Chaoyang He, FedML Inc. & University of Southern California

FedML: Social, Secure, Scalable, and Efficient Edge-Cloud Platform for Federated Learning

We provide an overview of the FedML platform (https://fedml.ai), which consists of (1) cutting-edge federated learning algorithms; (2) a lightweight and cross-platform Edge AI SDK for deployment over GPUs, smartphones, and IoTs; and (3) a user-friendly MLOps platform to simplify collaboration and real-world deployment; and (4) platform-supported vertical Solutions across a broad range of industries. We highlight several key design features and their algorithmic innovations. We also present a tutorial on using FedML.

Heiko Ludwig, IBM Almaden Research Center

Federated Learning for the Enterprise - Addressing Organizational and Regulatory Boundaries for Machine Learning

Increasing use of machine learning and other uses of data have led to increased regulation around the world. Under particular scrutiny is the gathering of large sets of person-identifying data. Data locality and use regulation as well as liability risk exposure is leading to an increased consideration of federated learning in enterprise use cases. Federated learning also provides increased opportunity for collaboration between companies on non-competitive models. This talk will review some use cases arising in enterprise federated learning, address some important issues arising and some recent approaches to solving these issues.