Home Research Papers Datasets State of the Art Pricing

Discover, visualize, and connect AI research papers. Explore the latest trends and insights in artificial intelligence research.

Product

Home
Research Papers
About

Support

Contact
Terms of Service
Privacy Policy

© 2026 Papersgraph. All rights reserved.

miscellaneous-2

Distributed Optimization

3260 papers • 126 benchmarks • 313 datasets

The goal of Distributed Optimization is to optimize a certain objective defined over millions of billions of data that is distributed over many machines by utilizing the computational power of these machines. Source: Analysis of Distributed StochasticDual Coordinate Ascent

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in distributed-optimization-4

Trend

Dataset

Best Model

Actions

No benchmarks available.

Libraries

i

Use these libraries to find distributed-optimization-4 models and implementations

gingsmith/cocoa

3 papers 90

Datasets

No datasets available.

Subtasks

No subtasks available.

Most implemented papers

SCAFFOLD: Stochastic Controlled Averaging for Federated Learning

Sebastian U. Stich, M. Mohri, A. Suresh, Sashank J. Reddi, Sai Praneeth Karimireddy, Satyen Kale•Sun Oct 13 2019

This work obtains tight convergence rates for FedAvg and proves that it suffers from `client-drift' when the data is heterogeneous (non-iid), resulting in unstable and slow convergence, and proposes a new algorithm (SCAFFOLD) which uses control variates (variance reduction) to correct for the ` client-drifts' in its local updates.

3548

Content

Introduction Benchmarks Datasets Subtasks Libraries Papers

2 papers 4,405

learning-at-home/hivemind

2 papers 1,867

MLOPTPSU/FedTorch

2 papers 178

0

Federated Optimization in Heterogeneous Networks

M. Zaheer, Ameet Talwalkar, Virginia Smith, Anit Kumar Sahu, Tian Li, Maziar Sanjabi•Thu Dec 13 2018

This work introduces a framework, FedProx, to tackle heterogeneity in federated networks, and provides convergence guarantees for this framework when learning over data from non-identical distributions (statistical heterogeneity), and while adhering to device-level systems constraints by allowing each participating device to perform a variable amount of work.

6966 0

ZOOpt: a toolbox for derivative-free optimization

Yu-Ren Liu, Yi-Qi Hu, Hong Qian, Yang Yu, Chao Qian•Sat Dec 30 2017

The ZOOpt toolbox that provides efficient derivative-free solvers and are designed easy to use, and particularly focuses on optimization problems in machine learning, addressing high-dimensional, noisy, and large-scale problems.

31 0

Secure Distributed Training at Scale

Max Ryabinin, Michael Diskin, Eduard A. Gorbunov, Alexander Borzunov•Sun Jun 20 2021

This work proposes a novel protocol for secure (Byzantine-tolerant) decentralized training that emphasizes communication efficiency and rigorously analyzed this protocol provides theoretical bounds for its resistance against Byzantine and Sybil attacks and shows that it has a marginal communication overhead.

17 0

L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework

Martin Jaggi, Michael I. Jordan, Virginia Smith, Simone Forte•Sat Dec 12 2015

This research presents a novel probabilistic approach to estimating the response of the immune system to laser-spot assisted, 3D image analysis of central nervous system injury.

28 0

CoCoA: A General Framework for Communication-Efficient Distributed Optimization

Martin Jaggi, Michael I. Jordan, Virginia Smith, Simone Forte, Chenxin Ma, Martin Takác•Mon Oct 31 2016

This work presents a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme and is applicable to a wide variety of problems in machine learning and signal processing, and extends the framework to cover general non-strongly-convex regularizers, including L1-regularized problems like lasso.

280 0

Robust Learning from Untrusted Sources

Christoph H. Lampert, Nikola Konstantinov•Mon Jan 28 2019

This work derives a procedure that allows for learning from all available sources, yet automatically suppresses irrelevant or corrupted data, and shows that this method provides significant improvements over alternative approaches from robust statistics and distributed optimization.

75 0

Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization

Mohammad Mahdi Kamani, Farzin Haddadpour, M. Mahdavi, V. Cadambe•Tue Oct 29 2019

This paper shows that for loss functions that satisfy the Polyak-Kojasiewicz condition, rounds of communication suffice to achieve a linear speed up, that is, an error of $O(1/pT)$, where $T$ is the total number of model updates at each worker.

219 0

FedDANE: A Federated Newton-Type Method

M. Zaheer, Ameet Talwalkar, Tian Li, Virginia Smith, Anit Kumar Sahu, Maziar Sanjabi•Thu Oct 31 2019

This work proposes FedDANE, an optimization method that is adapted from DANE, a method for classical distributed optimization, to handle the practical constraints of federated learning, and provides convergence guarantees for this method when learning over both convex and non-convex functions.

168 0

Training Large Neural Networks with Constant Memory using a New Execution Algorithm

B. Pudipeddi, Maral Mesmakhosroshahi, Jinwen Xi, S. Bharadwaj•Wed Feb 12 2020

A new relay-style execution technique called L2L (layer-to-layer) where at any given moment, the device memory is primarily populated only with the executing layer(s)'s footprint, which shows a new form of mixed precision for faster throughput and convergence.

71 0

Adding a benchmark result helps the community track progress.

Distributed Optimization | State-of-the-Art