FLEX (Federated Learning EXchange,FLEX) protocol is a set of standardized federal learning agreements designed by Tongdun AI Research Group。

Overview

FLEX logo

Click to view Chinese version

FLEX (Federated Learning Exchange) protocol is a set of standardized federal learning agreements designed by Tongdun AI Research Group. FLEX protocol sets the sequence of data exchange during the federal learning process between participants and methods for data encryption and decryption used before and after the exchange. Abiding by these agreements, participants can safely join the federation for providing data or federated services.

FLEX protocol consists of two parts:

  1. Application protocols: This part of protocol is designed for federated algorithms, especially for supporting the multi-party federated learning algorithms. The protocol sets the sequence of data exchange between multiple parties and the corresponding cryptographic algorithm. The communication part, in federation process, is also encapsulated here.
  2. Public components: This part is about the basic application algorithms and security agreement being dependent on upper-level application protocol, such as homomorphic encryption and secret sharing algorithm, etc.

FLEX protocol

FLEX protocol overview

This project realizes these two parts, mentioned by the FLEX white paper. As for the communication part, we use the Ionic Bond protocol interface, developed by Tongdun AI Research Group, as the practice interface. It only gives you a simple implementation as a reference.

Installation tutorial

FLEX protocol can be run directly with the source code. It supports Python 3.6 or any higher version and is available for environment variable settings.

export PYTHONPATH="/path/to/flex"

Firstly, install the basic dependent libraries, taking the Ubuntu system as an example:

apt install libgmp-dev, libmpfr-dev, libmpc-dev
pip install numpy, gmpy2, pycryptodome, scikit_learn, py_ecc, pandas

Through the tools provided in FLEX is also available. Run it through source directory with:

pip install .

After installation, the protocol can be imported by:

from flex.api import *

Test

FLEX provides basic test program, aiming to check the protocols during the running process. Generally, users need to install the FLEX protocol on three machines, playing the roles of Coordinator, Guest, and Host respectively. Before running the test program, users need to modify the “federal_info” according to their actual hostname or ip. There is also a stand-alone mode for users to simulate the whole process on one machine. See the test_intro for more details.

API and documentation

FLEX uses a unified API to import the upper-layer part of the protocol, while the typical process is getting the instance through “make_protocol”, and executing it by using “exchange”. Taking security aggregation as an example:

from flex.api import make_protocol
from flex.constants import OTP_SA_FT

# initialization
protocol = make_protocol(OTP_SA_FT, federal_info, sec_param, algo_param)
# excute
protocol.exchange(theta)

In the example, federal_info means the federal participant information; sec_param means the security parameter of the protocol, which specifies the cryptographic method and key length used in the protocol; algo_param means the algorithm hyperparameter, which can also be empty; Theta is the input of the protocol. See the api_intro for more details of parameter description and usage instructions. About public components, it can be imported from “flex.crypto” in module API. Taking Paillier homomorphic encryption algorithm as an example:

from flex.crypto.paillier.api import generate_paillier_encryptor_decryptor

# Generate Encryptor and Decryptor
pe, pd = generate_paillier_encryptor_decryptor(n_length = 2048)
# encryption
en_x = pe.encrypt(x)
en_y = pe.encrypt(y)
# sum
en_z = en_x + en_y
# decrypt
z = pd.decrypt(en_z)

See the crypto_intro for more details of public components.

Issues
  • 白皮书第22页表3.2.1 FMC协议流程

    白皮书第22页表3.2.1 FMC协议流程

    此处的“根据接收到的随机矩阵生成碎- 片”的具体实现是什么? 根据表中具体步骤,有

    X1碎片T X2 + U1 + U2 - R1T X2碎片 = X1T X2
    

    如何得到上述结果?R2在计算中未出现,其作用是什么?

    opened by lidh15 1
  • Update protocol_intro.md

    Update protocol_intro.md

    fix link bug

    opened by TDmengdan 0
Releases(V1.0)
Owner
同盾科技
同盾科技致力于解决银行,互联网金融,三方支付等行业的网络风险和欺诈问题,包括账号欺诈风险,交易欺诈风险,支付欺诈风险,商户欺诈风险,网络信用欺诈风险及企业内部欺诈风险等领域。
同盾科技
Python code for "Machine learning: a probabilistic perspective" (2nd edition)

Python code for "Machine learning: a probabilistic perspective" (2nd edition)

Probabilistic machine learning 4.2k Nov 26, 2021
PyTorch implementation of Federated Learning with Non-IID Data, and federated learning algorithms, including FedAvg, FedProx.

Federated Learning with Non-IID Data This is an implementation of the following paper: Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, Vik

Youngjoon Lee 4 Nov 20, 2021
Template for a Dataflow Flex Template in Python

Dataflow Flex Template in Python This repository contains a template for a Dataflow Flex Template written in Python that can easily be used to build D

STOIX 4 Nov 8, 2021
Automatically download the cwru data set, and then divide it into training data set and test data set

Automatically download the cwru data set, and then divide it into training data set and test data set.自动下载cwru数据集,然后分训练数据集和测试数据集

null 2 Nov 25, 2021
Dados Públicos de CNPJ disponibilizados pela Receita Federal do Brasil

Dados Públicos CNPJ Fonte oficial da Receita Federal do Brasil, aqui. Layout dos arquivos, aqui. A Receita Federal do Brasil disponibiliza bases com o

Aphonso Henrique do Amaral Rafael 53 Nov 11, 2021
Script em python para carregar os arquivos de cnpj dos dados públicos da Receita Federal em MYSQL.

cnpj-mysql Script em python para carregar os arquivos de cnpj dos dados públicos da Receita Federal em MYSQL. Dados públicos de cnpj no site da Receit

null 2 Nov 19, 2021
Consulta de CPF e CNPJ na Receita Federal com Web-Scraping

Repositório contendo scripts Python que realizam a consulta de CPF e CNPJ diretamente no site da Receita Federal.

Josué Campos 5 Nov 29, 2021
Plato: A New Framework for Federated Learning Research

a new software framework to facilitate scalable federated learning research.

System Group@Theory Lab 123 Nov 29, 2021
A Research-oriented Federated Learning Library and Benchmark Platform for Graph Neural Networks. Accepted to ICLR'2021 - DPML and MLSys'21 - GNNSys workshops.

FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks A Research-oriented Federated Learning Library and Benchmark Platform

FedML-AI 110 Dec 2, 2021
This python code will get requests from SET (The Stock Exchange of Thailand) a previously-close stock price and return it in Thai Baht currency using beautiful soup 4 HTML scrapper.

This python code will get requests from SET (The Stock Exchange of Thailand) a previously-close stock price and return it in Thai Baht currency using beautiful soup 4 HTML scrapper.

Andre 1 Dec 1, 2021
wikirepo is a Python package that provides a framework to easily source and leverage standardized Wikidata information

Python based Wikidata framework for easy dataframe extraction wikirepo is a Python package that provides a framework to easily source and leverage sta

Andrew Tavis McAllister 17 Nov 26, 2021
A logical, reasonably standardized, but flexible project structure for doing and sharing data science work.

Cookiecutter Data Science A logical, reasonably standardized, but flexible project structure for doing and sharing data science work. Project homepage

Jon C Cline 0 Sep 5, 2021
Code for paper: Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks

Group-CAM By Zhang, Qinglong and Rao, Lu and Yang, Yubin [State Key Laboratory for Novel Software Technology at Nanjing University] This repo is the o

zhql 68 Nov 23, 2021
A Python Oriented tool to Scrap WhatsApp Group Link using Google Dork it Scraps Whatsapp Group Links From Google Results And Gives Working Links.

WaGpScraper A Python Oriented tool to Scrap WhatsApp Group Link using Google Dork it Scraps Whatsapp Group Links From Google Results And Gives Working

Muhammed Rizad 12 Nov 22, 2021
Ningyu Jia(nj2459)/Mengyin Ma(mm5937) Call Analysis group project(Group 36)

Group and Section Group 36 Section 001 name and UNI Name UNI Ningyu Jia nj2459 Mengyin Ma mm5937 code explanation Parking.py (1) Calculate the rate of

null 1 Dec 2, 2021
Multiple paper open-source codes of the Microsoft Research Asia DKI group

?? Paper Code Collection (MSRA DKI Group) This repo hosts multiple open-source codes of the Microsoft Research Asia DKI Group. You could find the corr

Microsoft 162 Nov 22, 2021
scikit-learn addon to operate on set/"group"-based features

skl-groups skl-groups is a package to perform machine learning on sets (or "groups") of features in Python. It extends the scikit-learn library with s

Danica J. Sutherland 40 Jun 2, 2020
FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.

Detectron is deprecated. Please see detectron2, a ground-up rewrite of Detectron in PyTorch. Detectron Detectron is Facebook AI Research's software sy

Facebook Research 24.8k Dec 2, 2021
A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

MMF is a modular framework for vision and language multimodal research from Facebook AI Research. MMF contains reference implementations of state-of-t

Facebook Research 4.7k Dec 2, 2021
A telegram bot providing recon and research functions for bug bounty research

Bug Bounty Bot A telegram bot with commands to simplify bug bounty tasks Installation Use Road Map Installation BugBountyBot is open-source so you can

Tyler Butler 1 Oct 23, 2021
A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

MMF is a modular framework for vision and language multimodal research from Facebook AI Research. MMF contains reference implementations of state-of-t

Facebook Research 4.7k Dec 2, 2021
This repository contains the implementations related to the experiments of a set of publicly available datasets that are used in the time series forecasting research space.

TSForecasting This repository contains the implementations related to the experiments of a set of publicly available datasets that are used in the tim

Rakshitha Godahewa 37 Nov 21, 2021
The Social-Engineer Toolkit (SET) repository from TrustedSec - All new versions of SET will be deployed here.

?? The Social-Engineer Toolkit (SET) ?? Copyright 2020 The Social-Engineer Toolkit (SET) Written by: David Kennedy (ReL1K) @HackingDave Company: Trust

trustedsec 7.1k Dec 1, 2021
This is our ARTS test set, an enriched test set to probe Aspect Robustness of ABSA.

This is the repository for our 2020 paper "Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis". Data We provide

null 31 Oct 26, 2021
Open-Set Recognition: A Good Closed-Set Classifier is All You Need

Open-Set Recognition: A Good Closed-Set Classifier is All You Need Code for our paper: "Open-Set Recognition: A Good Closed-Set Classifier is All You

null 58 Nov 24, 2021
Script that receives an Image (original) and a set of images to be used as "pixels" in reconstruction of the Original image using the set of images as "pixels"

picinpics Script that receives an Image (original) and a set of images to be used as "pixels" in reconstruction of the Original image using the set of

RodrigoCMoraes 1 Oct 24, 2021
The leading native Python SSHv2 protocol library.

Paramiko Paramiko: Python SSH module Copyright: Copyright (c) 2009 Robey Pointer <[email protected]> Copyright: Copyright (c) 2020 Jeff Forcier <

null 7.3k Nov 24, 2021
A python module to parse the Open Graph Protocol

OpenGraph is a module of python for parsing the Open Graph Protocol, you can read more about the specification at http://ogp.me/ Installation $ pip in

Erik Rivera 206 Nov 22, 2021