α-Indirect Control in Onion-like Networks

Overview

α-Indirect Control in Onion-like Networks

We propose a fast, accurate, and scalable algorithm to detect ultimate controlling entities in global corporate networks. α-ICON uses company-participant links to identify super-holders who exert control in networks with millions of nodes.

By exploiting onion-like properties of such networks we iteratively peel off the hanging vertices until a dense core remains. This procedure allows for a dramatic speed-up, uncovers meaningful structures, and handles circular ownership by design.

Read our paper with the applications. As a toy example, consider the below corporate network where α-ICON designates Mr Philip Mactaggart (in green) as the super-holder exerting control over all other entities, directly or indirectly held:

Installation

To replicate the analysis you need to clone this repository to your local machine. Then you need to install the required versions of R dependencies listed in DEPENDENCIES. code/helper_functions/install_dependencies.r automates this step, but you may still need to install the underlying libraries manually with Homebrew or apt-get, depending on your platform. Finally, you need to declare the environment variable ALPHAICON_PATH in bash pointing to the repository. Or, better yet, you can add it in your .Renviron with

> ~/.Renviron ">
user:~$ echo 'ALPHAICON_PATH="path_to_cloned_repository"' >> ~/.Renviron

The repository does not contain any data due to its size (10+ GB unpacked); most files in data/ and output/ folders are zero-byte placeholders. We provide a public Google Drive folder with the populated data/ and output/ directories. You may still need to unzip them manually.

A self-contained example of α-ICON is also available in Google Colaboratory.

Repository structure

data/
├─uk/ # Data on UK companies and participants
| ├ persons-with-significant-control-snapshot-2021-08-02.txt # Source PSC data
| ├ BasicCompanyDataAsOneFile-2021-08-01.csv # Source data on live companies in UK
| ├ sic_2007_code_list.csv # Standard Industrial Classification codes
| ├ psc_snapshot_2021-08-02.rdata # Processed People with Significant Control data
| └ uk_basic_companies_data_2021-08-01.rdata # Processed Basic Company data
|
├─corpwatch_api_tables_csv_14aug21/ # Data from CorpWatch Dump 
| ├ company_info.csv # Source companies data from SEC filings
| ├ cik_name_lookup.csv # Company name variants in SEC filings
| └ company_locations.csv # Company locations in SEC filings
|
code/
├─helper_functions/
| ├ install_dependencies.r # Installs R dependencies used in the project 
| └ compute_power_index.r # Computes Mizuno et al. (2020) DPI and NPI
|
├─data_preparation/
| └─uk/
|   ├ 1a_process_psc_snapshot.r # Prepare source PSC data
|   ├ 1b_process_companies_data.r # Prepare source data on live companies
|   ├ 2_psc_snapshot_to_participants_panel.r # PSC data to entity-participant info
|   └ 3_prepare_affiliated_entities_evaluation_data.r # Process CorpWatch data
|
├─alphaicon_paper/
| ├ 1_compute_alphaicon.ipynb # Jupyter Notebook w. α-ICON (also on Google Colab)
| ├ 2_compute_npi_dpi.r # Computation of Direct and Network Power Indices
| ├ 3_summary_stat_by_node_type.r # UK PSC network statistics by core/SH/ST/I
| ├ 4_illustrate_algorithm.r # Visualise selected networks
| ├ 5_algorithm_evaluation.r # Compute recall @ k and l for various algorithms
| └ 6_rank_top_holders.r # Examine the rankings of super-holders & Kendall's tau
|
output/
├─uk/
| ├ uk_organisations_participants_2021_long_2aug21.csv # Primary ownership data
| ├ uk_organisations_participation_graph_core_periphery_membership_6aug21.csv
| ├─npi_dpi/ # Mizuno et al. (2020) computation results on UK PSC data
| | └─10000iter/
| |   ├ uk_organisations_participants_2021_long_7sep21_dpi_10000iter.csv # DPI
| |   └ uk_organisations_participants_2021_long_7sep21_npi_10000iter.csv # NPI
| |
| ├─transitive/ # Computed α-ICON shares on equity shares or DPI weights
| | ├ uk_organisations_transitive_ownership_alpha*_2021_long_2aug21.csv # α = *
| | └ uk_organisations_transitive_ownership_alpha*_2021_long_7sep21_dpi_....csv
| |
└─alphaicon_paper/
  ├ uk_orgs_algorithm_evaluation_recall.csv # Algorithm recall by k
  ├ uk_orgs_algorithm_evaluation_recall_by_pathlength.csv # Algorithm recall by l
  ├ uk_organisations_top100_holders_2021_long_2aug21.csv # Top SH in PSC network
  ├ uk_organisations_top100_holders_diff_npi_dpi_2021_long_2aug21.csv # Top-100 SH
  |                          # with the largest difference betw. total DPI and NPI
  ├ uk_organisations_top100_holders_diff_transitive_dpi_2021_long_2aug21.csv
  |  # Top-100 SH with the largest difference betw. total DPI and α-ICON (α=0.999)
  ├ uk_organisations_top100_holders_diff_transitive_npi_2021_long_2aug21.csv
  |  # Top-100 SH with the largest difference betw. total NPI and α-ICON (α=0.999)
  └ network_examples/ # Visualisations of selected networks

We provide an annotated Makefile that documents the data analysis in our papers.

To build the ‘α-Indirect Control in Onion-like Networks’ paper run make alphaicon_paper when in the repository folder.

Please note that those commands will not produce any publication-ready output files (e.g. tables or figures): the export statements are commented out in the code. Our intention is to make the analysis pipeline transparent to the readers with the aid of make:

alphaicon_dependencies

Licence

Creative Commons License
Creative Commons License Attribution 4.0 International (CC BY 4.0).

Copyright © the respective contributors, as shown by the AUTHORS file.

People with Significant Control data is distributed by Companies House under Open Government Licence v3.0.

Free Company Data Product is distributed by Companies House under Open Government Licence v3.0.

Contacts

Dmitriy Skougarevskiy, Ph.D.

[email protected]

You might also like...
Python3 program to control Elgato Ring Light on your local network without Elgato's Control Center software

Elgato Light Controller I'm really happy with my Elgato Key Light from an illumination perspective. However, their control software has been glitchy f

A python API act as Control Center to control your Clevo Laptop via wmi on windows.

ClevoPyControlCenter A python API act as Control Center to control your Clevo Laptop via wmi on windows. Usage # pip3 install pymi from clevo_wmi impo

Python implementation of ZMP Preview Control approach for biped robot control.
Python implementation of ZMP Preview Control approach for biped robot control.

ZMP Preview Control This is the Python implementation of ZMP Preview Control app

ROS-UGV-Control-Interface - Control interface which can be used in any UGV
ROS-UGV-Control-Interface - Control interface which can be used in any UGV

ROS-UGV-Control-Interface Cam Closed: Cam Opened:

Hand Gesture Volume Control is AIML based project which uses image processing to control the volume of your Computer.
Hand Gesture Volume Control is AIML based project which uses image processing to control the volume of your Computer.

Hand Gesture Volume Control Modules There are basically three modules Handtracking Program Handtracking Module Volume Control Program Handtracking Pro

TurtleBot Control App - TurtleBot Control App With Python
TurtleBot Control App - TurtleBot Control App With Python

TURTLEBOT CONTROL APP INDEX: 1. Introduction 2. Environments 2.1. Simulated Envi

Inverted-pendulum-with-fuzzy-control - Inverted pendulum with fuzzy control

Fuzzy Inverted Pendulum Basically, this project consists of an inverted pendulum

PowerGym is a Gym-like environment for Volt-Var control in power distribution systems.

Overview PowerGym is a Gym-like environment for Volt-Var control in power distribution systems. The Volt-Var control targets minimizing voltage violat

🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

Boltons boltons should be builtins. Boltons is a set of over 230 BSD-licensed, pure-Python utilities in the same spirit as — and yet conspicuously mis

🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

Boltons boltons should be builtins. Boltons is a set of over 230 BSD-licensed, pure-Python utilities in the same spirit as — and yet conspicuously mis

You like pytorch? You like micrograd? You love tinygrad! ❤️
You like pytorch? You like micrograd? You love tinygrad! ❤️

For something in between a pytorch and a karpathy/micrograd This may not be the best deep learning framework, but it is a deep learning framework. Due

🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

Boltons boltons should be builtins. Boltons is a set of over 230 BSD-licensed, pure-Python utilities in the same spirit as — and yet conspicuously mis

A C-like hardware description language (HDL) adding high level synthesis(HLS)-like automatic pipelining as a language construct/compiler feature.
A C-like hardware description language (HDL) adding high level synthesis(HLS)-like automatic pipelining as a language construct/compiler feature.

██████╗ ██╗██████╗ ███████╗██╗ ██╗███╗ ██╗███████╗ ██████╗ ██╔══██╗██║██╔══██╗██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝ ██████╔╝██║██████╔╝█

 Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks (MAPDN)
Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks (MAPDN)

Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks (MAPDN) This is the implementation of the paper Multi-Age

Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks

Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks This repository contains the code and data for the corresp

Draw like Bob Ross using the power of Neural Networks (With PyTorch)!
Draw like Bob Ross using the power of Neural Networks (With PyTorch)!

Draw like Bob Ross using the power of Neural Networks! (+ Pytorch) Learning Process Visualization Getting started Install dependecies Requires python3

A Pytorch Implementation of Domain adaptation of object detector using scissor-like networks

A Pytorch Implementation of Domain adaptation of object detector using scissor-like networks Please follow Faster R-CNN and DAF to complete the enviro

OSMnx: Python for street networks. Retrieve, model, analyze, and visualize street networks and other spatial data from OpenStreetMap.

OSMnx OSMnx is a Python package that lets you download geospatial data from OpenStreetMap and model, project, visualize, and analyze real-world street

Comments
  • Switch to an open source software license

    Switch to an open source software license

    Could you please re-license this project under an open source software license, such as MIT or GPL?

    Commercial entities will have a hard time using this code in production with the CC BY 4.0 license, which is not intended for software.

    opened by eligrey 1
Owner
European University at St. Petersburg (EUSP)
Russia's Independent Not-For-Profit University Founded in 1994. Social Sciences and Humanities Graduate Degrees & Programs in Russian and English.
European University at St. Petersburg (EUSP)
whm also known as wifi-heat-mapper is a Python library for benchmarking Wi-Fi networks and gather useful metrics that can be converted into meaningful easy-to-understand heatmaps.

whm also known as wifi-heat-mapper is a Python library for benchmarking Wi-Fi networks and gather useful metrics that can be converted into meaningful easy-to-understand heatmaps.

Nischay Mamidi 128 Dec 16, 2022
A python module to create random networks using network models

networkgen A python module to create random networks using network models Usage $

Serkan Demirci 3 Sep 26, 2022
Handy Tool to check the availability of onion site and to extract the title of submitted onion links.

This tool helps is to quickly investigate a huge set of onion sites based by checking its availability which helps to filter out the inactive sites and collect the site title that might helps us to categories what site we are handling.

Balaji 13 Nov 25, 2022
Code for "Modeling Indirect Illumination for Inverse Rendering", CVPR 2022

Modeling Indirect Illumination for Inverse Rendering Project Page | Paper | Data Preparation Set up the python environment conda create -n invrender p

ZJU3DV 116 Jan 3, 2023
The Devils Eye is an OSINT tool that searches the Darkweb for onion links and descriptions that match with the users query without requiring the use for Tor.

The Devil's Eye searches the darkweb for information relating to the user's query and returns the results including .onion links and their description

Richard Mwewa 135 Dec 31, 2022
This repository holds code and data for our PETS'22 article 'From "Onion Not Found" to Guard Discovery'.

From "Onion Not Found" to Guard Discovery (PETS'22) This repository holds the code and data for our PETS'22 paper titled 'From "Onion Not Found" to Gu

Lennart Oldenburg 3 May 4, 2022
dos-atack-tor script de python que permite usar conexiones cebollas para atacar paginas .onion o paginas convencionales via tor.

script de python que permite usar conexiones cebollas para atacar paginas .onion o paginas convencionales via tor. tiene capacidad de ajustar la cantidad de informacion a enviar, el numero de hilos a usar, el tiempo de duracion del ataque, atacar a un directorio en especial y etc. Su ataque es sencillo, trata de saturar un recurso en base a peticiones GET en rutas del servidor excesivamente lentas.

Desmon 2 Jun 1, 2022
This script allows you to make a onion host instantly.

Installation It only works in Debian based Linux distros. Clone the repo: git clone https://github.com/0xStevenson/Auto-Tor-Host.git Go to the direct

Steven 4 Feb 22, 2022
The ABR Control library is a python package for the control and path planning of robotic arms in real or simulated environments.

The ABR Control library is a python package for the control and path planning of robotic arms in real or simulated environments. ABR Control provides API's for the Mujoco, CoppeliaSim (formerly known as VREP), and Pygame simulation environments, and arm configuration files for one, two, and three-joint models, as well as the UR5 and Kinova Jaco 2 arms. Users can also easily extend the package to run with custom arm configurations. ABR Control auto-generates efficient C code for generating the control signals, or uses Mujoco's internal functions to carry out the calculations.

Applied Brain Research 277 Jan 5, 2023
Control System Packer is a lightweight, low-level program to transform energy equations into the compact libraries for control systems.

Control System Packer is a lightweight, low-level program to transform energy equations into the compact libraries for control systems. Packer supports Python ?? , C ?? and C++ ?? libraries.

mirnanoukari 31 Sep 15, 2022