distributed-blockchain-based-secure-file-dedupe
-
Searching is Distributed, Block and Access List for each upload is unique and it is stored in a single VM to reduce overhead in the metadata (blockchain based).
-
Instructions for 4 nodes (including a storage node) docker swarm setup:
-
get.docker.com .. install docker on all VMs (run both commands under "This script is meant for the quick & easy installation")
-
ssh into all VMs with sudo access (or add docker usergroup)
-
Run "docker swarm init" on the leader node.
-
Run "docker swarm join-token manager", copy paste the output to all other VMs, so that all other nodes will be added to the swarm as manager.
-
To confirm all the nodes present in swarm, run "docker node ls"
-
Create a common network (attachable) among all nodes on the swarm using "docker network create --driver overlay --attachable dedupe"
-
To confirm, run "docker network ls" on all the VMs and check the scope(==swarm) and driver(==overlay)
-
Build and push the image "docker-compose build", "docker-compose push".
-
Pull the image for dedupe nodes separately "docker image pull abhishekvtangod/blockdedupe"
-
Pull the image for cloud node "docker image pull abhishekvtangod/blockdedupe_cloud"
-
Run the containers on all VMs.
- docker container run -d --name node1 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=1 -p 5001:5000 --hostname localhost abhishekvtangod/blockdedupe
- docker container run -d --name node2 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=2 -p 5002:5000 --hostname localhost abhishekvtangod/blockdedupe
- docker container run -d --name node3 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=3 -p 5003:5000 --hostname localhost abhishekvtangod/blockdedupe
- docker container run -d --name cloud --network dedupe --env PYTHONUNBUFFERED=1 -p 5000:5000 --hostname localhost abhishekvtangod/blockdedupe_cloud
-
Access all the urls from individual ips of VM with appropriate ports(change port mappings according to your ease and avoid confusions).
-
Pure Blockchain Implementation: https://github.com/lanxeon