List of changes:

• lighter build of apache airflow image (removed some uused stuff)
• secured install of the keeper
  • customize password of airflow user, pg
  • autogeneration of the fernet key for metadata encryption
  • nginx entry point with self-signed ssl certificate to secure GUI
  • introduced fail2ban to avoid ssh bruteforce
• automate deploy on custom platform via make script
• automate deploy via terraform on aws for CI
• automate sql tables creation
• automate import of variables and connections
• update to the latest python version and pg
• introduced loki for centralized logging (WIP, not functional at the moment)
• introduced statsd exporter for monitoring (WIP, not functional at the moment)

In order to test dags execution and in general that the keeper will work when some changes are made i'm working on a staging keeper to test those changes, this is part of the CI stack. All changes will be executed against this new keeper

Keepers have been using my tellor-airflow image but we need to incorporate a docker image here and build it without relying on the one I pushed to docker hub months age.

This issue is to make a Dockerfile to use instead of my dockerhub image. A docker file for this can be found here: https://github.com/tellor-io/airflow

Many distribute calls fail for BAD_EXCHANGE_RATE which just means the exchange price on sushi/quickswap is no good compared to the "global" price reported by tellor. Right now we just send these txns and let them fail, wasting gas.

This issue is to add a pre-send check in the ContractInteractionOperator to check that a transaction won't fail, something like:

txn = self.function(...).buildTransaction(..)
if not confirm_success(txn):
   return False
...

where confirm_success will use web4py to verify the txn will be successful (without executing it).

This task involves converting the schedules for each DAG to use a Variable. Create variables like:

distribution-schedule-interval -> "0 * * * *"
harvest-schedule-interval -> "0 0 * * *"

Then in the DAGs:

schedule_interval = Variables.get("distribution-schedule-interval", "0 * * * *")

 dag = DAG("ricochet_stream_watch",
          max_active_runs=1,
          catchup=False,
          default_args=default_args,
          schedule_interval=schedule_interval)

secured install of the keeper: customized password of airflow user, pg autogeneration of the fernet key for metadata encryption nginx entry point with self-signed ssl certificate to secure GUI introduced fail2ban to avoid ssh bruteforce All changes can be found here: https://github.com/Ricochet-Exchange/ricochet-keeper/pull/39

This needs to be tested by configuring all variables and by creating sql tables,

• A new version of apache airflow is used
• newer python libraries are used
• permissions issue with logs folder is corrected
• airflow sources are necessarily in repo in order to build the container (should this be automated before building the container?)
• Useless grafana container is removed from the build
• docker-compose up takes considerably more time for building the complete stack

Stale price data for the rexmarkets is common when the prices aren't being updated by keepers. This task is to trigger oracle updates as part of the distribute_v2_workflow. Reference the ricochet_tellor_reporter workflow for the code to trigger oracle updates.

We need to be able to keep on multiple networks. Superfluid is deployed on Gnosis Chain and Avalanche. Getting REX setup on these networks will increase volume.

As a first step, the existing DAGs should be changed so that if there's multiple networks, then there will be multiple DAGs.

A single DAG script usually creates 1 dag, the pattern is 1 file 1 dag. However, we can use a single file to make multiple DAGs.

One solution is to add a variable networks = ['polygon', 'gnosis', 'avalanche'] and then at the top of the DAG files use:

for network in Variables.networks:
   dag = DAG("ricochet_distribute_{network}", ...)

A single DAG file will then make multiple DAGs:

ricochet_distribute_polygon
ricochet_distribute_gnsois
ricochet_distribute_avalanche

A mechanism was needed to test changes in DAG's and updates of airflow. this mechanism is described in the PR down below https://github.com/Ricochet-Exchange/ricochet-keeper/pull/39

Suggested by a Superfluid member.

Goal: trigger a notification if the price has not been updated in four hours. I'm not sure how to grab keeper status. I don't run a keeper node. I hope this can be clarified.