Know your customer pipeline in apache air flow

Related tags

General Utilities KYC_pipline
Overview

KYC_pipline

Know your customer pipeline in apache air flow

For a successful pipeline run take these steps:

  1. Run you Airflow server
  2. Admin -> connection -> create
  3. Triger the input_dag
  4. Before triggering the File_ process dag, move one of the JSON files into the \tmp folder (for example the request_1411.json). In the program we must give the right name of the JSON file to load.
  5. Triger the File_process dag

Input:

For our data, when I read the assignment and what I understood was that I had to create a bunch of dummy data as JSON files with random dates. This can be done with this main.py (Fourthline\Python codes\Generating data) which I turned into a Dag in Airflow as Input_dag.py.

Transformation:

These where the next steps I took for transforming my data: • Read JSON file from a file • Transform the data into an integer • Check the fraud of the request • Agent checks the request • We create the SQLite database • Fill the database with the decision • Create a decision json file with the final output

Read JSON file: ...

Check Fraud: ...

Agent Check: ...

You might also like...
This code renames subtitle file names to your video files names, so you don't need to rename them manually.

Rename Subtitle This code renames your subtitle file names to your video file names so you don't need to do it manually Note: It only works for series

Mostafa Kazemi 4 Sep 12, 2021
Protect your eyes from eye strain using this simple and beautiful, yet extensible break reminder

Protect your eyes from eye strain using this simple and beautiful, yet extensible break reminder

Gobinath 1.2k Jan 1, 2023
Display your calendar on the wallpaper.
Display your calendar on the wallpaper.

wallCal Have your calendar appear as the wallpaper. disclaimer Use at your own risk. Don't blame me if you miss a meeting :-) Some parts of the script

null 7 Jun 14, 2022
Conveniently measures the time of your loops, contexts and functions.
Conveniently measures the time of your loops, contexts and functions.

Conveniently measures the time of your loops, contexts and functions.

Maciej J Mikulski 79 Nov 15, 2022
A monitor than send discord webhook when a specific monitored product has stock in your nearby pickup stores.
A monitor than send discord webhook when a specific monitored product has stock in your nearby pickup stores.

Welcome to Apple In-store Monitor This is a monitor that are not fully scaled, and might still have some bugs.

null 5 Jun 16, 2022
jfc is an utility to make reviewing ArXiv papers for your Journal Club easier.
jfc is an utility to make reviewing ArXiv papers for your Journal Club easier.

jfc is an utility to make reviewing ArXiv papers for your Journal Club easier.

Miguel M. 3 Dec 20, 2021
Delete all of your forked repositories on Github
Delete all of your forked repositories on Github

Fork Purger Delete all of your forked repositories on Github Installation Install using pip: pip install fork-purger Exploration Under construc

Redowan Delowar 29 Dec 17, 2022
This python program will display all SSID usernames and SSID passwords you once connected to your laptop
This python program will display all SSID usernames and SSID passwords you once connected to your laptop

Windows-Wifi-password-extractor This python program will display all SSID usernames and SSID passwords you once connected to your laptop How to run th

Bhaskar Pal 3 Apr 26, 2022
Go through a random file in your favourite open source projects!

Random Source Codes Never be bored again! Staring at your screen and just scrolling the great world wide web? Would you rather read through some code

Mridul Seth 1 Nov 3, 2022
Comments
  • Refer friends to join Pi Network

    Refer friends to join Pi Network

    Pi is a new digital currency developed by Stanford PhDs, with over 35 million members worldwide. To claim your Pi, follow this link https://minepi.com/connirmm58 and use my username (connirmm58) as your invitation code.

    opened by cm58 0
Owner
saeed
Computer Engineer !
saeed
GitHub
jsoooooooon derulo - Make sure your 'jason derulo' is featured as the first part of your json data

jsonderulo Make sure your 'jason derulo' is featured as the first part of your json data Install: # python pip install jsonderulo poetry add jsonderul

jesse 3 Sep 13, 2021
A simple tool that updates your pubspec.yaml file, of a Flutter project, without altering the structure of your file.

A simple tool that updates your pubspec.yaml file, of a Flutter project, without altering the structure of your file.

null 3 Dec 10, 2021
A python package for your Kali Linux distro that find the fastest mirror and configure your apt to use that mirror

Kali Mirror Finder Using Single Python File A python package for your Kali Linux distro that find the fastest mirror and configure your apt to use tha

MrSingh 6 Dec 12, 2022
Make your functions return something meaningful, typed, and safe!

Make your functions return something meaningful, typed, and safe! Features Brings functional programming to Python land Provides a bunch of primitives

dry-python 2.6k Jan 9, 2023
Quickly edit your slack posts.

Lightning Edit Quickly edit your Slack posts. Heavily inspired by @KhushrajRathod's LightningDelete. Usage: Note: Before anything, be sure to head ove

null 14 Nov 19, 2021
Manage your exceptions in Python like a PRO

A linter to manage all your python exceptions and try/except blocks (limited only for those who like dinosaurs).

Guilherme Latrova 353 Dec 31, 2022
Set of utilities for exporting/controlling your robot in Blender

Blender Robotics Utils This repository contains utilities for exporting/controlling your robot in Blender Maintainers This repository is maintained by

Robotology 33 Nov 30, 2022
This utility lets you draw using your laptop's touchpad on Linux.

FingerPaint This utility lets you draw using your laptop's touchpad on Linux. Pressing any key or clicking the touchpad will finish the drawing

Wazzaps 95 Dec 17, 2022
A small utility that sorts your files.

FileSorter A small utility that sorts your files. TODO: Scan directory to find files(thanks @corruptmemry for this!) Split extensions to determine fil

null 2 Jun 16, 2022
Give you a better view of your Docker registry disk usage.

registry-du Give you a better view of your Docker registry disk usage. This small tool will analysis your Docker registry(vanilla or Harbor both work)

Nova Kwok 16 Jan 7, 2023