An example using debezium and mysql with docker-compose

Overview

debezium-mysql

An example using debezium and mysql with docker-compose

The docker compose starts the Zookeeper, Kafka, Mysql and Debezium Connect.

After the containers getting healthy you can create a new connector, listening changes into mysql inventory database.

curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{ "name": "inventory-connector", "config": { "connector.class": "io.debezium.connector.mysql.MySqlConnector", "tasks.max": "1", "database.hostname": "mysql", "database.port": "3306", "database.user": "debezium", "database.password": "dbz", "database.server.id": "184054", "database.server.name": "dbserver1", "database.include.list": "inventory", "database.history.kafka.bootstrap.servers": "kafka:9092", "database.history.kafka.topic": "dbhistory.inventory" } }'

Therefore you can get the connectors calling

curl -H "Accept:application/json" localhost:8083/connectors/

You can also get the connectors by their names

curl -i -X GET -H "Accept:application/json" localhost:8083/connectors/inventory-connector

Instaling python library

pip install -r requirements.txt

Running the python example

python kafka-example.py

The aplication listen a topic called "dbserver1.inventory.customers"

You can see the columns, the values before and after the changes.

You might also like...
MySQL Connectivity based project. Contains various functions of a Store-Management-System

An Intermediate Level Python - MySQL Connectivity based project. Contains various functions of a Store-Management-System.

An example repository for how to generate results using PyBaMM

PyBaMM results This repository provides a template for generating results (for example, for a paper) using PyBaMM Installation Install PyBaMM using a

EFB Docker image with efb-telegram-master and efb-wechat-slave

efb-wechat-docker EFB Docker image with efb-telegram-master and efb-wechat-slave Features Container run by non-root user. Support add environment vari

A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

Streamify A data pipeline with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more! Description Objective The project will stre

The docker-based Open edX distribution designed for peace of mind
The docker-based Open edX distribution designed for peace of mind

Tutor: the docker-based Open edX distribution designed for peace of mind Tutor is a docker-based Open edX distribution, both for production and local

pydock - Docker-based environment manager for Python

pydock - Docker-based environment manager for Python ⚠️ pydock is still in beta mode, and very unstable. It is not recommended for anything serious. p

Project repository of Apache Airflow, deployed on Docker in Amazon EC2 via GitLab.

Airflow on Docker in EC2 + GitLab's CI/CD Personal project for simple data pipeline using Airflow. Airflow will be installed inside Docker container,

Like Docker, but for Squeak. You know, for kids.
Like Docker, but for Squeak. You know, for kids.

Squeaker Like Docker, but for Smalltalk images. You know, for kids. It's a small program that helps in automated derivation of configured Smalltalk im

This is a Docker-based pipeline for preparing sextractor-ready multiwavelength images

Pipeline for creating NB422-detected (ODI) catalog The repository contains a Docker-based pipeline for preprocessing observational data. The pipeline

Owner
Horácio Dias Baptista Neto
I'm a developer. I like games, Python, Java and JS.
Horácio Dias Baptista Neto
An example of Connecting a MySQL Database with Python Code

An example of Connecting And Query Data a MySQL Database with Python Code And How to install Table of contents General info Technologies Setup General

Mohammad Hosseinzadeh 1 Nov 23, 2021
A docker container (Docker Desktop) for a simple python Web app few unit tested

Short web app using Flask, tested with unittest on making massive requests, responses of the website, containerized

Omar 1 Dec 13, 2021
A framework that let's you compose websites in Python with ease!

Perry Perry <= A framework that let's you compose websites in Python with ease! Perry works similar to Qt and Flutter, allowing you to create componen

Linkus 13 Oct 9, 2022
A basic DIY-project made using Python and MySQL

Banking-Using-Python-MySQL This is a basic DIY-project made using Python and MySQL. Pre-Requisite needed:-->> MySQL command Line:- creating a database

ABHISHEK 0 Jul 3, 2022
Password manager using MySQL and Python 3.10.2

Password Manager Password manager using MySQL and Python 3.10.2 Installation Install my-project with github git clone https://github.com/AyaanSiddiq

null 1 Feb 18, 2022
Library management using python & MySQL

Library management using python & MySQL Dev/Editor: Pavan Ananth Sharma & MK Akash Introduction: This is an intermediate project which is a user-frie

Pavan Ananth Sharma 2 Jul 5, 2022
This is a menu driven Railway Reservation Project which is mainly based on the python-mysql connectivity.

Online-Railway-Reservation-System This is a menu driven Railway Reservation Project which is mainly based on the python-mysql connectivity. The projec

Ananya Gupta 1 Jan 9, 2022
Бэкапалка таблиц mysql 8 через брокер сообщений nats

nats-mysql-tables-backup Бэкап таблиц mysql 8 через брокер сообщений nats (проверено и работает в ubuntu 20.04, при наличии python 3.8) ПРИМЕРЫ: Ниже

Constantine 1 Dec 13, 2021
Registro Online (100% Python-Mysql)

Registro elettronico scritto in python, utilizzando database Mysql e Collegando Registro elettronico scritto in PHP

Sergiy Grimoldi 1 Dec 20, 2021
Simple Crud Python vs MySQL

Simple Crud Python vs MySQL The idea came when I was studying MySQ... A desire to create a python program that can give access to a "localhost" databa

Lucas 1 Jan 21, 2022