Kafka Connect JDBC Docker Image.

Overview

kafka-connect-jdbc

This is a dockerized version of the Confluent JDBC database connector.

Usage

This image is running the connect-standalone command with a given worker.properties and one or more connector.properties files. Worker properties can be mounted to /worker.properties, while a directory of one or more connector properties can be mounted to /connector.properties.d.

Minimal Working Example

A minimal working docker-compose example of kafka-connect-jdbc (including zookeeper, kafka and postgresql) is provided in the example/ directory.

  1. Clone repository
git clone https://github.com/mhorlacher/kafka-connect-jdbc.git
  1. Navigate to the example directory
cd kafka-connect-jdbc/example
  1. Run the example via docker-compose
docker-compose up
  1. Connect to the postgresql database (with password toor)
psql -h localhost -p 5432 -U postgres -d postgresdb
  1. Confirm that messages are written to the test table
SELECT * FROM test;

Version 0.4.0

Owner
Marc Horlacher
Bioinformatics PhD Student @ TUM and ICB (Helmholtz Center). Broad interest in statistics, machine-and deep-learning, software development and biology.
Marc Horlacher
Confluent's Kafka Python Client

Confluent's Python Client for Apache KafkaTM confluent-kafka-python provides a high-level Producer, Consumer and AdminClient compatible with all Apach

Confluent Inc. 2.9k Aug 10, 2022
Class to connect to XAMPP MySQL Database

MySQL-DB-Connection-Class Class to connect to XAMPP MySQL Database Basta fazer o download o mysql_connect.py e modificar os parâmetros que quiser. E d

Alexandre Pimentel 4 Jul 12, 2021
PyRemoteSQL is a python SQL client that allows you to connect to your remote server with phpMyAdmin installed.

PyRemoteSQL Python MySQL remote client Basically this is a python SQL client that allows you to connect to your remote server with phpMyAdmin installe

ProbablyX 2 Oct 16, 2021
Reading streams of Twitter data, save them to Kafka, then process with Kafka Stream API and Spark Streaming

Using Streaming Twitter Data with Kafka and Spark Reading streams of Twitter data, publishing them to Kafka topic, process message using Kafka Stream

Rustam Zokirov 1 Dec 6, 2021
Stream-Kafka-ELK-Stack - Weather data streaming using Apache Kafka and Elastic Stack.

Streaming Data Pipeline - Kafka + ELK Stack Streaming weather data using Apache Kafka and Elastic Stack. Data source: https://openweathermap.org/api O

Felipe Demenech Vasconcelos 2 Jan 20, 2022
Python-kafka-reset-consumergroup-offset-example - Python Kafka reset consumergroup offset example

Python Kafka reset consumergroup offset example This is a simple example of how

Willi Carlsen 1 Feb 16, 2022
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

Streamify A data pipeline with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more! Description Objective The project will stre

Ankur Chavda 181 Aug 12, 2022
Connect-4-AI - AI that plays Connect-4 using the minimax algorithm

Connect-4-AI Brief overview I coded up the Connect-4 (or four-in-a-row) game in

Favour Okeke 1 Feb 15, 2022
Build and Push docker image in Python (luigi + docker-py)

Docker build images workflow in Python Since docker hub stopped building images for free accounts, I've been looking for another way to do it. I could

Fabien D. 2 Mar 18, 2022
a Scrapy spider that utilizes Postgres as a DB, Squid as a proxy server, Redis for de-duplication and Splash to render JavaScript. All in a microservices architecture utilizing Docker and Docker Compose

This is George's Scraping Project To get started cd into the theZoo file and run: chmod +x script.sh then: ./script.sh This will spin up a Postgres co

George Reyes 6 Aug 12, 2022
A docker container (Docker Desktop) for a simple python Web app few unit tested

Short web app using Flask, tested with unittest on making massive requests, responses of the website, containerized

Omar 1 Dec 13, 2021
A minimal implementation of face-detection models using flask, gunicorn, nginx, docker, and docker-compose

Face-Detection-flask-gunicorn-nginx-docker This is a simple implementation of dockerized face-detection restful-API implemented with flask, Nginx, and

Pooya-Mohammadi 24 Aug 6, 2022
Python IMDB Docker - A docker tutorial to containerize a python script.

Python_IMDB_Docker A docker tutorial to containerize a python script. Build the docker in the current directory: docker build -t python-imdb . Run the

Sarthak Babbar 1 Dec 30, 2021
Django-Docker - Django Installation Guide on Docker

Guía de instalación del Framework Django en Docker Introducción: Con esta guía p

Victor manuel torres 2 Jan 4, 2022
Baota-docker - Deploying baota panel via docker

baota-docker Deploying baota panel via docker. 通过docker一键部署宝塔面板。 一、前言 好像很多人对这个感兴

Mr. Cat 13 Jul 30, 2022
Python client for Apache Kafka

Kafka Python client Python client for the Apache Kafka distributed stream processing system. kafka-python is designed to function much like the offici

Dana Powers 4.9k Aug 12, 2022
Confluent's Kafka Python Client

Confluent's Python Client for Apache KafkaTM confluent-kafka-python provides a high-level Producer, Consumer and AdminClient compatible with all Apach

Confluent Inc. 2.9k Aug 10, 2022
Demo project for real time anomaly detection using kafka and python

kafkaml-anomaly-detection Project for real time anomaly detection using kafka and python It's assumed that zookeeper and kafka are running in the loca

Rodrigo Arenas 31 Jul 30, 2022
Store events and publish to Kafka

Create an event from Django ORM object model, store the event into the database and also publish it into Kafka cluster.

Diag 5 Jun 30, 2022
Fully reproducible, Dockerized, step-by-step, tutorial on how to mock a "real-time" Kafka data stream from a timestamped csv file. Detailed blog post published on Towards Data Science.

time-series-kafka-demo Mock stream producer for time series data using Kafka. I walk through this tutorial and others here on GitHub and on my Medium

Maria Patterson 22 Aug 6, 2022
A minimal configuration for a dockerized kafka project.

Docker Kafka Quickstart A minimal configuration for a dockerized kafka project. Usage: Run this command to build kafka and zookeeper containers, and c

Nouamane Tazi 5 Jan 12, 2022
Microservice example with Python, Faust-Streaming and Kafka (Redpanda)

Microservices Orchestration with Python, Faust-Streaming and Kafka (Redpanda) Example project for PythonBenin meetup. It demonstrates how to use Faust

Lé 3 Jun 13, 2022
A real-time financial data streaming pipeline and visualization platform using Apache Kafka, Cassandra, and Bokeh.

Realtime Financial Market Data Visualization and Analysis Introduction This repo shows my project about real-time stock data pipeline. All the code is

null 4 Jan 1, 2022
Fetching tweets and integrating it with Kafka and PySpark

KafkaPySpark Zookeeper bin/zookeeper-server-start.sh config/zookeeper.properties Kafka Server bin/kafka-server-start.sh config/server.properties Kafka

Priyansh 2 Dec 29, 2021
Bigdata - This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster

Scrapy Cluster This Scrapy project uses Redis and Kafka to create a distributed

Hanh Pham Van 0 Jan 6, 2022
X-news - Pipeline data use scrapy, kafka, spark streaming, spark ML and elasticsearch, Kibana

X-news - Pipeline data use scrapy, kafka, spark streaming, spark ML and elasticsearch, Kibana

Nguyễn Quang Huy 3 May 9, 2022
A simple healthcheck wrapper to monitor Kafka.

kafka-healthcheck A simple healthcheck wrapper to monitor Kafka. Kafka Healthcheck is a simple server that provides a singular API endpoint to determi

Rodrigo Nicolas Garcia 2 Jun 10, 2022
Event-driven-model-serving - Unified API of Apache Kafka and Google PubSub

event-driven-model-serving Unified API of Apache Kafka and Google PubSub 1. Proj

Danny Toeun Kim 4 Feb 4, 2022