Store events and publish to Kafka

Last update: Nov 30, 2022

Overview

Kafka Event

Create an event from Django ORM object model, store the event into the database and also publish it into Kafka cluster.

Setup

Install the package:

pip install django-kafka-event-log

In your project's settings.py, include the app and add credentials for Kafka:

INSTALLED_APPS = [
    ...
    'events',
]

KAFKA_GROUP = 'KAFKA_GROUP'
KAFKA_BOOTSTRAP_SERVERS = 'KAFKA_BOOTSTRAP_SERVERS'
KAFKA_SECURITY_PROTOCOL = 'KAFKA_SECURITY_PROTOCOL'
KAFKA_SASL_MECHANISMS = 'KAFKA_SASL_MECHANISMS'
KAFKA_SASL_USERNAME = 'KAFKA_SASL_USERNAME'
KAFKA_SASL_PASSWORD = 'KAFKA_SASL_PASSWORD'

Note: the credential should be read from environment variables.

Usage

This application has only 1 interface; it is PublishKafkaEventUtil. Given myapp is where the model object locates, MyModelSerializer is the data presenter, we can call the Util like this:

from events.utils import PublishKafkaEventUtil
from myapp.serializers.mymodel_serializer import MyModelSerializer

PublishKafkaEventUtil.call(
    event_name='Created',
    model_object=model_obj,
    serializer=ModelSerializer,
    metadata={'purpose': 'testing'},
)

How To Release:

This package is configured to release if the Git tag version is the same as the VERSION in setup.py. So, make sure their values are correct.

git push origin develop git push origin tag ">

git commit -m "..."
git tag <version>
git push origin develop
git push origin tag

Comments

Fix poetry install
Fix poetry install This reverts commit 0ed593eec4cc9407f8702fd634b9ed404636d675.

Ticket:

Context:

Changes:

Security impact:

Check:

[ ] I added ticket number to my branch name and commit

[ ] I did self-reviewed my PR

[ ] I added tests to cover my feature/bug fix

[ ] My PR passed the CI checks.

[ ] Adjusted the Documentation if needed.
opened by nguqtruong 1
add timestamp to kafka message
Ticket:

Context:

Changes:

add timestamp to kafka message

Security impact:

Check:

[ ] I added ticket number to my branch name and commit

[x] I did self-reviewed my PR

[x] I added tests to cover my feature/bug fix

[x] My PR passed the CI checks.

[ ] Adjusted the Documentation if needed.
opened by nguqtruong 1

Plug and play continuous integration with django and jenkins

django-jenkins Plug and play continuous integration with Django and Jenkins Installation From PyPI: $ pip install django-jenkins Or by downloading th

941 Oct 22, 2022

Django project starter on steroids: quickly create a Django app AND generate source code for data models + REST/GraphQL APIs (the generated code is auto-linted and has 100% test coverage).

Create Django App 💛 We're a Django project starter on steroids! One-line command to create a Django app with all the dependencies auto-installed AND

68 Oct 19, 2022

Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

Stream Framework Activity Streams & Newsfeeds Stream Framework is a Python library which allows you to build activity streams & newsfeeds using Cassan

4.7k Jan 2, 2023

A Django chatbot that is capable of doing math and searching Chinese poet online. Developed with django, channels, celery and redis.

Django Channels Websocket Chatbot A Django chatbot that is capable of doing math and searching Chinese poet online. Developed with django, channels, c

8 Oct 28, 2022

Forward and backwards compatibility layer for Django 1.4, 1.7, 1.8, 1.9, 1.10, and 1.11

django-compat Forward and backwards compatibility layer for Django 1.4 , 1.7 , 1.8, 1.9, 1.10 and 1.11 Consider django-compat as an experiment based o

106 Mar 28, 2022

The best way to have DRY Django forms. The app provides a tag and filter that lets you quickly render forms in a div format while providing an enormous amount of capability to configure and control the rendered HTML.

django-crispy-forms The best way to have Django DRY forms. Build programmatic reusable layouts out of components, having full control of the rendered

4.6k Jan 7, 2023

Store events and publish to Kafka

Related tags

Overview

Kafka Event

Setup

Usage

How To Release:

You might also like...

Plug and play continuous integration with django and jenkins

Django project starter on steroids: quickly create a Django app AND generate source code for data models + REST/GraphQL APIs (the generated code is auto-linted and has 100% test coverage).

Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

A Django chatbot that is capable of doing math and searching Chinese poet online. Developed with django, channels, celery and redis.

Forward and backwards compatibility layer for Django 1.4, 1.7, 1.8, 1.9, 1.10, and 1.11

The best way to have DRY Django forms. The app provides a tag and filter that lets you quickly render forms in a div format while providing an enormous amount of capability to configure and control the rendered HTML.

Location field and widget for Django. It supports Google Maps, OpenStreetMap and Mapbox

A simple Blog Using Django Framework and Used IBM Cloud Services for Text Analysis and Text to Speech

Median and percentile for Django and MongoEngine

Comments

Fix poetry install

Ticket:

Context:

Changes:

Security impact:

Check:

add timestamp to kafka message

Ticket:

Context:

Changes:

Security impact:

Check:

Owner

Diag

Getdp-project - A Django-built web app that generates a personalized banner of events to come

Simple API written in Python using FastAPI to store and retrieve Books and Authors.

Store model history and view/revert changes from admin site.

Store model history and view/revert changes from admin site.

An API was build with Django to store and retrieve information about various musical instruments.

This website serves as an online database (hosted via SQLLite) for fictional businesses in the area to store contact information (name, email, phone number, etc.) for fictional customers.

Organize Django settings into multiple files and directories. Easily override and modify settings. Use wildcards and optional settings files.

A Django web application to receive, virus check and validate transfers of digital archival records, and allow archivists to appraise and accession those records.

Django application and library for importing and exporting data with admin integration.

Management commands to help backup and restore your project database and media files