Updates redisearch instance with igdb data used for kimosabe

Related tags

Django python redis asyncio
Overview

igdb-pdt

Update RediSearch with IGDB games data in the following Format:

{
  "game_slug": {
    "name": "game_name",
    "cover": "igdb_coverart_url",
    "thumb": "igdb_thumbart_url",
    "summary": "summary text",
    "alt_names_%i": "alt_name_i"
  }
}

The only games that are pulled in by this script have the following properties:

  • lan, offline, or online co-op
  • category of 0 (base game) or 9 (remaster)

Note: Only the name, alt_names, and summary are used as fields that impact the search, with weights 10, 10, and 1 respectively.

Usage

CLIENT_ID=client_id; CLIENT_SECRET=client_secret; ./populate_db.py

Flags

--mock: gets data from data.json file instead of IGDB.
--persist: save to RediSearch.
--output: display data from IGDB to console. (used for creating mock data.json via ./populate_db.py --output > data.json)

Environment Variables

CLIENT_ID: Client ID for IGDB. Required if not using mock file.
CLIENT_SECRET: Client Secret for IGDB. Required if not using mock file.
REDIS_HOSTNAME: Hostname for Redis instance. Required.
REDIS_PORT: Port of Redis. Defaults to 6379.

Docker image

Runs script every day at 2:30 AM via cronjob.

TODO

  • add support for caching authorization token, and only refreshing if expired, otherwise keep it
  • seperate logging and output
Comments
  • [Snyk] Security upgrade python from 3.8.8-slim to 3.8.10-slim

    [Snyk] Security upgrade python from 3.8.8-slim to 3.8.10-slim

    Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.

    Changes included in this PR

    • Dockerfile

    We recommend upgrading to python:3.8.10-slim, as this image has only 68 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.

    Some of the most important vulnerabilities in your base image include:

    | Severity | Priority Score / 1000 | Issue | Exploit Maturity | | :------: | :-------------------- | :---- | :--------------- | | high severity | 714 | Information Exposure
    SNYK-DEBIAN10-LIBGCRYPT20-1297893 | No Known Exploit | | high severity | 500 | Privilege Chaining
    SNYK-DEBIAN10-SYSTEMD-345386 | No Known Exploit | | high severity | 500 | Privilege Chaining
    SNYK-DEBIAN10-SYSTEMD-345386 | No Known Exploit | | high severity | 500 | Incorrect Privilege Assignment
    SNYK-DEBIAN10-SYSTEMD-345391 | No Known Exploit | | high severity | 500 | Incorrect Privilege Assignment
    SNYK-DEBIAN10-SYSTEMD-345391 | No Known Exploit |

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    opened by snyk-bot 1
  • [Snyk] Security upgrade python from 3.8.8-slim to 3.9.5-slim

    [Snyk] Security upgrade python from 3.8.8-slim to 3.9.5-slim

    Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.

    Changes included in this PR

    • Dockerfile

    We recommend upgrading to python:3.9.5-slim, as this image has only 68 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.

    Some of the most important vulnerabilities in your base image include:

    | Severity | Priority Score / 1000 | Issue | Exploit Maturity | | :------: | :-------------------- | :---- | :--------------- | | high severity | 714 | Information Exposure
    SNYK-DEBIAN10-LIBGCRYPT20-1297893 | No Known Exploit | | high severity | 500 | Privilege Chaining
    SNYK-DEBIAN10-SYSTEMD-345386 | No Known Exploit | | high severity | 500 | Privilege Chaining
    SNYK-DEBIAN10-SYSTEMD-345386 | No Known Exploit | | high severity | 500 | Incorrect Privilege Assignment
    SNYK-DEBIAN10-SYSTEMD-345391 | No Known Exploit | | high severity | 500 | Incorrect Privilege Assignment
    SNYK-DEBIAN10-SYSTEMD-345391 | No Known Exploit |

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    opened by snyk-bot 0
  • [Snyk] Security upgrade python from 3.8.8-slim to 3-slim

    [Snyk] Security upgrade python from 3.8.8-slim to 3-slim

    Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.

    Changes included in this PR

    • Dockerfile

    We recommend upgrading to python:3-slim, as this image has only 68 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.

    Some of the most important vulnerabilities in your base image include:

    | Severity | Priority Score / 1000 | Issue | Exploit Maturity | | :------: | :-------------------- | :---- | :--------------- | | high severity | 714 | Information Exposure
    SNYK-DEBIAN10-LIBGCRYPT20-1297893 | No Known Exploit | | high severity | 500 | Privilege Chaining
    SNYK-DEBIAN10-SYSTEMD-345386 | No Known Exploit | | high severity | 500 | Privilege Chaining
    SNYK-DEBIAN10-SYSTEMD-345386 | No Known Exploit | | high severity | 500 | Incorrect Privilege Assignment
    SNYK-DEBIAN10-SYSTEMD-345391 | No Known Exploit | | high severity | 500 | Incorrect Privilege Assignment
    SNYK-DEBIAN10-SYSTEMD-345391 | No Known Exploit |

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    opened by snyk-bot 0
  • [Snyk] Security upgrade python from 3.8.5-slim to 3.8.8-slim

    [Snyk] Security upgrade python from 3.8.5-slim to 3.8.8-slim

    Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.

    Changes included in this PR

    • Dockerfile

    We recommend upgrading to python:3.8.8-slim, as this image has only 64 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.

    Some of the most important vulnerabilities in your base image include:

    | Severity | Priority Score / 1000 | Issue | Exploit Maturity | | :------: | :-------------------- | :---- | :--------------- | | high severity | 571 | Use After Free
    SNYK-DEBIAN10-GNUTLS28-1085094 | No Known Exploit | | high severity | 571 | Use After Free
    SNYK-DEBIAN10-GNUTLS28-1085097 | No Known Exploit | | high severity | 571 | Incorrect Default Permissions
    SNYK-DEBIAN10-LIBZSTD-1080893 | No Known Exploit | | high severity | 571 | Incorrect Default Permissions
    SNYK-DEBIAN10-LIBZSTD-1080899 | No Known Exploit | | high severity | 714 | Integer Overflow or Wraparound
    SNYK-DEBIAN10-OPENSSL-1075326 | No Known Exploit |

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: ?? View latest project report

    🛠 Adjust project settings

    opened by snyk-bot 0
  • [Snyk] Security upgrade aiohttp from 3.6.2 to 3.7.4

    [Snyk] Security upgrade aiohttp from 3.6.2 to 3.7.4

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- medium severity | 435/1000
    Why? Has a fix available, CVSS 4.2 | Open Redirect
    SNYK-PYTHON-AIOHTTP-1079232 | aiohttp:
    3.6.2 -> 3.7.4
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
  • [Snyk] Security upgrade python from 3.9.5 to 3.12.0a3

    [Snyk] Security upgrade python from 3.9.5 to 3.12.0a3

    Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.

    Changes included in this PR

    • Dockerfile

    We recommend upgrading to python:3.12.0a3, as this image has only 272 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.

    Some of the most important vulnerabilities in your base image include:

    | Severity | Priority Score / 1000 | Issue | Exploit Maturity | | :------: | :-------------------- | :---- | :--------------- | | critical severity | 714 | Directory Traversal
    SNYK-DEBIAN10-DPKG-2847944 | No Known Exploit | | critical severity | 714 | Buffer Overflow
    SNYK-DEBIAN10-OPENSSL-1569403 | No Known Exploit | | critical severity | 714 | OS Command Injection
    SNYK-DEBIAN10-OPENSSL-2807585 | No Known Exploit | | critical severity | 714 | OS Command Injection
    SNYK-DEBIAN10-OPENSSL-2933515 | No Known Exploit | | critical severity | 714 | Integer Overflow or Wraparound
    SNYK-DEBIAN10-PYTHON37-3090928 | No Known Exploit |

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

    opened by snyk-bot 0
  • [Snyk] Security upgrade python from 3.9.5 to 3.9.15

    [Snyk] Security upgrade python from 3.9.5 to 3.9.15

    Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.

    Changes included in this PR

    • Dockerfile

    We recommend upgrading to python:3.9.15, as this image has only 295 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.

    Some of the most important vulnerabilities in your base image include:

    | Severity | Priority Score / 1000 | Issue | Exploit Maturity | | :------: | :-------------------- | :---- | :--------------- | | critical severity | 714 | Directory Traversal
    SNYK-DEBIAN10-DPKG-2847944 | No Known Exploit | | critical severity | 714 | Buffer Overflow
    SNYK-DEBIAN10-OPENSSL-1569403 | No Known Exploit | | critical severity | 714 | OS Command Injection
    SNYK-DEBIAN10-OPENSSL-2807585 | No Known Exploit | | critical severity | 714 | OS Command Injection
    SNYK-DEBIAN10-OPENSSL-2933515 | No Known Exploit | | high severity | 614 | Improper Input Validation
    SNYK-DEBIAN10-XZUTILS-2444279 | No Known Exploit |

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

    opened by snyk-bot 0
  • [Snyk] Security upgrade python from 3.9.5 to 3.10.7

    [Snyk] Security upgrade python from 3.9.5 to 3.10.7

    Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.

    Changes included in this PR

    • Dockerfile

    We recommend upgrading to python:3.10.7, as this image has only 281 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.

    Some of the most important vulnerabilities in your base image include:

    | Severity | Priority Score / 1000 | Issue | Exploit Maturity | | :------: | :-------------------- | :---- | :--------------- | | critical severity | 714 | Directory Traversal
    SNYK-DEBIAN10-DPKG-2847944 | No Known Exploit | | critical severity | 714 | Buffer Overflow
    SNYK-DEBIAN10-OPENSSL-1569403 | No Known Exploit | | critical severity | 714 | OS Command Injection
    SNYK-DEBIAN10-OPENSSL-2807585 | No Known Exploit | | critical severity | 714 | OS Command Injection
    SNYK-DEBIAN10-OPENSSL-2933515 | No Known Exploit | | high severity | 614 | Improper Input Validation
    SNYK-DEBIAN10-XZUTILS-2444279 | No Known Exploit |

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

    opened by snyk-bot 0
  • [Snyk] Security upgrade aiohttp from 3.7.4 to 3.8.0

    [Snyk] Security upgrade aiohttp from 3.7.4 to 3.8.0

    This PR was automatically created by Snyk using the credentials of a real user.


    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 686/1000
    Why? Proof of Concept exploit, Has a fix available, CVSS 7.3 | HTTP Header Injection
    SNYK-PYTHON-AIOHTTP-1584144 | aiohttp:
    3.7.4 -> 3.8.0
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

    opened by ZRui98 0
  • [Snyk] Security upgrade python from 3.9.5 to 3.9.13

    [Snyk] Security upgrade python from 3.9.5 to 3.9.13

    Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.

    Changes included in this PR

    • Dockerfile

    We recommend upgrading to python:3.9.13, as this image has only 289 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.

    Some of the most important vulnerabilities in your base image include:

    | Severity | Priority Score / 1000 | Issue | Exploit Maturity | | :------: | :-------------------- | :---- | :--------------- | | critical severity | 714 | Directory Traversal
    SNYK-DEBIAN10-DPKG-2847944 | No Known Exploit | | critical severity | 714 | Buffer Overflow
    SNYK-DEBIAN10-OPENSSL-1569403 | No Known Exploit | | critical severity | 714 | OS Command Injection
    SNYK-DEBIAN10-OPENSSL-2807585 | No Known Exploit | | critical severity | 714 | OS Command Injection
    SNYK-DEBIAN10-OPENSSL-2933515 | No Known Exploit | | high severity | 614 | Improper Input Validation
    SNYK-DEBIAN10-XZUTILS-2444279 | No Known Exploit |

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

    opened by snyk-bot 0
  • [Snyk] Security upgrade python from 3.9.5 to 3.11.0rc2

    [Snyk] Security upgrade python from 3.9.5 to 3.11.0rc2

    Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.

    Changes included in this PR

    • Dockerfile

    We recommend upgrading to python:3.11.0rc2, as this image has only 278 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.

    Some of the most important vulnerabilities in your base image include:

    | Severity | Priority Score / 1000 | Issue | Exploit Maturity | | :------: | :-------------------- | :---- | :--------------- | | critical severity | 714 | Directory Traversal
    SNYK-DEBIAN10-DPKG-2847944 | No Known Exploit | | critical severity | 714 | Buffer Overflow
    SNYK-DEBIAN10-OPENSSL-1569403 | No Known Exploit | | critical severity | 714 | OS Command Injection
    SNYK-DEBIAN10-OPENSSL-2807585 | No Known Exploit | | critical severity | 714 | OS Command Injection
    SNYK-DEBIAN10-OPENSSL-2933515 | No Known Exploit | | high severity | 614 | Improper Input Validation
    SNYK-DEBIAN10-XZUTILS-2444279 | No Known Exploit |

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

    opened by snyk-bot 0
Releases(v0.5.3)
Owner
6rotoms
6rotoms
GitHub
A simple Blog Using Django Framework and Used IBM Cloud Services for Text Analysis and Text to Speech

ElhamBlog Cloud Computing Course first assignment. A simple Blog Using Django Framework and Used IBM Cloud Services for Text Analysis and Text to Spee

Elham Razi 5 Dec 6, 2022
This Django app will be used to host Source.Python plugins, sub-plugins, and custom packages.

Source.Python Project Manager This Django app will be used to host Source.Python plugins, sub-plugins, and custom packages. Want to help develop this

null 2 Sep 24, 2022
Django Fett is an incomplete code generator used on several projects

Django Fett Django Fett is an incomplete code generator used on several projects. This is an attempt to clean it up and make it public for consumption

Jeff Triplett 6 Dec 31, 2021
Yummy Django API, it's the exclusive API used for the e-yummy-ke vue web app

Yummy Django API, it's the exclusive API used for the e-yummy-ke vue web app

Am.Chris_KE 1 Feb 14, 2022
Django application and library for importing and exporting data with admin integration.

django-import-export django-import-export is a Django application and library for importing and exporting data with included admin integration. Featur

null 2.6k Dec 26, 2022
A reusable Django model field for storing ad-hoc JSON data

jsonfield jsonfield is a reusable model field that allows you to store validated JSON, automatically handling serialization to and from the database.

Ryan P Kilby 1.1k Jan 3, 2023
Easily share data across your company via SQL queries. From Grove Collab.

SQL Explorer SQL Explorer aims to make the flow of data between people fast, simple, and confusion-free. It is a Django-based application that you can

Grove Collaborative 2.1k Dec 30, 2022
Django project starter on steroids: quickly create a Django app AND generate source code for data models + REST/GraphQL APIs (the generated code is auto-linted and has 100% test coverage).

Create Django App ?? We're a Django project starter on steroids! One-line command to create a Django app with all the dependencies auto-installed AND

imagine.ai 68 Oct 19, 2022
The Django Leaflet Admin List package provides an admin list view featured by the map and bounding box filter for the geo-based data of the GeoDjango.

The Django Leaflet Admin List package provides an admin list view featured by the map and bounding box filter for the geo-based data of the GeoDjango. It requires a django-leaflet package.

Vsevolod Novikov 33 Nov 11, 2022
A standalone package to scrape financial data from listed Vietnamese companies via Vietstock

Scrape Financial Data of Vietnamese Listed Companies - Version 2 A standalone package to scrape financial data from listed Vietnamese companies via Vi

Viet Anh (Vincent) Tran 45 Nov 16, 2022
This is django-import-export module that exports data into many formats

django-import-export This is django-import-export module which exports data into many formats, you can implement this in your admin panel. -> Dehydrat

Shivam Rohilla 3 Jun 3, 2021
Social Media Network Focuses On Data Security And Being Community Driven Web App

privalise Social Media Network Focuses On Data Security And Being Community Driven Web App The Main Idea: We`ve seen social media web apps that focuse

Privalise 8 Jun 25, 2021
django-dashing is a customisable, modular dashboard application framework for Django to visualize interesting data about your project. Inspired in the dashboard framework Dashing

django-dashing django-dashing is a customisable, modular dashboard application framework for Django to visualize interesting data about your project.

talPor Solutions 703 Dec 22, 2022
Faker is a Python package that generates fake data for you.

Faker is a Python package that generates fake data for you. Whether you need to bootstrap your database, create good-looking XML documents, fill-in yo

Daniele Faraglia 15.2k Jan 1, 2023
DCM is a set of tools that helps you to keep your data in your Django Models consistent.

Django Consistency Model DCM is a set of tools that helps you to keep your data in your Django Models consistent. Motivation You have a lot of legacy

Occipital 59 Dec 21, 2022
Realtime data read and write without page refresh using Ajax in Django.

Realtime read-write with AJAX Hey,this is the basic implementation type of ajax realtime read write from the database. where you can insert or view re

Mehedi Hasan 3 Dec 13, 2022
Fully Automated YouTube Channel ▶️with Added Extra Features.

Fully Automated Youtube Channel ▒█▀▀█ █▀▀█ ▀▀█▀▀ ▀▀█▀▀ █░░█ █▀▀▄ █▀▀ █▀▀█ ▒█▀▀▄ █░░█ ░░█░░ ░▒█░░ █░░█ █▀▀▄ █▀▀ █▄▄▀ ▒█▄▄█ ▀▀▀▀ ░░▀░░ ░▒█░░ ░▀▀▀ ▀▀▀░

sam-sepiol 249 Jan 2, 2023
This is a simple bot that can be used to upload images to a third-party cloud (image hosting). Currently, only the imgbb.com website supports the bot. I Will do future updates

TGImageHosting This is a simple bot that can be used to upload images to a third party cloud (image hosting). Currently, only the imgbb.com website su

Abhijith N T 0 Dec 26, 2021
[ArXiv 2021] Data-Efficient Instance Generation from Instance Discrimination

InsGen - Data-Efficient Instance Generation from Instance Discrimination Data-Efficient Instance Generation from Instance Discrimination Ceyuan Yang,

GenForce: May Generative Force Be with You 93 Dec 25, 2022
Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation

Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation This paper has been accepted and early accessed

Yun Liu 39 Sep 20, 2022