Full featured redis cache backend for Django.

Overview

Redis cache backend for Django

Jazzband GitHub Actions Coverage https://img.shields.io/pypi/v/django-redis.svg?style=flat

This is a Jazzband project. By contributing you agree to abide by the Contributor Code of Conduct and follow the guidelines.

Introduction

django-redis is a BSD licensed, full featured Redis cache and session backend for Django.

Why use django-redis?

  • Uses native redis-py url notation connection strings
  • Pluggable clients
  • Pluggable parsers
  • Pluggable serializers
  • Primary/secondary support in the default client
  • Comprehensive test suite
  • Used in production in several projects as cache and session storage
  • Supports infinite timeouts
  • Facilities for raw access to Redis client/connection pool
  • Highly configurable (can emulate memcached exception behavior, for example)
  • Unix sockets supported by default

Requirements

User guide

Installation

Install with pip:

$ python -m pip install django-redis

Configure as cache backend

To start using django-redis, you should change your Django cache settings to something like:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
        }
    }
}

django-redis uses the redis-py native URL notation for connection strings, it allows better interoperability and has a connection string in more "standard" way. Some examples:

  • redis://[[username]:[password]]@localhost:6379/0
  • rediss://[[username]:[password]]@localhost:6379/0
  • unix://[[username]:[password]]@/path/to/socket.sock?db=0

Three URL schemes are supported:

  • redis://: creates a normal TCP socket connection
  • rediss://: creates a SSL wrapped TCP socket connection
  • unix:// creates a Unix Domain Socket connection

There are several ways to specify a database number:

  • A db querystring option, e.g. redis://localhost?db=0
  • If using the redis:// scheme, the path argument of the URL, e.g. redis://localhost/0

When using Redis' ACLs, you will need to add the username to the URL (and provide the password with the Cache OPTIONS). The login for the user django would look like this:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://django@localhost:6379/0",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "PASSWORD": "mysecret"
        }
    }
}

An alternative would be write both username and password into the URL:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://django:mysecret@localhost:6379/0",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
        }
    }
}

In some circumstances the password you should use to connect Redis is not URL-safe, in this case you can escape it or just use the convenience option in OPTIONS dict:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "PASSWORD": "mysecret"
        }
    }
}

Take care, that this option does not overwrites the password in the uri, so if you have set the password in the uri, this settings will be ignored.

Configure as session backend

Django can by default use any cache backend as session backend and you benefit from that by using django-redis as backend for session storage without installing any additional backends:

SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "default"

Testing with django-redis

django-redis supports customizing the underlying Redis client (see "Pluggable clients"). This can be used for testing purposes.

In case you want to flush all data from the cache after a test, add the following lines to your test class:

from django_redis import get_redis_connection

def tearDown(self):
    get_redis_connection("default").flushall()

Advanced usage

Pickle version

For almost all values, django-redis uses pickle to serialize objects.

The latest available version of pickle is used by default. If you want set a concrete version, you can do it, using PICKLE_VERSION option:

CACHES = {
    "default": {
        # ...
        "OPTIONS": {
            "PICKLE_VERSION": -1  # Use the latest protocol version
        }
    }
}

Socket timeout

Socket timeout can be set using SOCKET_TIMEOUT and SOCKET_CONNECT_TIMEOUT options:

CACHES = {
    "default": {
        # ...
        "OPTIONS": {
            "SOCKET_CONNECT_TIMEOUT": 5,  # seconds
            "SOCKET_TIMEOUT": 5,  # seconds
        }
    }
}

SOCKET_CONNECT_TIMEOUT is the timeout for the connection to be established and SOCKET_TIMEOUT is the timeout for read and write operations after the connection is established.

Compression support

django-redis comes with compression support out of the box, but is deactivated by default. You can activate it setting up a concrete backend:

CACHES = {
    "default": {
        # ...
        "OPTIONS": {
            "COMPRESSOR": "django_redis.compressors.zlib.ZlibCompressor",
        }
    }
}

Let see an example, of how make it work with lzma compression format:

import lzma

CACHES = {
    "default": {
        # ...
        "OPTIONS": {
            "COMPRESSOR": "django_redis.compressors.lzma.LzmaCompressor",
        }
    }
}

Lz4 compression support (requires the lz4 library):

import lz4

CACHES = {
    "default": {
        # ...
        "OPTIONS": {
            "COMPRESSOR": "django_redis.compressors.lz4.Lz4Compressor",
        }
    }
}

Memcached exceptions behavior

In some situations, when Redis is only used for cache, you do not want exceptions when Redis is down. This is default behavior in the memcached backend and it can be emulated in django-redis.

For setup memcached like behaviour (ignore connection exceptions), you should set IGNORE_EXCEPTIONS settings on your cache configuration:

CACHES = {
    "default": {
        # ...
        "OPTIONS": {
            "IGNORE_EXCEPTIONS": True,
        }
    }
}

Also, you can apply the same settings to all configured caches, you can set the global flag in your settings:

DJANGO_REDIS_IGNORE_EXCEPTIONS = True

Log Ignored Exceptions

When ignoring exceptions with IGNORE_EXCEPTIONS or DJANGO_REDIS_IGNORE_EXCEPTIONS, you may optionally log exceptions using the global variable DJANGO_REDIS_LOG_IGNORED_EXCEPTIONS in your settings file:

DJANGO_REDIS_LOG_IGNORED_EXCEPTIONS = True

If you wish to specify the logger in which the exceptions are output, simply set the global variable DJANGO_REDIS_LOGGER to the string name and/or path of the desired logger. This will default to __name__ if no logger is specified and DJANGO_REDIS_LOG_IGNORED_EXCEPTIONS is True:

DJANGO_REDIS_LOGGER = 'some.specified.logger'

Infinite timeout

django-redis comes with infinite timeouts support out of the box. And it behaves in same way as django backend contract specifies:

  • timeout=0 expires the value immediately.
  • timeout=None infinite timeout
cache.set("key", "value", timeout=None)

Get ttl (time-to-live) from key

With Redis, you can access to ttl of any stored key, for it, django-redis exposes ttl function.

It returns:

  • 0 if key does not exists (or already expired).
  • None for keys that exists but does not have any expiration.
  • ttl value for any volatile key (any key that has expiration).
>>> from django.core.cache import cache
>>> cache.set("foo", "value", timeout=25)
>>> cache.ttl("foo")
25
>>> cache.ttl("not-existent")
0

Expire & Persist

Additionally to the simple ttl query, you can send persist a concrete key or specify a new expiration timeout using the persist and expire methods:

>>> cache.set("foo", "bar", timeout=22)
>>> cache.ttl("foo")
22
>>> cache.persist("foo")
>>> cache.ttl("foo")
None
>>> cache.set("foo", "bar", timeout=22)
>>> cache.expire("foo", timeout=5)
>>> cache.ttl("foo")
5

Locks

It also supports the Redis ability to create Redis distributed named locks. The Lock interface is identical to the threading.Lock so you can use it as replacement.

with cache.lock("somekey"):
    do_some_thing()

Scan & Delete keys in bulk

django-redis comes with some additional methods that help with searching or deleting keys using glob patterns.

>>> from django.core.cache import cache
>>> cache.keys("foo_*")
["foo_1", "foo_2"]

A simple search like this will return all matched values. In databases with a large number of keys this isn't suitable method. Instead, you can use the iter_keys function that works like the keys function but uses Redis server side cursors. Calling iter_keys will return a generator that you can then iterate over efficiently.

>>> from django.core.cache import cache
>>> cache.iter_keys("foo_*")
<generator object algo at 0x7ffa9c2713a8>
>>> next(cache.iter_keys("foo_*"))
"foo_1"

For deleting keys, you should use delete_pattern which has the same glob pattern syntax as the keys function and returns the number of deleted keys.

>>> from django.core.cache import cache
>>> cache.delete_pattern("foo_*")

Redis native commands

django-redis has limited support for some Redis atomic operations, such as the commands SETNX and INCR.

You can use the SETNX command through the backend set() method with the nx parameter:

>>> from django.core.cache import cache
>>> cache.set("key", "value1", nx=True)
True
>>> cache.set("key", "value2", nx=True)
False
>>> cache.get("key")
"value1"

Also, the incr and decr methods use Redis atomic operations when the value that a key contains is suitable for it.

Raw client access

In some situations your application requires access to a raw Redis client to use some advanced features that aren't exposed by the Django cache interface. To avoid storing another setting for creating a raw connection, django-redis exposes functions with which you can obtain a raw client reusing the cache connection string: get_redis_connection(alias).

>>> from django_redis import get_redis_connection
>>> con = get_redis_connection("default")
>>> con
<redis.client.Redis object at 0x2dc4510>

WARNING: Not all pluggable clients support this feature.

Connection pools

Behind the scenes, django-redis uses the underlying redis-py connection pool implementation, and exposes a simple way to configure it. Alternatively, you can directly customize a connection/connection pool creation for a backend.

The default redis-py behavior is to not close connections, recycling them when possible.

Configure default connection pool

The default connection pool is simple. For example, you can customize the maximum number of connections in the pool by setting CONNECTION_POOL_KWARGS in the CACHES setting:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        # ...
        "OPTIONS": {
            "CONNECTION_POOL_KWARGS": {"max_connections": 100}
        }
    }
}

You can verify how many connections the pool has opened with the following snippet:

from django_redis import get_redis_connection

r = get_redis_connection("default")  # Use the name you have defined for Redis in settings.CACHES
connection_pool = r.connection_pool
print("Created connections so far: %d" % connection_pool._created_connections)

Since the default connection pool passes all keyword arguments it doesn't use to its connections, you can also customize the connections that the pool makes by adding those options to CONNECTION_POOL_KWARGS:

CACHES = {
    "default": {
        # ...
        "OPTIONS": {
            "CONNECTION_POOL_KWARGS": {"max_connections": 100, "retry_on_timeout": True}
        }
    }
}

Use your own connection pool subclass

Sometimes you want to use your own subclass of the connection pool. This is possible with django-redis using the CONNECTION_POOL_CLASS parameter in the backend options.

Customize connection factory

If none of the previous methods satisfies you, you can get in the middle of the django-redis connection factory process and customize or completely rewrite it.

By default, django-redis creates connections through the django_redis.pool.ConnectionFactory class that is specified in the global Django setting DJANGO_REDIS_CONNECTION_FACTORY.

class ConnectionFactory(object):
    def get_connection_pool(self, params: dict):
        # Given connection parameters in the `params` argument, return new
        # connection pool. It should be overwritten if you want do
        # something before/after creating the connection pool, or return
        # your own connection pool.
        pass

    def get_connection(self, params: dict):
        # Given connection parameters in the `params` argument, return a
        # new connection. It should be overwritten if you want to do
        # something before/after creating a new connection. The default
        # implementation uses `get_connection_pool` to obtain a pool and
        # create a new connection in the newly obtained pool.
        pass

    def get_or_create_connection_pool(self, params: dict):
        # This is a high layer on top of `get_connection_pool` for
        # implementing a cache of created connection pools. It should be
        # overwritten if you want change the default behavior.
        pass

    def make_connection_params(self, url: str) -> dict:
        # The responsibility of this method is to convert basic connection
        # parameters and other settings to fully connection pool ready
        # connection parameters.
        pass

    def connect(self, url: str):
        # This is really a public API and entry point for this factory
        # class. This encapsulates the main logic of creating the
        # previously mentioned `params` using `make_connection_params` and
        # creating a new connection using the `get_connection` method.
        pass

Use the sentinel connection factory

In order to facilitate using Redis Sentinels, django-redis comes with a built in sentinel connection factory, which creates sentinel connection pools. In order to enable this functionality you should add the following:

Pluggable parsers

redis-py (the Python Redis client used by django-redis) comes with a pure Python Redis parser that works very well for most common task, but if you want some performance boost, you can use hiredis.

hiredis is a Redis client written in C and it has its own parser that can be used with django-redis.

"OPTIONS": {
    "PARSER_CLASS": "redis.connection.HiredisParser",
}

Pluggable clients

django-redis is designed for to be very flexible and very configurable. For it, it exposes a pluggable backends that make easy extend the default behavior, and it comes with few ones out the box.

Default client

Almost all about the default client is explained, with one exception: the default client comes with replication support.

To connect to a Redis replication setup, you should change the LOCATION to something like:

"LOCATION": [
    "redis://127.0.0.1:6379/1",
    "redis://127.0.0.1:6378/1",
]

The first connection string represents the primary server and the rest to replica servers.

WARNING: Replication setup is not heavily tested in production environments.

Shard client

This pluggable client implements client-side sharding. It inherits almost all functionality from the default client. To use it, change your cache settings to something like this:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": [
            "redis://127.0.0.1:6379/1",
            "redis://127.0.0.1:6379/2",
        ],
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.ShardClient",
        }
    }
}

WARNING: Shard client is still experimental, so be careful when using it in production environments.

Herd client

This pluggable client helps dealing with the thundering herd problem. You can read more about it on link: Wikipedia

Like previous pluggable clients, it inherits all functionality from the default client, adding some additional methods for getting/setting keys.

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.HerdClient",
        }
    }
}

This client exposes additional settings:

  • CACHE_HERD_TIMEOUT: Set default herd timeout. (Default value: 60s)

Pluggable serializer

The pluggable clients serialize data before sending it to the server. By default, django-redis serializes the data using the Python pickle module. This is very flexible and can handle a large range of object types.

To serialize using JSON instead, the serializer JSONSerializer is also available.

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "SERIALIZER": "django_redis.serializers.json.JSONSerializer",
        }
    }
}

There's also support for serialization using MsgPack (that requires the msgpack library):

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "SERIALIZER": "django_redis.serializers.msgpack.MSGPackSerializer",
        }
    }
}

Pluggable Redis client

django-redis uses the Redis client redis.client.StrictClient by default. It is possible to use an alternative client.

You can customize the client used by setting REDIS_CLIENT_CLASS in the CACHES setting. Optionally, you can provide arguments to this class by setting REDIS_CLIENT_KWARGS.

CACHES = {
    "default": {
        "OPTIONS": {
            "REDIS_CLIENT_CLASS": "my.module.ClientClass",
            "REDIS_CLIENT_KWARGS": {"some_setting": True},
        }
    }
}

Closing Connections

The default django-redis behavior on close() is to keep the connections to Redis server.

You can change this default behaviour for all caches by the DJANGO_REDIS_CLOSE_CONNECTION = True in the django settings (globally) or (at cache level) by setting CLOSE_CONNECTION: True in the OPTIONS for each configured cache.

Setting True as a value will instruct the django-redis to close all the connections (since v. 4.12.2), irrespectively of its current usage.

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "CLOSE_CONNECTION": True,
        }
    }
}

License

Copyright (c) 2011-2015 Andrey Antukh <[email protected]>
Copyright (c) 2011 Sean Bleier

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in the
   documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
   derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS`` AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Comments
  • Started typing codebase

    Started typing codebase

    I started giving types to the codebase using type hinting since python3.5 supports them and python3.5 is the minimum version required by the library.

    I started with serializers and compressors.

    I have some doubts about the client, specifically about key, version and prefix.

    Redis supports everything that can be byte encoded like stated here https://redis.io/topics/data-types-intro

    Redis keys are binary safe, this means that you can use any binary sequence as a key, from a string like "foo" to the content of a JPEG file. The empty string is also a valid key.
    

    But other caches like memcached and django own BaseCache seem to point to key and prefix being a string and version an integer.

    My question is, leaving Any instead of a more specific type or to enforce the types that I just mentioned?

    By the way django-stubs here https://github.com/typeddjango/django-stubs/blob/master/django-stubs/core/cache/backends/base.pyi have specific types in some methods and on some not.

    opened by WisdomPill 25
  • django-redis is incompatible with redis-py >= 3

    django-redis is incompatible with redis-py >= 3

    From the redis-py release notes:

    Only bytes, strings and numbers (ints, longs and floats) are acceptable for keys and values.

    See https://github.com/andymccurdy/redis-py/blob/9b03af26dc829beea232a3248768de933f4c3b67/CHANGES#L9-L15

    The CacheKey class used by django-redis is no longer supported.

    I'm happy to open a PR, but I wanted to ask first if it's better to drop CacheKey or to cast keys to strings right before sending them to redis-py? The latter would be more work, but retain the intended use of CacheKey:

    A stub string class that we can use to check if a key was created already.

    opened by michael-k 24
  • add redis sentinel support

    add redis sentinel support

    I know that there were multiple requests, and most of them were denied since the primary author was not working with sentinel. Now that this project got passed to JazzBand, would it be possible to request sentinel support for this project?

    enhancement help needed 
    opened by L1ghtman2k 21
  • adjust GitHub actions and tox config

    adjust GitHub actions and tox config

    Changes to GitHub actions:

    • The service definition was duplicated on squash
    • Merge lint and test workflow files into ci.yml
    • Update for Django 3.2 and only test unreleased library versions with Python 3.9 to limit permutations
    • Move redis sentinel startup to a script to more easily use in local development

    Changes to README:

    • Fix the GitHub badge definition

    Changes to tests and testing environment:

    • Move tox config to setup.cfg
    • Remove runtest-*.py wrappers in favor of using runtest.py --settings=<settings>
    • Move test settings into their own directory
    • Fix unix socket tests

    x-ref: https://github.com/ymyzk/tox-gh-actions/pull/60 will remove the need for lint-dj31-redislatest

    opened by terencehonles 19
  • URL notation in LOCATION

    URL notation in LOCATION

    Hi,

    Since this package uses redis-py under the hood, I think it would be better to support the URL notation of the redis-py package directly in LOCATION instead of defining a new notation.

    For example: django-redis: unix:/path/to/socket:1 redis-py: unix:///path/to/socket?db=1

    django-redis: 127.0.0.1:6379:1 redis-py: redis://127.0.0.1:6379/1

    redis-py URL notation:

    redis://[:password]@localhost:6379/0
    rediss://[:password]@localhost:6379/0
    unix://[:password]@/path/to/socket.sock?db=0
    

    Details here: https://github.com/andymccurdy/redis-py/blob/master/redis/connection.py#L733 There is no need to develop anything, the URL can just be passed to the redis client/connection pool.

    What do you think?

    Thanks

    enhancement 
    opened by YAmikep 18
  • New maintainer needed.

    New maintainer needed.

    Hi,

    Unfortunatelly or not i'm no longer using python on my daily job and right now i dont have time to maintain this package. So for not just abandon it, i'm looking for a new maintainer.

    help needed 
    opened by niwinz 17
  • Roadmap for next release

    Roadmap for next release

    There are a few new things for the next release already.

    Can we discuss on what is blocking the next release? Maybe call it even 5.1.0

    I would say to solve the problem with redis-py>=4.0.0 is the only blocker

    @terencehonles what do you think?

    question 
    opened by WisdomPill 16
  • wrong number of arguments for 'set' command

    wrong number of arguments for 'set' command

    When i use `request.session[0]='xxx', return HttpResponse(None), then raise a ResponseError:wrong number of arguments for 'set' command

    it's raised from 'django_redis/client/default.py in set', why ?

    invalid 
    opened by sevenguin 16
  • Is TIMEOUT setting respected by django-redis?

    Is TIMEOUT setting respected by django-redis?

    Hello, can anybody confirm that TIMEOUT setting is respected by django-redis? This is my configuration (I want key/values to never expire):

    CACHES = {
        'default': {
            'BACKEND': 'redis_cache.cache.RedisCache',
            'LOCATION': '127.0.0.1:6379',
            'KEY_PREFIX': 'my-prefix',
            'OPTIONS': {
                'TIMEOUT': None
            }
        }
    }
    
    opened by honi 15
  • Sentinel commands have migrated in redis main

    Sentinel commands have migrated in redis main

    Redis pre release test throwing error in CI

    • Test Python 3.9, Django 3.2, Redis.py master
    • Test Python 3.9, Django 3.2 main, Redis.py master
      self = Sentinel<sentinels=[127.0.0.1:26379]>, service_name = 'default_service'
      
          def discover_master(self, service_name):
              """
              Asks sentinel servers for the Redis master's address corresponding
              to the service labeled ``service_name``.
          
              Returns a pair (address, port) or raises MasterNotFoundError if no
              master is found.
              """
              for sentinel_no, sentinel in enumerate(self.sentinels):
                  try:
      >               masters = sentinel.sentinel_masters()
      E               AttributeError: 'Redis' object has no attribute 'sentinel_masters'
      
      .tox/py39-dj32-redismaster/lib/python3.9/site-packages/redis/sentinel.py:211: AttributeError
      _____ ERROR at teardown of DjangoRedisCacheTestEscapePrefix.test_iter_keys _____
      
      self = <tests.test_backend.DjangoRedisCacheTestEscapePrefix testMethod=test_iter_keys>
      
          def tearDown(self):
    
    
    bug 
    opened by sarathak 14
  • Follow Django cache interface

    Follow Django cache interface

    django-redis 3.8.1 breaks the following code: https://bitbucket.org/psagers/django-auth-ldap/src/4e041084354eab8c2beefc850148e7f7c769d9e5/django_auth_ldap/backend.py?at=default#cl-824

    It seems that django-redis is not following the Django cache interface.

    https://docs.djangoproject.com/en/1.7/topics/cache/#basic-usage

    Should be able to set timeout without using a keyword argument.

    opened by limeburst 14
  • django-redis TypeError: __init__() got an unexpected keyword argument 'ssl'

    django-redis TypeError: __init__() got an unexpected keyword argument 'ssl'

    Describe the bug this bug happend when I want to make connection to redis cluster with use TLS

    To Reproduce Steps to reproduce the behavior:

    1. here is my config on settings.py:
    'redis-cluster-tls': {
        'BACKEND': 'django_redis.cache.RedisCache',
        'LOCATION': 'rediss://:[email protected]',
        'OPTIONS': {
            'REDIS_CLIENT_CLASS': 'rediscluster.RedisCluster',
            'CONNECTION_POOL_CLASS': 'rediscluster.connection.ClusterConnectionPool',
            'CONNECTION_POOL_KWARGS': {
                'skip_full_coverage_check': True,
                'ssl': True,
            },
            "SOCKET_CONNECT_TIMEOUT": 20
            "SOCKET_TIMEOUT": 20
        },
        "KEY_PREFIX": 'dev'
    }
    

    Expected behavior it can get value and set to redis cluster with use TLS (Connection Type: OSS Cluster)

    Stack trace

    traceback: Traceback (most recent call last):
      File "/Users/macbook2015/.local/share/virtualenvs/xxxxx-CWlIwagZ/lib/python3.9/site-packages/rediscluster/connection.py", line 364, in get_connection_by_node
        connection = self._available_connections.get(node["name"], []).pop()
    IndexError: pop from empty list
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/Users/macbook2015/.local/share/virtualenvs/'-CWlIwagZ/lib/python3.9/site-packages/rediscluster/client.py", line 615, in _execute_command
        connection = self.connection_pool.get_connection_by_node(node)
      File "/Users/macbook2015/.local/share/virtualenvs/xxxxx-CWlIwagZ/lib/python3.9/site-packages/rediscluster/connection.py", line 366, in get_connection_by_node
        connection = self.make_connection(node)
      File "/Users/macbook2015/.local/share/virtualenvs/xxxxx-CWlIwagZ/lib/python3.9/site-packages/rediscluster/connection.py", line 273, in make_connection
        connection = self.connection_class(host=node["host"], port=node["port"], **self.connection_kwargs)
      File "/Users/macbook2015/.local/share/virtualenvs/xxxxx-CWlIwagZ/lib/python3.9/site-packages/redis/connection.py", line 828, in __init__
        super(SSLConnection, self).__init__(**kwargs)
    TypeError: __init__() got an unexpected keyword argument 'ssl'
    

    Environment (please complete the following information): Python==3.9 Django==4.1.4 django-redis==5.2.0 redis==3.5.3 redis-py-cluster==2.1.3

    additional context: If I try some changes on redis-py package acctually its fixed the problem but can you guys help me to fix in django-redis instead of fix on redis-py

    before:

    class SSLConnection(Connection):
    
        def __init__(self, ssl_keyfile=None, ssl_certfile=None,
                     ssl_cert_reqs='required', ssl_ca_certs=None,
                     ssl_check_hostname=False, **kwargs):
            if not ssl_available:
                raise RedisError("Python wasn't built with SSL support")
                
            super(SSLConnection, self).__init__(**kwargs)
    
            self.keyfile = ssl_keyfile
            self.certfile = ssl_certfile
    

    after:

    class SSLConnection(Connection):
    
        def __init__(self, ssl_keyfile=None, ssl_certfile=None,
                     ssl_cert_reqs='required', ssl_ca_certs=None,
                     ssl_check_hostname=False, **kwargs):
            if not ssl_available:
                raise RedisError("Python wasn't built with SSL support")
            
            if 'ssl' in kwargs:
                del kwargs['ssl']
            
            super(SSLConnection, self).__init__(**kwargs)
    
            self.keyfile = ssl_keyfile
            self.certfile = ssl_certfile
    
    bug 
    opened by yosmelvin 0
  • Python 3.11 Support

    Python 3.11 Support

    Hi! This package is not currently documented as compatible with Python 3.11, so tools like poetry can't install it to a codebase running said version.

    Please consider updating in so users with the latest python version can use your package

    Thanks!

    opened by rmortes 1
  • Multiple databases support

    Multiple databases support

    I’m currently using django to build a multi tenant service. The DB is selected dynamically based on the url subdomain, custom database router and a dict that maps subdomain to the DB

    The django docs indicate that database caches create separate tables for each database. How is it handled by django-redis ? Is there default support for multiple databases? Do you use the db name as a prefix/root key? Or is it built assuming one DB per project ?

    question 
    opened by VaZark 0
  • Django unit tests hang forever when redis is not reachable

    Django unit tests hang forever when redis is not reachable

    When my redis instance is not reachable, it seems that my unit tests hang indefinitely. No single test is executed, though, just a hang after the system checks.

    I have tried setting the timeouts, socket timeouts, etc, without any luck.

    Is there some way to get this to fail faster, is there something I am configuring incorrectly?

    project ❯ ./manage.py test
    Found 262 test(s).
    Creating test database for alias 'default'...
    System check identified no issues (0 silenced).
    # <hangs here>
    # ^C^C to exit
    
    CACHES = {
        "default": {
            "BACKEND": "django_redis.cache.RedisCache",
            "LOCATION": "redis://localhost:6379/1",
            "TIMEOUT": 5,
            "OPTIONS": {
                "SOCKET_TIMEOUT": 5,
                "SOCKET_CONNECT_TIMEOUT": 5
            }
        }
    }
    question 
    opened by awbacker 0
Owner
Jazzband
We are all part of this
Jazzband
A simple app that provides django integration for RQ (Redis Queue)

Django-RQ Django integration with RQ, a Redis based Python queuing library. Django-RQ is a simple app that allows you to configure your queues in djan

RQ 1.6k Dec 28, 2022
Mr. Queue - A distributed worker task queue in Python using Redis & gevent

MRQ MRQ is a distributed task queue for python built on top of mongo, redis and gevent. Full documentation is available on readthedocs Why? MRQ is an

Pricing Assistant 871 Dec 25, 2022
RQ (Redis Queue) integration for Flask applications

Flask-RQ RQ (Redis Queue) integration for Flask applications Resources Documentation Issue Tracker Code Development Version Installation $ pip install

Matt Wright 205 Nov 6, 2022
Asynchronous tasks in Python with Celery + RabbitMQ + Redis

python-asynchronous-tasks Setup & Installation Create a virtual environment and install the dependencies: $ python -m venv venv $ source env/bin/activ

Valon Januzaj 40 Dec 3, 2022
Redis-backed message queue implementation that can hook into a discord bot written with hikari-lightbulb.

Redis-backed FIFO message queue implementation that can hook into a discord bot written with hikari-lightbulb. This is eventually intended to be the backend communication between a bot and a web dashboard.

thomm.o 7 Dec 5, 2022
Beatserver, a periodic task scheduler for Django 🎵

Beat Server Beatserver, a periodic task scheduler for django channels | beta software How to install Prerequirements: Follow django channels documenta

Raja Simon 130 Dec 17, 2022
A multiprocessing distributed task queue for Django

A multiprocessing distributed task queue for Django Features Multiprocessing worker pool Asynchronous tasks Scheduled, cron and repeated tasks Signed

Ilan Steemers 1.7k Jan 3, 2023
A Django app that integrates with Dramatiq.

django_dramatiq django_dramatiq is a Django app that integrates with Dramatiq. Requirements Django 1.11+ Dramatiq 0.18+ Example You can find an exampl

Bogdan Popa 261 Dec 25, 2022
Django database backed celery periodic task scheduler with support for task dependency graph

Djag Scheduler (Dj)ango Task D(AG) (Scheduler) Overview Djag scheduler associates scheduling information with celery tasks The task schedule is persis

Mohith Reddy 3 Nov 25, 2022
Queuing with django celery and rabbitmq

queuing-with-django-celery-and-rabbitmq Install Python 3.6 or above sudo apt-get install python3.6 Install RabbitMQ sudo apt-get install rabbitmq-ser

null 1 Dec 22, 2021
Full featured redis cache backend for Django.

Redis cache backend for Django This is a Jazzband project. By contributing you agree to abide by the Contributor Code of Conduct and follow the guidel

Jazzband 2.5k Jan 3, 2023
A Redis cache backend for django

Redis Django Cache Backend A Redis cache backend for Django Docs can be found at http://django-redis-cache.readthedocs.org/en/latest/. Changelog 3.0.0

Sean Bleier 1k Dec 15, 2022
A Redis cache backend for django

Redis Django Cache Backend A Redis cache backend for Django Docs can be found at http://django-redis-cache.readthedocs.org/en/latest/. Changelog 3.0.0

Sean Bleier 1k Dec 15, 2022
Cache-house - Caching tool for python, working with Redis single instance and Redis cluster mode

Caching tool for python, working with Redis single instance and Redis cluster mo

Tural 14 Jan 6, 2022
Django package to log request values such as device, IP address, user CPU time, system CPU time, No of queries, SQL time, no of cache calls, missing, setting data cache calls for a particular URL with a basic UI.

django-web-profiler's documentation: Introduction: django-web-profiler is a django profiling tool which logs, stores debug toolbar statistics and also

MicroPyramid 77 Oct 29, 2022
Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python.

DiskCache is an Apache2 licensed disk and file backed cache library, written in pure-Python, and compatible with Django.

Grant Jenks 1.7k Jan 5, 2023
Jira-cache - Jira cache with python

Direct queries to Jira have two issues: they are sloooooow many queries are impo

John Scott 6 Oct 8, 2022
Full-featured django project start tool.

django-start-tool Introduction django-start-tool is a full-featured replacement for django-admin startproject which provides cli for creating the same

Georgy Gnezdilov 0 Aug 30, 2022
Django-gmailapi-json-backend - Email backend for Django which sends email via the Gmail API through a JSON credential

django-gmailapi-json-backend Email backend for Django which sends email via the

Innove 1 Sep 9, 2022
Backend Boilerplate using Django,celery,Redis

Backend Boilerplate using Django,celery,Redis

Daniel Mawioo 2 Sep 14, 2022