Cachier

Persistent, stale-free, local and cross-machine caching for Python functions.

from cachier import cachier
import datetime

@cachier(stale_after=datetime.timedelta(days=3))
def foo(arg1, arg2):
  """foo now has a persistent cache, trigerring recalculation for values stored more than 3 days."""
  return {'arg1': arg1, 'arg2': arg2}

Contents

Installation
Features
- Future features
Use
Cachier Cores
Contributing
Credits

Installation

Install cachier with:

pip install cachier

For the latest version supporting Python 2.7 please use:

pip install 'cachier==1.2.8'

Features

Pure Python.
Compatible with Python 3.5+ (and Python 2.7 up until version 1.2.8).
Supported and tested on Linux, OS X and Windows.
A simple interface.
Defining "shelf life" for cached values.
Local caching using pickle files.
Cross-machine caching using MongoDB.
Thread-safety.

Cachier is NOT:

Meant as a transient cache. Python's @lru_cache is better.
Especially fast. It is meant to replace function calls that take more than... a second, say (overhead is around 1 millisecond).

Future features

S3 core.
Multi-core caching.
Cache replacement policies

Cachier provides a decorator which you can wrap around your functions to give them a persistent cache. The positional and keyword arguments to the wrapped function must be hashable (i.e. Python's immutable built-in objects, not mutable containers). Also, notice that since objects which are instances of user-defined classes are hashable but all compare unequal (their hash value is their id), equal objects across different sessions will not yield identical keys.

Setting up a Cache

You can add a default, pickle-based, persistent cache to your function - meaning it will last across different Python kernels calling the wrapped function - by decorating it with the cachier decorator (notice the ()!).

from cachier import cachier

@cachier()
def foo(arg1, arg2):
  """Your function now has a persistent cache mapped by argument values!"""
  return {'arg1': arg1, 'arg2': arg2}

Resetting a Cache

The Cachier wrapper adds a clear_cache() function to each wrapped function. To reset the cache of the wrapped function simply call this method:

foo.clear_cache()

Cache Shelf Life

Setting Shelf Life

You can set any duration as the shelf life of cached return values of a function by providing a corresponding timedelta object to the stale_after parameter:

import datetime

@cachier(stale_after=datetime.timedelta(weeks=2))
def bar(arg1, arg2):
  return {'arg1': arg1, 'arg2': arg2}

Now when a cached value matching the given arguments is found the time of its calculation is checked; if more than stale_after time has since passed, the function will be run again for the same arguments and the new value will be cached and returned.

This is useful for lengthy calculations that depend on a dynamic data source.

Fuzzy Shelf Life

Sometimes you may want your function to trigger a calculation when it encounters a stale result, but still not wait on it if it's not that critical. In that case, you can set next_time to True to have your function trigger a recalculation in a separate thread, but return the currently cached stale value:

@cachier(next_time=True)

Further function calls made while the calculation is being performed will not trigger redundant calculations.

Working with unhashable arguments

As mentioned above, the positional and keyword arguments to the wrapped function must be hashable (i.e. Python's immutable built-in objects, not mutable containers). To get around this limitation the hash_params parameter of the cachier decorator can be provided with a callable that gets the args and kwargs from the decorated function and returns a hash key for them.

@cachier(hash_params=hash_my_custom_class)
def calculate_super_complex_stuff(custom_obj):
  # amazing code goes here

Per-function call arguments

Cachier also accepts several keyword arguments in the calls of the function it wraps rather than in the decorator call, allowing you to modify its behaviour for a specific function call.

Ignore Cache

You can have cachier ignore any existing cache for a specific function call by passing ignore_cache=True to the function call. The cache will neither be checked nor updated with the new return value.

@cachier()
def sum(first_num, second_num):
  return first_num + second_num

def main():
  print(sum(5, 3, ignore_cache=True))

Overwrite Cache

You can have cachier overwrite an existing cache entry - if one exists - for a specific function call by passing overwrite_cache=True to the function call. The cache will not be checked but will be updated with the new return value.

Verbose Cache Call

You can have cachier print out a detailed explanation of the logic of a specific call by passing verbose_cache=True to the function call. This can be useful if you are not sure why a certain function result is, or is not, returned.

Cachier Cores

Pickle Core

The default core for Cachier is pickle based, meaning each function will store its cache is a separate pickle file in the ~/.cachier directory. Naturally, this kind of cache is both machine-specific and user-specific.

You can configure cachier to use another directory by providing the cache_dir parameter with the path to that directory:

@cachier(cache_dir='~/.temp/.cache')

You can slightly optimise pickle-based caching if you know your code will only be used in a single thread environment by setting:

@cachier(pickle_reload=False)

This will prevent reading the cache file on each cache read, speeding things up a bit, while also nullifying inter-thread functionality (the code is still thread safe, but different threads will have different versions of the cache at times, and will sometime make unnecessary function calls).

Setting the optional argument separate_files to True will cause the cache to be stored in several files: A file per argument set, per function. This can help if your per-function cache files become too large.

from cachier import cachier

@cachier(separate_files=True)
def foo(arg1, arg2):
  """Your function now has a persistent cache mapped by argument values, split across several files, per argument set"""
  return {'arg1': arg1, 'arg2': arg2}

You can get the fully qualified path to the directory of cache files used by cachier (~/.cachier by default) by calling the cache_dpath() function:

>>> foo.cache_dpath()
    "/home/bigus/.cachier/"

MongoDB Core

You can set a MongoDB-based cache by assigning mongetter with a callable that returns a pymongo.Collection object with writing permissions:

  from pymongo import MongoClient

  def my_mongetter():
      client = MongoClient(get_cachier_db_auth_uri())
      db_obj = client['cachier_db']
      if 'someapp_cachier_db' not in db_obj.list_collection_names():
          db_obj.create_collection('someapp_cachier_db')
      return db_obj['someapp_cachier_db']

@cachier(mongetter=my_mongetter)

This allows you to have a cross-machine, albeit slower, cache. This functionality requires that the installation of the pymongo python package.

In certain cases the MongoDB backend might leave a deadlock behind, blocking all subsequent requests from being processed. If you encounter this issue, supply the wait_for_calc_timeout with a reasonable number of seconds; calls will then wait at most this number of seconds before triggering a recalculation.

@cachier(mongetter=False, wait_for_calc_timeout=2)

Memory Core

You can set an in-memory cache by assigning the backend parameter with 'memory':

@cachier(backend='memory')

Note, however, that cachier's in-memory core is simple, and has no monitoring or cap on cache size, and can thus lead to memory errors on large return values - it is mainly intended to be used with future multi-core functionality. As a rule, Python's built-in lru_cache is a much better stand-alone solution.

Contributing

Package author and current maintainer is Shay Palachy ([email protected]); You are more than welcome to approach him for help. Contributions are very welcomed.

Installing for development

Clone:

git clone [email protected]:shaypal5/cachier.git

Install in development mode with test dependencies:

cd cachier
pip install -e ".[test]"

Running the tests

To run the tests, call the pytest command in the repository's root, or:

python -m pytest

To run only pickle core related tests, use:

pytest -m mongo

Adding documentation

This project is documented using the numpy docstring conventions, which were chosen as they are perhaps the most widely-spread conventions that are both supported by common tools such as Sphinx and result in human-readable docstrings (in my personal opinion, of course). When documenting code you add to this project, please follow these conventions.

Additionally, if you update this README.rst file, use python setup.py checkdocs to validate it compiles.

Credits

Created by Shay Palachy ([email protected]).

Other major contributors:

cthoyt - Base memory core implementation.

amarczew - The hash_params kwarg.

non-senses - The wait_for_calc_timeout kwarg.

Elad Rapapor - Multi-file Pickle core, a.k.a separate_files (released on v1.5.3).

Notable bugfixers:

MichaelRazum.

Eric Ma - The iNotify bugfix (released on v1.5.3).

Ofir - The iNotify bugfix (released on v1.5.3).

When using cachier with some long-running functions, I get the following error:

OSError: inotify instance limit reached

A fuller stack trace is in the details below. I have tried my best to obscure out proprietary information, so please forgive me if it kind of messes up the syntax.

What would be the best next step here to debug? (No pressure on your side, @shaypal5, I have no problems debugging on my own, just wanted to know your thoughts.)

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
 in 
----> 1 s = my_long_func("arg")
~/path/to/my/env/lib/python3.7/site-packages/cachier/core.py in func_wrapper(*args, **kwds)
169                         return _calc_entry(core, key, func, args, kwds)
170             _print('No entry found. No current calc. Calling like a boss.')
--> 171             return _calc_entry(core, key, func, args, kwds)
172
173         def clear_cache():
~/path/to/my/env/lib/python3.7/site-packages/cachier/core.py in _calc_entry(core, key, func, args, kwds)
65         core.mark_entry_being_calculated(key)
66         # _get_executor().submit(core.mark_entry_being_calculated, key)
---> 67         func_res = func(*args, **kwds)
68         core.set_entry(key, func_res)
69         # _get_executor().submit(core.set_entry, key, func_res)
~/path/to/my/src.py in my_long_func.py(kwarg, another_arg)
47 @cachier(stale_after=timedelta(weeks=1))
48 def my_long_func.py(kwarg: str, another_arg: bool = False):
---> 49     res_df = wrapped_func(kwarg=kwarg, another_arg="string")
50     if another_arg:
51         return res_df.query("valid_data == True")
~/path/to/my/env/lib/python3.7/site-packages/cachier/core.py in func_wrapper(*args, **kwds)
165                     _print('No value but being calculated. Waiting.')
166                     try:
--> 167                         return core.wait_on_entry_calc(key)
168                     except RecalculationNeeded:
169                         return _calc_entry(core, key, func, args, kwds)
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
191         if observer.isAlive():
192             # print('Timedout waiting. Starting again...')
--> 193             return self.wait_on_entry_calc(key)
194         # print("Returned value: {}".format(event_handler.value))
195         return event_handler.value
~/path/to/my/env/lib/python3.7/site-packages/cachier/pickle_core.py in wait_on_entry_calc(self, key)
187             recursive=True
188         )
--> 189         observer.start()
190         observer.join(timeout=1.0)
191         if observer.isAlive():
~/path/to/my/env/lib/python3.7/site-packages/watchdog/observers/api.py in start(self)
253     def start(self):
254         for emitter in self._emitters:
--> 255             emitter.start()
256         super(BaseObserver, self).start()
257
~/path/to/my/env/lib/python3.7/site-packages/watchdog/utils/init.py in start(self)
108
109     def start(self):
--> 110         self.on_thread_start()
111         threading.Thread.start(self)
112
~/path/to/my/env/lib/python3.7/site-packages/watchdog/observers/inotify.py in on_thread_start(self)
119     def on_thread_start(self):
120         path = unicode_paths.encode(self.watch.path)
--> 121         self._inotify = InotifyBuffer(path, self.watch.is_recursive)
122
123     def on_thread_stop(self):
~/path/to/my/env/lib/python3.7/site-packages/watchdog/observers/inotify_buffer.py in init(self, path, recursive)
33         BaseThread.init(self)
34         self._queue = DelayedQueue(self.delay)
---> 35         self._inotify = Inotify(path, recursive)
36         self.start()
37
~/path/to/my/env/lib/python3.7/site-packages/watchdog/observers/inotify_c.py in init(self, path, recursive, event_mask)
186         inotify_fd = inotify_init()
187         if inotify_fd == -1:
--> 188             Inotify._raise_error()
189         self._inotify_fd = inotify_fd
190         self._lock = threading.Lock()
~/path/to/my/env/lib/python3.7/site-packages/watchdog/observers/inotify_c.py in _raise_error()
413             raise OSError("inotify watch limit reached")
414         elif err == errno.EMFILE:
--> 415             raise OSError("inotify instance limit reached")
416         else:
417             raise OSError(os.strerror(err))
</pre>
</details>

		                                  
		                                    bug

opened by ericmjl 16

Change `wait_on_entry_calc` to use while loop instead of recursion

@shaypal5 I'm submitting this PR to hopefully make cachier a bit more usable with long-running functions.

I had noticed that the recursion pattern in wait_on_entry_calc was causing the issues I was seeing in #24. I changed the implementation here, and ran a few test cases on a project I was working on.

For short-running functions, nothing breaks. The cache gets loaded correctly, and the function executes correctly. For functions that are already cached, I made sure to use overwrite_cache=True to force overwriting of the cache. ~~For long-running functions, the function just keeps on executing nicely until it finishes up and pickles down the result.~~ I take this back - I am still measuring to see whether this code works properly.

UPDATE 3:52 PM: I think this is working as expected. It's a long-running database query that we cache locally. I can see in pgadmin that the data are being pulled.

If you see that this is an erroneous PR, please let me know. Otherwise, please go to town with your critiques, and let me know what you'd like fixed up, I am happy to continue working on the PR till you're satisfied :smile:.
good first issue

opened by ericmjl 12

Fix wheel creation

Without this, attempting to build a wheel produces:

$ python setup.py bdist_wheel
/path/to/venv/lib/python2.7/site-packages/setuptools/dist.py:333: UserWarning: Normalizing '1.2.1.post.dev3' to '1.2.1.post0.dev3'
  normalized_version,
running bdist_wheel
error: invalid truth value ''

opened by mikeage 10

Add "In Memory" cache backend.

This looks like a solid cache tool, but lack of an in memory backend to both avoid filesystem interaction and not require an external dependency like running mongodb, is a pretty big hinderance for me.
enhancement

opened by techdragon 8
Is there a way to disable the caching for unit-testing of the client code?

The pickling conflicts with the mock objects otherwise, and it seems like a clean solution could be to have a way to globally disarm all caching.

Since no one else has brought this question up, I assume there's a better way to work around this that I'm unaware of.
question

opened by nima 7
The cache file is too large

If a function is repeatedly called with different parameters, I find that the file in the cache directory still has only one file named after the function name. This leads to a huge file size when the function is called many times, so it reads And the speed of rewriting is not as good as when the cache is not used.
enhancement

opened by shanghaiyangming 7
Built-in hash() method is not deterministic, which breaks cachier since #17
Cachier is broken for me since #17 for string arguments (probably for other built-in objects as well). This comes from the fact that the built-in hash() method is not deterministic between runs in python. Therefore calling it on the arguments of the function for caching purpose does not work between multiple runs because it will hash to different values.

The problematic call to hash() is here.

Repro

Run this script twice in a row and it will re-execute the method test() each time without relying on the cache:

import time from cachier import cachier @cachier(pickle_reload=False) def test(text): time.sleep(1) print(text) print(hash(text)) return 0 start_time = time.time() text = 'foo' print(hash(text)) test(text) print(time.time() - start_time)
bug
opened by louismartin 7
Added RecalculationNeeded Exception
Hello,

in this scenario we cachier is getting a deadlock:

cache backend: mongodb

process_1 is calling an expensive function

process_2 is calling an expensive function

process_3 is calling clear_cache

Then there is a deadlock. I guess in a multi-process environment it doesn't make sense to use the pickle_code. So that fix is only applied on the mongo core.

This can happen quite often, at least in my environment. For some reason, the expensive function doesn't finalize or needs too long. Therefore process_3 is responsible to clear the cache.
opened by MichaelRazum 7
Compatibility changes from 0.x -> 1.x

Hi, I love cachier!

I'm just investigating some things in our stack that appear to be broken with the release of cachier 1.1x: https://github.com/biolink/biolink-api/issues/53

Is there a list of changes for the 1.x series?
bug

opened by cmungall 7

Add support for Windows

Hello,

It doesn't work on Windows due to lack of fcntln :

pip install cachier

In [1]: from cachier import cache
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-4d9fcdd09a5c> in <module>()
----> 1 from cachier import cache

c:\users\x\appdata\local\programs\python\python35\lib\site-packages\cachier\__init__.py in <module>()
----> 1 from .core import cachier
      2 from ._version import get_versions
      3 __version__ = get_versions()['version']
      4 del get_versions

c:\users\x\appdata\local\programs\python\python35\lib\site-packages\cachier\core.py in <module>()
     26     from concurrent.futures import ThreadPoolExecutor
     27 import time   # to sleep when waiting on Mongo cache
---> 28 import fcntl  # to lock on pickle cache IO
     29
     30 import pymongo

ImportError: No module named 'fcntl'

If you opt not to fix it you should mention it in the README and setup files

enhancement

opened by tburette 7

Several pickle files

I'm pretty sure I fixed the test that wasn't passing, it was a bug in the test of course. If you can run the tests again I would be very glad (like the previous time when I PR the tests don't seem to run properly).

opened by erap129 6
stale_after attribute not used anywhere?

It seems that the stale_after attribute in BaseCore is not used anywhere. Would it make sense to remove it as an attribute?

https://github.com/shaypal5/cachier/blob/2663f09f72a6256cfa80cfe92054949e33baa0f8/cachier/base_core.py#L16

opened by louismartin 0

Override stale after

Allows user to override the stale_after parameter when calling the function. I have a use case where I cache http get requests but I want to have different stale durations depending on the endpoint that I am calling.

Usage

from datetime import timedelta
import time

from cachier import cachier


@cachier(stale_after=timedelta(days=1), pickle_reload=False)
def test():
    print("Inside function")
    return 1


print("First run")
test()
print("Second run")
test()
time.sleep(1)
print("Third run with overriden stale_after")
test(stale_after=timedelta(seconds=1))

First run
Inside function
Second run
Third run with overriden stale_after
Inside function

Tests

Tests in tests/test_pickle_core.py pass

opened by louismartin 3

Cannot use `cachier` with `tqdm.contrib.concurrent.process_map` or `thread_map`

Hi, When using @cachier with tqdm utility for multiprocessing or multithreading, cachier always get stuck in what seems to be an infinite loop. I tried playing with the decorator args, but without success.
bug

opened by NicolasMICAUX 3
Allow concurrent cache reads with pickle core

I have a use case where my cache is large, rarely written to, but read by many different threads at the same time thus causing timeout errors because only one thread can read the cache at the same time.

This PR allows multiple threads to read the cache using a shared lock but still preserve the thread-safety by requesting an exclusive lock on cache writes.

Therefore if multiple threads want to read the cache then they can do so simultaneously, but if only a single thread wants to write to it, then it needs to acquire an exclusive lock that prevents all other threads to read or write to the cache.

I tested locally for my use case, but would that potentially cause other issues that I didn't think of?
enhancement

opened by louismartin 10
Consider using `functools` for argument hashing
Looking at how arg and kwarg hashing is done now, it is a little different from how the stdlib does it with functools.cache (see https://github.com/python/cpython/blob/3.10/Lib/functools.py#L448)

I think this builtin method is

more efficient

more bugfree (note the kwd_mark that separates the args from kwargs)

better maintained

What about switching to just using functools._make_key as the default hash function? Important to consider if the order of kwargs is important, they decided it was NOT, but I think swapping to this would be breaking for users here.

As a side note, monogo_core.get_entry() and memory_core.get_entry() have duplicated hashing logic that I think could get factored out?
enhancement
opened by NickCrews 1
Multiple concurrent writers (and readers) with shared NFS mount

I'm seeing an issue with cachier where I'm getting Bad File Descriptor errors. Just want to make sure I'm not completely abusing it here, I have multiple writers (and readers) accessing a shared cache directory via NFS.

Am I way off the mark here for thinking this would ever work correctly?
bug question complex issue

opened by matthewcummings 5

v2.0.0(Dec 6, 2022)

Cachier now supports pickle-based caching for cases where two identically-named methods of different classes are defined in the same module. This changes the naming convention for the pickle files in which the per-function caches are saved, and so it a breaking change.
Source code(tar.gz)
Source code(zip)
v1.5.4(Nov 25, 2021)

Source code(tar.gz)
Source code(zip)
v1.5.3(Oct 10, 2021)
Released features:

A new in-memory core.

Multi-file cache for pickle, with a file per-argument set, to handle large cache files.

The wait_for_calc_timeout option for MongoDB cores.

iNotify bug fix for pickle cores.

Source code(tar.gz)
Source code(zip)
v1.5.0(Nov 26, 2020)

Source code(tar.gz)
Source code(zip)
v1.4.3(Sep 7, 2020)

Source code(tar.gz)
Source code(zip)
v1.4.2(Mar 20, 2020)

Source code(tar.gz)
Source code(zip)
v1.4.1(Feb 20, 2020)

Source code(tar.gz)
Source code(zip)
v1.4.0(Feb 20, 2020)

Source code(tar.gz)
Source code(zip)
v1.3.3(Feb 8, 2020)

Source code(tar.gz)
Source code(zip)
v1.3.0(Feb 8, 2020)

Source code(tar.gz)
Source code(zip)
v1.2.8(Dec 3, 2019)

Source code(tar.gz)
Source code(zip)
v1.2.7(Dec 3, 2019)

Source code(tar.gz)
Source code(zip)
v1.2.5(Jan 29, 2019)

Source code(tar.gz)
Source code(zip)
v1.2.1(Apr 29, 2017)

Including minor documentation fixes.
Source code(tar.gz)
Source code(zip)
v1.2.0(Apr 29, 2017)

Source code(tar.gz)
Source code(zip)