Installing
$ pip install "thumbor>=7.0.0"
$ thumbor-doctor
Keep on reading for the changes you need to be aware of. If you experience any problems, please run thumbor-doctor --nocolor and create an issue with your problem and the results of running thumbor-doctor.
Introduction
This release notes will be slightly different from others as we need to have more of a conversation on what happened. We'll detail the changes below, but first let's explain them.
We set out to update thumbor to work with python3. During the work required to do that, it became clear that we were going to break most extensions with this release.
At this point we made a decision to not only update thumbor to python3, but completely modernize it.
Some of the 7.0.0 features:
- Python 3.6+ support. Python 2 is not supported anymore (does not even work).
- Updated tornado to release 6.0.3. This should fix many bugs/vulnerabilities we still had due to old tornado.
- Updated pillow version to >= 7.0.0, <9 (meaning you should get latest pillow as it is in 8.4.0).
- All tests now run using py.test.
- Updated every dependency we could update.
While updating the codebase to support new tornado and python 3 we also decided to get rid of the callback hell that tormented the committers for the last 10 years or so. The new codebase uses proper async/await idioms for all the asynchronous code.
As a bonus for this update, we added a new concept to customizing thumbor: handler lists. It has been a request for a long time to allow thumbor extensions to add new handlers to thumbor. Well, now you can!
Now that you understand why we broke compatibility with many constructs previously in place, be assured that the committers worked hard to make the transition as easy as possible. We added a compatibility layer to ease the transition to the new way of writing Loaders, Result Storages and Storages. This means you get to use your current extensions as long as they are python 3 compatible. We even monkey-patched tornado (very safely) to ensure these extensions can keep using return_future as it has been removed from tornado entirely.
Last, but not least, we decided not to support python 2 anymore. We'll keep updating the 6.7.x branch with bug fixes, but new features are going to be focused on python3 and thumbor7.
Lets get to the details!
Breaking Changes
First let's see what breaks (apart from not supporting python 2, that is).
Storages
Previously storages were implemented like this (this is NoStorage):
class Storage(BaseStorage):
def put(self, path, bytes):
return path
def put_crypto(self, path):
return path
def put_detector_data(self, path, data):
return path
@return_future
def get_crypto(self, path, callback):
callback(None)
@return_future
def get_detector_data(self, path, callback):
callback(None)
@return_future
def get(self, path, callback):
callback(None)
@return_future
def exists(self, path, callback):
callback(False)
def remove(self, path):
pass
With release 7.0.0 we modernize these functions to be async. The same storage in the new release:
class Storage(BaseStorage):
async def put(self, path, file_bytes):
return path
async def put_crypto(self, path):
return path
async def put_detector_data(self, path, data):
return path
async def get_crypto(self, path):
return None
async def get_detector_data(self, path):
return None
async def get(self, path):
return None
async def exists(self, path):
return False
async def remove(self, path):
pass
Much simpler, right? No funky callback business. The downside is that all storages out there need to be updated. It should be simply a matter of "asyncifying" them.
That said, if you can't (or don't know how) to update them, you can use the compatibility layer provided by thumbor to aid you:
thumbor.conf
# how to store the loaded images so we don't have to load
# them again with the loader
STORAGE = 'thumbor.compatibility.storage'
## Storage that will be used with the compatibility layer, instead of the
## compatibility storage. Please only use this if you can't use up-to-date
## storages.
## Defaults to: None
COMPATIBILITY_LEGACY_STORAGE = 'tc_aws.storages.s3_storage'
With this configuration, thumbor will understand how to call the methods with callbacks from previous releases.
Loaders
The same applies to loaders. Let's see what a loader looked like in 6.7.2:
@return_future
def load(context, path, callback):
# does a lot of stuff and :(
callback(loaded_image)
Now in 7.0.0 it is a simple async function:
async def load(context, path):
# does a lot of stuff with no callbacks whatsoever
return loader_result
It's much simpler to understand the new loaders (and to create new ones), but the same issue as above: previous loaders won't work with thumbor 7.0.0+ (until asyncified).
Important note: going forward, loaders should return LoaderResult object. While returning plain image bytes still works, it's encouraged to update loaders to return object.
Once again, if you can't (or don't know how) to update them, you can use the compatibility layer provided by thumbor to aid you:
thumbor.conf
LOADER = 'thumbor.compatibility.loader'
## Loader that will be used with the compatibility layer, instead of the
## compatibility loader. Please only use this if you can't use up-to-date
## loaders.
## Defaults to: None
# COMPATIBILITY_LEGACY_LOADER = 'tc_aws.loaders.s3_loader'
With this configuration, thumbor will understand how to call the methods with callbacks from previous releases.
Result Storages
You get the gist, right? NoStorage Result Storage in 6.7.2:
class Storage(BaseStorage):
# THIS METHOD IS NOT EVEN ASYNC :(
def put(self, bytes):
return ''
@return_future
def get(self, callback):
callback(None)
Now in 7.0.0+:
class Storage(BaseStorage):
async def put(self, image_bytes):
return ""
async def get(self):
return None
Previous result storages won't work with thumbor 7.0.0+ (until asyncified).
Getting repetitive here, but if you can't (or don't know how) to update them, you can use the compatibility layer provided by thumbor to aid you:
thumbor.conf
RESULT_STORAGE = 'thumbor.compatibility.result_storage'
## Result Storage that will be used with the compatibility layer, instead of the
## compatibility result storage. Please only use this if you can't use up-to-
## date result storages.
## Defaults to: None
COMPATIBILITY_LEGACY_RESULT_STORAGE = 'tc_aws.result_storages.s3_storage'
With this configuration, thumbor will understand how to call the methods with callbacks from previous releases.
Tornado Compatibility
If any of the compatibility extensions are used (storage, loader, result storage), then thumbor verifies whether tornado has a tornado.concurrent.return_future decorator. If it does, then thumbor won't change it. If it's not there, we add a decorator that won't do anything.
The intention here is to ensure that previous extensions keep working with the compatibility layer.
Please understand that the compatibility layer will NOT:
- Update your extension to be python 3 compliant;
- Make your extension non-blocking. If you have blocking methods, they'll keep blocking thumbor anyway.
Filters
Same ol', same ol'. Let's check format filter in 6.7.2.
class Filter(BaseFilter):
@filter_method(BaseFilter.String)
def format(self, format):
if format.lower() not in ALLOWED_FORMATS:
logger.debug('Format not allowed: %s' % format.lower())
self.context.request.format = None
else:
logger.debug('Format specified: %s' % format.lower())
self.context.request.format = format.lower()
Now the same filter in 7.0.0:
class Filter(BaseFilter):
@filter_method(BaseFilter.String)
async def format(self, file_format):
if file_format.lower() not in ALLOWED_FORMATS:
logger.debug("Format not allowed: %s", file_format.lower())
self.context.request.format = None
else:
logger.debug("Format specified: %s", file_format.lower())
self.context.request.format = file_format.lower()
Hardly noticeable, right? Except now you get to run async code in your filters without arcane magic or callbacks. As with all the others, previous filters won't work with thumbor 7.0.0+ (until asyncified).
For this and detectors (below) we do not provide compatibility layer as we understand them to be easier to update. If you have a scenario that requires these, let us know.
Detectors
These were also modernized. Let's take a look at our face detector in 6.7.2:
class Detector(CascadeLoaderDetector):
# details removed for clarity
def detect(self, callback):
features = self.get_features()
if features:
for (left, top, width, height), neighbors in features:
top = self.__add_hair_offset(top, height)
self.context.request.focal_points.append(
FocalPoint.from_square(left, top, width, height, origin="Face Detection")
)
callback()
else:
self.next(callback)
Now the same detector in 7.0.0+:
class Detector(CascadeLoaderDetector):
# details removed for clarity
async def detect(self):
features = self.get_features()
if features:
for (left, top, width, height), _ in features:
top = self.__add_hair_offset(top, height)
self.context.request.focal_points.append(
FocalPoint.from_square(
left, top, width, height, origin="Face Detection"
)
)
return
await self.next()
No more callbacks and now detector pipeline works by awaiting the next detector or just a plain early return to stop the pipe. As with the other updates, previous detectors will not work with thumbor 7.0.0 (until asyncified).
For detectors we do not provide compatibility layer as we understand them to be easier to update. If you have a scenario that requires these, let us know.
Improvements
Thumbor Testing Tools
This one is for you, an extension author! Thumbor now includes a thumbor.testing module that ships with it and allows extensions to create tests that get thumbor up and running easier. An example test from our code:
from preggy import expect
from tornado.testing import gen_test
from thumbor.testing import TestCase
from thumbor.storages.no_storage import Storage as NoStorage
class NoStorageTestCase(TestCase):
def get_image_url(self, image):
return "s.glbimg.com/some/{0}".format(image)
@gen_test
async def test_store_image_should_be_null(self):
iurl = self.get_image_url("source.jpg")
storage = NoStorage(None)
stored = await storage.get(iurl)
expect(stored).to_be_null()
# many more tests!
Notice how the test can now test only what it needs to. We're using py.test now and that allows async test methods.
WARNING: If you use thumbor.testing you need to also add "mock==3.,>=3.0.5" and "pyssim==0.,>=0.4.0" as dev dependencies of your package, as these are used in thumbor.testing. We didn't add these as dependencies for thumbor as it would require every thumbor user to also install them alongside thumbor and they are only useful for testing.
Revised Docs
The many fundamental changes in thumbor prompted a major review of the docs and improvement of many areas in it. Please help us improve the docs if you feel like contributing.
Handler Lists
Not all is breaking changes, though. Now you get to extend thumbor with new handlers! And it is as simple as creating a module with:
from typing import Any, cast
from thumbor.handler_lists import HandlerList
from my.handlers.index import IndexHandler # This is your code <--
def get_handlers(context: Any) -> HandlerList:
something_enabled = cast(bool, self.context.config.SOMETHING_ENABLED)
if not something_enabled:
return []
return [
(r"/my-url/?", IndexHandler, {"context": self.context}),
]
Then including it in thumbor.conf:
from thumbor.handler_lists import BUILTIN_HANDLERS
# Two things worth noticing here:
# 1) The handler list order indicates precedence, so whatever matches first will be executed;
# 2) Please do not forget thumbor's built-ins or you'll kill thumbor functionality.
HANDLER_LISTS = BUILTIN_HANDLERS + [
"my.handler_list',
]
Acknowledgements
This release would not be possible without the work of thumbor's commiters. I'd like to call out the work of @kkopachev that has enabled me (@heynemann) to move forward with this release.
Again thanks @kkopachev for all the code reviews and thanks the whole of thumbor community for catching errors when I hadn't even written the release notes! That's nothing short of amazing.
Now onto the not comprehensive list of changes in this release!
Fixes/Features
- Fix typo (thanks @timgates42);
- Python 3 support (thanks @kkopachev);
- Removed callbacks in favor of async-await;
- Simplified pillow feature checks (thanks @kkopachev);
- Extensive work to improve codebase quality (lint errors should be few now);
- Using stdlib which instead of a custom one (thanks @kkopachev);
- Major documentation review;
- Added handler lists to allow further extension;
- Update classifiers and add python_requires to help pip (thanks @hugovk);
- Ability to run thumbor in multiprocess mode (thanks @kkopachev);
- Python 3.6 support (thanks @amanagr);
- New
thumbor-doctor command to help diagnose issues with installs;
- New compatibility layer for using legacy storages, loaders and result storages.
Contribution List
- Asyncawait2 by @heynemann in https://github.com/thumbor/thumbor/pull/1257
- [IMPR] Use stdlib's shutil.which instead of custom one by @kkopachev in https://github.com/thumbor/thumbor/pull/1262
- [IMPR] Revising docs for 7.0.0 release by @heynemann in https://github.com/thumbor/thumbor/pull/1267
- [FEAT] URL Routers support by @heynemann in https://github.com/thumbor/thumbor/pull/1261
- [FIX] Ability to listen on unix sockets by @kkopachev in https://github.com/thumbor/thumbor/pull/1272
- [IMPR] Performance tests by @heynemann in https://github.com/thumbor/thumbor/pull/1273
- Update classifiers and add python_requires to help pip by @hugovk in https://github.com/thumbor/thumbor/pull/1283
- [FEAT] Ability to run thumbor in multiprocess mode by @kkopachev in https://github.com/thumbor/thumbor/pull/1276
- Add Python 3.6 Support to thumbor library. by @amanagr in https://github.com/thumbor/thumbor/pull/1282
- [FEAT] thumbor-doctor by @heynemann in https://github.com/thumbor/thumbor/pull/1284
- cherry-picked [FIX] Ability to listen to a socket passed as file descriptor by @kkopachev in https://github.com/thumbor/thumbor/pull/1287
- [IMPR] Add docs describing how to run thumbor multiprocess by @kkopachev in https://github.com/thumbor/thumbor/pull/1288
- Update libraries.rst by @marlonnardi in https://github.com/thumbor/thumbor/pull/1295
- Minor documentation typo by @sunny in https://github.com/thumbor/thumbor/pull/1303
- Pin py3exiv2 to a version < 0.8.0 by @znerol in https://github.com/thumbor/thumbor/pull/1318
- [DOCS] Update instructions on installing OSX deps by @scorphus in https://github.com/thumbor/thumbor/pull/1320
- Make python binary configurable in Makefile by @gi11es in https://github.com/thumbor/thumbor/pull/1328
- [FIX] Make fill with blur engine-agnostic by @scorphus in https://github.com/thumbor/thumbor/pull/1323
- [FEAT] Improvement of our build and CircleCI by @heynemann in https://github.com/thumbor/thumbor/pull/1341
- [FIX] Exclude py3exiv2 version 0.9.3 by @scorphus in https://github.com/thumbor/thumbor/pull/1340
- Allow remotecv 3.x by @gi11es in https://github.com/thumbor/thumbor/pull/1331
- Fix HTTP 500 when request path points to directory by @f90d83a8 in https://github.com/thumbor/thumbor/pull/1329
- Upgrade pillow and pytz requirements by @andersk in https://github.com/thumbor/thumbor/pull/1321
- UPLOAD_MAX_SIZE is calculated in bytes by @istovatis in https://github.com/thumbor/thumbor/pull/1315
- add timings and counters for Smart and None Smart option by @mohammedelhakim in https://github.com/thumbor/thumbor/pull/1251
- Configure GitHub Actions by @scorphus in https://github.com/thumbor/thumbor/pull/1324
- [fix] Lint fixes by @heynemann in https://github.com/thumbor/thumbor/pull/1343
- [FIX] Add pylint to GitHub Actions workflow too by @scorphus in https://github.com/thumbor/thumbor/pull/1344
- correct typo in usage.rst by @dessert1 in https://github.com/thumbor/thumbor/pull/1355
- [Fix] Docs: code formatting error and link error by @ThesllaDev in https://github.com/thumbor/thumbor/pull/1369
- DOCS readme correction by @christopherdarling in https://github.com/thumbor/thumbor/pull/1368
- [IMPR] Compatibility loader,storage,res storage by @heynemann in https://github.com/thumbor/thumbor/pull/1342
- fix(gifv): Adding webm required bitrate parameter by @guilhermef in https://github.com/thumbor/thumbor/pull/1377
- Fix/lint by @heynemann in https://github.com/thumbor/thumbor/pull/1378
- Updated min pillow version by @heynemann in https://github.com/thumbor/thumbor/pull/1376
New Contributors
- @hugovk made their first contribution in https://github.com/thumbor/thumbor/pull/1283
- @amanagr made their first contribution in https://github.com/thumbor/thumbor/pull/1282
- @marlonnardi made their first contribution in https://github.com/thumbor/thumbor/pull/1295
- @sunny made their first contribution in https://github.com/thumbor/thumbor/pull/1303
- @f90d83a8 made their first contribution in https://github.com/thumbor/thumbor/pull/1329
- @istovatis made their first contribution in https://github.com/thumbor/thumbor/pull/1315
- @mohammedelhakim made their first contribution in https://github.com/thumbor/thumbor/pull/1251
- @dessert1 made their first contribution in https://github.com/thumbor/thumbor/pull/1355
- @ThesllaDev made their first contribution in https://github.com/thumbor/thumbor/pull/1369
- @christopherdarling made their first contribution in https://github.com/thumbor/thumbor/pull/1368
Full Changelog: https://github.com/thumbor/thumbor/compare/6.7.1...7.0.0
Pypi release: https://pypi.org/project/thumbor/7.0.0/
Source code(tar.gz)
Source code(zip)
The thumbor-6.4.2 RPM is available into AXIVO repository. We use 
from https://newsroom.notified.com/gul-pr/media/68282/vilda-rosenblad-foto-anna-lena-ahlstromjpg)
1.5k Jan 1, 2023
5 May 30, 2022
1 Dec 18, 2021
4 Jan 28, 2022
350 Jan 2, 2023
7 Oct 29, 2022
93 Dec 30, 2022
1 Jan 2, 2022
6 Aug 5, 2022
44 Nov 17, 2022
53 Dec 29, 2022
39 Nov 25, 2022
2 May 1, 2022
3 Sep 26, 2022
12.9k Jan 7, 2023
1 Feb 13, 2022
9.3k Dec 31, 2022