Thumbor-bootcamp - learning and contribution experience with ❤️ and 🤗 from the thumbor team

I am using virtualenv and also pyenv. My python version is 3.10.2 and I am using macOS. I already updated the version of pip and the setuptools.

When I run pip install - r requirements I get this error, for all the packages that is in the requirement file:

When I run ``make setup``` I get this error:

Could you help me with this issue ?

Support Question

What's going on?

I was trying to run make test and it gave me an error .

❯ make test
running build_ext
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_alpha.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_saturation.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_fill.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_noise.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_brightness.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_round_corner.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_sharpen.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_bounding_box.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_composite.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_convolution.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_nine_patch.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_contrast.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_colorize.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_rgb.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_equalize.cpython-310-darwin.so -> thumbor/ext/filters
copying build/lib.macosx-12.0-arm64-3.10/thumbor/ext/filters/_curve.cpython-310-darwin.so -> thumbor/ext/filters
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
# Server
redis_version:6.2.6
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:c6f3693d1aced7d9
redis_mode:standalone
os:Darwin 21.1.0 arm64
arch_bits:64
multiplexing_api:kqueue
atomicvar_api:c11-builtin
gcc_version:4.2.1
process_id:44749
process_supervised:no
run_id:6b364404d15f04844a4a2bb9111725797aadbc88
tcp_port:6668
server_time_usec:1643899349835257
uptime_in_seconds:1
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:16509397
executable:/Users/nicolasgabriel/code/open-source/thumbor/redis-server
config_file:/Users/nicolasgabriel/code/open-source/thumbor/redis.conf
io_threads_active:0

# Clients
connected_clients:1
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# Memory
used_memory:1113968
used_memory_human:1.06M
used_memory_rss:3342336
used_memory_rss_human:3.19M
used_memory_peak:1113968
used_memory_peak_human:1.06M
used_memory_peak_perc:100.18%
used_memory_overhead:1028448
used_memory_startup:1028448
used_memory_dataset:85520
used_memory_dataset_perc:100.00%
allocator_allocated:1028448
allocator_active:3304448
allocator_resident:3304448
total_system_memory:8589934592
total_system_memory_human:8.00G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:3.21
allocator_frag_bytes:2276000
allocator_rss_ratio:1.00
allocator_rss_bytes:0
rss_overhead_ratio:1.01
rss_overhead_bytes:37888
mem_fragmentation_ratio:3.25
mem_fragmentation_bytes:2313888
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:0
mem_aof_buffer:0
mem_allocator:libc
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1643899348
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:1
total_commands_processed:1
instantaneous_ops_per_sec:0
total_net_input_bytes:41
total_net_output_bytes:5
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:2
total_writes_processed:1
io_threaded_reads_processed:0
io_threaded_writes_processed:0

# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:deebe7483d165e6fda9c5088764029551b9d947b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:0.004316
used_cpu_user:0.001417
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000

# Modules

# Errorstats

# Cluster
cluster_enabled:0

# Keyspace
ImportError while loading conftest '/Users/nicolasgabriel/code/open-source/thumbor/tests/conftest.py'.
tests/conftest.py:5: in <module>
    import tests.base  # NOQA
tests/base.py:18: in <module>
    from thumbor.testing import DetectorTestCase as ThumborDetectorTestCase
thumbor/testing.py:22: in <module>
    from thumbor.app import ThumborServiceApp
thumbor/app.py:15: in <module>
    from thumbor.handlers.imaging import ImagingHandler
thumbor/handlers/__init__.py:24: in <module>
    from thumbor.engines import BaseEngine, EngineResult
thumbor/engines/__init__.py:23: in <module>
    import cairosvg
../../../.venv/thumborenv/lib/python3.10/site-packages/cairosvg/__init__.py:26: in <module>
    from . import surface  # noqa isort:skip
../../../.venv/thumborenv/lib/python3.10/site-packages/cairosvg/surface.py:9: in <module>
    import cairocffi as cairo
../../../.venv/thumborenv/lib/python3.10/site-packages/cairocffi/__init__.py:48: in <module>
    cairo = dlopen(
../../../.venv/thumborenv/lib/python3.10/site-packages/cairocffi/__init__.py:45: in dlopen
    raise OSError(error_message)  # pragma: no cover
E   OSError: no library called "cairo-2" was found
E   no library called "cairo" was found
E   no library called "libcairo-2" was found
E   cannot load library 'libcairo.so.2': dlopen(libcairo.so.2, 0x0002): tried: 'libcairo.so.2' (no such file), '/usr/local/lib/libcairo.so.2' (no such file), '/usr/lib/libcairo.so.2' (no such file), '/Users/nicolasgabriel/code/open-source/thumbor/libcairo.so.2' (no such file), '/usr/local/lib/libcairo.so.2' (no such file), '/usr/lib/libcairo.so.2' (no such file)
E   cannot load library 'libcairo.2.dylib': dlopen(libcairo.2.dylib, 0x0002): tried: 'libcairo.2.dylib' (no such file), '/usr/local/lib/libcairo.2.dylib' (no such file), '/usr/lib/libcairo.2.dylib' (no such file), '/Users/nicolasgabriel/code/open-source/thumbor/libcairo.2.dylib' (no such file), '/usr/local/lib/libcairo.2.dylib' (no such file), '/usr/lib/libcairo.2.dylib' (no such file)
E   cannot load library 'libcairo-2.dll': dlopen(libcairo-2.dll, 0x0002): tried: 'libcairo-2.dll' (no such file), '/usr/local/lib/libcairo-2.dll' (no such file), '/usr/lib/libcairo-2.dll' (no such file), '/Users/nicolasgabriel/code/open-source/thumbor/libcairo-2.dll' (no such file), '/usr/local/lib/libcairo-2.dll' (no such file), '/usr/lib/libcairo-2.dll' (no such file)
make[1]: *** [unit] Error 4
make: *** [test] Error 2

I seems that one of the required packages was not found, I tried to install it but it raises this error bellow:

ERROR: Could not find a version that satisfies the requirement cairo-2 (from versions: none)
ERROR: No matching distribution found for cairo-2

Someone can help me ?

How do you want to be called: Arielle Pronouns: She/her

What do you expect to take from this experience: I'm interested in getting started with contributing to open source and getting a chance to gain exposure to new tools and projects :)

Brief Summary of You (career, skills, areas of interest): 2.5 years of experience as a software engineer working on web applications. I'm a full stack engineer but currently more focused on backend work.

Daily Availability (hs): about 5 hours per week

How do you want to be called: Cristiano Pronouns: (e.g. He/Him/His)

What do you expect to take from this experience: I stopped contributing to open source for while and want to enjoy this opportunity to get closer to this environment. Also, I really like to work with python and improve my skills, face situations to extract more from language, etc.

Brief Summary of You (career, skills, areas of interest): I am a software engineer with 7y of experience with product development processes and systems design. I already worked in multiple Computer Science fields and really interested to face new, big and complex challenges.

Daily Availability (hs): 4-5hrs a week

How do you want to be called: nicolastaveira Pronouns: (e.g. He/Him/His) He/Him/His

What do you expect to take from this experience:

• I want to learn more about how an open-source project works and level up my python skills.
• I want to make networking with great engineers because I believe that networking is what moves the world.
• One of my goals also is to work more with technologies for distributed systems like docker, redis, Kubernetes, distributed event streaming platform, etc.
• I believe that working in open-source projects will develop skills necessary to be an Engineer Manager in the future.

Brief Summary of You (career, skills, areas of interest): 3y of software engineering, mostly in the back-end but I like to understand about the product and the features end-to-end, because of this I am also interested in front-end. My favorite language is python and one of my strengths as a software engineer is my knowledge about data structures and algorithms and my communication skills.

Daily Availability (hs): (e.g. 2h per day to work on Thumbor, or 5h a week, just an idea) 4 hours week

Areas of Expertise

open-source contributions

Summary

There's a cool project called thumbor-padding that needs to be upgraded to thumbor 7 (Thumbor 7 release notes - Look at the filters part!)

Involved Modules

List the modules the bootcamper has to interact with, like:

• thumbor-padding - padding.py

Task Relevance

Expands the list of projects with thumbor 7 compatibility as well as add a nice capability for users.

Further Details

• First you should "asyncify" the filter in the repository
• Then write tests to verify that the filter is indeed working
• Then do a PR to the repository owner (@heynemann can approve)
• Then we release a new version of thumbor-padding and 💰profit!

Proposed Solution

Just need to transform the filter into an async function as well as fix any issues in the codebase with python 3.

Remember to create a new virtualenv, install all the dependencies (usually can be done with pip install -e .[tests] but the setup.py for this project also needs to be updated. You can check thumbor's setup.py for that).

How to complete this task?

To complete this task, follow this workflow:

1. Fork the involved repositories
2. In each repository there's a documentation on how to install dependencies and initialize your environment
3. Hack, in no particular order:
   • Write code & tests
   • Write new tests
   • Write docs
   • Improve design
   • Check that all tests pass
   • Repeat until you're satisfied
4. Submit a pull request.

Areas of Expertise

thumbor, open-source, testing, python, maintainability

Creating your own open source project will teach a lot of the concepts required for participating in another's open source project. Great experience!

Summary

Create a new project thumbor-upload in thumbor's org and move the upload functionality into it. Once it has been moved, create a L2 task here to remove upload from thumbor's codebase (or do it in two PRs yourself).

Involved Modules

• thumbor - upload

Don't forget to also check the tests and docs on uploading.

Task Relevance

Removing code from thumbor's codebase is as important as adding. The leaner the project is, the faster we can move and onboard new people.

Further Details

When you pick up this task, tag @heynemann to create the project for you and add you to the project team of contributors.

Then:

• Create the structure to run tests
• Add tests
• Add code
• Add github action to run the code on commit
• Add linting
• Add linting to the github action
• Add a nice README explaining how to use this new extensions
• Tag a version 0.1.0 (git tag 0.1.0 && git push --tags)
• Ask @heynemann to submit to pypi and he'll add you to the list of maintainers.

Task Workflow

To complete this task, follow this workflow:

1. Fork the involved repositories
2. In each repository there's a documentation on how to install dependencies and initialize your environment
3. Hack, in no particular order:
   • Write code & tests
   • Write new tests
   • Write docs
   • Improve design
   • Check that all tests pass
   • Repeat until you're satisfied
4. Submit a pull request.

Areas of Expertise

Python

Summary

Change the thumbor command to use click.

Involved Modules

• thumbor - server

Task Relevance

By upgrading the handling of the cli parts of thumbor to a more established library like click we ensure maintainability for the future.

Further Details

Completing this task means the thumbor command in thumbor is handled by click and not by optparse (deprecated module in Python 3).

How to complete this task?

To complete this task, follow this workflow:

1. Fork the involved repositories
2. In each repository there's a documentation on how to install dependencies and initialize your environment
3. Hack, in no particular order:
   • Write code & tests
   • Write new tests
   • Write docs
   • Improve design
   • Check that all tests pass
   • Repeat until you're satisfied
4. Submit a pull request.

Areas of Expertise

Python

Summary

Change the thumbor-doctor command to use click.

Involved Modules

• thumbor - url_composer

Task Relevance

By upgrading the handling of the cli parts of thumbor to a more established library like click we ensure maintainability for the future.

Further Details

Completing this task means the thumbor-doctor command in thumbor is handled by click and not by optparse (deprecated module in Python 3).

How to complete this task?

To complete this task, follow this workflow:

1. Fork the involved repositories
2. In each repository there's a documentation on how to install dependencies and initialize your environment
3. Hack, in no particular order:
   • Write code & tests
   • Write new tests
   • Write docs
   • Improve design
   • Check that all tests pass
   • Repeat until you're satisfied
4. Submit a pull request.

Areas of Expertise

Python

Summary

Change the thumbor-url command to use click.

Involved Modules

• thumbor - url_composer

Task Relevance

By upgrading the handling of the cli parts of thumbor to a more established library like click we ensure maintainability for the future.

Further Details

Completing this task means the thumbor-url command in thumbor is handled by click and not by optparse (deprecated module in Python 3).

How to complete this task?

To complete this task, follow this workflow:

1. Fork the involved repositories
2. In each repository there's a documentation on how to install dependencies and initialize your environment
3. Hack, in no particular order:
   • Write code & tests
   • Write new tests
   • Write docs
   • Improve design
   • Check that all tests pass
   • Repeat until you're satisfied
4. Submit a pull request.

Areas of Expertise

MyPy, static typing

You will learn more about how to use statically typed python code. mypy is an optional static type checker for python. It provides compile-time type checking.

Summary

In this task you'll fix thumbor/context.py to be typed.

Involved Modules

• thumbor - thumbor/context.py

Task Relevance

Providing type safety is important both for thumbor and the projects that depend on it. This way we can eliminate type errors, like None where you don't expect None or an integer where you expect a string.

How to complete this task?

To complete this task, read mypy docs on how to get started and run mypy thumbor/context.py from thumbor's virtualenv.

When creating this task, this was the output of running mypy thumbor/context.py:

../thumbor/thumbor/context.py:31: error: Function is missing a type annotation  [no-untyped-def]
../thumbor/thumbor/context.py:41: error: Incompatible types in assignment (expression has type "None", variable has type "ContextImporter")  [assignment]
../thumbor/thumbor/context.py:64: error: Function is missing a type annotation  [no-untyped-def]
../thumbor/thumbor/context.py:67: error: Function is missing a type annotation  [no-untyped-def]
../thumbor/thumbor/context.py:75: error: Function is missing a type annotation  [no-untyped-def]
../thumbor/thumbor/context.py:104: error: Function is missing a return type annotation  [no-untyped-def]
../thumbor/thumbor/context.py:108: error: Function is missing a type annotation  [no-untyped-def]
../thumbor/thumbor/context.py:111: error: Function is missing a return type annotation  [no-untyped-def]
../thumbor/thumbor/context.py:111: note: Use "-> None" if function does not return a value
../thumbor/thumbor/context.py:131: error: Function is missing a type annotation  [no-untyped-def]
../thumbor/thumbor/context.py:232: error: Function is missing a type annotation  [no-untyped-def]
../thumbor/thumbor/context.py:237: error: Function is missing a type annotation  [no-untyped-def]
../thumbor/thumbor/context.py:281: error: Function is missing a return type annotation  [no-untyped-def]
../thumbor/thumbor/context.py:281: note: Use "-> None" if function does not return a value
Found 12 errors in 1 file (checked 1 source file)

This gives you all errors in the file you will fix. Don't worry if there are types of other packages (other thumbor files or thumbor dependencies) failing. If that's the case we can ignore these for now using this kind of annotation:

import module_without_mypy  # type: ignore

For more details check this page.

Once you get to a point where running mypy thumbor/context.py returns no errors, go ahead and submit a PR. The output should look like this:

Success: no issues found in 1 source file

Task Workflow

The workflow for completing tasks in thumbor goes like this:

1. Fork the involved repositories
2. In each repository there's a documentation on how to install dependencies and initialize your environment
3. Hack, in no particular order:
   • Write code & tests
   • Write new tests
   • Write docs
   • Improve design
   • Check that all tests pass
   • Repeat until you're satisfied
4. Submit a pull request.