Hey Adam,

Thanks for this lib - looks very useful - unfortunately I wanted to fold it into a platform I work on and I got an error that might raise its head for other users - but it's not an implementation issue on your side, more a general 'Apple M1 Docker Stuff' observation that I suspect will eventually be addressed somewhere far away from this library

In short, I think that watchfiles itself craps out on Dockerised environments that are running on Apple's M1 Silicon.

I installed the library and rebuilt my Docker containers, but got a 502 bringing the development server up. I ran docker logs for the container and it gave me the following traceback:

Traceback (most recent call last):
  File "/app/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python3.10/dist-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3.10/dist-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python3.10/dist-packages/django/core/management/base.py", line 414, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/lib/python3.10/dist-packages/django/core/management/commands/runserver.py", line 74, in execute
    super().execute(*args, **options)
  File "/usr/lib/python3.10/dist-packages/django/core/management/base.py", line 460, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python3.10/dist-packages/django/core/management/commands/runserver.py", line 111, in handle
    self.run(**options)
  File "/usr/lib/python3.10/dist-packages/django/core/management/commands/runserver.py", line 118, in run
    autoreload.run_with_reloader(self.inner_run, **options)
  File "/usr/lib/python3.10/dist-packages/django/utils/autoreload.py", line 680, in run_with_reloader
    start_django(reloader, main_func, *args, **kwargs)
  File "/usr/lib/python3.10/dist-packages/django/utils/autoreload.py", line 661, in start_django
    reloader.run(django_main_thread)
  File "/usr/lib/python3.10/dist-packages/django/utils/autoreload.py", line 344, in run
    self.run_loop()
  File "/usr/lib/python3.10/dist-packages/django/utils/autoreload.py", line 350, in run_loop
    next(ticker)
  File "/usr/lib/python3.10/dist-packages/django_watchfiles/__init__.py", line 21, in tick
    for file_changes in watcher:
  File "/usr/lib/python3.10/dist-packages/watchfiles/main.py", line 85, in watch
    watcher = RustNotify([str(p) for p in paths], debug)
_rust_notify.WatchfilesRustInternalError: Error creating watcher: Function not implemented (os error 38)

The pertinent part of this seems to be:

  File "/usr/lib/python3.10/dist-packages/watchfiles/main.py", line 85, in watch
    watcher = RustNotify([str(p) for p in paths], debug)
_rust_notify.WatchfilesRustInternalError: Error creating watcher: Function not implemented (os error 38)

Which basically notes that in watchfiles, we're failing here.

This os error, I think is basically not to do with your implementation at all - this error seems quite frequent in other libraries when running certain code in Dockerised environments on Apple's M1 Silicon:

• https://github.com/Azure/azure-functions-core-tools/issues/2901
• https://github.com/meilisearch/MeiliSearch/issues/1195

For what it's worth there are zero actions / fixes to take here - it's more for if people on the newer machines running Dockerised environments run into this, that there's a record that this can happen. There is discussion that rebuilding your Docker images to usemulti-arch images can solve this, but this is not a problem for this library itself.

It is possible this could also be useful report in watchfiles itself, I just logged it here because this is the lib that I used to invoke it.

Give me a shout if anything's unclear on this - just hoping to help someone else if they wander through asking. :)

Nice work!

Would it make sense to expose filtering of files that are watched, possibly via some setting? Maybe some regexp list or pointing to a custom filtering class to use.

I tried this out in a project with a sqlite db (where the db-file was in the source root). On db writes the application would restart 🙃

I was using watchman for reloading django. I tried django-watchfiles, but the default filter wasn't enough for my use case. Since I already had a .watchmanconfig for my watch ignore lists, I decided to use it in a custom filter for watchfiles. Not sure this is the best design, but something like this will be useful.

Hi there,

Thank you very much for creating this awesome replacement for watchman I am facing an error. Where i cant run the project under pipenv and django.

Repository link : https://github.com/baseplate-admin/CoreProject/tree/540f8e030cf54a865c962bd7e886b76948ccb65c/backend

Python version : 3.11.1

PS C:\Users\Asus\Desktop\CoreProject\backend> pipenv run python .\django_core\manage.py runserver 
Watching for file changes with WatchfilesReloader
Performing system checks...

System check identified no issues (0 silenced).
December 15, 2022 - 16:14:12
Django version 4.1.4, using settings 'core.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
Traceback (most recent call last):
  File "C:\Users\Asus\Desktop\CoreProject\backend\django_core\manage.py", line 22, in <module>
    main()
  File "C:\Users\Asus\Desktop\CoreProject\backend\django_core\manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django\core\management\__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django\core\management\__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django\core\management\base.py", line 402, in run_from_argv
    self.execute(*args, **cmd_options)
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django\core\management\commands\runserver.py", line 74, in execute
    super().execute(*args, **options)
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django\core\management\base.py", line 448, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django\core\management\commands\runserver.py", line 111, in handle
    self.run(**options)
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django\core\management\commands\runserver.py", line 118, in run
    autoreload.run_with_reloader(self.inner_run, **options)
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django\utils\autoreload.py", line 680, in run_with_reloader
    start_django(reloader, main_func, *args, **kwargs)
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django\utils\autoreload.py", line 661, in start_django
    reloader.run(django_main_thread)
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django\utils\autoreload.py", line 344, in run
    self.run_loop()
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django\utils\autoreload.py", line 350, in run_loop
    next(ticker)
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\django_watchfiles\__init__.py", line 21, in tick
    for file_changes in watcher:
  File "C:\Users\Asus\.virtualenvs\backend-FjxXGjsR\Lib\site-packages\watchfiles\main.py", line 119, in watch
    with RustNotify([str(p) for p in paths], debug, force_polling, poll_delay_ms, recursive) as watcher:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: Input watch path is neither a file nor a directory.

This issue is pretty similar to: https://github.com/adamchainz/django-watchfiles/issues/2

I'm re-opening/duplicating the original issue with some additional context as I think some filtering is not properly taking effect here. I'm seeing the server restart/reload when anything in sqlite is touched (something which doesn't happen with the default StatReloader)

Is this behaviour known + intended/expected? (If not I'm happy to investigate a bit further)

I guess I could easily move sqlite out of my project and into a separate dir (where it won't trigger reloads)

Below is a comparison between the default/watchfiles reloaders

https://user-images.githubusercontent.com/4996338/192100292-4799de5e-c667-4ce4-9fba-5f837ba3bdc3.mov

e.g. when I login to the admin panel:

❯ ./manage.py runserver
Watching for file changes with WatchfilesReloader
Performing system checks...

System check identified no issues (0 silenced).
September 24, 2022 - 13:04:54
Django version 4.1.1, using settings 'e.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
watcher: INotifyWatcher { channel: Sender { .. }, waker: Waker { inner: Waker { fd: File { fd: 7, path: "anon_inode:[eventfd]", read: true, write: true } } } }
raw-event: Event { kind: Create(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Access(Close(Write)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Remove(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Create(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Access(Close(Write)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Remove(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Create(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Access(Close(Write)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Remove(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
[24/Sep/2022 13:05:05] "POST /admin/login/?next=/admin/ HTTP/1.1" 302 0
raw-event: Event { kind: Access(Close(Write)), paths: ["/home/jackevans/code/e/db.sqlite3"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
[24/Sep/2022 13:05:05] "GET /admin/ HTTP/1.1" 200 3888
raw-event: Event { kind: Access(Close(Write)), paths: ["/home/jackevans/code/e/db.sqlite3"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
/home/jackevans/code/e/db.sqlite3-journal changed, reloading.
Watching for file changes with WatchfilesReloader
Performing system checks...

I've replaced autoreload.get_reloader with autoreload.run_with_reloader to get verbosity level to setup debug mode. Other settings you can get from the project settings WATCHFILES, e.g.:

WATCHFILES = {
    'watch_filter': 'path.to.custom_filter',
    'raise_interrupt': False,
    'debug': False,
}

Is there a way to change the default verbosity when using Django watchfiles? I'd like to remove these lines:

watcher: INotifyWatcher { channel: Sender { .. }, waker: Waker { inner: Waker { fd: File { fd: 7, path: "anon_inode:[eventfd]", read: true, write: true } } } }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/workspaces/mango/diagram.puml"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }

Thanks.

EDIT: I found a workaround by:

• installing watchfiles as a dependency package
• cloning your app into my source code (to make it my own app)
• changing watcher = watchfiles.watch(*roots, debug=True) to watcher = watchfiles.watch(*roots, debug=False)

It works but it would be great to have a config option directly from settings :)

Hi Adam,

First, thank for all your work. It helped me a lot!

I used to use Watchman after having read your article about it. I had two issues with it:

• having to build my own arm64 binary for using it on mac m1.
• and now Python 3.10 is not compatible (the fix is not released)

In the meantime, I just found out Watchfiles and your related django-watchfiles package. I installed it along with django-browser-reload to make Tailwind CSS work with hot reloading. Everything's great so far.

So the question is: is there any point to keep using watchman rather than Watchfiles with Django?

It not, are you planing to upgrade your blog post and maybe your book too?

Thanks!

Seems that adding or modifying any file below the PWD triggers auto-reload behaviour. This a bit of a pain if, for example, you have a JS transpiler running in parallel: you end up triggering two reloads with every JS file save, the first when you edit your JS and the second when the transpiler saves it result.

(PS Thanks a lot for making this and routing around Facebook's really broken stewardship of pywatchman!)