Firstly, thanks for Whitenoise, I've really appreciated it.

I've bumped into a small problem with Whitenoise 2.0. I have the following as my STATIC_URL:

STATIC_URL = 'https://static.example.com/'

This makes whitenoise fail to load correctly, since get_prefix_url returns '/', and so you get the odd error (from here):

Both STATIC_URL and STATIC_ROOT settings must be configured to use DjangoWhiteNoise

Reverting to whitenoise==1.0.6 works. I think the breaking change is 25f5757b41f5d876252dff873ecb88b177793a3e.

I'm not quite sure what the breaking change was intending to do, so I'm not quite sure what an appropriate patch would be. I think it's something to do with local development based on the release notes of 2.0.

Adding the whitenoise middleware can greatly slow down your tests. If you have processes static previously and your static directory is full of static files. Then the middleware, on each test method, process the list of files over and over. So your tests are now slowed by a function of how many static files you have.

On startup this is called:

    def update_files_dictionary(self, root, prefix):
        for directory, _, filenames in os.walk(root, followlinks=True):
            for filename in filenames:
                path = os.path.join(directory, filename)
                url = prefix + os.path.relpath(path, root).replace('\\', '/')
                self.files[url] = self.get_static_file(path, url)

And if you have thousands of static files, this will slow it down.

Perhaps this list could be lazy loaded on first call rather than init.

I am getting the following test failures with the new release:

======================================================================
ERROR: test_get_brotli (tests.test_django_whitenoise.DjangoWhiteNoiseTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/python-whitenoise/src/whitenoise-4.1.3/tests/test_django_whitenoise.py", line 94, in test_get_brotli
    self.assertEqual(brotli.decompress(response.content), self.static_files.js_content)
brotli.error: BrotliDecompress failed

----------------------------------------------------------------------

Test environment contains the following Arch Linux x86_64 packages:

Packages (13) python-appdirs-1.4.3-2  python-chardet-3.0.4-2  python-idna-2.8-1  python-packaging-19.0-1  python-pyparsing-2.4.0-1
              python-pytz-2019.1-1  python-six-1.12.0-1  python-sqlparse-0.2.4-2  python-urllib3-1.25.3-1  python-brotli-1.0.7-1
              python-django-2.2.3-1  python-requests-2.22.0-1  python-setuptools-1:41.0.1-1

Python Version

3.10

Django Version

4.0

Package Version

6.1

Description

I am using whitenoise to distribute static content. during an execution of runserver, in a development environment, I have the misfortune of experiencing a failed build. I get the below error whenever I run "python manage.py collectstatic" Using the default Django storage engine gives no error. I'm using whitenoise v6.1.0. I've given my static folder full permissions, yet the error persists

""" You have requested to collect static files at the destination location as specified in your settings: /storage/emulated/0/django/test_e/staticfiles This will overwrite existing files! Are you sure you want to do this? Type 'yes' to continue, or 'no' to cancel: yes Traceback (most recent call last): File "/storage/emulated/0/django/test_e/manage.py", line 22, in main() File "/storage/emulated/0/django/test_e/manage.py", line 18, in main execute_from_command_line(sys.argv) File "/data/data/com.termux/files/home/.local/share/virtualenvs/test_e-SjLmU3eY/lib/python3.10/site-packages/django/core/management/init.py", line 446, in execute_from_command_line utility.execute() File "/data/data/com.termux/files/home/.local/share/virtualenvs/test_e-SjLmU3eY/lib/python3.10/site-packages/django/core/management/init.py", line 440, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/data/data/com.termux/files/home/.local/share/virtualenvs/test_e-SjLmU3eY/lib/python3.10/site-packages/django/core/management/base.py", line 414, in run_from_argv self.execute(*args, **cmd_options) File "/data/data/com.termux/files/home/.local/share/virtualenvs/test_e-SjLmU3eY/lib/python3.10/site-packages/django/core/management/base.py", line 460, in execute output = self.handle(*args, **options) File "/data/data/com.termux/files/home/.local/share/virtualenvs/test_e-SjLmU3eY/lib/python3.10/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 209, in handle collected = self.collect() File "/data/data/com.termux/files/home/.local/share/virtualenvs/test_e-SjLmU3eY/lib/python3.10/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 148, in collect for original_path, processed_path, processed in processor: File "/data/data/com.termux/files/home/.local/share/virtualenvs/test_e-SjLmU3eY/lib/python3.10/site-packages/whitenoise/storage.py", line 162, in post_process_with_compression for name, compressed_name in self.compress_files(files_to_compress): File "/data/data/com.termux/files/home/.local/share/virtualenvs/test_e-SjLmU3eY/lib/python3.10/site-packages/whitenoise/storage.py", line 199, in compress_files for compressed_path in compressor.compress(path): File "/data/data/com.termux/files/home/.local/share/virtualenvs/test_e-SjLmU3eY/lib/python3.10/site-packages/whitenoise/compress.py", line 77, in compress yield self.write_data(path, compressed, ".br", stat_result) File "/data/data/com.termux/files/home/.local/share/virtualenvs/test_e-SjLmU3eY/lib/python3.10/site-packages/whitenoise/compress.py", line 122, in write_data os.utime(filename, (stat_result.st_atime, stat_result.st_mtime)) PermissionError: [Errno 1] Operation not permitted """

Python Version

3.9.5

Django Version

4.0.2

Package Version

6.0.0

Description

There are many SO posts in which folks ask "How do I configure Django to serve my index.html when I visit routes handled by a frontend router?" (such as react-router, react-router-dom, or Angular's router). For example, when I visit myDjangoURL.com/app/frontendRoute, how do I serve a file in app/build/index.html that loads a React/Angular app with a browser-based router that handles the /frontendRoute route?

This, this, this, and this all have answers that suggest configuring Django wildcard URLs/paths and returning something like django.shortcuts.render(request,'index.html') or django.views.generic.TemplateView. Of course django.contrib.staticfiles.views.serve is not available in production (when django.conf.settings.DEBUG is False), but everyone asking this question really wants a function like serve, and not a function like render (maybe the implementation of these functions is similar enough that it doesn't matter, but the function names suggest something is wrong). "Serving" one single static file as a rendered template is probably okay performance-wise if Whitenoise is managing and serving all the built dependencies linked to in index.html.

I think it creates a few problems, though:

• Users might have to add their static build folder to their django.conf.settings.TEMPLATES, where it doesn't belong, possibly causing template name conflicts (because this build folder may not be managed in the same way as a Python app)
• Using render might cause weird behavior considering Javascript frameworks might use some of the same syntax as Django's/Jinja's templating language (e.g. {{variablename}} might cause a templating/rendering error).
• Obviously the index.html file would not be a version-hashed copy in the static folder, which could fail to update in caches somewhere downstream, defeating the purpose of Whitenoise's versioning efforts for this one file.

Whitenoise already supports some "interface sugar" by serving an "index" file (WHITENOISE_INDEX_FILE = True in Django) when the static file root is served. Considering the proliferation of frontend routing (evidenced by the StackOverflow posts I linked to above), I think you should expose a frontend routing interface to be used in Django's settings. I think this interface could look something like the following:

WHITENOISE_FRONTEND_ROUTES_BASE = '/app/' # defaults to STATIC_URL

that automatically adds a catchall route that serves index.html for any route starting with '/app/

or

WHITENOISE_FRONTEND_ROUTES_BASES = [('/app/', 'app/index.html'), ('/webapp/' , 'webapp/index.html')] # for multiple frontends served by one Django/Whitenoise app

The interface would probably need some way to both include and exclude certain routes/filepaths, and Whitenoise might want to add a quick check to ensure none of these routes conflict with existing static content routes (and if so, maybe it automatically changes file and directory names in the STATIC_ROOT folder so that static content can still be served without this issue).

I don't 100% understand how Whitenoise does its thing, but it makes the case for replacing Nginx unless you know how to configure everything according the best practices. This reverse-proxying configuration seems like a very easy concern compared to everything else Whitenoise does, and I think it seems wrong for users (like me) to set up Nginx for the sole purpose of it having config options to allow frontend routing.

If this is already possible in a first-class supported way using Django/Whitenoise, please close the issue but please explain what I'm missing (and what the others on the SO posts are missing). Please note that if this is supported, I don't think it's clear from the documentation.

At present, runserver_nostatic works by simply shadowing the runserver command from contrib.staticfiles and replacing it with the runserver command from Django core. However, this won't work with Channels, which needs its own runserver command.

A possible approach might be to require that runserver_nostatic be at the top of the INSTALLED_APPS list and for it to discover the next highest priority runserver command (by inspecting the app registry and delegate to that, after setting the appropriate --no-static flag.

This is obviously more complex and fragile, but it would be nice to have a simple (for the user) solution which was agnostic as to whether Channels was being used.

I am trying to use whitenoise to serve my static files for a django project.

I receive a Server Error (500) When debug = False STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

When I run the program with STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage' the static files are served correctly.

When using whitenoise in DEBUG=True, templates request files like: /static/rest_framework/css/bootstrap-tweaks.css which are properly served up.

With DEBUG=False the template requests: /static/rest_framework/css/bootstrap-tweaks.964c59eecf05.css and this file is not properly served.

What is causing this suffix to be inserted?

When I am running with DEBUG=False , If I manually navigate to /static/rest_framework/css/bootstrap-tweaks.css, the content is served.

I am trying to push my django project to heroku but each time i try to python manage.py collectstatic, I am get this error every time i set STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage' YET When i run python manage.py collectstatic with the default STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage' It runs without an issue. I originally thought one/some of my static files had issues and went through each and everyone of the tens of files but still got the same error. Then i decided to default the STATICFILES_STORAGE setting and to my surprise, python manage.py collectstatic worked.

I would like for someone to help me trouble shoot on this one. Because the issue seems to be related to whitenoise.

I am using Python 3.9.4 and django 3.2

Below is the error i am getting.

Traceback (most recent call last):
  File "/home/nmj/PROJECTS/abc/blueMust/mysite/src/manage.py", line 22, in <module>
    main()
  File "/home/nmj/PROJECTS/abc/blueMust/mysite/src/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/home/nmj/PROJECTS/abc/SubmissionMgtSyst/mysite/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/home/nmj/PROJECTS/abc/SubmissionMgtSyst/mysite/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/nmj/PROJECTS/abc/SubmissionMgtSyst/mysite/venv/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/nmj/PROJECTS/abc/SubmissionMgtSyst/mysite/venv/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/home/nmj/PROJECTS/abc/SubmissionMgtSyst/mysite/venv/lib/python3.9/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 187, in handle
    collected = self.collect()
  File "/home/nmj/PROJECTS/abc/SubmissionMgtSyst/mysite/venv/lib/python3.9/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 128, in collect
    for original_path, processed_path, processed in processor:
  File "/home/nmj/PROJECTS/abc/SubmissionMgtSyst/mysite/venv/lib/python3.9/site-packages/whitenoise/storage.py", line 148, in post_process_with_compression
    for name, hashed_name, processed in files:
  File "/home/nmj/PROJECTS/abc/SubmissionMgtSyst/mysite/venv/lib/python3.9/site-packages/whitenoise/storage.py", line 88, in post_process
    for name, hashed_name, processed in files:
  File "/home/nmj/PROJECTS/abc/SubmissionMgtSyst/mysite/venv/lib/python3.9/site-packages/django/contrib/staticfiles/storage.py", line 406, in post_process
    yield from super().post_process(*args, **kwargs)
  File "/home/nmj/PROJECTS/abc/SubmissionMgtSyst/mysite/venv/lib/python3.9/site-packages/django/contrib/staticfiles/storage.py", line 231, in post_process
    for name, hashed_name, processed, _ in self._post_process(paths, adjustable_paths, hashed_files):
  File "/home/nmj/PROJECTS/abc/SubmissionMgtSyst/mysite/venv/lib/python3.9/site-packages/django/contrib/staticfiles/storage.py", line 288, in _post_process
    content = original_file.read().decode("utf-8")   # original line
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

@edmorley I've just pushed a beta of WhiteNoise 3.0. Changes and documentation are here: http://whitenoise.evans.io/en/latest/changelog.html

If you have time, I'd love feedback from you or anyone else at Mozilla who's using WhiteNoise before I unleash it on the general public.

Thanks!

Currently, Whitenoise sets an Access-Control-Allow-Origin: * header by default for all font files. This was done so that your fonts continue to work in Firefox if you're using a CDN.

I've since discovered other cases where CORS headers are needed, for instance the respond.js media query polyfill for IE which loads stylesheets over XHR.

I'm tempted to set Access-Control-Allow-Origin: * header by default for all files. My reasoning is that:

1. it will mean that putting a CDN in front of Whitenoise just works, without any nasty cross-origin surprises;
2. given that Whitenoise only serves public, static files there are no security implications to allowing cross-origin requests.

I'm aware though that automatically setting these headers is a slightly surprising thing for an application to do and I'd really welcome feedback on whether people think it's a good idea.

Description

The Zopfli compressor generates gzip-compatible compressed files smaller than zlib, and often pretty close to Brotli. For example I use the Python bindings to compress some web assets here:

ORIGINAL   ZOPFLI    (.gz)  BROTLI   (.br)  FILE
---------  ---------------  --------------  ------------------------------------------
    6,138      1,686   27%     1,398   23%  normalize-8.0.1.css
  318,964    161,987   51%   141,848   44%  inconsolata-33dd95333e9e.ttf
      515        199   39%       178   35%  fonts-8e4fe55c0023.css
   22,141      4,573   21%     4,037   18%  main-b75bac5769ba.css
    9,310      2,512   27%     2,355   25%  main-b75bac5769ba.css.map
    5,805      1,149   20%     1,043   18%  icon-bd3da638f8c2.svg
    2,544      1,056   42%       976   38%  lettertype-dc874be920ee.svg
    8,957      1,940   22%     1,879   21%  logotype-a492afd710d0.svg
   24,838      4,849   20%     4,555   18%  icon-4a76ea6aede6.ico

The main downside is that Zopfli is much slower than zlib, like Brotli.

Currently at 96% - the report can be seen and downloaded in HTML format from any run on master.

Here's a version at time of creating issue:

 Name                                                                 Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------------------------------------------------
src/whitenoise/__init__.py                                               3      0      0      0   100%
src/whitenoise/base.py                                                 177      5     84      9    95%   142, 155->152, 164, 171->179, 188, 211->213, 235->exit, 257, 261
src/whitenoise/compress.py                                              84      0     34      1    99%   169->168
src/whitenoise/media_types.py                                           17      0      6      0   100%
src/whitenoise/middleware.py                                           108      4     46      5    94%   88->91, 89->91, 92, 126->130, 132, 170-171
src/whitenoise/responders.py                                           202      5     92      9    95%   92-96, 109->111, 129, 134, 145->147, 170->176, 176->181, 229->exit, 277->280, 285
src/whitenoise/runserver_nostatic/__init__.py                            0      0      0      0   100%
src/whitenoise/runserver_nostatic/management/__init__.py                 0      0      0      0   100%
src/whitenoise/runserver_nostatic/management/commands/__init__.py        0      0      0      0   100%
src/whitenoise/runserver_nostatic/management/commands/runserver.py      26      3     12      4    82%   20->exit, 24-25, 37->34, 39, 48->exit
src/whitenoise/storage.py                                              120      3     58     10    93%   29->31, 36->exit, 49, 92, 96->109, 100->109, 135->137, 149->151, 167->169, 187
src/whitenoise/string_utils.py                                          12      1      4      1    88%   6
tests/__init__.py                                                        0      0      0      0   100%
tests/conftest.py                                                        9      0      0      0   100%
tests/django_settings.py                                                14      0      0      0   100%
tests/django_urls.py                                                     2      0      0      0   100%
tests/test_compress.py                                                  57      0      6      0   100%
tests/test_django_whitenoise.py                                        139      0     16      0   100%
tests/test_media_types.py                                               18      0      0      0   100%
tests/test_runserver_nostatic.py                                        10      0      0      0   100%
tests/test_storage.py                                                   63      0     16      0   100%
tests/test_string_utils.py                                              26     13      4      0    57%   8, 11-12, 15-18, 23, 26, 29, 32, 35, 38
tests/test_whitenoise.py                                               241      0     26      0   100%
tests/utils.py                                                          44      0     12      0   100%
----------------------------------------------------------------------------------------------------------------
TOTAL                                                                 1372     34    416     39    96%

After getting it to 100% we can enforce it on CI.

Hi,

I help beginners a lot on the Unofficial Django Discord, and a very common problem is various problems with static files. I always ask them to install whitenoise obviously, but then I realized that whitenoise doesn't have great error messages for when they have other problems than just "set DEBUG=False" :P

This patch will make the error message much more informative and should help beginners a lot when trying to figure out why their static files don't work.

(This is sort of related to my other work with django-fastdev but this specific change made more sense in whitenoise I thought)

This PR aims to do the following

• [x] Create async whitenoise class
• [ ] Add tests
• [ ] Use aiofile
• [ ] Support Django ASGI middleware
• [ ] Update the docs to show ASGI configuration

See original PR: #261

Currently invalid values like gzip brotli spam still pass, and valid values like gzip;q=0 ("don't send me gzip") are intepreted as "send me gzip". Also the "no encoding" case relies on matching r"" within the string (?).

The header should be parsed correctly. Logic can be borrowed from Django's request.accepts().