Pipeline is an asset packaging library for Django.

Overview

Pipeline

Jazzband GitHub Actions Coverage Documentation Status

Pipeline is an asset packaging library for Django, providing both CSS and JavaScript concatenation and compression, built-in JavaScript template support, and optional data-URI image and font embedding.

Installation

To install it, simply:

pip install django-pipeline

Documentation

For documentation, usage, and examples, see : https://django-pipeline.readthedocs.io

Comments
  • Load times have increased dramatically since upgrade to 1.4.x

    Load times have increased dramatically since upgrade to 1.4.x

    Hello,

    We just tried upgrading django-pipeline from version 1.3.25 to 1.4.x (multiple minor versions tested, including latest: 1.4.6). Every request rendering static files went from 0.5 sec to about 20 secs to render.

    We tried to debug what led to this increase but could not find it. After downgrading to 1.3 the load times were again at 0.5 secs per page.

    opened by bobvoorneveld 44
  • When less file exists in a different folder depth than output, staticfiles dies looking for images

    When less file exists in a different folder depth than output, staticfiles dies looking for images

    I could swear this worked in 1.1.27...

    When any of my less files specify background: url(''), the url is relative assuming that all compressed CSS will end up in css/*.css, and therefore the image urls being with '../img/'.

    However, it seems that pipeline has passed paths off to django staticfiles that are all relative to their original less files, which leads to the image not being found:

    Traceback (most recent call last):
    ...
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/management/commands/collectstatic.py", line 163, in handle_noargs
        collected = self.collect()
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/management/commands/collectstatic.py", line 120, in collect
        for original_path, processed_path, processed in processor:
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 270, in post_process
        content = pattern.sub(converter, content)
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 213, in converter
        hashed_url = self.url(unquote(joined_result), force=True)
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 160, in url
        hashed_name = self.hashed_name(clean_name).replace('\\', '/')
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 123, in hashed_name
        (clean_name, self))
    ValueError: The file 'less/img/wilford-brimley-diabeetus-cat.jpg' could not be found with <simple_pipeline.myapp.pipeline_storage.S3PipelineStorage object at 0x1027f2a90>.
    

    No to go figure out if/where this ever worked...

    [1] http://pastebin.com/MGtcj9rA (full stacktrace here)

    question 
    opened by estebistec 27
  • Do not create compiled JS and CSS in source directory

    Do not create compiled JS and CSS in source directory

    If I have a file, 'foo.coffee', when I reference that file through the tag, it creates a file next to it called 'foo.js'. The problem with this is it makes it very difficult to have a version control system ignore these files. It is very unlikely that I would want to blanket ignore .css or .js files, so that isn't an option. Instead, I'm left ignoring files individually when I add a new less or coffee file.

    It would be nice if pipeline placed the files in their own directory, so I could blanket ignore that directory. As an example, if I have '{static}/foo/bar.coffee', pipeline could place the compiled file in '{static}/{pipelined}/foo/bar.js', where "pipelined" could be a configurable setting.

    opened by SoftwareMaven 21
  • FinderStorage don't write where it should

    FinderStorage don't write where it should

    https://github.com/cyberdelia/django-pipeline/blob/master/pipeline/storage.py#L89 <- bug here

    path is string and os.path.dirname(name) is string too.

    Searching substring in this case is so confusing - need more complicated logic.

    bug 
    opened by dreadatour 21
  • Less compiler not respecting relative paths with @import

    Less compiler not respecting relative paths with @import

    I'm running django-pipeline version 1.1.22 (latest), and the following isn't working yet in .less files:

     @import "bootstrap-1.4/bootstrap.less";
    

    The reason one would do this would be to include the Less library and be able to predefine variables and be able to use the mixins exposed by the library in one's own Less code. For instance:

    @basefont: 14px; /* in bootstrap, it's 13 by default, so we override it before it's imported */
    
    @import "bootstrap-1.4/bootstrap.less";
    
    .ghostly { /* we use bootstrap's defined .opacity mixin to create this effect. */
        .opacity(90);
    }
    

    Currently, this doesn't work and yields:

    Environment:
    
    
    Request Method: GET
    Request URL: http://localhost:8000/
    
    Django Version: 1.3
    Python Version: 2.7.1
    Installed Applications:
    ['django.contrib.auth',
     'django.contrib.contenttypes',
     'django.contrib.sessions',
     'django.contrib.sites',
     'django.contrib.messages',
     'django.contrib.admin',
     'django.contrib.admindocs',
     'django.contrib.staticfiles',
     'django_bcrypt',
     'pipeline',
     'reversion',
     'south',
     'myproject.web']
    Installed Middleware:
    ('django.middleware.common.CommonMiddleware',
     'django.contrib.sessions.middleware.SessionMiddleware',
     'django.middleware.csrf.CsrfViewMiddleware',
     'django.contrib.auth.middleware.AuthenticationMiddleware',
     'django.contrib.messages.middleware.MessageMiddleware')
    
    
    Template error:
    In template /home/rfkrocktk/Desktop/myproject/src/myproject/web/templates/desktop/root.html, error at line 6
       Caught CompilerError while rendering: /var/lib/gems/1.8/gems/less-2.0.7/lib/less/loader.rb:46:in `error': file 'bootstrap-1.4/bootstrap.less' wasn't found. (Less::ParseError)
        from at Function.importer (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/index.js:114:13)
        from at Object.push (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:73:25)
        from at Object.<anonymous> (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/tree/import.js:30:17)
        from at Object.import (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:953:28)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.directive (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:967:29)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.primary (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:428:80)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.parse (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:264:42)
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:65:in `parse'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:13:in `calljs'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:27:in `lock'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:25:in `lock'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:12:in `calljs'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:63:in `parse'
        from /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:22
        from /usr/local/bin/lessc:19:in `load'
        from /usr/local/bin/lessc:19
    
       1 : {% load static %}
    
    
       2 : {% load compressed %}
    
    
       3 : <!DOCTYPE html>
    
    
       4 : <html>
    
    
       5 :  <head>
    
    
       6 :      {% block stylesheets %} {% compressed_css 'all' %} {% endblock %}
    
    
       7 :      <title>{% block title %}{% endblock %}&nbsp;&raquo;&nbsp;myproject</title>
    
    
       8 :  </head>
    
    
       9 :  <body>
    
    
       10 :         {% block body %}{% endblock %}
    
    
       11 :         {% block scripts %}{% compressed_js 'all' %}{% endblock %}
    
    
       12 :     </body>
    
    
       13 : </html>
    
    
       14 : 
    
    Traceback:
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/core/handlers/base.py" in get_response
      111.                         response = callback(request, *callback_args, **callback_kwargs)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/views/generic/simple.py" in direct_to_template
      28.     return HttpResponse(t.render(c), mimetype=mimetype)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
      123.             return self._render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
      117.         return self.nodelist.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
      744.                 bits.append(self.render_node(node, context))
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
      73.             result = node.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
      127.         return compiled_parent._render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
      117.         return self.nodelist.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
      744.                 bits.append(self.render_node(node, context))
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
      73.             result = node.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
      127.         return compiled_parent._render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
      117.         return self.nodelist.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
      744.                 bits.append(self.render_node(node, context))
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
      73.             result = node.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
      64.             result = block.nodelist.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
      744.                 bits.append(self.render_node(node, context))
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
      73.             result = node.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/templatetags/compressed.py" in render
      30.             package['paths'] = self.packager.compile(package['paths'])
    File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/packager.py" in compile
      50.         return self.compiler.compile(paths)
    File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/__init__.py" in compile
      25.                         compiled_content = compiler.compile_file(content, storage.path(path))
    File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/less/__init__.py" in compile_file
      17.         content = self.execute_command(command)
    File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/__init__.py" in execute_command
      88.             raise CompilerError(error)
    
    Exception Type: TemplateSyntaxError at /
    Exception Value: Caught CompilerError while rendering: /var/lib/gems/1.8/gems/less-2.0.7/lib/less/loader.rb:46:in `error': file 'bootstrap-1.4/bootstrap.less' wasn't found. (Less::ParseError)
        from at Function.importer (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/index.js:114:13)
        from at Object.push (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:73:25)
        from at Object.<anonymous> (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/tree/import.js:30:17)
        from at Object.import (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:953:28)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.directive (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:967:29)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.primary (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:428:80)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.parse (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:264:42)
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:65:in `parse'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:13:in `calljs'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:27:in `lock'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:25:in `lock'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:12:in `calljs'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:63:in `parse'
        from /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:22
        from /usr/local/bin/lessc:19:in `load'
        from /usr/local/bin/lessc:19
    
    bug awaiting response 
    opened by naftulikay 17
  • Compiler @import don't work

    Compiler @import don't work

    With the below situation, our 'base' bundle is still being compressed and written to a file called 'css/base.css' (where it should be 'css/thatguyismystyle.base.min.css' )

    The template tag still writes out the desired filename (which is 'css/thatguyismystyle.base.min.css' )


    Django==1.3
    django-pipeline==1.2.5
    django-staticfiles==1.2.1
    

        from os.path import join
        from settings import SERVER_ROOT
        from collections import OrderedDict as OD
    
        NODE_BIN = join(SERVER_ROOT, 'node_modules/.bin')
    
        #
        # Pipelined CSS Files
        #
    
    
        PIPELINE_CSS = OD((
            ('base', {
                'source_filenames': (
                    'css/base.less',
                ),
    
                'output_filename':
                    'css/thatguyismystyle.base.min.css',
    
                'extra_context': {
                    'media': 'screen',
                },
            }),
            ('theme', {
                'source_filenames': (
                    'css/theme/base.less',
                ),
                'output_filename':
                    'css/theme/thatguyismystyle.theme.min.css',
    
                'extra_context': {
                    'media': 'screen',
                },
            }),
            ('theme-light', {
                'source_filenames': (
                    'themes/light/css/base.less'
                ),
                'output_filename':
                    'themes/thatguyismystyle.colour.light.min.css',
                'extra_context': {
                    "media": "screen",
                }
            }),
            ('theme-dark', {
                'source_filenames': (
                    'themes/dark/css/base.less'
                ),
                'output_filename':
                    'themes/thatguyismystyle.colour.dark.min.css',
                'extra_context': {
                    "media": "screen",
                }
            }),
        ))
    
        #
        # Pipelined JS FILES
        #
    
        BASE_JS_FRAMEWORK = (
            "js/lib/modernizr.min.js",
            "js/lib/jquery.min.1.7.1.js",
            "js/lib/history.js/jquery.history.js",
            "js/lib/moment.min.pretty.js",
            "js/lib/moment.strftime.min.js",
        )
        BOOTSTRAP_JS = (
            'js/lib/bootstrap/bootstrap-alert.js',
            'js/lib/bootstrap/bootstrap-button.js',
            'js/lib/bootstrap/bootstrap-carousel.js',
            'js/lib/bootstrap/bootstrap-collapse.js',
            'js/lib/bootstrap/bootstrap-dropdown.js',
            'js/lib/bootstrap/bootstrap-modal.js',
            'js/lib/bootstrap/bootstrap-popover.js',
            'js/lib/bootstrap/bootstrap-scrollspy.js',
            'js/lib/bootstrap/bootstrap-tab.js',
            'js/lib/bootstrap/bootstrap-tooltip.js',
            'js/lib/bootstrap/bootstrap-transition.js',
            'js/lib/bootstrap-contrib/*.js',
        )
        CKEDITOR_JS = (
            'js/lib/ckeditor/*.js',
        )
        TINYMCS_JS = (
            'js/lib/tinymce/tiny_mce.js',
            'js/lib/tinymce/langs/*.js',
            'js/lib/tinymce/jquery.tinymce.js',
        )
    
    
        PIPELINE_JS = OD((
            ('backend', {
                'source_filenames': (
                    'js/lib/jquery.js',
                ) + BOOTSTRAP_JS
                  + CKEDITOR_JS + (
                    'js/lib/jquery.*.js',
                    'js/lib/thatguyismystyle.*.js',
                    'js/application.js',
                ),
                'output_filename': 'js/.js',
            }),
            ('profile-pages', {
                'source_filenames': (
                ),
                'output_filename': 'js/profile.js'
            })
        ))
        PIPELINE_JS = dict()
    
        PIPELINE_DISABLE_WRAPPER = True
    
        #
        # Compilers
        #
    
        PIPELINE_STORAGE = 'staticfiles.storage.StaticFilesStorage'
        STATICFILES_STORAGE = 'staticfiles.storage.StaticFilesStorage'
        PIPELINE=True
    
        PIPELINE_COMPILERS = (
            'pipeline.compilers.less.LessCompiler',
            'pipeline.compilers.coffee.CoffeeScriptCompiler',
        )
        PIPELINE_LESS_BINARY = join( NODE_BIN, 'lessc' )
    
        #
        # Compressors
        #
        PIPELINE_JS_COMPRESSOR = 'pipeline.compressors.jsmin.JSMinCompressor'
        PIPELINE_CSS_COMPRESSOR = 'pipeline.compressors.cssmin.CssminCompressor'
    
    
        STATICFILES_FINDERS = (
         'staticfiles.finders.AppDirectoriesFinder',
        )
    
        STATICFILES_IGNORE_PATTERNS = (
            "*.txt",
            "tests",
            "css/*.old",
            "/opt/webfiles/common/*.txt",
            "/opt/webfiles/common/temp",
        )
    
    bug 
    opened by airtonix 15
  • Ignore missing files

    Ignore missing files

    In large projects that use dozens of third-party libraries, invariably, some library will reference a url(../../..somefile) which doesn't exist. When running manage.py collectstatic with django-pipeline, this will result in an incredibly unhelpful error like:

    ValueError: The file 'tinymce/js/tinymce/plugins/compat3x/img/icons.gif' could not be found with <pipeline.storage.PipelineCachedStorage object at 0x1ca6750>.
    

    Not only does pipeline not mention which file attempts to include this file, searching for img/icons.gif myself results in 42 matches. Even if I could locate the issue, I can't easily fix the missing file, since it's a third-party library. This means pipeline is effectively blocking all my deployments because of a missing file I don't even use.

    Ideally, pipeline should report which resource is referencing a missing file and allow specifying an option to ignore it. This is very similar to the https://github.com/adamcharnock/django-pipeline-forgiving fork, although I think this would be a good idea for the main branch.

    opened by chrisspen 14
  • 1.4.2 requires absolute paths support from static storage

    1.4.2 requires absolute paths support from static storage

    I'm using the following static storage:

    class S3PipelineStorage(PipelineMixin, S3BotoStorage):
        """S3 storage with pipeline magic"""
        pass
    

    Which is almost the same as the one from pipeline docs (even simpler because it doesnt use CachedFilesMixin) . It's fine with 1.3.x but breaks with 1.4.2 because it tries to run sass compiler using absolute path from s3. S3BotoStorage doesn't support absolute paths and throws this long exception:

    /home/bender/myapps/foo/manage.py in <module>()
          8     from django.core.management import execute_from_command_line
          9 
    ---> 10     execute_from_command_line(sys.argv)
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/__init__.pyc in execute_from_command_line(argv)
        383     """
        384     utility = ManagementUtility(argv)
    --> 385     utility.execute()
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/__init__.pyc in execute(self)
        375             sys.stdout.write(self.main_help_text() + '\n')
        376         else:
    --> 377             self.fetch_command(subcommand).run_from_argv(self.argv)
        378 
        379 
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/base.pyc in run_from_argv(self, argv)
        286         handle_default_options(options)
        287         try:
    --> 288             self.execute(*args, **options.__dict__)
        289         except Exception as e:
        290             if options.traceback or not isinstance(e, CommandError):
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/base.pyc in execute(self, *args, **options)
        336                     not options.get('skip_checks')):
        337                 self.check()
    --> 338             output = self.handle(*args, **options)
        339             if output:
        340                 if self.output_transaction:
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/base.pyc in handle(self, *args, **options)
        531         if args:
        532             raise CommandError("Command doesn't accept any arguments")
    --> 533         return self.handle_noargs(**options)
        534 
        535     def handle_noargs(self, **options):
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.pyc in handle_noargs(self, **options)
        166             raise CommandError("Collecting static files cancelled.")
        167 
    --> 168         collected = self.collect()
        169         modified_count = len(collected['modified'])
        170         unmodified_count = len(collected['unmodified'])
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.pyc in collect(self)
        112             processor = self.storage.post_process(found_files,
        113                                                   dry_run=self.dry_run)
    --> 114             for original_path, processed_path, processed in processor:
        115                 if isinstance(processed, Exception):
        116                     self.stderr.write("Post-processing '%s' failed!" % original_path)
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/storage.pyc in post_process(self, paths, dry_run, **options)
         28         from pipeline.packager import Packager
         29         packager = Packager(storage=self)
    ---> 30         for package_name in packager.packages['css']:
         31             package = packager.package_for('css', package_name)
         32             output_file = package.output_filename
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/packager.pyc in pack_stylesheets(self, package, **kwargs)
         94         return self.pack(package, self.compressor.compress_css, css_compressed,
         95                          output_filename=package.output_filename,
    ---> 96                          variant=package.variant, **kwargs)
         97 
         98     def compile(self, paths, force=False):
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/packager.pyc in pack(self, package, compress, signal, **kwargs)
        103         if self.verbose:
        104             print("Saving: %s" % output_filename)
    --> 105         paths = self.compile(package.paths, force=True)
        106         content = compress(paths, **kwargs)
        107         self.save_file(output_filename, content)
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/packager.pyc in compile(self, paths, force)
         97 
         98     def compile(self, paths, force=False):
    ---> 99         return self.compiler.compile(paths, force=force)
        100 
        101     def pack(self, package, compress, signal, **kwargs):
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/compilers/__init__.pyc in compile(self, paths, force)
         54         else:
         55             with futures.ThreadPoolExecutor(max_workers=multiprocessing.cpu_count()) as executor:
    ---> 56                 return list(executor.map(_compile, paths))
         57 
         58     def output_path(self, path, extension):
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_base.pyc in map(self, fn, *iterables, **kwargs)
        578             for future in fs:
        579                 if timeout is None:
    --> 580                     yield future.result()
        581                 else:
        582                     yield future.result(end_time - time.time())
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_base.pyc in result(self, timeout)
        398                 raise CancelledError()
        399             elif self._state == FINISHED:
    --> 400                 return self.__get_result()
        401 
        402             self._condition.wait(timeout)
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_base.pyc in __get_result(self)
        357     def __get_result(self):
        358         if self._exception:
    --> 359             reraise(self._exception, self._traceback)
        360         else:
        361             return self._result
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_compat.pyc in reraise(exc, traceback)
        105     def reraise(exc, traceback):
        106         locals_ = {'exc_type': type(exc), 'exc_value': exc, 'traceback': traceback}
    --> 107         exec('raise exc_type, exc_value, traceback', {}, locals_)
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/thread.pyc in run(self)
         59 
         60         try:
    ---> 61             result = self.fn(*self.args, **self.kwargs)
         62         except BaseException:
         63             e, tb = sys.exc_info()[1:]
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/compilers/__init__.pyc in _compile(input_path)
         34                 if compiler.match_file(input_path):
         35                     output_path = self.output_path(input_path, compiler.output_extension)
    ---> 36                     infile = self.storage.path(input_path)
         37                     outfile = self.output_path(infile, compiler.output_extension)
         38                     outdated = compiler.is_outdated(input_path, output_path)
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/files/storage.pyc in path(self, name)
         85         accessed using open() should *not* implement this method.
         86         """
    ---> 87         raise NotImplementedError("This backend doesn't support absolute paths.")
         88 
         89     # The following methods form the public API for storage systems, but with
    
    NotImplementedError: This backend doesn't support absolute paths
    
    opened by rjagielski 12
  • pipeline.exceptions.CompressorError

    pipeline.exceptions.CompressorError

    I occasionally (once every 10 runs or so?) get this error when django-pipeline is compressing my JS:

    Traceback (most recent call last):
      File "manage.py", line 10, in <module>
        execute_from_command_line(sys.argv)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
        utility.execute()
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
        self.fetch_command(subcommand).run_from_argv(self.argv)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
        self.execute(*args, **options.__dict__)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
        output = self.handle(*args, **options)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/base.py", line 415, in handle
        return self.handle_noargs(**options)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 173, in handle_noargs
        collected = self.collect()
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 118, in collect
        for original_path, processed_path, processed in processor:
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/storage.py", line 39, in post_process
        packager.pack_javascripts(package)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/packager.py", line 110, in pack_javascripts
        return self.pack(package, self.compressor.compress_js, js_compressed, templates=package.templates, **kwargs)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/packager.py", line 104, in pack
        content = compress(paths, **kwargs)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/__init__.py", line 65, in compress_js
        js = getattr(compressor(verbose=self.verbose), 'compress_js')(js)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/yuglify.py", line 13, in compress_js
        return self.compress_common(js, 'js', settings.PIPELINE_YUGLIFY_JS_ARGUMENTS)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/yuglify.py", line 10, in compress_common
        return self.execute_command(command, content)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/__init__.py", line 237, in execute_command
        raise CompressorError(stderr)
    pipeline.exceptions.CompressorError: Killed
    

    The only way I have found to continue is to restart the vagrant box that I am developing on.

    opened by ErikEvenson 12
  • AttributeError: 'CachedFileFinder' object has no attribute 'storages'

    AttributeError: 'CachedFileFinder' object has no attribute 'storages'

    I'm having the following problem. Not sure if it's a misconfiguration on my end or a bug.

    I'm using Bower to install some components. I've added to STATICFILES_DIRS the directory where the components are installed.

    Here are some relevant parts from settings.py

    STATICFILES_FINDERS = (
        'pipeline.finders.AppDirectoriesFinder',
        'pipeline.finders.FileSystemFinder',
        'pipeline.finders.PipelineFinder',
        'pipeline.finders.CachedFileFinder',
    )
    STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'
    DEBUG = False
    

    When I run collectstatic, I get the following error:

    AttributeError: 'CachedFileFinder' object has no attribute 'storages'
    

    However, If I remove CachedFileFinder, collectstatic works fine.

    opened by honi 12
  • What should be happening?

    What should be happening?

    I've begun implementing Pipeline in our app. I've added the middleware, items to the requirements.txt file, the compressor variable, and the PIPELINE_JS object.

    PIPELINE_JS = {
        'activity': {
            'source_filenames': (
                'assets/js/libs/src/core.libs.js',
                'assets/js/libs/src/datepicker.js',
            ),
            'output_filename': 'assets/js/libs/activity.js',
        }
    }
    

    I've included the template tag on the page, and output the tag call like so:

    {% compressed_js 'activity' %}

    The page loads without error, but nothing appears to be happening. I wanted to make sure something should be happening at this point. I don't see any outputted files, and viewing the rendered source of the page shows an empty line where the call to the template was.

    So what should I expect is happening?

    Also, as a side note, it would be AWESOME if you could include a super simple app using pipeline as a demo. Something that users could download and run locally to see what happens.

    Looking forward to your response.

    opened by commadelimited 12
  • [QUESTION] Should not found files not raise an error while executing collectstatic

    [QUESTION] Should not found files not raise an error while executing collectstatic

    When executing collectstatic django-pipeline does not raise an error if a file defined in PIPELINE config is not found. Is this on purpose? Do I miss a settings? I would like to get an error if file which should be compiled does not exist.

    opened by Schwankenson 0
  • Add Django 4.1 support

    Add Django 4.1 support

    Only a small change was required in the tests to handle this change about removing the type="text/css" attributes for CSS link tags:

    2022-09-29_12-14


    It might make sense when releasing a new version to drop support for Python 3.6 and Django 2.2, 3.0 and 3.1 since they have all reached end of life. If you want I can push that in this PR or seperately

    opened by kevinmarsh 3
  • swapped the project_infile path with the infile path on the compile_f…

    swapped the project_infile path with the infile path on the compile_f…

    collectstatic static fails with the project_infile variable in the compile_file method because the finder looks for static files in the static directories not from static_root.

    opened by OzzyGrammer 0
  • [Question][Bug] Infile paramater value wrong after updating from 1.6.14 to 2.0.7

    [Question][Bug] Infile paramater value wrong after updating from 1.6.14 to 2.0.7

    We had to upgrade django-pipeline from 1.6.14 to at least 2.0.7 since we are currently in the process of upgrading django to version 3.2. After we made the update, our static files were not behaving the same anymore (css not loading). After a few days of investigating I saw that the infile parameter that is passed into the compile_file function in the LessCompiler class were not the same as the value we usually got before we made the update.

    Seems like the infile value comes from compile function on the Compiler class in the init file.

        def compile(self, paths, compiler_options={}, force=False):
            def _compile(input_path):
                for compiler in self.compilers:
                    compiler = compiler(verbose=self.verbose, storage=self.storage)
                    if compiler.match_file(input_path):
                        try:
                            infile = self.storage.path(input_path)
                        except NotImplementedError:
                            infile = finders.find(input_path)
                        project_infile = finders.find(input_path)
                        outfile = compiler.output_path(infile, compiler.output_extension)
                        outdated = compiler.is_outdated(project_infile, outfile)
                        compiler.compile_file(project_infile, outfile,
                                              outdated=outdated, force=force,
                                              **compiler_options)
    
                        return compiler.output_path(input_path, compiler.output_extension)
                else:
                    return input_path
    

    I see the infile value is never passed into the compile_file function. No matter what happens in the try except, another variable called project_infile is created underneath and passed to compile_file, which in our case has the wrong value. Infile has the correct value. On the older version that we used the project_infile didnt exist and the infile value was passed to compile_file. Thank you in advance.

    opened by Willem-Nieuwoudt 0
  • [Feature Request]: self.request to context in render_js

    [Feature Request]: self.request to context in render_js

    This is a feature request to add self.request to context in render_js on JavascriptNode.

    This would be handy for cases where request context variables need to be rendered in the js template, for example, with django-csp you would be able to add a nonce in a custom template without having to extend JavascriptNode and writing a new tag to use it.

    My suggestion is to add a key to extra_context which you can declare in settings with a boolean value that defaults to false. If true it adds self.request to context which will then be usable in the template.

    If it's something anybody else is interested in, I could create a pull request for it.

    opened by mfvictor 0
Releases(2.0.8)
  • 2.0.8(Jan 10, 2022)

    • Added Django 4.0 compatibility. Thanks to @kevinmarsh (#760)
    • Add tests for Django 4.0, Python 3.9 and Python 3.10. Thank to @kevinmarsh (#739)
    • Introduce CODE_OF_CONDUCT.md for the project. Thank to @hugovk (#758)
    • Add precision in the documentation for PipelineCachedStorage. Thank to @gatsinski (#739)
    • Drop support for slimit compressor (#765) due to package not released an official version for Python 3 and not any new package release from 2013.
    • Edit github actions matrix: django 3.2.9 support python 3.10, remove python 4.0 (doesn't exist) and exclude pypy-3.8 for django-main.
    • Add .pre-commit-config.yaml. Thanks to @hugovk (#762)
    • Update package.json due to CoffeeScript on NPM has moved to "coffeescript"
    • Update setup.py with Django 4.0 and Python 3.10
    Source code(tar.gz)
    Source code(zip)
  • 2.0.7(Oct 21, 2021)

  • 2.0.6(Jan 18, 2021)

  • 2.0.5(Aug 16, 2020)

    • Adding Django 3.1 compatibility.
    • CachedStaticFilesStorage is removed from Django. Add a check of the current version to prevent error while importing. Thank to @vmsp
    • Context in django.template.base is removed from Django and not used anymore in django-pipeline.
    • Fixing widgets tests of django-pipeline due to Media.render_js change in Django. More information in Django ticket #31892
    Source code(tar.gz)
    Source code(zip)
  • 2.0.4(May 29, 2020)

  • 2.0.3(May 23, 2020)

    • Add subclass of ManifestStaticFilesStorage. Thank to @jhpinson
    • Change the documentation to use PipelineManifestStorage in configuration instead of PipelineCachedStorage now deprecated.
    • Change import MutableMapping from collections.abc. Thank to @colons
    • Fix Middleware to properly decode HTML. Thank to @gatsinski
    • Keep mimetypes as str. Thank to @benspaulding
    • Based on #642 add 'NonPackagingPipelineManifestStorage' and update the documentation: storages.rst. Thank to @kronion
    • Remove futures from pipeline setup.py requirements.
    Source code(tar.gz)
    Source code(zip)
  • 2.0.0(May 23, 2020)

    • Definitely drop the support of Python 2.
    • Drop support for Python 3.5 (not compatible with PEP 498).
    • Remove 'decorator.py' how was used for backward compatibility between python 2 and 3 for metaclass inheritance on PipelineFormMedia.
    • Replace 'format' by 'fstring' (PEP 498: Literal String Interpolation).
    • Remove of old imports form 'django.utils.six' and these fixes (1.7.0).
    • Remove tests of uncovered versions of Python and Django.
    • Replace tests for Pypy by Pypy3.
    • Explicitly specify when files are read / write in binary mode.
    • Set opening files for tests to deal with universal newlines.
    • Upgrade documentation version to 2.0 to follow the project version.
    Source code(tar.gz)
    Source code(zip)
  • 1.7.0(Dec 28, 2019)

    Release the last major version of django-pipeline working on Python 2.

    • Thank you for all the modifications made since version 1.6.14, which we cannot quote.
    • Apply an optimization for compilation. Thank to @blankser
    • Edit setup.py to follow the recommendation of the documentation. Thank to @shaneikennedy
    • Add tests for Django 3.0 and Python 3.8
    • Add alternatives imports for django.utils.six, who has been removed in Django 3.0
    Source code(tar.gz)
    Source code(zip)
  • 1.6.13(May 25, 2017)

    • Fix forward-slashed paths on Windows. Thanks to @etiago
    • Fix CSS URL detector to match quotes correctly. Thanks to @vskh
    • Add a compiler_options dict to compile, to allow passing options to custom compilers. Thanks to @sassanh
    • Verify support for Django 1.11. Thanks to @jwhitlock
    Source code(tar.gz)
    Source code(zip)
  • 1.6.12(Mar 13, 2017)

Meta package to combine turbo-django and stimulus-django

Hotwire + Django This repository aims to help you integrate Hotwire with Django ?? Inspiration might be taken from @hotwired/hotwire-rails. We are sti

Hotwire for Django 31 Aug 9, 2022
django-reversion is an extension to the Django web framework that provides version control for model instances.

django-reversion django-reversion is an extension to the Django web framework that provides version control for model instances. Requirements Python 3

Dave Hall 2.8k Jan 2, 2023
Django-environ allows you to utilize 12factor inspired environment variables to configure your Django application.

Django-environ django-environ allows you to use Twelve-factor methodology to configure your Django application with environment variables. import envi

Daniele Faraglia 2.7k Jan 7, 2023
Rosetta is a Django application that eases the translation process of your Django projects

Rosetta Rosetta is a Django application that facilitates the translation process of your Django projects. Because it doesn't export any models, Rosett

Marco Bonetti 909 Dec 26, 2022
Cookiecutter Django is a framework for jumpstarting production-ready Django projects quickly.

Cookiecutter Django Powered by Cookiecutter, Cookiecutter Django is a framework for jumpstarting production-ready Django projects quickly. Documentati

Daniel Feldroy 10k Dec 31, 2022
Django project starter on steroids: quickly create a Django app AND generate source code for data models + REST/GraphQL APIs (the generated code is auto-linted and has 100% test coverage).

Create Django App ?? We're a Django project starter on steroids! One-line command to create a Django app with all the dependencies auto-installed AND

imagine.ai 68 Oct 19, 2022
django-quill-editor makes Quill.js easy to use on Django Forms and admin sites

django-quill-editor django-quill-editor makes Quill.js easy to use on Django Forms and admin sites No configuration required for static files! The ent

lhy 139 Dec 5, 2022
A Django chatbot that is capable of doing math and searching Chinese poet online. Developed with django, channels, celery and redis.

Django Channels Websocket Chatbot A Django chatbot that is capable of doing math and searching Chinese poet online. Developed with django, channels, c

Yunbo Shi 8 Oct 28, 2022
A handy tool for generating Django-based backend projects without coding. On the other hand, it is a code generator of the Django framework.

Django Sage Painless The django-sage-painless is a valuable package based on Django Web Framework & Django Rest Framework for high-level and rapid web

sageteam 51 Sep 15, 2022
A beginner django project and also my first Django project which involves shortening of a longer URL into a short one using a unique id.

Django-URL-Shortener A beginner django project and also my first Django project which involves shortening of a longer URL into a short one using a uni

Rohini Rao 3 Aug 8, 2021
Dockerizing Django with Postgres, Gunicorn, Nginx and Certbot. A fully Django starter project.

Dockerizing Django with Postgres, Gunicorn, Nginx and Certbot ?? Features A Django stater project with fully basic requirements for a production-ready

null 8 Jun 27, 2022
pytest-django allows you to test your Django project/applications with the pytest testing tool.

pytest-django allows you to test your Django project/applications with the pytest testing tool.

pytest-dev 1.1k Dec 14, 2022
APIs for a Chat app. Written with Django Rest framework and Django channels.

ChatAPI APIs for a Chat app. Written with Django Rest framework and Django channels. The documentation for the http end points can be found here This

Victor Aderibigbe 18 Sep 9, 2022
django-dashing is a customisable, modular dashboard application framework for Django to visualize interesting data about your project. Inspired in the dashboard framework Dashing

django-dashing django-dashing is a customisable, modular dashboard application framework for Django to visualize interesting data about your project.

talPor Solutions 703 Dec 22, 2022
Django-MySQL extends Django's built-in MySQL and MariaDB support their specific features not available on other databases.

Django-MySQL The dolphin-pony - proof that cute + cute = double cute. Django-MySQL extends Django's built-in MySQL and MariaDB support their specific

Adam Johnson 504 Jan 4, 2023
Django-Audiofield is a simple app that allows Audio files upload, management and conversion to different audio format (mp3, wav & ogg), which also makes it easy to play audio files into your Django application.

Django-Audiofield Description: Django Audio Management Tools Maintainer: Areski Contributors: list of contributors Django-Audiofield is a simple app t

Areski Belaid 167 Nov 10, 2022
django Filer is a file management application for django that makes handling of files and images a breeze.

django Filer is a file management application for django that makes handling of files and images a breeze.

django CMS Association 1.6k Jan 6, 2023
Twitter Bootstrap for Django Form - A simple Django template tag to work with Bootstrap

Twitter Bootstrap for Django Form - A simple Django template tag to work with Bootstrap

tzangms 557 Oct 19, 2022
Blog focused on skills enhancement and knowledge sharing. Tech Stack's: Vue.js, Django and Django-Ninja

Blog focused on skills enhancement and knowledge sharing. Tech Stack's: Vue.js, Django and Django-Ninja

Wanderson Fontes 2 Sep 21, 2022