Silky smooth profiling for Django

Related tags

Testing django-silk
Overview

Silk

GitHub Actions GitHub Actions PyPI Download PyPI Python Versions Jazzband

Silk is a live profiling and inspection tool for the Django framework. Silk intercepts and stores HTTP requests and database queries before presenting them in a user interface for further inspection:

SECURITY NOTE: Because Silk stores all HTTP requests into the database in plain text, it will store the request's sensitive information into the database in plain text (e.g. users' passwords!). This is a massive security concern. An issue has been created for this here.

Contents

Requirements

Silk has been tested with:

  • Django: 2.2, 3.0, 3.1
  • Python: 3.6, 3.7, 3.8, 3.9

Installation

Via pip into a virtualenv:

pip install django-silk

In settings.py add the following:

MIDDLEWARE = [
    ...
    'silk.middleware.SilkyMiddleware',
    ...
]

INSTALLED_APPS = (
    ...
    'silk'
)

Note: The middleware placement is sensitive. If the middleware before silk.middleware.SilkyMiddleware returns from process_request then SilkyMiddleware will never get the chance to execute. Therefore you must ensure that any middleware placed before never returns anything from process_request. See the django docs for more information on this.

Note: If you are using django.middleware.gzip.GZipMiddleware, place that before silk.middleware.SilkyMiddleware, otherwise you will get an encoding error.

If you want to use custom middleware, for example you developed the subclass of silk.middleware.SilkyMiddleware, so you can use this combination of settings:

# Specify the path where is the custom middleware placed
SILKY_MIDDLEWARE_CLASS = 'path.to.your.middleware.MyCustomSilkyMiddleware'

# Use this variable in list of middleware
MIDDLEWARE = [
    ...
    SILKY_MIDDLEWARE_CLASS,
    ...
]

To enable access to the user interface add the following to your urls.py:

urlpatterns += [url(r'^silk/', include('silk.urls', namespace='silk'))]

before running migrate:

python manage.py makemigrations

python manage.py migrate

python manage.py collectstatic

Silk will automatically begin interception of requests and you can proceed to add profiling if required. The UI can be reached at /silk/

Alternative Installation

Via github tags:

pip install https://github.com/jazzband/silk/archive/<version>.tar.gz

You can install from master using the following, but please be aware that the version in master may not be working for all versions specified in requirements

pip install -e git+https://github.com/jazzband/django-silk.git#egg=silk

Features

Silk primarily consists of:

  • Middleware for intercepting Requests/Responses
  • A wrapper around SQL execution for profiling of database queries
  • A context manager/decorator for profiling blocks of code and functions either manually or dynamically.
  • A user interface for inspection and visualisation of the above.

Request Inspection

The Silk middleware intercepts and stores requests and responses in the configured database. These requests can then be filtered and inspecting using Silk's UI through the request overview:

It records things like:

  • Time taken
  • Num. queries
  • Time spent on queries
  • Request/Response headers
  • Request/Response bodies

and so on.

Further details on each request are also available by clicking the relevant request:

SQL Inspection

Silk also intercepts SQL queries that are generated by each request. We can get a summary on things like the tables involved, number of joins and execution time (the table can be sorted by clicking on a column header):

Before diving into the stack trace to figure out where this request is coming from:

Profiling

Turn on the SILKY_PYTHON_PROFILER setting to use Python's built-in cProfile profiler. Each request will be separately profiled and the profiler's output will be available on the request's Profiling page in the Silk UI.

SILKY_PYTHON_PROFILER = True

If you would like to also generate a binary .prof file set the following:

SILKY_PYTHON_PROFILER_BINARY = True

When enabled, a graph visualisation generated using gprof2dot and viz.js is shown in the profile detail page:

A custom storage class can be used for the saved generated binary .prof files:

SILKY_STORAGE_CLASS = 'path.to.StorageClass'

The default storage class is silk.storage.ProfilerResultStorage, and when using that you can specify a path of your choosing. You must ensure the specified directory exists.

# If this is not set, MEDIA_ROOT will be used.
SILKY_PYTHON_PROFILER_RESULT_PATH = '/path/to/profiles/'

A download button will become available with a binary .prof file for every request. This file can be used for further analysis using snakeviz or other cProfile tools

Silk can also be used to profile specific blocks of code/functions. It provides a decorator and a context manager for this purpose.

For example:

from silk.profiling.profiler import silk_profile


@silk_profile(name='View Blog Post')
def post(request, post_id):
    p = Post.objects.get(pk=post_id)
    return render_to_response('post.html', {
        'post': p
    })

Whenever a blog post is viewed we get an entry within the Silk UI:

Silk profiling not only provides execution time, but also collects SQL queries executed within the block in the same fashion as with requests:

Decorator

The silk decorator can be applied to both functions and methods

from silk.profiling.profiler import silk_profile


# Profile a view function
@silk_profile(name='View Blog Post')
def post(request, post_id):
    p = Post.objects.get(pk=post_id)
    return render_to_response('post.html', {
        'post': p
    })


# Profile a method in a view class
class MyView(View):
    @silk_profile(name='View Blog Post')
    def get(self, request):
        p = Post.objects.get(pk=post_id)
        return render_to_response('post.html', {
            'post': p
        })

Context Manager

Using a context manager means we can add additional context to the name which can be useful for narrowing down slowness to particular database records.

def post(request, post_id):
    with silk_profile(name='View Blog Post #%d' % self.pk):
        p = Post.objects.get(pk=post_id)
        return render_to_response('post.html', {
            'post': p
        })

Dynamic Profiling

One of Silk's more interesting features is dynamic profiling. If for example we wanted to profile a function in a dependency to which we only have read-only access (e.g. system python libraries owned by root) we can add the following to settings.py to apply a decorator at runtime:

SILKY_DYNAMIC_PROFILING = [{
    'module': 'path.to.module',
    'function': 'MyClass.bar'
}]

which is roughly equivalent to:

class MyClass(object):
    @silk_profile()
    def bar(self):
        pass

The below summarizes the possibilities:

"""
Dynamic function decorator
"""

SILKY_DYNAMIC_PROFILING = [{
    'module': 'path.to.module',
    'function': 'foo'
}]

# ... is roughly equivalent to
@silk_profile()
def foo():
    pass

"""
Dynamic method decorator
"""

SILKY_DYNAMIC_PROFILING = [{
    'module': 'path.to.module',
    'function': 'MyClass.bar'
}]

# ... is roughly equivalent to
class MyClass(object):

    @silk_profile()
    def bar(self):
        pass

"""
Dynamic code block profiling
"""

SILKY_DYNAMIC_PROFILING = [{
    'module': 'path.to.module',
    'function': 'foo',
    # Line numbers are relative to the function as opposed to the file in which it resides
    'start_line': 1,
    'end_line': 2,
    'name': 'Slow Foo'
}]

# ... is roughly equivalent to
def foo():
    with silk_profile(name='Slow Foo'):
        print (1)
        print (2)
    print(3)
    print(4)

Note that dynamic profiling behaves in a similar fashion to that of the python mock framework in that we modify the function in-place e.g:

""" my.module """
from another.module import foo

# ...do some stuff
foo()
# ...do some other stuff

,we would profile foo by dynamically decorating my.module.foo as opposed to another.module.foo:

SILKY_DYNAMIC_PROFILING = [{
    'module': 'my.module',
    'function': 'foo'
}]

If we were to apply the dynamic profile to the functions source module another.module.foo after it has already been imported, no profiling would be triggered.

Custom Logic for Profiling

Sometimes you may want to dynamically control when the profiler runs. You can write your own logic for when to enable the profiler. To do this add the following to your settings.py:

This setting is mutually exclusive with SILKY_PYTHON_PROFILER and will be used over it if present. It will work with SILKY_DYNAMIC_PROFILING.

def my_custom_logic(request):
    return 'profile_requests' in request.session

SILKY_PYTHON_PROFILER_FUNC = my_custom_logic # profile only session has recording enabled.

You can also use a lambda.

# profile only session has recording enabled.
SILKY_PYTHON_PROFILER_FUNC = lambda request: 'profile_requests' in request.session

Code Generation

Silk currently generates two bits of code per request:

Both are intended for use in replaying the request. The curl command can be used to replay via command-line and the python code can be used within a Django unit test or simply as a standalone script.

Configuration

Authentication/Authorisation

By default anybody can access the Silk user interface by heading to /silk/. To enable your Django auth backend place the following in settings.py:

SILKY_AUTHENTICATION = True  # User must login
SILKY_AUTHORISATION = True  # User must have permissions

If SILKY_AUTHORISATION is True, by default Silk will only authorise users with is_staff attribute set to True.

You can customise this using the following in settings.py:

def my_custom_perms(user):
    return user.is_allowed_to_use_silk

SILKY_PERMISSIONS = my_custom_perms

You can also use a lambda.

SILKY_PERMISSIONS = lambda user: user.is_superuser

Request/Response bodies

By default, Silk will save down the request and response bodies for each request for future viewing no matter how large. If Silk is used in production under heavy volume with large bodies this can have a huge impact on space/time performance. This behaviour can be configured with the following options:

SILKY_MAX_REQUEST_BODY_SIZE = -1  # Silk takes anything <0 as no limit
SILKY_MAX_RESPONSE_BODY_SIZE = 1024  # If response body>1024 bytes, ignore

Meta-Profiling

Sometimes it is useful to be able to see what effect Silk is having on the request/response time. To do this add the following to your settings.py:

SILKY_META = True

Silk will then record how long it takes to save everything down to the database at the end of each request:

Note that in the above screenshot, this means that the request took 29ms (22ms from Django and 7ms from Silk)

Recording a Fraction of Requests

On high-load sites it may be helpful to only record a fraction of the requests that are made. To do this add the following to your settings.py:

Note: This setting is mutually exclusive with SILKY_INTERCEPT_FUNC.

SILKY_INTERCEPT_PERCENT = 50 # log only 50% of requests

Custom Logic for Recording Requests

On high-load sites it may also be helpful to write your own logic for when to intercept requests. To do this add the following to your settings.py:

Note: This setting is mutually exclusive with SILKY_INTERCEPT_PERCENT.

def my_custom_logic(request):
    return 'record_requests' in request.session

SILKY_INTERCEPT_FUNC = my_custom_logic # log only session has recording enabled.

You can also use a lambda.

# log only session has recording enabled.
SILKY_INTERCEPT_FUNC = lambda request: 'record_requests' in request.session

Limiting request/response data

To make sure silky garbage collects old request/response data, a config var can be set to limit the number of request/response rows it stores.

SILKY_MAX_RECORDED_REQUESTS = 10**4

The garbage collection is only run on a percentage of requests to reduce overhead. It can be adjusted with this config:

SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENT = 10

Enable query analysis

To enable query analysis when supported by the dbms a config var can be set in order to execute queries with the analyze features.

SILKY_ANALYZE_QUERIES = True

Clearing logged data

A management command will wipe out all logged data:

python manage.py silk_clear_request_log

Contributing

Jazzband

This is a Jazzband project. By contributing you agree to abide by the Contributor Code of Conduct and follow the guidelines.

Development Environment

Silk features a project named project that can be used for silk development. It has the silk code symlinked so you can work on the sample project and on the silk package at the same time.

In order to setup local development you should first install all the dependencies for the test project. From the root of the project directory:

pip install -r requirements.txt

You will also need to install silk's dependencies. From the root of the git repository:

pip install -e .

At this point your virtual environment should have everything it needs to run both the sample project and silk successfully.

Before running, you must set the DB_ENGINE and DB_NAME environment variables:

export DB_ENGINE=sqlite3
export DB_NAME=db.sqlite3

For other combinations, check tox.ini.

Now from the root of the sample project apply the migrations

python manage.py migrate

Now from the root of the sample project directory start the django server

python manage.py runserver

Running the tests

cd project
./tests/test_migrations.sh
python manage.py test --noinput

Happy profiling!

Issues
  • IntegrityError: duplicate key value violates unique constraint

    IntegrityError: duplicate key value violates unique constraint "silk_response_request_id_key"

    After activating silk certain urls began erring with:

    IntegrityError: duplicate key value violates unique constraint "silk_response_request_id_key" DETAIL: Key (request_id)=(1166) already exists.

    Sentry stack trace: http://toolbox1.tedc.de:9000/bidev/esldj/group/131/

    bug 
    opened by synotna 90
  • Support Django 3.2

    Support Django 3.2

    Currently, support for Django 3.2 is broken, as per ~~this issue.~~ (EDIT: the linked issue isn't related to Django 3.2, though support was broken before the PR I have submitted, and I now believe it is fixed)

    Therefore, I created this issue to track progress on fixing django-silk in Django 3.2

    opened by Jorl17 37
  • Create new release of silk

    Create new release of silk

    I think once the jazzband migration is complete, it would be a good time to make a new release of silk, either manually or by setting up automatic releases.

    opened by albertyw 25
  • Prepare Release 4.0

    Prepare Release 4.0

    Since this is a major release as we are dropping python 2 support, we will move to 4.x instead of 3.x.

    • [x] Check the support with Django 3.0
    • [x] Test django silk if its working fine.
    • [x] Prepare a change log
    opened by nasirhjafri 22
  • `apps.py` added with `default_auto_field` set

    `apps.py` added with `default_auto_field` set

    This is to avoid unnecessary migrations caused by default "id" field type change from AutoField -> BigAutoField in Django 3.2.

    new test added

    • testing if proper AppConfig class is loaded by Django
    • testing if package has no pending migrations and will not generate new migrations when running makemigrations on user-end.
    opened by daadu 19
  • Migrations not working w/ Django 1.6.11

    Migrations not working w/ Django 1.6.11

    Traceback (most recent call last):
      File "manage.py", line 13, in <module>
        execute_from_command_line(sys.argv)
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
        utility.execute()
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
        self.fetch_command(subcommand).run_from_argv(self.argv)
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
        self.execute(*args, **options.__dict__)
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
        output = self.handle(*args, **options)
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/south/management/commands/migrate.py", line 111, in handle
        ignore_ghosts = ignore_ghosts,
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/south/migration/__init__.py", line 173, in migrate_app
        Migrations.calculate_dependencies()
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/south/migration/base.py", line 229, in calculate_dependencies
        migration.calculate_dependencies()
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/south/migration/base.py", line 363, in calculate_dependencies
        for migration in self._get_dependency_objects("depends_on"):
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/south/migration/base.py", line 343, in _get_dependency_objects
        for app, name in getattr(self.migration_class(), attrname, []):
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/south/migration/base.py", line 315, in migration_class
        return self.migration().Migration
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/south/utils/__init__.py", line 62, in method
        value = function(self)
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/south/migration/base.py", line 304, in migration
        raise exceptions.UnknownMigration(self, sys.exc_info())
    south.exceptions.UnknownMigration: Migration 'silk:0001_initial' probably doesn't exist.
    Traceback (most recent call last):
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/south/migration/base.py", line 302, in migration
        migration = __import__(full_name, {}, {}, ['Migration'])
      File "/Users/matthew/.virtualenvs/nauman/lib/python2.7/site-packages/silk/migrations/0001_initial.py", line 4, in <module>
        from django.db import models, migrations
    ImportError: cannot import name migrations
    

    The solution from https://github.com/django-silk/silk/issues/64 did not fix this.

    wontfix 
    opened by sgtsquiggs 17
  • NoReverseMatch: Reverse for 'summary' with arguments '()' and keyword arguments '{}' not found. 0 pattern(s) tried: []

    NoReverseMatch: Reverse for 'summary' with arguments '()' and keyword arguments '{}' not found. 0 pattern(s) tried: []

    I am getting NoReverseMatch error. I tried placing middleware at top but its giving same error.

    These are my middleware classes:

    MIDDLEWARE_CLASSES = (
        'django.contrib.sessions.middleware.SessionMiddleware',
        'subdomains.middleware.SubdomainURLRoutingMiddleware',
        'corsheaders.middleware.CorsMiddleware',
        'django.middleware.common.CommonMiddleware',
        'django.middleware.csrf.CsrfViewMiddleware',
        'django.contrib.auth.middleware.AuthenticationMiddleware',
        'django.contrib.messages.middleware.MessageMiddleware',
        'django.middleware.clickjacking.XFrameOptionsMiddleware',
        'social.apps.django_app.middleware.SocialAuthExceptionMiddleware',
        'django_mobile.middleware.MobileDetectionMiddleware',
        'django_mobile.middleware.SetFlavourMiddleware',
    )
    

    and its the traceback-

    Traceback (most recent call last):
      File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
        self.result = application(self.environ, self.start_response)
      File "/home/ashish/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 64, in __call__
        return self.application(environ, start_response)
      File "/home/ashish/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 168, in __call__
        self.load_middleware()
      File "/home/ashish/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 44, in load_middleware
        mw_class = import_string(middleware_path)
      File "/home/ashish/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/utils/module_loading.py", line 26, in import_string
        module = import_module(module_path)
      File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
        __import__(name)
      File "/home/ashish/.virtualenvs/myapp/local/lib/python2.7/site-packages/silk/middleware.py", line 31, in <module>
        fpath = silky_reverse('summary')
      File "/home/ashish/.virtualenvs/myapp/local/lib/python2.7/site-packages/silk/middleware.py", line 27, in silky_reverse
        r = reverse(name, *args, **kwargs)
      File "/home/ashish/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 551, in reverse
        return iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs))
      File "/home/ashish/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 468, in _reverse_with_prefix
        (lookup_view_s, args, kwargs, len(patterns), patterns))
    NoReverseMatch: Reverse for 'summary' with arguments '()' and keyword arguments '{}' not found. 0 pattern(s) tried: []
    [24/Aug/2015 14:15:14] "GET /silk HTTP/1.1" 500 59
    

    Is this error due to middleware placement or its some other issue ?

    opened by a1Gupta 14
  • Django 2.0 support

    Django 2.0 support

    I have this error after update:

     File "/Users/izvr/Projects/bet/env/lib/python3.6/site-packages/silk/models.py", line 135, in Response
        request = OneToOneField(Request, related_name='response', db_index=True)
    TypeError: __init__() missing 1 required positional argument: 'on_delete'
    
    opened by volodchenkov 13
  • DB Deadlock when stress testing with silk

    DB Deadlock when stress testing with silk

    I got a deadlock error when testing with the latest version of silk (1.1.0).

    Here is the DB output:

            Process 149: DELETE FROM "silk_response" WHERE "silk_response"."request_id" IN ('629f685a-26dc-41ca-ba9d-cc8af1998ef9', '8336effc-217c-47a7-8cc2-6944b8aede7f', 'aa12852b-5f5b-40fa-a647-cc3ddd2ef4ba', '2b2d5e84-e595-4ccf-b06c-78db181b4f24', '3f94c00d-f0c3-4021-8def-703e363e4f8f', 'e88b9162-0455-4e78-ba7b-029694b493d8')
            Process 150: DELETE FROM "silk_response" WHERE "silk_response"."request_id" IN ('629f685a-26dc-41ca-ba9d-cc8af1998ef9', '9ad245bb-4b6a-49cf-b32c-1a547ee8c11f', '93c50572-f6ec-416c-a9d6-42798f1631dd', '1ffd6a86-f93e-48f8-a593-86184e26c417', '3f94c00d-f0c3-4021-8def-703e363e4f8f')
            Process 151: DELETE FROM "silk_response" WHERE "silk_response"."request_id" IN ('629f685a-26dc-41ca-ba9d-cc8af1998ef9', '9ad245bb-4b6a-49cf-b32c-1a547ee8c11f', 'aa12852b-5f5b-40fa-a647-cc3ddd2ef4ba', '8336effc-217c-47a7-8cc2-6944b8aede7f', '2b2d5e84-e595-4ccf-b06c-78db181b4f24', '93c50572-f6ec-416c-a9d6-42798f1631dd', '1ffd6a86-f93e-48f8-a593-86184e26c417', '3f94c00d-f0c3-4021-8def-703e363e4f8f', 'e88b9162-0455-4e78-ba7b-029694b493d8')
            Process 144: DELETE FROM "silk_response" WHERE "silk_response"."request_id" IN ('629f685a-26dc-41ca-ba9d-cc8af1998ef9', '9ad245bb-4b6a-49cf-b32c-1a547ee8c11f', 'aa12852b-5f5b-40fa-a647-cc3ddd2ef4ba', '8336effc-217c-47a7-8cc2-6944b8aede7f', '2b2d5e84-e595-4ccf-b06c-78db181b4f24', '93c50572-f6ec-416c-a9d6-42798f1631dd', '1ffd6a86-f93e-48f8-a593-86184e26c417', '3f94c00d-f0c3-4021-8def-703e363e4f8f', 'e88b9162-0455-4e78-ba7b-029694b493d8')
    2018-01-26 01:03:55.996 UTC [148] HINT:  See server log for query details.
    2018-01-26 01:03:55.996 UTC [148] STATEMENT:  DELETE FROM "silk_response" WHERE "silk_response"."request_id" IN ('629f685a-26dc-41ca-ba9d-cc8af1998ef9', '9ad245bb-4b6a-49cf-b32c-1a547ee8c11f', 'aa12852b-5f5b-40fa-a647-cc3ddd2ef4ba', '8336effc-217c-47a7-8cc2-6944b8aede7f', '2b2d5e84-e595-4ccf-b06c-78db181b4f24', '1ffd6a86-f93e-48f8-a593-86184e26c417', '3f94c00d-f0c3-4021-8def-703e363e4f8f', 'e88b9162-0455-4e78-ba7b-029694b493d8')
    2018-01-26 01:03:56.024 UTC [144] ERROR:  deadlock detected
    2018-01-26 01:03:56.024 UTC [144] DETAIL:  Process 144 waits for AccessExclusiveLock on tuple (443,8) of relation 16579 of database 12994; blocked by process 149.
            Process 149 waits for ShareLock on transaction 152027; blocked by process 150.
            Process 150 waits for AccessExclusiveLock on tuple (440,10) of relation 16579 of database 12994; blocked by process 151.
            Process 151 waits for ShareLock on transaction 152109; blocked by process 144.
    

    Output from pip freeze:

    amqp==2.2.2
    autopep8==1.3.4
    billiard==3.5.0.3
    blessings==1.6.1
    bpython==0.17
    cached-property==1.3.1
    celery==4.1.0
    certifi==2018.1.18
    chardet==3.0.4
    coreapi==2.3.3
    coreschema==0.0.4
    coverage==4.4.2
    curtsies==0.2.11
    Django==1.11.9
    django-auth-ldap==1.3.0
    django-cachalot==1.5.0
    django-extensions==1.9.9
    django-model-utils==3.1.1
    django-prettyjson==0.3.0
    django-rest-swagger==2.1.2
    django-silk==1.1.0
    django-url-filter==0.3.4
    djangorestframework==3.7.7
    drf-extensions==0.3.1
    drf-nested-routers==0.90.0
    enum-compat==0.0.2
    gevent==1.2.2
    gprof2dot==2016.10.13
    greenlet==0.4.12
    gunicorn==19.7.1
    idna==2.6
    itypes==1.1.0
    Jinja2==2.10
    kombu==4.1.0
    Markdown==2.6.11
    MarkupSafe==1.0
    mercury==1.0.0
    mock==2.0.0
    openapi-codec==1.3.2
    pbr==3.1.1
    psycopg2==2.7.3.2
    pycodestyle==2.3.1
    Pygments==2.2.0
    pyldap==2.4.45
    python-dateutil==2.6.1
    python-memcached==1.59
    pytz==2017.3
    requests==2.18.4
    simplejson==3.13.2
    six==1.11.0
    sqlparse==0.2.4
    standardjson==0.3.1
    stevedore==1.28.0
    typing==3.6.4
    uritemplate==3.0.0
    urllib3==1.22
    vine==1.1.4
    wcwidth==0.1.7
    Werkzeug==0.14.1
    

    Thanks -Selim

    opened by selimt 12
  • Quality of life improvements in Requests tab

    Quality of life improvements in Requests tab

    I'm using you're package quite often and wanted to give something back to this project 😄

    This PR focuses on improving experience on Request tab - one improvement per commit:

    • Add message when there are no requests to display: when there are no matches instead of displaying an empty page, let user know that there are no filter results
    • Add 'Clear all filters' button - when user fills in a few filters, clearing them is rather tedious task so I thought it would be nice to have a button to clear filters.
    • Add small margin for filter selects: I think this change makes filter section looks much better.

    This branch is based off 4.2.0 tag, please let me know if I need to rebase.

    opened by glujan 11
  • upgrade 4.1.0 to 4.2.0 cause access to wrong table

    upgrade 4.1.0 to 4.2.0 cause access to wrong table

    I am using silk on a system which does not managed the table, its migration model like this

    migrations.CreateModel(
                name='User',
                fields=[
                    ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                    ('password', models.CharField(max_length=128, verbose_name='password')),
                    ('last_login', models.DateTimeField(blank=True, null=True, verbose_name='last login')),
                    ('is_superuser', models.BooleanField(default=False, help_text='Designates that this user has all permissions without explicitly assigning them.', verbose_name='superuser status')),
                    ('username', models.CharField(error_messages={'unique': 'A user with that username already exists.'}, help_text='Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only.', max_length=150, unique=True, validators=[django.contrib.auth.validators.UnicodeUsernameValidator()], verbose_name='username')),
                    ('first_name', models.CharField(blank=True, max_length=150, verbose_name='first name')),
                    ('last_name', models.CharField(blank=True, max_length=150, verbose_name='last name')),
                    ('email', models.EmailField(blank=True, max_length=254, verbose_name='email address')),
                    ('is_staff', models.BooleanField(default=False, help_text='Designates whether the user can log into this admin site.', verbose_name='staff status')),
                    ('is_active', models.BooleanField(default=True, help_text='Designates whether this user should be treated as active. Unselect this instead of deleting accounts.', verbose_name='active')),
                    ('date_joined', models.DateTimeField(default=django.utils.timezone.now, verbose_name='date joined')),
                    ('display_name', models.CharField(blank=True, max_length=255, null=True)),
                    ('role', models.IntegerField(choices=[(0, 'System'), (1, 'Admin'), (2, 'Operator'), (3, 'CustomerService'), (100, 'Merchant'), (200, 'Platform'), (300, 'Agent'), (1000, 'Player')], default=2)),
                    ('google2fa_secret', models.CharField(max_length=100)),
                    ('update_time', models.DateTimeField(auto_now=True)),
                ],
                options={
                    'db_table': 'backend_user',
                    'managed': False,
                },
                managers=[
                    ('objects', django.contrib.auth.models.UserManager()),
                ],
            )
    

    it is work fine on 4.1.0, but I encounter below issue on 4.2.0

    2021-11-02 23:17:19,237 [WARNING][unicorn_common.middleware:34] - get_jwt_user() UnicornAuthentication failed. message: (1146, "Table 'olympus.backend_user' doesn't exist")
    2021-11-02 23:17:19,296 [ERROR][django.request:224] - Internal Server Error: /games/games/
    Traceback (most recent call last):
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
        return self.cursor.execute(sql, params)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 73, in execute
        return self.cursor.execute(query, args)
      File "/home/venv_olympus/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
        res = self._query(query)
      File "/home/venv_olympus/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
        db.query(q)
      File "/home/venv_olympus/lib/python3.8/site-packages/MySQLdb/connections.py", line 259, in query
        _mysql.connection.query(self, query)
    MySQLdb._exceptions.ProgrammingError: (1146, "Table 'olympus.backend_user' doesn't exist")
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "/home/venv_olympus/lib/python3.8/site-packages/asgiref/sync.py", line 482, in thread_handler
        raise exc_info[1]
      File "/home/venv_olympus/lib/python3.8/site-packages/django/core/handlers/exception.py", line 38, in inner
        response = await get_response(request)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/core/handlers/base.py", line 231, in _get_response_async
        response = await wrapped_callback(request, *callback_args, **callback_kwargs)
      File "/home/venv_olympus/lib/python3.8/site-packages/asgiref/sync.py", line 444, in __call__
        ret = await asyncio.wait_for(future, timeout=None)
      File "/usr/lib/python3.8/asyncio/tasks.py", line 455, in wait_for
        return await fut
      File "/home/venv_olympus/lib/python3.8/site-packages/asgiref/current_thread_executor.py", line 22, in run
        result = self.fn(*self.args, **self.kwargs)
      File "/home/venv_olympus/lib/python3.8/site-packages/asgiref/sync.py", line 486, in thread_handler
        return func(*args, **kwargs)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
        return view_func(*args, **kwargs)
      File "/home/venv_olympus/lib/python3.8/site-packages/rest_framework/viewsets.py", line 125, in view
        return self.dispatch(request, *args, **kwargs)
      File "/home/venv_olympus/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
        response = self.handle_exception(exc)
      File "/home/venv_olympus/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
        self.raise_uncaught_exception(exc)
      File "/home/venv_olympus/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
        raise exc
      File "/home/venv_olympus/lib/python3.8/site-packages/rest_framework/views.py", line 497, in dispatch
        self.initial(request, *args, **kwargs)
      File "/home/venv_olympus/lib/python3.8/site-packages/rest_framework/views.py", line 414, in initial
        self.perform_authentication(request)
      File "/home/venv_olympus/lib/python3.8/site-packages/rest_framework/views.py", line 324, in perform_authentication
        request.user
      File "/home/venv_olympus/lib/python3.8/site-packages/rest_framework/request.py", line 227, in user
        self._authenticate()
      File "/home/venv_olympus/lib/python3.8/site-packages/rest_framework/request.py", line 380, in _authenticate
        user_auth_tuple = authenticator.authenticate(self)
      File "/home/venv_olympus/lib/python3.8/site-packages/rest_framework_simplejwt/authentication.py", line 42, in authenticate
        return self.get_user(validated_token), validated_token
      File "/home/venv_olympus/lib/python3.8/site-packages/unicorn_common/authentication.py", line 25, in get_user
        user = auth_model.objects.get(pk=_id)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
        return getattr(self.get_queryset(), name)(*args, **kwargs)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/models/query.py", line 425, in get
        num = len(clone)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/models/query.py", line 269, in __len__
        self._fetch_all()
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/models/query.py", line 1308, in _fetch_all
        self._result_cache = list(self._iterable_class(self))
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/models/query.py", line 53, in __iter__
        results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
      File "/home/venv_olympus/lib/python3.8/site-packages/silk/sql.py", line 94, in execute_sql
        query_dict['analysis'] = _explain_query(q, params)
      File "/home/venv_olympus/lib/python3.8/site-packages/silk/sql.py", line 56, in _explain_query
        cur.execute(prefixed_query, params)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/backends/utils.py", line 98, in execute
        return super().execute(sql, params)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute
        return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
        return executor(sql, params, many, context)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
        return self.cursor.execute(sql, params)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
        raise dj_exc_value.with_traceback(traceback) from exc_value
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
        return self.cursor.execute(sql, params)
      File "/home/venv_olympus/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 73, in execute
        return self.cursor.execute(query, args)
      File "/home/venv_olympus/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
        res = self._query(query)
      File "/home/venv_olympus/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
        db.query(q)
      File "/home/venv_olympus/lib/python3.8/site-packages/MySQLdb/connections.py", line 259, in query
        _mysql.connection.query(self, query)
    django.db.utils.ProgrammingError: (1146, "Table 'olympus.backend_user' doesn't exist")
    

    it seems something change from 4.1.0 to 4.2.0 at below function

      File "/home/venv_olympus/lib/python3.8/site-packages/silk/sql.py", line 94, in execute_sql
        query_dict['analysis'] = _explain_query(q, params)
      File "/home/venv_olympus/lib/python3.8/site-packages/silk/sql.py", line 56, in _explain_query
        cur.execute(prefixed_query, params)
    

    I am not sure what change cause this issue, but I think it should work the same way like 4.1.0 without error.

    opened by caesarleong 11
  • EXPLAIN query are executed despite SILKY_ANALYZE_QUERIES=False

    EXPLAIN query are executed despite SILKY_ANALYZE_QUERIES=False

    My settings are:

    SILKY_PYTHON_PROFILER = True
    SILKY_META = True
    SILKY_AUTHENTICATION = True  # User must login
    SILKY_AUTHORISATION = True  # User must have permissions
    SILKY_PYTHON_PROFILER_BINARY = True
    SILKY_IGNORE_PATHS = ["/api/healthz/"]
    SILKY_MAX_RECORDED_REQUESTS = 10**4
    SILKY_ANALYZE_QUERIES = False
    

    But when I inspect the queries that were executed using 'connection.queries', I see that all the queries are doubled by silk to get the "explain" of the query. For instance:

    SELECT "gdw_tag"."kind", ARRAY_AGG("gdw_tag"."id" ) AS "ids" FROM "gdw_tag" WHERE "gdw_tag"."id" IN (17) GROUP BY "gdw_tag"."kind" ORDER BY "gdw_tag"."kind" ASC
    EXPLAIN SELECT "gdw_tag"."kind", ARRAY_AGG("gdw_tag"."id" ) AS "ids" FROM "gdw_tag" WHERE "gdw_tag"."id" IN (17) GROUP BY "gdw_tag"."kind" ORDER BY "gdw_tag"."kind" ASC
    
    

    How can I suppress this ?

    Thanks in advance Loic

    opened by quertenmont 0
  • Proposal: Drop dependency on pytz in favor of zoneinfo

    Proposal: Drop dependency on pytz in favor of zoneinfo

    Hi, pytz is/was a good llibrary. But it's obsolete since PEP 615. In the long run, you should not use this 3rd party package, but the language integrated functionality. Even big frameworks like Django (Django 4.0 Release Info) already dropped pytz.

    opened by aDramaQueen 0
  • Column

    Column "silk_request.path" must appear in the GROUP BY clause

    I have tested this on silk 4.2.0, 4.3.0, and Django 3.0.4

    SELECT "silk_request"."id", "silk_request"."path", "silk_request"."query_params", "silk_request"."raw_body", "silk_request"."body", "silk_request"."method", "silk_request"."start_time", "silk_request"."view_name", "silk_request"."end_time", "silk_request"."time_taken", "silk_request"."encoded_headers", "silk_request"."meta_time", "silk_request"."meta_num_queries", "silk_request"."meta_time_spent_queries", "silk_request"."pyprofile", "silk_request"."prof_file", "silk_request"."num_sql_queries", COUNT("silk_sqlquery"."id") AS "t" FROM "silk_request" LEFT OUTER JOIN "silk_sqlquery" ON ("silk_request"."id" = "silk_sqlquery"."request_id") GROUP BY "silk_request"."id" ORDER BY "t" DESC LIMIT 1

    The following query is generated in the following function: _time_spent_in_db_by_view

    I have seen this issue in the older versions, but it seemed to be fixed.

    opened by fatalispm 0
  • [BUG] Encoding SQL queries to UTF8

    [BUG] Encoding SQL queries to UTF8

    Explain query function is trying to encode the query to utf8, which may raise an exception when there is no valid encode for certain chars in the query.

    File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 317, in __getitem__
        qs._fetch_all()
      File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 1324, in _fetch_all
        self._result_cache = list(self._iterable_class(self))
      File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 51, in __iter__
        results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
      File "/usr/local/lib/python3.10/site-packages/silk/sql.py", line 93, in execute_sql
        query_dict['analysis'] = _explain_query(self.connection, q, params)
      File "/usr/local/lib/python3.10/site-packages/silk/sql.py", line 55, in _explain_query
        cur.execute(prefixed_query, params)
      File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 66, in execute
        return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
      File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
        return executor(sql, params, many, context)
      File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
        return self.cursor.execute(sql, params)
      File "/usr/local/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 73, in execute
        return self.cursor.execute(query, args)
      File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 206, in execute
        res = self._query(query)
      File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 321, in _query
        self._post_get_result()
      File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 355, in _post_get_result
        self._rows = self._fetch_row(0)
      File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 328, in _fetch_row
        return self._result.fetch_row(size, self._fetch_type)
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb4 in position 550: invalid start byte
    
    opened by yuri1992 0
  • Attempt to access queries without initialisation. Is the middleware installed correctly?

    Attempt to access queries without initialisation. Is the middleware installed correctly?

    I am getting this error when trying to silk_profile decorator .i.e. @silk_profile(name='...')

    backend_1 | Exception when performing meta profiling, dumping trace below backend_1 | Traceback (most recent call last): backend_1 | File "/usr/local/lib/python3.9/site-packages/silk/profiling/profiler.py", line 159, in wrapped_target backend_1 | self._start_queries() backend_1 | File "/usr/local/lib/python3.9/site-packages/silk/profiling/profiler.py", line 78, in _start_queries backend_1 | self._queries_before = self._query_identifiers_from_collector() backend_1 | File "/usr/local/lib/python3.9/site-packages/silk/profiling/profiler.py", line 74, in _query_identifiers_from_collector backend_1 | return [x for x in DataCollector().queries] backend_1 | File "/usr/local/lib/python3.9/site-packages/silk/collector.py", line 67, in queries backend_1 | return self._get_objects(TYP_QUERIES) backend_1 | File "/usr/local/lib/python3.9/site-packages/silk/collector.py", line 76, in _get_objects backend_1 | self._raise_not_configured( backend_1 | File "/usr/local/lib/python3.9/site-packages/silk/collector.py", line 99, in _raise_not_configured backend_1 | raise SilkNotConfigured(err + ' Is the middleware installed correctly?') backend_1 | silk.errors.SilkNotConfigured: Attempt to access queries without initialisation. Is the middleware installed correctly? backend_1 | Attempt to access queries without initialisation. Is the middleware installed correctly?

    in my setting page SILKY_PYTHON_PROFILER = True SILKY_META = True

    Added silk in app section silk

    Added middleware 'silk.middleware.SilkyMiddleware',

    FTR, Also I have tried placing the silk middleware in different indexes in the MIDDLEWARE list on Django setting page. Can someone please help me if I am missing any step or doing something wrong?

    opened by Joish 0
Releases(5.0.1)
  • 5.0.1(Jul 3, 2022)

  • 4.4.1(Jul 3, 2022)

  • 5.0.0(Jun 27, 2022)

  • 4.4.0(Jun 27, 2022)

    4.4.0 (2022-06-20)

    :release-by: Albert Wang (@albertyw) Full Changelog

    Features/Enhancements:

    • Switch 'Apply' and 'Clear all filters' ordering
    • Make filters on Requests tab more visible
    • Add small margin for filter selects
    • Add 'Clear all filters' button
    • Add message when there are no requests to display
    • Making the error logging more accurate and explicit
    • Fixing #530 - Adding support for SILKY_EXPLAIN_FLAGS

    Maintenance and Cleanup:

    • Remove unused js compilation pipeline (#561)
    • Fix pre-commit-config

    Dependencies:

    • Update jquery to 3.6.0 and jquery-ui to 1.13.1 [#508]
    • [pre-commit.ci] pre-commit autoupdate (#560, #571)
    • Add django-upgrade to pre-commit hooks (#566)

    Moved to 5.0.0

    • Drop support for Django 2.2 (EOL) (#567)
    Source code(tar.gz)
    Source code(zip)
  • 4.3.0(Mar 5, 2022)

    What's Changed

    • Jazzband: Synced file(s) with jazzband/.github by @jazzband-bot in https://github.com/jazzband/django-silk/pull/518
    • [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/jazzband/django-silk/pull/520
    • Fix installation instructions in README by @side2k in https://github.com/jazzband/django-silk/pull/521
    • [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/jazzband/django-silk/pull/523
    • [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/jazzband/django-silk/pull/524
    • [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/jazzband/django-silk/pull/526
    • Add Python 3.10 compatibility by @hramezani in https://github.com/jazzband/django-silk/pull/527
    • Django main by @hramezani in https://github.com/jazzband/django-silk/pull/528
    • Remove unneeded dependency Pygments by @jayvdb in https://github.com/jazzband/django-silk/pull/532
    • [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/jazzband/django-silk/pull/533
    • [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/jazzband/django-silk/pull/535
    • Replace assertDictContainsSubset by @albertyw in https://github.com/jazzband/django-silk/pull/536
    • Use correct db in a multi db setup by @glujan in https://github.com/jazzband/django-silk/pull/538
    • Update django version by @eduzen in https://github.com/jazzband/django-silk/pull/544
    • [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/jazzband/django-silk/pull/552
    • Some tests misusing assertTrue for comparisons fix by @code-review-doctor in https://github.com/jazzband/django-silk/pull/550
    • Standardize supported django versions by @albertyw in https://github.com/jazzband/django-silk/pull/553
    • Drop support for python 3.6 by @albertyw in https://github.com/jazzband/django-silk/pull/554
    • Remove duplicate requirements.txt dependencies by @albertyw in https://github.com/jazzband/django-silk/pull/555
    • Pin dependencies in requirements.txt by @albertyw in https://github.com/jazzband/django-silk/pull/556
    • Release version v4.3.0 by @albertyw in https://github.com/jazzband/django-silk/pull/557

    New Contributors

    • @jazzband-bot made their first contribution in https://github.com/jazzband/django-silk/pull/518
    • @side2k made their first contribution in https://github.com/jazzband/django-silk/pull/521
    • @jayvdb made their first contribution in https://github.com/jazzband/django-silk/pull/532
    • @glujan made their first contribution in https://github.com/jazzband/django-silk/pull/538
    • @eduzen made their first contribution in https://github.com/jazzband/django-silk/pull/544
    • @code-review-doctor made their first contribution in https://github.com/jazzband/django-silk/pull/550

    Full Changelog: https://github.com/jazzband/django-silk/compare/4.2.0...4.3.0

    Source code(tar.gz)
    Source code(zip)
  • 4.2.0(Oct 23, 2021)

    What's Changed

    • #427 Passed WSGI request to SILKY_PYTHON_PROFILER_FUNC by @nasirhjafri in https://github.com/jazzband/django-silk/pull/438
    • Added Django 3.1 to Travis test matrix by @smithdc1 in https://github.com/jazzband/django-silk/pull/441
    • Do not crash when silk app is not included in urls by @asalahli in https://github.com/jazzband/django-silk/pull/443
    • Add the SILKY_JSON_ENSURE_ASCII configuration item to support Chinese by @eshxcmhk in https://github.com/jazzband/django-silk/pull/420
    • Add row view for requests page by @mierz00 in https://github.com/jazzband/django-silk/pull/440
    • RequestModelFactory: fallback if request body too large, fix #162 by @ge0rg in https://github.com/jazzband/django-silk/pull/451
    • Add query execution plan to sql_detail by @MRoci in https://github.com/jazzband/django-silk/pull/452
    • Add Python 3.9 compatibility by @tirkarthi in https://github.com/jazzband/django-silk/pull/404
    • Replace url with path for Django 4.0 by @MuslimBeibytuly in https://github.com/jazzband/django-silk/pull/445
    • Migrate to GitHub Actions. by @jezdez in https://github.com/jazzband/django-silk/pull/460
    • Rename Django's dev branch to main. by @jezdez in https://github.com/jazzband/django-silk/pull/463
    • Fix transaction error for mysql by @yuekui in https://github.com/jazzband/django-silk/pull/464
    • Fix django 3.2 support by @Jorl17 in https://github.com/jazzband/django-silk/pull/466
    • parse query when counting joins by @MRoci in https://github.com/jazzband/django-silk/pull/453
    • Update django support by @hramezani in https://github.com/jazzband/django-silk/pull/467
    • Temporary fix for testing Django 2.2 by @Andrew-Chen-Wang in https://github.com/jazzband/django-silk/pull/486
    • Fix egg metadata error by @Archmonger in https://github.com/jazzband/django-silk/pull/485
    • DOC: Remove outdated security notice by @MT-Cash in https://github.com/jazzband/django-silk/pull/491
    • fix incorrectly made decorator by @Archmonger in https://github.com/jazzband/django-silk/pull/492
    • Fixed a bug that the profile tab could not be opened when the source … by @Taikono-Himazin in https://github.com/jazzband/django-silk/pull/490
    • Generate missing row.css from sass file by @przxmek in https://github.com/jazzband/django-silk/pull/480
    • Ensure sorting between most db queries requests by @ErwinJunge in https://github.com/jazzband/django-silk/pull/484
    • Don't return null values when computing "Most Time Overall" summary by @ErwinJunge in https://github.com/jazzband/django-silk/pull/482
    • fix: DB connection to ClearDB when multiple databases by @prayashm in https://github.com/jazzband/django-silk/pull/469
    • Don't return null values when computing "Most Time Overall" summary by @ErwinJunge in https://github.com/jazzband/django-silk/pull/483
    • fix: DataCollector sql_queries model not found on filter(request=self… by @armanjtehrani in https://github.com/jazzband/django-silk/pull/476
    • Add tests that access the actual DB by @marius-mather in https://github.com/jazzband/django-silk/pull/493
    • Update Django Support in the README.md by @starryrbs in https://github.com/jazzband/django-silk/pull/474
    • apps.py added with default_auto_field set by @daadu in https://github.com/jazzband/django-silk/pull/495
    • Make sensitive keys set configurable by @jeffrey-effendy in https://github.com/jazzband/django-silk/pull/494
    • Revert "apps.py added with default_auto_field set" by @auvipy in https://github.com/jazzband/django-silk/pull/497
    • remove python 2 style super() call from models by @auvipy in https://github.com/jazzband/django-silk/pull/499
    • remove python 2 style codes from across the codebase by @auvipy in https://github.com/jazzband/django-silk/pull/500
    • doc(readme): minor correction by @daadu in https://github.com/jazzband/django-silk/pull/501
    • Fix broken test on Windows 10 (SQLite) by @Archmonger in https://github.com/jazzband/django-silk/pull/504
    • Remove Make Migrations by @Archmonger in https://github.com/jazzband/django-silk/pull/503
    • Support Simple Pre-Commit CI by @Archmonger in https://github.com/jazzband/django-silk/pull/507
    • Add isort to CI by @Archmonger in https://github.com/jazzband/django-silk/pull/509
    • Remove garbage files by @Archmonger in https://github.com/jazzband/django-silk/pull/510
    • Add pyupgrade to CI by @Archmonger in https://github.com/jazzband/django-silk/pull/511
    • Add flake8, autopep8, and pycln to CI by @Archmonger in https://github.com/jazzband/django-silk/pull/512
    • Remove obsolete files by @Archmonger in https://github.com/jazzband/django-silk/pull/513
    • [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/jazzband/django-silk/pull/516
    • Docln by @auvipy in https://github.com/jazzband/django-silk/pull/519

    New Contributors

    • @smithdc1 made their first contribution in https://github.com/jazzband/django-silk/pull/441
    • @asalahli made their first contribution in https://github.com/jazzband/django-silk/pull/443
    • @mierz00 made their first contribution in https://github.com/jazzband/django-silk/pull/440
    • @ge0rg made their first contribution in https://github.com/jazzband/django-silk/pull/451
    • @MRoci made their first contribution in https://github.com/jazzband/django-silk/pull/452
    • @tirkarthi made their first contribution in https://github.com/jazzband/django-silk/pull/404
    • @MuslimBeibytuly made their first contribution in https://github.com/jazzband/django-silk/pull/445
    • @jezdez made their first contribution in https://github.com/jazzband/django-silk/pull/460
    • @yuekui made their first contribution in https://github.com/jazzband/django-silk/pull/464
    • @Jorl17 made their first contribution in https://github.com/jazzband/django-silk/pull/466
    • @hramezani made their first contribution in https://github.com/jazzband/django-silk/pull/467
    • @Andrew-Chen-Wang made their first contribution in https://github.com/jazzband/django-silk/pull/486
    • @Archmonger made their first contribution in https://github.com/jazzband/django-silk/pull/485
    • @MT-Cash made their first contribution in https://github.com/jazzband/django-silk/pull/491
    • @Taikono-Himazin made their first contribution in https://github.com/jazzband/django-silk/pull/490
    • @przxmek made their first contribution in https://github.com/jazzband/django-silk/pull/480
    • @ErwinJunge made their first contribution in https://github.com/jazzband/django-silk/pull/484
    • @prayashm made their first contribution in https://github.com/jazzband/django-silk/pull/469
    • @armanjtehrani made their first contribution in https://github.com/jazzband/django-silk/pull/476
    • @marius-mather made their first contribution in https://github.com/jazzband/django-silk/pull/493
    • @starryrbs made their first contribution in https://github.com/jazzband/django-silk/pull/474
    • @daadu made their first contribution in https://github.com/jazzband/django-silk/pull/495
    • @jeffrey-effendy made their first contribution in https://github.com/jazzband/django-silk/pull/494
    • @pre-commit-ci made their first contribution in https://github.com/jazzband/django-silk/pull/516

    Full Changelog: https://github.com/jazzband/django-silk/compare/4.1.0...4.2.0

    Source code(tar.gz)
    Source code(zip)
  • 4.1.0(Aug 7, 2020)

    4.1.0 (2020-08-07)

    Full Changelog

    New features/Implemented enhancements:

    Fixed bugs:

    Closed issues:

    • _mask_credentials uses UGC in a regex substitution #410 (barm)
    • Django Silk is not compatible with Django 3.1: EmptyResultSet is removed in Django 3.1 #431 (Tirzono)

    Merged pull requests:

    • Wrap re.sub() in try-except #412 (bambookchos)
    • Replace the call to re.findall with re.sub in _mask_credentials so matched values are not treated as regex patterns #413 (ThePumpingLemma)
    • Capture entire key name during cleansing in _mask_credentials #414 (ThePumpingLemma)
    • Clear DB error when configuring silk to use a non-' default' database #417 (eshxcmhk)
    • Fix force_text RemovedInDjango40Warning #422 (justinmayhew)
    • Make compatible with Django 3.1 #432 (Tirzono)
    • Update README.md django-silk is tested with Django 3.1 #433 (Tirzono)
    Source code(tar.gz)
    Source code(zip)
  • 4.0.1(Mar 12, 2020)

    4.0.1 (2020-03-12)

    Full Changelog

    New features/Implemented enhancements:

    Fixed bugs:

    Closed issues:

    • The "Clear DB" page doesn't work with PostgreSQL #395 (Ikalou)

    Merged pull requests:

    Source code(tar.gz)
    Source code(zip)
  • 4.0.0(Jan 21, 2020)

    New features/Implemented enhancements:

    Fixed bugs:

    Closed issues:

    • Ability to clean up all requests/queries #365
    • Use bulk_create to save number of queries #369
    • Headers are not sanitized #375
    • Django 3 support #382
    • Support functional cProfile enable #390

    Merged pull requests:

    Source code(tar.gz)
    Source code(zip)
  • 3.0.4(Sep 5, 2019)

    Implemented enhancements:

    • templates: limit select width to its container one #351 (xrmx)
    • Clean up RemovedInDjango30Warning with {% load staticfiles %} #353 (devmonkey22)
    • Simplify pattern masking and handle dicts #355 (Chris7)

    Fixed bugs:

    Closed issues:

    • Clean up RemovedInDjango30Warning warning re load staticfiles in Django 2.1+ #352

    Merged pull requests:

    • Fix masking sensitive data in batch JSON request #342 (nikolaik)
    • Fix project url on PyPi #343 (luzfcb)
    • templates: limit select width to its container one #351 (xrmx)
    • Clean up RemovedInDjango30Warning with {% load staticfiles %} #353 (devmonkey22)
    • Simplify pattern masking and handle dicts #355 (Chris7)
    Source code(tar.gz)
    Source code(zip)
  • 3.0.3(Aug 13, 2019)

    Implemented enhancements:

    • templates: limit select width to its container one #351 (xrmx)
    • Clean up RemovedInDjango30Warning with {% load staticfiles %} #353 (devmonkey22)
    • Simplify pattern masking and handle dicts #355 (Chris7)

    Fixed bugs:

    Closed issues:

    • Clean up RemovedInDjango30Warning warning re load staticfiles in Django 2.1+ #352

    Merged pull requests:

    • Fix masking sensitive data in batch JSON request #342 (nikolaik)
    • Fix project url on PyPi #343 (luzfcb)
    • templates: limit select width to its container one #351 (xrmx)
    • Clean up RemovedInDjango30Warning with {% load staticfiles %} #353 (devmonkey22)
    • Simplify pattern masking and handle dicts #355 (Chris7)
    Source code(tar.gz)
    Source code(zip)
  • 3.0.2(Apr 23, 2019)

    Implemented enhancements:

    Fixed bugs:

    • Long url path causes Http 500 #312

    Closed issues:

    • Permission checking is skipped due to order of silk_profile decorator #336
    • Support gprof2dot 2017.09.19 #332
    • Duplicate #310 #328
    • Profiling management commands #327
    • NoReverseMatch at /cart/detail/ Reverse for 'cart_add' with arguments not found. #324
    • Request body sanitization #305
    • How to profile middleware? #303
    • Disabling Silk for specific URLs #292
    • silk_clear_request_log fails on Postgres #290
    • silk profile is not work, with dango-version 2.0.2 and django-silk version 2.0.0 #277
    • DataError: value too long for type character varying(190) #179

    Merged pull requests:

    Source code(tar.gz)
    Source code(zip)
  • 3.0.1(Jul 3, 2018)

  • 3.0.0(May 28, 2018)

    • Dropped support for Django<1.11.0
    • Dropped support for python 3.3
    • Fix string/byte typing (#268)
    • Fix collecting meta profiling in Django > 2 (#274)
    • Fix FORCE_SCRIPT_NAME (#278)
    • Revert opening sql queries in new tab
    Source code(tar.gz)
    Source code(zip)
  • 2.0.0(Jan 28, 2018)

    • Fix migration for request.prof_file field (#245)
    • Dropped support for Django 1.5
    • Dropped support for Django 1.6
    • Dropped support for Django 1.7 (#247)
    • Updated documentation (#250)
    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Dec 27, 2017)

    • Support for Django 2 and Python 3.6
    • Support garbage collecting old profiling data
    • Moved project to jazzband
    • Better data visualizations (#194)
    • Added ability to filter requests by http verb (#192)
    • Various UI fixes
    • Various bug and testing fixes
    Source code(tar.gz)
    Source code(zip)
  • 1.0.0(Mar 25, 2017)

    • Improved MySQL support (#167 , #38)
      • Changes length of Request.path and Request.view_name fields
      • Length changes accounts for utf8bm4
    • Updates travis matrix and requirements dependencies versions (#165)
      • Drops Django 1.7 Support & starts Django 1.11 testing
    • Fixes scenario where ValueError exception is raised therefore causing no profiling to register (#161 , #164)
    • Adds configuration for wheels support (#168 , #149)
    • Adds Code style improvements for readability (#166)

    IMPORTANT As a major version release, expect some unintended quirks over existing installations. @smaccona highlighted a couple MySQL scenario's here . PostgreSQL users should be fine. Upgrade accordingly.

    Thanks to ( @auvipy, @smaccona , @PeRDy )

    Source code(tar.gz)
    Source code(zip)
  • 0.7.3(Feb 13, 2017)

    • Improves profile storage location (#152, #151 )
      • ProfileDownloadView now available.
    • Stores cProfiler into threaded local storage (#153, #155)
    • Upgrades testing library.

    Thanks to ( @r3m0t , @Drache91).

    Source code(tar.gz)
    Source code(zip)
  • 0.7.2(Dec 3, 2016)

    • Fixes Django 1.10 middleware instantiation issue #148
    • Fixes Middleware exception handling to be SQL agnostic #142, #144, #146, #150
    • Adds TravisCI testing of various SQL backends #145

    Thanks to ( @mattjegan , @hanleyhansen , @lockie )

    Source code(tar.gz)
    Source code(zip)
  • 0.7.1(Oct 1, 2016)

    • In the event of an SQL OperationalError while trying to process a request due to a large query, a try catch was introduced to retry the processing (#140).

    Thanks to @hanleyhansen.

    Source code(tar.gz)
    Source code(zip)
  • 0.7.0(Sep 21, 2016)

    • You can now choose to save binary profile to a path/location of your choosing. (#131)
    • Middleware update to SilkyMiddleware allows for Django 1.10 compatibility. (#138)
    • csrf imports in requests.py have been swapped to avoid emitting errors. (#136)
    • Manifest file update to allow compilation of project on Windows machines. (#139)
    • Logging has been namespaced for finer-grained control of logging output across Silk components. (#133)

    Thanks to (@javaguirre, @blag, @shanx, @leifdenby, @aehlke )

    Source code(tar.gz)
    Source code(zip)
  • 0.6.2(Jul 28, 2016)

  • 0.6.1(Jul 13, 2016)

    • Fix for install issue via setup.py file (#125, #126 , #127 , #128 ).
      • Manifest file includes all readme files.

    Thanks to (@hanleyhansen, @chid, @SzySteve)

    Source code(tar.gz)
    Source code(zip)
  • 0.6.0(Jul 12, 2016)

    • Ability to export .profile as binary downloads
    • Fixes to local development project to allow continued development
    • Fixes to all unit tests for a green TravisCI build
    • Usage of Django's six module versus a statically imported version
    • Upgrade of Pillow requirement for successful installation

    Thanks to: ( @hanleyhansen, @auvipy, @chrono, @svisser )

    Source code(tar.gz)
    Source code(zip)
  • 0.5.7(Mar 21, 2016)

    • Django 1.9 support
    • Base64 encoding of raw request body for support of binary responses. (i.e. images)
    • Ability to sort ascending/descending order in the RequestsView.
    • Unit test enhancements.
    • Stability tweaks and small performance improvements.

    Thanks to ( @Alkalit , @trik , @blag , @Wrhector , @digitaldavenyc )

    Source code(tar.gz)
    Source code(zip)
  • 0.5.6(Sep 6, 2015)

  • 0.5.2(Apr 15, 2015)

  • 0.5.1(Apr 8, 2015)

  • 0.5(Apr 8, 2015)

  • v0.4(Aug 17, 2014)

    Thanks to @JoshData for the following:

    • Adds a setting to not log every request, which makes this easier to use on a high-load site
    • Use Python's cProfile class to profile all method calls in a request (with a setting to turn it on)
    • Adds a management command to clear the request/response/etc. tables
    • Doesn't compute a reverse or load the configuration on every request
    • Adds some indexes to the models
    • On the SQL queries page, display the query start time as relative to the request start time
    Source code(tar.gz)
    Source code(zip)
    django-silk-0.4.tar.gz(1.13 MB)
Owner
Jazzband
Jazzband
Mixer -- Is a fixtures replacement. Supported Django, Flask, SqlAlchemy and custom python objects.

The Mixer is a helper to generate instances of Django or SQLAlchemy models. It's useful for testing and fixture replacement. Fast and convenient test-

Kirill Klenov 843 Aug 2, 2022
A Django plugin for pytest.

Welcome to pytest-django! pytest-django allows you to test your Django project/applications with the pytest testing tool. Quick start / tutorial Chang

pytest-dev 1.1k Aug 12, 2022
Django test runner using nose

django-nose django-nose provides all the goodness of nose in your Django tests, like: Testing just your apps by default, not all the standard ones tha

Jazzband 871 Jul 26, 2022
Useful additions to Django's default TestCase

django-test-plus Useful additions to Django's default TestCase from REVSYS Rationale Let's face it, writing tests isn't always fun. Part of the reason

REVSYS 523 Aug 4, 2022
Mixer -- Is a fixtures replacement. Supported Django, Flask, SqlAlchemy and custom python objects.

The Mixer is a helper to generate instances of Django or SQLAlchemy models. It's useful for testing and fixture replacement. Fast and convenient test-

Kirill Klenov 741 Feb 4, 2021
Object factory for Django

Model Bakery: Smart fixtures for better tests Model Bakery offers you a smart way to create fixtures for testing in Django. With a simple and powerful

Model Bakers 560 Jul 31, 2022
Test django schema and data migrations, including migrations' order and best practices.

django-test-migrations Features Allows to test django schema and data migrations Allows to test both forward and rollback migrations Allows to test th

wemake.services 342 Aug 14, 2022
Useful additions to Django's default TestCase

django-test-plus Useful additions to Django's default TestCase from REVSYS Rationale Let's face it, writing tests isn't always fun. Part of the reason

REVSYS 524 Aug 6, 2022
A feature flipper for Django

README Django Waffle is (yet another) feature flipper for Django. You can define the conditions for which a flag should be active, and use it in a num

null 925 Aug 3, 2022
Silky smooth profiling for Django

Silk Silk is a live profiling and inspection tool for the Django framework. Silk intercepts and stores HTTP requests and database queries before prese

Jazzband 3.4k Aug 11, 2022
Silky smooth profiling for Django

Silk Silk is a live profiling and inspection tool for the Django framework. Silk intercepts and stores HTTP requests and database queries before prese

Jazzband 3.4k Aug 8, 2022
PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Smooth ReLU in PyTorch Unofficial PyTorch reimplementation of the Smooth ReLU (SmeLU) activation function proposed in the paper Real World Large Scale

Christoph Reich 8 May 31, 2022
Dual languaged (rus+eng) tool for packing and unpacking archives of Silky Engine.

SilkyArcTool English Dual languaged (rus+eng) GUI tool for packing and unpacking archives of Silky Engine. It is not the same arc as used in Ai6WIN. I

Tester 3 May 24, 2022
Silk is a live profiling and inspection tool for the Django framework.

Silk is a live profiling and inspection tool for the Django framework. Silk intercepts and stores HTTP requests and database queries before presenting them in a user interface for further inspection:

Jazzband 3.5k Aug 13, 2022
This code extends the neural style transfer image processing technique to video by generating smooth transitions between several reference style images

Neural Style Transfer Transition Video Processing By Brycen Westgarth and Tristan Jogminas Description This code extends the neural style transfer ima

Brycen Westgarth 98 Aug 4, 2022
Tensorflow Implementation of SMU: SMOOTH ACTIVATION FUNCTION FOR DEEP NETWORKS USING SMOOTHING MAXIMUM TECHNIQUE

SMU A Tensorflow Implementation of SMU: SMOOTH ACTIVATION FUNCTION FOR DEEP NETWORKS USING SMOOTHING MAXIMUM TECHNIQUE arXiv https://arxiv.org/abs/211

Fuhang 5 Jan 18, 2022
A smooth and powerful Telegram Userbot made to make Telegram easier.

| Xᴇɴᴏ Bᴏᴛ Is One Of The Fastest & Smoothest Bot On Telegram Based on Telethon|

SimpleBoy 1 Dec 1, 2021
(OLD REPO) Line-by-line profiling for Python - Current repo ->

line_profiler and kernprof line_profiler is a module for doing line-by-line profiling of functions. kernprof is a convenient script for running either

Robert Kern 3.6k Aug 11, 2022
Soda SQL Data testing, monitoring and profiling for SQL accessible data.

Soda SQL Data testing, monitoring and profiling for SQL accessible data. What does Soda SQL do? Soda SQL allows you to Stop your pipeline when bad dat

Soda Data Monitoring 38 Jul 15, 2022
Line-by-line profiling for Python

line_profiler and kernprof NOTICE: This is the official line_profiler repository. The most recent version of line-profiler on pypi points to this repo

OpenPyUtils 1.3k Aug 11, 2022
Create HTML profiling reports from pandas DataFrame objects

Pandas Profiling Documentation | Slack | Stack Overflow Generates profile reports from a pandas DataFrame. The pandas df.describe() function is great

null 9.3k Aug 10, 2022
Create HTML profiling reports from pandas DataFrame objects

Pandas Profiling Documentation | Slack | Stack Overflow Generates profile reports from a pandas DataFrame. The pandas df.describe() function is great

null 6.8k Feb 18, 2021
Create HTML profiling reports from pandas DataFrame objects

Pandas Profiling Documentation | Slack | Stack Overflow Generates profile reports from a pandas DataFrame. The pandas df.describe() function is great

null 9.3k Aug 8, 2022
Create HTML profiling reports from pandas DataFrame objects

Pandas Profiling Documentation | Slack | Stack Overflow Generates profile reports from a pandas DataFrame. The pandas df.describe() function is great

null 9.3k Aug 8, 2022
An ultra fast tiny model for lane detection, using onnx_parser, TensorRTAPI, torch2trt to accelerate. our model support for int8, dynamic input and profiling. (Nvidia-Alibaba-TensoRT-hackathon2021)

Ultra_Fast_Lane_Detection_TensorRT An ultra fast tiny model for lane detection, using onnx_parser, TensorRTAPI to accelerate. our model support for in

steven.yan 116 Aug 8, 2022
Digital Twin Mobility Profiling: A Spatio-Temporal Graph Learning Approach

Digital Twin Mobility Profiling: A Spatio-Temporal Graph Learning Approach This is the implementation of traffic prediction code in DTMP based on PyTo

chenxin 1 Dec 19, 2021
Source code for CIKM 2021 paper for Relation-aware Heterogeneous Graph for User Profiling

RHGN Source code for CIKM 2021 paper for Relation-aware Heterogeneous Graph for User Profiling Dependencies torch==1.6.0 torchvision==0.7.0 dgl==0.7.1

Big Data and Multi-modal Computing Group, CRIPAC 4 Jun 22, 2022
VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.

VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.

null 2.3k Aug 15, 2022
Intelligent Employer Profiling Platform.

Intelligent Employer Profiling Platform Setup Instructions Generating Model Data Ensure that Python 3.9+ and pip is installed. Install project depende

Harvey Donnelly 2 Jan 9, 2022