Django package to log request values such as device, IP address, user CPU time, system CPU time, No of queries, SQL time, no of cache calls, missing, setting data cache calls for a particular URL with a basic UI.

Overview

django-web-profiler's documentation:

Documentation Status https://travis-ci.org/MicroPyramid/django-web-profiler.svg?branch=master Latest Release https://coveralls.io/repos/github/MicroPyramid/django-web-profiler/badge.svg?branch=master Code Health Latest Release

Introduction:

django-web-profiler is a django profiling tool which logs, stores debug toolbar statistics and also a set of URL's statistics using a management command. It logs request values such as device, ip address, user cpu time, system cpu time, No of queries, sql time, no of cache calls, missing, setting data cache calls for a particular url.

It provides a basic UI, which will differentiate development url statistics, production level statistics which generates using a management command.

Source Code is available in Micropyramid Repository(https://github.com/MicroPyramid/django-web-profiler).

Modules used:

  • Python >= 2.6 (or Python 3.4)
  • Django = 1.11.15
  • Django Compressor = 2.1.1
  • Django Debug Toolbar = 1.8
  • requests = 2.20.0

Installation Procedure

  1. Install django-web-profiler using the following command:

    pip install django-web-profiler
    
            (or)
    
    git clone git://github.com/micropyramid/django-web-profiler.git
    
    cd django-web-profiler
    
    python setup.py install
    
  2. Add app name in settings.py:

    INSTALLED_APPS = [
       '..................',
       'compressor',
       'debug_toolbar',
       'django_web_profiler',
       '..................'
    ]
    
  3. Add 'django_web_profiler.middleware.DebugLoggingMiddleware' to your project middlewares:

    MIDDLEWARE = [
       '.....................................................',
       'django_web_profiler.middleware.DebugLoggingMiddleware',
       '.....................................................',
    ]
    
    Disable 'debug_toolbar.middleware.DebugToolbarMiddleware' if you've already using it.
    
  4. Make sure that 'debug-toolbar' has enabled for your application. After installing debug toolbar, add the following details to settings.py:

    INTERNAL_IPS = ('127.0.0.1',)
    
  5. After installing/cloning, add the following details in settings file about urls, logger names:

    URLS = ['http://stage.testsite.com/', 'http://stage.testsite.com/testing/']
    
  6. Add the following logger to your existing loggers and create a folder called 'logs' where all profiler log files are stored:

        'request-logging': {
            'level': 'DEBUG',
            'handlers': ['console', 'file_log'],
            'propagate': False,
        },
    
    Here file_log is a handler which contains a path where log files are stored.
    

Sample Application

  1. Install application requirements using the following command:

    pip install -r requirements.txt
    
  2. Load the application load using the following command:

    python sandbox/manage.py loaddata sandbox/fixtures/users.json
    
  3. Using the following command, we can generate url statistics in production environment i.e debug=False:

    python sandbox/manage.py logging_urls
    

We are always looking to help you customize the whole or part of the code as you like.

Visit our Django Development page Here

We welcome your feedback and support, raise github ticket if you want to report a bug. Need new features? Contact us here

or

mailto:: "[email protected]"

Issues
  • Bump django from 1.11.21 to 1.11.23 in /sandbox

    Bump django from 1.11.21 to 1.11.23 in /sandbox

    Bumps django from 1.11.21 to 1.11.23.

    Commits
    • 9748977 [1.11.x] Bumped version for 1.11.23 release.
    • 869b34e [1.11.x] Fixed CVE-2019-14235 -- Fixed potential memory exhaustion in django....
    • ed682a2 [1.11.x] Fixed CVE-2019-14234 -- Protected JSONField/HStoreField key and inde...
    • 52479ac [1.11.x] Fixed CVE-2019-14233 -- Prevented excessive HTMLParser recursion in ...
    • 42a66e9 [1.11.X] Fixed CVE-2019-14232 -- Adjusted regex to avoid backtracking issues ...
    • 693046e [1.11.x] Added stub release notes for security releases.
    • 6d054b5 [1.11.x] Added CVE-2019-12781 to the security release archive.
    • 7c849b9 [1.11.x] Post-release version bump.
    • 480380c [1.11.x] Bumped version for 1.11.22 release.
    • 32124fc [1.11.x] Fixed CVE-2019-12781 -- Made HttpRequest always trust SECURE_PROXY_S...
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot ignore this [patch|minor|major] version will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump django from 1.11.21 to 1.11.29 in /sandbox

    Bump django from 1.11.21 to 1.11.29 in /sandbox

    Bumps django from 1.11.21 to 1.11.29.

    Commits
    • f1e3017 [1.11.x] Bumped version for 1.11.29 release.
    • 02d97f3 [1.11.x] Fixed CVE-2020-9402 -- Properly escaped tolerance parameter in GIS f...
    • e643833 [1.11.x] Pinned PyYAML < 5.3 in test requirements.
    • d0e3eb8 [1.11.x] Added CVE-2020-7471 to security archive.
    • 9a62ed5 [1.11.x] Post-release version bump.
    • e09f09b [1.11.x] Bumped version for 1.11.28 release.
    • 001b063 [1.11.x] Fixed CVE-2020-7471 -- Properly escaped StringAgg(delimiter) parameter.
    • 7fd1ca3 [1.11.x] Fixed timezones tests for PyYAML 5.3+.
    • 121115d [1.11.x] Added CVE-2019-19844 to the security archive.
    • 2c4fb9a [1.11.x] Post-release version bump.
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Sample app

    Sample app

    opened by nikhila05 0
  • added badges

    added badges

    opened by nikhila05 0
  • django version support

    django version support

    • [x] 1.8
    • [x] 1.11
    • [x] 2.0
    opened by ashwin31 0
  • Typo? django-web-packer => django-web-profiler

    Typo? django-web-packer => django-web-profiler

    Believe this should reference "django-web-profiler", not "django-web-packer".

    opened by itsderek23 0
  • updated to django 2.0

    updated to django 2.0

    opened by nikhila05 0
  • Bump django from 1.11.21 to 2.2.24 in /sandbox

    Bump django from 1.11.21 to 2.2.24 in /sandbox

    Bumps django from 1.11.21 to 2.2.24.

    Commits
    • 2da029d [2.2.x] Bumped version for 2.2.24 release.
    • f27c38a [2.2.x] Fixed CVE-2021-33571 -- Prevented leading zeros in IPv4 addresses.
    • 053cc95 [2.2.x] Fixed CVE-2021-33203 -- Fixed potential path-traversal via admindocs'...
    • 6229d87 [2.2.x] Confirmed release date for Django 2.2.24.
    • f163ad5 [2.2.x] Added stub release notes and date for Django 2.2.24.
    • bed1755 [2.2.x] Changed IRC references to Libera.Chat.
    • 63f0d7a [2.2.x] Refs #32718 -- Fixed file_storage.test_generate_filename and model_fi...
    • 5fe4970 [2.2.x] Post-release version bump.
    • 61f814f [2.2.x] Bumped version for 2.2.23 release.
    • b8ecb06 [2.2.x] Fixed #32718 -- Relaxed file name validation in FileField.
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump django-debug-toolbar from 1.8 to 1.11.1 in /sandbox

    Bump django-debug-toolbar from 1.8 to 1.11.1 in /sandbox

    Bumps django-debug-toolbar from 1.8 to 1.11.1.

    Changelog

    Sourced from django-debug-toolbar's changelog.

    1.11.1 (2021-04-14)

    • Fixed SQL Injection vulnerability, CVE-2021-30459. The toolbar now calculates a signature on all fields for the SQL select, explain, and analyze forms.

    1.11 (2018-12-03)

    • Use defer on all <script> tags to avoid blocking HTML parsing, removed inline JavaScript.
    • Stop inlining images in CSS to avoid Content Security Policy errors altogether.
    • Reformatted the code using black <https://github.com/ambv/black>__.
    • Added the Django mail panel to the list of third-party panels.
    • Convert system check errors to warnings to accomodate exotic configurations.
    • Fixed a crash when explaining raw querysets.
    • Fixed an obscure unicode error with binary data fields.
    • Added MariaDB and Python 3.7 builds to the CI.

    1.10.1 (2018-09-11)

    • Fixed a problem where the duplicate query detection breaks for non-hashable query parameters.
    • Added support for structured types when recording SQL.
    • Made Travis CI also run one test no PostgreSQL.
    • Added fallbacks for inline images in CSS.
    • Improved cross-browser compatibility around URLSearchParams usage.
    • Fixed a few typos and redundancies in the documentation, removed mentions of django-debug-toolbar's jQuery which aren't accurate anymore.

    1.10 (2018-09-06)

    • Removed support for Django < 1.11.
    • Added support and testing for Django 2.1 and Python 3.7. No actual code changes were required.
    • Removed the jQuery dependency. This means that django-debug-toolbar now requires modern browsers with support for fetch, classList etc.
    • Added support for the server timing header.
    • Added a differentiation between similar and duplicate queries. Similar queries are what duplicate queries used to be (same SQL, different parameters).
    • Stopped hiding frames from Django's contrib apps in stacktraces by default.

    ... (truncated)

    Commits
    • bc08f69 Merge pull request from GHSA-pghf-347x-c2gj
    • c201ce3 django-debug-toolbar 1.11
    • 0a75be1 Update the change log
    • a4a5393 Merge pull request #1121 from matthiask/mariadb
    • 48a0e2e Reformat settings using black
    • 901aed7 Mark binary payload as binary (same thing BinaryField.get_db_prep_value does)
    • ad091e6 Test with a real BinaryField
    • 2f3193e Remove the MySQL USER
    • 762e5d9 Run tests with MariaDB too on Travis CI
    • e78ac8c Merge pull request #1107 from dbowd/patch-1
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Releases(0.1.4)
Owner
MicroPyramid
Python, django, AWS, devops, reactjs, Salesforce consulting company.
MicroPyramid
Python's missing debug print command and other development tools.

python devtools Python's missing debug print command and other development tools. For more information, see documentation. Install Just pip install de

Samuel Colvin 453 Jan 16, 2022
Graphical Python debugger which lets you easily view the values of all evaluated expressions

birdseye birdseye is a Python debugger which records the values of expressions in a function call and lets you easily view them after the function exi

Alex Hall 1.4k Jan 21, 2022
Middleware that Prints the number of DB queries to the runserver console.

Django Querycount Inspired by this post by David Szotten, this project gives you a middleware that prints DB query counts in Django's runserver consol

Brad Montgomery 308 Dec 21, 2021
Auto-detecting the n+1 queries problem in Python

nplusone nplusone is a library for detecting the n+1 queries problem in Python ORMs, including SQLAlchemy, Peewee, and the Django ORM. The Problem Man

Joshua Carp 762 Jan 19, 2022
A configurable set of panels that display various debug information about the current request/response.

Django Debug Toolbar The Django Debug Toolbar is a configurable set of panels that display various debug information about the current request/respons

Jazzband 6.8k Jan 14, 2022
A configurable set of panels that display various debug information about the current request/response.

Django Debug Toolbar The Django Debug Toolbar is a configurable set of panels that display various debug information about the current request/respons

Jazzband 6.8k Jan 17, 2022
A package containing a lot of useful utilities for Python developing and debugging.

Vpack A package containing a lot of useful utilities for Python developing and debugging. Features Sigview: press Ctrl+C to print the current stack in

volltin 14 Dec 3, 2021
A drop-in replacement for Django's runserver.

About A drop in replacement for Django's built-in runserver command. Features include: An extendable interface for handling things such as real-time l

David Cramer 1.3k Jan 9, 2022
A drop-in replacement for Django's runserver.

About A drop in replacement for Django's built-in runserver command. Features include: An extendable interface for handling things such as real-time l

David Cramer 1.3k Jan 9, 2022
Silky smooth profiling for Django

Silk Silk is a live profiling and inspection tool for the Django framework. Silk intercepts and stores HTTP requests and database queries before prese

Jazzband 3.2k Jan 13, 2022
Run-time type checker for Python

This library provides run-time type checking for functions defined with PEP 484 argument (and return) type annotations. Four principal ways to do type

Alex Grönholm 780 Jan 14, 2022
Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

starlette context Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automat

Tomasz Wójcik 214 Jan 10, 2022
Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

starlette context Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automat

Tomasz Wójcik 216 Jan 13, 2022
Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

starlette context Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automat

Tomasz Wójcik 110 Feb 16, 2021
PyPika is a python SQL query builder that exposes the full richness of the SQL language using a syntax that reflects the resulting query. PyPika excels at all sorts of SQL queries but is especially useful for data analysis.

PyPika - Python Query Builder Abstract What is PyPika? PyPika is a Python API for building SQL queries. The motivation behind PyPika is to provide a s

KAYAK 1.5k Jan 16, 2022
Django GUID attaches a unique correlation ID/request ID to all your log outputs for every request.

Django GUID Now with ASGI support! Django GUID attaches a unique correlation ID/request ID to all your log outputs for every request. In other words,

snok 212 Jan 12, 2022
missing-pixel-filler is a python package that, given images that may contain missing data regions (like satellite imagery with swath gaps), returns these images with the regions filled.

Missing Pixel Filler This is the official code repository for the Missing Pixel Filler by SpaceML. missing-pixel-filler is a python package that, give

SpaceML 7 Sep 21, 2021
Django URL Shortener is a Django app to to include URL Shortening feature in your Django Project

Django URL Shortener Django URL Shortener is a Django app to to include URL Shortening feature in your Django Project Install this package to your Dja

Rishav Sinha 4 Nov 18, 2021
tfquery: Run SQL queries on your Terraform infrastructure. Query resources and analyze its configuration using a SQL-powered framework.

??️ tfquery ??️ Run SQL queries on your Terraform infrastructure. Ask questions that are hard to answer ?? What is tfquery? tfquery is a framework tha

Mazin Ahmed 261 Dec 30, 2021
Given a metadata file with relevant schema, an SQL Engine can be run for a subset of SQL queries.

Mini-SQL-Engine Given a metadata file with relevant schema, an SQL Engine can be run for a subset of SQL queries. The query engine supports Project, A

Prashant Raj 1 Dec 3, 2021
Fully Automated YouTube Channel ▶️with Added Extra Features.

Fully Automated Youtube Channel ▒█▀▀█ █▀▀█ ▀▀█▀▀ ▀▀█▀▀ █░░█ █▀▀▄ █▀▀ █▀▀█ ▒█▀▀▄ █░░█ ░░█░░ ░▒█░░ █░░█ █▀▀▄ █▀▀ █▄▄▀ ▒█▄▄█ ▀▀▀▀ ░░▀░░ ░▒█░░ ░▀▀▀ ▀▀▀░

sam-sepiol 1 Jan 13, 2022
Py address book gui - An address book with graphical user interface developed with Python Tkinter

py_address_book_gui An address book with graphical user interface developed with

Milton 3 Jan 8, 2022
Inject an ID into every log message from a Django request. ASGI compatible, integrates with Sentry, and works with Celery

Django GUID Now with ASGI support! Django GUID attaches a unique correlation ID/request ID to all your log outputs for every request. In other words,

snok 211 Jan 10, 2022
Locationinfo - A script helps the user to show network information such as ip address

Description This script helps the user to show network information such as ip ad

Roxcoder 1 Dec 30, 2021
Django app for building dashboards using raw SQL queries

django-sql-dashboard Django app for building dashboards using raw SQL queries Brings a useful subset of Datasette to Django. Currently only works with

Simon Willison 317 Jan 15, 2022
Luis M. Capdevielle 1 Jan 14, 2022
Log processor for nginx or apache that extracts user and user sessions and calculates other types of useful data for bot detection or traffic analysis

Log processor for nginx or apache that extracts user and user sessions and calculates other types of useful data for bot detection or traffic analysis

David Puerta Martín 1 Nov 11, 2021
Soda SQL Data testing, monitoring and profiling for SQL accessible data.

Soda SQL Data testing, monitoring and profiling for SQL accessible data. What does Soda SQL do? Soda SQL allows you to Stop your pipeline when bad dat

Soda Data Monitoring 675 Jan 13, 2022
Easily share data across your company via SQL queries. From Grove Collab.

SQL Explorer SQL Explorer aims to make the flow of data between people fast, simple, and confusion-free. It is a Django-based application that you can

Grove Collaborative 2k Jan 18, 2022