Django package to log request values such as device, IP address, user CPU time, system CPU time, No of queries, SQL time, no of cache calls, missing, setting data cache calls for a particular URL with a basic UI.

MicroPyramid

Last update: Oct 29, 2022

Related tags

Overview

django-web-profiler's documentation:

https://travis-ci.org/MicroPyramid/django-web-profiler.svg?branch=master

https://coveralls.io/repos/github/MicroPyramid/django-web-profiler/badge.svg?branch=master

Introduction:

django-web-profiler is a django profiling tool which logs, stores debug toolbar statistics and also a set of URL's statistics using a management command. It logs request values such as device, ip address, user cpu time, system cpu time, No of queries, sql time, no of cache calls, missing, setting data cache calls for a particular url.

It provides a basic UI, which will differentiate development url statistics, production level statistics which generates using a management command.

Source Code is available in Micropyramid Repository(https://github.com/MicroPyramid/django-web-profiler).

Modules used:

Python >= 2.6 (or Python 3.4)

Django = 1.11.15

Django Compressor = 2.1.1

Django Debug Toolbar = 1.8

requests = 2.20.0

Installation Procedure

Install django-web-profiler using the following command:

pip install django-web-profiler

        (or)

git clone git://github.com/micropyramid/django-web-profiler.git

cd django-web-profiler

python setup.py install

Add app name in settings.py:

INSTALLED_APPS = [
   '..................',
   'compressor',
   'debug_toolbar',
   'django_web_profiler',
   '..................'
]

Add 'django_web_profiler.middleware.DebugLoggingMiddleware' to your project middlewares:

MIDDLEWARE = [
   '.....................................................',
   'django_web_profiler.middleware.DebugLoggingMiddleware',
   '.....................................................',
]

Disable 'debug_toolbar.middleware.DebugToolbarMiddleware' if you've already using it.

Make sure that 'debug-toolbar' has enabled for your application. After installing debug toolbar, add the following details to settings.py:
```
INTERNAL_IPS = ('127.0.0.1',)
```
After installing/cloning, add the following details in settings file about urls, logger names:
```
URLS = ['http://stage.testsite.com/', 'http://stage.testsite.com/testing/']
```

Add the following logger to your existing loggers and create a folder called 'logs' where all profiler log files are stored:

    'request-logging': {
        'level': 'DEBUG',
        'handlers': ['console', 'file_log'],
        'propagate': False,
    },

Here file_log is a handler which contains a path where log files are stored.

Sample Application

Install application requirements using the following command:
```
pip install -r requirements.txt
```

Load the application load using the following command:

python sandbox/manage.py loaddata sandbox/fixtures/users.json

Using the following command, we can generate url statistics in production environment i.e debug=False:
```
python sandbox/manage.py logging_urls
```

We are always looking to help you customize the whole or part of the code as you like.

Visit our Django Development page Here

We welcome your feedback and support, raise github ticket if you want to report a bug. Need new features? Contact us here

or

mailto:: "[email protected]"

You might also like...

Silky smooth profiling for Django

Silk Silk is a live profiling and inspection tool for the Django framework. Silk intercepts and stores HTTP requests and database queries before prese

3.7k Jan 1, 2023

Run-time type checker for Python

This library provides run-time type checking for functions defined with PEP 484 argument (and return) type annotations. Four principal ways to do type

1.1k Jan 5, 2023

Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

starlette context Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automat

300 Dec 26, 2022

Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

starlette context Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automat

300 Dec 26, 2022

Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

starlette context Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automat

110 Feb 16, 2021

PyPika is a python SQL query builder that exposes the full richness of the SQL language using a syntax that reflects the resulting query. PyPika excels at all sorts of SQL queries but is especially useful for data analysis.

PyPika - Python Query Builder Abstract What is PyPika? PyPika is a Python API for building SQL queries. The motivation behind PyPika is to provide a s

1.9k Jan 4, 2023

Django GUID attaches a unique correlation ID/request ID to all your log outputs for every request.

Django GUID Now with ASGI support! Django GUID attaches a unique correlation ID/request ID to all your log outputs for every request. In other words,

300 Dec 29, 2022

missing-pixel-filler is a python package that, given images that may contain missing data regions (like satellite imagery with swath gaps), returns these images with the regions filled.

Missing Pixel Filler This is the official code repository for the Missing Pixel Filler by SpaceML. missing-pixel-filler is a python package that, give

11 Jul 19, 2022

Fully Automated YouTube Channel ▶️with Added Extra Features.

Fully Automated Youtube Channel ▒█▀▀█ █▀▀█ ▀▀█▀▀ ▀▀█▀▀ █░░█ █▀▀▄ █▀▀ █▀▀█ ▒█▀▀▄ █░░█ ░░█░░ ░▒█░░ █░░█ █▀▀▄ █▀▀ █▄▄▀ ▒█▄▄█ ▀▀▀▀ ░░▀░░ ░▒█░░ ░▀▀▀ ▀▀▀░

249 Jan 2, 2023

tfquery: Run SQL queries on your Terraform infrastructure. Query resources and analyze its configuration using a SQL-powered framework.

🌩️ tfquery 🌩️ Run SQL queries on your Terraform infrastructure. Ask questions that are hard to answer 🚀 What is tfquery? tfquery is a framework tha

311 Dec 21, 2022

Given a metadata file with relevant schema, an SQL Engine can be run for a subset of SQL queries.

Mini-SQL-Engine Given a metadata file with relevant schema, an SQL Engine can be run for a subset of SQL queries. The query engine supports Project, A

1 Dec 3, 2021

Django URL Shortener is a Django app to to include URL Shortening feature in your Django Project

Django URL Shortener Django URL Shortener is a Django app to to include URL Shortening feature in your Django Project Install this package to your Dja

4 Nov 18, 2021

Py address book gui - An address book with graphical user interface developed with Python Tkinter

py_address_book_gui An address book with graphical user interface developed with

4 Feb 1, 2022

EBay-email-tracker - Scapes an entire search page of a particular item on eBay and sends regular updates to an email address

Introduction This is a project I built with the sole intent to learn more about

1 Jan 14, 2022

Locationinfo - A script helps the user to show network information such as ip address

Description This script helps the user to show network information such as ip ad

1 Dec 30, 2021

An extension package of 🤗 Datasets that provides support for executing arbitrary SQL queries on HF datasets

datasets_sql A 🤗 Datasets extension package that provides support for executing arbitrary SQL queries on HF datasets. It uses DuckDB as a SQL engine

19 Dec 15, 2022

Inject an ID into every log message from a Django request. ASGI compatible, integrates with Sentry, and works with Celery

Django GUID Now with ASGI support! Django GUID attaches a unique correlation ID/request ID to all your log outputs for every request. In other words,

300 Dec 29, 2022

Django app for building dashboards using raw SQL queries

django-sql-dashboard Django app for building dashboards using raw SQL queries Brings a useful subset of Datasette to Django. Currently only works with

383 Jan 6, 2023

Soda SQL Data testing, monitoring and profiling for SQL accessible data.

Soda SQL Data testing, monitoring and profiling for SQL accessible data. What does Soda SQL do? Soda SQL allows you to Stop your pipeline when bad dat

51 Jan 1, 2023

Comments

Bump django from 1.11.21 to 1.11.29 in /sandbox
Bumps django from 1.11.21 to 1.11.29.

Commits

f1e3017 [1.11.x] Bumped version for 1.11.29 release.

02d97f3 [1.11.x] Fixed CVE-2020-9402 -- Properly escaped tolerance parameter in GIS f...

e643833 [1.11.x] Pinned PyYAML < 5.3 in test requirements.

d0e3eb8 [1.11.x] Added CVE-2020-7471 to security archive.

9a62ed5 [1.11.x] Post-release version bump.

e09f09b [1.11.x] Bumped version for 1.11.28 release.

001b063 [1.11.x] Fixed CVE-2020-7471 -- Properly escaped StringAgg(delimiter) parameter.

7fd1ca3 [1.11.x] Fixed timezones tests for PyYAML 5.3+.

121115d [1.11.x] Added CVE-2019-19844 to the security archive.

2c4fb9a [1.11.x] Post-release version bump.

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump django from 1.11.21 to 1.11.23 in /sandbox
Bumps django from 1.11.21 to 1.11.23.

Commits

9748977 [1.11.x] Bumped version for 1.11.23 release.

869b34e [1.11.x] Fixed CVE-2019-14235 -- Fixed potential memory exhaustion in django....

ed682a2 [1.11.x] Fixed CVE-2019-14234 -- Protected JSONField/HStoreField key and inde...

52479ac [1.11.x] Fixed CVE-2019-14233 -- Prevented excessive HTMLParser recursion in ...

42a66e9 [1.11.X] Fixed CVE-2019-14232 -- Adjusted regex to avoid backtracking issues ...

693046e [1.11.x] Added stub release notes for security releases.

6d054b5 [1.11.x] Added CVE-2019-12781 to the security release archive.

7c849b9 [1.11.x] Post-release version bump.

480380c [1.11.x] Bumped version for 1.11.22 release.

32124fc [1.11.x] Fixed CVE-2019-12781 -- Made HttpRequest always trust SECURE_PROXY_S...

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot ignore this [patch|minor|major] version will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump django from 1.11.21 to 2.2.24 in /sandbox
Bumps django from 1.11.21 to 2.2.24.

Commits

2da029d [2.2.x] Bumped version for 2.2.24 release.

f27c38a [2.2.x] Fixed CVE-2021-33571 -- Prevented leading zeros in IPv4 addresses.

053cc95 [2.2.x] Fixed CVE-2021-33203 -- Fixed potential path-traversal via admindocs'...

6229d87 [2.2.x] Confirmed release date for Django 2.2.24.

f163ad5 [2.2.x] Added stub release notes and date for Django 2.2.24.

bed1755 [2.2.x] Changed IRC references to Libera.Chat.

63f0d7a [2.2.x] Refs #32718 -- Fixed file_storage.test_generate_filename and model_fi...

5fe4970 [2.2.x] Post-release version bump.

61f814f [2.2.x] Bumped version for 2.2.23 release.

b8ecb06 [2.2.x] Fixed #32718 -- Relaxed file name validation in FileField.

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
Bump django-debug-toolbar from 1.8 to 1.11.1 in /sandbox
Bumps django-debug-toolbar from 1.8 to 1.11.1.

Changelog

Sourced from django-debug-toolbar's changelog.

1.11.1 (2021-04-14)

Fixed SQL Injection vulnerability, CVE-2021-30459. The toolbar now calculates a signature on all fields for the SQL select, explain, and analyze forms.

1.11 (2018-12-03)

Use defer on all <script> tags to avoid blocking HTML parsing, removed inline JavaScript.

Stop inlining images in CSS to avoid Content Security Policy errors altogether.

Reformatted the code using black <https://github.com/ambv/black>__.

Added the Django mail panel to the list of third-party panels.

Convert system check errors to warnings to accomodate exotic configurations.

Fixed a crash when explaining raw querysets.

Fixed an obscure unicode error with binary data fields.

Added MariaDB and Python 3.7 builds to the CI.

1.10.1 (2018-09-11)

Fixed a problem where the duplicate query detection breaks for non-hashable query parameters.

Added support for structured types when recording SQL.

Made Travis CI also run one test no PostgreSQL.

Added fallbacks for inline images in CSS.

Improved cross-browser compatibility around URLSearchParams usage.

Fixed a few typos and redundancies in the documentation, removed mentions of django-debug-toolbar's jQuery which aren't accurate anymore.

1.10 (2018-09-06)

Removed support for Django < 1.11.

Added support and testing for Django 2.1 and Python 3.7. No actual code changes were required.

Removed the jQuery dependency. This means that django-debug-toolbar now requires modern browsers with support for fetch, classList etc.

Added support for the server timing header.

Added a differentiation between similar and duplicate queries. Similar queries are what duplicate queries used to be (same SQL, different parameters).

Stopped hiding frames from Django's contrib apps in stacktraces by default.

... (truncated)

Commits

bc08f69 Merge pull request from GHSA-pghf-347x-c2gj

c201ce3 django-debug-toolbar 1.11

0a75be1 Update the change log

a4a5393 Merge pull request #1121 from matthiask/mariadb

48a0e2e Reformat settings using black

901aed7 Mark binary payload as binary (same thing BinaryField.get_db_prep_value does)

ad091e6 Test with a real BinaryField

2f3193e Remove the MySQL USER

762e5d9 Run tests with MariaDB too on Travis CI

e78ac8c Merge pull request #1107 from dbowd/patch-1

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0

Releases(0.1.4)

0.1.4(Mar 24, 2018)

Updated the package to django 2.0
Source code(tar.gz)
Source code(zip)
0.1.3(Jun 9, 2017)

A simple profiling tool to analyze Django requests.
Source code(tar.gz)
Source code(zip)

Owner

MicroPyramid

Python, django, AWS, devops, reactjs, Salesforce consulting company.

GitHub

Django package to log request values such as device, IP address, user CPU time, system CPU time, No of queries, SQL time, no of cache calls, missing, setting data cache calls for a particular URL with a basic UI.

Related tags

Overview

django-web-profiler's documentation:

Introduction:

Installation Procedure

Sample Application

You might also like...

Silky smooth profiling for Django

Run-time type checker for Python

Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

PyPika is a python SQL query builder that exposes the full richness of the SQL language using a syntax that reflects the resulting query. PyPika excels at all sorts of SQL queries but is especially useful for data analysis.

Django GUID attaches a unique correlation ID/request ID to all your log outputs for every request.

missing-pixel-filler is a python package that, given images that may contain missing data regions (like satellite imagery with swath gaps), returns these images with the regions filled.

Fully Automated YouTube Channel ▶️with Added Extra Features.

tfquery: Run SQL queries on your Terraform infrastructure. Query resources and analyze its configuration using a SQL-powered framework.

Given a metadata file with relevant schema, an SQL Engine can be run for a subset of SQL queries.

Django URL Shortener is a Django app to to include URL Shortening feature in your Django Project

Py address book gui - An address book with graphical user interface developed with Python Tkinter

EBay-email-tracker - Scapes an entire search page of a particular item on eBay and sends regular updates to an email address

Locationinfo - A script helps the user to show network information such as ip address

An extension package of 🤗 Datasets that provides support for executing arbitrary SQL queries on HF datasets

Inject an ID into every log message from a Django request. ASGI compatible, integrates with Sentry, and works with Celery

Django app for building dashboards using raw SQL queries

Soda SQL Data testing, monitoring and profiling for SQL accessible data.

Comments

Bump django from 1.11.21 to 1.11.29 in /sandbox

Bump django from 1.11.21 to 1.11.23 in /sandbox

Bump django from 1.11.21 to 2.2.24 in /sandbox

Bump django-debug-toolbar from 1.8 to 1.11.1 in /sandbox

1.11.1 (2021-04-14)

1.11 (2018-12-03)

1.10.1 (2018-09-11)

1.10 (2018-09-06)

Releases(0.1.4)

0.1.4(Mar 24, 2018)

0.1.3(Jun 9, 2017)

Owner

MicroPyramid

Python's missing debug print command and other development tools.

Graphical Python debugger which lets you easily view the values of all evaluated expressions

Middleware that Prints the number of DB queries to the runserver console.

Auto-detecting the n+1 queries problem in Python

A configurable set of panels that display various debug information about the current request/response.

A configurable set of panels that display various debug information about the current request/response.

Trace all method entries and exits, the exit also prints the return value, if it is of basic type

A package containing a lot of useful utilities for Python developing and debugging.

A drop-in replacement for Django's runserver.

A drop-in replacement for Django's runserver.