A simple, immutable URL class with a clean API for interrogation and manipulation.

Related tags

purl
Overview

purl - A simple Python URL class

A simple, immutable URL class with a clean API for interrogation and manipulation. Supports Pythons 2.7, 3.3, 3.4, 3.5, 3.6 and pypy.

Also supports template URLs as per RFC 6570

Contents:

https://secure.travis-ci.org/codeinthehole/purl.png

Docs

http://purl.readthedocs.org/en/latest/

Install

From PyPI (stable):

$ pip install purl

From Github (unstable):

$ pip install git+git://github.com/codeinthehole/purl.git#egg=purl

Use

Construct:

>>> from purl import URL

# String constructor
>>> from_str = URL('https://www.google.com/search?q=testing')

# Keyword constructor
>>> from_kwargs = URL(scheme='https', host='www.google.com', path='/search', query='q=testing')

# Combine
>>> from_combo = URL('https://www.google.com').path('search').query_param('q', 'testing')

URL objects are immutable - all mutator methods return a new instance.

Interrogate:

>>> u = URL('https://www.google.com/search?q=testing')
>>> u.scheme()
'https'
>>> u.host()
'www.google.com'
>>> u.domain()
'www.google.com'
>>> u.username()
>>> u.password()
>>> u.netloc()
'www.google.com'
>>> u.port()
>>> u.path()
'/search'
>>> u.query()
'q=testing'
>>> u.fragment()
''
>>> u.path_segment(0)
'search'
>>> u.path_segments()
('search',)
>>> u.query_param('q')
'testing'
>>> u.query_param('q', as_list=True)
['testing']
>>> u.query_param('lang', default='GB')
'GB'
>>> u.query_params()
{'q': ['testing']}
>>> u.has_query_param('q')
True
>>> u.has_query_params(('q', 'r'))
False
>>> u.subdomains()
['www', 'google', 'com']
>>> u.subdomain(0)
'www'

Note that each accessor method is overloaded to be a mutator method too, similar to the jQuery API. Eg:

>>> u = URL.from_string('https://github.com/codeinthehole')

# Access
>>> u.path_segment(0)
'codeinthehole'

# Mutate (creates a new instance)
>>> new_url = u.path_segment(0, 'tangentlabs')
>>> new_url is u
False
>>> new_url.path_segment(0)
'tangentlabs'

Hence, you can build a URL up in steps:

>>> u = URL().scheme('http').domain('www.example.com').path('/some/path').query_param('q', 'search term')
>>> u.as_string()
'http://www.example.com/some/path?q=search+term'

Along with the above overloaded methods, there is also a add_path_segment method for adding a segment at the end of the current path:

>>> new_url = u.add_path_segment('here')
>>> new_url.as_string()
'http://www.example.com/some/path/here?q=search+term'

Couple of other things:

  • Since the URL class is immutable it can be used as a key in a dictionary
  • It can be pickled and restored
  • It supports equality operations
  • It supports equality operations

URL templates can be used either via a Template class:

>>> from purl import Template
>>> tpl = Template("http://example.com{/list*}")
>>> url = tpl.expand({'list': ['red', 'green', 'blue']})
>>> url.as_string()
'http://example.com/red/green/blue'

or the expand function:

>>> from purl import expand
>>> expand(u"{/list*}", {'list': ['red', 'green', 'blue']})
'/red/green/blue'

A wide variety of expansions are possible - refer to the RFC for more details.

Changelog

v1.5 - 2019-03-10

  • Allow @ in passwords.

v1.4 - 2018-03-11

  • Allow usernames and passwords to be removed from URLs.

v1.3.1

  • Ensure paths always have a leading slash.

v1.3

  • Allow absolute URLs to be converted into relative.

v1.2

  • Support password-less URLs.
  • Allow slashes to be passed as path segments.

v1.1

  • Support setting username and password via mutator methods

v1.0.3

  • Handle some unicode compatibility edge-cases

v1.0.2

  • Fix template expansion bug with no matching variables being passed in. This ensures purl.Template works correctly with the URLs returned from the Github API.

v1.0.1

  • Fix bug with special characters in paths not being escaped.

v1.0

  • Slight tidy up. Document support for PyPy and Python 3.4.

v0.8

  • Support for RFC 6570 URI templates

v0.7

  • All internal strings are unicode.
  • Support for unicode chars in path, fragment, query, auth added.

v0.6

  • Added append_query_param method
  • Added remove_query_param method

v0.5

  • Added support for Python 3.2/3.3 (thanks @pmcnr and @mitchellrj)

v0.4.1

  • Added API docs
  • Added to readthedocs.org

v0.4

  • Modified constructor to accept full URL string as first arg
  • Added add_path_segment method

v0.3.2

  • Fixed bug port number in string when using from_string constructor

v0.3.1

  • Fixed bug with passing lists to query param setter methods

v0.3

  • Added support for comparison and equality
  • Added support for pickling
  • Added __slots__ so instances can be used as keys within dictionaries

Contribute

Clone, create a virtualenv then install purl and the packages required for testing:

$ git clone [email protected]:codeinthehole/purl.git
$ cd purl
$ mkvirtualenv purl  # requires virtualenvwrapper
(purl) $ make

Ensure tests pass using:

(purl) $ ./runtests.sh

or:

$ tox
Issues
  • Passwordless URL support

    Passwordless URL support

    I've added tests and my solution of #19

    opened by mieciu 4
  • Added support for username/password setting

    Added support for username/password setting

    Those lines of code could actually help with elegant build of full URL, like this:

    ELASTICSEARCH_URL = URL.from_string("FOUNDELASTICSEARCH_URL"))\
                               .scheme('https')\
                               .username(myuser)\
                               .password(os.environ['password'])\
                               .port('9243')
    
    opened by mieciu 4
  • Cannot be installed using buildout

    Cannot be installed using buildout

    When I try to install purl 1.2 using buildout I get the following error:

    Traceback (most recent call last):
      File "/Users/denis/.buildout/eggs/setuptools-19.4-py3.5.egg/setuptools/sandbox.py", line 154, in save_modules
      File "/Users/denis/.buildout/eggs/setuptools-19.4-py3.5.egg/setuptools/sandbox.py", line 195, in setup_context
      File "/Users/denis/.buildout/eggs/setuptools-19.4-py3.5.egg/setuptools/sandbox.py", line 239, in run_setup
      File "/Users/denis/.buildout/eggs/setuptools-19.4-py3.5.egg/setuptools/sandbox.py", line 269, in run
      File "/Users/denis/.buildout/eggs/setuptools-19.4-py3.5.egg/setuptools/sandbox.py", line 238, in runner
      File "/Users/denis/.buildout/eggs/setuptools-19.4-py3.5.egg/setuptools/sandbox.py", line 46, in _execfile
      File "/var/folders/mm/5rlp4h9906n3npgjt96lm0fw0000gn/T/easy_install-4zdguc7q/purl-1.2/setup.py", line 5, in <module>
      File "/var/folders/mm/5rlp4h9906n3npgjt96lm0fw0000gn/T/easy_install-4zdguc7q/purl-1.2/purl/__init__.py", line 1, in <module>
      File "/var/folders/mm/5rlp4h9906n3npgjt96lm0fw0000gn/T/easy_install-4zdguc7q/purl-1.2/purl/url.py", line 10, in <module>
    ImportError: No module named 'six'
    

    The problem seems to be that you try to import the package in setup.py, when it's not yet installed. This works if six is installed, or indeed if pip is used (because pip has a workaround for this). What you should be doing is either define the version twice in the __init__.py as well as setup.py. Or you open __init__.py as text and parse the version statically.

    opened by href 3
  • fixed typo on unicode_unquote

    fixed typo on unicode_unquote

    Hi,

    I noticed there was a typo - fixed. On a different topic - what do you think about serializing the object ? Something like - to_dict.

    opened by silviud 3
  • Add RFC6570 (Uri template) support

    Add RFC6570 (Uri template) support

    Right now, there's a python package https://github.com/uri-templates/uritemplate-py that implements RFC6570 url template expanding, but it will be fantastic to get that code merged into purl .

    opened by inean 2
  • passwordless URL support in string representation

    passwordless URL support in string representation

    Hi,

    Today I've discovered quite weird behaviour - please have a look at snippet below:

    >>> from purl import URL
    >>> url = URL(scheme='postgres', username='user', host='127.0.0.1', port='5432', path='/db_name')
    >>> url.as_string()
    u'postgres://127.0.0.1:5432/db_name'        # Username disappeared 
    >>> url = URL(scheme='postgres', username='user', password='pass', host='127.0.0.1', port='5432', path='/db_name')
    >>> url.as_string()
    u'postgres://user:[email protected]:5432/db_name'        # In combination w/ passord it works
    

    Of course its being stored properly:

    >>> url = URL(scheme='postgres', username='user', host='127.0.0.1', port='5432', path='/db_name')
    >>> url
    _URLTuple(host='127.0.0.1', username='user', password=None, scheme='postgres', port='5432', path='/db_name', query=None, fragment=N
    one)
    >>> url.username
    <bound method URL.username of _URLTuple(host='127.0.0.1', username='user', password=None, scheme='postgres', port='5432', path='/db
    _name', query=None, fragment=None)>
    >>> url.username()
    u'user'
    

    I think that passwordless URLs should be supported, as - for instance - following postgres URL is fully proper: psql "postgres://[email protected]:5432/template1"

    It should be relatively easy to fix, I'll try to submit a patch this week, but what do you think about this?

    opened by mieciu 2
  • decode all urls to ascii

    decode all urls to ascii

    urlparse.parse_qs will return bad value, when input url is a unicode instance and contains encoded non-ascii data. Problem described in details here: http://www.lexev.org/en/2013/parse-url-which-chontains-unicode-query-using-urlp/

    For some reason, django's request.get_full_path() returns url as unicode instance. Faced that problem during developing a faceted search in django-oscar

    Tests are attached.

    In short:

    >>> import urlparse, urllib
    >>> value = urllib.quote(u'значение'.encode('utf8'))
    >>> query = u"key=%s" % value
    >>> query
    u'key=%D0%B7%D0%BD%D0%B0%D1%87%D0%B5%D0%BD%D0%B8%D0%B5'
    >>> query_dict = urlparse.parse_qs(query)
    >>> query_dict
    {u'key': [u'\xd0\xb7\xd0\xbd\xd0\xb0\xd1\x87\xd0\xb5\xd0\xbd\xd0\xb8\xd0\xb5']}
    >>> urllib.urlencode(query_dict, doseq=True)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "../urllib.py", line 1337, in urlencode
        l.append(k + '=' + quote_plus(str(elt)))
    UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-15: ordinal not in range(128)
    

    But, when construct query without u, it works fine:

    >>> query = "key=%s" % value
    >>> query_dict = urlparse.parse_qs(query)
    >>> query_dict
    {'key': ['\xd0\xb7\xd0\xbd\xd0\xb0\xd1\x87\xd0\xb5\xd0\xbd\xd0\xb8\xd0\xb5']}
    >>> urllib.urlencode(query_dict, doseq=True)
    'key=%D0%B7%D0%BD%D0%B0%D1%87%D0%B5%D0%BD%D0%B8%D0%B5'
    
    opened by st4lk 2
  • Enable URL concatenation

    Enable URL concatenation

    Example:

    >>> str(URL('/some_path/') + URL('../other_path/'))
    '/other_path/'
    
    opened by szuliq 2
  • Python 3.x compatibility refactoring.

    Python 3.x compatibility refactoring.

    This pull request implements some changes which make purl compatible with Python 2.6 through 3.2 (and I assume also 3.3).

    opened by paparomeo 2
  • Added a origin method

    Added a origin method

    Hello. This is KAZY. This is a very nice library.

    Well, I am developing a crawler. I thought it would be very useful to have the ability to extract the origin from the URL. I'd be happy to accept your commit if you agree with me.

    I welcome your comments and suggestions.

    Thank you.

    opened by KAZYPinkSaurus 0
  • Bump pip from 19.0.3 to 19.2

    Bump pip from 19.0.3 to 19.2

    Bumps pip from 19.0.3 to 19.2.

    Changelog

    Sourced from pip's changelog.

    19.2 (2019-07-22)

    Deprecations and Removals

    • Drop support for EOL Python 3.4. ([#6685](https://github.com/pypa/pip/issues/6685) <https://github.com/pypa/pip/issues/6685>_)
    • Improve deprecation messages to include the version in which the functionality will be removed. ([#6549](https://github.com/pypa/pip/issues/6549) <https://github.com/pypa/pip/issues/6549>_)

    Features

    • Credentials will now be loaded using keyring when installed. ([#5948](https://github.com/pypa/pip/issues/5948) <https://github.com/pypa/pip/issues/5948>_)
    • Fully support using --trusted-host inside requirements files. ([#3799](https://github.com/pypa/pip/issues/3799) <https://github.com/pypa/pip/issues/3799>_)
    • Update timestamps in pip's --log file to include milliseconds. ([#6587](https://github.com/pypa/pip/issues/6587) <https://github.com/pypa/pip/issues/6587>_)
    • Respect whether a file has been marked as "yanked" from a simple repository (see PEP 592 <https://www.python.org/dev/peps/pep-0592/>__ for details). ([#6633](https://github.com/pypa/pip/issues/6633) <https://github.com/pypa/pip/issues/6633>_)
    • When choosing candidates to install, prefer candidates with a hash matching one of the user-provided hashes. ([#5874](https://github.com/pypa/pip/issues/5874) <https://github.com/pypa/pip/issues/5874>_)
    • Improve the error message when METADATA or PKG-INFO is None when accessing metadata. ([#5082](https://github.com/pypa/pip/issues/5082) <https://github.com/pypa/pip/issues/5082>_)
    • Add a new command pip debug that can display e.g. the list of compatible tags for the current Python. ([#6638](https://github.com/pypa/pip/issues/6638) <https://github.com/pypa/pip/issues/6638>_)
    • Display hint on installing with --pre when search results include pre-release versions. ([#5169](https://github.com/pypa/pip/issues/5169) <https://github.com/pypa/pip/issues/5169>_)
    • Report to Warehouse that pip is running under CI if the PIP_IS_CI environment variable is set. ([#5499](https://github.com/pypa/pip/issues/5499) <https://github.com/pypa/pip/issues/5499>_)
    • Allow --python-version to be passed as a dotted version string (e.g. 3.7 or 3.7.3). ([#6585](https://github.com/pypa/pip/issues/6585) <https://github.com/pypa/pip/issues/6585>_)
    • Log the final filename and SHA256 of a .whl file when done building a wheel. ([#5908](https://github.com/pypa/pip/issues/5908) <https://github.com/pypa/pip/issues/5908>_)
    • Include the wheel's tags in the log message explanation when a candidate wheel link is found incompatible. ([#6121](https://github.com/pypa/pip/issues/6121) <https://github.com/pypa/pip/issues/6121>_)
    • Add a --path argument to pip freeze to support --target installations. ([#6404](https://github.com/pypa/pip/issues/6404) <https://github.com/pypa/pip/issues/6404>_)
    • Add a --path argument to pip list to support --target installations. ([#6551](https://github.com/pypa/pip/issues/6551) <https://github.com/pypa/pip/issues/6551>_)

    Bug Fixes

    • Set sys.argv[0] to the underlying setup.py when invoking setup.py via the setuptools shim so setuptools doesn't think the path is -c. ([#1890](https://github.com/pypa/pip/issues/1890) <https://github.com/pypa/pip/issues/1890>_)
    • Update pip download to respect the given --python-version when checking "Requires-Python". ([#5369](https://github.com/pypa/pip/issues/5369) <https://github.com/pypa/pip/issues/5369>_)
    • Respect --global-option and --install-option when installing from a version control url (e.g. git). ([#5518](https://github.com/pypa/pip/issues/5518) <https://github.com/pypa/pip/issues/5518>_)
    • Make the "ascii" progress bar really be "ascii" and not Unicode. ([#5671](https://github.com/pypa/pip/issues/5671) <https://github.com/pypa/pip/issues/5671>_)
    • Fail elegantly when trying to set an incorrectly formatted key in config. ([#5963](https://github.com/pypa/pip/issues/5963) <https://github.com/pypa/pip/issues/5963>_)
    • Prevent DistutilsOptionError when prefix is indicated in the global environment and --target is used. ([#6008](https://github.com/pypa/pip/issues/6008) <https://github.com/pypa/pip/issues/6008>_)
    • Fix pip install to respect --ignore-requires-python when evaluating links. ([#6371](https://github.com/pypa/pip/issues/6371) <https://github.com/pypa/pip/issues/6371>_)

    ... (truncated)

    Commits
    • 0e64295 Generate NEWS
    • 0df416d Bump version for release
    • 5f0aa2a Generate AUTHORS.txt
    • 8582f7e Reduce dependency on ctypes when discovering glibc version. (#6678)
    • e308497 Merge pull request #6743 from chrahunt/maint/remove-path-copytree
    • 9281a7a Remove copytree from tests.lib.path.Path.
    • 0d28601 Remove copy from tests.lib.path.Path. (#6746)
    • c275e9d Drop a useless import in favor of explicitness
    • 3732e79 Remove normpath from tests.lib.path.Path.
    • 358e690 Remove move from tests.lib.path.Path.
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Don't correct parse host

    Don't correct parse host

    Don't work with cyrillic domain

    from purl import URL
    
    url = "www.мастеркофе.рф"
    parse_url = URL(url)
    
    print(parse_url.host()) <- return ""
    

    Don't detect host

    opened by temi4 0
  • Bump version to v1.6

    Bump version to v1.6

    null

    opened by codeinthehole 0
  • Added 3 new methods to the URL class

    Added 3 new methods to the URL class

    First of all, I love this project! It saved me time on my projects, a couple of times already. However, while using it I thought to myself that there is someplace to improve it.

    Added 3 new methods:

    • add_path_segments: To add multiple path segments to the URL at once, instead of using the add_path_segment method multiple times.
    • append_query_params: The same concept - to add multiple query parameters at once.
    • remove_query_params: To remove multiple query parameters at once.

    I have also updated the README file to save you some work 😊 Please check my documentation, comments, and docstrings! I tried to follow the original format and syntax, but I'm pretty sure I failed at that 😅

    opened by RealA10N 0
  • Custom scheme is not respected in as_string

    Custom scheme is not respected in as_string

    ipython
    Python 3.7.3 (default, Apr 12 2019, 14:40:22)
    Type 'copyright', 'credits' or 'license' for more information
    IPython 7.3.0 -- An enhanced Interactive Python. Type '?' for help.
    
    In [1]: from purl import URL
    
    In [2]: url = URL("foobar:///my_path")
    
    In [3]: url.scheme()
    Out[3]: 'foobar'
    
    In [4]: url.as_string()
    Out[4]: '/my_path'
    

    I expect Out[4] to be "foobar:///my_path"

    opened by denizdogan 0
  • Update conda package

    Update conda package

    https://anaconda.org/hargup/purl (or something similar) needs some ❤️

    opened by dazza-codes 0
  • Add resolve() method (or function)

    Add resolve() method (or function)

    Here's the helper function I wrote:

    from purl import URL
    from urllib import parse
    
    def resolve(base, url):
        """
        Resolves a target URL relative to a base URL in a manner similar to that of a Web browser resolving an anchor tag HREF
        :param base: str|URL
        :param url: str|URL
        :return: URL
        """
        if isinstance(base, URL):
            baseurl = base
        else:
            baseurl = URL(base)
    
        if isinstance(url, URL):
            relurl = url
        else:
            relurl = URL(url)
    
        if relurl.host():
            return relurl
    
        if relurl.path():
            return URL(
                scheme=baseurl.scheme(),
                host=baseurl.host(),
                port=baseurl.port(),
                path=parse.urljoin(baseurl.path(), relurl.path()),
                query=relurl.query(),
                fragment=relurl.fragment(),
            )
        elif relurl.query() or '?' in url:
            return URL(
                scheme=baseurl.scheme(),
                host=baseurl.host(),
                port=baseurl.port(),
                path=baseurl.path(),
                query=relurl.query(),
                fragment=relurl.fragment(),
            )
        elif relurl.fragment() or '#' in url:
            return URL(
                scheme=baseurl.scheme(),
                host=baseurl.host(),
                port=baseurl.port(),
                path=baseurl.path(),
                query=baseurl.query(),
                fragment=relurl.fragment(),
            )
        return baseurl
    

    Usage:

    >>> base = URL('http://user:[email protected]:8080/path/to/some/doc.html?q=query#frag')
    ...
    >>> print(resolve(base, '../home'))
    http://user:[email protected]:8080/path/to/home
    
    >>> print(resolve(base, 'doc2.html'))
    http://user:[email protected]:8080/path/to/some/doc2.html
    
    >>> print(resolve(base, '?'))
    http://user:[email protected]:8080/path/to/some/doc.html
    
    >>> print(resolve(base, '?q=git'))
    http://user:[email protected]:8080/path/to/some/doc.html?q=git
    
    >>> print(resolve(base, '#'))
    http://user:[email protected]:8080/path/to/some/doc.html?q=query
    
    opened by pix666 3
  • add_path_segment: slashes in path segments getting quoted (readability)

    add_path_segment: slashes in path segments getting quoted (readability)

    Hi there,

    I think I just ran into the "reverse" issue of #22 – readability of URLs with (now quoted) / in path seqments for e.g. logging and other text outputs (or lack thereof).

    Is there a quick and easy fix for this – removing the quotes – which I might not be aware of?

    opened by keikoro 2
Owner
David Winterbottom
Head of Software Engineering at Octopus Energy
David Winterbottom
python3 flask based python-url-shortener microservice.

python-url-shortener This repository is for managing all public/private entity specific api endpoints for an organisation. In this case we have entity

Asutosh Parida 1 Oct 18, 2021
A URL builder for genius :D

genius-url A URL builder for genius :D Usage from gurl import genius_url

ꌗᖘ꒒ꀤ꓄꒒ꀤꈤꍟ 12 Aug 14, 2021
find all the URL of a site with a specific Regex

href this program will find all the link with a spesfic Regex pattern from a site. what it will do in any site there are a lots of url that may you ne

Arya Shabane 9 Oct 17, 2021
This is a no-bullshit file hosting and URL shortening service that also runs 0x0.st. Use with uWSGI.

This is a no-bullshit file hosting and URL shortening service that also runs 0x0.st. Use with uWSGI.

mia 1.5k Oct 25, 2021
🔗 Generate Phishing URLs 🔗

URLer ?? Generate Phishing URLs ?? URLer Table Of Contents General Information Preview Installation Disclaimer Credits Social Media Bug Report General

mrblackx 4 Sep 27, 2021
A python code for url redirect check

A python code for url redirect check

Fayas Noushad 1 Oct 24, 2021
URL Shortener in Flask - Web service using Flask framework for Shortener URLs

URL Shortener in Flask Web service using Flask framework for Shortener URLs Install Create Virtual env $ python3 -m venv env Install requirements.txt

Rafnix Guzman 1 Sep 21, 2021
A url redirect status check module for python

A url redirect status check module for python

Fayas Noushad 2 Oct 24, 2021
UDdup - URLs Deduplication Tool

UDdup - URLs Deduplication Tool The tool gets a list of URLs, and removes "duplicate" pages in the sense of URL patterns that are probably repetitive

Rotem Reiss 125 Sep 8, 2021
a url shortener project from semicolonworld

Url Shortener With Django Written by Semicolon World

null 3 Aug 24, 2021
A friendly library for parsing HTTP request arguments, with built-in support for popular web frameworks, including Flask, Django, Bottle, Tornado, Pyramid, webapp2, Falcon, and aiohttp.

webargs Homepage: https://webargs.readthedocs.io/ webargs is a Python library for parsing and validating HTTP request objects, with built-in support f

marshmallow-code 1.2k Oct 19, 2021
Have you ever wondered: Where does this link go? The REDLI Tool follows the path of the URL.

Have you ever wondered: Where does this link go? The REDLI Tool follows the path of the URL. It allows you to see the complete path a redirected URL goes through. It will show you the full redirection path of URLs, shortened links, or tiny URLs.

JAYAKUMAR 28 Oct 20, 2021
Astra is a tool to find URLs and secrets.

Astra finds urls, endpoints, aws buckets, api keys, tokens, etc from a given url/s. It combines the paths and endpoints with the given domain and give

Stinger 128 Sep 30, 2021
:electric_plug: Generating short urls with python has never been easier

pyshorteners A simple URL shortening API wrapper Python library. Installing pip install pyshorteners Documentation https://pyshorteners.readthedocs.i

Ellison 316 Oct 25, 2021
encurtador de links feito com python

curt-link encurtador de links feito com python! instalação Linux: $ git clone https://github.com/bydeathlxncer/curt-link $ cd curt-link $ python3 url.

bydeathlxncer 4 Sep 16, 2021
declutters url lists for crawling/pentesting

uro Using a URL list for security testing can be painful as there are a lot of URLs that have uninteresting/duplicate content; uro aims to solve that.

Somdev Sangwan 253 Oct 19, 2021
C++ library for urlencode.

liburlencode C library for urlencode.

Khaidi Chu 4 Jul 1, 2021
A tool programmed to shorten links/mask links

A tool programmed to shorten links/mask links

Anontemitayo 3 Oct 19, 2021