Returns unicode slugs

Overview

Python Slugify

A Python slugify application that handles unicode.

status-image version-image coverage-image

Overview

Best attempt to create slugs from unicode strings while keeping it DRY.

Notice

This module, by default installs and uses text-unidecode (GPL & Perl Artistic) for its decoding needs.

However, there is an alternative decoding package called Unidecode (GPL). It can be installed as python-slugify[unidecode] for those who prefer it.

How to install

easy_install python-slugify |OR| easy_install python-slugify[unidecode]
-- OR --
pip install python-slugify |OR| pip install python-slugify[unidecode]

Options

def slugify(
    text,
    entities=True,
    decimal=True,
    hexadecimal=True,
    max_length=0,
    word_boundary=False,
    separator='-',
    save_order=False,
    stopwords=(),
    regex_pattern=None,
    lowercase=True,
    replacements=()
  ):
  """
  Make a slug from the given text.
  :param text (str): initial text
  :param entities (bool): converts html entities to unicode (foo & bar -> foo-bar)
  :param decimal (bool): converts html decimal to unicode (Ž -> Ž -> z)
  :param hexadecimal (bool): converts html hexadecimal to unicode (Ž -> Ž -> z)
  :param max_length (int): output string length
  :param word_boundary (bool): truncates to end of full words (length may be shorter than max_length)
  :param save_order (bool): if parameter is True and max_length > 0 return whole words in the initial order
  :param separator (str): separator between words
  :param stopwords (iterable): words to discount
  :param regex_pattern (str): regex pattern for allowed characters
  :param lowercase (bool): activate case sensitivity by setting it to False
  :param replacements (iterable): list of replacement rules e.g. [['|', 'or'], ['%', 'percent']]
  :return (str): slugify text
  """

How to use

from slugify import slugify

txt = "This is a test ---"
r = slugify(txt)
self.assertEqual(r, "this-is-a-test")

txt = '影師嗎'
r = slugify(txt)
self.assertEqual(r, "ying-shi-ma")

txt = 'C\'est déjà l\'été.'
r = slugify(txt)
self.assertEqual(r, "c-est-deja-l-ete")

txt = 'Nín hǎo. Wǒ shì zhōng guó rén'
r = slugify(txt)
self.assertEqual(r, "nin-hao-wo-shi-zhong-guo-ren")

txt = 'Компьютер'
r = slugify(txt)
self.assertEqual(r, "kompiuter")

txt = 'jaja---lol-méméméoo--a'
r = slugify(txt, max_length=9)
self.assertEqual(r, "jaja-lol")

txt = 'jaja---lol-méméméoo--a'
r = slugify(txt, max_length=15, word_boundary=True)
self.assertEqual(r, "jaja-lol-a")

txt = 'jaja---lol-méméméoo--a'
r = slugify(txt, max_length=20, word_boundary=True, separator=".")
self.assertEqual(r, "jaja.lol.mememeoo.a")

txt = 'one two three four five'
r = slugify(txt, max_length=13, word_boundary=True, save_order=True)
self.assertEqual(r, "one-two-three")

txt = 'the quick brown fox jumps over the lazy dog'
r = slugify(txt, stopwords=['the'])
self.assertEqual(r, 'quick-brown-fox-jumps-over-lazy-dog')

txt = 'the quick brown fox jumps over the lazy dog in a hurry'
r = slugify(txt, stopwords=['the', 'in', 'a', 'hurry'])
self.assertEqual(r, 'quick-brown-fox-jumps-over-lazy-dog')

txt = 'thIs Has a stopword Stopword'
r = slugify(txt, stopwords=['Stopword'], lowercase=False)
self.assertEqual(r, 'thIs-Has-a-stopword')

txt = "___This is a test___"
regex_pattern = r'[^-a-z0-9_]+'
r = slugify(txt, regex_pattern=regex_pattern)
self.assertEqual(r, "___this-is-a-test___")

txt = "___This is a test___"
regex_pattern = r'[^-a-z0-9_]+'
r = slugify(txt, separator='_', regex_pattern=regex_pattern)
self.assertNotEqual(r, "_this_is_a_test_")

txt = '10 | 20 %'
r = slugify(txt, replacements=[['|', 'or'], ['%', 'percent']])
self.assertEqual(r, "10-or-20-percent")

txt = 'ÜBER Über German Umlaut'
r = slugify(txt, replacements=[['Ü', 'UE'], ['ü', 'ue']])
self.assertEqual(r, "ueber-ueber-german-umlaut")

For more examples, have a look at the test.py file.

Command Line Options

With the package, a command line tool called slugify is also installed.

It allows convenient command line access to all the features the slugify function supports. Call it with -h for help.

The command can take its input directly on the command line or from STDIN (when the --stdin flag is passed):

$ echo "Taking input from STDIN" | slugify --stdin
taking-input-from-stdin
$ slugify taking input from the command line
taking-input-from-the-command-line

Please note that when a multi-valued option such as --stopwords or --replacements is passed, you need to use -- as separator before you start with the input:

$ slugify --stopwords the in a hurry -- the quick brown fox jumps over the lazy dog in a hurry
quick-brown-fox-jumps-over-lazy-dog

Running the tests

To run the tests against the current environment:

python test.py

Contribution

Please read the (wiki) page prior to raising any PRs.

License

Released under a (MIT) license.

Version

X.Y.Z Version

`MAJOR` version -- when you make incompatible API changes,
`MINOR` version -- when you add functionality in a backwards-compatible manner, and
`PATCH` version -- when you make backwards-compatible bug fixes.

Sponsors

Neekware Inc.

Comments
  • HTML entity decoding cannot be disabled

    HTML entity decoding cannot be disabled

    I'm trying a very simple test as we look at moving from the outdated awesome-slugify to python-slugify. For backward compatibility reasons we are trying to match its present behavior.

    For the data involved this can be accomplished after jumping through a few small hoops, but not all of them can be accomplished within python-slugify using the options available.

    One hoop is trivial (you can't seem to force the conversion of , to - short of pre-replacing the comma with something else.)

    However, the other hoop is more odd: I can't seem to disable HTML entity decoding.

    In [1]: slugify.slugify('1-209-3-Ž', entities=True)
    Out[1]: '1-209-3-z'
    
    In [2]: slugify.slugify('1-209-3-Ž', entities=False)
    Out[2]: '1-209-3-z'
    

    If entities isn't what mediates this conversion, what does, and is it configurable? Currently (like commas) replacing #& with something like _ seems to be the only way to disable that behavior when necessary.

    opened by MartinFalatic 11
  • Question/Bug: how do I preserve dashes and not replace them with separator?

    Question/Bug: how do I preserve dashes and not replace them with separator?

    I am trying to slugify a string, replacing non-matching characters with an underscore, but I don't want to consider a dash a non-matching character. This is easily achieved with regex but it looks like there is some special slugify behaviour when it comes to dashes which is preventing this working. I can't see anything in the docs or code, about how to work around this.

    What did I do?

    slugify.slugify("test-foo bar", separator='_', regex_pattern=r'[^-a-zA-Z0-9_]+')
    

    What happened? test_foo_bar

    What did I expect? test-foo_bar

    opened by adamcunnington-mlg 10
  • \' shoud be replaced by separator

    \' shoud be replaced by separator

    In your example:

    txt = 'C\'est déjà l\'été.'
    r = slugify(txt)
    self.assertEqual(r, "cest-deja-lete")
    

    I believe the result should be 'c-est-deja-l-ete', not 'cest-deja-lete'.

    opened by sfermigier 8
  • Add a tox file to automatically run the tests and add Python 3.9

    Add a tox file to automatically run the tests and add Python 3.9

    This adds a tox.ini config file to allow automatic running of the tests. This also:

    • Adds tox environments to run formatting and coverage
    • Advertises the existing Python 3.8 support which was being run against Travis
    • Also adds Python 3.9 support

    Testing notes

    You can run the tests against all of the versions of Python by running:

    • pip install tox
    • tox

    After much task running you should get a summary like this:

      py39: commands succeeded
      py38: commands succeeded
      py37: commands succeeded
      py36: commands succeeded
      py35: commands succeeded
      py27: commands succeeded
      pypy: commands succeeded
      pypy3: commands succeeded
      congratulations :)
    

    You can also run the formatting and coverage through tox with:

    • tox -e format
    • tox- e coverage

    I'm not sure how far / if you'd like to transit these functions to tox at all, so for the moment they either replicate or call out to the external functions, but it might be a nice idea to swap it around and let tox manage all the dependencies and tasks etc.

    opened by jon-betts 7
  • What are the few gotchas like when migrating from other packages, etc.

    What are the few gotchas like when migrating from other packages, etc.

    Gotchas ...

    Migrating form awesome-slugify

    • Entity decoding is fully disabled only if you specify all three options (entities=False, hexadecimal=False, decimal=False) (https://github.com/un33k/python-slugify/issues/79)
    opened by un33k 7
  • Name conflict with other slugify package

    Name conflict with other slugify package

    When working on the following code:

    from slugify import slugify
    fname = slugify("cc / loc") + ".png"
    

    My IDE (PyCharm 2018.2.5) will suggest to automatically download a package, but the wrong one. And this results in an error: NameError: name 'unicode' is not defined

    The namespace of this package should be changed, or the half implemented package should be removed from PIP.

    A tool like this should work out of the box.

    opened by EmileSonneveld 7
  • Strange install behavior - pip always installs unidecode

    Strange install behavior - pip always installs unidecode

    I'm trying to test installing python-slugify with unidecode vs text-unidecode for Airflow, however, for some reason I'm getting the unidecode package installed by pip always even when passing the explicit flag for text-unidecode. I can see in the setup.py that this shouldn't happen, so I'm not really sure how it is.

    Can you replicate what I'm experiencing below?

    Note: I'm using pyenv and virtualenvwrapper commands.

    Case 1 - Without SLUGIFY_USES_TEXT_UNIDECODE (expected unidecode)

    t-mbp:~ taylor$ pyenv shell 3.6.5 && mktmpenv -i python-slugify
    Using base prefix '/Users/taylor/.pyenv/versions/3.6.5'
    New python executable in /Users/taylor/.local/share/virtualenvs/tmp-63bc530c9c5a5f1/bin/python3.6
    Also creating executable in /Users/taylor/.local/share/virtualenvs/tmp-63bc530c9c5a5f1/bin/python
    Installing setuptools, pip, wheel...done.
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-63bc530c9c5a5f1/bin/predeactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-63bc530c9c5a5f1/bin/postdeactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-63bc530c9c5a5f1/bin/preactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-63bc530c9c5a5f1/bin/postactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-63bc530c9c5a5f1/bin/get_env_details
    Collecting python-slugify
    Collecting Unidecode>=0.04.16 (from python-slugify)
      Using cached https://files.pythonhosted.org/packages/59/ef/67085e30e8bbcdd76e2f0a4ad8151c13a2c5bce77c85f8cad6e1f16fb141/Unidecode-1.0.22-py2.py3-none-any.whl
    Installing collected packages: Unidecode, python-slugify
    Successfully installed Unidecode-1.0.22 python-slugify-1.2.5
    This is a temporary environment. It will be deleted when you run 'deactivate'.
    

    Case 2 - With SLUGIFY_USES_TEXT_UNIDECODE=yes (expected text-unidecode)

    t-mbp:~ taylor$ pyenv shell 3.6.5 && export SLUGIFY_USES_TEXT_UNIDECODE=yes && mktmpenv -i python-slugify
    Using base prefix '/Users/taylor/.pyenv/versions/3.6.5'
    New python executable in /Users/taylor/.local/share/virtualenvs/tmp-6e020af8417d51b/bin/python3.6
    Also creating executable in /Users/taylor/.local/share/virtualenvs/tmp-6e020af8417d51b/bin/python
    Installing setuptools, pip, wheel...done.
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-6e020af8417d51b/bin/predeactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-6e020af8417d51b/bin/postdeactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-6e020af8417d51b/bin/preactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-6e020af8417d51b/bin/postactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-6e020af8417d51b/bin/get_env_details
    Collecting python-slugify
    Collecting Unidecode>=0.04.16 (from python-slugify)
      Using cached https://files.pythonhosted.org/packages/59/ef/67085e30e8bbcdd76e2f0a4ad8151c13a2c5bce77c85f8cad6e1f16fb141/Unidecode-1.0.22-py2.py3-none-any.whl
    Installing collected packages: Unidecode, python-slugify
    Successfully installed Unidecode-1.0.22 python-slugify-1.2.5
    This is a temporary environment. It will be deleted when you run 'deactivate'.
    

    Case 3 - With text-unidecode pre-installed (expected text-unidecode)

    Last login: Tue Aug 28 15:21:47 on ttys004
    t-mbp:~ taylor$ pyenv shell 3.6.5 && export SLUGIFY_USES_TEXT_UNIDECODE=yes && mktmpenv -i text-unidecode -i python-slugify
    Using base prefix '/Users/taylor/.pyenv/versions/3.6.5'
    New python executable in /Users/taylor/.local/share/virtualenvs/tmp-8bec3e8a00b8e0b/bin/python3.6
    Also creating executable in /Users/taylor/.local/share/virtualenvs/tmp-8bec3e8a00b8e0b/bin/python
    Installing setuptools, pip, wheel...done.
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-8bec3e8a00b8e0b/bin/predeactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-8bec3e8a00b8e0b/bin/postdeactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-8bec3e8a00b8e0b/bin/preactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-8bec3e8a00b8e0b/bin/postactivate
    virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-8bec3e8a00b8e0b/bin/get_env_details
    Collecting text-unidecode
      Using cached https://files.pythonhosted.org/packages/79/42/d717cc2b4520fb09e45b344b1b0b4e81aa672001dd128c180fabc655c341/text_unidecode-1.2-py2.py3-none-any.whl
    Installing collected packages: text-unidecode
    Successfully installed text-unidecode-1.2
    Collecting python-slugify
    Collecting Unidecode>=0.04.16 (from python-slugify)
      Using cached https://files.pythonhosted.org/packages/59/ef/67085e30e8bbcdd76e2f0a4ad8151c13a2c5bce77c85f8cad6e1f16fb141/Unidecode-1.0.22-py2.py3-none-any.whl
    Installing collected packages: Unidecode, python-slugify
    Successfully installed Unidecode-1.0.22 python-slugify-1.2.5
    This is a temporary environment. It will be deleted when you run 'deactivate'.
    
    opened by tedmiston 7
  • Error installing version 1.2.2 with buildout

    Error installing version 1.2.2 with buildout

    Hello,

    When installing version 1.2.2 with buildout I have this error :

    
    Getting distribution for 'python-slugify'.
    zip_safe flag not set; analyzing archive contents...
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 2270, in main
        **kw
      File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
        dist.run_commands()
      File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 411, in run
        self.easy_install(spec, not self.no_deps)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 655, in easy_install
        return self.install_item(None, spec, tmpdir, deps, True)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 700, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 881, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1120, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1106, in run_setup
        run_setup(setup_script, args)
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 258, in run_setup
        raise
      File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 198, in setup_context
        yield
      File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 169, in save_modules
        saved_exc.resume()
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 144, in resume
        six.reraise(type, exc, self._tb)
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 157, in save_modules
        yield saved
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 198, in setup_context
        yield
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 255, in run_setup
        DirectorySandbox(setup_dir).run(runner)
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 285, in run
        return func()
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 253, in runner
        _execfile(setup_script, ns)
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 47, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-nJRC3I/python-slugify-1.2.2/setup.py", line 89, in <module>
      File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
        dist.run_commands()
      File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/usr/lib/python2.7/site-packages/setuptools/command/bdist_egg.py", line 209, in run
        os.path.join(archive_root, 'EGG-INFO'), self.zip_safe()
      File "/usr/lib/python2.7/site-packages/setuptools/command/bdist_egg.py", line 245, in zip_safe
        return analyze_egg(self.bdist_dir, self.stubs)
      File "/usr/lib/python2.7/site-packages/setuptools/command/bdist_egg.py", line 355, in analyze_egg
        safe = scan_module(egg_dir, base, name, stubs) and safe
      File "/usr/lib/python2.7/site-packages/setuptools/command/bdist_egg.py", line 392, in scan_module
        code = marshal.load(f)
    ValueError: bad marshal data (unknown type code)
    An error occurred when trying to install /tmp/tmpN77DXmget_dist/python-slugify-1.2.2.tar.gz. Look above this message for any errors that were output by easy_install.
    While:
      Installing eggs.
      Getting distribution for 'python-slugify'.
    

    I do not have this problem when using version 1.2.1

    opened by ericdupo 7
  • Installing python-slugify on Python 2.7 does not install a compatible version of slugify

    Installing python-slugify on Python 2.7 does not install a compatible version of slugify

    Hello,

    It seems that since python-slugify 6.0.0 has been released, the appropriate version of slugify is no longer installed when using Python 2.7.

    Back in 5.0.2, the python_requires argument of setup.py was set to >=3.6, meaning that this version was not eligible for Python 2.7 and that pip would have automatically installed slugify 4.0.1 (where the python_requires expression fits Python 2.7) as expected.

    Since 6.x, python_requires has been set to ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*" meaning that this version is installed by pip on Python 2.7 (instead of slugify 4.0.1) and cannot work.

    It could be fixed, but I think that if it is fixed (for instance) in a future 6.1.3 version, then pip on Python 2.7 will simply install the 6.1.2 version instead and the initial issue will remain.

    When developing a library that uses slugify and that still supports Python 2.7, the version dependency has to be set explicitly like I did here: https://github.com/lemoncheesecake/lemoncheesecake/blob/09f4bd84b53f599d52c9749bebbe078e6e1d5f15/setup.py#L15

    Best regards,

    Nicolas.

    opened by ndelon 6
  • Syntax error in slugify/slugify.py

    Syntax error in slugify/slugify.py

    Hello,

    The current master branch (commit c096bcdd76b0ef216716eccef5b19839266e1372) gives a syntax error in slugify/slugify.py

    File ".../site-packages/slugify/slugify.py", line 69 replacements: typing.Iterable[typing.Iterable[str]] = ()): ^ SyntaxError: invalid syntax

    Have a good day

    opened by janderson-atb-potsdam 5
  • Fix misleading pattern name and documentation

    Fix misleading pattern name and documentation

    The ALLOWED_CHARS_PATTERN and regex_pattern are actually disallowed characters in the final slug. Also since the code lowers the case in line 141, there's no need for a separate regex pattern for lowercase strings

    opened by mrezzamoradi 5
  • Looking for 1-2 contributors  to de-risk the project as it is now used by 21k+ projects

    Looking for 1-2 contributors to de-risk the project as it is now used by 21k+ projects

    python-slugify is now used by 21k+ projects and many major companies. As such, it would be smart to have more than one person with admin access to the project, especially, in times of pandemics.

    If you would like to contribute in de-risking python-slugify and posses solid python background, with keen interests in supporting open source projects for the benefit of many; - send me an email with links to a few python projects you are maintaining.

    I will select two individuals and promote them to admin level.

    I'd appreciate your consideration.

    Val

    opened by un33k 2
Owner
Val Neekman
Val is a Solutions Architect with a passion for Typescript-Angular, Python-Django, UX and Performant Software. He is the Principal Consultant at Neekware Inc.
Val Neekman
A slugifier that works in unicode

Unicode Slugify Unicode Slugify is a slugifier that generates unicode slugs. It was originally used in the Firefox Add-ons web site to generate slugs

Mozilla 315 Nov 21, 2022
A python tool to convert Bangla Bijoy text to Unicode text.

Unicode Converter A python tool to convert Bangla Bijoy text to Unicode text. Installation Unicode Converter can be installed via PyPi. Make sure pip

Shahad Mahmud 10 Sep 29, 2022
Returns unicode slugs

Python Slugify A Python slugify application that handles unicode. Overview Best attempt to create slugs from unicode strings while keeping it DRY. Not

Val Neekman (AvidCoder) 1.3k Dec 23, 2022
Fixes mojibake and other glitches in Unicode text, after the fact.

ftfy: fixes text for you >>> print(fix_encoding("(ง'⌣')ง")) (ง'⌣')ง Full documentation: https://ftfy.readthedocs.org Testimonials “My life is li

Luminoso Technologies, Inc. 3.4k Jan 8, 2023
A slugifier that works in unicode

Unicode Slugify Unicode Slugify is a slugifier that generates unicode slugs. It was originally used in the Firefox Add-ons web site to generate slugs

Mozilla 315 Nov 21, 2022
Python library that measures the width of unicode strings rendered to a terminal

Introduction This library is mainly for CLI programs that carefully produce output for Terminals, or make pretend to be an emulator. Problem Statement

Jeff Quast 305 Dec 25, 2022
Fixes mojibake and other glitches in Unicode text, after the fact.

ftfy: fixes text for you >>> print(fix_encoding("(ง'⌣')ง")) (ง'⌣')ง Full documentation: https://ftfy.readthedocs.org Testimonials “My life is li

Luminoso Technologies, Inc. 3.4k Dec 29, 2022
Fixes mojibake and other glitches in Unicode text, after the fact.

ftfy: fixes text for you >>> print(fix_encoding("(ง'⌣')ง")) (ง'⌣')ง Full documentation: https://ftfy.readthedocs.org Testimonials “My life is li

Luminoso Technologies, Inc. 2.9k Feb 17, 2021
Detects members having unicode names. Public bot: @scarletwitchprobot

✨ Scarletwitch bot ✨ Detects unicode names members in a tg chat & provides a option to take action on that user ! Public bot: @scarletwitchprobot Supp

ÁÑÑÍHÌLÅTØR SPÄRK 18 Nov 12, 2022
My own Unicode compression algorithm

Zee Code ZCode is a custom compression algorithm I originally developed for a competition held for the Spring 2019 Datastructures and Algorithms cours

Vahid Zehtab 2 Oct 20, 2021
These scripts look for non-printable unicode characters in all text files in a source tree

find-unicode-control These scripts look for non-printable unicode characters in all text files in a source tree. find_unicode_control.py should work w

Siddhesh Poyarekar 25 Aug 30, 2022
A python tool to convert Bangla Bijoy text to Unicode text.

Unicode Converter A python tool to convert Bangla Bijoy text to Unicode text. Installation Unicode Converter can be installed via PyPi. Make sure pip

Shahad Mahmud 10 Sep 29, 2022
Unicode fuzzer for various purposes

UnicodeToy Unicode fuzzer for various purposes Unicode based on version 14.0 features Generate the shortest xss domain payload Generate unicode str, u

null 33 Nov 27, 2022
A quick script to spot the usage of Unicode Bidi (bidirectional) characters that could lead to an Invisible Backdoor

Invisible Backdoor Detector is a little Python script that allows you to spot and remove Bidi characters that could lead to an invisible backdoor. If you don't know what that is you should check the related paragraph.

SecSI 28 Dec 29, 2022
A Happy and lightweight Python Package that searches Google News RSS Feed and returns a usable JSON response and scrap complete article - No need to write scrappers for articles fetching anymore

GNews ?? A Happy and lightweight Python Package that searches Google News RSS Feed and returns a usable JSON response ?? As well as you can fetch full

Muhammad Abdullah 273 Dec 31, 2022
Visual Weather api. Returns beautiful pictures with the current weather.

VWapi Visual Weather api. Returns beautiful pictures with the current weather. Installation: sudo apt update -y && sudo apt upgrade -y sudo apt instal

Hotaru 33 Nov 13, 2022
emoji-math computes the given python expression and returns either the value or the nearest 5 emojis as measured by cosine similarity.

emoji-math computes the given python expression and returns either the value or the nearest 5 emojis as measured by cosine similarity.

Andrew White 13 Dec 11, 2022
An OSINT tool that searches for devices directly connected to the internet (IoT) with a user specified query. It returns results for Webcams, Traffic lights, Refridgerators, Smart TVs etc.

An OSINT tool that searches for devices directly connected to the internet (IoT) with a user specified query. It returns results for Webcams, Traffic

Richard Mwewa 48 Nov 20, 2022
Given a string or a text file with plain text , returns his encryption using SHA256 method

Encryption using SHA256 Given a string or a .txt file with plain text. Returns his encryption using SHA256 method Requirements : pip install pyperclip

yuno 3 Jan 24, 2022
This CLI give the possibility to do a queries in Star Wars API and returns a JSON in a terminal.

Star Wars CLI (swcli) This CLI give the possibility to do a queries in Star Wars API and returns a JSON in a terminal. Install $ pip install swcli Qu

Pery Lemke 5 Nov 5, 2021