Pygments is a generic syntax highlighter written in Python

Last update: Jan 6, 2023

Overview

Welcome to Pygments

This is the source of Pygments. It is a generic syntax highlighter written in Python that supports over 500 languages and text formats, for use in code hosting, forums, wikis or other applications that need to prettify source code.

Installing

... works as usual, use pip install Pygments to get published versions, or python setup.py install to install from a checkout.

Documentation

... can be found online at https://pygments.org/ or created with Sphinx by

cd doc
make html

Development

... takes place on GitHub, where the Git repository, tickets and pull requests can be viewed.

Continuous testing runs on GitHub workflows:

The authors

Pygments is maintained by Georg Brandl, e-mail address georg@python.org and Matthäus Chajdas.

Many lexers and fixes have been contributed by Armin Ronacher, the rest of the Pocoo team and Tim Hatch.

The code is distributed under the BSD 2-clause license. Contributors making pull requests must agree that they are able and willing to put their contributions under that license.

Comments

Is there a low-baggage html option?

(Original issue 1514 created by gcallah on 2019-05-08T16:07:56.411437+00:00)

The package seems oriented towards processing files or at least large chunks of code. But I want to feed in perhaps as little as a single line, and get back the line with minimal html inserted: really just the span tags, no divs or anything like that. Is this currently possible? I couldn’t see it in the docs.
T-feature X-imported major

opened by Anteru 24
Add Macaulay2 lexer

Macaulay2 is a computer algebra system that is widely used for research in algebraic geometry and commutative algebra. It comes with its own programming language, and it would be great if support were added to Pygments, e.g., for syntax highlighting Macaulay2 code in journal articles using minted.

The lists of symbols are generated automatically using the script make-M2-symbols.m2 in the Macaulay2 source. See https://github.com/Macaulay2/M2/pull/2040.
lexing

opened by d-torrance 19
Error parsing Python raw f-string with backslash
Python(3) code using raw f-strings containing a backslash within escaped braces (i.e. double braces) cannot be parsed. A minimum example is

fr'{{\S'

Some pointers:

Works: f'{{\S'.

Works: r'{{\S'

Works: rf'{{'.

Works: rf'\S'.

Also do not work: rf'{{\S'.

The S is not important, though some character must be there in order for the syntax to be valid.

This might seem like a pretty esoteric case. For me it comes up a lot when writing LaTeX in Matplotlib.
bug good first issue minor lexing
opened by jmd-dk 19
Moving pygments.rb to pygments organization

Hello, guys! I've started an effort to reanimate pygments.rb project (a Ruby gem that exposes pygments to Ruby programs): https://github.com/tmm1/pygments.rb/issues/197

The problem is, pygments.rb is currently a personal repo, so previous developer cannot grant me admin rights and it would be more convenient if pygments.rb belonged to an organization.

Would you accept pygments.rb into pygments GitHub organization?

opened by slonopotamus 19
Support for css variables (aka custom properties)
(Original issue 1356 created by aleray on 2017-06-15T22:15:43.804108+00:00)

Hi,

I noticed that the css lexer doesn't support yet css custom properties like this:

#!css :root { --page-width: 148.5mm; --page-height: 210mm; --margin-top: 30mm; --margin-right: 20mm; --margin-bottom: 30mm; --margin-left: 20mm; --column-count: 2; --column-gap: 9pt; }

Is this something planned somehow?

Thanks!
bug good first issue X-imported major
opened by Anteru 19
Add support for Arturo language

This PR adds Arturo Language highlight support. 🙂

Simple, expressive & portable programming language for efficient scripting

Arturo's Github Arturo's Website

opened by RickBarretto 18
Replace tests that assert on token output with auto-updatable samples

This extracts the code samples from each test that used to do assert list(lexer.get_tokens(fragment)) == tokens, puts each of them into their own file, and gives the ability to automatically update such files if there are changes to the code affecting them.

To perform such an update in-place, run pytest tests/test_lexers.py --update-goldens. ~~The functionality comes from the 'pytest-golden' plugin (disclaimer: written by me!)~~

When testing without the update flag, this just asserts that the token list does not change from the stored list.

So, the tests still serve the same purpose as before (and you can see the diffs both from pytest and in code review), just that they don't need to be manually updated. One only needs to provide the interesting code sample.

Note to reviewers:

tests/test_lexers.py is the newly added file, which is the entry point for these tests.

The conversion work was mostly mechanical, but I'll make sure to look through all of it again if the overall idea is liked.

opened by oprypin 18
tox and GitHub checks are not aligned

After opening #1555 the regex linter warned me about a problem in my patch. This was not caught during extensive testing using tox, and indicates that tox and the GitHub checks are not aligned with each other.

tox must be updated to use the same regex linter that GitHub checks is using so that contributors can catch problems before submitting PR's.

opened by kurtmckee 18
new lexer request: dot (graphviz)
(Original issue 1024 created by psuter on 2014-08-12T08:29:53.471324+00:00)

The DOT language is used by the open source graph visualization software Graphviz to represent structural information as diagrams of abstract graphs and networks.

The language grammar is described at http://www.graphviz.org/doc/info/lang.html

An example:

#!dot digraph G { Hello->Pygments }

Would be nice to have a lexer for this in Pygments.
T-feature X-imported major
opened by Anteru 16
Release new version

(Original issue 1007 created by nobbz on 2014-06-20T10:11:46.098017+00:00)

Hi!

There was a pullrequest about a year ago that has been merged, but pygments installed via pip seems not to contain this changeset.

So when is a new pygments release planned?

The pullrequest in question is #193.
X-imported major T-proposal

opened by Anteru 16
Support ANSI (ECMA-48) color-coded text input

(Original issue 770 created by mgorny on 2012-05-14T06:18:43.247173+00:00)

I feel like many pastebin users would appreciate such a feature in pygments: reading (already colored) data stream with ECMA-48 color codes embedded (like console program output) and outputting it in any format pygments support.

I guess such a feature would require adding a new token types to represent console colors.

If there's interest in this, I can assemble a patch for it.
X-imported T-proposal minor

opened by Anteru 16

2.14.0: documentation build fails

Looks like something recently has changed because now documetation build fails with

+ /usr/bin/sphinx-build -n -T -b man doc build/sphinx/man
Running Sphinx v5.3.0
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [man]: all manpages
updating environment: [new config] 28 added, 0 changed, 0 removed
reading sources... [ 96%] languages
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/cmd/build.py", line 281, in build_main
    app.build(args.force_all, args.filenames)
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 347, in build
    self.builder.build_update()
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 307, in build_update
    self.build(['__all__'], to_build)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 326, in build
    updated_docnames = set(self.read())
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 433, in read
    self._read_serial(docnames)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 454, in _read_serial
    self.read_doc(docname)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 510, in read_doc
    publisher.publish()
  File "/usr/lib/python3.8/site-packages/docutils/core.py", line 224, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/usr/lib/python3.8/site-packages/sphinx/io.py", line 104, in read
    self.parse()
  File "/usr/lib/python3.8/site-packages/docutils/readers/__init__.py", line 76, in parse
    self.parser.parse(self.input, document)
  File "/usr/lib/python3.8/site-packages/sphinx/parsers.py", line 78, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/usr/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 169, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "/usr/lib/python3.8/site-packages/docutils/statemachine.py", line 233, in run
    context, next_state, result = self.check_line(
  File "/usr/lib/python3.8/site-packages/docutils/statemachine.py", line 445, in check_line
    return method(match, context, next_state)
  File "/usr/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2785, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/usr/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 325, in section
    self.new_subsection(title, lineno, messages)
  File "/usr/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 391, in new_subsection
    newabsoffset = self.nested_parse(
  File "/usr/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 279, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/usr/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/usr/lib/python3.8/site-packages/docutils/statemachine.py", line 233, in run
    context, next_state, result = self.check_line(
  File "/usr/lib/python3.8/site-packages/docutils/statemachine.py", line 445, in check_line
    return method(match, context, next_state)
  File "/usr/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2355, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/usr/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2367, in explicit_construct
    return method(self, expmatch)
  File "/usr/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2104, in directive
    return self.run_directive(
  File "/usr/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2154, in run_directive
    result = directive_instance.run()
  File "/usr/lib/python3.8/site-packages/pygments/sphinxext.py", line 78, in run
    raise Exception('invalid argument for "pygmentsdoc" directive')
Exception: invalid argument for "pygmentsdoc" directive

Exception occurred:
  File "/usr/lib/python3.8/site-packages/pygments/sphinxext.py", line 78, in run
    raise Exception('invalid argument for "pygmentsdoc" directive')
Exception: invalid argument for "pygmentsdoc" directive
The full traceback has been saved in /tmp/sphinx-err-c8js501f.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

opened by kloczek 1

backslash is not defined in `PostgresLexer`

I stumbled on this problem multiple times. I need to lex an SQL script for PostgreSQL, specifically for creating and connecting to DB and creating tables in it. Everything works fine, but I don't know why such command as \connect is not defined. With minted package for TeX-family typesetting systems, the output of PDF file shows that the backslash in \connect command is not defined (or probably the command itself):

Other PostgreSQL lexers work even worse (some other commands are also not defined). After running python3 -I _postgres_builtins.py nothing has changed. Is this a bug or unimplemented syntax?

opened by Andrew15-5 5
Add a token type like Generic.EmphStrong to allow bold italic styling

For markup languages like Markdown, syntaxes like ***bold italic*** create a bold italic styled text. Unfortunately I didn't find a way to achieve this since we only have Generic.Emph and Generic.Strong token type. Given that such syntaxes exist in many markup languages, should we add a token type like Generic.EmphStrong?

opened by diskdance 1
New language support: hare
https://harelang.org/

"Hare is a systems programming language designed to be simple, stable, and robust."

https://github.com/colinkiama/hare-sublime-syntax

https://github.com/sudison/harelang-intellij
opened by alexcoder04 0
Add lexer for Portugol language

Portugol is a simplified programming language based on Pascal language. This language is used for teach programming for people that speak Portuguese but don't understand English.

https://www.apoioinformatica.inf.br/produtos/visualg/linguagem

opened by LorhanSohaky 1

MATLAB: Comments not recognized as nestable

The MATLAB lexer does not recognize nested multiline comments. The below code is proof that Highlight.js (the lib used by GitHub) gets it right:

% These are the derivatives for each term
d = [0 -1 0];

%{
  %{
    (Example of a nested comment, indentation is for cosmetics (and ignored).)
  %}
  We form the sequence, following the Taylor formula.
  Note that we're operating on a vector.
%}
seq = d .* (x - c).^n ./(factorial(n))

% We add-up to get the Taylor approximation
approx = sum(seq)

As for Pygments: obraz

The black section of text between %} and %} should be highlighted like a comment, not like normal text.

opened by Keyacom 0

Releases(2.14.0)

2.14.0(Jan 1, 2023)
Added lexers:

Arturo (#2259)

GAP session (#2211)

Fift (#2249)

func (#2232)

Jsonnet (#2239)

Minecraft schema (#2276)

MIPS (#2228)

Phix (#2222)

Portugol (#2300)

TL-b (#2247)

World of Warcraft TOC format (#2244, #2245)

Wren (#2271)

Updated lexers:

Abap: Update keywords (#2281)

Alloy: Update for Alloy 6 (#1963)

C family (C, C++ and many others):

Fix an issue where a chunk would be wrongly recognized as a function definition due to braces in comments (#2210)

Improve parantheses handling for function definitions (#2207, #2208)

C#: Fix number and operator recognition (#2256, #2257)

CSound: Updated builtins (#2268)

F#: Add .fsx file extension (#2282)

gas (GNU assembler): recognize braces as punctuation (#2230)

HTTP: Add CONNECT keyword (#2242)

Inform 6: Fix lexing of properties and doubles (#2214)

INI: Allow comments that are not their own line (#2217, #2161)

Java properties: Fix issue with whitespace-delimited keys, support comments starting with ! and escapes, no longer support undocumented ; and // comments (#2241)

LilyPond: Improve heuristics, add \maxima duration (#2283)

LLVM: Add opaque pointer type (#2269)

Macaulay2: Update keywords (#2305)

Minecraft-related lexers (SNB and Minecraft function) moved to pygments.lexers.minecraft (#2276)

Nim: General improvements (#1970)

Nix: Fix single quotes inside indented strings (#2289)

Objective J: Fix catastrophic backtracking (#2225)

NASM: Add support for SSE/AVX/AVX-512 registers as well as 'rel' and 'abs' address operators (#2212)

Powershell:

Add local: keyword (#2254)

Allow continuations without markers (#2262, #2263)

Solidity: Add boolean operators (#2292)

Spice: Add enum keyword and fix a bug regarding binary, hexadecimal and octal number tokens (#2227)

YAML: Accept colons in key names (#2277)

Fix make mapfiles when Pygments is not installed in editable mode (#2223)

Support more filetypes and compression types in autopygmentize (#2219)

Merge consecutive tokens in Autohotkey, Clay (#2248)

Add .nasm as a recognized file type for NASM (#2280)

Add *Spec.hs as a recognized file type for HSpec (#2308)

Add *.pyi (for typing stub files) as a recognized file type for Python (#2331)

The HTML lexer no longer emits empty spans for whitespace (#2304)

Fix IRCFormatter inserting linenumbers incorrectly (#2270)

Source code(tar.gz)
Source code(zip)
2.13.0(Aug 15, 2022)
Added lexers:

COMAL-80 (#2180)

JMESPath (#2174, #2175, #2179, #2182)

Sql+Jinja (#2148)

Updated lexers:

Ada: support Ada 2022 (#2121); disable recognition of namespaces because it disturbs lexing of aspects (#2125)

Agda: allow straight quotes in module names (#2163)

C family (C, C++ and many others): allow comments between elements of function headers, e.g. between the arguments and the opening brace for the body (#1891)

C++: Resolve several cases of Error tokens (#2207, #2208)

Coq: add some common keywords, improve recognition of Set and qualified identifiers (#2158)

F*: Allow C-style comments anywhere in a line

Fortran: fix catastrophic backtracking with backslashes in strings (#2194)

Go: add support for generics (#2167)

Inform: update for version 6.40 (#2190)

Isabelle: recognize cartouches (#2089)

Java: support multiline strings aka. text blocks (#2132)

Kotlin: add value modifier (#2142)

LilyPond: add some missing builtins

Macaulay2: update builtins (#2139)

Matlab session: fix traceback when a line continuation ellipsis appears in the output (#2166)

.NET: add aliases for LibreOffice Basic, OpenOfficeBasic and StarOffice Basic (#2170)

Nim: use Name.Builtin instead of Keyword.Type (#2136)

PHP: fix \"$var\" inside strings (#2105)

Python: only recognize \N, \u and \U escape sequences in string literals, but not in bytes literals where they are not supported (#2204)

Tcl: support ${name} variables (#2145)

Terraform: accept leading whitespace for << heredoc delimiters (#2162)

Teraterm: Various improvements (#2165)

Spice: add support for the recently added features including more builtin functions and bin, oct, hex number formats (#2206)

Added styles:

GitHub dark (#2192)

StarOffice (#2168)

Nord (nord and nord-darker; #2189, #1799, #1678)

Pygments now tries to use the importlib.metadata module to discover plugins instead of the slower pkg_resources (#2155). In particular, this largely speeds up the pygmentize script when the lexer is not specified.

importlib.metadata is only available in the Python standard library since Python 3.8. For older versions, there exists an importlib_metadata backport on PyPI. For this reason, Pygments now defines a packaging extra plugins, which adds a requirement on importlib_metadata if the Python version is older than 3.8. Thus, in order to install Pygments with optimal plugin support even for old Python versions, you should do:

pip install pygments[plugins]

Pygments still falls back on pkg_resources if neither importlib.metadata nor importlib_metadata is found, but it will be slower.

Silently ignore BrokenPipeError in the command-line interface (#2193).

The HtmlFormatter now uses the linespans attribute for anchorlinenos if the lineanchors attribute is unset (#2026).

The highlight, lex and format functions no longer wrongly report "argument must be a lexer/formatter instance, not a class" in some cases where this is not the actual problem (#2123).

Fix warnings in doc build (#2124).

The codetagify filter now recognizes FIXME tags by default (#2150).

The pygmentize command now recognizes if the COLORTERM environment variable is set to a value indicating that true-color support is available. In that case, it uses the TerminalTrueColorFormatter by default (#2160)

Remove redundant caches for filename patterns (#2153)

Use new non-deprecated Pillow API for text bounding box in ImageFormatter (#2198)

Remove default_style (#930, #2183)

Stop treating DeprecationWarnings as errors in the unit tests (#2196)

Source code(tar.gz)
Source code(zip)
2.12.0(Apr 24, 2022)
Added lexers:

Cplint (#2045)

Macaulay2 (#1791)

Minecraft (#2107)

Qlik (#1925)

UnixConfigLexer for "colon-separated" config files, like /etc/passwd (#2112)

Updated lexers:

Agda: Update keyword list (#2017)

C family: Fix identifiers after case statements (#2084)

Clojure: Highlight ratios (#2042)

Csound: Update to 6.17 (#2064)

CSS: Update the list of properties (#2113)

Elpi:

Fix catastrophic backtracking (#2053, #2061)

Fix handling of -> (#2028)

Futhark: Add missing tokens (#2118)

Gherkin: Add But (#2046)

Inform6: Update to 6.36 (#2050)

JSON: Support C comments in JSON (#2049). Note: This doesn't mean the JSON parser now supports JSONC or JSON5 proper, just that it doesn't error out when seeing a /* */ or // style comment. If you need proper comment handling, consider using the JavaScript lexer.

LilyPond:

Fix incorrect lexing of names containing a built-in (#2071)

Fix properties containing dashes (#2099)

PHP: Update builtin function and keyword list (#2054, #2056)

Scheme: Various improvements (#2060)

Spice: Update the keyword list, add new types (#2063, #2067)

Terraform:

Support non-idiomatic comments (#2065, #2066)

Fix class name lexing (#2097)

Add plugins argument to get_all_lexers().

Bump minimal Python version to 3.6 (#2059)

Fix multiple lexers marking whitespace as Text (#2025)

Remove various redundant uses of re.UNICODE (#2058)

Associate .resource with the Robot framework (#2047)

Associate .cljc with Clojure (#2043)

Associate .tpp with C++ (#2031)

Remove traces of Python 2 from the documentation (#2039)

The native style was updated to meet the WCAG AAA contrast guidelines (#2038)

Fix various typos (#2030)

Fix Groff formatter not inheriting token styles correctly (#2024)

Various improvements to the CI (#2036)

The Ada lexer has been moved to a separate file (#2117)

When linenos=table is used, the <table> itself is now wrapped with a <div class="highlight"> tag instead of placing it inside the <td class="code"> cell (#632.) With this change, the output matches the documented behavior. :warning: If you have subclassed HtmlFormatter.wrap, you may have to adjust the logic.

Source code(tar.gz)
Source code(zip)
2.11.2(Jan 6, 2022)
Updated lexers:

C-family: Fix incorrect handling of labels (#2022, #1996, #1182)

Java: Fixed an issue with record keywords result in Error tokens in some cases (#2018)

Fix links to line numbers not working correctly (#2014)

Remove underline from Whitespace style in the Tango theme (#2020)

Fix IRC and Terminal256 formatters not backtracking correctly for custom token types, resulting in some unstyled tokens (#1986)

Source code(tar.gz)
Source code(zip)
2.11.1(Dec 31, 2021)
Updated lexers:

C-family: Handle return types with multiple tokens (e.g. unsigned int) (#2008)

JSON: Fix a regression which caused whitespace before : to result in Error tokens (#2010)

SPICE: Various improvements (#2009)

Source code(tar.gz)
Source code(zip)
2.11.0(Dec 30, 2021)
Added lexers:

BDD (#1803)

Elpi (#1894)

LilyPond (#1845, #1968, #1971, #2001). This comes with a custom style as well.

Maxima (#1885)

Rita (#1541, #2003)

Savi (#1863)

Sed (#1935)

Sophia contracts (#1974)

Spice (#1980)

.SRCINFO (#1951)

Updated lexers:

ABNF: Allow one-character rules (#1804)

Assembly: Fix incorrect token endings (#1895, #1961)

Bibtex: Distinguish between comment and commentary (#1899, #1806)

C family: Support unicode identifiers (#1848)

CDDL: Fix slow lexing speed (#1959)

Debian control: Add missing fields (#1946)

Devicetree: Recognize hexadecimal addresses for nodes (#1949)

GDScript: Add void data type (#1948)

GSQL

Fix comment handling (#2002)

Fix catastrophic backtracking (#2006)

HTML, XML: Improve comment handling (#1896)

Java: Add yield (#1941) and sealed classes/record (#1902)

Makefiles (#1860, #1898)

objdump-nasm: Improve handling of --no-show-raw-insn dumps (#1981)

Prolog: Support escaped \ inside quoted strings (#1479)

Python:

Support ~ in tracebacks (#2004)

Support the pattern matching keywords (#1797, #1994)

RobotFramework: Improve empty brace handling (#1921, #1922)

Terraform

Add the 'set' type (#1909)

Support heredocs (#1909)

Added styles:

Dracula (#1796)

Friendly Grayscale (#1040, #1273)

LilyPond (#1845) -- to be used for the LilyPond language.

One-Dark (#1924, #1979)

There is new infrastructure in place to improve style accessibility. The default style has been updated to conform to WCAG recommendations. All styles are now checked for sufficient contrast by default to prevent regressions. (#1919, #1937, #1938, #1940)

Clean up unused imports (#1887)

Fix multiple lexers producing repeated single-character tokens

Fix multiple lexers marking whitespace as Text (#1237, #1905, #1908, #1914, #1911, #1923, #1939, #1957, #1978)

Remove duplicated assignments in the Paraiso style (#1934)

pygmentize supports JSON output for the various list functions now, making it easier to consume them from scripts. (#1437, #1890)

Use the shell lexer for kshrc files (#1947)

Use the ruby lexer for Vagrantfile files (#1936)

Use the C lexer for .xbm and .xpm files (#1802)

Add a groff formatter (#1873)

Update documentation (#1928)

Line anchors now link to themselves (#1973)

Add official support for Python 3.10 (#1917)

Fix several missing colors in dark styles: Gruvbox dark, Monokai, Rrt, Sas, Strata dark (#1955)

Associate more file types with man pages

The HtmlFormatter can now emit tooltips for each token to ease debugging of lexers (#1822)

Add f90 as an alias for fortran (#2000)

Source code(tar.gz)
Source code(zip)
2.10.0(Aug 15, 2021)
Added lexers:

ASC armored files (#1807)

GSQL (#1809, #1866)

Javascript REPL (#1825)

procfile (#1808)

Smithy (#1878, #1879)

Updated lexers:

C-family: Fix preprocessor token issues (#1830)

C# (#1573, #1869)

CSound (#1837)

Fennel (#1862)

JavaScript (#1741, #1814)

LLVM (#1824)

Rust

Fix lexing of "break" and "continue" (#1843)

Improve attribute handling (#1813)

Scala: Add support for the \ operator (#1857)

Swift (#1767, #1842)

Tcl: Allow , and @ in strings (#1834, #1742)

Fix assert statements in TNT lexer.

Token types across all lexers have been unified (using the most common token type name) (#1816, #1819)

Improve Jasmin min score analysis (#1619)

Add new alias for Go files (#1827)

Fix multi-line console highlighting (#1833)

Add a new trivial lexer which outputs everything as Text.Generic.Output (#1835, #1836)

Use the .ini lexer for systemd files (#1849)

Fix a FutureWarning related to words() (#1854)

pwsh is now recognized as an alias for PowerShell (#1876)

Source code(tar.gz)
Source code(zip)
2.9.0(May 3, 2021)
Added lexers:

APDL, gcode (#1714)

Kuin (#1300)

NestedText (#1578)

OMG IDL (#1595)

TEAL (#1671)

ThingsDB (#1295)

WebAssembly (#1416, #1564)

Updated lexers:

AMDGPU (#1717, #1775)

APL (#1747)

C/C++: Improve namespace handling (#1722, #1561, #1719, #1746)

Chapel (#1743)

Coq (#1721)

Cython (#853)

Groovy (#1765)

JavaScript (#1741)

Julia (#1715)

Octave: Allow multiline and block-percent comments (#1726)

PowerShell: Improve lexing of : (#1682, #1758)

PromQL (#1783)

Python: Improve float parsing (#1768, #1740)

Rust (#1061)

Scala: Rewrite to support Scala3 (#1694, #1035, #1121)

Swift (#1767)

Terraform: Support 0.14 syntax (#1756)

Velocity: Detect multi-line patterns (#1776)

Add Pango formatter (#1727)

Autopygmentize uses file first instead of pygments -N (#1786)

Fix links (#1716)

Fix issue with LaTeX formatter and minted (#1734, #1735, #1736, #1737)

Improve alias order (#1780)

Improve line number colors (#1779, #1778)

Fix CTag related issue (#1724)

Recognize .leex as Elixir templates

Updated filename handling in HTML formatter if linenos='table' (#1757)

Previously the filename would be emitted within the <td> holding the code, but outside the <pre>. This would invariably break the a lignment with line numbers.

Now if filename is specified, a separate <tr> is emitted before the table content which contains a single <th> with colspan=2 so it spans both the line number and code columns. The filename is still within <span class="filename">...</span> so any existing styles should still apply, although the CSS path may need to change.

For an example of the new output format see table_cls_step_1_start_1_special_0_noanchor_filename.html in the tests/html_linenos_expected_output/ directory.

For more details and discussion see the issue https://github.com/pygments/pygments/issues/1757

Added styles:

Gruvbox light+dark (#1763)

Source code(tar.gz)
Source code(zip)
2.8.1(Mar 7, 2021)
Fix issue with LaTeX formatter and minted (#1734, #1735, #1736, #1737)

Source code(tar.gz)
Source code(zip)
2.8.0(Feb 14, 2021)
Added lexers:

AMDGPU (#1626)

CDDL (#1379, #1239)

Futhark (#1691)

Graphviz/DOT (#1657, #731)

Updated lexers:

AutoIt: Support single quoted strings (#1667, #1663)

C/C++ & related: Fix mishandling '*/' (#1695)

Cocoa: Add builtin types (#1703)

Console (#1672)

Eiffel: Fix performance issues (#1658)

Fortran: Improve combined keyword detection (#1677, #1188)

J: Fix operator ? lexing (#1700, #1149)

JavaScript/TypeScript: Fix escapes in backtick strings (#1679, #1686)

Kotlin: Improve string interpolation, modifier keyword handling, and various small issues (#1699)

LESS: Support single-line comments (#1046)

Matlab:

Add support for class properties (#1466)

Update builtin functions (#1705)

Various cleanups (#1673)

Python: Improve handling of raw f-strings (#1681, #1683)

Ruby: Better method name handling (#1531)

Stata: Updated keywords (#1470)

Added styles:

Material (#1662)

Zenburn (#1659)

The pygmentize script now uses argparse, all options should work as before

Add pygmentize -C option to guess a lexer from content

With this release, Pygments moves to a new internal testing system (#1649.) See Contributing.md for details. The main advantage of this new change is a much better test coverage of all existing example lexers. It also makes it much easier to add new test snippets.

Make guessing prefer Python 3 lexer

Do not guess MIME or SQL without reason

Changed setuptools to use a declarative config through setup.cfg. Building Pygments now requires setuptools 39.2+.

Add markdown to MarkdownLexer aliases (#1687)

Change line number handling

In <table> based output, the td.linenos element will have either a normal or special class attached. Previously, only special line numbers got a class. This prevents styles from getting applied twice - once via <pre>, once via <span class="special">. This also means that td.linenos pre is no longer styled, instead, use td.linenos .normal and td.linenos .special.

In the "inline" style, the DOM element order was changed. The line number is added first, then the line is wrapped is wrapped by the highlighter. This fixes lines not being fully highlighted.

The visual output for inline and non-inline line numbers & highlighting, as well as class-based and inline styling is now consistent.

Line number styles are set to background-color: transparent and color: inherit by default. This works much better with dark styles which don't have colors set for line numbers.

Remove "raw" alias from RawTokenLexer, so that it cannot be selected by alias.

Fix RawTokenLexer to work in Python 3 and handle exceptions.

Add prompt colors to the Solarized theme (#1529)

Image formatter supports background colors now (#1374)

Add support for anchors in conjunction with inline line numbers (#1591)

Modernize the codebase using pyupgrade (#1622)

Add support for line numbers to the terminal256 formatter (#1674, #1653)

Improve analyze_text logic for ECL (#1610)

Improve analyze_text logic for CBM Basic V2 (#1607)

Improve LaTeX formatter (#1708, #1709)

Source code(tar.gz)
Source code(zip)
2.7.4(Jan 12, 2021)
Updated lexers:

Apache configurations: Improve handling of malformed tags (#1656)

CSS: Add support for variables (#1633, #1666)

Crystal (#1650, #1670)

Coq (#1648)

Fortran: Add missing keywords (#1635, #1665)

Ini (#1624)

JavaScript and variants (#1647 -- missing regex flags, #1651)

Markdown (#1623, #1617)

Shell

Lex trailing whitespace as part of the prompt (#1645)

Add missing in keyword (#1652)

SQL - Fix keywords (#1668)

Typescript: Fix incorrect punctuation handling (#1510, #1511)

Fix infinite loop in SML lexer (#1625)

Fix backtracking string regexes in JavaScript/TypeScript, Modula2 and many other lexers (#1637)

Limit recursion with nesting Ruby heredocs (#1638)

Fix a few inefficient regexes for guessing lexers

Fix the raw token lexer handling of Unicode (#1616)

Revert a private API change in the HTML formatter (#1655) -- please note that private APIs remain subject to change!

Fix several exponential/cubic-complexity regexes found by Ben Caller/Doyensec (#1675)

Fix incorrect MATLAB example (#1582)

Thanks to Google's OSS-Fuzz project for finding many of these bugs.
Source code(tar.gz)
Source code(zip)
2.7.3(Dec 6, 2020)
Updated lexers:

Ada (#1581)

HTML (#1615, #1614)

Java (#1594, #1586)

JavaScript (#1605, #1589, #1588)

JSON (#1569 -- this is a complete rewrite)

Lean (#1601)

LLVM (#1612)

Mason (#1592)

MySQL (#1555, #1551)

Rust (#1608)

Turtle (#1590, #1553)

Deprecated JsonBareObjectLexer, which is now identical to JsonLexer (#1600)

The ImgFormatter now calculates the exact character width, which fixes some issues with overlapping text (#1213, #1611)

Documentation fixes (#1609, #1599, #1598)

Fixed duplicated Juttle language alias (#1604, #1606)

Added support for Kotlin scripts (#1587)

Removed CSS rule which forced margin to 0

Source code(tar.gz)
Source code(zip)
2.7.2(Oct 24, 2020)
Updated lexers:

Latex (PR#1517, #1516)

LLVM (PR#1565)

SPARQL (PR#1559)

Fixed Python console/traceback lexer problems with custom exceptions without messages (#1548)

Allow loading ttc fonts on Mac/image formatter (#1223)

Improve analyze_text across a variety of lexers (PR#1549)

Removed CSS rule which forced the vertical padding to 0 for line numbers (PR#1583, #1579)

Fix TNTLexer crashing on unexpected EOL (#1568, PR#1570)

regexlint can be now run locally as part of tox tests (PR#1557)

Fix typos (PR#1550, #1562)

Add Python 3.9 as a supported version (PR#1554)

Source code(tar.gz)
Source code(zip)
2.7.1(Sep 26, 2020)
Fixed a regression in the JSON lexer (#1544)

Source code(tar.gz)
Source code(zip)
2.7.0(Sep 12, 2020)
Added lexers:

Arrow (PR#1481, PR#1499)

BARE (PR#1488)

Devicetree (PR#1434)

F* (PR#1409)

GDScript (PR#1457)

Pointless (PR#1494)

PromQL (PR#1506)

PsySH (PR#1438)

Singularity (PR#1285)

TiddlyWiki5 (PR#1390)

TNT (PR#1414)

YANG (PR#1408, PR#1428)

Updated lexers:

APL (PR#1503)

C++ (PR#1350, which also fixes: #1222, #996, #906, #828, #1162, #1166, #1396)

Chapel (PR#1423)

CMake (#1491)

CSound (#1509)

Cython (PR#1507)

Dart (PR#1449)

Fennel (PR#1535)

Fortran (PR#1442)

GAS (PR#1530)

HTTP (PR#1432, #1520, PR#1521)

Inform 6 (PR#1461)

Javascript (PR#1533)

JSON (#1065, PR#1528)

Lean (PR#1415)

Matlab (PR#1399)

Markdown (#1492, PR#1495)

MySQL (#975, #1063, #1453, PR#1527)

NASM (PR#1465)

Nim (PR#1426)

PostgreSQL (PR#1513)

PowerShell (PR#1398, PR#1497)

Protobuf (PR#1505)

Robot (PR#1480)

SQL (PR#1402)

SystemVerilog (PR#1436, PR#1452, PR#1454, PR#1460, PR#1462, PR#1463, PR#1464, PR#1471, #1496, PR#1504)

TeraTerm (PR#1337)

XML (#1502)

Added a new filter for math symbols (PR#1406)

The Kconfig lexer will match Kconfig derivative names now (PR#1458)

Improved HTML formatter output (PR#1500)

.markdown is now recognized as an extension for Markdown files (PR#1476)

Fixed line number colors for Solarized (PR#1477, #1356)

Improvements to exception handling (PR#1478)

Improvements to tests (PR#1532, PR#1533, PR#1539)

Various code cleanups (PR#1536, PR#1537, PR#1538)

Source code(tar.gz)
Source code(zip)
2.6.1(Mar 8, 2020)
Running Pygments on Python 2.x is no longer supported. (The Python 2 lexer still exists.)

Added lexers:

Linux kernel logs (PR#1310)

LLVM MIR (PR#1361)

MiniScript (PR#1397)

Mosel (PR#1287, PR#1326)

Parsing Expression Grammar (PR#1336)

ReasonML (PR#1386)

Ride (PR#1319, PR#1321)

Sieve (PR#1257)

USD (PR#1290)

WebIDL (PR#1309)

Updated lexers:

Apache2 (PR#1378)

Chapel (PR#1357)

CSound (PR#1383)

D (PR#1375, PR#1362)

Idris (PR#1360)

Perl6/Raku lexer (PR#1344)

Python3 (PR#1382, PR#1385)

Rust: Updated lexer to cover more builtins (mostly macros) and miscellaneous new syntax (PR#1320)

SQL: Add temporal support keywords (PR#1402)

The 256-color/true-color terminal formatters now support the italic attribute in styles (PR#1288)

Support HTTP 2/3 header (PR#1308)

Support missing reason in HTTP header (PR#1322)

Boogie/Silver: support line continuations and triggers, move contract keywords to separate category (PR#1299)

GAS: support C-style comments (PR#1291)

Fix names in S lexer (PR#1330, PR#1333)

Fix numeric literals in Ada (PR#1334)

Recognize .mjs files as Javascript (PR#1392)

Recognize .eex files as Elixir (PR#1387)

Fix re.MULTILINE usage (PR#1388)

Recognize pipenv and poetry dependency & lock files (PR#1376)

Improve font search on Windows (#1247)

Remove unused script block (#1401)

Source code(tar.gz)
Source code(zip)
2.5.2(Nov 29, 2019)
Fix incompatibility with some setuptools versions (PR#1316)

Fix lexing of ReST field lists (PR#1279)

Fix lexing of Matlab keywords as field names (PR#1282)

Recognize double-quoted strings in Matlab (PR#1278)

Avoid slow backtracking in Vim lexer (PR#1312)

Fix Scala highlighting of types (PR#1315)

Highlight field lists more consistently in ReST (PR#1279)

Fix highlighting Matlab keywords in field names (PR#1282)

Recognize Matlab double quoted strings (PR#1278)

Add some Terraform keywords

Update Modelica lexer to 3.4

Update Crystal examples

Source code(tar.gz)
Source code(zip)
2.5.1(Nov 26, 2019)

This is a release to fix a packaging-related issue (#1280). No functional differences to 2.5.0.
Source code(tar.gz)
Source code(zip)
2.5.0(Nov 26, 2019)
Added lexers:

Email (PR#1246)

Erlang, Elxir shells (PR#823, #1521)

Notmuch (PR#1264)

Scdoc (PR#1268)

Solidity (#1214)

Zeek (new name for Bro) (PR#1269)

Zig (PR#820)

Updated lexers:

Apache2 Configuration (PR#1251)

Bash sessions (#1253)

CSound (PR#1250)

Dart

Dockerfile

Emacs Lisp

Handlebars (PR#773)

Java (#1101, #987)

Logtalk (PR#1261)

Matlab (PR#1271)

Praat (PR#1277)

Python3 (PR#1255)

Ruby

YAML (#1528)

Velocity

Added styles:

Inkpot (PR#1276)

The PythonLexer class is now an alias for the former Python3Lexer. The old PythonLexer is available as Python2Lexer. Same change has been done for the PythonTracebackLexer. The python3 option for the PythonConsoleLexer is now true by default.

Bump NasmLexer priority over TasmLexer for .asm files (fixes #1326)

Default font in the ImageFormatter has been updated (#928, PR#1245)

Test suite switched to py.test, removed nose dependency (#1490)

Reduce TeraTerm lexer score -- it used to match nearly all languages (#1256)

Treat Skylark/Starlark files as Python files (PR#1259)

Image formatter: actually respect line_number_separator option

Add LICENSE file to wheel builds

Agda: fix lambda highlighting

Dart: support @ annotations

Dockerfile: accept FROM ... AS syntax

Emacs Lisp: add more string functions

GAS: accept registers in directive arguments

Java: make structural punctuation (braces, parens, colon, comma) Punctuation, not Operator (#987)

Java: support var contextual keyword (#1101)

Matlab: Fix recognition of function keyword (PR#1271)

Python: recognize .jy filenames (#976)

Python: recognize f string prefix (#1156)

Ruby: support squiggly heredocs

Shell sessions: recognize Virtualenv prompt (PR#1266)

Velocity: support silent reference syntax

Source code(tar.gz)
Source code(zip)
2.4.2(Nov 10, 2019)
Fix encoding error when guessing lexer with given encoding option (#1438)

Source code(tar.gz)
Source code(zip)
2.4.1(Nov 10, 2019)
Updated lexers:

Coq (#1430)

MSDOS Session (PR#734)

NASM (#1517)

Objective-C (PR#813, #1508)

Prolog (#1511)

TypeScript (#1515)

Support CSS variables in stylesheets (PR#814, #1356)

Fix F# lexer name (PR#709)

Fix TerminalFormatter using bold for bright text (#1480)

Source code(tar.gz)
Source code(zip)
2.4.0(Nov 10, 2019)
Added lexers:

Augeas (PR#807)

BBC Basic (PR#806)

Boa (PR#756)

Charm++ CI (PR#788)

DASM16 (PR#807)

FloScript (PR#750)

FreeFem++ (PR#785)

Hspec (PR#790)

Pony (PR#627)

SGF (PR#780)

Slash (PR#807)

Slurm (PR#760)

Tera Term Language (PR#749)

TOML (PR#807)

Unicon (PR#731)

VBScript (PR#673)

Updated lexers:

Apache2 (PR#766)

Cypher (PR#746)

LLVM (PR#792)

Makefiles (PR#766)

PHP (#1482)

Rust

SQL (PR#672)

Stan (PR#774)

Stata (PR#800)

Terraform (PR#787)

YAML

Add solarized style (PR#708)

Add support for Markdown reference-style links (PR#753)

Add license information to generated HTML/CSS files (#1496)

Change ANSI color names (PR#777)

Fix catastrophic backtracking in the bash lexer (#1494)

Fix documentation failing to build using Sphinx 2.0 (#1501)

Fix incorrect links in the Lisp and R lexer documentation (PR#775)

Fix rare unicode errors on Python 2.7 (PR#798, #1492)

Fix lexers popping from an empty stack (#1506)

TypoScript uses .typoscript now (#1498)

Updated Trove classifiers and pip requirements (PR#799)

Source code(tar.gz)
Source code(zip)
2.3.1(Nov 10, 2019)
Updated lexers:

ASM (PR#784)

Chapel (PR#735)

Clean (PR#621)

CSound (PR#684)

Elm (PR#744)

Fortran (PR#747)

GLSL (PR#740)

Haskell (PR#745)

Hy (PR#754)

Igor Pro (PR#764)

PowerShell (PR#705)

Python (PR#720, #1299, PR#715)

SLexer (PR#680)

YAML (PR#762, PR#724)

Fix invalid string escape sequences

Fix FutureWarning introduced by regex changes in Python 3.7

Source code(tar.gz)
Source code(zip)
2.3.0(Nov 10, 2019)
Added lexers:

Fennel (PR#783)

HLSL (PR#675)

Updated lexers:

Dockerfile (PR#714)

Minimum Python versions changed to 2.7 and 3.5

Added support for Python 3.7 generator changes (PR#772)

Fix incorrect token type in SCSS for single-quote strings (#1322)

Use terminal256 formatter if TERM contains 256 (PR#666)

Fix incorrect handling of GitHub style fences in Markdown (PR#741, #1389)

Fix %a not being highlighted in Python3 strings (PR#727)

Source code(tar.gz)
Source code(zip)