~~This is not quite the finished product yet, but it outlines the basic functionality of what I'm looking to implement~~ now basically complete

closes #13

This provides a hook for passing variables to the parsing/render process, which in-turn provides a simple way of allowing plugins to accept variable arguments from the API or CLI.

~~Note this is back-incompatible for parser plugins, since update_mdit now also gets passed this env dict.~~

Context

I have a repo that uses mdformat as a pre-commit hook; it works beautifully on Linux and macOS.

If I clone the repo on Windows, under Git's default settings (core.autocrlf = true), the Markdown files in the working tree have CRLF newlines. When the pre-commit hook runs, it replaces newlines with LF (as expected in the absence of --end-of-line). This leads to the confusing situation where:

• pre-commit reports no errors, yet
• after the commit, there remain unstaged changes (consisting of CRLF -> LF replacement).

Specifying --end-of-line=crlf can fix this, but we would need to specify this on Windows only, and there is no way to do so in .pre-commit-config.yaml or .mdformat.toml, as far as I can tell.

It is probably also possible to work around this using .gitattributes, but that adds more things to maintain.

Proposal

• Add the choice native, in addition to the current choices (lf, crlf), to the --end-of-line option.
  • And similarly to the end_of_line key of .mdformat.toml.
• When --end-of-line=native is given, use the value of os.linesep to determine the newlines to use.
• The default newline remains lf to maintain compatibility.

The choice of the name native is bikesheddable (perhaps auto?).

Tasks and updates

• [x] Settle on a name (native, auto, or other)
• [ ] Implement

I use a Markdown linter with pre-commit and GitLab CI pipelines. It's useful but I'd like to use a formatter before running the linter: one of the biggest problems I have is that, as well as raising rule violations that require manual correction, the linter frequently raises large numbers of trivial formatting rule violations that could very easily be corrected automatically by a formatter. If they are going to be coupled, it's really important that the formatter and linter work together and don't conflict. I would like to suggest that mdformat be designed from the beginning to integrate with Markdown linters, particularly markdownlint [1].

I expect that this means mdformat will have to observe the markdownlint rules and the rule structure, to avoid creating linting conflicts. The advantage of this is that there's an existing structure and code (largely regexps) which can inform mdformat development.

What do you think?

[1] There are two Markdown linters I'm aware of: a Ruby version https://github.com/markdownlint/markdownlint and a node.js version: https://github.com/DavidAnson/markdownlint, both named "markdownlint" and the respective authors and maintainers work together to ensure that there is minimal divergence of rules between the two versions.

Describe the problem/need and solution

Problem

Right now, mdformat breaks indentation for admonitions. Below is a repeatable example based on the first snippet from the admonition documentation

~/Developer > pipx install mdformat-gfm --include-deps
  installed package mdformat-gfm 0.3.5, installed using Python 3.10.2
  These apps are now globally available
    - markdown-it
    - mdformat
done! ✨ 🌟 ✨
~/Developer > pbpaste > ad_test.md
~/Developer > bat ad_test.md
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: ad_test.md
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ !!! type "optional explicit title within double quotes"
   2   │     Any number of other indented markdown elements.
   3   │
   4   │     This is the second paragraph.
───────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
~/Developer > mdformat ad_test.md
~/Developer > bat ad_test.md
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: ad_test.md
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ !!! type "optional explicit title within double quotes"
   2   │ Any number of other indented markdown elements.
   3   │
   4   │ ```
   5   │ This is the second paragraph.
   6   │ ```
───────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Solution

admonitions are not part of the CommonMark specification and would be best handled by a plugin (i.e. mdformat-admonition)

I would be happy to try writing one, but I wanted to open an issue first so that I'm not duplicating effort. I saw that there was at least one other issue with admonition problems, but there didn't seem to be any updates since: #53 (https://github.com/executablebooks/mdformat/issues/53#issuecomment-928306196)

• I naively think the plugin shouldn't be too difficult to implement
  • I found the plugin template: https://github.com/executablebooks/mdformat-plugin
    • and the mdformat-tables, mdformat-frontmatter, mdformat-footnote, and mdformat source will be useful to reference
  • Update: As a reference, here is a plugin that was recently ported with a good implementation

Benefit

Would be nice to have support for admonitions (particular for mkdocs which uses the markdown extension)

Guide for implementation

No response

Tasks and updates

No response

Hmm, this is interesting. Running mdformat on:

# mdformat-tables

[![Build Status][ci-badge]][ci-link]
[![codecov.io][cov-badge]][cov-link]
[![PyPI version][pypi-badge]][pypi-link]

An [mdformat](https://github.com/executablebooks/mdformat) plugin for rendering tables.

[ci-badge]: https://github.com/executablebooks/mdformat-tables/workflows/CI/badge.svg?branch=master
[ci-link]: https://github.com/executablebooks/mdformat/actions?query=workflow%3ACI+branch%3Amaster+event%3Apush
[cov-badge]: https://codecov.io/gh/executablebooks/mdformat-tables/branch/master/graph/badge.svg
[cov-link]: https://codecov.io/gh/executablebooks/mdformat-tables
[pypi-badge]: https://img.shields.io/pypi/v/mdformat-tables.svg
[pypi-link]: https://pypi.org/project/mdformat-tables

goes to

# mdformat-tables

[![Build Status](https://github.com/executablebooks/mdformat-tables/workflows/CI/badge.svg?branch=master)](<https://github.com/executablebooks/mdformat/actions?query=workflow%3ACI+branch%3Amaster+event%3Apush>)
[![codecov.io](https://codecov.io/gh/executablebooks/mdformat-tables/branch/master/graph/badge.svg)](<https://codecov.io/gh/executablebooks/mdformat-tables>)
[![PyPI version](https://img.shields.io/pypi/v/mdformat-tables.svg)](<https://pypi.org/project/mdformat-tables>)

An [mdformat](<https://github.com/executablebooks/mdformat>) plugin for rendering tables.

I'd say that's not ideal behaviour 😬

If not in "core", I think we at least want a plugin that can "inhibit" this type of behaviour.

Describe the problem

I have a README.md which has something like this:

text
<br/>
text

For some reason the new release of mdformat now wants to add 4 spaces infront of the <br/>. This breaks all my build pipelines. Since I see no reason why there must be 4 spaces I would say this is a bug.

Could you please patch this and make a patch release?

Many thanks Philip

Link to your repository or website

https://github.com/telekom/HPOflow/blob/main/README.md

Steps to reproduce

see above

The version of Python you're using

3.9

Your operating system

Linux

Versions of your packages

No response

Additional context

No response

Utilise the new store_labels option I introduced in mardown-it-py, to keep link/image reference labels, instead of their resolved href. Then output the (sorted) references at the end of the document.

Notes:

• I've left a "hook" to turn this feature on/off: markdown_it.options["mdformat"] = {"keep_references": True}, this could be turned into a CLI option, if desired.
• References are always normalized using str.upper() in markdown-it, but, as a style choice, I convert them to str.lower()
• I've added a finalize key-word argument to MDRenderer.render. This is for use by plugins (like mdformat-tables) that need to run "nested rendering", during which they do not want these references output.

Describe the bug

context running mdformat --check --end-of-line=crlf on a correctly formated markdonw file result in error.

expectation I expected mdformat to succede and nothing to happen

bug But instead an error is rased that the file is not formated:

$ mdformat --check --end-of-line=crlf test.md
Error: File "D:\Downloads\temp\mdformat\test.md" is not formatted.

related code

I belive the problem thems from here:

https://github.com/executablebooks/mdformat/blob/24f9d25960d28efc75cbfa051881bac5fc46a842/src/mdformat/_cli.py#L85-L87

debugging reveald the content of formatted_str and original_str to have different lineendings:

original_str: '# Test file\r\n\r\nHello world\r\n'
formatted_str: '# Test file\n\nHello world\n'

I assume that the formating process internally does a universal newline conversion, resulting in all newline types beeing converted to \n

Reproduce the bug

1. create a well formated mardkown file test.md with dos-style line endings (\r\n):

   # Test file
   
   Hello world
   
   

2. run mdformat on it with the --check and --end-of-line=crlf options:

   mdformat --check --end-of-line=crlf test.md
   

3. get error

List your environment

OS: windows 10 Python: v3.10.6 mdformat: 0.7.15

Describe the bug

- [KiCAD](KiCAD)
- [KiCAD - Scripts](KiCAD - Scripts)
- [KiCAD - CNC](KiCAD - CNC)

is converted to

- [KiCAD](KiCAD)
- \[KiCAD - Scripts\](KiCAD - Scripts)
- \[KiCAD - CNC\](KiCAD - CNC)

Rendering the links useless.

There are links that are properly resolved in gitlab.

expectation The link is maintained.

problem This is a problem because this breaks the expected behaviour after applying mdformat.

Reproduce the bug

Apply mdformat on

- [KiCAD](KiCAD)
- [KiCAD - Scripts](KiCAD - Scripts)
- [KiCAD - CNC](KiCAD - CNC)

List your environment

I suggest you update the hint to get the version (jupyter-hook does not exist).

$ mdformat --version
mdformat 0.7.13 (mdformat_frontmatter: 0.4.1, mdformat_gfm: 0.3.5,
mdformat_tables: 0.4.1)
$ python --version
Python 3.9.10

pre-commit configuration:

  - repo: https://github.com/executablebooks/mdformat
    # Do this before other tools "fixing" the line endings
    rev: 0.7.13
    hooks:
      - id: mdformat
        name: Format Markdown
        entry: mdformat  # Executable to run, with fixed options
        language: python
        types: [markdown]
        args: [--wrap, '75', --number]
        additional_dependencies:
          - mdformat-toc

Description / Summary

mdformat supports three configuration options via switches to the command line interface:

  --number              apply consecutive numbering to ordered lists
  --wrap {keep,no,INTEGER}
                        paragraph word wrap mode (default: keep)
  --end-of-line {lf,crlf}
                        output file line ending mode (default: lf)

However, there does not seem to be support for configuring these options from a file that could be committed to source control.

Value / benefit

A configuration file would assist in ensuring consistent behavior between executing manually in a terminal, an editor integration, and when running as a pre-commit hook.

Implementation details

I find yaml files to be excellent for implementing configurations that are more often parsed by humans than by programs. But I do not have a strong opinion about this. toml is also an option that is gaining traction in the python ecosystem.

To discover the configuration file, I suggest this logic to follow the xdg base directory spec:

1. Check ./.mdformat-config.yaml
2. If not found, check $XDG_CONFIG_HOME/mdformat-config.yaml
3. If not found, check $HOME/mdformat-config.yaml
4. If not found, check $HOME/.mdformat-config.yaml

If command-line switches are passed, they should over-ride conflicting options in the configuration file.

Tasks to complete

• [ ] Decide on a configuration file format.
• [ ] Decide on a discovery path for the configuration file.
• [ ] Code it!

Context

Currently, mdformat plugins are allowed to implement a add_cli_options function to add arguments to the mdformat ArgumentParser, but there is no way to allow those options to be added to the .mdformat.toml file without monkey patching the mdformat._conf module to add the new options to the DEFAULT_OPTS dictionary or _validate_keys and _validate_values functions.

Proposal

Add a way for plugins to declare acceptable options and values for the mdformat.toml file.

Along the way, maybe also slightly reformat how the options are declared, so that they are not hard-coded in two different places (_conf and _cli) independently. An option for this would be to have them declared only inside _conf, with a way for plugins to alter them, and then have both the _cli.ArgumentParser and the _conf validation functions read and use them.

Tasks and updates

No response

Context

I am using mdformat but having troubles with the current thematic break rendering composed of 70 underscore characters, which I am rather used to typing as three dashes ---.

My (very personal and opinionated) reasons being:

1. The default vim highlighting for Markdown syntax breaks when multiple underscores are put together, because it conflicts with the italics format. Notice that this can probably be solved with Vim plugins or configuration tweaks, but the origin of the problem is about raw vim with no plugins.
2. Typing --- while writing a document is quick and easy, while typing a line of exactly 70 underscores is comparatively much harder and needs some sort of copy/paste action. My usual coding workflow involves code style checkers like mdformat and black as pre-commit hooks, but in general I try to type in code and documentation that is already valid to minimize the number of times that these tools have to modify what I have typed, which forces me to redo the review, git add and pre-commit loop.

I admit that these are very personal reasons, so I do not wish to engange in any discussions about the current rendering style and the rationale behind it.

Instead, I thought best to take the route of coding a new plugin which overrides the rendering style for thematic breaks with three dashes, so anyone who wants can take advantage of it without having to do any change in the main mdformat project: https://github.com/csala/mdformat-simple-breaks

Proposal

The proposal is just to add the plugin to the corresponding list in the documentation I'm happy to make a PR for it.

Tasks and updates

• [ ] Discuss and decide if a PR is wanted
• [ ] Do the corresponding PR

Describe the bug

context When I run MdFormat on this notebook through nbqa…

expectation I expected to get nested lists indented a certain number of spaces.

bug But instead some are indented two spaces, while some are indented three.

diff --git hw_0.ipynb hw_0.ipynb
index 4aaa010..ef99e82 100644
--- hw_0.ipynb
+++ hw_0.ipynb
@@ -11,16 +11,16 @@
     "Kaggle, like JupyterHub and Google Colab, is built around Jupyter notebooks. They are not _exactly_ the same, but will feel similar. For this homework
:\n",
     "\n",
     "1. Do following Kaggle [Learn Python](https://www.kaggle.com/learn/python) tutorials and exercises. You do not need to turn these in.\n",
-    "    - [Hello, Python](https://www.kaggle.com/colinmorris/hello-python)\n",
-    "    - [Booleans and Conditionals](https://www.kaggle.com/colinmorris/booleans-and-conditionals)\n",
-    "    - [Strings and Dictionaries](https://www.kaggle.com/colinmorris/strings-and-dictionaries)\n",
+    "   - [Hello, Python](https://www.kaggle.com/colinmorris/hello-python)\n",
+    "   - [Booleans and Conditionals](https://www.kaggle.com/colinmorris/booleans-and-conditionals)\n",
+    "   - [Strings and Dictionaries](https://www.kaggle.com/colinmorris/strings-and-dictionaries)\n",
     "1. Read about:\n",
-    "    - [Tuples, lists, and dictionaries](http://sthurlow.com/python/lesson06/)\n",
-    "    - [Errors and exceptions](https://swcarpentry.github.io/python-novice-inflammation/09-errors/index.html)\n",
+    "   - [Tuples, lists, and dictionaries](http://sthurlow.com/python/lesson06/)\n",
+    "   - [Errors and exceptions](https://swcarpentry.github.io/python-novice-inflammation/09-errors/index.html)\n",
     "1. Do the coding challenges below.\n",
     "1. Do the following Kaggle tutorials and exercises. You do not need to turn these in.\n",
-    "    - [Python Lists](https://www.kaggle.com/colinmorris/lists)\n",
-    "    - [Creating, Reading, and Writing with Pandas](https://www.kaggle.com/residentmario/creating-reading-and-writing)\n",
+    "   - [Python Lists](https://www.kaggle.com/colinmorris/lists)\n",
+    "   - [Creating, Reading, and Writing with Pandas](https://www.kaggle.com/residentmario/creating-reading-and-writing)\n",
     "1. Post a question, per the instructions at the bottom.\n",
     "1. [Turn in this notebook.](https://python-public-policy.afeld.me/en/latest/README.html#turning-in-assignments)\n",
     "\n",
@@ -163,8 +163,8 @@
     "We will now make an interactive tool to assess people for unemployment benefits. We will use a simplified version of [the rules from the New York Stat
e Department of Labor](https://www.labor.ny.gov/formsdocs/ui/TC318.3e.pdf#page=18), as follows. To be elegible, one must:\n",
     "\n",
     "- Have lost employment\n",
-    "- Have made at least $10,000 in the past year\n",
-    "    - Ask for their salary rather than as a yes-or-no question\n",
+    "- Have made at least \\$10,000 in the past year\n",
+    "  - Ask for their salary rather than as a yes-or-no question\n",
     "- Be ready, willing and able to work immediately\n",
     "- Be actively seeking work\n",
     "\n",

problem This is a problem because I'm publishing that notebook via JupyterBook, and it's Markdown parser only seems to support nested lists indented 3+ spaces.

cc https://github.com/executablebooks/mdformat/issues/331#issuecomment-1224279288 for the option to make that configurable for more fragile Markdown parsers

Reproduce the bug

1. Download this notebook

2. Run Mdformat through nbqa:

   nbqa mdformat hw_0.ipynb --nbqa-md
   

List your environment

$ jupyter-book --version
Jupyter Book      : 0.13.1
External ToC      : 0.2.4
MyST-Parser       : 0.15.2
MyST-NB           : 0.13.2
Sphinx Book Theme : 0.3.3
Jupyter-Cache     : 0.4.3
NbClient          : 0.5.13
$ nbqa --version
nbqa 1.5.3
$ mdformat --version
mdformat 0.7.16 (mdformat_myst: 0.1.5, mdformat_tables: 0.4.1, mdformat_frontmatter: 0.4.1)

Context

The plugins documentation shows how to pass plugins to mdformat.text, but not how to do this using the Renderer API.

At the moment I have rendered = MDRenderer().render(tokens, md.options, env) which works fine for plain markdown, but not if I have front_matter tokens which then gives the error below.

It would be good if the documentation made it clearer if this can be done using mdformat.renderer directly.

Traceback (most recent call last):
  File "/Users/g/gbin/mdh", line 25, in <module>
    run(fle, findtext, replacetext)
  File "/Users/g/gbin/markdown_helper/fix_image_paths.py", line 31, in run
    fix_image_paths(text, findtext, replacetext)
  File "/Users/g/gbin/markdown_helper/fix_image_paths.py", line 23, in fix_image_paths
    rendered = MDRenderer().render(tokens, md.options, env)
  File "/opt/homebrew/lib/python3.10/site-packages/mdformat/renderer/__init__.py", line 57, in render
    return self.render_tree(tree, options, env, finalize=finalize)
  File "/opt/homebrew/lib/python3.10/site-packages/mdformat/renderer/__init__.py", line 91, in render_tree
    text = tree.render(render_context)
  File "/opt/homebrew/lib/python3.10/site-packages/mdformat/renderer/_tree.py", line 11, in render
    text = renderer(self, context)
  File "/opt/homebrew/lib/python3.10/site-packages/mdformat/renderer/_context.py", line 50, in render_children
    return separator.join(out for out in render_outputs if out)
  File "/opt/homebrew/lib/python3.10/site-packages/mdformat/renderer/_context.py", line 50, in <genexpr>
    return separator.join(out for out in render_outputs if out)
  File "/opt/homebrew/lib/python3.10/site-packages/mdformat/renderer/_context.py", line 49, in <genexpr>
    render_outputs = (child.render(context) for child in node.children)
  File "/opt/homebrew/lib/python3.10/site-packages/mdformat/renderer/_tree.py", line 10, in render
    renderer = context.renderers[self.type]
KeyError: 'front_matter'

Proposal

• Extend the docs with an example how to use plugins with Renderer if this is possible
• I am happy to write the docs and more examples covering frontmatter, tables, etc if someone can show me a basic example of how use Renderer() with a plugin.

Tasks and updates

No response

Context

markdownlint has rules for this:

• MD049
• MD050

They seem like good rules and mdformat can do this.

Proposal

Pick a style for emphasis and strength, and have mdformat swap the other to the chosen format.

I don't have a strong opinion about which should win, but if it's a coin-toss then I guess _Italic_ and **Strong** would be easiest for humans to differentiate?

Tasks and updates

No response