Somewhat related to https://github.com/pradyunsg/sphinx-theme-builder/issues/6: I tried out the workflow of using a conda environment, and installing the exact required version of node in that environment (in the idea that the conda-provided node would be detected as matching "system" node, and thus used by sphinx-theme-builder / nodeenv). But it seems this is not fully working.

docs-theme2 is a conda env in which I installed the exact node version that is required (+ all dependencies to build the demo docs + sphinx-theme-builder):

(docs-theme2) ~/pydata-sphinx-theme$ stb serve docs/
[stb] Invoking sphinx-autobuild to run.
[sphinx-autobuild] pre-build
[sphinx-autobuild] > /home/joris/miniconda3/envs/docs-theme2/bin/python3.9 -m sphinx_theme_builder compile
[stb] # nodeenv does not exist.
[stb] # Generating new nodeenv with NodeJS 14.17.4!
[stb] # Will use system nodeJS, since version matches.
[stb] $ python -m nodeenv --node=system /home/joris/scipy/repos/pydata-sphinx-theme/.nodeenv
[stb] (nodeenv) $ node --version
v10.19.0
error: nodeenv-version-mismatch

× The `nodeenv` for this project is unhealthy.
╰─> There is a mismatch between what is present in the environment (v10.19.0) and the expected version of NodeJS (v14.17.4).

hint: Deleting the .nodeenv directory and trying again may work.

Link: https://sphinx-theme-builder.rtfd.io/errors/#nodeenv-version-mismatch
Command exited with exit code: 3

So it detects that the conda installed node version is matching the required version, but then nodeenv links to the actual system provided one (apparently I have a system wide node with version 10.19.0), and not the conda provided node. Resulting in a "broken" workflow. This might also be an issue to open with nodeenv.

Note: I am not sure how useful it is to actually try this (I am personally happy with the workflow to let nodeenv manage the node installation), but it would be good to know if this could work or will never work, to ensure we can document this for people using conda (and potentially having node installed in their environment) (note: I personally don't want to use this workflow myself (I am happy with the node that nodeenv installs), but some users might want to

Description

On windows, it doesn't seem possible to install themes that have been built with STB.

The installation step installs a new nodejs, but then there is a version mismatch between the present and expected versions.

Here's the error message that is given:

   command: 'C:\hostedtoolcache\windows\Python\3.9.9\x64\python.exe' 'C:\hostedtoolcache\windows\Python\3.9.9\x64\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' build_editable 'C:\Users\RUNNER~1\AppData\Local\Temp\tmpupz5wwwn'
       cwd: D:\a\pydata-sphinx-theme\pydata-sphinx-theme
  Complete output (17 lines):
  [stb] # nodeenv does not exist.
  [stb] # Generating new nodeenv with NodeJS 14.18.1!
  [stb] # The system `node` has a different version: {}
  [stb] $ python -m nodeenv --node=14.18.1
  D:\a\pydata-sphinx-theme\pydata-sphinx-theme\.nodeenv
   * Install prebuilt node (14.18.1) ..... done.
  symbolic link created for D:\a\pydata-sphinx-theme\pydata-sphinx-theme\.nodeenv\Scripts\nodejs.exe <<===>> node.exe
  [stb] (nodeenv) $ node --version
  v16.13.1
  error: nodeenv-version-mismatch
  x The `nodeenv` for this project is unhealthy.
  
  There is a mismatch between what is present in the environment (v16.13.1) and the expected version of NodeJS (v14.18.1).
  
  hint: Deleting the .nodeenv directory and trying again may work.
  
  Link: sphinx-theme-builder.rtfd.io/errors/#nodeenv-version-mismatch

Replication

I opened a PR in the PyData Sphinx Theme to replicate this.

Here's a GitHub Action job that fails with this error upon installing a theme that has been built with the STB: https://github.com/pydata/pydata-sphinx-theme/runs/4655097854?check_suite_focus=true#step:4:224

Here are the action steps needed to produce this error (with the pydata sphinx theme):

https://github.com/pydata/pydata-sphinx-theme/blob/85d7b5bc00cec96efd794bcc6893779cc7584c2b/.github/workflows/tests.yml#L34-L46

Assume that runs-on is windows-latest

Description

When this package installs npm dependencies, it does so with a --no-save flag:

https://github.com/pradyunsg/sphinx-theme-builder/blob/10450cfeb65bc3610a6708ee731069c707978fef/src/sphinx_theme_builder/_internal/nodejs.py#L150

This means that dependencies aren't frozen into a package-lock.json file. As far as I can tell, it's not possible to run this command without the --no-save unless you did it manually with stb npm install.

It would be helpful if this package provided guidance for how authors can create package-lock.json (even if just telling people to do it manually with stb npm install)

I've had a hard time getting the sphinx-2i2c-theme building on ReadTheDocs, unless I manually include node as a part of the build environment. However, I thought that this package should automatically install node as-needed in order to build wheels.

Here's one example of a failed sphinx build:

Building wheels for collected packages: sphinx-2i2c-theme, livereload
  Building wheel for sphinx-2i2c-theme (pyproject.toml): started
  Building wheel for sphinx-2i2c-theme (pyproject.toml): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /home/docs/checkouts/readthedocs.org/user_builds/2i2c-team-compass/envs/304/bin/python /home/docs/checkouts/readthedocs.org/user_builds/2i2c-team-compass/envs/304/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpcyc47z8e
       cwd: /tmp/pip-req-build-jza17p6x
  Complete output (26 lines):
  [stb] # nodeenv does not exist.
  [stb] # Generating new nodeenv with NodeJS 16.13.0!
  Traceback (most recent call last):
    File "/home/docs/checkouts/readthedocs.org/user_builds/2i2c-team-compass/envs/304/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
      main()
    File "/home/docs/checkouts/readthedocs.org/user_builds/2i2c-team-compass/envs/304/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/docs/checkouts/readthedocs.org/user_builds/2i2c-team-compass/envs/304/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/tmp/pip-build-env-frl5r2ot/overlay/lib/python3.8/site-packages/sphinx_theme_builder/__init__.py", line 95, in build_wheel
      return _generate_wheel(
    File "/tmp/pip-build-env-frl5r2ot/overlay/lib/python3.8/site-packages/sphinx_theme_builder/_internal/distributions.py", line 157, in generate_wheel_distribution
      generate_assets(project, production=not editable)
    File "/tmp/pip-build-env-frl5r2ot/overlay/lib/python3.8/site-packages/sphinx_theme_builder/_internal/nodejs.py", line 176, in generate_assets
      create_nodeenv(nodeenv, project.node_version)
    File "/tmp/pip-build-env-frl5r2ot/overlay/lib/python3.8/site-packages/sphinx_theme_builder/_internal/nodejs.py", line 120, in create_nodeenv
      if _should_use_system_node(node_version=node_version):
    File "/tmp/pip-build-env-frl5r2ot/overlay/lib/python3.8/site-packages/sphinx_theme_builder/_internal/nodejs.py", line 99, in _should_use_system_node
      process = subprocess.run(["node", "--version"], capture_output=True)
    File "/home/docs/.asdf/installs/python/3.8.12/lib/python3.8/subprocess.py", line 493, in run
      with Popen(*popenargs, **kwargs) as process:
    File "/home/docs/.asdf/installs/python/3.8.12/lib/python3.8/subprocess.py", line 858, in __init__
      self._execute_child(args, executable, preexec_fn, close_fds,
    File "/home/docs/.asdf/installs/python/3.8.12/lib/python3.8/subprocess.py", line 1704, in _execute_child
      raise child_exception_type(errno_num, err_msg, err_filename)
  FileNotFoundError: [Errno 2] No such file or directory: 'node'
  ----------------------------------------
  ERROR: Failed building wheel for sphinx-2i2c-theme

So it seems like the theme builder is correctly logging that it needs to download node, but it doesn't seem to happen. This error goes away if I configure the ReadTheDocs build to include nodejs.

I think that this is related to this function, which is trying to allow for a non-0 exit code, but somehow this isn't working within ReadTheDocs...

https://github.com/pradyunsg/sphinx-theme-builder/blob/df7f089add6726064f0966741268e803c2d2d8b2/src/sphinx_theme_builder/_internal/nodejs.py#L98-L105

What's happening?

Certain redistributors don't like that they need to provide a specific version of NodeJS, down to the exact patch version, for building Sphinx themes with sphinx-theme-builder.

This restriction exists so that theme authors don't encounter pain due to redistributors who mess things up, ignoring the details of how the package works because "the build is successful in our build system" (which is not the same as "everything about the package works the same as upstream, and is supported by them"). This is especially true for Sphinx themes, since the failure modes are "things look wrong" rather than "the compiler raised an exception" and there's many components involved here -- the JS tooling use to build it, the Python build time dependencies, the Python runtime dependencies (including Sphinx's dependencies, whose versions change the rendered markup) and the interactions between all of them.

Reproducer

1. Ensure you have NodeJS 16.x as the first entry on PATH.
2. Run STB_USE_SYSTEM_NODE=true pip install https://github.com/pradyunsg/furo/archive/refs/tags/2022.06.21.zip
3. See a failure due to a mismatch.

Expectation

The restriction should be, with another flag, relaxed to enforce the major version and a minimum patch version of NodeJS. This has a low-enough risk of meddling horribly with the JS-based build, that it should not result in additional need for the theme authors to deal with difficult-to-diagnose issues because a redistributor broke their package.

Code of Conduct

• [X] I agree to follow the Code of Conduct.

This adds a short topic on asset compilation (in a new folder called topic/) and adds a few points of information that I recently had to figure out the hard way. I think that this topic is common-enough that it's worth documenting as a dedicated page.

I did this because it felt like reference/ wasn't quite the right place for this (assuming you're defining reference similar to how diataxis defines reference material. But happy to move it to a different place if there's a better spot for it

My current understanding is that:

1. It will first check if the "system" node (I suppose what is available if you run node --version in a console on your system) has the exact same version as the required version (node-version in pyproject.toml), and if those match, use the system node.
2. If those version don't match, nodeenv will install the required node version

Is that correct?

But in both cases nodeenv will still create a .nodeenv directory? (only in the first case it seems that the node executable in there is then basically a symlink to the system one. I suppose this is logic that lives in the nodeenv package)

What's happening?

When running stb new blank-theme, the package will try to download https://github.com/pradyunsg/sphinx-theme-template/archive/refs/heads/sphinx-theme-template-stable.zip to make a template. However, this file doesn't exist.

Reproducer

1. pip install from head, e.g. pip install git+https://github.com/pradyunsg/sphinx-theme-builder.git
2. Run stb new blank-theme in the repository root.
3. Error:

Zip repository https://github.com/pradyunsg/sphinx-theme-template/archive/refs/heads/sphinx-theme-template-stable.zip is not a valid zip archive:
error: cookiecutter-failed

× Cookiecutter failed to generate the project.
╰─> Exit code: 1

Expectation

The file should exist or the link should be corrected.

Code of Conduct

• [X] I agree to follow the Code of Conduct.

Discussed in https://github.com/pradyunsg/sphinx-theme-builder/discussions/28

This adds a short warning if package-lock.json doesn't exist. My rationale for adding it where I did is the following:

• When you install node for the first time, this is the only time that npm install is run.
• If you remove --no-save here, then whenever a person installs via the theme builder for the first time, it'd overwrite package-lock.json which will be confusing for reasons described in https://github.com/pradyunsg/sphinx-theme-builder/issues/14
• So, emit a warning when you compile with stb that tells the user what command to run to create this file.
• That way, the act of creating or updating that file will always be an intentional step.

closes #14

Checking the new wheel for pydata-sphinx-theme 0.8.0 (https://pypi.org/project/pydata-sphinx-theme/0.8.0/#files), I noticed that it still includes the pydata_sphinx_theme/assets/ scripts and styles (the original source), in addition to the compiled files located at pydata_sphinx_theme/theme/pydata_sphinx_theme/static/.

Just wondering if that is intentional?

This would help ensure that the Error Index page stays up to date with the errors declared in the codebase.

• Check that all errors are listed in there.
• Check that outdated ones are marked as such.
• Check that all of them mention user-actionable steps.

This would basically be something that helps theme authors provide + maintain translations for their theme.

It will be a combination of documentation and some build-time logic for managing + validating the translations, as part of generating the distributions.

Context

Currently, the extra-compiled-assets configuration assumes that compiled assets are placed somewhere in the static folder of the theme. These are then bundled with any documentation that uses this theme.

However, there are some cases where you compile assets that are not meant to be bundled with the theme.

An example of this is HTML partials that are meant for use in Jinja templates, that define hashes for assets. Currently, these snippets of non-valid HTML (they contain pathto( calls and such) are still bundled with the theme in the _static folder. This isn't a huge problem, but it does mean that things like link auto-checkers will complain about incorrect links (because it discovers the items in the _static folder).

Another example of this is translation machinery (e.g. the .mo and .po files that you may want to generate programmatically, so that the human-editable translation files are in a more readable location (but not bundled with the theme)

What I tried

I tried adding a relative path to extra-compiled-assets (e.g. ../partials/mypartial.html). But at least for stb serve this doesn't work because the regex doesn't match relative paths. As a result, the function loops endlessly because it keeps detecting a change that is outside of the static folder.

The project.compiled_assets looks like this on the book theme for example:

$ project.compiled_assets

('src/sphinx_book_theme/theme/sphinx_book_theme/static/scripts/sphinx-book-theme.js',
 'src/sphinx_book_theme/theme/sphinx_book_theme/static/styles/sphinx-book-theme.css',
 'src/sphinx_book_theme/theme/sphinx_book_theme/static/../partials/sbt-webpack-macros.html',
 'src/sphinx_book_theme/theme/sphinx_book_theme/static/scripts/sphinx-book-theme.js.map',
 'src/sphinx_book_theme/theme/sphinx_book_theme/static/styles/sphinx-book-theme.css.map',
 'src/sphinx_book_theme/theme/sphinx_book_theme/static/../partials/sbt-webpack-macros.html.map')

Potential fix

I noticed that the compiled_assets property is documented as relative to Project root, but that the code itself keeps these assets relative to the static directory:

https://github.com/pradyunsg/sphinx-theme-builder/blob/df7f089add6726064f0966741268e803c2d2d8b2/src/sphinx_theme_builder/_internal/project.py#L374-L386

If those files were stored relative to the project root (which I guess would be src/<themename>?, then we wouldn't run into problems if compiled files did not exist in the static directory.

• [x] #22
  • [x] Behaviour reference
  • [x] Error reference
• [ ] stb new template
• [ ] CI setup
• [ ] ~100% test coverage on all supported platforms (3 OS x 5 Python)