Warning about footnotes when building the Spanish version of the guide

[flpm]

After merging PR 342, there is now a warning when building the Spanish version of the guide:

/home/felipe/github/pyOpenSci/python-package-guide/tutorials/add-license-coc.md:30: WARNING: inconsistent footnote references in translated message. original: ['[^mit]', '[^bsd3]'], translated: []
/home/felipe/github/pyOpenSci/python-package-guide/tutorials/add-license-coc.md:30: WARNING: inconsistent footnote references in translated message. original: ['[^mit]', '[^bsd3]'], translated: []

Here is the entire nox log:

❯ nox -s docs-test -- -D language=es
nox > Running session docs-test
nox > Re-using existing virtual environment at .nox/docs-test.
nox > python -m pip install -e .
nox > sphinx-build -b html -W --keep-going -E -a . _build/html -D language=es
Running Sphinx v7.3.7
loading translations [es]... done
myst v3.0.1: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions={'deflist', 'attrs_block', 'colon_fence'}, disable_syntax=[], all_links_external=False, links_external_new_tab=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, fence_as_directive=set(), number_code_blocks=[], title_to_header=False, heading_anchors=3, heading_slug_func=None, html_meta={}, footnote_transition=False, words_per_minute=200, substitutions={}, linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', enable_checkboxes=False, suppress_warnings=[], highlight_code_blocks=True)
myst-nb v1.1.1: NbParserConfig(custom_formats={}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, eval_name_regex='^[a-zA-Z_][a-zA-Z0-9_]*$', execution_mode='auto', execution_cache_path='', execution_excludepatterns=(), execution_timeout=30, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_streams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, code_prompt_show='Show code cell {type}', code_prompt_hide='Hide code cell {type}', number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_markdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False)
Using jupyter-cache at: /home/felipe/github/pyOpenSci/python-package-guide/_build/.jupyter_cache
The default value for `navigation_with_keys` will change to `False` in the next release. If you wish to preserve the old behavior for your site, set `navigation_with_keys=True` in the `html_theme_options` dict in your `conf.py` file. Be aware that `navigation_with_keys = True` has negative accessibility implications: https://github.com/pydata/pydata-sphinx-theme/issues/1492
building [mo]: all of 7 po files
WARNING: Start of line didn't match any expected keyword.tion.mo
WARNING: Problem on line 98: 'flexibilidad."'
writing output... [100%] locales/es/LC_MESSAGES/package-structure-code.mo
building [html]: all source files
updating environment: [new config] 44 added, 0 changed, 0 removed
reading sources... [100%] tutorials/setup-py-to-pyproject-toml
/home/felipe/github/pyOpenSci/python-package-guide/tutorials/add-license-coc.md:30: WARNING: inconsistent footnote references in translated message. original: ['[^mit]', '[^bsd3]'], translated: []
/home/felipe/github/pyOpenSci/python-package-guide/tutorials/add-license-coc.md:30: WARNING: inconsistent footnote references in translated message. original: ['[^mit]', '[^bsd3]'], translated: []
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
copying assets... copying static files... done
copying extra files... done
done
writing output... [100%] tutorials/setup-py-to-pyproject-toml
generating indices... genindex done
writing additional pages... search done
copying images... [100%] images/tutorials/test-pypi-package.png
dumping search index in Spanish (code: es)... done
dumping object inventory... done
sphinx-sitemap: sitemap.xml was generated for URL https://www.pyopensci.org/python-package-guide/ in /home/felipe/github/pyOpenSci/python-package-guide/_build/html/sitemap.xml
build finished with problems, 2 warnings.
nox > Command sphinx-build -b html -W --keep-going -E -a . _build/html -D language=es failed with exit code 1
nox > Session docs-test failed.

The original file (/tutorials/add-license-coc.md) does have the footnotes defined at the bottom (line 199):

## Footnotes

[^mit]: https://opensource.org/license/mit/
[^bsd3]: https://opensource.org/license/bsd-3-clause/

This will need to be solved before the Spanish translation can be release since the build in CI will fail.

[lwasser]

I am not sure if this is helpful @flpm, but I did notice -- below is the English page in .md format. (Excuse me if you already noticed this!).

## Footnotes

[^mit]: https://opensource.org/license/mit/
[^bsd3]: https://opensource.org/license/bsd-3-clause/

The translation files look like this

#: ../../tutorials/add-license-coc.md:201
msgid "https://opensource.org/license/mit/" <- NOTE that [^bsd3] is not a part of the line
msgstr ""

#: ../../tutorials/add-license-coc.md:202
msgid "https://opensource.org/license/bsd-3-clause/"
msgstr ""

i tried this to ensure the footnote is included in the spanish version and it didn't work.

msgid "https://opensource.org/license/mit/"
msgstr "[^bsd3]: https://opensource.org/license/bsd-3-clause/"

#: ../../tutorials/add-license-coc.md:202
msgid "https://opensource.org/license/bsd-3-clause/"
msgstr "[^bsd3]: https://opensource.org/license/bsd-3-clause/"

i just wonder if there is a bug in the sphinx translation extension or if we could ask over there about this issue? we can also ask other folks who have worked on translation with sphinx -- like maybe @melissawm . i'm just taking a wild guess.

[melissawm]

Hi all! Thanks for the ping ❤️ Unfortunately I am not that familiar with the sphinx extension and can't reproduce this failure locally - nox -s docs-test -- -D language=es runs fine for me. Wild guess but I think @OriolAbril may be able to help?

[OriolAbril]

I think this is due to this issue in the myst-parser: executablebooks/MyST-Parser#690. There seems to be a workaround available in the issue itself but it hasn't been "upgraded" to a PR.

I think maintainers of myst-parser/myst-nb are stretched quite thin and can't work much (or at all) on internationalization stuff. I do use sphinx for translations from time to time but there are several rough edges still and I don't really have the knowledge nor availability for in depth testing and fixing. I am interested on getting transaltions with sphinx+myst to work smoothly though so it would be gretlat to get some coordination going.

I also have executablebooks/MyST-NB#600 which similarly to the issue above is waiting for maintainer availability.

[tkoyama010]

@all-contributors
please add @melissawm for question.
please add @OriolAbril for question.

[allcontributors]

@tkoyama010

I've put up a pull request to add @melissawm! 🎉

I've put up a pull request to add @OriolAbril! 🎉

[flpm]

After @melissawm 's comment, I tried again and starting from a clean slate, it works without warnings on the main branch. I can't understand why, but I will investigate, but it does not seem to be a block for release at this point.

Thank you so much @melissawm and @OriolAbril for taking a look!

[lwasser]

@melissawm @OriolAbril - thank you both so much for the help here! It seems like something has been magically fixed. A few of us now don't see the issue. Maybe there was a patch release somewhere? Regardless thank you all for the thought and effort 👐🏻 and thank you @tkoyama010 for being on top of all contributor additions!!

[flpm]

I did some more research on this. On my computer, it works using the virtual env in the build-language session but not in the virtual env for the docs-test session. And there are some version differences:

❯ diff docs-test-env.txt build-lang-env.txt
6c6
< sphinx_design==0.6.0
---
> sphinx_design==0.6.1
8,10c8,10
< sphinxcontrib-applehelp==1.0.8
< sphinxcontrib-devhelp==1.0.6
< sphinxcontrib-htmlhelp==2.0.5
---
> sphinxcontrib-applehelp==2.0.0
> sphinxcontrib-devhelp==2.0.0
> sphinxcontrib-htmlhelp==2.1.0
12,13c12,13
< sphinxcontrib-qthelp==1.0.7
< sphinxcontrib-serializinghtml==1.1.10
---
> sphinxcontrib-qthelp==2.0.0
> sphinxcontrib-serializinghtml==2.0.0

The virtual env for build-language is newer, because it is a new session we created recently. So it looks like the problem might have solved itself with an update in the sphinx modules, but it persist in some places because the cached virtual env.

Since we do not fix version in pyproject.toml, we should look into a way to tell nox to check for latest versions every time it runs a session, or we risk having these unexpected version differences across nox sessions in the same machine.

[flpm]

🙌 @ucodery found an issue with our nox config that explains this behavior!

[sneakers-the-rat]

Do we wanna use pdm to make/use a lockfile as well?

[flpm]

@ucodery helped figure out what was happening, from a discussion in Slack:

AH! I see the problem

python-package-guide/noxfile.py

Line 6 in cdbdf85

nox.options.reuse_existing_virtualenvs = True

Will close this issue now and create another one to discuss removing this parameter.

[flpm]

Solved.