0.13.2: sphinx fails on generate man page

[kloczek]

Describe the bug

context
I'm trying to package your module as the rpm package.
Part of that process is building sphinx based documentation as man page (ad on all unixes man page is still most basic forrm of the documentation).

expectation
Should be possible to generate MyST-NB documentation as the man page.

bug
sphinx fails with:

[tkloczko@devel-g2v MyST-NB-0.13.2]$ tox -e docs-update man
GLOB sdist-make: /home/tkloczko/rpmbuild/BUILD/MyST-NB-0.13.2/setup.py
docs-update inst-nodeps: /home/tkloczko/rpmbuild/BUILD/MyST-NB-0.13.2/.tox/.tmp/package/1/myst-nb-0.13.2.zip
docs-update installed: backcall==0.2.0,decorator==5.1.1,fastjsonschema==2.15.3,ipython==7.11.0,ipython-genutils==0.2.0,jedi==0.18.1,mistune==0.8.4,mpmath==1.2.1,myst-nb @ file:///home/tkloczko/rpmbuild/BUILD/MyST-NB-0.13.2/.tox/.tmp/package/1/myst-nb-0.13.2.zip,parso==0.8.3,pexpect==4.8.0,pickleshare==0.7.5,prompt-toolkit==3.0.29,ptyprocess==0.7.0,Pygments==2.11.2,pytz==2022.1,Send2Trash==1.8.0,snowballstemmer==2.2.0,traitlets==5.1.1,wcwidth==0.2.5,webencodings==0.5.1
docs-update run-test-pre: PYTHONHASHSEED='2243809959'
docs-update run-test: commands[0] | sphinx-build -nW --keep-going -b man docs/ docs/_build/man
WARNING: test command found but not installed in testenv
  cmd: /usr/bin/sphinx-build
  env: /home/tkloczko/rpmbuild/BUILD/MyST-NB-0.13.2/.tox/docs-update
Maybe you forgot to specify a dependency? See also the allowlist_externals envconfig setting.

DEPRECATION WARNING: this will be an error in tox 4 and above!
Running Sphinx v4.5.0
making output directory... done
CoconutWarning: Failed to update Coconut Jupyter kernel installation; the 'coconut' kernel might not work properly as a result (try again with 'sudo').
Coconut: Installing Jupyter kernels 'coconut_py', 'coconut_py2', 'coconut_py3'...
Coconut: Successfully installed Jupyter kernels: 'coconut_py', 'coconut_py2', 'coconut_py3'
myst v0.17.1: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=['amsmath', 'colon_fence', 'deflist', 'dollarmath', 'html_image'], linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', disable_syntax=[], all_links_external=False, url_schemes=['http', 'https', 'mailto', 'ftp'], ref_domains=None, highlight_code_blocks=True, number_code_blocks=[], title_to_header=False, heading_anchors=None, heading_slug_func=None, html_meta=[], footnote_transition=True, substitutions=[], sub_delimiters=['{', '}'], words_per_minute=200)

MyST NB Configuration Error:
`nb_render_priority` not set for builder: man
ERROR: InvocationError for command /usr/bin/sphinx-build -nW --keep-going -b man docs/ docs/_build/man (exited with code 2)
docs-update run-test-post: commands[0] | echo 'open file:///home/tkloczko/rpmbuild/BUILD/MyST-NB-0.13.2/docs/_build/man/index.html'
open file:///home/tkloczko/rpmbuild/BUILD/MyST-NB-0.13.2/docs/_build/man/index.html
_________________________________________________________________________________ summary __________________________________________________________________________________
ERROR:   docs-update: commands failed

problem
I suppose that myst-mb has not finished backend generating roff output.

Reproduce the bug

tox -e docs-update man

List your environment

list of modules installed in build env:

Package                       Version
----------------------------- -----------------
alabaster                     0.7.12
anyio                         3.5.0
argon2-cffi                   21.1.0
attrs                         21.4.0
Babel                         2.9.1
backcall                      0.2.0
beautifulsoup4                4.10.0
bleach                        4.1.0
build                         0.7.0
cffi                          1.15.0
charset-normalizer            2.0.12
click                         8.1.2
coconut                       1.6.0
codespell                     2.1.0
colorama                      0.4.4
coverage                      6.3.2
cPyparsing                    2.4.7.1.1.0
cycler                        0.11.0
debugpy                       1.6.0
decorator                     5.1.1
defusedxml                    0.7.1
distlib                       0.3.4
distro                        1.7.0
docutils                      0.17.1
entrypoints                   0.4
extras                        1.0.0
fastjsonschema                2.15.3
filelock                      3.6.0
fixtures                      3.0.0
fonttools                     4.32.0
gitdb                         4.0.9
GitPython                     3.1.27
gpg                           1.17.1-unknown
greenlet                      1.1.2
html5lib                      1.1
idna                          3.3
imagesize                     1.3.0
importlib-metadata            4.11.3
importlib-resources           5.7.0
iniconfig                     1.1.1
ipykernel                     6.13.0
ipython                       7.30.1
ipython-genutils              0.2.0
ipywidgets                    7.7.0
jedi                          0.18.1
Jinja2                        3.1.1
jsonschema                    4.4.0
jupyter-cache                 0.5.0
jupyter-client                7.2.2
jupyter-core                  4.9.2
jupyter-server                1.16.0
jupyter-server-mathjax        0.2.5
jupyter-sphinx                0.3.2
jupyterlab-pygments           0.1.2
jupytext                      1.13.7
kiwisolver                    1.3.2
libcomps                      0.1.18
markdown-it-py                2.0.1
MarkupSafe                    2.1.1
matplotlib                    3.5.1
matplotlib-inline             0.1.3
mdit-py-plugins               0.3.0
mdurl                         0.1.0
mistune                       0.8.4
myst-parser                   0.17.1
nbclient                      0.5.13
nbconvert                     6.4.5
nbdime                        3.1.1
nbformat                      5.3.0
nest-asyncio                  1.5.5
notebook                      6.4.8
numpy                         1.22.3
olefile                       0.46
packaging                     21.3
pandas                        1.4.2
pandocfilters                 1.5.0
parso                         0.8.3
pbr                           5.8.1
pep517                        0.12.0
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        9.1.0
pip                           22.0.4
platformdirs                  2.5.1
pluggy                        1.0.0
ply                           3.11
prometheus-client             0.14.1
prompt-toolkit                3.0.29
psutil                        5.9.0
ptyprocess                    0.7.0
py                            1.11.0
pycparser                     2.21
Pygments                      2.11.2
PyGObject                     3.42.0
pyparsing                     3.0.8
pyrsistent                    0.18.1
pytest                        7.1.1
pytest-cov                    3.0.0
pytest-datadir                1.3.1
pytest-regressions            2.3.1
python-dateutil               2.8.2
pytz                          2022.1
PyYAML                        6.0
pyzmq                         22.3.0
requests                      2.27.1
rpm                           4.17.0
Send2Trash                    1.8.0
setuptools                    62.0.0
six                           1.16.0
smmap                         5.0.0
sniffio                       1.2.0
snowballstemmer               2.2.0
soupsieve                     2.3.2.post1
Sphinx                        4.5.0.dev20220404
sphinx-copybutton             0.4.0
sphinx-togglebutton           0.2.3
sphinxcontrib-applehelp       1.0.2.dev20220410
sphinxcontrib-devhelp         1.0.2.dev20220410
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1.dev20220410
sphinxcontrib-qthelp          1.0.3.dev20220410
sphinxcontrib-serializinghtml 1.1.5
SQLAlchemy                    1.4.35.dev0
tabulate                      0.8.9
terminado                     0.13.3
testpath                      0.6.0
testtools                     2.5.0
toml                          0.10.2
tomli                         2.0.1
tornado                       6.1
tox                           3.24.5
traitlets                     5.1.1
typing_extensions             4.1.1
urllib3                       1.26.9
virtualenv                    20.13.4
wcwidth                       0.2.5
webencodings                  0.5.1
websocket-client              1.3.2
wheel                         0.37.1
widgetsnbextension            3.6.0
zipp                          3.8.0

[chrisjsewell]

Heya, yeh cheers for the feedback, this will be fixed in #380

[kloczek]

Is that PR can be applied on top of last release?

[chrisjsewell]

After merging #380, this now works

Note there are some warnings, but this IMO is to be expected for these output types that man does not support:

WARNING: No mime type available in priority list for builder 'man' ('text/html') [mystnb.mime_priority]
/Users/chrisjsewell/Documents/GitHub/MyST-NB-actual/docs/examples/interactive.md:62: WARNING: No mime type available in priority list for builder 'man' ('text/html') [mystnb.mime_priority]
/Users/chrisjsewell/Documents/GitHub/MyST-NB-actual/docs/examples/interactive.md:90: WARNING: No mime type available in priority list for builder 'man' ('text/html') [mystnb.mime_priority]
/Users/chrisjsewell/Documents/GitHub/MyST-NB-actual/docs/examples/interactive.md:90: WARNING: No mime type available in priority list for builder 'man' ('application/javascript') [mystnb.mime_priority]
/Users/chrisjsewell/Documents/GitHub/MyST-NB-actual/docs/examples/interactive.md:95: WARNING: No mime type available in priority list for builder 'man' ('text/html') [mystnb.mime_priority]
/Users/chrisjsewell/Documents/GitHub/MyST-NB-actual/docs/examples/interactive.md:95: WARNING: No mime type available in priority list for builder 'man' ('application/javascript') [mystnb.mime_priority]
/Users/chrisjsewell/Documents/GitHub/MyST-NB-actual/docs/examples/interactive.md:95: WARNING: No mime type available in priority list for builder 'man' ('text/html') [mystnb.mime_priority]
/Users/chrisjsewell/Documents/GitHub/MyST-NB-actual/docs/examples/interactive.md:95: WARNING: No mime type available in priority list for builder 'man' ('application/javascript') [mystnb.mime_priority]

[kloczek]

Yupi 😄

[kloczek]

There are to many paches to integrate with my build procedure so will wait on next release.

[kloczek]

Q: is it known ETA of next release? 🤔

[chrisjsewell]

Q: is it known ETA of next release? 🤔

Next release!? I've already done two since 0.13.2 😅

[kloczek]

OK so looks like it is another issue with generate man page in 0.15.0.

Coconut: Installing Jupyter kernels 'coconut_py', 'coconut_py2', 'coconut_py3'...
Coconut: Successfully installed Jupyter kernels: 'coconut_py', 'coconut_py2', 'coconut_py3'
myst v0.17.2: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=['amsmath', 'colon_fence', 'deflist', 'dollarmath', 'html_image'], linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', disable_syntax=[], all_links_external=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, highlight_code_blocks=True, number_code_blocks=[], title_to_header=False, heading_anchors=None, heading_slug_func=None, footnote_transition=True, sub_delimiters=('{', '}'), words_per_minute=200)
myst-nb v0.15.0: NbParserConfig(custom_formats={'.Rmd': ('jupytext.reads', {'fmt': 'Rmd'}, False)}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, execution_mode='cache', execution_cache_path='', execution_excludepatterns=(), execution_timeout=60, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_streams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_markdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False)
Using jupyter-cache at: /home/tkloczko/rpmbuild/BUILD/MyST-NB-0.15.0/build/sphinx/.jupyter_cache
loading intersphinx inventory from https://docs.python.org/3.8/objects.inv...
loading intersphinx inventory from https://jupyterbook.org/objects.inv...
loading intersphinx inventory from https://myst-parser.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://markdown-it-py.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://nbclient.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://nbformat.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://www.sphinx-doc.org/en/master/objects.inv...
intersphinx inventory has moved: https://jupyterbook.org/objects.inv -> https://jupyterbook.org/en/stable/objects.inv
building [mo]: targets for 0 po files that are out of date
building [man]: all manpages
updating environment: [new config] 22 added, 0 changed, 0 removed
reading sources... [  9%] authoring/custom-formats
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app.build(args.force_all, filenames)
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 330, in build
    self.builder.build_update()
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 283, in build_update
    self.build(['__all__'], to_build)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 300, in build
    updated_docnames = set(self.read())
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 407, in read
    self._read_serial(docnames)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 428, in _read_serial
    self.read_doc(docname)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 468, in read_doc
    doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
  File "/usr/lib/python3.8/site-packages/sphinx/io.py", line 181, in read_doc
    pub.publish()
  File "/usr/lib/python3.8/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/usr/lib/python3.8/site-packages/sphinx/io.py", line 101, in read
    self.parse()
  File "/usr/lib/python3.8/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/tkloczko/rpmbuild/BUILD/MyST-NB-0.15.0/myst_nb/sphinx_.py", line 123, in parse
    nb_renderer: NbElementRenderer = load_renderer(renderer_name)(
  File "/home/tkloczko/rpmbuild/BUILD/MyST-NB-0.15.0/myst_nb/core/render.py", line 849, in load_renderer
    raise EntryPointError(f"No Entry Point found for {RENDER_ENTRY_GROUP}:{name}")
myst_nb.core.render.EntryPointError: No Entry Point found for myst_nb.renderers:default

Exception occurred:
  File "/home/tkloczko/rpmbuild/BUILD/MyST-NB-0.15.0/myst_nb/core/render.py", line 849, in load_renderer
    raise EntryPointError(f"No Entry Point found for {RENDER_ENTRY_GROUP}:{name}")
myst_nb.core.render.EntryPointError: No Entry Point found for myst_nb.renderers:default
The full traceback has been saved in /tmp/sphinx-err-aqtv4l29.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

[kloczek]

Or it could be some sphinx-design issue ..

[tkloczko@devel-g2v SPECS]$ cat /tmp/sphinx-err-aqtv4l29.log
# Sphinx version: 4.5.0
# Python version: 3.8.13 (CPython)
# Docutils version: 0.17.1 release
# Jinja2 version: 3.1.1
# Last messages:
#   loading intersphinx inventory from https://nbformat.readthedocs.io/en/latest/objects.inv...
#   loading intersphinx inventory from https://www.sphinx-doc.org/en/master/objects.inv...
#   intersphinx inventory has moved: https://jupyterbook.org/objects.inv -> https://jupyterbook.org/en/stable/objects.inv
#   building [mo]: targets for 0 po files that are out of date
#   building [man]: all manpages
#   updating environment:
#   [new config]
#   22 added, 0 changed, 0 removed
#   reading sources... [  4%] authoring/basics
#   reading sources... [  9%] authoring/custom-formats
# Loaded extensions:
#   sphinx.ext.mathjax (4.5.0) from /usr/lib/python3.8/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from /usr/lib/python3.8/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.2) from /usr/lib/python3.8/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (2.0.0) from /usr/lib/python3.8/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.5) from /usr/lib/python3.8/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.3) from /usr/lib/python3.8/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /usr/lib/python3.8/site-packages/alabaster/__init__.py
#   sphinx_togglebutton (0.3.1) from /usr/lib/python3.8/site-packages/sphinx_togglebutton/__init__.py
#   myst_nb (0.15.0) from /home/tkloczko/rpmbuild/BUILD/MyST-NB-0.15.0/myst_nb/__init__.py
#   sphinx_copybutton (0.4.0) from /usr/lib/python3.8/site-packages/sphinx_copybutton/__init__.py
#   sphinx_book_theme (unknown version) from /usr/lib/python3.8/site-packages/sphinx_book_theme/__init__.py
#   sphinx.ext.intersphinx (4.5.0) from /usr/lib/python3.8/site-packages/sphinx/ext/intersphinx.py
#   sphinx.ext.autodoc.preserve_defaults (1.0) from /usr/lib/python3.8/site-packages/sphinx/ext/autodoc/preserve_defaults.py
#   sphinx.ext.autodoc.type_comment (4.5.0) from /usr/lib/python3.8/site-packages/sphinx/ext/autodoc/type_comment.py
#   sphinx.ext.autodoc (4.5.0) from /usr/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py
#   sphinx.ext.viewcode (4.5.0) from /usr/lib/python3.8/site-packages/sphinx/ext/viewcode.py
#   sphinx_design (0.0.13) from /usr/lib/python3.8/site-packages/sphinx_design/__init__.py
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app.build(args.force_all, filenames)
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 330, in build
    self.builder.build_update()
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 283, in build_update
    self.build(['__all__'], to_build)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 300, in build
    updated_docnames = set(self.read())
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 407, in read
    self._read_serial(docnames)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 428, in _read_serial
    self.read_doc(docname)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 468, in read_doc
    doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
  File "/usr/lib/python3.8/site-packages/sphinx/io.py", line 181, in read_doc
    pub.publish()
  File "/usr/lib/python3.8/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/usr/lib/python3.8/site-packages/sphinx/io.py", line 101, in read
    self.parse()
  File "/usr/lib/python3.8/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/tkloczko/rpmbuild/BUILD/MyST-NB-0.15.0/myst_nb/sphinx_.py", line 123, in parse
    nb_renderer: NbElementRenderer = load_renderer(renderer_name)(
  File "/home/tkloczko/rpmbuild/BUILD/MyST-NB-0.15.0/myst_nb/core/render.py", line 849, in load_renderer
    raise EntryPointError(f"No Entry Point found for {RENDER_ENTRY_GROUP}:{name}")
myst_nb.core.render.EntryPointError: No Entry Point found for myst_nb.renderers:default