Skip to content

Commit 04f3bbb

Browse files
authored
🔀 MERGE: Improve Notebook Output Rendering (#243)
This merge encapsulates a major refactor of the notebook output rendering process. The process was rewritten to be more module and pluggable, hopefully allowing for people to customise the way outputs are rendered for there own purposes. this is includes lots of additional configuration/improvements, such as ANSI and Markdown rendering, stdout/stderr removal, and utilisation of metadata to control image formatting. Also, error reporting and testing has been improved.
2 parents 0b09901 + 3368a64 commit 04f3bbb

43 files changed

Lines changed: 1660 additions & 434 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

docs/_static/patch.css

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
/* CSS that should eventually go in sphinx-book-theme */
2+
3+
dd {
4+
margin-top: 3px;
5+
margin-bottom: 10px;
6+
margin-left: 30px;
7+
}

docs/api/index.rst

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
.. _api/main:
2+
3+
Python API
4+
==========
5+
6+
.. toctree::
7+
:maxdepth: 2
8+
9+
nodes
10+
render_outputs
11+
12+
Miscellaneous
13+
-------------
14+
15+
.. autoclass:: myst_nb.ansi_lexer.AnsiColorLexer
16+
:members:
17+
:undoc-members:
18+
:show-inheritance:

docs/api/nodes.rst

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
.. _api/nodes:
2+
3+
AST Nodes
4+
---------
5+
6+
.. automodule:: myst_nb.nodes
7+
8+
.. autoclass:: myst_nb.nodes.CellNode
9+
:members:
10+
:undoc-members:
11+
:show-inheritance:
12+
13+
.. autoclass:: myst_nb.nodes.CellInputNode
14+
:members:
15+
:undoc-members:
16+
:show-inheritance:
17+
18+
.. autoclass:: myst_nb.nodes.CellOutputNode
19+
:members:
20+
:undoc-members:
21+
:show-inheritance:
22+
23+
.. autoclass:: myst_nb.nodes.CellOutputBundleNode
24+
:members:
25+
:undoc-members:
26+
:show-inheritance:

docs/api/render_outputs.rst

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
.. _api/output_renderer:
2+
3+
Output Renderer
4+
---------------
5+
6+
.. automodule:: myst_nb.render_outputs
7+
8+
.. autoclass:: myst_nb.render_outputs.CellOutputsToNodes
9+
:members:
10+
:undoc-members:
11+
:show-inheritance:
12+
13+
.. autoexception:: myst_nb.render_outputs.MystNbEntryPointError
14+
:members:
15+
:undoc-members:
16+
:show-inheritance:
17+
18+
.. autofunction:: myst_nb.render_outputs.load_renderer
19+
20+
21+
.. autoclass:: myst_nb.render_outputs.CellOutputRendererBase
22+
:members:
23+
:undoc-members:
24+
:show-inheritance:
25+
:special-members: __init__
26+
27+
.. autoclass:: myst_nb.render_outputs.CellOutputRenderer
28+
:members:
29+
:undoc-members:
30+
:show-inheritance:

docs/conf.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@
3434
"sphinx_togglebutton",
3535
"sphinx_copybutton",
3636
"sphinx.ext.intersphinx",
37+
"sphinx.ext.autodoc",
38+
# "sphinx.ext.viewcode"
3739
]
3840

3941
# Add any paths that contain templates here, relative to this directory.
@@ -63,19 +65,31 @@
6365
}
6466

6567
intersphinx_mapping = {
68+
"python": ("https://docs.python.org/3.8", None),
6669
"jb": ("https://jupyterbook.org/", None),
6770
"myst": ("https://myst-parser.readthedocs.io/en/latest/", None),
6871
"markdown_it": ("https://markdown-it-py.readthedocs.io/en/latest", None),
6972
"nbclient": ("https://nbclient.readthedocs.io/en/latest", None),
7073
"nbformat": ("https://nbformat.readthedocs.io/en/latest", None),
74+
"sphinx": ("https://www.sphinx-doc.org/en/3.x", None),
7175
}
7276

7377
intersphinx_cache_limit = 5
7478

79+
nitpick_ignore = [
80+
("py:class", "docutils.nodes.document"),
81+
("py:class", "docutils.nodes.Node"),
82+
("py:class", "docutils.nodes.container"),
83+
("py:class", "docutils.nodes.system_message"),
84+
("py:class", "nbformat.notebooknode.NotebookNode"),
85+
("py:class", "pygments.lexer.RegexLexer"),
86+
]
87+
7588
# Add any paths that contain custom static files (such as style sheets) here,
7689
# relative to this directory. They are copied after the builtin static files,
7790
# so a file named "default.css" will overwrite the builtin "default.css".
7891
html_static_path = ["_static"]
92+
html_css_files = ["patch.css"]
7993

8094
copybutton_selector = "div:not(.output) > div.highlight pre"
8195

docs/index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,9 @@ Finally, here is documentation on contributing to the development of MySt-NB
8181

8282
```{toctree}
8383
:titlesonly:
84+
:maxdepth: 1
8485
develop/contributing
86+
api/index
8587
GitHub Repo <https://github.com/executablebooks/myst-nb>
8688
```
8789

docs/use/formatting_outputs.md

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
---
2+
jupytext:
3+
text_representation:
4+
extension: .md
5+
format_name: myst
6+
format_version: '0.8'
7+
jupytext_version: '1.4.1'
8+
kernelspec:
9+
display_name: Python 3
10+
language: python
11+
name: python3
12+
---
13+
14+
(use/format)=
15+
# Formatting code outputs
16+
17+
(use/format/priority)=
18+
## Render priority
19+
20+
When Jupyter executes a code cell it can produce multiple outputs, and each of these outputs can contain multiple [MIME media types](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types), for use by different output formats (like HTML or LaTeX).
21+
22+
MyST-NB stores a default priority dictionary for most of the common [Sphinx builders](https://www.sphinx-doc.org/en/master/usage/builders/index.html), which you can be also update in your `conf.py`.
23+
For example, this is the default priority list for HTML:
24+
25+
```python
26+
nb_render_priority = {
27+
"html": (
28+
"application/vnd.jupyter.widget-view+json",
29+
"application/javascript",
30+
"text/html",
31+
"image/svg+xml",
32+
"image/png",
33+
"image/jpeg",
34+
"text/markdown",
35+
"text/latex",
36+
"text/plain",
37+
)
38+
}
39+
```
40+
41+
:::{seealso}
42+
[](use/format/cutomise), for a more advanced means of customisation.
43+
:::
44+
45+
## Removing stdout and stderr
46+
47+
In some cases you may not wish to display stdout/stderr outputs in your final documentation,
48+
for example, if they are only for debugging purposes.
49+
You can tell MyST-NB to remove these outputs using the `remove-stdout` and `remove-stderr` [cell tags](https://jupyter-notebook.readthedocs.io/en/stable/changelog.html#cell-tags), like so:
50+
51+
````md
52+
```{code-cell} ipython3
53+
:tags: [remove-input,remove-stdout,remove-stderr]
54+
55+
import pandas, sys
56+
print("this is some stdout")
57+
print("this is some stderr", file=sys.stderr)
58+
# but what I really want to show is:
59+
pandas.DataFrame({"column 1": [1, 2, 3]})
60+
```
61+
````
62+
63+
```{code-cell} ipython3
64+
:tags: [remove-input,remove-stdout,remove-stderr]
65+
66+
import pandas, sys
67+
print("this is some stdout")
68+
print("this is some stderr", file=sys.stderr)
69+
# but what I really want to show is:
70+
pandas.DataFrame({"column 1": [1, 2, 3]})
71+
```
72+
73+
(use/format/images)=
74+
## Images
75+
76+
With the default renderer, for any image types output by the code, we can apply formatting *via* cell metadata.
77+
The keys should be placed under `myst`, then for the image we can apply all the variables of the standard [image directive](https://docutils.sourceforge.io/docs/ref/rst/directives.html#image):
78+
79+
- **width**: length or percentage (%) of the current line width
80+
- **height**: length
81+
- **scale**: integer percentage (the "%" symbol is optional)
82+
- **align**: "top", "middle", "bottom", "left", "center", or "right"
83+
- **classes**: space separated strings
84+
- **alt**: string
85+
86+
Units of length are: 'em', 'ex', 'px', 'in', 'cm', 'mm', 'pt', 'pc'
87+
88+
We can also set a caption (which is rendered as [CommonMark](https://commonmark.org/)) and name, by which to reference the figure:
89+
90+
````md
91+
```{code-cell} ipython3
92+
---
93+
myst:
94+
image:
95+
width: 200px
96+
alt: fun-fish
97+
classes: shadow bg-primary
98+
figure:
99+
caption: |
100+
Hey everyone its **party** time!
101+
name: fun-fish
102+
---
103+
from IPython.display import Image
104+
Image("images/fun-fish.png")
105+
```
106+
````
107+
108+
```{code-cell} ipython3
109+
---
110+
myst:
111+
image:
112+
width: 300px
113+
alt: fun-fish
114+
classes: shadow bg-primary
115+
figure:
116+
caption: |
117+
Hey everyone its **party** time!
118+
name: fun-fish
119+
---
120+
from IPython.display import Image
121+
Image("images/fun-fish.png")
122+
```
123+
124+
Now we can link to the image from anywhere in our documentation: [swim to the fish](fun-fish)
125+
126+
(use/format/markdown)=
127+
## Markdown
128+
129+
Markdown output is parsed by MyST-Parser, currently with the configuration set to `myst_commonmark_only=True` (see [MyST configuration options](myst:intro/config-options)).
130+
131+
The parsed Markdown is integrated into the wider documentation, and so it is possible, for example, to include internal references:
132+
133+
```{code-cell} ipython3
134+
from IPython.display import display, Markdown
135+
display(Markdown('**_some_ markdown** and an [internal reference](use/format/markdown)!'))
136+
```
137+
138+
and even internal images can be rendered!
139+
140+
```{code-cell} ipython3
141+
display(Markdown('![figure](../_static/logo.png)'))
142+
```
143+
144+
(use/format/ansi)=
145+
## ANSI Outputs
146+
147+
By default, the standard output/error streams and text/plain MIME outputs may contain ANSI escape sequences to change the text and background colors.
148+
149+
```{code-cell} ipython3
150+
import sys
151+
print("BEWARE: \x1b[1;33;41mugly colors\x1b[m!", file=sys.stderr)
152+
print("AB\x1b[43mCD\x1b[35mEF\x1b[1mGH\x1b[4mIJ\x1b[7m"
153+
"KL\x1b[49mMN\x1b[39mOP\x1b[22mQR\x1b[24mST\x1b[27mUV")
154+
```
155+
156+
This uses the built-in {py:class}`~myst_nb.ansi_lexer.AnsiColorLexer` [pygments lexer](https://pygments.org/).
157+
You can change the lexer used in the `conf.py`, for example to turn off lexing:
158+
159+
```python
160+
nb_render_text_lexer = "none"
161+
```
162+
163+
The following code[^acknowledge] shows the 8 basic ANSI colors it is based on.
164+
Each of the 8 colors has an “intense” variation, which is used for bold text.
165+
166+
[^acknowledge]: Borrowed from [nbsphinx](https://nbsphinx.readthedocs.io/en/0.7.1/code-cells.html#ANSI-Colors)!
167+
168+
```{code-cell} ipython3
169+
text = " XYZ "
170+
formatstring = "\x1b[{}m" + text + "\x1b[m"
171+
172+
print(
173+
" " * 6
174+
+ " " * len(text)
175+
+ "".join("{:^{}}".format(bg, len(text)) for bg in range(40, 48))
176+
)
177+
for fg in range(30, 38):
178+
for bold in False, True:
179+
fg_code = ("1;" if bold else "") + str(fg)
180+
print(
181+
" {:>4} ".format(fg_code)
182+
+ formatstring.format(fg_code)
183+
+ "".join(
184+
formatstring.format(fg_code + ";" + str(bg)) for bg in range(40, 48)
185+
)
186+
)
187+
```
188+
189+
:::{note}
190+
ANSI also supports a set of 256 indexed colors.
191+
This is currently not supported, but we hope to introduce it at a later date
192+
(raise an issue on the repository if you require it!).
193+
:::
194+
195+
(use/format/cutomise)=
196+
## Customise the render process
197+
198+
The render process is goverened by subclasses of {py:class}`myst_nb.render_outputs.CellOutputRendererBase`, which dictate how to create the `docutils` AST nodes for a particular MIME type. the default implementation is {py:class}`~myst_nb.render_outputs.CellOutputRenderer`.
199+
200+
Implementations are loaded *via* Python [entry points](https://packaging.python.org/guides/distributing-packages-using-setuptools/#entry-points), in the `myst_nb.mime_render` group.
201+
So it is possible to inject your own subclass to handle rendering.
202+
203+
For example, the renderers loaded in this package are:
204+
205+
```python
206+
entry_points={
207+
"myst_nb.mime_render": [
208+
"default = myst_nb.render_outputs:CellOutputRenderer",
209+
"inline = myst_nb.render_outputs:CellOutputRendererInline",
210+
],
211+
}
212+
```
213+
214+
You can then select the renderer plugin in your `conf.py`:
215+
216+
```python
217+
nb_render_plugin = "default"
218+
```

docs/use/hiding.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ right to show it.
5050
5151
# This cell has a hide-input tag
5252
fig, ax = plt.subplots()
53-
ax.scatter(*data, c=data[0], s=data[0])
53+
points =ax.scatter(*data, c=data[0], s=data[0])
5454
```
5555

5656
Here's a cell with a `hide-output` tag:
@@ -60,7 +60,7 @@ Here's a cell with a `hide-output` tag:
6060
6161
# This cell has a hide-output tag
6262
fig, ax = plt.subplots()
63-
ax.scatter(*data, c=data[0], s=data[0])
63+
points =ax.scatter(*data, c=data[0], s=data[0])
6464
```
6565

6666
And the following cell has a `hide-cell` tag:
@@ -70,7 +70,7 @@ And the following cell has a `hide-cell` tag:
7070
7171
# This cell has a hide-cell tag
7272
fig, ax = plt.subplots()
73-
ax.scatter(*data, c=data[0], s=data[0])
73+
points =ax.scatter(*data, c=data[0], s=data[0])
7474
```
7575

7676
(use/hiding/markdown)=
@@ -152,7 +152,7 @@ the page at all.
152152
153153
# This cell has a remove-input tag
154154
fig, ax = plt.subplots()
155-
ax.scatter(*data, c=data[0], s=data[0])
155+
points =ax.scatter(*data, c=data[0], s=data[0])
156156
```
157157

158158
Here's a cell with a `remove-output` tag:

docs/use/images/fun-fish.png

89.9 KB
Loading

docs/use/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,5 +10,6 @@ start
1010
myst
1111
execute
1212
hiding
13+
formatting_outputs
1314
glue
1415
```

0 commit comments

Comments
 (0)