Update getting started and reference docs

docs/examples/custom-formats.Rmd

@@ -33,7 +33,7 @@ nb_custom_formats = {
 
 :::{important}
 
-By default, Markdown cells in the Notebook will be parsed using the same MyST parser configuration as for other Markdown files ([see available configuration options](start/config-options)).
+By default, Markdown cells in the Notebook will be parsed using the same MyST parser configuration as for other Markdown files ([see available configuration options](config/reference)).
 
 But, if this is incompatible with your file format, then you can specify for the Markdown to be parsed as **strictly CommonMark**, using a third argument:
 
@@ -56,7 +56,7 @@ This key should be a list mapping each cell to the starting line number in the o
 }
 ```
 
-This mapping allows for "true" error reporting, as described in [](start/error-reporting).
+This mapping allows for "true" error reporting, as described in [](myst/error-reporting).
 
 ## Using Jupytext
 

docs/use/config-reference.md

@@ -0,0 +1,79 @@
+
+(config/reference)=
+# Configuration reference
+
+This page lists the configuration options that are available to control MyST-NB.
+For more detailed explanation of when to use each option, see the other sections in the documentation.
+
+## MyST Parser configuration
+
+The MyST-NB parser derives from {ref}`the base MyST-Parser <myst:intro/get-started>`, and so all the same configuration options are available.
+See the {ref}`MyST configuration options <myst:sphinx/config-options>` for the full set of options.
+
+## Notebook execution configuration
+
+This configuration is used to control how Jupyter Notebooks are executed at build-time.
+
+`````{list-table}
+:header-rows: 1
+
+* - Option
+  - Default
+  - Description
+* - `jupyter_cache`
+  - ""
+  - Path to jupyter_cache, [see here](execute/cache) for details.
+* - `execution_excludepatterns`
+  - ()
+  - Exclude certain file patterns from execution, [see here](execute/config) for details.
+* - `jupyter_execute_notebooks`
+  - "auto"
+  - The logic for executing notebooks, [see here](execute/config) for details.
+* - `execution_in_temp`
+  - `False`
+  - If `True`, then a temporary directory will be created and used as the command working directory (cwd), if `False` then the notebook's parent directory will be the cwd.
+* - `execution_allow_errors`
+  - `False`
+  - If `False`, when a code cell raises an error the execution is stopped, if `True` then all cells are always run.
+    This can also be overridden by metadata in a notebook, [see here](execute/allow_errors) for details.
+* - `execution_timeout`
+  - 30
+  - The maximum time (in seconds) each notebook cell is allowed to run.
+    This can also be overridden by metadata in a notebook, [see here](execute/timeout) for details.
+* - `execution_show_tb`
+  - `False`
+  - Show failed notebook tracebacks in stdout (in addition to writing to file).
+`````
+
+## Notebook parsing and output rendering
+
+These configuration options affect the look and feel of notebook parsing and output rendering.
+
+`````{list-table}
+:header-rows: 1
+
+* - Option
+  - Default
+  - Description
+* - `nb_custom_formats`
+  - `{}`
+  - Define custom functions for conversion of files to notebooks, [see here](examples/custom_formats) for details.
+* - `nb_render_priority`
+  - `{}`
+  - Dict override for MIME type render priority, [see here](use/format/priority) for details.
+* - `nb_render_plugin`
+  - `default`
+  - Entry point pointing toward a code cell output renderer, [see here](use/format/cutomise) for details.
+* - `nb_render_text_lexer`
+  - `myst-ansi`
+  - pygments lexer for rendering text outputs, [see here](use/format/ansi) for details.
+* - `nb_render_key`
+  - `render`
+  - The top-level cell metadata key, to store render control data, [see here](use/format/images) for examples.
+* - `nb_output_stderr`
+  - `show`
+  - One of 'show', 'remove', 'warn', 'error' or 'severe', [see here](use/format/stderr) for details.
+* - `nb_merge_streams`
+  - `False`
+  - If `True`, ensure all stdout / stderr output streams are merged into single outputs. This ensures deterministic outputs.
+`````

docs/use/execute.md

@@ -13,7 +13,7 @@ kernelspec:
   name: python3
 ---
 
-# Executing and caching your content
+# Execute and cache your content
 
 MyST-NB can automatically run and cache notebooks contained in your project using [jupyter-cache].
 Notebooks can either be run each time the documentation is built, or cached locally so that re-runs occur only when code cells have changed.
@@ -63,7 +63,7 @@ execution_excludepatterns = ['list', 'of', '*patterns']
 Any file that matches one of the items in `execution_excludepatterns` will not be executed.
 
 (execute/cache)=
-## Caching the notebook execution
+## Cache execution outputs
 
 As mentioned above, you can **cache the results of executing a notebook page** by setting:
 
@@ -123,7 +123,7 @@ This global value can also be overridden per notebook by adding this to you note
 ```
 
 (execute/allow_errors)=
-## Dealing with code that raises errors
+## Raise errors in code cells
 
 In some cases, you may want to intentionally show code that doesn't work (e.g., to show the error message).
 You can achieve this at "three levels":

docs/use/formatting_outputs.md

@@ -12,7 +12,7 @@ kernelspec:
 ---
 
 (use/format)=
-# Formatting code outputs
+# Format code outputs
 
 (use/format/priority)=
 ## Render priority
@@ -42,8 +42,11 @@ nb_render_priority = {
 [](use/format/cutomise), for a more advanced means of customisation.
 :::
 
+(use/format/stdout-stderr)=
+## stdout and stderr
+
 (use/format/stderr)=
-## Removing stdout and stderr
+### Remove stdout or stderr
 
 In some cases you may not wish to display stdout/stderr outputs in your final documentation,
 for example, if they are only for debugging purposes.
@@ -80,6 +83,22 @@ This can be set to:
 - `"remove-warn"`: remove all stderr, but log a warning to sphinx if any found
 - `"warn"`, `"error"` or `"severe"`: log to sphinx, at a certain level, if any found.
 
+### Group stdout and stderr into single streams
+
+Cells may print multiple things to `stdout` and `stderr`.
+For example, if a cell prints status updates throughout its execution, each of these is often printed to `stdout`.
+By default, these outputs may be split across multiple items, and will be rendered as separate "chunks" in your built documentation.
+
+If you'd like each of the outputs in `stderr` and `stdout` to be merged into a single stream for each, use the following configuration:
+
+```python
+nb_merge_streams = True
+```
+
+This will ensure that all `stderr` and `stdout` outputs are merged into a single group.
+This also makes cell outputs more deterministic.
+Normally, slight differences in timing may result in different orders of `stderr` and `stdout` in the cell output, while this setting will sort them properly.
+
 (use/format/images)=
 ## Images
 

docs/use/glue.md

@@ -13,7 +13,7 @@ kernelspec:
 
 (glue)=
 
-# Inserting variables into pages with `glue`
+# Insert variables into pages with `glue`
 
 You often wish to run analyses in one notebook and insert them into your
 documents text elsewhere. For example, if you'd like to include a figure,

docs/use/hiding.md

@@ -11,7 +11,7 @@ kernelspec:
   name: python3
 ---
 
-# Hiding cell contents
+# Hide cell contents
 
 You can use Jupyter Notebook **cell tags** to control some of the behavior of
 the rendered notebook. This uses the [**`sphinx-togglebutton`**](https://sphinx-togglebutton.readthedocs.io/en/latest/)
@@ -22,7 +22,7 @@ package to add a little button that toggles the visibility of content.[^download
 
 (use/hiding/code)=
 
-## Hiding code cells
+## Hide code cells
 
 You can use **cell tags** to control the content hidden with code cells.
 Add the following tags to a cell's metadata to control
@@ -75,7 +75,7 @@ points =ax.scatter(*data, c=data[0], s=data[0])
 
 (use/hiding/markdown)=
 
-## Hiding markdown cells
+## Hide markdown cells
 
 There are two ways to hide markdown cells. First, **you can add the `hide-input`**
 cell metadata. This triggers the same hiding behavior described above for
@@ -132,7 +132,7 @@ with `#`. If you'd like to use headings, do one of the following:
 
 (use/removing)=
 
-## Removing parts of cells
+## Remove parts of cells
 
 Sometimes, you want to entirely remove parts of a cell so that it doesn't make it
 into the output at all. To do this, you can use the same tag pattern described above,

docs/use/index.md

@@ -12,4 +12,5 @@ execute
 hiding
 formatting_outputs
 glue
+config-reference
 ```

docs/use/myst.md

@@ -52,3 +52,25 @@ source_suffix = {
     '.myst': 'myst-nb',
 }
 ```
+
+(myst/error-reporting)=
+
+## Error reporting in Sphinx
+
+When Sphinx encounters and error or raises a warning, it will print the location and source file of the text that generated that error.
+This works slightly differently depending on whether you use markdown files or Jupyter Notebook files.
+
+For markdown (`.md`) files, Sphinx will correctly report the line number that the error or warning is associated with:
+
+```
+source/path:4: (WARNING/2) Duplicate reference definition: abc
+```
+
+For Jupyter Notebook (`.ipynb`) files, these errors also correspond to a cell index.
+To allow for this, we use a special format of line number corresponding to: `<CELL_INDEX> * 10000 + LINE_NUMBER`.
+
+For example, the following error corresponds to **Cell 1, line 4**:
+
+```
+source/path:10004: (WARNING/2) Duplicate reference definition: abc
+```

docs/use/start.md

@@ -1,18 +1,15 @@
 # Get started
 
 This section covers how to get started with the `MyST-NB` Sphinx extension.
-The Sphinx extension allows you to read markdown (`.md`) and Jupyter Notebook (`.ipynb`)
-files into your Sphinx site. It also enables you to write [MyST Markdown](myst.md)
-in your pages.
+The Sphinx extension allows you to read markdown (`.md`) and Jupyter Notebook (`.ipynb`) files into your Sphinx site.
+It also enables you to write [MyST Markdown](myst.md) in your pages.
 
-## Install and activate
-
-To install `myst-nb`, do the following:
+To get started with the extension, follow these steps.
 
 * **Install** `myst-nb` with the following command:
 
-  ```bash
-  pip install myst-nb
+  ```console
+  $ pip install myst-nb
   ```
 
 * **Activate** the `myst_nb` extension in your Sphinx site by adding it to your list of
@@ -25,94 +22,15 @@ To install `myst-nb`, do the following:
   ]
   ```
 
-Once you do this, MyST-NB will now parse both markdown (`.md`), Jupyter notebooks (`.ipynb`), and even [text-based Notebooks](markdown.md) (`.md`) into your Sphinx site
-(see also [custom notebook formats](examples/custom_formats)).
-
-(start/error-reporting)=
-
-## Sphinx Error Reporting
-
-For `.md` files, sphinx will correctly report the line number that the error or warning is associated with:
-
-```
-source/path:4: (WARNING/2) Duplicate reference definition: abc
-```
-
-For `.ipynb` files, these errors also correspond to a cell index.
-To allow for this, we use a special format of line number corresponding to: `<CELL_INDEX> * 10000 + LINE_NUMBER`, for example:
-
-```
-source/path:10004: (WARNING/2) Duplicate reference definition: abc
-```
-
-(start/config-options)=
-
-## MyST-NB configuration options
-
-The MyST-NB parser derives from {ref}`the base MyST-Parser <myst:intro/get-started>`, and so all the same configuration options are available.
-See the {ref}`MyST configuration options <myst:sphinx/config-options>` for the full set of options.
-
-MyST-NB then adds some additional configuration, specific to notebooks.
-Firstly for execution:
-
-`````{list-table}
-:header-rows: 1
+* **Add MyST and notebook content** to your documentation's source files.
+  Sphinx will now be able to parse **markdown files** written in [MyST Markdown](https://myst-parser.readthedocs.io), Jupyter Notebooks (ending in `.ipynb`), and [Jupyter Notebooks written in plain-text with MyST markdown](markdown.md).
+  Make sure to include paths to your content in a Sphinx `toctree` directive!
 
-* - Option
-  - Default
-  - Description
-* - `jupyter_cache`
-  - ""
-  - Path to jupyter_cache, [see here](execute/cache) for details.
-* - `execution_excludepatterns`
-  - ()
-  - Exclude certain file patterns from execution, [see here](execute/config) for details.
-* - `jupyter_execute_notebooks`
-  - "auto"
-  - The logic for executing notebooks, [see here](execute/config) for details.
-* - `execution_in_temp`
-  - `False`
-  - If `True`, then a temporary directory will be created and used as the command working directory (cwd), if `False` then the notebook's parent directory will be the cwd.
-* - `execution_allow_errors`
-  - `False`
-  - If `False`, when a code cell raises an error the execution is stopped, if `True` then all cells are always run.
-    This can also be overridden by metadata in a notebook, [see here](execute/allow_errors) for details.
-* - `execution_timeout`
-  - 30
-  - The maximum time (in seconds) each notebook cell is allowed to run.
-    This can also be overridden by metadata in a notebook, [see here](execute/timeout) for details.
-* - `execution_show_tb`
-  - `False`
-  - Show failed notebook tracebacks in stdout (in addition to writing to file).
-`````
+* **Build your documentation**. MyST-NB will now parse any markdown (`.md`), Jupyter notebooks (`.ipynb`), and [text-based Notebooks](markdown.md) (`.md`) into your Sphinx site, and include them in the outputs.
 
-Then for parsing and output rendering:
+## Next steps
 
-`````{list-table}
-:header-rows: 1
+There is a lot more that you can do with MyST-NB.
+For example, you can define [custom text-based notebook formats](examples/custom_formats), [execute and cache notebook content](execute.md), and [format cell outputs](formatting_outputs.md).
 
-* - Option
-  - Default
-  - Description
-* - `nb_custom_formats`
-  - `{}`
-  - Define custom functions for conversion of files to notebooks, [see here](examples/custom_formats) for details.
-* - `nb_render_priority`
-  - `{}`
-  - Dict override for MIME type render priority, [see here](use/format/priority) for details.
-* - `nb_render_plugin`
-  - `default`
-  - Entry point pointing toward a code cell output renderer, [see here](use/format/cutomise) for details.
-* - `nb_render_text_lexer`
-  - `myst-ansi`
-  - pygments lexer for rendering text outputs, [see here](use/format/ansi) for details.
-* - `nb_render_key`
-  - `render`
-  - The top-level cell metadata key, to store render control data, [see here](use/format/images) for examples.
-* - `nb_output_stderr`
-  - `show`
-  - One of 'show', 'remove', 'warn', 'error' or 'severe', [see here](use/format/stderr) for details.
-* - `nb_merge_streams`
-  - `False`
-  - If `True`, ensure all stdout / stderr output streams are merged into single outputs. This ensures deterministic outputs.
-`````
+Check out the sections to the left under [Using with Sphinx](index.md) for more information.