Skip to content

Commit b59613c

Browse files
hdpriest-uihdpriest-ui
committed
more documentation overhaul
1 parent bb30fb5 commit b59613c

17 files changed

Lines changed: 71 additions & 322 deletions

‎documentation/source/contributing.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ The source code and latest updates are available on GitHub: `hdpriest-ui/icrn_ma
77

88
Developer Guide: Creating New Kernels
99
-------------------------------------
10-
- To contribute new R kernels (library environments) to the central catalog, see the maintainer section for a step-by-step walkthrough using Bioconductor as an example.
10+
- To contribute new R kernels (library environments) to the central catalog, see the contributor section for a step-by-step walkthrough using Bioconductor as an example.
1111

1212
Development Environment
1313
-----------------------

‎documentation/source/contributor/index.rst‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,13 @@
11
Kernel Contributor Guide
22
========================
33

4-
This section provides guidance for contributors who want to create and submit new R kernels to the ICRN catalog.
4+
This section provides guidance for contributors who want to create and submit new kernels to the ICRN catalog, as well as maintainers who manage and index kernels.
55

66
.. toctree::
77
:maxdepth: 2
88

9-
creating_r_kernels
109
kernel_requirements
10+
creating_r_kernels
11+
r_kernels
12+
python_kernels
1113

documentation/source/maintainer/jupyter_pieces.rst renamed to documentation/source/contributor/jupyter_pieces.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
Adding Jupyter Pieces (Maintainer)
2-
=================================
1+
Adding Jupyter Pieces
2+
=====================
33

44
.. note::
55
Instructions for adding Jupyter integration to Python kernels are forthcoming.

‎documentation/source/contributor/kernel_requirements.rst‎

Lines changed: 22 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,27 @@ Each kernel submission must include:
2727
* - Package List
2828
- Complete list of included packages with versions
2929

30+
Indexing Kernels
31+
----------------
32+
33+
The ``kernel_indexer`` script can be used to create package manifests and inventory all kernels in a repository. This is useful for:
34+
35+
- Tracking which packages are included in each kernel
36+
- Generating package inventories for documentation
37+
- Creating indexes for package discovery
38+
39+
To index all Python kernels in a repository:
40+
41+
.. code-block:: bash
42+
43+
./kernel_indexer index --kernel-root /path/to/repository --language Python
44+
45+
This creates a ``package_manifest.json`` file in each kernel directory containing:
46+
- Kernel name and version
47+
- Language and language version
48+
- Complete list of packages with versions and sources
49+
- Indexing timestamp
50+
3051
Required Directory Structure
3152
----------------------------
3253

@@ -48,7 +69,6 @@ Python Kernels
4869
.. code-block:: text
4970
5071
<kernel_root>/Python/<kernel_name>/<version>/
51-
├── conda-meta/ # Conda environment metadata
5272
├── lib/
5373
│ └── python3.x/
5474
│ └── site-packages/
@@ -77,23 +97,6 @@ The ``package_manifest.json`` file must conform to this schema:
7797
]
7898
}
7999
80-
Catalog Entry
81-
-------------
82-
83-
After creating the kernel, add an entry to ``icrn_kernel_catalog.json``:
84-
85-
.. code-block:: json
86-
87-
{
88-
"R": {
89-
"mykernel": {
90-
"1.0": {
91-
"path": "R/mykernel/1.0",
92-
"description": "Description of the kernel"
93-
}
94-
}
95-
}
96-
}
97100
98101
Quality Requirements
99102
--------------------
@@ -118,7 +121,7 @@ Naming Conventions
118121
Version Guidelines
119122
------------------
120123

121-
- Follow semantic versioning: ``MAJOR.MINOR`` or ``MAJOR.MINOR.PATCH``
124+
- Follow semantic versioning: ``MAJOR.MINOR``
122125
- For upstream project kernels, match their versioning
123126
- Increment version for any package updates
124127
- Never modify an existing version in-place

documentation/source/maintainer/python_kernels.rst renamed to documentation/source/contributor/python_kernels.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
Python Kernels (Maintainer)
2-
==========================
1+
Python Kernels
2+
==============
33

4-
This section provides guidance for maintainers on creating and managing Python kernels for the ICRN platform.
4+
This section provides guidance for creating and managing Python kernels for the ICRN platform.
55

66
Creating Python Kernels
77
----------------------

documentation/source/maintainer/r_kernels.rst renamed to documentation/source/contributor/r_kernels.rst

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
R Kernels (Maintainer)
2-
======================
1+
R Kernels
2+
=========
33

4-
This section provides guidance for maintainers on creating and managing R kernels for the ICRN platform.
4+
This section provides guidance for creating and managing R kernels for the ICRN platform.
55

66
Creating R Kernels
77
-----------------
@@ -29,6 +29,7 @@ Before distributing kernels, ensure they:
2929
- Include all required dependencies
3030
- Work with the ICRN Jupyter environment
3131
- Pass validation tests
32+
- that permissions are set correctly: general users should have read access, while group and owner will need read, write, and execute
3233

3334
.. note::
3435
Detailed step-by-step instructions for creating R kernels are available in the kernel creation documentation.

‎documentation/source/index.rst‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,15 @@ Illinois Computes Library & Kernel Manager
66

77
The Illinois Computes Library & Kernel Manager is a tool for managing R library environments for the Illinois Computes Research Notebook (ICRN) platform. It enables users to easily install, switch, and manage complex R package sets without manual configuration.
88

9-
.. figure:: demo_resources/icrn_libraries_mngr_switch.gif
10-
:align: center
11-
:width: 600
9+
Kernel Information Browser
10+
--------------------------
1211

13-
Example: Instantly switching from one large R Kernel to another.
12+
For advanced searching and detailed kernel information, visit the ICRN Kernel Information web interface:
13+
14+
- **Production**: https://kernels.ncsa.illinois.edu
15+
- **Development**: https://kernels.cori-dev.ncsa.illinois.edu/
16+
17+
The web interface allows you to browse all kernels by language, search for specific packages across all kernels, view complete package manifests for each kernel, and copy CLI commands for kernel operations.
1418

1519
.. toctree::
1620
:maxdepth: 2
@@ -19,7 +23,6 @@ The Illinois Computes Library & Kernel Manager is a tool for managing R library
1923
user/index
2024
administrator/index
2125
contributor/index
22-
maintainer/index
2326
contributing
2427
troubleshooting
2528
reference

‎documentation/source/maintainer/index.rst‎

Lines changed: 0 additions & 10 deletions
This file was deleted.

‎documentation/source/troubleshooting.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,4 +136,4 @@ If you continue to experience issues:
136136
3. **Review configuration**: Ensure all paths and permissions are correct
137137
4. **Report issues**: Create an issue on GitHub with detailed error information
138138

139-
For more detailed information, see the :doc:`user_guide` and :doc:`maintainer_guide`.
139+
For more detailed information, see the :doc:`user/index` and :doc:`contributor/index`.

‎documentation/source/user/campus_cluster/slurm_examples.rst‎

Lines changed: 0 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -58,34 +58,3 @@ Python Kernel Batch Job
5858
# Run Python script
5959
python my_analysis.py
6060
61-
GPU Kernel Batch Job
62-
-------------------
63-
64-
.. code-block:: bash
65-
66-
#!/bin/bash
67-
#SBATCH --job-name=gpu_kernel_job
68-
#SBATCH --output=gpu_kernel_job_%j.out
69-
#SBATCH --error=gpu_kernel_job_%j.err
70-
#SBATCH --time=02:00:00
71-
#SBATCH --nodes=1
72-
#SBATCH --ntasks=1
73-
#SBATCH --cpus-per-task=4
74-
#SBATCH --mem=16G
75-
#SBATCH --gres=gpu:1
76-
77-
# Load required modules
78-
module load python
79-
module load jupyter
80-
module load cuda
81-
module load cudnn
82-
83-
# Get and use GPU kernel
84-
icrn_manager kernels get Python <gpu_kernel_name> <version>
85-
icrn_manager kernels use Python <gpu_kernel_name> <version>
86-
87-
# Run GPU-enabled script
88-
python my_gpu_analysis.py
89-
90-
.. note::
91-
Adjust resource requirements (memory, CPU, GPU) based on your specific needs.

0 commit comments

Comments
 (0)
⚡