Merge branch 'main' into chore/10-gh-actions

Author: blms

CHANGELOG.rst

@@ -1,15 +1,18 @@
 Change & Version Information
 ============================
 
-The following is a summary of changes and improvements to
-:mod:`neuxml`.  New features in each version should be listed, with
-any necessary information about installation or upgrade notes.
-
-0.1.0
+1.0.0
 -----
 
-* Fork `eulxml <https://github.com/emory-libraries/eulxml>`_
-  package from version `1.1.3` with the new name `neuxml`
+Initial release of `neuxml.` This is a hard fork of `eulxml <https://github.com/emory-libraries/eulxml>`_
+version 1.1.3, which drops Django forms integration and support for older versions of Python.
+
+* Rename `eulxml` to `neuxml` throughout
 * Remove `forms` submodule and drop Django requirements
-* Add GitHub workflow for pypi publication
+* Add GitHub Actions workflow for PyPI publication
 * Update for Python 3.12 compatibility
+* Store a default XML catalog and schemas in codebase and released package
+* Migrate build environment to Hatchling with `pypackage.toml`
+* Migrate test environment from Nose to Pytest
+
+For a record of the pre-existing functionality, refer to the `eulxml changelog <https://github.com/emory-libraries/eulxml/blob/master/CHANGELOG.rst>`_.

DEVNOTES.rst

@@ -59,40 +59,3 @@ in that module, and store the schemas and catalog file in the subdirectory
 
 To specify other remote schema URLs and catalog locations, use the provided
 keyword arguments ``xsd_schemas``, ``xmlcatalog_dir``, and ``xmlcatalog_file``.
-
-Migration from ``eulxml``
--------------------------
-
-A convenience script has been included under ``scripts/`` called
-``migrate_eulxml.py`` to migrate your project from ``eulxml`` to ``neuxml``,
-which will replace any usage of the package name in all ``.py`` files in the
-passed directory and subdirectories. After upgrading to Python 3.12+ and
-updating the package in your project's dependencies, you can run the script::
-
-    python scripts/migrate_eulxml.py /path/to/your/project
-
-In addition to renaming the package from ``eulxml`` to ``neuxml``, the usage
-of indirect imports from ``neuxml.xmlmap`` has been removed, and definitions
-must be imported directly from its submodules instead. The migration script
-will also attempt to make these changes automatically.
-
-For example, imports have changed from this style:
-
-.. code-block:: python
-
-    from neuxml import xmlmap
-
-    xmlmap.XmlObject
-    xmlmap.Field
-
-to this style:
-
-.. code-block:: python
-
-    from neuxml.xmlmap import core, fields
-
-    core.XmlObject
-    fields.Field
-
-Class imports are also acceptable (e.g. ``from neuxml.xmlmap.core import
-XmlObject``).

APACHE-LICENSE-2.0.txt LICENSEAPACHE-LICENSE-2.0.txt renamed to LICENSE

@@ -0,0 +1,59 @@
+Migration from ``eulxml``
+-------------------------
+
+A convenience script called ``migrate-eulxml`` has been included in order
+to migrate your project from ``eulxml`` to ``neuxml``, which will replace
+any usage of the package name in all ``.py`` files in the passed directory
+and subdirectories. After upgrading to Python 3.12+ and installing the
+updated package, you can run the script::
+
+    migrate-eulxml /path/to/your/project
+
+Or you can run the script directly without installing the package::
+
+    python neuxml/utils/migrate_eulxml.py /path/to/your/project
+
+In addition to renaming the package from ``eulxml`` to ``neuxml``, the usage
+of indirect imports from ``neuxml.xmlmap`` has been removed, and definitions
+must be imported directly from its submodules instead. The migration script
+will also attempt to make these changes automatically, but may not work in
+all cases, so be sure to check your imports manually as well.
+
+For example, imports have changed from this style:
+
+.. code-block:: python
+
+    from neuxml import xmlmap
+
+    xmlmap.XmlObject
+    xmlmap.Field
+
+to this style:
+
+.. code-block:: python
+
+    from neuxml.xmlmap import core, fields
+
+    core.XmlObject
+    fields.Field
+
+Class and function imports are also acceptable (e.g. ``from neuxml.xmlmap.core
+import XmlObject``).
+
+----
+
+If you would like to automatically replace all instances of ``eulxml`` with
+``neuxml`` but otherwise complete the import migration manually, you may use
+the following one-line shell script. 
+
+On MacOS:
+
+.. code-block:: shell
+
+   find . -name '*.py' -print0 | xargs -0 sed -i '' -e 's/eulxml/neuxml/g'
+
+Or on other Unix-based operating systems:
+
+.. code-block:: shell
+
+   find . -name '*.py' -print0 | xargs -0 sed -i 's/eulxml/neuxml/g'

MIGRATION.rst

@@ -2,60 +2,71 @@
 neuxml
 ======
 
-**package**
-  .. image:: https://img.shields.io/pypi/v/neuxml.svg
-    :target: https://pypi.python.org/pypi/neuxml
-    :alt: PyPI
+.. image:: https://img.shields.io/pypi/v/neuxml.svg
+  :target: https://pypi.python.org/pypi/neuxml
+  :alt: PyPI
 
-  .. image:: https://img.shields.io/github/license/Princeton-CDH/neuxml.svg
-    :alt: License
+.. image:: https://img.shields.io/github/license/Princeton-CDH/neuxml.svg
+  :alt: License
 
-  .. image:: https://img.shields.io/pypi/dm/neuxml.svg
-    :alt: PyPI downloads
+.. image:: https://img.shields.io/pypi/pyversions/neuxml
+  :alt: PyPI - Python Version
 
-neuxml is a `Python <http://www.python.org/>`_ module that provides
-utilities and classes for interacting with XML that allow the
-definition of re-usable XML objects that can be accessed, updated and
-created as standard Python types.
+.. image:: https://img.shields.io/pypi/dm/neuxml.svg
+  :alt: PyPI downloads
 
-**neuxml.xpath** provides functions and classes for parsing XPath
-expressions using `PLY <http://www.dabeaz.com/ply/>`_.
+`neuxml` is a Python library that provides utilities and classes for
+object-oriented access to XML. `neuxml` makes it possible to define reusable
+python classes to access, update, and create XML content as standard Python types.
 
-**neuxml.xmlmap** makes it easier to map XML to Python objects in a
-nicer, more pythonic and object-oriented way than typical DOM access
-usually provides.  XML can be read, modified, and even created from
-scratch (in cases where the configured XPath is simple enough to
-determine the nodes that should be constructed).
+**neuxml.xmlmap** makes it possible to map XML content to Python objects in a
+pythonic and object-oriented way, which is easier to use than typical DOM access.
+With the `neuxml.xmlmap.core.XmlObject` class, XML can be read, modified, and even
+created from scratch in some cases, as long as the configured XPath can
+be used to construct new nodes.
 
-Dependencies
-============
-
-**neuxml** depends on `PLY <http://www.dabeaz.com/ply/>`_ and `lxml
-<http://lxml.de/>`_.
+Object-oriented access depends on **neuxml.xpath**, which provides functions and
+classes for parsing XPath expressions using `PLY <http://www.dabeaz.com/ply/>`_.
 
+Installation
+============
 
-Contact Information
-===================
+We recommend using pip to install the officially released versions from PyPI:
 
-**eulxml** was created by the `Center for Digital Humanities at Princeton <https://cdh.princeton.edu/>`_.
+```console
+pip install neuxml
+```
 
-cdhdevteam@princeton.edu
+It is also possible to install directly from GitHub. Use a branch or tag name,
+e.g. `@develop` or `@1.0` to install a specific tagged version or branch.
 
+```console
+pip install git+https://github.com/Princeton-CDH/neuxml.git@develop#egg=neuxml
+```
 
 License
 =======
+
 **neuxml** is distributed under the Apache 2.0 License.
 
 
 Development History
 ===================
 
-This codebase was forked from a package called **eulxml**, originally developed
-by Emory University Libraries. To see and interact with the full development
-history of **eulxml**, see `eulxml <https://github.com/emory-libraries/eulxml>`_.
+`neuxml` is a hard fork of `eulxml <https://github.com/emory-libraries/eulxml>`_,
+which was originally developed by Emory University Libraries from 2011-2016.
+`neuxml` has been updated for compatibility with current versions of Python
+and drops the support for Django form integration. The full development history
+for the `eulxml` package is available at the original repository:  https://github.com/emory-libraries/eulxml
+
+
+Technical documentation
+=======================
 
+For instructions on developer setup, unit testing, XML catalog file management,
+and migrating from `eulxml`, refer to ``DEVNOTES.rst``.
 
-Developer instructions
-======================
+Migration from ``eulxml``
+=========================
 
-For development instructions and notes, see ``DEVNOTES.rst``.
+If migrating from a previous ``eulxml`` installation, see ``MIGRATION.rst``.

README.rst

@@ -19,7 +19,7 @@
 from importlib import resources
 from contextlib import ExitStack
 
-__version__ = "0.1.0"
+__version__ = "1.0.0"
 
 # Paths for XML catalog file & directory
 

neuxml/__init__.py

@@ -81,31 +81,34 @@ def fix_xmlmap(code):
     return code
 
 
-"""Handle command line args"""
-
-parser = argparse.ArgumentParser()
-parser.add_argument(
-    "project_path",
-    type=pathlib.Path,
-    help="The path to the root of a project using eulxml",
-)
-
-args = parser.parse_args()
-
-
 def is_valid(path):
     """consider hidden files/folders and venv site-packages invalid for migration"""
     return not any(
         (part for part in path.parts if part.startswith(".") or "site-packages" in part)
     )
 
 
-# migrate every valid python file in project path and subdirectories
-for filepath in args.project_path.glob("**/*.py"):
-    if is_valid(filepath):
-        try:
-            code = filepath.read_text(encoding="utf-8")
-            migrated_code = fix_xmlmap(re.sub("eulxml", "neuxml", code))
-            filepath.write_text(migrated_code, encoding="utf-8")
-        except Exception as e:
-            print(f"Failed to process {filepath}: {e}")
+def main(arg_list=None):
+    """Handle command line arguments and run migration functions"""
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "project_path",
+        type=pathlib.Path,
+        help="The path to the root of a project using eulxml",
+    )
+
+    args = parser.parse_args(arg_list)
+
+    # migrate every valid python file in project path and subdirectories
+    for filepath in args.project_path.glob("**/*.py"):
+        if is_valid(filepath):
+            try:
+                code = filepath.read_text(encoding="utf-8")
+                migrated_code = fix_xmlmap(re.sub("eulxml", "neuxml", code))
+                filepath.write_text(migrated_code, encoding="utf-8")
+            except Exception as e:
+                print(f"Failed to process {filepath}: {e}")
+
+
+if __name__ == "__main__":
+    main()

scripts/migrate_eulxml.py neuxml/utils/migrate_eulxml.pyscripts/migrate_eulxml.py renamed to neuxml/utils/migrate_eulxml.py

@@ -36,6 +36,9 @@ dev = [
     "pre-commit",
 ]
 
+[project.scripts]
+migrate-eulxml = "neuxml.utils.migrate_eulxml:main"
+
 [project.urls]
 Repository = "https://github.com/Princeton-CDH/neuxml"
 Changelog = "https://github.com/Princeton-CDH/neuxml/blob/main/CHANGELOG.rst"

pyproject.toml

@@ -0,0 +1,14 @@
+from eulxml import xmlmap
+from eulxml.xmlmap import teimap
+from eulxml.xmlmap import (
+    IntegerField, NodeField, NodeListField,
+    SimpleBooleanField, StringField, XmlObject,
+    load_xmlobject_from_string, mods
+)
+
+class TeiElement(teimap.Tei):
+    folio_start = xmlmap.StringField("@from")
+    folio_end = xmlmap.StringField("@to")
+
+class CustomObject(xmlmap.XmlObject):
+    xml = xmlmap.load_xmlobject_from_file("file")

test/test_utils/fixtures/old_import.txt

@@ -0,0 +1,64 @@
+from io import StringIO
+import pathlib
+import re
+import sys
+from unittest.mock import patch
+
+from neuxml.utils.migrate_eulxml import is_valid, main
+
+
+class TestMigrateEulxml:
+    def test_is_valid(self):
+        # hidden files/folders, site-packages should not be valid
+        bad_path = pathlib.Path("/project/.git/example.py")
+        assert not is_valid(bad_path)
+        bad_path = pathlib.Path(
+            "/project/venv/lib/python3.12/site-packages/pytest/__main__.py"
+        )
+        assert not is_valid(bad_path)
+        good_path = pathlib.Path("/project/example.py")
+        assert is_valid(good_path)
+
+    def test_main(self):
+        """integration tests for the migration script, which
+        should hit every case and thus every helper function"""
+        mock_args = ["/fake/path"]
+        mock_file = "test/test_utils/fixtures/old_import.txt"
+        with patch.object(pathlib.Path, "glob") as mock_glob:
+            mock_glob.return_value = [pathlib.Path(mock_file)]
+            with patch.object(pathlib.Path, "write_text") as mock_write_text:
+                main(mock_args)
+                output_text = mock_write_text.call_args[0][0]
+
+                # should replace all eulxml with neuxml
+                assert "eulxml" not in output_text
+                assert "neuxml" in output_text
+
+                # should replace `import xmlmap` with submodule imports
+                assert "import xmlmap" not in output_text
+                assert "from neuxml.xmlmap import core, fields" in output_text
+
+                # should categorize imports correctly: core, fields, module
+                assert re.search(
+                    r"from neuxml\.xmlmap\.core import.*XmlObject", output_text
+                )
+                assert re.search(
+                    r"from neuxml\.xmlmap\.fields import.*NodeField", output_text
+                )
+                assert "from neuxml.xmlmap import mods" in output_text
+
+                # should replace inline references appropriately
+                assert "xmlmap.StringField" not in output_text
+                assert "fields.StringField" in output_text
+                assert "xmlmap.XmlObject" not in output_text
+                assert "core.XmlObject" in output_text
+                assert "xmlmap.load_" not in output_text
+                assert "core.load_" in output_text
+
+                # should print exceptions with file path
+                mock_write_text.side_effect = Exception("msg")
+                with patch.object(sys, "stdout", new_callable=StringIO) as mock_stdout:
+                    main(mock_args)
+                    assert (
+                        f"Failed to process {mock_file}: msg" in mock_stdout.getvalue()
+                    )