Update from GeoServer translate work #12
Conversation
- Add encoding='utf-8' to all file read operations in translate.py - Add error handling with errors='replace' for graceful UTF-8 fallbacks - Fix UnicodeDecodeError on Windows systems with international content - Addresses CP1252 vs UTF-8 encoding conflicts in RST scanning
Add missing import errno jodygarnett#11
This reverts commit 87323b7.
- Add encoding='utf-8' to all file read operations in translate.py - Add error handling with errors='replace' for graceful UTF-8 fallbacks - Fix UnicodeDecodeError on Windows systems with international content - Addresses CP1252 vs UTF-8 encoding conflicts in RST scanning
- Detect indented list-table directives (indentation > 0) - Remove indentation to enable proper Pandoc conversion - Add HTML comments documenting indentation removal - Fail migration if de-indentation impossible (validation) - Process all files during migration, not just those in reports Features: - detect_nested_tables() finds all indented tables with line numbers - deindent_nested_table() removes indentation and adds documentation - Integration in preprocess_rst() before other block directives - Proper detection avoiding false positives in code examples - Handles deep nesting (9+ spaces) correctly Testing: - test_nested_table_simple.rst: single 3-space indented table - test_nested_table_multiple.rst: 3 tables at various indent levels - test_nested_table_deep.rst: deep nesting in admonition - test_nested_table_mixed.rst: mixed indented/non-indented tables - test_nested_table_code_example.rst: code blocks not affected - test_non_indented_table.rst: non-indented tables unchanged All test cases pass with correct table detection and indentation removal.
Phases complete: - Phase 1: Clean branch setup - Phase 2: Nested table detection implementation - Phase 3: De-indentation with comments - Phase 4: Integration into preprocessing pipeline - Phase 5: Comprehensive test suite (all passing) - Phase 6: Real-world testing on GeoServer docs Bugs fixed: - Code block detection false positives - Directive line handling in nested contexts Testing status: - 6 test files created and passing - Real GeoServer documentation tested successfully - Nested tables correctly detected and processed
Phase 7 deliverables: - Created NESTED_TABLE_FIX.md (3600+ word technical documentation) - Updated CHANGES with v0.6.2 entry - Bumped version to 0.6.2 in __init__.py - Created tests/test_nested_tables.py with 17 comprehensive tests - All unit tests passing (100%) Test coverage: - 9 tests for detect_nested_tables() function - 5 tests for deindent_nested_table() function - 2 integration tests - 1 edge case test Documentation includes: - Problem analysis and root cause - Solution architecture - Implementation details - Testing results - Troubleshooting guide - Future improvement suggestions Ready for Phase 8: Final merge to main branch
Documented steps for Phase 8: - Verify branch status - Clean up temporary files - Create PR and merge to geoserver-minimal - Tag version 0.6.2 - Document release notes Branch readiness: All code complete and tested Commits on branch: 4 commits from f4e1c22 baseline Ready for production merge
|
@peter-afrigis can we start to merge this bit by bit, rather than have such a long standing thing? |
48 hours later, I am still fighting nested tables. |
|
When we get into things like nested tables, it starts to become worthwhile to edit the RST origional. How many nested tables are there? And if you are fighting hard it is "faster" to steal a page example over to this project for rapid troubleshooting. |
…rs(), collect_paths(), collect_path()
…to enable migrate command execution
… [STDERR-*]), keep [v0.6.4] preprocessing markers
- Fixed critical indentation bug in _safe_list_table() function (line 1838)
- Block content now properly indented under directive (3 spaces minimum)
- Removed double {indent} from option formatting
- Ensures pandoc correctly recognizes list-table directives as table structures
- All list-table blocks now render as proper tables, not empty patterns
This fix complements v0.6.11 nested-table fix:
- v0.6.11: De-indents nested list-tables (indent > 0)
- v0.6.13: Indents top-level list-table content under directive
…_rst() and preprocess_rst()
…rt for Annotated; add typing-extensions dependency and tests
|
@peter-afrigis for https://geoserver.org/geoserver/en/user/extensions/authkey/#installation page the variables are not being filled in like I think that is done using macros (which makes pages slower then sphinx) so I turned it off by default. To enable it for a page you have to add a snippet to the top of the page. Generating locally I see the page does contain: ---
render_macros: true
---So I expect the substitutions variables changed in sphinx over the time period. I was working to make main So the valid substations should be:
And using that we should be able to build up the download macros, reading the logic config also has:
config.py uses these values for external link generation: |
|
There is some logic here for config handling external links: And the example But I do not see a translate.yml for external links in the geoserver user docs. |
|
I tried creating a local translate.yml with extlinks and running locally but did not get anywhere. Let's catch up, and if needed remove this stuff. |
Peter has been working on the translate script filling in gaps for docs.geoserver.org - which is much harder than other projects attempted.