MDEV-25529 Auto-create: Pre-existing historical data is not partitioned as specified by ALTER

[midenok]

Adds logic into prep_alter_part_table() for AUTO to check the history
range (vers_get_history_range()) and based on (max_ts - min_ts)
difference compute the number of created partitions and set STARTS
value to round down min_ts value (vers_set_starts()) if it was not
specified by user or if the user specified it incorrectly. In the
latter case it will print warning about wrongly specified user value.

In case of fast ALTER TABLE, f.ex. when partitioning already exists,
the above logic is ignored unless FORCE clause is specified. When user
specifies partition list explicitly the above logic is ignored even
with FORCE clause.

vers_get_history_range() detects if the index can be used for row_end
min/max stats and if so it gets it with ha_index_first() and
HA_READ_BEFORE_KEY (as it must ignore current data). Otherwise it does
table scan to read the stats. There is test_mdev-25529 debug keyword
to check the both and compare results. A warning is printed if the
algorithm uses slow scan.

Field_vers_trx_id::get_timestamp() is implemented for TRX_ID based
versioning to get epoch value. It works in vers_get_history_range()
but since partitioning is not enabled for TRX_ID versioning create
temporary table fails with error, requiring timestamp-based system
fields. This method will be useful when partitioning will be enabled
for TRX_ID which is mostly performance problems to solve.

Static key_cmp was renamed to key_eq to resolve compilation after
key.h was included as key_cmp was already declared there.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[Copilot]

Pull request overview

This PR addresses MDEV-25529 by improving ALTER TABLE behavior for system-versioned tables so that auto-created SYSTEM_TIME partitions correctly account for pre-existing historical rows, and by expanding supported FORCE/partitioning syntax while adding more informative warnings.

Changes:

• Add logic to compute the historical timestamp range (min/max) and derive an appropriate number of history partitions during ALTER with AUTO partitioning.
• Improve ALTER TABLE grammar to accept additional ordering of FORCE, PARTITION BY ..., and REMOVE PARTITIONING.
• Introduce my_timespec_t utilities and new warnings/messages, plus add regression coverage in versioning/partition test suites.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
sql/table.h	Declares TABLE::vers_get_history_range() helper.
sql/sql_table.cc	Implements history-range detection using either an index on ROW END or a scan fallback.
sql/sql_partition.cc	Uses history-range data to adjust AUTO-created versioning partitions during ALTER.
sql/partition_info.h / sql/partition_info.cc	Tracks whether STARTS was explicitly specified; factors out STARTS rounding into vers_set_starts().
include/my_time.h	Introduces my_timespec_t, comparisons, and min/max sentinels.
sql/sql_class.h / sql/sql_class.cc	Adds THD timestamp formatting helper and TimestampString wrapper for warnings.
sql/sql_type.h / sql/sql_type.cc	Adds Temporal_hybrid constructor from my_timespec_t.
sql/sql_time.h / sql/sql_time.cc	Replaces COMBINE macro with reusable interval2sec/interval2usec helpers.
sql/sql_yacc.yy	Extends ALTER TABLE grammar to accept additional FORCE/partitioning command orderings.
sql/share/errmsg-utf8.txt	Renames/extends warning text and adds new warning identifiers for versioning partition behavior.
sql/field.h / sql/field.cc	Adds Field_vers_trx_id::get_timestamp(pos, sec_part) implementation for TRX_ID-based versioning fields.
sql/event_data_objects.cc	Expands doc comment for next-time computation with interval/timezone/DST context.
mysql-test/suite/versioning/t/partition2.test + result	Adds regression test coverage for MDEV-25529 and related warnings.
mysql-test/suite/versioning/r/partition.result	Updates expected warning text output.
mysql-test/main/partition.test + result	Adds coverage for improved FORCE/partitioning syntax parsing.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

This PR improves ALTER TABLE behavior for SYSTEM_TIME partitioning with AUTO by detecting the existing historical data range and auto-adjusting STARTS/partition count so that pre-existing history is partitioned as intended. It also refines FORCE syntax parsing, adds new warning messages, and introduces helper utilities for timestamp/time conversions plus regression tests.

Changes:

• Add TABLE::vers_get_history_range() to compute min/max history timestamps (index-based when possible, else scan with warning) and use it during AUTO partition preparation to set STARTS/partition count.
• Improve ALTER TABLE ... FORCE syntax handling in the parser and update warning messages (more informative STARTS warning + new warnings).
• Add new tests covering MDEV-25529 scenarios (AUTO/FORCE behavior, warnings, slow-scan path, TRX_ID versioning rejection).

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
sql/table.h	Declares TABLE::vers_get_history_range() for versioned+partitioned logic.
sql/sql_table.cc	Implements history range detection using index-first/HA_READ_BEFORE_KEY or table scan with warning.
sql/sql_partition.cc	Uses history range during ALTER PARTITION AUTO to correct STARTS and compute required partition count.
sql/sql_yacc.yy	Extends ALTER TABLE grammar to support improved FORCE positioning.
sql/partition_info.h	Adds starts_clause flag and declares vers_set_starts().
sql/partition_info.cc	Tracks explicit STARTS clause, factors out STARTS rounding into vers_set_starts(), and tweaks default partition setup error handling.
sql/sql_class.h / sql/sql_class.cc	Adds timestamp-to-string helpers and TimestampString wrapper for warning formatting.
sql/sql_type.h / sql/sql_type.cc	Adds Temporal_hybrid(THD*, my_timespec_t) ctor for timestamp formatting.
sql/sql_time.h / sql/sql_time.cc	Adds reusable interval-to-seconds/microseconds helpers and uses them in interval addition code.
include/my_time.h	Introduces my_timespec_t plus comparison helpers/operators and min/max sentinels.
sql/field.h / sql/field.cc	Adds Field_vers_trx_id::get_timestamp() to obtain commit timestamp epoch seconds for TRX_ID versioning.
sql/share/errmsg-utf8.txt	Renames/extends STARTS warning and adds new warnings for wrong STARTS and slow ROW END scan.
mysql-test/*	Adds new versioning partition test + updates expected results for warning text changes.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

This PR improves ALTER TABLE ... PARTITION BY SYSTEM_TIME ... AUTO for system-versioned tables by auto-adjusting STARTS and the number of generated history partitions based on pre-existing history data, and by extending parser support for FORCE placement.

Changes:

• Add history range detection (TABLE::vers_get_history_range) to drive AUTO partition creation (including warning paths for slow scans).
• Add helper logic to round/adjust STARTS (partition_info::vers_set_starts) and emit new warnings when user-provided STARTS is inconsistent with history/query time.
• Extend ALTER TABLE grammar for FORCE + partitioning syntax and add/adjust test coverage and expected results.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
sql/table.h	Exposes TABLE::vers_get_history_range() for partitioning logic.
sql/sql_table.cc	Implements TABLE::vers_get_history_range() using index-first/scan fallback and emits warnings.
sql/sql_partition.cc	Uses history range to recompute AUTO history partition count/STARTS; adds vers_calc_hist_parts().
sql/partition_info.h	Tracks whether STARTS was explicitly specified; adds vers_set_starts() API.
sql/partition_info.cc	Implements vers_set_starts() and updates interval/STARTS warning behavior.
sql/sql_yacc.yy	Accepts additional ALTER TABLE partitioning command orderings (FORCE syntax improvements).
include/my_time.h	Introduces my_timespec_t + comparisons and MIN/MAX constants.
sql/sql_type.h / sql/sql_type.cc	Adds Temporal_hybrid(THD*, my_timespec_t) constructor used for formatting.
sql/sql_class.h / sql/sql_class.cc	Adds THD::timestamp_to_string() and TimestampString helper for warnings.
sql/sql_time.h / sql/sql_time.cc	Replaces COMBINE macro with interval2sec/interval2usec helpers for interval math.
sql/field.h / sql/field.cc	Implements Field_vers_trx_id::get_timestamp() (future TRX_ID work; used by history scanning infra).
sql/share/errmsg-utf8.txt	Adds new warnings (WARN_VERS_*) and updates message text to include timestamps.
sql/event_data_objects.cc	Expands function comment (doc clarification).
mysql-test/suite/versioning/t/partition2.test	Adds new regression tests for MDEV-25529 behavior.
mysql-test/suite/versioning/r/partition2.result	Expected output for the new versioning partition tests.
mysql-test/suite/versioning/r/partition.result	Updates expected warning text format.
mysql-test/main/partition.test	Adds tests validating improved ALTER TABLE FORCE syntax.
mysql-test/main/partition.result	Expected output for FORCE syntax tests.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds support for AUTO history partition creation during ALTER TABLE ... PARTITION BY SYSTEM_TIME by detecting existing historical timestamp range and adjusting STARTS/partition count accordingly, along with improved FORCE placement in ALTER syntax and new warnings/tests.

Changes:

• Implement history-range detection (TABLE::vers_get_history_range) to drive AUTO partition creation and STARTS correction.
• Add helper time/timestamp utilities (my_timespec type, interval helpers, timestamp-to-string) and new warnings for user feedback.
• Extend SQL grammar and add regression tests covering FORCE syntax and versioning partition AUTO behaviors.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
sql/table.h	Declares TABLE::vers_get_history_range() API for partitioning/versioning logic.
sql/sql_yacc.yy	Expands ALTER TABLE grammar to allow partitioning/remove partitioning before alter_list (FORCE syntax improvements).
sql/sql_type.h	Adds Temporal_hybrid ctor from my_timespec_t for timestamp formatting.
sql/sql_type.cc	Implements new Temporal_hybrid(THD*, my_timespec_t) constructor.
sql/sql_time.h	Introduces interval2sec() / interval2usec() helpers used by interval arithmetic.
sql/sql_time.cc	Switches date interval arithmetic to the new interval helpers (removes local COMBINE macro).
sql/sql_table.cc	Includes key handling header and renames comparator to avoid collision; adds TABLE::vers_get_history_range() implementation.
sql/sql_partition.cc	Adds AUTO history partition calculation and wiring into ALTER partition preparation.
sql/sql_class.h	Adds THD::timestamp_to_string() and TimestampString helper for warning formatting.
sql/sql_class.cc	Implements THD::timestamp_to_string().
sql/share/errmsg-utf8.txt	Adds/renames warnings for STARTS beyond query time, wrong STARTS, and slow scan.
sql/partition_info.h	Tracks whether STARTS was explicitly specified and exposes vers_set_starts().
sql/partition_info.cc	Implements vers_set_starts(), improves OOM error path, and sets starts_clause.
sql/field.h	Declares Field_vers_trx_id::get_timestamp() override.
sql/field.cc	Implements TRX_ID-based get_timestamp() for future TRX_ID versioning support.
sql/event_data_objects.cc	Clarifies documentation for interval scheduling behavior.
mysql-test/suite/versioning/t/partition2.test	Adds new test coverage for MDEV-25529 auto partitioning + FORCE behaviors.
mysql-test/suite/versioning/r/partition2.result	Captures expected results/warnings for the new versioning partition tests.
mysql-test/suite/versioning/r/partition.result	Updates expected warning text due to improved STARTS warning message.
mysql-test/main/partition.test	Adds tests for ALTER TABLE FORCE syntax placements.
mysql-test/main/partition.result	Captures expected output/errors for new FORCE syntax tests.
include/my_time.h	Adds my_timespec_t and comparison helpers/operators for timestamp ranges.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

This PR enhances SYSTEM_TIME partitioning AUTO behavior during ALTER TABLE so that existing historical rows are partitioned according to the requested interval/STARTS settings, and expands parser support for FORCE placement in ALTER TABLE partitioning clauses.

Changes:

• Add TABLE::vers_get_history_range() to detect historical min/max ROW END timestamps (index-based when possible, otherwise scan with warning) and use it to auto-size partitions and auto-adjust STARTS for history.
• Extend ALTER TABLE grammar to allow FORCE-style partitioning/removal syntax variants (partitioning/removal before or after alter_list).
• Add new warnings/messages and extensive MTR coverage for AUTO/FORCE/history edge cases.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
sql/table.h	Declares TABLE::vers_get_history_range() API.
sql/sql_table.cc	Implements vers_get_history_range() and resolves key_cmp naming conflict.
sql/sql_partition.cc	Uses history range to compute AUTO partition count and to correct/derive STARTS (with warnings); adds vers_calc_hist_parts().
sql/partition_info.h	Adds starts_clause tracking and vers_set_starts() declaration.
sql/partition_info.cc	Tracks explicit STARTS clause, factors out STARTS rounding into vers_set_starts(), and improves OOM error path in default partition setup.
include/my_time.h	Introduces my_timespec_t plus min/max sentinels and comparisons.
sql/sql_type.h	Declares Temporal_hybrid(THD*, my_timespec_t) ctor.
sql/sql_type.cc	Implements Temporal_hybrid ctor from (sec,usec) for formatting.
sql/sql_class.h	Adds THD::timestamp_to_string() overloads and TimestampString helper.
sql/sql_class.cc	Implements THD::timestamp_to_string().
sql/sql_time.h	Adds interval2sec() / interval2usec() helpers.
sql/sql_time.cc	Uses interval2usec() instead of local COMBINE macro.
sql/sql_yacc.yy	Allows partitioning/removal clauses to appear before/after alter_list to support improved FORCE syntax.
sql/share/errmsg-utf8.txt	Renames/extends STARTS warning text and adds new versioning partitioning warnings.
sql/field.h	Declares Field_vers_trx_id::get_timestamp() override.
sql/field.cc	Implements Field_vers_trx_id::get_timestamp() for TRX_ID versioning.
sql/event_data_objects.cc	Expands comment explaining interval scheduling logic.
mysql-test/suite/versioning/t/partition2.test	New test suite covering AUTO/FORCE/history range behavior and warnings.
mysql-test/suite/versioning/r/partition2.result	Expected results for new versioning partition AUTO tests.
mysql-test/suite/versioning/r/partition.result	Updates expected warning text for STARTS beyond query timestamp.
mysql-test/main/partition.test	Adds tests for improved ALTER TABLE ... FORCE partitioning syntax acceptance/rejection.
mysql-test/main/partition.result	Expected output for new FORCE syntax tests.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.