[Bug]: AI Generator "Generate with AI" fails after site domain change

[chrisdavidmiles]

Prerequisites

• I've read and understood the contribution guidelines.
• I've searched for any related issues and avoided creating a duplicate issue.

Please give us a description of what happened

Background Information

When a WordPress site's URL changes after the AI Generator tokens have been initially requested (e.g., migrating from a hosting provider's temporary/staging URL to a production domain), the AI Generator's "Generate with AI" feature fails. The Yoast API continues to use the old callback URLs that were registered during the initial token_request(), and token_refresh() never re-registers updated callback URLs. This issue is not specific to hosting providers since provisioning WordPress sites at temporary staging URLs before customers point their real domains is a common pattern. (But I'm replicating this at Bluehost.)

In Token_Manager::token_request(), the callback_url and refresh_callback_url are sent to the Yoast AI service. These URLs are derived from get_rest_url() at the time of the initial request. However, Token_Manager::token_refresh() does not re-send callback URLs, it only sends the code_challenge. The Yoast API reuses the originally registered callback URLs for all subsequent token delivery.

If the site URL changes after the initial token request (e.g., from abc.def.staging-site.net to customer-website.com), the API's stored callback URLs point to the old domain. The TLS handshake fails (the cert covers the new domain, not the old one), and the callback never lands. See src/ai/authorization/application/token-manager.php — token_request() (line ~207) sends callback URLs; token_refresh() (line ~250) does not and src/ai/generator/infrastructure/wordpress-urls.php — builds callback URLs from get_rest_url().

Proposed fix

Add a self-healing URL mismatch detection to Token_Manager::get_or_request_access_token():

1. Store the callback URL used during the last successful token_request() in the wpseo option (ai_generator_callback_url)
2. Before using or refreshing existing tokens, compare the stored URL to the current get_rest_url()-derived URL
3. If they differ, delete the user's stale tokens and force a fresh token_request() with the correct callback URLs
4. For existing sites upgrading (where no stored URL exists yet), fall back to comparing the existing home_url option against the current get_home_url()

This approach:

• Self-heals all existing affected sites on their next "Generate with AI" click
• Prevents future staleness after any domain change
• Requires no server-side API changes
• Requires no manual intervention or CLI access
• Follows the existing pattern used by Indexable_HomeUrl_Watcher for detecting URL changes

Step-by-step reproduction instructions

1. Install WordPress at a temporary/staging URL (e.g., abc.def.mybluehost.me) — this is the default provisioning behavior for Bluehost and many other managed WordPress hosts.
2. Install and activate Yoast SEO Premium
3. Create a post, add a focus keyphrase, and click "Generate with AI" — this triggers the initial token_request() which registers callback URLs using the staging domain.
4. Change the WordPress site URL to a production domain (e.g., example.com) via Settings > General or through the hosting provider's domain mapping tool.
5. Create or edit a post and click "Generate with AI" again
6. The feature fails — the Yoast API attempts to call back to the old staging URL, which either no longer resolves or fails TLS validation.

Note: Yoast SEO does hook into update_option_home (via Indexable_HomeUrl_Watcher) to detect URL changes, but this only fires when the home option is updated through WordPress's PHP update_option() API. If a hosting providers change site URLs using wp search-replace or direct MySQL updates, the option hooks will not fire so Yoast will not detect the change.

Expected results

1. After a site URL change, the AI Generator should detect that the registered callback URLs are stale and automatically re-register with the current site URL
2. The "Generate with AI" feature should work without requiring manual intervention after a domain migration

Actual results

1. The AI Generator silently fails because token_refresh() reuses the old callback URLs stored on the Yoast API server
2. The Yoast API attempts to deliver tokens to the old staging URL (e.g., temp123.mybluehost.me/wp-json/yoast/v1/ai_generator/refresh_callback)
3. The callback fails (DNS resolution failure, TLS mismatch, or 404) and the user receives no AI suggestions
4. There is no self-recovery mechanism — the feature remains broken until the user manually clears their AI generator tokens from the database

Screenshots, screen recording, code snippet

---

The issue may be in the request/refresh flow asymmetry:

// token_request() sends callback URLs (token-manager.php ~line 207):
$request_body = [
    'callback_url'         => $this->urls->get_callback_url(),
    'refresh_callback_url' => $this->urls->get_refresh_callback_url(),
    // ...
];

// token_refresh() does NOT send callback URLs (token-manager.php ~line 250):
$request_body = [
    'code_challenge' => \hash( 'sha256', $code_verifier->get_code() ),
    // No callback URLs — API reuses the stale ones from token_request()
];

Which editor is affected (or editors)

• Block Editor
• Gutenberg Editor
• Elementor Editor
• Classic Editor
• Other (please specify in additional info)

Which browser is affected (or browsers)

• Chrome
• Firefox
• Safari
• Other (please specify in additional info)

Device you are using

N/A

Operating system

N/A

PHP version

8.4

WordPress version

6.9.4

WordPress Theme

No response

Yoast SEO version

27.3

Gutenberg plugin version (if relevant)

No response

Elementor plugin version (if relevant)

No response

Classic Editor plugin version (if relevant)

No response

Relevant plugins in case of a bug

No response