Skip to content

Adds check to make sure we only populate the llms.txt when we manage it.#22262

Merged
leonidasmi merged 13 commits intofeature/llms-txtfrom
557-make-sure-that-we-dont-deleteoverwrite-llmstxt-files-when-were-not-supposed-to
May 16, 2025
Merged

Adds check to make sure we only populate the llms.txt when we manage it.#22262
leonidasmi merged 13 commits intofeature/llms-txtfrom
557-make-sure-that-we-dont-deleteoverwrite-llmstxt-files-when-were-not-supposed-to

Conversation

@thijsoo
Copy link
Copy Markdown
Contributor

@thijsoo thijsoo commented May 13, 2025

Context

  • Adds check to make sure we only populate the llms.txt when we manage it.

Summary

This PR can be summarized in the following changelog entry:

  • Adds check to make sure we only populate the llms.txt when we manage it.

Relevant technical choices:

  • When the llms.txt file is not managed by us (because it got created/edited manually), we don't delete the cron job, because if the file ever gets deleted, we want to automatically re-generate it.

Test instructions

Test instructions for the acceptance test before the PR gets merged

This PR can be acceptance tested by following these steps:

Test pre-existing LLMs.txt files

  • In a fresh site, manually create a LLMs.txt file
  • Install this RC and enable the LLMs.txt feature
  • Make sure that that the LLMs.txt file was not overwritten and stayed as it was when you created it manually
  • Run the wpseo_llms_txt_population cron and again confirm that the file didnt change.
  • Disable the feature and confirm that the file didnt get deleted.

Test manually tweaking Yoast's LLMs.txt file
To properly test this

  • disable the llms.txt feature in the settings
  • remove the file if you have it
  • delete the wpseo_llms_txt_content_hash option from the db
  • and possibly delete the wpseo_llms_txt_population cron if it exists
    After you did the above:
  • Enable the feature and make sure an llms.txt is created. Open it by going to /llms.txt and take note of the content.
  • Make sure the cron is registered with the WP Crontrol plugin.
  • Change the tagline of your site on /wp-admin/options-general.php.
  • Hit Run now on the cron and make sure its updated in the llms.txt file.
  • Open the file on your file system and edit a character. -> this will mean we are no longer managing your llms.txt file.
  • Change your tagline again. and run the cron again. Make sure the file is not updated.
  • Disable the feature in the settings and make sure the file is not deleted.
  • Re-enable the feature and make sure the file is not updated.
  • Remove the file from the file system and run the cron again or disable and re-enable the feature.
  • Make sure its created again -> from this moment we are managing it again.
  • Disable the feature and make sure its deleted again.
  • Enable the feature once more and again make sure it's created.

Test having no access to filesystem

  • disable the llms.txt feature in the settings
  • remove the file if you have it
  • delete the wpseo_llms_txt_content_hash option from the db
  • and possibly delete the wpseo_llms_txt_population cron if it exists
  • create a llms.txt file manually in your site's root
  • Add the following snippet to your site:

add_filter( 'filesystem_method', 'disable_filesystem_method' );

function disable_filesystem_method( $url ) {
	return false;
}
  • enable the LLMs.txt feature
  • Make sure that that the LLMs.txt file was not overwritten and stayed as it was when you created it manually
  • Run the wpseo_llms_txt_population cron and again confirm that the file didnt change.
  • Disable the feature and confirm that the file didnt get deleted.
  • Manually delete the file and re-enable the feature
  • Confirm that the file didnt get created
  • Remove the filter and run the wpseo_llms_txt_population cron
  • Confirm that now the file has been created
  • Add the filter again and change the tagline of your site
  • Run the wpseo_llms_txt_population cron and confirm that your file didnt change
  • Remove the filter and run the cron again
  • Confirm that the file did change now and has the new tagline.

Relevant test scenarios

  • Changes should be tested with the browser console open
  • Changes should be tested on different posts/pages/taxonomies/custom post types/custom taxonomies
  • Changes should be tested on different editors (Default Block/Gutenberg/Classic/Elementor/other)
  • Changes should be tested on different browsers
  • Changes should be tested on multisite

Test instructions for QA when the code is in the RC

  • QA should use the same steps as above.

QA can test this PR by following these steps:

Impact check

This PR affects the following parts of the plugin, which may require extra testing:

UI changes

  • This PR changes the UI in the plugin. I have added the 'UI change' label to this PR.

Other environments

  • This PR also affects Shopify. I have added a changelog entry starting with [shopify-seo], added test instructions for Shopify and attached the Shopify label to this PR.

Documentation

  • I have written documentation for this change. For example, comments in the Relevant technical choices, comments in the code, documentation on Confluence / shared Google Drive / Yoast developer portal, or other.

Quality assurance

  • I have tested this code to the best of my abilities.
  • During testing, I had activated all plugins that Yoast SEO provides integrations for.
  • I have added unit tests to verify the code works as intended.
  • If any part of the code is behind a feature flag, my test instructions also cover cases where the feature flag is switched off.
  • I have written this PR in accordance with my team's definition of done.
  • I have checked that the base branch is correctly set.

Innovation

  • No innovation project is applicable for this PR.
  • This PR falls under an innovation project. I have attached the innovation label.
  • I have added my hours to the WBSO document.

Fixes #

@thijsoo thijsoo added the changelog: non-user-facing Needs to be included in the 'Non-userfacing' category in the changelog label May 13, 2025
@leonidasmi leonidasmi added this to the feature/llms-txt milestone May 15, 2025
Copy link
Copy Markdown
Contributor

@leonidasmi leonidasmi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CR: Some minor comments.

But also, I was checking the implementation and a weird thing is happening when you disable the llms.txt feature.

When that feature is toggled off, the can_regenerate() is run to see if we should delete the file and when that checked the llms_txt_content_hash option, it was an empty string even though we have content in the file. Can you look into that as well?

Comment thread src/llms-txt/infrastructure/file/wordpress-file-system-adapter.php Outdated
Comment thread src/llms-txt/infrastructure/file/wordpress-llms-txt-permission-gate.php Outdated
Comment thread src/llms-txt/infrastructure/file/wordpress-llms-txt-permission-gate.php Outdated
Comment thread src/llms-txt/infrastructure/file/wordpress-llms-txt-permission-gate.php Outdated
Base automatically changed from 606-add-content-in-the-llmstxt-file to feature/llms-txt May 15, 2025 07:56
@thijsoo thijsoo force-pushed the 557-make-sure-that-we-dont-deleteoverwrite-llmstxt-files-when-were-not-supposed-to branch from cb30e43 to e36f1a9 Compare May 15, 2025 10:10
@coveralls
Copy link
Copy Markdown

coveralls commented May 15, 2025

Pull Request Test Coverage Report for Build a3fd93ef8b3de5980c14439fb56ac9dcb33863bc

Details

  • 0 of 55 (0.0%) changed or added relevant lines in 4 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.5%) to 52.334%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/llms-txt/application/file/commands/populate-file-command-handler.php 0 6 0.0%
src/llms-txt/application/file/commands/remove-file-command-handler.php 0 6 0.0%
src/llms-txt/infrastructure/file/wordpress-llms-txt-permission-gate.php 0 11 0.0%
src/llms-txt/infrastructure/file/wordpress-file-system-adapter.php 0 32 0.0%
Totals Coverage Status
Change from base Build 8ea7ca73e94dd24480d92b94cb862bd63761d666: -0.5%
Covered Lines: 29150
Relevant Lines: 56848

💛 - Coveralls

…s-when-were-not-supposed-to' of github.com:Yoast/wordpress-seo into 557-make-sure-that-we-dont-deleteoverwrite-llmstxt-files-when-were-not-supposed-to
@leonidasmi
Copy link
Copy Markdown
Contributor

@thijsoo I've also pushed a couple of commits, can you review them too?

leonidasmi and others added 5 commits May 15, 2025 15:18
…s-when-were-not-supposed-to' of github.com:Yoast/wordpress-seo into 557-make-sure-that-we-dont-deleteoverwrite-llmstxt-files-when-were-not-supposed-to
…eoverwrite-llmstxt-files-when-were-not-supposed-to
Copy link
Copy Markdown
Contributor

@leonidasmi leonidasmi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CR + Acceptance is ✅

I'm merging this but I've also pushed this commit to cover another edge case first, @thijsoo can you give it a quick second look?

@leonidasmi leonidasmi merged commit 0453727 into feature/llms-txt May 16, 2025
25 checks passed
@leonidasmi leonidasmi deleted the 557-make-sure-that-we-dont-deleteoverwrite-llmstxt-files-when-were-not-supposed-to branch May 16, 2025 07:07
@leonidasmi leonidasmi mentioned this pull request May 21, 2025
18 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog: non-user-facing Needs to be included in the 'Non-userfacing' category in the changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants