Clean up the DB from now-deleted or now-made-private pages

[leonidasmi]

Context

Summary

This PR can be summarized in the following changelog entry:

• Deals with the DB cleanup of pages that had been selected for the llms.txt file but then got deleted or made private.

Relevant technical choices:

Test instructions

Test instructions for the acceptance test before the PR gets merged

This PR can be acceptance tested by following these steps:

• Select a couple of pages to be added manually in the llms.txt file
• After saving the settings, do the following to those saved pages (make sure at least one of them are from the contact/about/term/etc pages and one of them from the content pages):
  • Make one private
  • Make one draft
  • Make one scheduled
  • Make one noindexed
  • Make one password-protected
  • Delete one
• Find the wpseo_llmstxt option in the DB, paste its content in unserialize.com and take a note of the result
• Run the scheduled action and confirm that you dont see any pages that are private/draft/etc in the llms.txt file
• This should have also cleaned up the IDs of the private pages from the db
• Find again the wpseo_llmstxt option in the DB, paste its content in unserialize.com and compare the result with the previous one
• The diff between the results should be the IDs of the pages you edited in the previous steps
• Run the scheduled action again and confirm that the llms.txt file didnt change
• Go to the llms.txt settings, confirm that you see the pages you expected, add another page, save and re-run the scheduled action
• Confirm that the llms.tx file's only change was the new page you added.
• Repeat the above tests, but this time with indexables reset and disabled
  • The only difference would be that the noindexed pages will be treated as public ones.

Relevant test scenarios

• Changes should be tested with the browser console open
• Changes should be tested on different posts/pages/taxonomies/custom post types/custom taxonomies
• Changes should be tested on different editors (Default Block/Gutenberg/Classic/Elementor/other)
• Changes should be tested on different browsers
• Changes should be tested on multisite

Test instructions for QA when the code is in the RC

• QA should use the same steps as above.

Impact check

This PR affects the following parts of the plugin, which may require extra testing:

UI changes

• This PR changes the UI in the plugin. I have added the 'UI change' label to this PR.

Other environments

• This PR also affects Shopify. I have added a changelog entry starting with [shopify-seo], added test instructions for Shopify and attached the Shopify label to this PR.

Documentation

• I have written documentation for this change. For example, comments in the Relevant technical choices, comments in the code, documentation on Confluence / shared Google Drive / Yoast developer portal, or other.

Quality assurance

• I have tested this code to the best of my abilities.
• During testing, I had activated all plugins that Yoast SEO provides integrations for.
• I have added unit tests to verify the code works as intended.
• If any part of the code is behind a feature flag, my test instructions also cover cases where the feature flag is switched off.
• I have written this PR in accordance with my team's definition of done.
• I have checked that the base branch is correctly set.

Innovation

• No innovation project is applicable for this PR.
• This PR falls under an innovation project. I have attached the innovation label.
• I have added my hours to the WBSO document.

Fixes #

[coveralls]

Pull Request Test Coverage Report for Build 6e307514f3a03186b8b1812970167fe4467a7efa

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 7 (0.0%) changed or added relevant lines in 1 file are covered.
• 2 unchanged lines in 1 file lost coverage.
• Overall coverage decreased (-0.002%) to 53.093%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/llms-txt/infrastructure/content/manual-post-collection.php	0	7	0.0%

Files with Coverage Reduction	New Missed Lines	%
src/llms-txt/infrastructure/markdown-services/content-types-collector.php	2	92.86%

Totals
Change from base Build 4a74e5e94e15b448b98a98b2e98b5a622916b36d:	-0.002%
Covered Lines:	29990
Relevant Lines:	57366

---

💛 - Coveralls

[thijsoo]

CR + ACC 👍