Inaccurate analysis when keyphrase contains a period

[mmikhan]

Reporter: @amboutwe
From: https://yoast.atlassian.net/browse/IM-2011

Please give us a description of what happened

A . is used to determine when a sentence ends but if the keyphrase contains a dot, the analysis report is inaccurate.

Customer set the keyphrase as 11. SSW (German) which is a shortened way to say 11th week of pregnancy. In the below copy, the keyword is not recognized:

Das zweite Trimester steht fast vor der Tür. Damit naht auch Besserung, was einige Deiner Symptome betrifft. In der 11. SSW hast Du Dich bestimmt schon über sämtliche Themen informiert. Dabei hast Du sicherlich bemerkt, dass es trotz aller Recherche ständig noch etwas Neues zu entdecken gibt.

This issue is related to Yoast/YoastSEO.js#745 where the sentence tokenizer incorrectly splits at an ordinal number, frustrating the keyphrase recognition.

• Also, note this discussion on Slack.
• We’re going to make this for German only. There are also false positives, like It happened in 2000., so it’s probably best to check for numbers with 2-3 digits. We will do this in LR-10 Improves sentence recognition for German with ordinal numbers #18560.
• Check whether LR-10 Improves sentence recognition for German with ordinal numbers #18560 resolves this issue.
• Check whether there are also other languages that use a dot for ordinal numbers (instead of morphology, like 11th). See Ordinal indicator. If there’s other languages, create issues to tackle these.
• In this issue, let’s also look at all other linked observations in the GitHub issues (and in HelpScout) and see if they have been resolved (I’m pretty sure some are).
  • If they have been solved, ping IM to close the GitHub issue with a comment which version should have resolved the issue.
  • If they have not been solved, please create a separate issue.

Observations from the linked GitHub issues and HelpScout conversations:

• #689136 - That’s the issue about with 11. SSW as keyphrase.
• Keyword matching in URL fails when there is a period in the keyword.  YoastSEO.js#998 - .net as a keyphrase should match a slug with net.
• #724453 - Is similar about using .NET as a keyphrase (but then in capital letters). This was solved through in https://yoast.atlassian.net/browse/LINGO-1488
• #718868 - Here a keyphrase with a dot in Turkish creates an error for the subheadings assessment. I’ll attach the video. If we can reproduce, we could create a new issue for it.

Screenshots, screen recording, code snippet

ScreenRecorder-error-on.turkish-wordpress.1.mp4

Used versions

• WordPress version: 5.6

[mmikhan]

From comments:

@hdvos

• This issue is more complex than the 3 that was assigned to it

@hdvos

• I am almost done solving LR-10 Improves sentence recognition for German with ordinal numbers #18560. I think this indeed solves [IM-2011: Inaccurate analysis when keyphrase contains a period as well. But it would need some extra testing just to be sure

@mhkuu

• Plan: we’re solving LR-10 Improves sentence recognition for German with ordinal numbers #18560 first, and then see if all the issues linked to this issue

@hannaw93

• I’m not sure this has a language-independent solution. Especially since there is often a space after the period that should still not count as a sentence ending. The problem is close to the abbreviations problem

@hdvos

• My guess would be that it is not because of the ordinal number but as a result of nouns beginning with a capital in german so it is more sensitive for dots occuring in the sentence. I think that in English this would fail as well, (or at least if the next word is capitalized). But in german it is more prevalent due to the habit of both starting nouns with capitals and ending ordinals in dots. I am not entirely sure, but it might be beneficial to look at this issue with a broader perspective “dots in german sentences“ or sth.

@amboutwe In some languages, this may cause an error. For example when using Turkish, the following happens:

• Keyphrase: 6.sınıf 8.ünite ingilizce kelimeler
• Error: ‘subheadingsKeyword’ değerlendirmesi sırasında bir hata oluştu
• (translates to An error occurred while evaluating 'subheadingsKeyword')

[mmikhan]

Please inform the customer of conversation # 689136 when this conversation has been closed.

[mmikhan]

Please inform the customer of conversation # 718868 when this conversation has been closed.

[mmikhan]

Please inform the customer of conversation # 724453 when this conversation has been closed.

[mmikhan]

Please inform the customer of conversation # 930762 when this conversation has been closed.

[mhkuu]

This bug has many linked issues, so let's first check for each one of them whether they are resolved / worth resolving.

Also, if you need access to the HelpScout conversations, please let me know!

[hannaw93]

(so far) conclusions from the investigation:

1. The bug regarding 11. SSW has been fixed here and works as expected in splitting sentences and keyphrase recognition. The only issue is a orange instead of red score in the Keyphrase in SEO title assessment (details in 2nd comment this issue).

2. It looks like Turkish also has ordinal numbers with periods. From brief research, periods are used to abbreviate ordinal numbers (so 1. instead of 1st, but also as an adjective rather that only for lists). It must be noted that this does not cause issues for sentence splitting or keyphrase recognition that we have seen in German because:

• there is no space between the ordinal number and the noun in Turkish, and a space is required between sentences to be detected as such.
• countable nouns in Turkish do not start with capital letter (that is German-specific), so what follows the period does not look like a sentence beginning.
  There is only a problem for keyphrase in SEO title (details in 2nd comment this issue).
  So while the sentence splitting problem on ordinal numbers is German-specific, the slug problem happens in all languages where a period can be part of the keyphrase.

3. The slug always detects only letters as part of a word, and any sentence splitters (period, comma, inc.) turn into hyphens. For English an example would be that the slug for a post with a keyphrase .rar and a title Files in .rar will be files-in-rar instead of files-in-.rar. A separate issue is needed for the fix that makes the slug take into account exceptions for how the slug words, which are in this case words that contain a period. Another option is to stop the slug from stripping dots, impact of which may be big and needs to be investigated. The straightforward workaround for the user (to change the slug manually) does not work, at least not if you save the post or refresh the page (see attached video). That strips off the period. Apparently, urls are expected to be able to contain periods.

Screen.Recording.2023-01-30.at.17.09.25.mov

An issue is created to fix this:

No issues were found with detecting the keyphrase in slug though, even when the slug does not have a period while the keyphrase doesn't (see attached image).

[hannaw93]

The exact issue for Turkish (Keyphrase in Subheadings breaking) was not reproduced. No error shows in any assessment using the reproducing steps.

However, using the keyphrase 6.sınıf 8.ünite in the title gives this feedback for Keyphrase in SEO title: Does not contain the exact match. Try to write the exact match of your keyphrase in the SEO title and put it at the beginning of the title. instead of The exact match of the focus keyphrase appears at the beginning of the SEO title. Good job!. This is language independent. In English and German, with a keyphrase .rar and a title Files in .rar, the results are the same. There is also no difference whether there is a space the keyphrase (like 11. SSW for German) or not.

An issue to fix this: https://github.com/orgs/Yoast/projects/51?pane=issue&itemId=19229605

The keyphrase is recognised properly though in other keyphrase assessments, including the keyphrase in subheading assessment.

[hannaw93]

The research into other languages with an ordinal period can be limited to languages that have both ordinal periods and nouns starting with a capital letter, because

• keyphrase recognition assessments (except keyphrase in SEO title which gives orange instead of green) and sentence splitting work properly when the period is followed by either a non-capitalized noun or no space. This aligns with how sentence splitting and keyphrase matching works, and is shown through the bug German and tesing Turkish and English.
• the slug matching issue is language-independent

No other cases except German that match these conditions were found so far.

[hannaw93]

Leaving the issue in CR for a second pair of eyes on created issues (i guess also from Martijn and/or Manuel) and on whether it's the case that we don't need to research more languages with ordinal periods.