Fix infinite loop caused by incorrect timestamp tokens prediction

[andrewchernyh]

#810

[jongwook]

Thank you!

[Jeronymous]

The infinite loop can still happen, so I suggest to go further with this bugfix

[Jeronymous]

This is not enough to prevent the infinite loop (see discussion #924) because it is not preventing the model to always output <|0.00|>

Suggestion:

                timestamp_last = max(timestamps[-1], self.tokenizer.timestamp_begin + 1) # Avoid to emit <|0.00|> again
                logits[k, self.tokenizer.timestamp_begin : timestamp_last] = -np.inf

[Jeronymous]

I think a better suggestion is:

                if last_was_timestamp and not penultimate_was_timestamp:
                    timestamp_last = timestamps[-1]
                else:
                    timestamp_last = timestamps[-1] + 1
                logits[k, self.tokenizer.timestamp_begin : timestamp_last] = -np.inf

to force that timestamps are strictly increasing after a speech segment / increasing between the end of a speech segment and the start of the next one.

Is anyone looking at this?

[FernanOrtega]

Great solution @Jeronymous! I checked it and it works.

With your permission, Im gonna create a new PR to speed up this change. Im gonna mention your suggestion.