The Mem0 paper reporting Zep scores on the Locomo benchmark had several implementation and methodology issues (including instructions that contradicted the way Zep displays information). We at Zep ran the benchmark on a corrected implementation.
We would appreciate updating your chart with these corrected results!
The scores for the correct implementation can be found here.
The result data can be found here.
The implementation can be found here.
And an article discussing the issues with the implementation in the paper can be found here.
The Mem0 paper reporting Zep scores on the Locomo benchmark had several implementation and methodology issues (including instructions that contradicted the way Zep displays information). We at Zep ran the benchmark on a corrected implementation.
We would appreciate updating your chart with these corrected results!
The scores for the correct implementation can be found here.
The result data can be found here.
The implementation can be found here.
And an article discussing the issues with the implementation in the paper can be found here.