Skip to content

Commit 0916343

Browse files
junzhinmzr1996
andauthored
[Benchmark] Add MMRarebench dataset support (#1495)
* [Benchmark] Add MMRarebench dataset support - Add 4-track medical rare disease benchmark - Support HuggingFace datasets for automatic download - Support base64-encoded images in TSV files * [Benchmark] Fix MMRarebench lint issues * [Benchmark] Fix MMRarebench lint: remove unused import, fix isort, add explicit smp imports - Remove unused DEBUG_MESSAGE import (F401) - Add noqa F401,F403 to star import from ..smp - Add explicit imports for load, dump, read_ok, np to resolve F405 - Fix isort ordering in mmrarebench.py and __init__.py * [Benchmark] Fix MMRarebench lint issues in mmrarebench.py * [Benchmark] Fix isort: remove extra blank line in mmrarebench.py * [Benchmark] Fix isort: move mmrarebench import to correct alphabetical position in __init__.py * [Benchmark] Sync lint config from main and fix MMRarebench imports - Sync .pre-commit-config.yaml: add isort hook, upgrade flake8 to 7.1.2 - Sync __init__.py with upstream main (explicit imports), add MMRarebench - Remove redundant `from ..smp import *` in mmrarebench.py --------- Co-authored-by: Ma Zerun <mzr1996@163.com>
1 parent 2d5b16c commit 0916343

2 files changed

Lines changed: 890 additions & 0 deletions

File tree

vlmeval/dataset/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,8 @@
8686
from .mmmath import MMMath
8787
from .mmoral_opg_closed import MMOral_OPG_CLOSED
8888
from .mmoral_opg_open import MMOral_OPG_OPEN
89+
from .mmrarebench import (MMRarebenchCrossmodal, MMRarebenchDiagnosis, MMRarebenchExamination,
90+
MMRarebenchTreatment)
8991
from .mmsafetybench import MMSafetyBenchDataset
9092
from .mmsibench import MMSIBench, MMSIVideoBench
9193
from .moat import MOAT
@@ -294,6 +296,7 @@ def evaluate(self, eval_file, **judge_kwargs):
294296
MMSafetyBenchDataset, MSSBenchDataset, SIUODataset, SIUOGenDataset, SIUOMCQDataset, M3oralBenchDataset, # noqa: E501
295297
Design2Code, VLADBench, SSIBenchDataset, NPMM, SGI_Bench_Experimental_Reasoning, MMOral_OPG_OPEN, MMOral_OPG_CLOSED, # noqa: E501
296298
SciDocBench,
299+
MMRarebenchDiagnosis, MMRarebenchTreatment, MMRarebenchCrossmodal, MMRarebenchExamination,
297300
]
298301

299302
# add by EASI team

0 commit comments

Comments
 (0)