LLaDA-2.0-mini baseline

[tianbing111]

Hello, DMax is a very good work, and I have successfully reproduced it. However, I'm encountering some issues testing the LLaDA-2.0-mini baseline. Could you please provide me with a test script and parameter configuration for the LLaDA-2.0-mini baseline?

[czg1225]

Hi @tianbing111

Thank you for your interest in our work!

For the evaluation of LLaDA-2.0-mini, we use the official dInfer framework with the following settings:

• gen_length: 2048
• confidence_threshold: 0.95
• block_size: 32
• temperature: 0
• shot setting: 0-shot for all tasks

The configuration files for each task are available in our repository. In addition, we include post-processing scripts to extract answers from model outputs.

For simplicity, you can:

1. Replace the folder:

   dInfer/python/dinfer/decode
   

   with:

   dInfer/python/dinfer/decode_llada_origin
   

2. Change the model path from DMax to LLaDA-2.0-mini

3. Follow the remaining evaluation steps as described in the repository README.

Thanks again. I hope this helps.