there is not data parallel in training code?

[JoJoistheBestOne]

hi, even when turn on the fsdp, there is not splited data by rank, the dummy_data_loader will yield the same batch across all the ranks.

please help to notifiy it clearly.