Same dataset iteration on different gpu cards?

[xuehy]

In dataset implementation, item is fetched by random.choice. The code sets the same random seeds on different processes, and as a result, at every iteration, different processes will get the same data for training. Is this a bug?

[potatoQi]

Same question.

[pixeli99]

Thanks for pointing this out, @xuehy, and thanks @potatoQi for echoing the concern.

Yes, if different processes (especially on different GPUs) are getting the exact same data at each iteration due to the same random seed, this is generally considered a bug or an undesirable setup for distributed training.

I will fix it!

Thanks again.