Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Adding Google 1 Billion Benchmark Dataset to PyTorch dataset #645

@h56cho

Description

@h56cho

Hello,

Currently, for language modelling, PyTorch has 3 built-in datasets (WikiText103, WikiText2, Penn Treebank). Would it be possible to add the Google 1 Billion Benchmark dataset as one of the PyTorch language modelling built-in data?

The link to the Google 1 Billion Benchmark dataset is below:

http://www.statmt.org/lm-benchmark/

Thanks,

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions