Language Models and the Dataset

hello,I don’t understand the mean of the Variabel num_subseqs_pre_example which confused me with the general mean of batch_size. In my opinion, mean of the parameter batch_size is the num of subseq per batch, and I think the Variable num_subseqs_per_example should change as the num_batch, what do you think?@astonzhang

Thanks. Given a set of subseqs, the example in num_subseqs_pre_example means a list of subseqs such that batch_size (e.g., 32) of such examples recover the entire set of subseqs. To make it less confusing, I’ve renamed it as num_batches: