最近在github上学习新模型时遇到了作者写的:
def prepare_dataloader(opt):
“”" Load data and prepare dataloader. “”"
def load_data(name, dict_name):
with open(name, 'rb') as f:
data = pickle.load(f, encoding='latin-1')
num_types = data['dim_process']
data = data[dict_name]
return data, int(num_types)
print('[Info] Loading train data...')
train_data, num_types = load_data(opt.data + 'train.pkl', 'train')
print('[Info] Loading dev data...')
dev_data, _ = load_data(opt.data + 'dev.pkl', 'dev')
print('[Info] Loading test data...')
test_data, _ = load_data(opt.data + 'test.pkl', 'test')
trainloader = get_dataloader(train_data, opt.batch_size, shuffle=True)
testloader = get_dataloader(test_data, opt.batch_size, shuffle=False)
return trainloader, testloader, num_types
我反复确认作者给的数据是CSV文件,但是作者的源码写的是读取pkl文件,我也尝试直接将csv文件直接导入进去是行不通,请问各位是我理解错了吗,还是有什么方法可以改变。