Should we do this as a batch instead of one sentence at a time? #43
Description
Reference Code: https://github.com/NirantK/Hinglish/blob/2798fda87b9b28fa1d7921203ed466c9fd23a28d/hinglishutils.py#L401-#L408
Recommended approach from Huggingface from here:
batch_sentences = ["Hello I'm a single sentence",
"And another sentence",
"And the very very last one"]
encoded_inputs = tokenizer(batch_sentences)
print(encoded_inputs)Metadata
Metadata
Assignees
Labels
No labels