Overflow error: Loading a big libsvm file with load_svmlight_file() #10727

Description

I am using scikit-learn (0.19.1) in the following setup: Ubuntu 17.10 (x86_64) with numpy-1.14.1 and scipy-1.0.0

When I tried loading a big libsvm file (1696462087 rows) like this:

from sklearn.datasets import load_svmlight_file
X_train, y_train = load_svmlight_file('/home/user_name/sets/big_dataset.txt')

I got the following error:

...
File "sklearn/datasets/_svmlight_format.pyx", line 114, in sklearn.datasets._svmlight_format._load_svmlight_file

OverflowError: signed integer is greater than maximum

I googled around and found a similar issue which, however, was related to the query_id parameter. By looking at the _svmlight_format.pyx function I can see that both indices and indptr arrays are of type interger and the overflow error occurs when the following assignment is attempted:

indptr[len(indptr) - 1] = len(data)

Any remedy for such a problem? Would the fix be the same as in the previous issue?

P.S. I would like to avoid splitting a big libsvm file into many small chunks.