"ValueError: Cannot take the length of Shape with unknown rank". error when passing tf.data.Dataset tensors to model.fit #24520
Description
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): "18.04.1 LTS (Bionic Beaver)"
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: no
- TensorFlow installed from (source or binary): binary
- TensorFlow version (use command below): tf.VERSION = 1.12.0
- Python version: python3.6
- Bazel version (if compiling from source): no
- GCC/Compiler version (if compiling from source): no
- CUDA/cuDNN version: cuda9.0 with cuDNN 7.4.1
- GPU model and memory: GTX 1080 with 8 GB
You can collect some of this information using our environment capture script
You can also obtain the TensorFlow version with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
Describe the current behavior
I am trying to pass the tfrecords read through tf.data.Dataset api into the model.fit .
Since the images could be of different sizes, I am storing the image shapes into tfrecords itself which are
later on read and applied to the img data using tf.reshape . But the tensorflow.keras is unable to determine the shape of this image data at this stage and throws the error.
def _parse_function(proto):
keys_to_features = {"im_path": tf.FixedLenSequenceFeature([], tf.string, allow_missing=True),
"im_shape": tf.FixedLenSequenceFeature([], tf.int64, allow_missing=True),
"im_arr": tf.FixedLenSequenceFeature([], tf.string, allow_missing=True),
"label": tf.FixedLenSequenceFeature([], tf.int64, allow_missing=True),
}
parsed_features = tf.parse_single_example(serialized=proto, features=keys_to_features)
parsed_features['im_arr'] = parsed_features['im_arr'][0]
parsed_features['label'] = parsed_features['label'][0]
parsed_features['im_arr'] = tf.decode_raw(parsed_features['im_arr'], tf.uint8)
parsed_features['im_arr'] = tf.reshape(parsed_features['im_arr'], parsed_features['im_shape'])
return parsed_features['im_arr'], parsed_features['label']
The error thrown is as follows :
Traceback (most recent call last):
File "issue/IssueScript.py", line 75, in <module>
verbose=1)
File "opt/github/example-classification/env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1536, in fit
validation_split=validation_split)
File "opt/github/example-classification/env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 992, in _standardize_user_data
class_weight, batch_size)
File "opt/github/example-classification/env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1117, in _standardize_weights
exception_prefix='input')
File "opt/github/example-classification/env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_utils.py", line 284, in standardize_input_data
data = [standardize_single_array(x) for x in data]
File "opt/github/example-classification/env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_utils.py", line 284, in <listcomp>
data = [standardize_single_array(x) for x in data]
File "opt/github/example-classification/env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_utils.py", line 218, in standardize_single_array
if x.shape is not None and len(x.shape) == 1:
File "opt/github/example-classification/env/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 579, in __len__
raise ValueError("Cannot take the length of Shape with unknown rank.")
ValueError: Cannot take the length of Shape with unknown rank.
So as a debugging step, I removed the length check present in the standardize_single_array function by changing the check as (note the False and part which bypasses the length check)
if x is None:
return None
if False and (x.shape is not None and len(x.shape) == 1):
if tensor_util.is_tensor(x):
return array_ops.expand_dims(x, axis=1)
else:
return np.expand_dims(x, 1)
return x
Then I get the following error
Traceback (most recent call last):
File "issue/IssueScript.py", line 75, in <module>
verbose=1)
File "opt/github/example-classification/env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1536, in fit
validation_split=validation_split)
File "opt/github/example-classification/env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 992, in _standardize_user_data
class_weight, batch_size)
File "opt/github/example-classification/env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1154, in _standardize_weights
exception_prefix='target')
File "opt/github/example-classification/env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_utils.py", line 323, in standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking target: expected activation_4 to have 2 dimensions, but got array with shape (None,)
I did the same with the above error. I removed the check present at line 323 by commenting out the length check as follows.
"""
if len(data_shape) != len(shape):
raise ValueError('Error when checking ' + exception_prefix +
': expected ' + names[i] + ' to have ' +
str(len(shape)) + ' dimensions, but got array '
'with shape ' + str(data_shape))
"""
Now the training proceeds smoothly without error. I believe there is issue with tf.reshape when tensors are supplied as a shape to the function.
Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
Code : https://github.com/dineshdharme/tensorflow-issue1
Just run : python3 issue/IssueScript.py
I have also added a tfrecords generating script tfrecords_utils.py which you can call by
To generate tfrecords file using the image data present in the data folder :
python3 issue/tfrecords_utils.py