S3 Storage breaking smart detectors? #57
Description
Hello, I am getting these errors:
ubuntu@ip-10-0-255-97:~$ thumbor -c /etc/thumbor-config.conf -l debug
libdc1394 error: Failed to initialize libdc1394
2016-05-10 00:19:59 root:DEBUG thumbor running at 0.0.0.0:8888
2016-05-10 00:20:06 thumbor:DEBUG Cleaning key: gfr/source-assets/img/2016/05/09/9524d1f3-027e-4335-97c1-b12bc462780f.jpg
2016-05-10 00:20:06 thumbor:DEBUG Cleansed key: gfr/source-assets/img/2016/05/09/9524d1f3-027e-4335-97c1-b12bc462780f.jpg
2016-05-10 00:20:06 thumbor:DEBUG Cleaning key: gfr/source-assets/img/2016/05/09/9524d1f3-027e-4335-97c1-b12bc462780f.jpg
2016-05-10 00:20:06 thumbor:DEBUG Cleansed key: gfr/source-assets/img/2016/05/09/9524d1f3-027e-4335-97c1-b12bc462780f.jpg
2016-05-10 00:20:06 thumbor:DEBUG METRICS: inc: storage.hit:1
2016-05-10 00:20:06 thumbor:DEBUG Cleaning key: gfr/source-assets/img/2016/05/09/9524d1f3-027e-4335-97c1-b12bc462780f.detectors.txt
2016-05-10 00:20:06 thumbor:DEBUG Cleansed key: gfr/source-assets/img/2016/05/09/9524d1f3-027e-4335-97c1-b12bc462780f.detectors.txt
2016-05-10 00:31:41 botocore.utils:DEBUG URI updated to: https://<bucketpath>/img/2016/05/09/9524d1f3-027e-4335-97c1-b12bc462780f.detectors.txt
2016-05-10 00:31:41 botocore.auth:DEBUG Calculating signature using hmacv1 auth.
2016-05-10 00:31:41 botocore.auth:DEBUG HTTP request method: GET
2016-05-10 00:20:06 tornado.application:ERROR Future exception was never retrieved: Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 1017, in run
yielded = self.gen.send(value)
File "/usr/local/lib/python2.7/dist-packages/thumbor/handlers/__init__.py", line 177, in get_image
self.filters_runner.apply_filters(thumbor.filters.PHASE_AFTER_LOAD, transform)
File "/usr/local/lib/python2.7/dist-packages/thumbor/filters/__init__.py", line 81, in apply_filters
callback()
File "/usr/local/lib/python2.7/dist-packages/thumbor/handlers/__init__.py", line 175, in transform
self.context.transformer.transform(after_transform_cb)
File "/usr/local/lib/python2.7/dist-packages/thumbor/transformer.py", line 102, in transform
self.smart_detect()
File "/usr/local/lib/python2.7/dist-packages/thumbor/transformer.py", line 151, in smart_detect
self.do_smart_detection().result()
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 233, in result
self._check_done()
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 311, in _check_done
raise Exception("DummyFuture does not support blocking for results")
Exception: DummyFuture does not support blocking for results
2016-05-10 00:30:46 botocore.parsers:DEBUG Response headers: <tornado.httputil.HTTPHeaders object at 0x7fc4c9a11050>
2016-05-10 00:30:46 botocore.parsers:DEBUG Response body:
<botocore.response.StreamingBody object at 0x7fc4c9c72110>
I removed all lines with botocore (except those last few which are somewhat interesting in that they happen around the exception) or x-amz-security on them, but there were no errors from botocore, it was properly getting the files and putting them ( I can see the results on the bucket itself).
If I disable all detectors no error messages pop up (but of course smart detection doesn't do diddly), if I switch to the default file storage (literally the only change is commenting the STORAGE = 'tc_aws.storages.s3_storage' line) then everything works perfectly.
I will try to have more feedback tomorrow but if you guys could point me to where in the code I can look for issues I'd appreciate it. I suspect it has something to do with the recent async changes, specifically aws.storage.AwsStorage.put_detector_data being changed to a Future as a side effect of aws.storage.AwsStorage.set being changed. It's the only big difference I can see with thumbor.storages.file_storage.
P.S. This is the content of one of the .detectors.txt in S3:
[{"origin": "Face Detection", "height": 49, "width": 49, "y": 51.120000000000005, "x": 106, "z": 2401}, {"origin": "Face Detection", "height": 47, "width": 47, "y": 53.36, "x": 186, "z": 2209}]