Memory Usage Explosion #753
Description
We are experiencing memory usage explosion on Ubuntu 14.04.1 LTS with Thumbor v6.0.1 and Pillow 3.2.0. Our memory usage can hit ~97%. Then it will return to ~60% memory usage and after some time it will hit again ~97% memory usage.
After some investigation, there is possility that resize method in thumbor/engines/pil.py is the root cause. If we run gc.collect() in resize method, our memory usage is never hit 97% again.
Thumbor request URL
Sample url http://localhost:7001/unsafe/1920x1080/filename.jpg
Script to reproduce
#! /usr/bin/env ruby
require 'open-uri'
@total = 0
count = 100000
(5000..(5000+count)).each do |i|
size = 2000 + rand(2000)
wxh = [size, size].join('x')
url = "http://localhost:7001/unsafe/#{wxh}/filename.jpg"
start_time = Time.now
open url
end_time = Time.now
duration = end_time - start_time
@total += duration
puts "got: #{url} in #{duration.round(2)}"
end
puts "That took: #{@total.round(2)} seconds"
puts "Average #{(@total / count).round(2)} per image"Expected behaviour
Memory usage should below 90%
Actual behaviour
Memory usage hit ~97%
Operating system
14.04.1 LTS
Thumbor v6.0.1
Pillow 3.2.0
thumbor.conf
MAX_AGE = 31536000
ALLOW_ANIMATED_GIFS = False
LOADER = 'thumbor.loaders.file_loader'
STORAGE = 'thumbor.storages.mixed_storage'
SECURITY_KEY = 'example'
ALLOW_UNSAFE_URL = True
FILE_LOADER_ROOT_PATH = '/vagrant/images'
REDIS_STORAGE_SERVER_HOST = 'example.com'
REDIS_STORAGE_SERVER_PORT = 6379
REDIS_STORAGE_SERVER_DB = 0
REDIS_STORAGE_SERVER_PASSWORD = None
MIXED_STORAGE_CRYPTO_STORAGE = 'tc_redis.storages.redis_storage'
MIXED_STORAGE_DETECTOR_STORAGE = 'tc_redis.storages.redis_storage'
DETECTORS = [
'thumbor.detectors.face_detector',
'thumbor.detectors.profile_detector',
'thumbor.detectors.glasses_detector',
'thumbor.detectors.feature_detector'
]
Metadata
Metadata
Assignees
Labels
No labels