Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@homm
Copy link
Member

@homm homm commented Nov 24, 2016

Implements source ROI for resampling. Explanation

In a loose sens, im.resize(size, resample, box) is very similar to im.crop(box).resize(size, resample). But there are two differences:

  1. Box dimensions are not required to be integer. Now it is possible to create resized images with subpixel accuracy.

  2. While box points out mathematical constraints, it doesn't limit actual pixels. Border lines and rows still depends on pixels outside of the box.

This reveals several use cases:

Tile resampling

In opposite to crop + resize, tiles resized from the source image using box argument, will perfect match pixels from image resized without crop.

Original image

lossy

tiles.py
#!/usr/bin/env python

from __future__ import division, print_function

import sys
import time
from PIL import Image


rnd = lambda f: int(round(f))

def split(size, tiles):
    scale = size / tiles
    for i in range(tiles):
        yield (rnd(scale * i), rnd(scale * (i + 1)))


def tiles_crop(im, size, tiles):
    tiled = Image.new(im.mode, size)
    scale = (im.size[0] / size[0], im.size[1] / size[1])

    for y0, y1 in split(size[1], tiles[1]):
        for x0, x1 in split(size[0], tiles[0]):
            tile = im.crop((rnd(x0 * scale[0]), rnd(y0 * scale[1]),
                            rnd(x1 * scale[0]), rnd(y1 * scale[1])))
            tile = tile.resize((x1 - x0, y1 - y0), Image.BICUBIC)
            tiled.paste(tile, (x0, y0))

    return tiled


def tiles_box(im, size, tiles):
    tiled = Image.new(im.mode, size)
    scale = (im.size[0] / size[0], im.size[1] / size[1])

    for y0, y1 in split(size[1], tiles[1]):
        for x0, x1 in split(size[0], tiles[0]):
            box = (x0 * scale[0], y0 * scale[1],
                   x1 * scale[0], y1 * scale[1])
            tile = im.resize((x1 - x0, y1 - y0), Image.BICUBIC, box)
            tiled.paste(tile, (x0, y0))

    return tiled


if __name__ == '__main__':
    im = Image.open(sys.argv[1])
    width = 320
    height = int(width / im.width * im.height)

    start = time.time()
    for _ in range(10):
        tiled = im.resize((width, height), Image.BICUBIC)
    print('>>> ref', time.time() - start)
    tiled.save("{}.ref.png".format(sys.argv[1]))

    for tiles in [(i, i) for i in range(4, 30, 3)]:
        start = time.time()
        for _ in range(10):
            tiled = tiles_crop(im, (width, height), tiles)
        print('>>> crop', tiles, time.time() - start)
        tiled.save("{}.crop.{}x{}.png".format(sys.argv[1], *tiles))

        start = time.time()
        for _ in range(10):
            tiled = tiles_box(im, (width, height), tiles)
        print('>>> box', tiles, time.time() - start)
        tiled.save("{}.box.{}x{}.png".format(sys.argv[1], *tiles))

        print()

  • Reference image (resized from 957 × 701 to 320 × 234 in one pass)
  • image resized with 19 × 19 tiles with box
  • image resized with 19 × 19 tiles with crop
  • inverted diff between reference and cropped tiles

lossy jpg ref lossy jpg box 19x19 lossy jpg crop 19x19lossy jpg ref

Multistep resampling

In some cases we can perform one very cheap (or even free) supersampling resampling (e.g. during JPEG decoding) and then resize to target dimensions. The problem is that JPEG supersampling works only with whole pixels (for example, scaling 8×8 pixel into one). That is why 256×256 and 249×249 images both will be scaled to 32x32 image. This introduces geometric distortion. But with subsampling scaling we can scale 31.125×31.125 image to target dimension without geometric distortion.

Operations reordering

This may be very similar to previous topic. If we have some crop and resize operations sequence, we used to have to perform it in defined order. Otherwise we would have cumulative rounding and geometric distortion errors. With subpixel resampling box, we can reorder operations on our need: for example, merge all operations in single resample with box.

_imaging.c Outdated
{"resize", (PyCFunction)_resize, 1},
// There were two methods for image resize before.
// Starting from Pillow 2.7.0 stretch is depreciated.
{"stretch", (PyCFunction)_resize, 1},
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The deprecated internal method is removed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to hoist that into its own PR?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I'll do

PIL/Image.py Outdated
return self.im.putpixel(xy, value)

def resize(self, size, resample=NEAREST):
def resize(self, size, resample=NEAREST, roi=None):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another possible name is box=None like in Image.crop().

I would not refuse help with English docstring. My suggestion:

The region of interest for the source image. A (left, upper, right, lower)-tuple of floats. If not provided, the entire source image is used.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like box for a parameter name. This parameter runs the equivalent of resize(crop(box)), so matching the Image.crop name makes sense.

@homm
Copy link
Member Author

homm commented Nov 29, 2016

I've found performance regression in .resize(size, box) over .crop(box).resize(size) related to two-steps vertical/horizontal resampling. So, more work required.

@homm homm changed the title Region of interest for resampling Region of interest (box) for resampling Dec 1, 2016
@homm homm removed the Do Not Merge label Dec 2, 2016
@homm
Copy link
Member Author

homm commented Dec 2, 2016

I've updated PR info.

I wasn't expected such massive changes, unfortunately. The main problem was that ImagingResampleHorizontal/Vertical functions used to calculate coefficients on its own. This led to code blowing, so I decided to move precompute_coeffs call to parent function (now it is called ImagingResampleInner). As a bonus: ImagingResampleHorizontal/Vertical functions now don't allocate any memory and always successful, so it is a bit easier to understand the code in ImagingResampleInner.

ResampleHorizontal(imTemp, imIn, yroi_min,
ksize_horiz, bounds_horiz, kk_horiz);
}
free(bounds_horiz);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Memory leak if this step is not performed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved

if previous step was not performed. */
ImagingDelete(imTemp);
if ( ! imOut)
free(bounds_vert);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Memory leak if this step is not performed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved

@homm
Copy link
Member Author

homm commented Dec 2, 2016

More bugs were fixed

@wiredfool
Copy link
Member

@homm Are you happy with this one?

@homm homm added this to the 4.3.0 milestone Aug 9, 2017
@homm homm removed the Needs Rebase label Aug 11, 2017
@homm
Copy link
Member Author

homm commented Aug 11, 2017

@wiredfool Now this looks good :-)

# Conflicts:
#	libImaging/Resample.c
@homm homm removed the Needs Rebase label Aug 17, 2017
@homm
Copy link
Member Author

homm commented Aug 17, 2017

Rebased

@wiredfool wiredfool merged commit f5a8ece into python-pillow:master Aug 22, 2017
@homm homm deleted the resample-roi branch August 22, 2017 23:05
@homm homm mentioned this pull request Dec 17, 2019
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants