Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add LinearNDInterpolator to cupyx.scipy.interpolate#8035

Merged
takagi merged 15 commits into
cupy:mainfrom
andfoy:add_linear_nd_interp
Mar 11, 2024
Merged

Add LinearNDInterpolator to cupyx.scipy.interpolate#8035
takagi merged 15 commits into
cupy:mainfrom
andfoy:add_linear_nd_interp

Conversation

@andfoy

@andfoy andfoy commented Dec 18, 2023

Copy link
Copy Markdown
Contributor

See #7186
Depends on #7985

@andfoy andfoy mentioned this pull request Dec 18, 2023
51 tasks
@emcastillo emcastillo self-assigned this Dec 19, 2023
@emcastillo emcastillo added cat:feature New features/APIs prio:medium labels Dec 19, 2023
@andfoy andfoy force-pushed the add_linear_nd_interp branch 2 times, most recently from 5a9d68b to 51f0b09 Compare January 31, 2024 00:15
@andfoy

andfoy commented Jan 31, 2024

Copy link
Copy Markdown
Contributor Author

This PR is passing tests w.r.t the new version of Delaunay in #7985

@emcastillo emcastillo assigned takagi and unassigned emcastillo Feb 6, 2024
@andfoy andfoy force-pushed the add_linear_nd_interp branch 4 times, most recently from fb06e1b to c3312bb Compare February 12, 2024 23:30
@andfoy andfoy force-pushed the add_linear_nd_interp branch from 9fc1cb7 to b3af097 Compare February 14, 2024 22:21
@andfoy

andfoy commented Feb 21, 2024

Copy link
Copy Markdown
Contributor Author

These are the latest benchmark results for this PR as of 6989659. Most of the performance improvement is related to the "find point in triangle" operation, which is critical in order to find the triangle over which the interpolation should take place, this operation was implemented by means of a linear quadtree built upon the Z-curve of the centers of the triangles output by Delaunay.

This will yield the "closest" triangle according to the encoding, which happens to be also a close triangle in the neighborhood of the actual one, here two things can happen:

  1. The closest triangle is the one we are looking for
  2. The enclosing triangle is near the closest based on encoding.

For the first case, the search will yield the same triangle, since it is the one enclosing the point. For the second case the search will recursively explore the neighbors based on the distance from the center to the point, as well as the "nearest point in the triangle close to the search point" (an optimization problem in a graph of triangles), the procedure will take place until the enclosing triangle is found, or until the closest triangle is found (another minimum was not found in the neighborhood).

The following complexities take place, if this was a linear process:

  1. O(T) to encode the triangles, where T is the number of triangles
  2. O(T log T) to sort the triangles
  3. O(P log T) to search the "closest triangle" according to encoding to a point, with P the number of query points
  4. O(1) to refine the search (no more than 10 comparisons take place)

This is better than a naïve O(PT) linear search

Benchmark script
from itertools import product

import cupy as cp
import numpy as np
from cupy import testing
from cupyx.profiler import benchmark

from cupyx.scipy.spatial.delaunay_2d._kernels import DELAUNAY_MODULE
from cupyx.scipy.interpolate import LinearNDInterpolator as GPULinearND
from scipy.interpolate import LinearNDInterpolator as CPULinearND

from tqdm import tqdm

DELAUNAY_MODULE.compile()

# input_sizes = [5, 10, 100, 1000, 5000]
input_sizes = [100, 1000, 5000, 10000, 50000, 100000, 500000]
# input_sizes = [50000]
# input_sizes = [100, 1000, 5000, 10000, 50000]
modules = [(cp, GPULinearND, 'CuPy'), (np, CPULinearND, 'SciPy')]
kwargs = []
# kwargs = [('order', [1, 2, 4, 6, 10])]
# kwargs = [('extrapolate', [True])]
kwargs = list(product(*[list(product([x[0]], x[1])) for x in kwargs]))
measurement = {}


def compute_circumcenter(tri_points, xp):
    D = (tri_points[:, 0, 0] * (tri_points[:, 1, 1] - tri_points[:, 2, 1]) +
         tri_points[:, 1, 0] * (tri_points[:, 2, 1] - tri_points[:, 0, 1]) +
         tri_points[:, 2, 0] * (tri_points[:, 0, 1] - tri_points[:, 1, 1]))
    D = 2 * D

    out = xp.empty((tri_points.shape[0], 2), dtype=xp.float64)
    out[:, 0] = ((tri_points[:, 0] ** 2).sum(-1) * (
                    tri_points[:, 1, 1] - tri_points[:, 2, 1]) +
                 (tri_points[:, 1] ** 2).sum(-1) * (
                     tri_points[:, 2, 1] - tri_points[:, 0, 1]) +
                 (tri_points[:, 2] ** 2).sum(-1) * (
                     tri_points[:, 0, 1] - tri_points[:, 1, 1])) / D

    out[:, 1] = ((tri_points[:, 0] ** 2).sum(-1) * (
                    tri_points[:, 2, 0] - tri_points[:, 1, 0]) +
                 (tri_points[:, 1] ** 2).sum(-1) * (
                     tri_points[:, 0, 0] - tri_points[:, 2, 0]) +
                 (tri_points[:, 2] ** 2).sum(-1) * (
                     tri_points[:, 1, 0] - tri_points[:, 0, 0])) / D
    return out


def compute_random_points(tri_points, xp):
    bary = xp.empty((tri_points.shape[0], 3), xp.float64)
    s = testing.shaped_random((tri_points.shape[0],), np, np.float64,
                              scale=1.0)
    t = testing.shaped_random((tri_points.shape[0],), np, np.float64,
                              scale=1-s)
    bary[:, 0] = xp.asarray(s)
    bary[:, 1] = xp.asarray(t)
    bary[:, 2] = xp.asarray(1 - s - t)

    # breakpoint()

    return (xp.expand_dims(bary, -1) * tri_points).sum(1)


def gather_time(prof):
    cpu_time = prof.cpu_times.mean() * 1000
    gpu_time = prof.gpu_times.mean() * 1000
    return max(cpu_time, gpu_time)


def input_creation(xp, fn, size, **kwargs):
    # np.linspace()
    # # breakpoint()
    # x = xp.linspace(0, 10, int(np.sqrt(size[0])))
    # y = xp.linspace(0, 10, int(np.sqrt(size[0])))

    # noise_x = testing.shaped_random((int(np.sqrt(size[0]))**2,), xp, xp.float64)
    # noise_y = testing.shaped_random((int(np.sqrt(size[0]))**2,), xp, xp.float64, seed=1234)
    # x += noise_x
    # y += noise_y
    # X, Y = xp.meshgrid(x, y)
    # points = xp.c_[X.reshape(-1) + noise_x, Y.reshape(-1) + noise_y]
    points = testing.shaped_random(size + (2,), xp, xp.float64)
    values = testing.shaped_random(points.shape, xp, xp.float64)
    instance = fn(points, values)
    # breakpoint()
    # search = testing.shaped_random(size + (2,), xp, xp.float64, 5, seed=1111)
    search = compute_random_points(points[instance.tri.simplices], xp)
    # search = points[instance.tri.simplices].mean(1)
    return instance, search


def closure(fn, sig):
    return fn(sig)


for kwarg_comb in kwargs:
    kwarg_measurement = measurement.get(kwarg_comb, {})
    measurement[kwarg_comb] = kwarg_measurement
    kwarg_comb = dict(kwarg_comb)
    for size in tqdm(input_sizes):
        size_measurement = kwarg_measurement.get(size, {})
        kwarg_measurement[size] = size_measurement
        for xp, cls, _id in modules:
            b, x = input_creation(xp, cls, (size,), **kwarg_comb)
            prof = benchmark(closure, (b, x), n_repeat=100)
            size_measurement[_id] = gather_time(prof)


lines = []
for kwarg_comb in kwargs:
    kwarg_line = ', '.join([f'{x}={y}' for x, y in kwarg_comb])
    lines.append(f'### `{kwarg_line}`\n')
    lines.append('| Size | CuPy (ms) | SciPy (ms) | Speedup |')
    lines.append('|:----:|:---------:|:----------:|:-------:|')
    kwarg_measurement = measurement[kwarg_comb]
    for size in input_sizes:
        comp = f'{size}'
        size_measurement = kwarg_measurement[size]
        times = []
        for _, _, _id in modules:
            time = size_measurement[_id]
            times.append(time)
            time_info = f'{time:3f}'
            comp = f'{comp} | {time_info}'
        speedup = times[1] / times[0]
        comp = f'{comp} | {speedup:3f}'
        lines.append(f'| {comp} |')
    lines.append('\n')


print('\n'.join(lines))
Size CuPy (ms) SciPy (ms) Speedup
100 0.171086 0.043524 0.254399
1000 0.244974 0.493022 2.012551
5000 0.519537 2.389944 4.600146
10000 0.943060 4.808400 5.098720
50000 2.624869 24.539790 9.348959
100000 5.255700 57.220216 10.887269
500000 25.023617 436.359276 17.437898

@andfoy andfoy force-pushed the add_linear_nd_interp branch from 1726e5d to 4eaab46 Compare February 21, 2024 15:35
@andfoy andfoy marked this pull request as ready for review February 21, 2024 16:05
@andfoy andfoy force-pushed the add_linear_nd_interp branch from 770ee94 to a0840cb Compare March 4, 2024 22:31
@takagi

takagi commented Mar 8, 2024

Copy link
Copy Markdown
Contributor

/test full

@takagi takagi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@takagi takagi merged commit 8fb6d04 into cupy:main Mar 11, 2024
@takagi takagi added this to the v14.0.0a1 milestone Mar 11, 2024
@takagi

takagi commented Mar 11, 2024

Copy link
Copy Markdown
Contributor

Thanks, @andfoy! Would you also add LinearNDInterpolator to the document (docs/source/reference/scipy_interpolate.rst) in another PR?

@andfoy andfoy deleted the add_linear_nd_interp branch March 18, 2024 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants