Add resident device change call #1517

lslusarczyk · 2025-09-11T12:30:55Z

UMF part of https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/supported/sycl_ext_oneapi_peer_access.asciidoc feature.

UR/llvm part is in intel/llvm#19257

include/umf/memory_pool_ops.h

src/libumf.map

bratpiorka · 2025-09-11T12:39:25Z

src/memory_provider.c

+        hProvider->provider_priv, deviceIndex, isAdding);
+    checkErrorAndSetLastProvider(res, hProvider);
+    return res;
+}


add empty line

src/provider/provider_level_zero.c

bratpiorka · 2025-09-11T12:48:49Z

src/provider/provider_cuda.c

@@ -802,6 +802,15 @@ static umf_result_t cu_memory_provider_get_allocation_properties_size(
    return UMF_RESULT_ERROR_INVALID_ARGUMENT;
 }

+static umf_result_t


add a "default" handler in memory_provider.c (see umfDefaultCtlHandle) and do not modify CUDA provider

Where exactly in the code you think "default" is missing?

do not modify CUDA provid

Rejected. umf_memory_provider_ops_t type received a new field and it should be somehow initialized in UMF_CUDA_MEMORY_PROVIDER_OPS as all other fields. Please explain if you still want me to remove this change.

Please initialize cuda ops .ext_resident_device_change to NULL.
Next, define the umfDefaultResidentDeviceChange in src/memory_provider.c that returns UMF_RESULT_ERROR_NOT_SUPPORTED and set this handler for each created provider if the ext_resident_device_change is set to NULL (see how this is done for other default handlers like "umfDefaultCloseIPCHandle")

ok, applied

bratpiorka · 2025-09-11T12:51:31Z

src/provider/provider_tracking.c

@@ -1336,6 +1336,15 @@ static umf_result_t trackingGetAllocationPropertiesSize(
        p->hUpstream, memory_property_id, size);
 }

+static umf_result_t trackingResidentDeviceChange(void *provider,


Rejected. umf_memory_provider_ops_t type received a new field and it should be somehow initialized in UMF_TRACKING_MEMORY_PROVIDER_OPS as all other fields. Please explain if you still want me to remove this change.

applied in the same way as in cuda provider

bratpiorka · 2025-09-11T12:51:53Z

test/common/provider_null.c

@@ -162,6 +162,15 @@ static umf_result_t nullGetAllocationPropertiesSize(
    return UMF_RESULT_SUCCESS;
 }

+static umf_result_t nullResidentDeviceChange(void *provider,


rejected, same reason as above

bratpiorka · 2025-09-12T08:15:51Z

src/provider/provider_level_zero.c

    if (!hParams) {
        LOG_ERR("Level Zero memory provider params handle is NULL");
        return UMF_RESULT_ERROR_INVALID_ARGUMENT;
    }

-    if (deviceCount && !hDevices) {
-        LOG_ERR("Resident devices array is NULL, but deviceCount is not zero");
+    if (residentDevicesCount && !residentDevicesIndices) {


please also check if residentDevicesCount == 0 but residentDevicesIndices != NULL

additionally, should indices be unique?

ok, added checking for uniqueness

bratpiorka · 2025-09-12T08:18:39Z

src/provider/provider_level_zero.c

@@ -930,6 +987,155 @@ static umf_result_t ze_memory_provider_get_allocation_properties_size(
    return UMF_RESULT_ERROR_INVALID_ARGUMENT;
 }

+struct ze_memory_provider_resident_device_change_data {
+    bool isAdding;


bratpiorka · 2025-09-12T08:19:39Z

src/provider/provider_level_zero.c

+        change_data->source_memory_provider
+            ->device_handles[change_data->peer_device_index];
+
+    // TODO: add assertions to UMF and change it to be an assertion


use ASSERT() macros from src/utils/utils_assert.h

Postponed to some future PR. I need here a permanent ASSERT, not a one that compiles on in debug config like ASSERT does.

bratpiorka · 2025-09-12T08:21:28Z

src/provider/provider_level_zero.c

+            info->props.base, info->props.base_size);
+    } else {
+        result = ZE_RESULT_SUCCESS;
+        // TODO: currently not implemented call evict here


do you plan to add the missing code here?

Not in this PR.

bratpiorka · 2025-09-12T08:24:18Z

src/provider/provider_level_zero.c

+    if (ze_provider->resident_device_count == 0 ||
+        existing_peer_index == ze_provider->resident_device_count) {
+        // not found
+        if (!isAdding) { // impossible for UR, should be an assertion


if this should be an assertion, please use the ASSERT() macro from src/utils/utils_assert.h
Also, move the comment to the next line

To be changed in future PR when umf will have permanent assertions. Comments moved.

bratpiorka · 2025-09-12T08:24:27Z

src/provider/provider_level_zero.c

+
+static umf_result_t
+ze_memory_provider_resident_device_change(void *provider, uint32_t device_index,
+                                          bool isAdding) {


bratpiorka · 2025-09-12T08:24:46Z

src/provider/provider_level_zero.c

+        // adding case
+        if (ze_provider->device_count <=
+            ze_provider
+                ->resident_device_count) { // impossible for UR, should be an assertion


if this should be an assertion, please use the ASSERT() macro from src/utils/utils_assert.h
Also, move the comment to the next line

postponed when UMF have permanent assertions, comments moved

bratpiorka · 2025-09-12T08:24:54Z

src/provider/provider_level_zero.c

+        }
+
+        if (ze_provider->device_count <=
+            device_index) { // impossible for UR, should be an assertion


applied as above

bratpiorka · 2025-09-12T08:25:32Z

src/provider/provider_level_zero.c

+
+    } else {
+        // found
+        if (isAdding) { // impossible for UR, should be an assertion


applied as above

bratpiorka · 2025-09-12T08:27:28Z

src/provider/provider_level_zero.c

+        LOG_ERR("umfMemoryTrackerIterateAll did not manage to do some change "
+                "numFailed:%d, numSuccess:%d",
+                privData.success_changes, privData.failed_changes);
+        return UMF_RESULT_ERROR_INVALID_ARGUMENT; // probably some other result is better, best just change into assertion


you could use UMF_RESULT_ERROR_MEMORY_PROVIDER_SPECIFIC
or change to assert

src/critnib/critnib.c

lukaszstolarczuk · 2025-09-15T09:02:40Z

src/libumf.map

@@ -54,6 +54,7 @@ UMF_1.0 {
        umfMemoryProviderPurgeForce;
        umfMemoryProviderPurgeLazy;
        umfMemoryProviderPutIPCHandle;
+        umfMemoryProviderResidentDeviceChange;


pls move down in the file (after a comment # Added in UMF_1.0)

actually, mea culpa, pls make a new sction, we're already past v1.0 - we should have now # Added in UMF_1.1 section

lukaszstolarczuk · 2025-09-15T09:02:58Z

src/libumf.def

@@ -60,6 +60,7 @@ EXPORTS
    umfMemoryProviderPurgeForce
    umfMemoryProviderPurgeLazy
    umfMemoryProviderPutIPCHandle
+    umfMemoryProviderResidentDeviceChange


pls move down in the file (after a comment ; Added in UMF_1.0)

first of all, wrong function was moved - umfMemoryProviderResidentDeviceChange is the new one

mea culpa, it should be already in ; Added in UMF_1.1 section

lukaszstolarczuk · 2025-09-15T09:06:07Z

src/memory_provider.c

@@ -606,3 +619,16 @@ umf_result_t umfMemoryProviderGetAllocationPropertiesSize(
    checkErrorAndSetLastProvider(res, hProvider);
    return res;
 }
+
+umf_result_t
+umfMemoryProviderResidentDeviceChange(umf_memory_provider_handle_t hProvider,


do we have any tests for this function? including negative tests

no, success path will be tested by UR tests
failure path does not happen as UR is the only one using this functionality and it never issues wrong parameters

If you think some test case is important and possible to test within UMF please write which case and please guide me where to add it and what other test to use as example.

UR is an separate project - we need at least few simple test cases to do not decrease code coverage.

ok, I've added some tests in 88f672b

still missing tests for new functions, even simple negative test would be better than nothing (you can put them in generic API tests: test/memoryProviderAPI.cpp)

lplewa · 2025-09-15T10:18:31Z

include/umf/memory_provider_ops.h

+    /// @brief Adds or removes devices on which allocations should be made
+    ///        resident.
+    /// @param provider handle to the memory provider
+    /// @param device_index identifier of device
+    /// @param is_adding Boolean indicating if peer is to be removed or added
+    /// @return UMF_RESULT_SUCCESS on success or appropriate error code on
+    ///         failure.
+    umf_result_t (*ext_resident_device_change)(void *provider,
+                                               uint32_t device_index,
+                                               bool is_adding);
+


This function should not be in OPS. This is something very specific to L0 provider. Resident devices are passed to L0 provider thru provider specific params, so control of them should be olso be done through provider specific API, this is why i think with should be implemented through CTL

Rejected. We discussed this in teams (4th Aug, 25) we do it as ops (Piotr, sorry in Polish): co do tego API co Ty robisz (w stylu daj mi wszystkie zaalokowane page'e) ja bym chyba sugerował zrobić API a nie robić przez CTLa, Rafał: no to faktycznie chyba dedykowane API lepiej pasuje niż CTL.

CTL should be for statistics, not a universal, hard-to-read tool to implement any API, Łukasz (me): Być może będziecie chcieli uprościć ext_ctl by nie był maszynką, którą mozna zaimplementować wszystko, a jedynie służył do statystyk. Ale to już ja się na tym nie znam - zostawiam do przemyśleń.

We talked about API to iterate over all allocations, not about API to modify some internal settings of the L0

I can see it both ways. My comment during that discussion was about a generic functionality implementable for all providers. This isn't that.
I also remember making a point about CTL being useful for provider/pool-specific functionality.

On the other hand, adding an API function is simpler.

lplewa · 2025-09-15T10:18:33Z

include/umf/providers/provider_level_zero.h

 /// @brief  Set the resident devices in the parameters struct.
 /// @param  hParams handle to the parameters of the Level Zero Memory Provider.
-/// @param  hDevices array of devices for which the memory should be made resident.
-/// @param  deviceCount number of devices for which the memory should be made resident.
+/// @param  hDevices array of all devices for which the memory can be made resident.
+/// @param  deviceCount number of devices for which the memory can be made resident.
+/// @param  residentDevicesIndices array of indices in all devices array to devices for which the memory should be made resident.
+/// @param  residentDevicesCount number of items in indices array.
 /// @return UMF_RESULT_SUCCESS on success or appropriate error code on failure.
 umf_result_t umfLevelZeroMemoryProviderParamsSetResidentDevices(
    umf_level_zero_memory_provider_params_handle_t hParams,
-    ze_device_handle_t *hDevices, uint32_t deviceCount);
+    ze_device_handle_t *hDevices, uint32_t deviceCount,
+    uint32_t *residentDevicesIndices, uint32_t residentDevicesCount);


This is an API(and ABI) break - we are post 1.0 release so you cannot do changes like this.

Shall I bump some number? Only UR uses this API and I am changing UR right now as well.

Even if UR uses only this API, old UR should work with new umf. And we will not do 2.0 release just after 1.0.

You cannot brake API. If changes are needed we have to keep old function working correctly and add new one.

We can't break backwards compatibility with SYCL / UR. Changing a major version is a significant undertaking that involves updating the components across all the different layers (UMF is nearly the lowest-most component in the stack). We've been burned on this in the past, and it's very disruptive.

In this case, my suggestion would be to simply create a new function that sets the indices.

ldorau

14 / 16 files reviewed (will be continued)

pbalcer · 2025-09-16T08:54:51Z

include/umf/memory_provider_ops.h

+    ///         failure.
+    umf_result_t (*ext_resident_device_change)(void *provider,
+                                               uint32_t device_index,
+                                               bool is_adding);


I'd rather we have two functions - add/remove.

Add resident device change call

7430e14

lslusarczyk requested a review from a team as a code owner September 11, 2025 12:30

added missing doxygen documentation

ac552af

bratpiorka requested changes Sep 11, 2025

View reviewed changes

bratpiorka requested review from lplewa and ldorau September 11, 2025 12:53

fixed compilation error

a8baf31

bratpiorka reviewed Sep 12, 2025

View reviewed changes

lslusarczyk added 3 commits September 12, 2025 10:46

fix documentation spelling

c897d66

applied Rafal's comments

fa2049d

applied second round of Rafal's comments

9d7e12c

lslusarczyk force-pushed the memresident branch from d53a23f to 9d7e12c Compare September 15, 2025 08:16

lukaszstolarczuk reviewed Sep 15, 2025

View reviewed changes

lslusarczyk added 2 commits September 15, 2025 11:22

applied Lukasz's comments

ca69334

one more missed Lukasz comment applied

b15ecdd

lplewa requested changes Sep 15, 2025

View reviewed changes

added some tests for resident device feature

88f672b

lslusarczyk mentioned this pull request Sep 15, 2025

memory resident limit to enabled peers intel/llvm#19257

Draft

ldorau reviewed Sep 15, 2025

View reviewed changes

pbalcer reviewed Sep 16, 2025

View reviewed changes

Add resident device change call #1517

Are you sure you want to change the base?

Add resident device change call #1517

Uh oh!

Conversation

lslusarczyk commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lslusarczyk Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lslusarczyk Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

lslusarczyk commented Sep 11, 2025 •

edited

Loading

lslusarczyk Sep 12, 2025 •

edited

Loading

lslusarczyk Sep 15, 2025 •

edited

Loading

lplewa Sep 15, 2025 •

edited

Loading