Prepare PiecewiseLinearTwoPhaseMaterial for GPU support #4153

multitalentloes · 2024-07-30T09:54:14Z

Start work on property evaluation on GPUs

Make header with macros for function decoration needed by cuda/hip
decorate functions that must be callable from both the cpu and gpu
implement make_view functions that will be used in cuda kernels later to serve as the interface to the property data

atgeirr · 2024-07-30T11:04:39Z

opm/material/fluidmatrixinteractions/PiecewiseLinearTwoPhaseMaterialParams.hpp

+
+template <class TraitsT, class ContainerType, class ViewType>
+PiecewiseLinearTwoPhaseMaterialParams<TraitsT, ViewType> make_view(const PiecewiseLinearTwoPhaseMaterialParams<TraitsT, ContainerType>& params) {
+    ViewType SwPcwnSamples = make_view<const float, float>(params.SwPcwnSamples());


Should this be float unconditionally, and not depend on the underlying scalar type requested in the ViewType?

Good point, It should depend on the viewType which should store something that matches the containerType

I see a static_assert in your future...

I have changed the type and added the static assert now

multitalentloes · 2024-08-01T07:46:15Z

Jenkins build this please

multitalentloes · 2024-08-01T11:32:23Z

Jenkins build this please

kjetilly

Some comments. A lot of files include both config.h and gpuDecorators.h, but the former is only needed for the macros gpuDecorators.h, which already include config.h. Also, there seems to be some inconsistencies with the defintions in gpuDecorators.h

opm/common/utility/gpuDecorators.hpp

opm/material/common/EnsureFinalized.hpp

opm/material/common/MathToolbox.hpp

opm/material/densead/Evaluation6.hpp

opm/material/densead/Evaluation7.hpp

opm/material/densead/Evaluation8.hpp

opm/material/densead/Evaluation9.hpp

opm/material/fluidmatrixinteractions/PiecewiseLinearTwoPhaseMaterial.hpp

multitalentloes · 2024-08-05T15:00:37Z

Jenkins build this please

opm/common/utility/gpuDecorators.hpp

multitalentloes · 2024-08-06T09:44:15Z

Jenkins build this please

multitalentloes · 2024-08-06T11:05:28Z

Jenkins build this please

bska · 2024-08-07T08:53:10Z

I haven't looked at anything here, but for future reference we usually prefer that PRs be made from branches on your own fork instead of a development branch created directly in the main OPM/opm-common repository.

multitalentloes · 2024-08-07T09:00:05Z

Good catch @bska, this was probably caused by an unintentional git push --set-upstream=origin/branch_name instead of --set-upstream=myfork/branch_name when creating the remote branch. I am actually a bit surprised I was even allowed create a branch here, but yeah, lets delete it after the merge. It will be merged sometime next week after review from @atgeirr.

multitalentloes · 2024-08-07T12:05:47Z

Jenkins build this please

cuistl will soon be renamed gpuistl in opm-simulators

multitalentloes · 2024-08-12T08:28:42Z

Jenkins build this please

atgeirr · 2024-08-12T08:32:57Z

A comment on config.h: This file shall be included in all compilation units (= .cpp files) as the first thing, before any other includes. No header files shall include it! If a header file contains conditional compilation depending on config.h-defined things, it will get the definition of any such macros via the .cpp file including config.h, not by including it itself.

bska · 2024-08-12T08:43:07Z

[config.h] shall be included in all compilation units (= .cpp files) as the first thing, before any other includes. No header files shall include it!

True, although we do break that rule in a couple of important cases. For instance in PR OPM/opm-simulators#5227.

kjetilly · 2024-08-12T08:47:13Z

A comment on config.h: This file shall be included in all compilation units (= .cpp files) as the first thing, before any other includes. No header files shall include it! If a header file contains conditional compilation depending on config.h-defined things, it will get the definition of any such macros via the .cpp file including config.h, not by including it itself.

I get that from a technical standpoint this needs to be the case (since config.h does not have opm in it's path, therefore could conflict with other files, and hence can't be installed alongside of the rest of the header files) but this is also a very, very dangerous design. That means in practice that the inclusion of any header file that requires defines from config.h would behave differently when compiling OPM and when using it as a library, which in turn means that any such header files that requires defines from config.h probably can't be public (ie. can't be part of what gets (implicitly) included when using OPM as a library).

This problem has probably not been relevant up until now, but with the CUDA/HIP definitions that do rely on the defines in config.h, this does become a potential issue. While I believe one can mostly circumvent the issue since it (probably) won't change the ABI of the generated library files, so it might not matter in practice, but it's still too close for comfort. AFAIK, this is also a problem with say determining MPI support etc of an installed OPM library.

If we look at other libraries, they typically include a library_name/config.h file in the installation. Is there a good reason the same hasn't been done for the config files in OPM? So say opm/common/config.h etc. The change could probably be handled automatically.

multitalentloes · 2024-08-12T11:33:21Z

In light of these comments I have removed the include of the config.h file from gpuDecorators.hpp.

multitalentloes · 2024-08-12T11:33:26Z

Jenkins build this please

bska · 2024-08-13T10:09:18Z

The PR was merged and I think we should delete the branch now, at least unless it will be needed in the near future.

multitalentloes · 2024-08-13T12:05:30Z

@bska you can safely delete the branch now

bska · 2024-08-13T12:11:50Z

you can safely delete the branch now

Thanks for letting me know. I'll delete the branch now.

atgeirr reviewed Jul 30, 2024

View reviewed changes

multitalentloes changed the title ~~Prepare property evaluation for GPU acceleration~~ Prepare PiecewiseLinearTwoPhaseMaterial for GPU support Aug 1, 2024

multitalentloes marked this pull request as ready for review August 1, 2024 07:46

kjetilly reviewed Aug 5, 2024

View reviewed changes

multitalentloes force-pushed the gpu_property_evaluation branch 2 times, most recently from 7fab79f to 259d3ba Compare August 5, 2024 14:53

kjetilly requested changes Aug 6, 2024

View reviewed changes

multitalentloes force-pushed the gpu_property_evaluation branch from e38d557 to de6bbcc Compare August 6, 2024 09:43

multitalentloes mentioned this pull request Aug 7, 2024

Avoid exception macro in device code OPM/opm-simulators#5515

Merged

multitalentloes added 6 commits August 12, 2024 10:22

add gpuDecorators.hpp and use it in with AD

8ee21bb

fix inconsistency

a4fb430

improve gpuDecorators.hpp

a929b36

avoid warnings

6f5092e

make macrodefinitions more verbose to avoid clang warning

765f33e

change namespace to reflect incoming refactoring

71e216a

cuistl will soon be renamed gpuistl in opm-simulators

multitalentloes force-pushed the gpu_property_evaluation branch from ac44e51 to 71e216a Compare August 12, 2024 08:28

multitalentloes added 2 commits August 12, 2024 13:28

avoid including non-installable config.h in headerfile

67a6b6e

add doxygen documentation

296e361

kjetilly approved these changes Aug 13, 2024

View reviewed changes

atgeirr merged commit b2b8f28 into master Aug 13, 2024

bska deleted the gpu_property_evaluation branch August 13, 2024 12:11

Prepare PiecewiseLinearTwoPhaseMaterial for GPU support #4153

Prepare PiecewiseLinearTwoPhaseMaterial for GPU support #4153

Uh oh!

Conversation

multitalentloes commented Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

atgeirr Jul 30, 2024

Choose a reason for hiding this comment

Uh oh!

multitalentloes Jul 30, 2024

Choose a reason for hiding this comment

Uh oh!

atgeirr Jul 30, 2024

Choose a reason for hiding this comment

Uh oh!

multitalentloes Jul 30, 2024

Choose a reason for hiding this comment

Uh oh!

multitalentloes commented Aug 1, 2024

Uh oh!

multitalentloes commented Aug 1, 2024

Uh oh!

kjetilly left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

multitalentloes commented Aug 5, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

multitalentloes commented Aug 6, 2024

Uh oh!

multitalentloes commented Aug 6, 2024

Uh oh!

bska commented Aug 7, 2024

Uh oh!

multitalentloes commented Aug 7, 2024

Uh oh!

multitalentloes commented Aug 7, 2024

Uh oh!

multitalentloes commented Aug 12, 2024

Uh oh!

atgeirr commented Aug 12, 2024

Uh oh!

bska commented Aug 12, 2024

Uh oh!

kjetilly commented Aug 12, 2024

Uh oh!

multitalentloes commented Aug 12, 2024

Uh oh!

multitalentloes commented Aug 12, 2024

Uh oh!

bska commented Aug 13, 2024

Uh oh!

multitalentloes commented Aug 13, 2024

Uh oh!

bska commented Aug 13, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

multitalentloes commented Jul 30, 2024 •

edited

Loading