features/branson #462

rfhaque · 2024-12-01T04:32:38Z

Description

Adding a specification of Branson https://lanl.github.io/benchmarks/01_branson/branson.html.

We should work with @gshipman and @alexrlongne to make progress on incorporating Branson.

Type of PR: Adding an experiment
Add a new application.py and (maybe) package.py under a new directory for this benchmark
Add an experiment.py
Define an exec_mode=test and exec_mode=perf experiment
Change the git location in package.py once the source PR Update cmake script lanl/branson#53 is merged
Add to gitlab on 3 systems

rfhaque · 2024-12-01T04:34:52Z

repo/branson/package.py

+        cflags = " ".join(self.compiler.flags['cflags']) if 'cflags' in self.compiler.flags else ""
+        cxxflags = " ".join(self.compiler.flags['cxxflags']) if 'cxxflags' in self.compiler.flags else ""
+
+        args.append("-DCMAKE_C_FLAGS={} -I{}/src/random123/features".format(cflags, self.stage.source_path))
+        args.append("-DCMAKE_CXX_FLAGS={} -I{}/src/random123/features".format(cxxflags, self.stage.source_path))
+        args.append(f"-DBUILD_TESTING=OFF")


@pearce8 This is just a temporary fix for lassen. Need to work with the branson team to fix build issues on that system.

I'll look at the RNG package again and see if an update fixes this.

pearce8

Please add a dry run.

scheibelp

minor compatibility issue w/ #953 (should be resolved w/ merge from develop)

repo/branson/package.py

slabasan · 2025-09-29T20:00:34Z

@scheibelp NotImplementedError: "branson" cannot run with MPI only without inheriting from MpiOnlyExperiment. Choose from ['rocm', 'cuda', 'openmp']

scheibelp

I think this was just missing the MpiOnlyExperiment logic added in ce3ffb6#diff-9607e034d65374d4f8618d80af3e2d5033d973bdf09932cb07b5981232425232 - all the dry runs pass now.

I think someone other than me has to approve because I submitted the latest commits to this PR @slabasan, but I am adding an approving review to say the other parts look good to me.

rfhaque · 2025-09-30T02:27:06Z

@scheibelp Does sparta need to provide an mpi variant for this to work? mpi is always on in sparta and thus a variant is not needed

scheibelp · 2025-09-30T03:56:14Z

It appears to be necessary based on the logic added by ce3ffb6 but @michaelmckinsey1 am I missing a way for a package that always uses mpi to avoid this?

michaelmckinsey1 · 2025-09-30T17:14:16Z

It appears to be necessary based on the logic added by ce3ffb6 but @michaelmckinsey1 am I missing a way for a package that always uses mpi to avoid this?

@scheibelp All programming models run with MPI, but to run only with MPI the experiment must inherit from MpiOnlyExperiment. This is to differentiate between experiments that cannot be ran only with MPI experiment+mpi, such as babelstream+mpi. babelstream+mpi+cuda, babelstream+mpi+rocm, babelstream+mpi+openmp are all valid, however babelstream+mpi is not. I believe babelstream may be the only current experiment with this behavior.

scheibelp · 2025-09-30T18:16:05Z

I think what @rfhaque is saying is that it would never make sense to experiment init branson ~mpi, so the presence of an mpi variant is confusing.

@michaelmckinsey1 I think that means that if an experiment init branson would fail based on the logic in https://github.com/LLNL/benchpark/blob/develop/lib/benchpark/experiment.py#L279 (it wants mpi to be mentioned explicitly).

I was thinking perhaps https://github.com/LLNL/benchpark/blob/develop/.github/utils/dryruns.py was dispatching the wrong call, but it seems like it was generating the appropriate experiment init branson for e830b9d, but that it was rejected by the logic in experiment.py.

Is it your opinion such experiments should have an mpi variant? Or am I missing a way for an experiment to be set up this way?

michaelmckinsey1 · 2025-09-30T19:07:07Z

The mpi variant is true by default, so that is why experiment init branson specs to branson+mpi without needing to explicitly write +mpi. I think it makes sense to have an explicit mpi variant because of the babelstream example. I don't see the test where the logic is failing, but I can take a look if it is.

slabasan

LGTM, but waiting on the package.py to be updated upstream?

michaelmckinsey1 · 2025-10-03T22:25:48Z

This PR raised some confusion about the MpiOnlyExperiment mechanism. It is described #904 and in the docs for writing an experiment. It should be generally included for every experiment, except cases like babelstream, which can not be ran for babelstream+mpi, but can be ran for babelstream+mpi+cuda. So it is used to indicate whether running +mpi without any programming models is valid.

Inheriting from MpiOnlyExperiment is used to check spec validity

benchpark/lib/benchpark/experiment.py

Lines 259 to 296 in dd1b469

    
           # Explicitly ordered list. "mpi" first 
        
           models = ["mpi"] + ["openmp", "cuda", "rocm"] 
        
           invalid_models = [] 
        
           for model in models: 
        
               # Experiment specifying model in add_package_spec that it doesn't implement 
        
               if ( 
        
                   self.spec.satisfies("+" + model) 
        
                   and model not in self.programming_models 
        
               ): 
        
                   invalid_models.append(model) 
        
           # Case where there are no experiments specified in experiment.py 
        
           if len(self.programming_models) == 0: 
        
               raise NotImplementedError( 
        
                   f"Please specify a programming model in your {self.name}/experiment.py (e.g. MpiOnlyExperiment, OpenMPExperiment, CudaExperiment, ROCmExperiment). See other experiments for examples." 
        
               ) 
        
           elif len(invalid_models) > 0: 
        
               raise NotImplementedError( 
        
                   f'{invalid_models} are not valid programming models for "{self.name}". Choose from {self.programming_models}.' 
        
               ) 
        
           # Check if experiment is trying to run in MpiOnly mode without being an MpiOnlyExperiment 
        
           elif "mpi" not in str(self.spec) and not any( 
        
               self.spec.satisfies("+" + model) for model in models[1:] 
        
           ): 
        
               raise NotImplementedError( 
        
                   f'"{self.name}" cannot run with MPI only without inheriting from MpiOnlyExperiment. Choose from {self.programming_models}' 
        
               ) 
        
           if ( 
        
               sum([self.spec.satisfies(s) for s in ["+strong", "+weak", "+throughput"]]) 
        
               > 1 
        
           ): 
        
               raise BenchparkError( 
        
                   f"spec cannot specify multiple scaling options. {self.spec}" 
        
               ) 
        
           if sum([self.spec.satisfies(s) for s in ["+cuda", "+rocm", "+openmp"]]) > 1: 
        
               raise BenchparkError( 
        
                   f"spec cannot specify multiple mutually-exclusive programming models. {self.spec}" 
        
               )

. #931 may drive some changes to MpiOnlyExperiment.

codecov-commenter · 2025-10-09T23:13:07Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.43%. Comparing base (8672555) to head (0fd8824).
⚠️ Report is 1 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #462      +/-   ##
===========================================
+ Coverage    65.30%   65.43%   +0.12%     
===========================================
  Files           44       44              
  Lines         3240     3240              
  Branches       256      256              
===========================================
+ Hits          2116     2120       +4     
+ Misses        1117     1113       -4     
  Partials         7        7

see 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Riyaz Haque added 2 commits November 30, 2024 18:12

branson fix for lassen

ec5bd68

branson experiment fixes

31fb7d0

rfhaque requested review from august-knox and pearce8 December 1, 2024 04:32

github-actions bot added the application label Dec 1, 2024

rfhaque commented Dec 1, 2024

View reviewed changes

Branson experiment.py

e55474b

github-actions bot added the experiment New or modified experiment label Dec 2, 2024

lint

3a9d12b

pearce8 requested changes Dec 2, 2024

View reviewed changes

Merge remote-tracking branch 'origin/develop' into features/branson

d6a2140

slabasan marked this pull request as draft December 12, 2024 04:38

rfhaque and others added 2 commits December 16, 2024 08:44

Merge branch 'develop' into features/branson

9eceb18

Merge branch 'develop' into features/branson

a5bfa33

slabasan marked this pull request as ready for review December 18, 2024 17:38

Riyaz Haque added 6 commits January 2, 2025 19:39

Merge remote-tracking branch 'origin/develop' into features/branson

57d725b

Merge remote-tracking branch 'origin/develop' into features/branson

249120f

Merge remote-tracking branch 'origin/develop' into features/branson

1d20884

branson hip cuda implementation

8e232cf

Fix input file params

03f28fd

lint

f1d763f

pearce8 requested review from scheibelp and removed request for august-knox January 13, 2025 21:12

pearce8 assigned rfhaque and august-knox Jan 13, 2025

rfhaque and others added 5 commits January 15, 2025 10:50

venado system specs

1783331

Fix NVCC flags

2a261f7

Merge remote-tracking branch 'origin/develop' into systems/venado

6d0a753

Fix caliper version

752bae8

Add patches

e708c94

Riyaz Haque and others added 5 commits June 3, 2025 14:15

Merge remote-tracking branch 'origin/develop' into features/branson

c770d04

Merge remote-tracking branch 'origin/develop' into features/branson

4ab7199

Merge branch 'develop' into features/branson

8a7fab7

Merge with develop

6941d2f

Merge remote-tracking branch 'origin/develop' into features/branson

17bfb61

pearce8 mentioned this pull request Sep 10, 2025

Spack packages needing updates/testing after Spack 1.0 support #1048

Open

6 tasks

scheibelp reviewed Sep 10, 2025

View reviewed changes

repo/branson/package.py Outdated Show resolved Hide resolved

slabasan and others added 3 commits September 15, 2025 12:56

Merge branch 'develop' into features/branson

b9a6a73

Merge remote-tracking branch 'origin/develop' into features/branson

85afcf5

branson spack-v1.0 changes

b81cc09

rfhaque mentioned this pull request Sep 27, 2025

Branson status #1018

Open

20 tasks

Update experiment.py

e830b9d

scheibelp added 2 commits September 29, 2025 13:31

I think Branson needs to inherit mpionlyexperiment

da93eab

forgot import

10f49e9

scheibelp previously approved these changes Sep 30, 2025

View reviewed changes

Merge remote-tracking branch 'origin/develop' into features/branson

7c02c66

slabasan previously approved these changes Sep 30, 2025

View reviewed changes

Riyaz Haque added 2 commits October 8, 2025 10:15

Merge remote-tracking branch 'origin/develop' into features/branson

c56efe7

rpath changes

0fd8824

rfhaque dismissed stale reviews from scheibelp and slabasan via 0fd8824 October 9, 2025 18:42

Merge remote-tracking branch 'origin/develop' into features/branson

470e2cf

features/branson #462

Are you sure you want to change the base?

features/branson #462

Uh oh!

Conversation

rfhaque commented Dec 1, 2024 • edited by pearce8 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

rfhaque Dec 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alexrlongne Dec 9, 2024

Choose a reason for hiding this comment

Uh oh!

pearce8 left a comment

Choose a reason for hiding this comment

Uh oh!

scheibelp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

slabasan commented Sep 29, 2025

Uh oh!

scheibelp left a comment

Choose a reason for hiding this comment

Uh oh!

rfhaque commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

scheibelp commented Sep 30, 2025

Uh oh!

michaelmckinsey1 commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

scheibelp commented Sep 30, 2025

Uh oh!

michaelmckinsey1 commented Sep 30, 2025

Uh oh!

slabasan left a comment

Choose a reason for hiding this comment

Uh oh!

michaelmckinsey1 commented Oct 3, 2025

Uh oh!

codecov-commenter commented Oct 9, 2025

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

rfhaque commented Dec 1, 2024 •

edited by pearce8

Loading

rfhaque Dec 1, 2024 •

edited

Loading

rfhaque commented Sep 30, 2025 •

edited

Loading

michaelmckinsey1 commented Sep 30, 2025 •

edited

Loading