Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Perf: make carl_sta_trig O(n) regardless of window size - #3724

Open
jberg5 wants to merge 5 commits into
obspy:masterfrom
jberg5:faster-carl-sta-trig
Open

Perf: make carl_sta_trig O(n) regardless of window size#3724
jberg5 wants to merge 5 commits into
obspy:masterfrom
jberg5:faster-carl-sta-trig

Conversation

@jberg5

@jberg5 jberg5 commented Mar 28, 2026 •
edited
Loading

Copy link
Copy Markdown

What does this PR do?

The previous implementation of carl_sta_trig on an array of n elements over a window size m was O(nm). For every position in an output array, we have to compute the sum across a window of m elements.

The "trick" we can take advantage of here is that by computing the cumulative sum of the original array once, the window sum (for any window size!) can be computed in constant time by subtracting the value of the cumsum array at the end of the window from the value of the cumsum array at the start of the window.

To walk through an example:

import numpy as np
m = 9
n = 3  # window size
a = np.arange(1, m+1)  # array([1, 2, 3, 4, 5, 6, 7, 8, 9])
# pad with a zero for lookback alignment
cs = np.concatenate(([0.0], np.cumsum(a, dtype=np.float64)))  # array([ 0.,  1.,  3.,  6., 10., 15., 21., 28., 36., 45.])
out = np.zeros(len(a), dtype=np.float64)
out[n:] = cs[n:m] - cs[:m - n]
array([ 0.,  0.,  0.,  6.,  9., 12., 15., 18., 21.])

For larger window sizes, the vectorization + O(n) change produces a pronounced speedup:

⏺ ARM MacBook (Apple Silicon)                                                                                           
                                                                                                                       
  ┌───────────────────────────────────────────────────────┬─────────┬──────────┬───────┬─────────┬─────────────┐        
  │                        Dataset                        │ Params  │   Old    │  New  │ Speedup │  Triggers   │        
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ Virginia M5.8 @ IU.HRV (BHZ 40Hz, 7.2K samples, 6min) │ 1s/10s  │ 2.5ms    │ 0.1ms │ 19x     │ 153 = 153   │        
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ Virginia M5.8                                         │ 5s/30s  │ 10.2ms   │ 0.2ms │ 52x     │ 36 = 36     │        
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ Virginia M5.8                                         │ 0.5s/5s │ 1.3ms    │ 0.2ms │ 6x      │ 146 = 146   │        
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ IU.ANMO continuous (BHZ 40Hz, 144K samples, 1hr)      │ 1s/10s  │ 58ms     │ 3.4ms │ 17x     │ 56 = 56     │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ IU.ANMO                                               │ 5s/30s  │ 186ms    │ 1.8ms │ 102x    │ 466 = 466   │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ IU.ANMO                                               │ 0.5s/5s │ 29ms     │ 2.8ms │ 10x     │ 2 = 2       │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ Oklahoma OK.BCOK (HHZ 100Hz, 2.16M samples, 6hr)      │ 1s/10s  │ 5,855ms  │ 58ms  │ 102x    │ 2679 = 2679 │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ Oklahoma OK.BCOK                                      │ 5s/30s  │ 18,702ms │ 63ms  │ 297x    │ 780 = 780   │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ Oklahoma OK.BCOK                                      │ 0.5s/5s │ 3,021ms  │ 62ms  │ 49x     │ 2789 = 2789 │
  └───────────────────────────────────────────────────────┴─────────┴──────────┴───────┴─────────┴─────────────┘        
                                                            
  Intel Xeon x86_64 (GCP c2-standard-4, 3.1 GHz)                                                                        
                                                            
  ┌───────────────────────────────────────────────────────┬─────────┬──────────┬───────┬─────────┬─────────────┐        
  │                        Dataset                        │ Params  │   Old    │  New  │ Speedup │  Triggers   │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ Virginia M5.8 @ IU.HRV (BHZ 20Hz, 7.2K samples, 6min) │ 1s/10s  │ 2.6ms    │ 0.2ms │ 14x     │ 153 = 153   │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤
  │ Virginia M5.8                                         │ 5s/30s  │ 8.2ms    │ 0.2ms │ 44x     │ 36 = 36     │        
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ Virginia M5.8                                         │ 0.5s/5s │ 1.4ms    │ 0.2ms │ 8x      │ 146 = 146   │        
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ IU.ANMO continuous (BHZ 40Hz, 144K samples, 1hr)      │ 1s/10s  │ 164ms    │ 4.0ms │ 41x     │ 56 = 56     │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ IU.ANMO                                               │ 5s/30s  │ 527ms    │ 3.9ms │ 134x    │ 466 = 466   │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ IU.ANMO                                               │ 0.5s/5s │ 82ms     │ 3.9ms │ 21x     │ 2 = 2       │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ Oklahoma OK.BCOK (HHZ 100Hz, 2.16M samples, 6hr)      │ 1s/10s  │ 13,451ms │ 94ms  │ 143x    │ 2679 = 2679 │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ Oklahoma OK.BCOK                                      │ 5s/30s  │ 43,205ms │ 95ms  │ 456x    │ 780 = 780   │
  ├───────────────────────────────────────────────────────┼─────────┼──────────┼───────┼─────────┼─────────────┤        
  │ Oklahoma OK.BCOK                                      │ 0.5s/5s │ 6,750ms  │ 95ms  │ 71x     │ 2789 = 2789 │
  └───────────────────────────────────────────────────────┴─────────┴──────────┴───────┴─────────┴─────────────┘        
                                                            
  Trigger counts match exactly in all cases. Max relative difference never exceeds 1e-11.

AI used?

I asked claude code to do some profiling on common use cases and look for python loops that could be vectorized. It flagged this pretty quickly and identified that cumsum could be a big win here. I've had to modify its output a bunch to get rid of useless extra features and make it feel more idiomatic. Also it seems to love getting rid of original code comments, even though they're very helpful.

I also used it a lot for writing and running benchmarking scripts both locally on my macbook and on a gcloud vm.

PR Checklist

  • Correct base branch selected? master for new features, maintenance_?.?.x for bug fixes
  • Tests: Added new tests for any new features or fixed regressions
  • Changelog: Added a short note in CHANGELOG.txt (only obsolete if fixing a bug introduced after the last release)
  • Add the yellow ready for review label when you the PR is ready to be reviewed

Rare actions items:

  • First time contributors: Feel free to add your name to CONTRIBUTORS.txt
  • New modules: add the module to CODEOWNERS with your github handle

Issue labels

The PR can be flagged with the following "issue labels" to alter CI runs:

  • no_ci to skip CI builds while work-in-progress
  • build_docs to trigger automatic docs build to see how docs render for the PR
  • test_network to include "networked" tests if the PR touches any tests marked as "network"
  • upload_images to attach plots generated in tests as artifacts to the CI run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jberg5