Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add stricter gradient check for log marginal likelihood in Gaussian Processes #31543

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Add stricter gradient check for log marginal likelihood in Gaussian Processes #31543

wants to merge 13 commits into from

Conversation

snath-xoc
Copy link
Contributor

@snath-xoc snath-xoc commented Jun 13, 2025 •
edited
Loading

First step in fixing the error in the log-marginal likelihood gradient calculations in Gaussian Processes as noticed in #31366. Also related to #31289.

What does this implement/fix? Explain your changes.

Implements a stricter test for test_lml_gradient by replacing the manual gradient calculation usingscip.optimize.approx_fprime with scipy.differentiate.derivative and calculating the gradient over several different length_scales

@conradstevens and @lorentzenchr any suggestions are welcome (cc @ogrisel and @antoinebaker)

TO DO (perhaps within this PR or separately):

  • Implement chain rule calculation for theta under GaussianProcessRegressor
  • Implement chain rule calculation for the Max Absolute Posteriori (b) in _BinaryGaussianProcessClassifierLaplace

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 13, 2025 •
edited
Loading

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: 734abfd. Link to the linter CI: here

@snath-xoc
Copy link
Contributor Author

Somehow some commits from other branches got mixed up in this PR hence the weird number of commit messages (the files should be restored now)

@conradstevens
Copy link
Contributor

Are we going to go with scipy.differentiate as suggested by @lorentzenchr?

@snath-xoc
Copy link
Contributor Author

@conradstevens yes, your initial manual_grad implementation is similar to approx_fprime and derivative seems to yield similar results. I think they could be used interchangably, derivative is roughly based off taylor series approximation and seems to be more numerical. The main addition is that we check it over many different length-scales which is different from the previous implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
module:gaussian_process
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@snath-xoc @conradstevens