Adds performance tests, that compare to published NuGet #975
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
not ready for runs yet, as I can't yet force the benchmark to use different assembly versions
|
Not sure I made the CI/build script changes correctly. Plus we might have to restrict testing to Python 3.5 for now, as it is the only version supported by a published NuGet. |
ad92f96 to
2e28711
Compare
|
Excluded CI changes so that this can be merged independently. |
327c311 to
6a9f75e
Compare
6a9f75e to
250a490
Compare
Codecov Report
@@ Coverage Diff @@
## master #975 +/- ##
===========================================
+ Coverage 65.44% 86.71% +21.26%
===========================================
Files 1 1
Lines 301 301
===========================================
+ Hits 197 261 +64
+ Misses 104 40 -64
Continue to review full report at Codecov.
|
|
@filmor ready to review |
filmor
approved these changes
Oct 23, 2019
AlexCatarino
pushed a commit
to QuantConnect/pythonnet
that referenced
this pull request
Jun 27, 2020
Add performance tests that compare to published NuGet
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A new project
src\perf_tests\Python.PerformanceTests.csproj, that uses BenchmarkDotNet to benchmark compare current version with the latest version published on NuGet (2.3 as of now).During build the baseline and new version are copied into separate directories. Benchmark code then loads new or baseline based on an environment variable.
In the end, an "optimistic performance ratio" is computed, that has to be less than some target (usually close to 1.0) for the performance test to pass.