Summary
The committed XBEN results support the reported solve count of 100 out of 104 challenges
They do not reproduce the published average solve time of about 19 minutes or total cost of about 337 dollars
No aggregation script explains how those two figures were calculated
Recalculation from the repository
The calculation below used the 104 committed result.json files at benchmark commit 26f374238198ca38b622833b49af83064e87e350
| Measurement |
Recalculated value |
| Solved results |
100 |
| Mean top-level duration across solved results |
33.19 minutes |
| Mean Strix duration across solved results |
32.21 minutes |
| Recorded cost across 35 solved results with a numeric cost |
77.54 dollars |
| Recorded cost across all 38 results with a numeric cost |
147.02 dollars |
What remains unknown
The published figures may use data or calculation rules that are not committed
The mismatch does not by itself prove that the README is wrong
It does mean a reader cannot reproduce the figures from the repository alone
Request
Commit one versioned aggregation script that generates every benchmark table and summary from the result files
Report the sample count beside any metric with missing values
Document the timing fields, inclusion rules, and treatment of missing values next to that script
A generated-file check can then catch drift between the committed results and the README
Summary
The committed XBEN results support the reported solve count of 100 out of 104 challenges
They do not reproduce the published average solve time of about 19 minutes or total cost of about 337 dollars
No aggregation script explains how those two figures were calculated
Recalculation from the repository
The calculation below used the 104 committed
result.jsonfiles at benchmark commit26f374238198ca38b622833b49af83064e87e350What remains unknown
The published figures may use data or calculation rules that are not committed
The mismatch does not by itself prove that the README is wrong
It does mean a reader cannot reproduce the figures from the repository alone
Request
Commit one versioned aggregation script that generates every benchmark table and summary from the result files
Report the sample count beside any metric with missing values
Document the timing fields, inclusion rules, and treatment of missing values next to that script
A generated-file check can then catch drift between the committed results and the README