docs(skore): Update the user guide documentation - #2688
Conversation
GaetandeCast
left a comment
There was a problem hiding this comment.
Nice! Here is a first pass.
| Refer to the :ref:`cross_validation_metrics` section for more details on the | ||
| metrics available in `skore` for cross-validation. | ||
| - **Classification**: accuracy, precision, and recall. For **binary** classification, | ||
| ROC AUC is included as well, and if the estimator defines ``predict_proba``, Brier |
There was a problem hiding this comment.
ROC AUC is included as well
Only if predict_proba or decision_function is available
There was a problem hiding this comment.
ROC AUC is included as well
Only if
predict_probaordecision_functionis available
That's only for binary classification models, otherwise it is included only if predict_proba is available
| The **inspection** accessor surfaces model internals and explanations that depend on | ||
| the estimator family: for example coefficients for linear models, or feature | ||
| importance–style summaries for tree-based models. What is available follows the | ||
| estimator you passed to :func:`~skore.evaluate`. |
There was a problem hiding this comment.
This section could receive a bit of love. First a link to the inspection section of the API would be welcomed. Also this
feature importance style summaries for tree-based models.
is too vague IMO, call it "impurity based feature importance for tree-based models" or smth.
Also I'd rather we advertise the permutation importance first as it is the preferred method of model inspection (of those proposed in skore at least).
| The :meth:`EstimatorReport.metrics.roc` creates a :class:`RocCurveDisplay` object. | ||
| Here :meth:`EstimatorReport.metrics.roc` returns a :class:`RocCurveDisplay`. Other | ||
| report facets expose other display types (confusion matrices, prediction errors, data | ||
| summaries, and so on); see the report sections in the reference and |
There was a problem hiding this comment.
add link to "report sections" ?
auguste-probabl
left a comment
There was a problem hiding this comment.
Thanks for the PR. I think this could be a bit less verbose (e.g. the details on the default metrics), and could be rephrased with e.g. fewer emdashes and fewer emphases (*) which distract the eye.
| df = display.frame() | ||
| df.head() | ||
|
|
||
| For the full list of display classes and report accessors, see the API reference. |
There was a problem hiding this comment.
Add a link to the API reference
There was a problem hiding this comment.
Overuse of emdash is an LLM smell, can you rephrase?
| - **Classification**: accuracy, precision, and recall. For **binary** classification, | ||
| ROC AUC is included as well, and if the estimator defines ``predict_proba``, Brier | ||
| score is added. For **multiclass** classification, if ``predict_proba`` is | ||
| available, ROC AUC and log loss are added to the defaults. | ||
| - **Regression**: R² (``r2``) and root mean squared error (``rmse``). |
There was a problem hiding this comment.
This is currently up for debate here; do we want to be that specific?
There was a problem hiding this comment.
We will update when it is not true anymore :)
There was a problem hiding this comment.
I'd rather add this detailed info in the API docs of summarize and then link to it here; at least this way the docs might not get de-synced as easily.
There was a problem hiding this comment.
MAE and MAPE are also in the default regression metrics now
Co-authored-by: Gaétan de Castellane <[email protected]> Co-authored-by: Auguste Baum <[email protected]>
closes #1993
Revamp the documentation of the user guide. It is just an update given the recent changes that we have and it does not really change the narrative.
With the work on diagnosis, the user guide will become more interesting I think.