(semi-random thoughts while looking at groupings in multiple comparison #9573 and #9493)

How do we specify equivalence margin if there is no natural scale for the effect?

Letter/line grouping in multiple comparison likeTukey-hsd and Games-Howell has null and alternative hypothesis reversed for showing which groups are the same or equivalent.
To show equivalence groups, we should use pairwise equivalence tests.

Which effect size? #7006 and #7674

Initially effect sizes where used for power computation. That mixes up what's convenient or appropriate for power and sample size computation with what are "interpretable" measures of effect size.

Cohen's d can be considered as signal to noise ratio, i.e. we standardize the raw effect like difference in means by some population measure. Specifically, Cohen's d uses population (residual) standard deviation (while for power we need the normalized test statistic or noncentrality, normalized by the standard error of the mean difference).

example:
Tukey-hsd uses residual variance as pooled variance estimate, that then is used to compute the standard error of the mean differences.
Games-Howell uses group specific (residual) variance.
One possibility:
Use pooled variance to compute an "important" or "non-negligible" relative effect (e.g. 10% signal to noise), and use it to define equivalence (or non-inferiority) margins. The actual pairwise hypothesis tests can use other variance assumptions, e.g. group-specific, unequal variances as in Games-Howell.

In this way, the effect size is just a tool to define the null and alternative hypotheses. The actual analysis is independent of how, for example, the equivalence margins have been defined.
We can then further separate the "human interface" from the internal machinery. Similarly to separating reported effect sizes from the actual power computations (normalized noncentrality in power instead of a Cohen type effect size with a one letter name).