-
Couldn't load subscription status.
- Fork 60
Description
The MLPerf Results Messaging Guidelines contain the following:
Submitters (i.e., those who have submitted results for review and verification by MLCommons) are not allowed to publish any results for a given benchmark version before its official publication date.
For example, if an organization only submits ResNet50 results to the upcoming v3.1 round, they are not allowed to disclose these results until the publication date. (However, they can seemingly obtain a similar set of results for a previous version of the benchmark e.g. v3.0, and disclose them as unverified results anytime they like, including during the v3.1 review period.)
Non-submitters (i.e., those who have not submitted results for review and verification by MLCommons) are not allowed to publish any results until two weeks after the official publication date for that benchmark version.
Now, if the same organization , who has only submitted ResNet50 results, obtains some unverified results, say, for GPT3, can they disclose them right after the official publication date, without waiting for two weeks?
On the one hand, the answer could be yes: this organization is a submitter, since they have submitted something to this round, just not this benchmark. On the other hand, the answer should be no: they have submitted nothing for this benchmark.