Hello,

I’ve been working to reproduce the downstream task results as reported in your paper (Table 3). However, I noticed significant discrepancies in the results of some Taskonomy tasks, as shown in the attached table below. (red colours are the result that I obtained. Black colours are from paper.)

For tasks such as Edges and kpts2d, my reproduced results show a large deviation, while others (like Occlusion) align closely. I followed the evaluation instructions provided in the MultiMAE repository.

Questions:

1. Could these differences be due to missing details in the experimental setup not mentioned in the paper?
2. Is there a particular step in the codebase that might require additional adjustment to ensure reproducibility?

I would greatly appreciate your insights on whether this difference arises from my setup, or if there’s something I might have overlooked.