I would like to ask how long does it take for the entire pretraining process. For example, how many epoch is used during the pretraining process? and the estimated time taken for the pretraining to be finished?
I am also very interested in your weight release, when are you planning to release the weight?