Content generated in modulefile with puts command is applied on refresh evaluation mode, but only if modulefile is marked for refresh (which means this modulefile has defined shell alias or function).
This behavior is not consistent: puts is refreshed only if something else has to be refreshed.
Thanks to @fihuer for report.