install_redrock_templates script #327
Conversation
|
Really excited to see this update. |
There was a problem hiding this comment.
Thanks, this works very well. I tested using no arguments, using -v 0.8.1, using -o, and setting $RR_TEMPLATE_DIR and making sure that it didn't use it.
My only request is that you make a more detailed help statement for --outdir.
I expected the code to install the templates in a directory "templates" under the tree I specified in -o. Instead, it put the template files themselves directly in the tree. The implemented version is equally valid and doesn't need to be changed, but a more explicit help message for -o will hopefully prevent others from making the same assumption I did.
For the record, I also tried stress-testing this script with a (very) old tag, 0.3, and it failed. I don't think this is a problem since the data was downloaded and people shouldn't be using such old tags, but I'll note it here for completeness:
kremin@perlmutter:login15:/global/cfs/cdirs/desi/users/kremin/PRs/testing/redrock> install_redrock_templates -v 0.3
Traceback (most recent call last):
File "/global/cfs/cdirs/desi/users/kremin/PRs/testing/redrock/bin/install_redrock_templates", line 72, in <module>
sys.exit(main())
File "/global/cfs/cdirs/desi/users/kremin/PRs/testing/redrock/bin/install_redrock_templates", line 62, in main
templates = redrock.templates.load_templates()
File "/global/cfs/cdirs/desi/users/kremin/PRs/testing/redrock/py/redrock/templates.py", line 464, in load_templates
tx = Template(filename, zscan_galaxy=zscan_galaxy,
File "/global/cfs/cdirs/desi/users/kremin/PRs/testing/redrock/py/redrock/templates.py", line 144, in __init__
raise ValueError('Missing keyword RRIGM specifying IGM model to use')
ValueError: Missing keyword RRIGM specifying IGM model to use
|
Thanks for the review. I updated the help string to "write template files into this directory instead of into the code installation" which hopefully is a little bit clearer. For the record, redrock-templates 0.7.2 is the oldest version of the templates that is compatible with the current version of the redrock code. This is the version used for EDR/Fuji. Also FYI starting with version 0.9, tags are backwards compatible by including the earlier versions of the templates and using the templates-default.txt config file to define what the current set is. |
This PR adds an
install_redrock_templatesscript, similar in spirit to theinstall_desimodel_datascript in desihub/desimodel#179. This new script installs the Redrock templates to the code data/templates directory. If$RR_TEMPLATE_DIRis set it will be used, but if it is not set then the templates installed into the code directory will be used. This enables users to install redrock + templates and then not have to also set$RR_TEMPLATE_DIR, but that envvar remains available to point Redrock to custom sets of templates if needed.I'm not suggesting any operational change to how we install and use redrock+templates at NERSC, but this simplifies work on laptops and for end-users, e.g. for tutorials.
Details
Redrock previously had partial support for default templates installed in
py/redrock/templates. I expanded this support and moved the location topy/redrock/data/templatesto go along with other data and to avoid ambiguity withpy/redrock/templates.py.I added a function
redrock.templates.get_template_dirto standardize the parsing of whether$RR_TEMPLATE_DIRis set and where to find the default templates if needed.Option
--versioncan be used to specify a specific tag or branch; default main. Main is installed as a full git clone so that it can be updated again in the future if needed; tags are installed with--depth 1to get just that tag without the rest of the history.Option
--template-urlcan overridehttps://github.com/desihub/redrock-templates, e.g. if someone wants to point to their own fork or template repo.Option
--outdircan be used to specify a different output directory.Testing
In addition to the updated unit tests, I checked this by running
install_redrock_templatesfor my copy of Redrock and then running rrdesi_mpi with and without$RR_TEMPLATE_DIRset.Outputs in /pscratch/sd/s/sjbailey/desi/redrock show the the only differences are header timestamps and the keywords recording the templates location; the data are bitwise identical.