dev-guide

This README describes how the astropy-learn organization is designed, how it functions, and how to add a new tutorial.

Design

Repos

• action--execute-convert: a custom GitHub action that builds (executes and converts to html and pdf) a given notebook.
• action--pre-process: a custom GitHub action that populates the configuration script used by Jupyter Book to build a given notebook.
• astropy-tutorials: keeps material used across all tutorials, including: Python scripts used in the above custom GitHub actions to build the notebooks, a template config file for Jupyter Book (which is populated by action--pre-process and used by action--execute-convert), [dev requirements] used by _action--execute_convert, a list of valid tutorial keywords, and the site logo. Any organization-wide issues should be opened in this repo.
• learn-astropy: hosts the website homepage (https://learn.astropy.org) and serves the rendered (HTML) notebooks to the site.
• tutorial--template: a template repo for new tutorials.
• tutorial--*: each tutorial (or book) is its own repo, emulating the tutorial--template file structure.

How the repos interact

Within a tutorial-- repo, when the build.yml file is triggered (either by a push to main, on a schedule or manually), action--pre-process is called to populate the Jupyter Book config script for that notebook, with the action itself running the associated Python script kept in the astropy-tutorials repo. The default build schedule is weekly, as of this edit.

Then build.yml workflow then calls action--execute-convert (which itself runs the associated Python script kept in the astropy-tutorials repo) to build the .ipynb notebook into .html (and .pdf). This step determines if a single tutorial page or a Jupyter Book 'book' (collection of notebooks) should be built, as checked in the Python script. The Jupyter Book HTML build outputs are pushed to the converted branch in the tutorial repo.

To deploy content to the website, the learn-astropy repo's deploy.yaml workflow runs (triggered either by a push to main, on a schedule or manually) is used. The Gatsby site is built, and using a Python script the HTML outputs of all tutorials are collected from their respective converted branch in their repo and installed into the Gatsby site. The tutorials are then indexed for Algolia using learn-astropy-librarian. The site is deployed to GitHub pages for the learn-astropy repo, and this is served to the custom domain learn.astropy.org (see the corresponding DNS settings if you have the permissions to view it). The default deployment schedule is weekly, as of this edit.

Steps to add a new tutorial

1. A new tutorial (or a collection of tutorials as a Jupyter 'book') should be proposed by the authors as an issue in astropy-tutorials, which has an issue template for this. Once it is decided the proposal should be a new tutorial, make a new repo in the astropy-learn organization, selecting the tutorial--template repo as a template. Either ask the authors in the issue to open a PR to this new repo with their notebook(s), or add the notebooks yourself, along with the following changes using the information provided by the authors in the issue:

   a) Update AUTHORS.md with a comma-separated list of author names, GitHub usernames and if possible ORCID IDs, e.g.:

   Jane Doe (@janed, 0000-0000-0000-0000), John Doe (@johnd, 0000-0000-0000-0001)
   

   b) Update requirements.txt with Python requirements for the notebook (or the set of notebooks if this is a book).

   c) Add the .ipynb notebook(s) to the top-level directory. If this is a book, their filenames should be preceded by a number and an underscore, in the order they should appear in the book (starting with 1), e.g.:

   1_intro-to-modeling
   2_applying-model-to-data
   

   d) If this is a book, index.md should be renamed index-NAME.md, for example index-models.md, and it should consist of text summarizing the contents of the individual notebooks. It will be the fist page of the book. If this is a single notebook rather than a book, the file should be left as index.md and should be empty.

   e) If this is a book, _toc.yml should not be empty; it will be the book's table of contents and should have the below format. If this is a single notebook rather than a book, the file should be empty.

   format: jb-book
   root: index-modeling
   chapters:
   - file: 1_intro-to-modeling
   - file: 2_applying-model-to-data
   

   f) Include any data files needed by the notebook, if their file size is <~ 1 MB. If they are larger, do not include them, and instead ensure they are correctly linked in the notebook per the contribution guide. This will either require a PR merge with the data file(s) in the https://github.com/astropy/astropy-data repo (the data should be at http://data.astropy.org/tutorials/TUTORIAL-NAME/, and a variable like tutorialpath in the notebook should correspond to this), or the author should've uploaded the data files to Zenodo. When you add or modify data files in a commit, make sure the only thing in that commit involves the data files. That is, do not edit the notebook and add/change data files in the same commit. If you accidentally include large files in a commit, you can strip them out via git rebase -i, deleting the commits that include the data files.

2. Additionally, always do the following:

   a) Update metadata.yml with the tutorial name (or book name if this is a book), a slug of the name (expected to match the name of the .ipynb file in the case of a single notebook, case-sensitive; or to be the name of the repo in the case of a book), and the 'source' (name of the repo), e.g.:

    title: "User-defined Model"
    slug: "User-Defined-Model"
    source: "tutorial--user-defined-model"
   

   b) Ensure the notebook's first cell has the format specified in the contribution guide and that the headers in that cell have no typos (they're used to find metadata like keywords that are displayed on the site).

3. Next, review the PR with the tutorial. Once merged, the new repo's build.yml workflow will build the notebook for deployment to the site. The workflow can also be manually triggered. Finally:

a) Add a badge for the notebook's 'build' workflow to the astropy-tutorials README - copy the text there for an existing badge and update it with the new repo name in the appropriate parts.

b) The issue in astropy-tutorials can be closed.