Like k9s, but for Slurm clusters. stui makes interacting with Slurm clusters intuitive and fast for everyone, without getting in the way of more experienced users.

• List and view nodes and jobs, filter by partition and state
• Quickly search nodes/jobs lists with regular expressions across columns, sort by any column
• Select multiple nodes/jobs and run scontrol commands on them, run scancel on jobs, or copy rows to clipboard
• View individual node details (scontrol show node equivalent)
• View individual job details (scontrol show job equivalent)
• Show sdiag output for scheduler diagnostics
• (if Slurm accounting is enabled) Explore historical job accounting from sacct tables, search across rows with regular expressions, filtering by partition and state. View individual job details (sacct -j equivalent, with all available columns)
• (if Slurm accounting is enabled) Explore sacctmgr tables, search across rows with regular expressions
• Configure table views with specific columns/content of your choice
• Optimized to minimize load on the Slurm scheduler by only fetching the data user is looking at. Default configs make ~1 request per minute after initial startup.

stui requires no configuration - if you can talk to your Slurm cluster with squeue/scontrol, you can run stui. Several configuration options are available and detailed below.

curl -sL stui.dev/install | sh 

Installation script can be found here for reference.

curl -sL github.com/Antvirf/stui/releases/latest/download/stui_Linux_x86_64.tar.gz | tar xzv stui

You can then move the binary to a more convenient location, e.g. sudo mv stui /usr/local/bin.

With go 1.22 or newer installed;

go install github.com/antvirf/stui@latest
alias stui=~/go/bin/stui

# root required only if you want to move the binary to a system path
sudo mv ~/go/bin/stui /usr/local/bin

1. Ensure your Slurm binaries are working and you can talk to your cluster, e.g. sdiag shows a valid output.

2. Run stui. Use the -help flag to view arguments for additional configuration. Any configuration value (with the exception of -config-dirs) can be set using a configuration file, see step #4 for details. Command line arguments take precedence over all configuration files.

   Usage of ./stui:
     -config-dirs string
         comma-separated list of paths to directories with stui config files (default "/etc/stui.d/,/home/$USER/.config/stui.d/")
     -copied-lines-separator string
         string to use when separating copied lines in clipboard (default "\n")
     -copy-first-column-only
         if true, only copy the first column of the table to clipboard when copying (default true)
     -disable-mouse
         disable mouse input
     -job-columns-config string
         comma-separated list of scontrol fields to show in job view, use '//' to combine column or '++' to extend columns to full width. 'JobId', 'Partitions' and 'JobState' are always shown. (default "UserId,JobName++,Comment,RunTime,NodeList,QOS,NumCPUs,Mem")
     -load-sacct-data-from duration
         load sacct data starting from this long ago, specify as a duration, e.g. '1h', '2h'. This can be very slow on busy clusters, so use with caution. Set to 0 to not load any data from sacct. (default 30m0s)
     -log-level int
         log level, 0=none, 1=error, 2=info, 3=debug (default 2)
     -node-columns-config string
         comma-separated list of scontrol fields to show in node view, use '//' to combine column or '++' to extend columns to full width. 'NodeName', 'Partition' and 'State' are always shown. (default "CPULoad//CPUAlloc//CPUTot,AllocMem//RealMemory,CfgTRES++,ActiveFeatures++,Gres++,Reason")
     -partition string
         limit views to specific partition only, leave empty to show all partitions
     -refresh-interval duration
         interval when to refetch data, specify as a duration e.g. '300ms', '1s', '2m' (default 1m0s)
     -request-timeout duration
         timeout setting for fetching data, specify as a duration e.g. '300ms', '1s', '2m' (default 20s)
     -sacct-columns-config string
         comma-separated list of sacct fields to show in job view, use '//' to combine columns or '++' to extend columns to full width. 'JobIDRaw', 'Partitions' and 'State' are always shown. (default "QOS,Account,User,JobName++,NodeList,ReqCPUS//AllocCPUS,ReqMem,Elapsed,ExitCode,ReqTRES,AllocTRES++,Comment++,SubmitLine++")
     -show-all-columns
         if set, shows all columns for Nodes, Jobs and Accounting view Jobs, overriding other specific config
     -show-keyboard-shortcuts
         print keyboard shortcuts and exit
     -slurm-binaries-path string
         path where Slurm binaries like 'sinfo' and 'squeue' can be found, if not in $PATH
     -slurm-conf-location string
         path to slurm.conf for the desired cluster, if not set, fall back to SLURM_CONF env var or configless lookup if not set
     -version
         print version information and exit

3. Keyboard shortcuts within stui:

   GENERAL SHORTCUTS
   1        Switch to Nodes view (scontrol)
   2        Switch to Jobs view (scontrol)
   3        Switch to Jobs accounting view (sacct)
   4        Switch to Accounting Manager view (sacctmgr)
   5        Switch to Scheduler view (sdiag)
   k/j      Move selection up/down in table view
   h/l      Scroll left/right in table view
   Arrows   Scroll up/down/left/right in table view
   ?        Show this help
   Ctrl+A   Select all currently filtered rows
   Ctrl+I   Invert selection within filtered rows
   Ctrl+R   Refresh currently visible data
   Ctrl+C   Exit
   o        Sort table by column
   
   SHORTCUTS IN JOB/NODE VIEW
   /        Open search bar to filter rows by regex, 'esc' to close, 'enter' to go back to table
   p        Focus on partition selector, 'esc' to close
   s        Focus on state selector, 'esc' to close
   Space    Select/deselect row
   y        Copy selected content (either rows, or currently open details) to clipboard
   c        Open 'scontrol' prompt for selected items, or current row if no selection (opens prompt)
   Enter    Show details for selected row
   Esc      Close modal
   
   ADDITIONAL SHORTCUTS IN JOBS VIEW (SCONTROL)
   Ctrl+D   Open 'scancel' prompt for selected jobs, or current row if no selection
   
   ADDITIONAL SHORTCUTS IN ACCOUNTING MANAGER VIEW (SACCTMGR)
   e        Focus on Entity type selector, 'esc' to close

4. Configuring stui using a config file - pass in -config-dirs argument or create a .yaml/.yml files in the default locations of /etc/stui.d/ or /home/$USER/.config/stui.d./. Directories are processed in the given order. Files within each directory are processed in alphabetical order. Later files take precedence. Please note that plugin configs are concatenated, not merged.

   • Any command line arguments can be configured in the file, under the argumentOptions key. See example below.
   • For plugins:
     • Full list of available keybinds can be found here.
     • If several keybinds match, first plugin defined for that page takes priority.
     • Plugins are processed after existing keybinds, and cannot override the defaults.
     • Any column in a given table view is available for use, following standard Go template syntax.

   argumentOptions:
     refresh-interval: 60s
     request-timeout: 10s
     log-level: 4 # debug logs
   
   plugins:
     - name: Sstat a job
       # Available pages: `nodes`, `jobs`, `sacct`, `sacctmgr`
       activePage: jobs
       shortcut: "Ctrl-S"
       # Any column of a particular view can be used in a command template
       command: sstat {{.JobId}}
       # Whether to execute command immediately rather than open a prompt. Default is false.
       executeImmediately: true
       # Closes prompt immediately once command is executed. Default is false.
       # Only applies if `executeImmediately` is true.
       closePromptAfterExecute: false
   
     - name: Open logs from a remote HTTP server
       activePage: jobs
       shortcut: "Ctrl-U"
       command: firefox "https://localhost:8080/{{.JobId}}" > /dev/null &
       executeImmediately: true
       closePromptAfterExecute: true
   
     - name: Check node disk usage
       activePage: nodes
       shortcut: "Ctrl-S"
       command: ssh {{.NodeName}} 'df -h /'

The below helpers configure a locally running cluster with 888 virtual nodes across several partitions to help work on stui with realistic data. This builds Slurm from scratch, so refer to Slurm docs on build dependencies.

make build-cluster      # build Slurm with required options
make config-cluster     # copy mock config to /etc/slurm/
make run-cluster        # start `slurmctld` and `slurmd`
make setup-sacct        # set up sacct
make launch-jobs        # launch few hundred sleep jobs
make stop-cluster       # stop cluster

make setup                      # install pre-commit and download Go deps
GIT_TAG=0.0.8 make gh-release   # create release commit for given tag

stui...

• is self-contained and distributed as a single binary
• requires nothing special from Slurm to operate
• is designed for keyboard first
• is light and fast
• does not over-engineer / re-implement every feature of the Slurm binaries, but acts as a light UI to make those binaries more accessible and easier to use.

Which is in contrast to other existing projects:

• sview is the official Slurm GUI, but requires build-time configurations of Slurm, has dependencies on the OS itself
• CLIP-HPC/SlurmCommander is an extensive project but no longer maintained, being incompatible with Slurm 23.02 and onward. Additionally, it relies on Slurm build-time configs (JSON parser) which makes it inoperable in certain environments.
• mil-ad/stui is an older project, implementing a small terminal user interface that only supports viewing jobs, and being written in Python, needs extra work for distribution.

This is likely the result of tmux defaulting to a different colour mode than the terminal emulator being used to run it is expecting. You can usually fix this by adding export TERM=screen-256color to your shell RC files.