SetProgramOptions

The SetProgramOptions package extends ConfigParserEnhanced to enable the processing of .ini files that specify Program Options.

SetProgramOptions supports all the operations that ConfigParserEnhanced supports and adds some of its own.

Operation	Format	Defined By
use	use <section>	ConfigParserEnhanced
opt-set	opt-set Param1 [Param2..ParamN] [: <VALUE>]	SetProgramOptions
opt-remove	opt-remove Param [SUBSTR]	SetProgramOptions

INI File Format

A .ini file that can be processed by SetProgramOptions can be formatted like this:

[COMMAND_LS]
opt-set ls

This is perhaps the most simple thing we could do. Using gen_option_list('COMMAND_LS', generator="bash") would generate the command ls from this.

A more complex section which creates a CMake command call might look like this:

[COMMAND_CMAKE]
opt-set cmake
opt-set -G : Ninja
opt-set -D CMAKE_CXX_FLAGS : "-O3"

and this would generate the command cmake -G=Ninja -DCMAKE_CXX_FLAGS="-O3" when processed for bash output.

We can further epxand the CMake example with multiple sections, such as:

[CMAKE_COMMAND]
opt-set cmake
opt-set -G : Ninja

[CMAKE_OPTIONS_COMMON]
opt-set -D CMAKE_CXX_FLAGS : "-fopenmp"

[CMAKE_OPTIONS_APPLICATION]
opt-set -D MYAPP_FLAG1 : "foo"
opt-set -D MYAPP_FLAG2 : "bar"

[APPLICATION_PATH_TO_SOURCE]
opt-set /path/to/source/.

[APPLICATION_CMAKE_PROFILE_01]
use CMAKE_COMMAND
use CMAKE_OPTIONS_COMMON
use CMAKE_OPTIONS_APPLICATION
use APPLICATION_PATH_TO_SOURCE

[APPLICATION_CMAKE_PROFILE_02]
use APPLICATION_PROFILE_01
opt-remove MYAPP_FLAG2

This example is fairly simple but follows a pattern that larger projects might wish to follow when there are many configurations that may be getting tested. In this case we set up some common option groups and then create aggregation sections that will include the other sections to compose a full command line.

If we generate bash output for APPLICATION_CMAKE_PROFILE_01 we'll get cmake -G=Ninja -DCMAKE_CXX_FLAGS="-fopenmp" -DMYAPP_FLAG1="foo" -DMYAPP_FLAG2="bar" /path/to/source/.

Generating bash output for APPLICATION_CMAKE_PROFILE_02 clones APPLICATION_CMAKE_PROFILE_01 and then removes any entry containing the parameter MYAPP_FLAG2. This will result in a generated comand cmake -G=Ninja -DCMAKE_CXX_FLAGS="-fopenmp" -DMYAPP_FLAG1="foo" /path/to/source/..

Hopefully, this example shows some of the capabilities that SetProgramOptions provides for managing many build configurations within a single .ini file.

Variable Expansion within VALUE fields

Variables can be added to the VALUE fields in handled instructions, but they have their own format that must be used:

${VARNAME|VARTYPE}

• VARNAME is the variable name that you might expect for a bash style environment variable that might be defined like this: export VARNAME=VALUE.
• VARTYPE is the type of the variable that is being declared. For SetProgramOptions the only recognized type is ENV which defines environment variables. Subclasses such as SetProgramOptionsCMake define their own types.

We do not provide a default type for this because we wish it to be explicit that this is a pseudo-type and do not want it to be confused with some specific variable type since that meaning can change depending on the kind of generator being used. For example, ${VARNAME} is an environment variable within a bash context but in a CMake fragment file it would be an internal CMake variable and $ENV{VARNAME} would be an environment variable. By not providing a default we force type consideration to be made explicitly during the creation of the .ini file.

Operations Explained

use

The use operation is provided by ConfigParserEnhanced. Please see its documentation on this command and its use.

opt-set

Sets a generic command line style option.

The format of this is opt-set Param1 [Param2] [Param3] ... [ParamN] : [VALUE]

In a bash context, this operation attempts to generate an option for some command that will be executed. SetProgramOptions will concactenate the Params together and then append =VALUE if a VALUE field is present. For example, opt-set Foo Bar : Baz will become FooBar=Baz.

opt-remove

Removes existing entries that have been processed up to the point the opt-remove is encountered that match a pattern.

The format of this is opt-remove Param [SUBSTR]

When a remove is encountered, SetProgramOptions will search through all processed options and will delete any that contain any Param-i that matches Param. By default the parameters much be an exact match of Param, but if the optional SUBSTR parameter is provided then SetProgramOptions will treat Param as a substring and will remove all existing options if any parameter contains Param.

SetProgramOptions Examples

example-01.ini

[BASH_VERSION]
opt-set bash
opt-set --version

[LS_COMMAND]
opt-set ls

[LS_LIST_TIME_REVERSED]
opt-set "-l -t -r"

[LS_CUSTOM_TIME_STYLE]
opt-set --time-style : "+%Y-%m-%d %H:%M:%S"

[MY_LS_COMMAND]
use LS_COMMAND
use LS_LIST_TIME_REVERSED
use LS_CUSTOM_TIME_STYLE

example-01.py

#!/usr/bin/env python3
import setprogramoptions

filename = "example-01.ini"
section  = "MY_LS_COMMAND"

# Create SetProgramOptions instance
popts = setprogramoptions.SetProgramOptions(filename)

# Parse section
popts.parse_section(section)

# Generate the list of bash options for the command
bash_options = popts.gen_option_list(section, generator="bash")

# Print out the commands
print(" ".join(bash_options))

generates the output:

ls -l -t -r --time-style="+%Y-%m-%d %H:%M:%S"

SetProgramOptionsCMake

SetProgramOptionsCMake is a subclass of SetProgramOptions that adds additional additional operations and generators to handle processing CMake options:

• Adds opt-set-cmake-var.
• Adds cmake_fragment generator.
• Adds CMAKE type to variables.

New operations defined in SetProgramOptionsCMake:

Operation	Format	Defined By
opt-set-cmake-var	opt-set-cmake-var VARNAME [TYPE] [FORCE] [PARENT_SCOPE]: VALUE	SetProgramOptionsCMake

INI File Format

A .ini file that can be processed by SetProgramOptions can be formatted like this:

[SECTION_A]
opt-set cmake
opt-set-cmake-var MYVARIABLENAME  : VALUE
opt-set-cmake-var MYVARIABLENAME2 PARENT_SCOPE : VALUE

Variable Expansion for CMake Variables

SetProgramOptionsCMake extends the variable expansion options provided by SetProgramOptions by adding a new vartype: CMAKE which designates a variable as a "CMake variable":

${VARNAME|CMAKE}

A CMake variable in this context would be an internal variable that is known to CMake. Because this is not a variable that would be known outside of the context of .cmake files, this kind of variable is only applicable when generating CMake fragment files.

It is necessary to provide a CMake variant for variable expansions because the CMake syntax for variables is different than that used by Bash. In CMake fragment files:

• environment variables are written as $ENV{VARNAME}
• internal CMake variables are written as: ${VARNAME}

We can attempt to still allow these to be used if generating bash output but only if it can be resolved to something that is known to the calling environment (i.e., either a string or an environment variable). In this case, we cache the known values of the VARAIBLE as we process the .ini file and perform substitutions with the last known value. An exception should be thrown if the generator encounteres an unhandled CMake variable when generating bash output.

For example, to append -fopenmp to the CMAKE_CXX_FLAGS variable is something one might wish to do:

opt-set-cmake-var CMAKE_CXX_FLAGS : "${CMAKE_CXX_FLAGS|CMAKE} -fopenmp"

which is perfectly fine if we're generating a CMake fragment file:

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fopenmp")

but if the generator is Bash, it has no idea what to put in the -D option... we can't use ${CMAKE_CXX_FLAGS} because bash will think this is an environment variable. In this case, if CMAKE_CXX_FLAGS had already been set to something known then we can handle it. For example:

[COMMON_FLAGS]
opt-set-cmake-var CMAKE_CXX_FLAGS FORCE : "-O0 -g"
# ... many lines later ...
[SOME_CONFIGURATION]
use COMMON_FLAGS
opt-set-cmake-var CMAKE_CXX_FLAGS FORCE : "${CMAKE_CXX_FLAGS|CMAKE} -fopenmp"

might generate this cmake fragment:

set(CMAKE_CXX_FLAGS "-O0 -g" FORCE)
set(CMAKE_CXX_FLAGS "-O0 -g -fopenmp" FORCE)

or bash options:

-DCMAKE_CXX_FLAGS="-O0 -g"
-DCMAKE_CXX_FLAGS="-O0 -g -fopenmp"

We currently don't try and disambiguate these options internally within SetProgramOptions. This is something that is left up to the application using the tool. The reason is that in our testing it appears that the last value wins for bash commands... but within a CMake script there could be some command that uses the intermediate value of this variable and we don't currently perform any sort of use-def chain tracking. In the future, we may add some use-def awareness that could allow some optimization here.

Operations Explained

opt-set-cmake-var

This adds a CMake variable program option. These have a special syntax in bash that looks like -DVARNAME:TYPE=VALUE where the :TYPE is an optional parameter. If the type is left out then CMake assumes the value is a STRING.

We may not wish to generate bash only output though. For CMake files, we might wish to generate a cmake fragment file which is basically a snippet of CMake that can be loaded during a CMake call using the -S option: cmake -S cmake_fragment.cmake. The syntax within a CMake fragment file is the same as in a CMake script itself.

If the back-end generator is creating a CMake fragment file, the set command generated will use [CMake set syntax]. This looks something like set(<variable> <value>) but can also contain additional options. These extra options can be provided in the opt-set-cmake-var operation in the .ini file:

• FORCE -
  • By default, a set() operation does not overwrite entries in a CMake file. This can be added to force the value to be saved.
  • This is only applicable to generating cmake fragment files.
• PARENT_SCOPE - If provided, this option instructs CMake to set the variable in the scope that is above the current scope.
  • This is only applicable to generating cmake fragment files.
• TYPE - Specifies the TYPE the variable can be.
  • This is a positional argument and must always come after VARNAME.
  • Valid options for this are STRING (default), BOOL, PATH, INTERNAL, FILEPATH.
  • Adding a TYPE option implies that the CACHE and docstring parameters will be added to a set() command in a CMake fragment file according to the syntax: set(<variable> <value> CACHE <type> <docstring> [FORCE]) as illustrated on the CMake set() documentation.
  • This is applicable to both cmake fragment and bash generation.

SetProgramOptionsCMake Examples

example-02.ini

[CMAKE_COMMAND]
opt-set cmake

[CMAKE_GENERATOR_NINJA]
opt-set -G : Ninja

[CMAKE_GENERATOR_MAKEFILES]
opt-set -G : "Unix Makefiles"

[MYPROJ_OPTIONS]
opt-set-cmake-var  MYPROJ_CXX_FLAGS                         : "-O0 -fopenmp"
opt-set-cmake-var  MYPROJ_ENABLE_OPTION_A BOOL FORCE        : ON
opt-set-cmake-var  MYPROJ_ENABLE_OPTION_B BOOL PARENT_SCOPE : ON

[MYPROJ_SOURCE_DIR]
opt-set /path/to/source/dir

[MYPROJ_CONFIGURATION_NINJA]
use CMAKE_COMMAND
use CMAKE_GENERATOR_NINJA
use MYPROJ_OPTIONS
use MYPROJ_SOURCE_DIR

[MYPROJ_CONFIGURATION_MAKEFILES]
use CMAKE_COMMAND
use CMAKE_GENERATOR_MAKEFILES
use MYPROJ_OPTIONS
use MYPROJ_SOURCE_DIR

example-02.py

#!/usr/bin/env python3
import os
import sys

import setprogramoptions

filename = "example-02.ini"
popts = setprogramoptions.SetProgramOptionsCMake(filename)

section  = "MYPROJ_CONFIGURATION_NINJA"
popts.parse_section(section)

# Generate BASH output
print("")
print("Bash output")
print("-----------")
bash_options = popts.gen_option_list(section, generator="bash")
print(" \\\n   ".join(bash_options))

# Generate a CMake Fragment
print("")
print("CMake fragment output")
print("---------------------")
cmake_options = popts.gen_option_list(section, generator="cmake_fragment")
print("\n".join(cmake_options))

print("\nDone")

Generates the output:

$ python3 example-02.py

Bash output
-----------
cmake \
   -G=Ninja \
   -DMYPROJ_CXX_FLAGS="-O0 -fopenmp" \
   -DMYPROJ_ENABLE_OPTION_A:BOOL=ON \
   -DMYPROJ_ENABLE_OPTION_B:BOOL=ON \
   /path/to/source/dir

CMake fragment output
---------------------
set(MYPROJ_CXX_FLAGS "-O0 -fopenmp")
set(MYPROJ_ENABLE_OPTION_A ON CACHE BOOL "from .ini configuration" FORCE)
set(MYPROJ_ENABLE_OPTION_B ON CACHE BOOL "from .ini configuration" PARENT_SCOPE)

Done