ActiveConfigParser Class Reference

ActiveConfigParser provides extended functionality for the ConfigParser module. This class attempts to satisfy the following goals:

  1. Provide an extended “parsing” capability in .ini style files by allowing the construction of handlers that can be triggered by specially formatted options to provide enhanced parsing capabilities for .ini files.

  2. Enable chaining of [SECTIONS] within a single .ini file using the parsing capability noted in (1).

  3. Provide an extensible capability. We intend ActiveConfigParser to be used as a base class for other tools so that subclasses can add additional handlers for new ‘operations’ which can be used by the parser.

Note

ConfigParser SECTION entries are stored as a dict.

Because ConfigParser treats OPTIONS as dictionary objects storing { key: value } entries, the KEY fields must be unique within a SECTION or ConfigParser will raise a KeyError exception.

This is why the default behavior of the parser is to partition keys into three parts: <operation> <parameter> <everything else>: where <everything else> can be used to ‘uniqueify’ the overall OPTION field.

Normalization of Fields

Internally, the parser will apply a normalization transform to the operation and the parameter fields.

The following operations are applied to the operation field:

  • Replace occurrences of - with _.

Extending through Inheritance

One example of how we might extend activeconfigparser.ActiveConfigParser to add an operation that might prepend to an environment variable could be the following. Say we wish to prepend to the PATH environment variable by adding a new operation called envvar-prepend that might be invoked in this .ini snippet:

1[SET_PATH_VARS]
2envvar-prepend PATH A: /some/new/path
3envvar-prepend PATH B: /another/path/to/prepend/to/PATH

ActiveConfigParser will identify an operation envvar-prepend with a parameter PATH. It will then look for a method called _handler_envvar_prepend() to handle the operation:

 1def _handler_envvar_prepend(self, section_name, handler_parameters) -> int:
 2    op1,op2 = handler_parameters.op_params
 3    envvar_new = op2 # I think this was missing (if the envvar doesn't exist yet).
 4    if op1 in os.environ.keys():
 5        envvar_new = ":".join([
 6            op2,
 7            os.environ[op1]
 8        ])
 9    os.environ[op1] = envvar_new
10    return 0

The strings “A” and “B” on lines 2 and 3 of the .ini file are solely used to uniqueify the entries within the section SET_PATH_VARS.

Dealing with DEFAULT sections

The underlying ConfigParser instance inside activeconfigparser.ActiveConfigParser has a notion of a special default section that it can process. The name of this section can be changed but it defaults to “DEFAULT”.

Normally, the contents of DEFAULT are prepended to every section that is read by ConfigParser. This is a problem for ActiveConfigParser because the contents of this section will be re-applied to a sections content every time a use operation is encountered because use triggers the loading of a new section. When this happens, any values from the default section that had been changed will be re-set to their original values. This is not something we want, but it is useful to keep the concept of a “DEFAULT” section.

We implement this by changing the default section name for the internal ConfigParser object to a new value: CONFIGPARSERENHANCED_COMMON. Generally speaking, one should not include this in their .ini files for the reasons already presented. The name of this is controlled by the new property _internal_default_section_name.

To include the original concept of a default section we have another property called default_section_name which defaults to “DEFAULT”. This section is processed once at the start of parsing a new section and provides a behaviour that should be familiar to users of ConfigParser.

API Documentation

ActiveConfigParser

The ActiveConfigParser provides extended functionality for the ConfigParser module. This class enables configurable parsing of .ini files by splitting the key portion of an option in a configuration section into an operation and a parameter. For example, given this .ini file:

1[SECTION_NAME]
2key: value
3operation parameter: optional value
4operation parameter uniquestr: optional value

The built-in parser will process each OPTION (key: value pairs) in-order within a SECTION. During the parsing process we attempt to split the key field into two pieces consisting of an operation and a parameter.

If there is a handler method associated with the operation field that is extracted, then the parser will call that handler. Handlers are added as methods named like _handler_<operation>() (with or without the leading underscore, which denotes a method not intended to be overriden by subclasses) and will be called if they exist. For example, if the operation field resolves to use, then the parser looks for a method called _handler_use().

If the handler exists, then it is invoked. Otherwise the parser treats the OPTION field as a generic key: value pair as normal.

In this way, we can customize our processing by subclassing ActiveConfigParser and defining our own handler methods.

Authors:

Public Interface

class activeconfigparser.ActiveConfigParser(filename=None)[source]

Bases: LightweightDebugMessages, ExceptionControl

An enhanced .ini file parser built using ConfigParser.

Provides an enhanced version of the ConfigParser module, which enables some extended processing of the information provided in a .ini file.

See also

  • ConfigParser reference <https://docs.python.org/3/library/configparser.html>

__init__(filename=None)[source]

Constructor

Parameters:

filename (str,Path,list) – The .ini file or files to be loaded. If a str or Path is provided then we load only the one file. A list of strings or Path s can also be provided, which will be loaded by ConfigParser’s read() method.

property activeconfigparserdata

Enhanced configparserdata .ini file information data.

This property returns a parsed representation of the configparserdata that would be loaded from our .ini file. The data in this will return the contents of a section plus its parsed results. For example, if we have this in our .ini file:

[SEC A]
key A1: value A1

[SEC B]
use 'SEC A':
key B1: value B1

Extracting the data from ‘SEC B’ would result the contents of ‘SEC B’ + ‘SEC A’:

>>> ActiveConfigParserObj.activeconfigparserdata["SEC B"]
{
    'key A1': 'value A1',
    'key B1': 'value B1'
}

Options that resolve to a format matching an operation (operation parameter : value) what also have a handler defined (i.e, the operation is “handled”) will not be included in the data of a section. This is shown in the example above where the use 'SEC A': option in [SEC B] is not included in activeconfigparserdata["SEC B"] after processing.

Returns:

ActiveConfigParserData

Note

Subclasses should not override this.

assert_file_all_sections_handled() → int[source]

Checks that ALL the options within a file are fully handled. This calls assert_section_all_options_handled on all the sections of a .ini file.

Returns:

0 indicates a successful parse with no errors. Otherwise if problems

are detected then the return code will be nonzero. Note the return value only occurs if exception_control_level is 2 or lower (See Raises notes on this method for more details).

Return type:

int

Raises:

ValueError – A ValueError is raised if there are any unhandled options detected in the file(s) loaded in any section. The error can be suppressed by setting exception_control_level to suppress SERIOUS errors (2 or lower). If the exception is suppressed then this method will return a nonzero integer.

assert_section_all_options_handled(section_name: str, do_raise: bool = True) → int[source]

This method provides a validation capability against a section in a .ini file to validate that all of the entries are handled. The use case of this is fairly specific will flag any ‘unhandled’ options within a section under an expectation that the intent or design of the .ini file is that _all_ options are handled by some handler and _any_ that are not indicate an error or typo of some sort.

Parameters:

  • section_name (str) – The _name_ of the section that will be searched. Note: in ActiveConfigParser this includes the fully parsed section which includes the transitive closure of the dependency graph imposed by use operations.

  • do_raise (bool) – If True (default) then this assert will trigger an exception_control_event that may raise a ValueError if there are unhandled entries detected. If False then the return value is set to nonzero if unhandled entries are found but the ece is not triggered.

Returns:

0 indicates a successful parse with no errors, otherwise

any nonzero return value indicates there is at least one entry in the section that was not properly handled. IF a bad entry is found then the output will be a string containing a message that indicates what the bad entries are.

Return type:

int

Raises:

ValueError – A ValueError is raised if there are any unhandled options detected in the file(s) loaded in any section. The error can be suppressed by setting exception_control_level to suppress SERIOUS errors (2 or lower). If the exception is suppressed then this method will return a nonzero integer.

property configparser_delimiters: tuple

Delimiters for the configparser

Changing the value of this will trigger a reset of the cached data in the class since this fundamentally changes how ConfigParser will parse the .ini file.

This defaults to ('=', ':') which is the default in ConfigParser (https://docs.python.org/3/library/configparser.html)

Returns:

Returns a tuple containing the delimiters.

Return type:

tuple

Raises:

TypeError – If assignment of something other than a tuple, a list, or a str is attempted.

property configparserdata: ConfigParser

The raw results to a vanilla ConfigParser-processed .ini file.

This property is lazy-evaluated and will be processed the first time it is accessed.

Returns:

The object containing the contents of the loaded .ini file.

Return type:

ConfigParser

Raises:

  • ValueError – If the length of self.inifilepath is zero.

  • IOError – If any of the files in self.inifilepath don’t exist or are not files.

Note

Subclasses should not override this.

See also

property default_section_name
enter_handler(handler_parameters)[source]

General tasks to do when entering a handler

This executes some general operations that should be executed when entering a handler. Generally, these are just logging and console messages for debugging.

Parameters:

handler_parameters (HandlerParameters) – The parameters passed to the handler.

exit_handler(handler_parameters)[source]

General tasks to do when entering a handler

This executes some general operations that should be executed when entering a handler. Generally, these are just logging and console messages for debugging.

Parameters:

handler_parameters (HandlerParameters) – The parameters passed to the handler.

get_known_operations()[source]

Generate a list of known operation keywords based on the existing handlers defined in this class.

There is potentially some data loss here since there is nothing in ActiveConfigParser that would prevent someone from defining an operation of the form foo_bar rather than foo-bar but since the parser will always convert - to _, in terms of processing an operation there would be no distinction between foo-bar and foo_bar.

Returns:

A list of strings is returned containing the list of

known operations based on existing handlers.

Return type:

list

get_known_operations_message()[source]

Generate a string that lists valid operations.

Returns:

A string containing the message listing the valid operations.

Return type:

str

property inifilepath: list

Provides access to the path to the .ini file (or files).

inifilepath can be set to one of these things:

  1. A str contining a path to a .ini file.

  2. A pathlib.Path object pointing to a .ini file.

  3. A list of one or more of (1) or (2).

Entries in the list will be converted to pathlib.Path objects.

Returns:

A list containing the .ini files that will be processed.

Return type:

list

Note

Subclasses should not override this.

operation_handler()[source]

Implements the operation_handler decorator.

This decorator is used on handlers and implements entry and exit calls and exit value checks.

1@operation_handler
2def handler_my_operation(self, section_name, handler_parameters) -> int:
3    # do stuff
4    return 0
Parameters:

func_handler (Callable) – Reference to the handler function. This gets managed through Python’s decorator syntax.

parse_all_sections()[source]

Parse ALL sections in the .ini file.

This can be useful if the user wishes to pre-parse all the sections of an ini file in one go.

parse_section(section, initialize=True, finalize=True)[source]

Execute parser operations for the provided section.

Parameters:

  • section (str) – The section name that will be parsed and retrieved.

  • initialize (bool) – If True then handler_initialize() will be executed at the start of the search.

  • finalize (bool) – If True then handler_finalize() will be executed at the end of the search.

Returns:

data_shared property from HandlerParameters. Unless HandlerParameters is changed, this wil be a dict type.

property parse_section_last_result
unroll_to_str(section=None, space_around_delimiters=True, use_base_class_parser=True) → str[source]

Unroll a section or whole .ini file to a string

This method generates a string that is the equivalent to the original .ini file that is loaded with all the use operations processed so that the output will look like a regular .ini file.

If use_base_class_parser is set then we instantiate a base-class ActiveConfigParser that will load and (re)parse the inifilepath file(s) to generate a “clean” copy. Because all options that have handled operations in them will be stripped from the generated .ini file, the most common use case is that we just want the use operations processed and stripped, but this paramter gives the option to disable that capability.

Parameters:

  • section (str) – If a section name is provided we only generate the specified section (if it exists), otherwise we generate output for all sections.

  • space_around_delimiters (bool) – If True the delimiter is given an extra space of padding.

  • use_base_class_parser (bool) – If Tru then we instantiate a ActiveConfigParser object to parse the .ini file. If False then we use the current class (possibly a sub-class) of ActiveConfigParser, which will strip out all options that have handled operations.

Raises:

TypeError – If section is not a str or None object.

write(file_object, space_around_delimiters=True, section=None, use_base_class_parser=True) → int[source]

File writer utility for ActiveConfigParser objects.

Writes the output of unroll_to_str() to a file.

Parameters:

  • file_object (File Ptr) – Pointer to the file that should be written.

  • space_around_delimiters (bool) – If True the delimiter is given an extra space of padding.

  • section (str) – If a section name is provided we only generate the specified section (if it exists), otherwise we generate output for all sections.

Raises:

TypeError – If file_object is not a file pointer (instance or derivitive of io.IOBase).

Operation Handlers (Public)

These handlers are defined by ActiveConfigParser and may be overridden by subclasses to specialize parser behaviour. If you wish to do so, we recommend cutting and pasting the default handlers into your code and customizing from there.

ActiveConfigParser.handler_initialize(section_name, handler_parameters)[source]
ActiveConfigParser.handler_finalize(section_name, handler_parameters)[source]
ActiveConfigParser._generic_option_handler(section_name, handler_parameters)[source]

Operation Handlers (Private)

These handlers perform tasks internal to ActiveConfigParser and should not be overridden.

ActiveConfigParser._handler_use(section_name, handler_parameters)[source]

Parser Helpers (Private)

ActiveConfigParser._parse_section_r(section_name, handler_parameters=None, initialize=True, finalize=True)[source]

Recursive driver of the parser.

This is the main heavy lifter of the parser.

Parameters:

  • section_name (str) – The name of the section being processed.

  • handler_parameters (HandlerParameters) – The parameters passed to the handler.

  • initialize (bool) – If enabled and this level of recursion is the root level then we will call handler_initialize() to do some preprocessing activities.

  • finalize (bool) – If enabled and this level of recursion is the root level then we will call handler_finalize() to wrap up the recursion.

Returns:

data_shared

ActiveConfigParser._apply_transformation_to_operation(operation) → str[source]

Apply transformations to the operator parameters which are necessary for the parser and down-stream handlers.

Current transformations:

  • Repalce all occurrences of - with _

Returns:

A string containing the transformed operator parameter.

Return type:

str

ActiveConfigParser._apply_transformation_to_parameter(parameter) → str[source]

Applies transformations to the parameter field if necessary.

Current transformations:

  • none at this time

Returns:

A string containing the transformed operator parameter.

Return type:

str

ActiveConfigParser._new_handler_parameters(handler_parameters=None) → HandlerParameters[source]

Generate a new HandlerParameters object.

This is called inside the parser to generate HandlerParameters. If subclasses extend the HandlerParameters class, this can be overridden.

Parameters:

handler_parameters – an existing HandlerParameters object to copy the persistent state over from (i.e., data_shared, data_internal) for continuity in the new object.

Returns:

HandlerParameters object.

ActiveConfigParser._validate_handlerparameters(handler_parameters)[source]

Check HandlerParameters.

Check the handler_parameters object that’s being passed around to the handlers to verify that items in it have the proper types.

Raises:

TypeError – Raises a TypeError if handler_parameters.data_internal['processed_sections'] is not a set type.

ActiveConfigParser._check_handler_rval(handler_name, handler_rval) → 0[source]

Check the returned value from a handler.

Parameters:

  • handler_name (string) – The name of the handler.

  • handler_rval (int) – The return value from a handler being processed.

Returns:

Returns a 0 if everything is ok or a 1 if a warning occurred.

Return type:

int

Raises:

RuntimeError – If handler_rval > 0 and the exception_control_level is high enough, depending on the value of handler_rval.

ActiveConfigParser._launch_generic_option_handler(section_name, handler_parameters, sec_k, sec_v) → int[source]

Launcher for _generic_option_handler()

_generic_option_handler() is called in two places inside the recursive parser.

  1. It is called when the key:value pair in an option does not parse out to operation and parameter fields.

  2. It is called when the key:value pair in an option does parse to an operation and a parameter field, but there are no handlers defined for the operation.

This helper just manages the call in both places so thay’re done the same way.

Returns:

Returns the output value from _generic_option_handler()

Return type:

int

ActiveConfigParser._locate_class_method(method_name) → tuple[source]

Helper that locates a class method (if it exists)

Parameters:

method_name (str) – The name of the class method we’re searching for.

Returns:

A tuple containing the name and a reference to the method if it

is found or None if not found. (method_name, method_ref or None)

Return type:

tuple

ActiveConfigParser._locate_handler_method(operation) → tuple[source]

Convert operation to a handler name and get the reference to the handler.

This method converts the operation parameter (op) to a handler name and searches for the existence of a matching method.

Handlers may be of the format: _handler_<operation> for internal/private handlers (that should not be overridden) or handler_<operation> for handlers that are considered fair game for subclasses to override and customize.

In the case where there are both public and private handlers for the same operation defined, we will optionally raise a AmbiguousHandlerErorr if the exception_control_level is set high enough to trigger “SERIOUS” events. Otherwise, we will choose the PUBLIC handler definition to execute.

Parameters:

operation (str) – The operation parameter that is converted to a proper handler name of the form _handler_{operation}() or handler_{operation}().

Returns:

A tuple containing the (handler_name, handler_method) where:

  • handler_name is a string.

  • handler_method is either a reference to the handler method if it exists, or None if it does not exist.

Return type:

tuple

Raises:

AmbiguousHandlerError – An AmbiguousHandlerError might be raised if both a public and a private handler of the same name are defined and the exception_control_level is set to raise exceptions for “SERIOUS” events.

Private Methods

ActiveConfigParser._loginfo_add(typeinfo, entry) → None[source]

If in debug mode, we can use this to log operations by appending to _loginfo.

Parameters:

  • typeinfo (str) – The kind of operation this is. This generates the ‘type’ entry in the _loginfo dict. (Required)

  • entry (dict) – A dictionary containing log information that we’re appending. At minimum it should have: { type: typestring }.

ActiveConfigParser._loginfo_print(pretty=True) → None[source]

This is a helper to pretty-print the _loginfo object.

Inner Classes

ActiveConfigParserData

ActiveConfigParserData is an inner class of ActiveConfigParser that behaves somewhat like a partial implementation of a ConfigParser object that is customized to work in concert with ActiveConfigParser.

This class operates using lazy-evaluation of the parser method in ActiveConfigParser to process sections when requested.

The following table shows the main API and implementation differences between a ActiveConfigParserData object and a ConfigParser object, illustrating what subset of features a ActiveConfigParserData object provides:

ActiveConfigParserData Comparison

Property

activeconfigparserdata

configparser

defaults

X

sections

X

X

add_section

X

X

has_section

X

X

options

X

X

has_option

X

X

read

X

read_file

X

read_string

X

read_dict

X

get

X

X

getint

X

getfloat

X

getboolean

X

items()

X

X

items(section)

X

X

set

X

X

write

X

remove_option

X

optionxform

X

readfp

X

keys

X

__len__

X

X

__getitem__

X

X

__iter__

X

X

Public Interface

class ActiveConfigParser.ActiveConfigParserData(owner=None)[source]

Bases: LightweightDebugMessages, ExceptionControl

This class is intended to serve as a lite analog to ConfigParser to provide a similar result but with the ActiveConfigParser class’s ability to parse .ini files and follow entries that implement a use <section name> rule. In this case, when a section parses through, we will return sections with all options that were not handled by some handler.

For example, if we have this .ini file:

[SEC A]
opt1: value1-A
opt2: value2-A

[SEC B]
opt1: value1-B
use 'SEC A'

[SEC C]
use 'SEC A'
opt1: value1-C

and we pull the SEC B data from it using, say, options("SEC B"), the result we should expect is:

[SEC B]
opt1: value1-A
opt2: value2-A

but if we loaded options("SEC C") then we should get:

[SEC C]
opt1: value1-C
opt2: value2-A

Since the recursion of the use operations is a depth-first search, when there are entries with the same keys, then the ‘last one visited’ will win.

When we parse a particular section, the result for a given section name is the union of all options in the transitive closure of the directed acyclic graph generated by the use operations. For example:

[SEC A]
use 'SEC B'
opt1: value1-A
opt2: value2-A

[SEC B]
use 'SEC A'
opt1: value1-B

The results of options('SEC A') and options('SEC B) will be different:

>>> options("SEC A")
[SEC A]
opt1: value1-A
opt2: value2-A

>>> options("SEC B")
[SEC B]
opt1: value1-B
opt2: value1-A

Note

This MUST be an inner class of ActiveConfigParser because it contains a ‘hook’ back to the instance of ActiveConfigParser in in which this entry exists. This allows us to access the owner’s state so we can implement our lazy-evaluation and caching schemes. When an intance of ActiveConfigParser accesses a section via the activeconfigparserdata property, the parser will be invoked on this section to generate the result.

add_section(section, force=False)[source]

Add a new empty section.

Adds a new empty section if an existing one does not already exist. We can override this and force the new section if we enable the force option.

Parameters:

  • section (str) – The new section name.

  • force (boolean) – If True then we will FORCE the new section to be added regardless of whether or not this will overwrite an existing section.

Returns:

A dictionary containing the new section added.

Return type:

dict

property data: dict
get(section, option=None)[source]

Get a section/option pair, if it exists. If we have not parsed the section yet, we should run the parser to fully get the key data.

has_option(section, option) → bool[source]
has_section(section) → bool[source]

Checks if the section exists in the data.

Returns:

True if the section exists, False if otherwise.

Return type:

boolean

has_section_no_parse(section)[source]

Check for existence of the section without parsing

This will return True if the section exists or False if it does not. In this check, if the section does not exist the owner parser will NOT be invoked.

Returns:

True if the section exists in the data and

False if it does not.

Return type:

boolean

items(section=None)[source]

Iterator over all sections and their values in the .ini file.

keys()[source]

Returns an iterable of sections in the .ini file.

Returns:

containing the sections in the .ini file.

Return type:

iterable object

options(section)[source]
sections(parse=False)[source]

Returns an iterable of sections in the .ini file.

Parameters:

parse (str,bool) –

Determines whether or not we will parse the sections as well. This can be a string or a boolean type.

  • False: (default): Only returns the section names.

  • True : Return the section names but also execute a parse of the sections.

  • ”force”Same as True but we force a (re)parse of all sections

    even if they’ve already been parsed before.

Returns:

containing the sections in the .ini file.

Return type:

iterable object

set(section, option, value)[source]

Directly set an option. If the section is missing, we create an empty one.

ActiveConfigParser and ConfigParser Differences

There are some differences between ConfigParser and ActiveConfigParser that are worthwhile to note:

  1. ConfigParser.keys() and ActiveConfigParserData.keys() work slightly differently:

    • ConfigParser.keys() will return a KeyView object that when iterated will include the DEFAULT section even if it’s not present in the .ini file.

    • ActiveConfigParserData.keys() does not include the DEFAULT section if it does not exist in the file.