Issue

An issue has been reported on the current Python SDK stating that the Custom Search Command drops fields where the fields are selectively set as part of the command.
As per the analysis of the existing code, fields are being dropped because the RecordWriter in splunklib/searchcommands/internals.py uses the first row of results in order to determine the field names.

Proposed solution

Below is our current approach to solve this issue:
To prevent fields from being dropped, we iterate over the results and create a set of all possible field names set before writing the headers for the output. This implementation can be found in a branch here: develop...DVPL-8354.

Impact

The above solution does solve the original problem but it affects the performance of the Custom Search Command. As per the analysis, we found that the proposed solution would take approximately 15-25% more time depending on the system. This was tested with datasets of increasing sizes and the performance hit is relatively the same.
To effectively improve the performance, the underlying implementation of the Custom Search Command would require changes.

Feedback requested

We would love your feedback on this issue to help us decide on the fix.

• Is a 15-25% more time in the performance of Custom Search Commands acceptable for your application in favor of having the bug fix? Please indicate with an upvote or downvote on this issue.
• Please post comments below to provide any suggestions to improve the performance without changing underlying structure and implementation of Custom Search Commands.