Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add rule severity levels and CLI filtering #190

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
mikelolasagasti wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Add rule severity levels and CLI filtering #190

mikelolasagasti wants to merge 5 commits into from

Conversation

@mikelolasagasti
Copy link
Collaborator

@mikelolasagasti mikelolasagasti commented Nov 4, 2025

This PR introduces severity levels for checkmake rules and adds CLI options to control how violations are displayed and handled.

Summary of changes

  • Add Severity type (error, warning, info) to rules.RuleViolation.
  • Extend the Rule interface with DefaultSeverity().
  • Populate severities automatically in the validator.
  • Add CLI flags:
    • --include-info shows informational/style-level hints.
    • --errors-only displays only error-level violations.
  • Update exit behavior:
    • Checkmake now fails only if error-level violations are found.
    • Warnings and info no longer cause non-zero exit status.
  • Show severity in:
    • Text and JSON outputs.
    • list-rules command (new “Severity” column).
  • Assign default severities to built-in rules:
    • uniquetargets: error
    • phonydeclared: warning
    • minphony: warning
    • maxbodylength: info
    • timestampexpanded: warning

Motivation

Not all rules indicate real errors, some are style or best-practice recommendations.

Introducing severity levels allows users to:

  • Distinguish between errors, warnings, and informational hints.
  • Integrate checkmake more smoothly into CI pipelines (fail only on errors).
  • Explore best practices without enforcing them strictly.

Note

This initial PR doesn't include changes to README or man pages until code changes are approved.

Checklist

  • Description of proposed change
  • Documentation (README, docs/, man pages) is updated
  • Existing issue is referenced if there is one
  • Unit tests for the proposed change

Fixes #187

@mikelolasagasti
rules: introduce severity levels for rule violations
e376e45 
Adds a new Severity type ('error', 'warning', 'info') and extends the
Rule interface with DefaultSeverity(). RuleViolation now includes a
Severity field.

The validator automatically assigns a severity to violations using the
rule's default or configuration value.

This change introduces the concept of severity internally without
affecting CLI or output behavior.

Signed-off-by: Mikel Olasagasti Uranga <[email protected]>
@mikelolasagasti
cmd: add severity-based filtering and exit code handling
8b4c54d 
Adds --include-info and --errors-only flags to control which violations
are displayed. The command now exits with non-zero status only if
error-level violations are found.

Default behavior: show errors and warnings, hide info-level messages.

Also updates tests accordingly.

Signed-off-by: Mikel Olasagasti Uranga <[email protected]>
@mikelolasagasti
formatters: display severity level in text and JSON outputs
5ecc7c3 
Both the text and JSON formatters now include the severity field for
each violation. Default fallback is 'error' for backward compatibility.

Tests updated to validate presence and correctness of the severity field.

Signed-off-by: Mikel Olasagasti Uranga <[email protected]>
@mikelolasagasti
rules: define default severities for built-in rules
fe10fb1 
Each rule now defines a default severity level appropriate to its purpose:

  - UniqueTargets: error
  - Phonydeclared: warning
  - MinPhony: warning
  - MaxBodyLength: info
  - TimeStampExpanded: warning

This ensures consistent severity tagging across all rules.

Signed-off-by: Mikel Olasagasti Uranga <[email protected]>
@mikelolasagasti
cmd: show rule severity in list-rules output
3f3d2ad 
Enhances the 'list-rules' subcommand to display each rule's default
severity alongside its description.

Signed-off-by: Mikel Olasagasti Uranga <[email protected]>
@mikelolasagasti
Copy link
Collaborator Author

mikelolasagasti commented Nov 4, 2025 •
edited
Loading

With this change against https://github.com/reactive-firewall-org/multicast/blob/master/Makefile nothing is shown as Makefile is valid (against current rule set):

$ ./checkmake /tmp/multicast/Makefile
$ echo $?
0

Same Makefile but with --include-info shows maxbodylength rule hits but exits with 0:

$ ./checkmake /tmp/multicast/Makefile  --include-info
 SEVERITY      RULE           DESCRIPTION             FILE NAME         LINE NUMBER 
 info      maxbodylength  Target               /tmp/multicast/Makefile  178         
                          body for                                                  
                          "MANIFEST.in"                                             
                          exceeds                                                   
                          allowed length                                            
                          of 8 lines                                                
                          (27).                                                     
 info      maxbodylength  Target               /tmp/multicast/Makefile  292         
                          body for                                                  
                          "just-test"                                               
                          exceeds                                                   
                          allowed length                                            
                          of 8 lines                                                
                          (13).                                                     
(...)                           

$ echo $?
0

Same Makefile, but a custom config that sets maxbodylength's severity to error and exits with 1:

$ cat checkmake.ini 
[maxbodylength]
maxBodyLength = 8
severity = error

$ ./checkmake /tmp/multicast/Makefile 
 SEVERITY      RULE           DESCRIPTION             FILE NAME         LINE NUMBER 
 error     maxbodylength  Target               /tmp/multicast/Makefile  178         
                          body for                                                  
                          "MANIFEST.in"                                             
                          exceeds                                                   
                          allowed length                                            
                          of 8 lines                                                
                          (27).                                                     
(...)

$ echo $?
1

Checking against a Makefile that hits uniquetargets exits with 1:

$ ./checkmake fixtures/uniquetarget.make
 SEVERITY      RULE        DESCRIPTION            FILE NAME           LINE NUMBER 
 error     uniquetargets  Target "all"    fixtures/uniquetarget.make  14          
                          defined                                                 
                          multiple times                                          
                          (lines 5 and                                            
                          14).                                                    
 error     uniquetargets  Target "test"   fixtures/uniquetarget.make  17          
                          defined                                                 
                          multiple times                                          
                          (lines 11 and                                           
                          17).                                                    
Error: errors found (2)

$ echo $?
1

Against a Makefile hitting minphony or phonydeclared it warns and shows by default but exits with 0:

$ ./checkmake fixtures/missing_phony.make
 SEVERITY      RULE       DESCRIPTION            FILE NAME           LINE NUMBER 
 warning   minphony       Required      fixtures/missing_phony.make  22          
                          target                                                 
                          "all" must                                             
                          be declared                                            
                          PHONY.                                                 
 warning   minphony       Required      fixtures/missing_phony.make  22          
                          target                                                 
                          "test" must                                            
                          be declared                                            
                          PHONY.                                                 
 warning   phonydeclared  Target "all"  fixtures/missing_phony.make  16          
                          should be                                              
                          declared                                               
                          PHONY.                                                 

$ echo $?
0

json output also shows the severity of each hit:

$ ./checkmake -o json fixtures/*.make | jq -r '.[] | "\(.rule):\(.severity):\(.line_number)"'
Error: errors found (2)
minphony:warning:22
minphony:warning:22
phonydeclared:warning:16
minphony:warning:3
minphony:warning:3
minphony:warning:3
uniquetargets:error:14
uniquetargets:error:17

And so does list-rules:

$ ./checkmake list-rules
       NAME         SEVERITY       DESCRIPTION      
 maxbodylength      info      Target bodies should  
                              be kept simple and    
                              short (no more than 8 
                              lines).               
 minphony           warning   Minimum required      
                              phony targets         
                              must be present       
                              (all,clean,test).     
 phonydeclared      warning   Every target without  
                              a body needs to be    
                              marked PHONY          
 timestampexpanded  warning   timestamp variables   
                              should be simply      
                              expanded              
 uniquetargets      error     Targets should be     
                              uniquely defined;     
                              duplicates can cause  
                              recipe overrides or   
                              unintended merges.

@mikelolasagasti mikelolasagasti mentioned this pull request Nov 5, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

enhancement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[RFC] Introduce severity levels (error / warning / info) for rules

2 participants

@mikelolasagasti @obnoxxx