Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@jakub-zwolakowski
Copy link

Hi, @hawicz!

I contacted you in June about deploying the new tool TrustInSoft CI on JSON-C: #641

At the time you gave me valuable feedback concerning both what you found advantageous and inconvenient about the tool.

In the meantime TIS CI has matured and we have learned more about how to deploy it on open-source projects. I would like to propose you a new TIS CI configuration for JSON-C, which addresses your concerns.

First, you were concerned about compromising the readability of your code just to make some checks in TIS CI pass. We have added new options that allow to ignore certain alarms if you consider them spurious.

Second, we have learned how to make TIS CI configurations more robust and more advantageous for long-term Continuous Integration. The script regenerate.py can be used to recreate all the TIS CI configuration files automatically. And a whole batch of tests generated through fuzzing was included to increase the verification coverage and spot regressions.

About the files that are part of this PR:

  • tis.config : The main configuration file that describes all the 1377 tests to analyze (32 from your test suite, the rest from inputs created through fuzzing).
  • trustinsoft/common.config : Analysis options common to all tests.
  • trustinsoft/fuzz_inputs/*.json and trustinsoft/test_fuzz.c : The fuzzing inputs (generated from https://github.com/nst/JSONTestSuite) and the test driver for analyzing them.
  • trustinsoft/include/*.h : Three header files (generated when building JSON-C) that TIS CI requires for analyses.
  • trustinsoft/regenerate.py : A python script which recreates the TIS CI configuration files and which copies the 3 generated header files to trustinsoft/include.
  • README.md : Includes a TIS CI badge that displays the latest analysis status.

In this new configuration almost all the tests are green. That's because:

  1. the UBs which you considered worth fixing in June were already fixed,
  2. and I have configured TIS CI to ignore the UBs which you considered not worth fixing.

The one exception is the single red test which concerns the json_tokener_new_ex function. You explained that this function expects the depth parameter to be non-negative. So instead of looking for the problem inside the function's body I gave it a precondition depth >= 0. This precondition is violated by this test because it calls json_tokener_new_ex with depth equal -2. That's why the test is red.

Please let me know if this configuration is satisfactory to you or if you have any more concerns.

@coveralls
Copy link

Coverage Status

Coverage remained the same at 87.427% when pulling c66722c on jakub-zwolakowski:tis into df27756 on json-c:master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants