Add TRD: Tock Code Tiers #4694
Add TRD: Tock Code Tiers #4694
Conversation
doc/reference/trd-tiers.md
Outdated
|
|
||
| | Tier # | Tier | Description | | ||
| |--------|--------------|-----------------------------------------------------------------| | ||
| | 5 | Verified | Formally verified code with necessary proof | |
There was a problem hiding this comment.
I don't think "verified" is a reasonable hierarchical "security tier", esp. in comparison with the other tiers on this list.
Formal verification is one particular family of tools we can use to get assurance that our code is doing what it's supposed to, among many other types of tools. Just because something is verified doesn't mean that it's automatically important to the Tock security model. Conversely, just because something is not formally verified does not mean it is not an important component. Some things are just not (yet) tractable to be fully verified. And even then, the particular specification and guarantees matter.
I'd be much more comfortable with removing Verified from this list of tiers, and add progressively more encouragement for verification as a tool to gain high assurance to the other, higher tiers.
There was a problem hiding this comment.
In a practical sense, if a portion of Tock code is formally verified, even if that Tock code is on the periphery, would a change be expected to also be formally verified? I can imagine that going from formally verified to not formally verified would be a significant change with significant discussion. In that case, annotating that code as such is important.
There was a problem hiding this comment.
I view that as a largely orthogonal concern.
I'd argue that it's fine for a low-criticality component to go from a formally verified (for some set of guarantees), but problematic / unsuited implementation to one that is not formally verified, but better suited / cleaner / otherwise checked implementation.
I think that's not so much the case for components in critical areas of the codebase, where we ought to hold ourselves to a higher standard in terms of stability, quality, and correctness.
It's easy to assume that formally verified components are ultimately superior to anything else, but I think that's a fallacy as these can still have fundamental design issues, be inefficient, or even be entirely unsuited for their intended purpose. Depending on the exact method and tools used, and the exact specification against which the code is verified, you will obtain vastly different guarantees.
This is why I think that whether something is verified should expressly not be used to classify a component into a "tier" governing "importance to the OS, scrutiny in development, expectations for code review, and consensus needed to make changes". Instead, those tiers should inform whether formal verification is a necessary and useful tool to apply to a given component, and also which tool with what specification would be most suited to assure us of the guarantees we require.
There was a problem hiding this comment.
I do think it is important these convey the difficulty of changing the code.
I have probably been assuming "Verified" really means, in practice, "Critical and Verified". I think a strawman like verifying a obscure temperature sensor capsule illustrates the problem with that assumption, as while that would be verified it doesn't help us much to have that in the top trust tier.
I still think we should aim to have a tier above critical. I'll think more about this.
Specify this top tier is critical code ++. The explanation for the critical tier applies, but then there are also above-normal checks that must be present. This could be verification, but could be something else. This conveys that not only does the code matter, but that changing it requires adding the necessary checks.
|
Ok see what you think about "validated" code. Not married to that name. |
Pull Request Overview
Tock has something like 150 crates and 2200 source files, and the expectations for those files, in terms of importance to the OS, scrutiny in development, expectations for code review, and consensus needed to make changes, differ widely. Those expectations are largely implicit, with the understanding being based on convention, conversations in PRs, rough implications based on crate name/location, developer preference, rate of patches, importance to the threat model, and the impact of any bugs. This is hard for everyone to understand from the core team to new contributors. Having this be implicit leads to issues and can lead to issues, including:
This TRD proposes "code tiers" which is a mechanism to explicitly annotate Tock source code with a tier indicating how scrutinized and trusted the code is. This should make the implicit understanding we have today more explicit. Eventually, developers and reviewers should be able to use these tiers to better understand how changes will affect the Tock code base.
rendered
Testing Strategy
n/a
TODO or Help Wanted
The first opening of this PR is intended to socialize the idea and gather feedback. As of the original opening, we don't have a prototype of exactly what these annotations would look like.
Documentation Updated
/docs, or no updates are required.Formatting
make prepush.