Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit f286aed

Browse files
markpeekmarkpeek
committed
First pass update to CONTRIBUTING documentation
1 parent 17019ff commit f286aed

6 files changed

Lines changed: 258 additions & 52 deletions

File tree

‎CONTRIBUTE.rst‎

Lines changed: 0 additions & 48 deletions
This file was deleted.

‎CONTRIBUTING.md‎

Lines changed: 254 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,254 @@
1+
# Contributing to troposphere
2+
3+
# How to Get Help
4+
We have a Google Group, [cloudtools-dev](https://groups.google.com/forum/#!forum/cloudtools-dev),
5+
where you can ask questions and engage with the troposphere community. Issues and pull requests are always
6+
welcome!
7+
8+
# Contributing Example Code
9+
New example code should go into `troposphere/examples`. The expected
10+
CloudFormation Template should be stored in `troposphere/tests/examples_output/`.
11+
When tests are run the output of the code in the examples directory will
12+
be compared with the expected results in the `example_output` directory.
13+
14+
# Core troposphere code base
15+
16+
## A brief historical comment
17+
18+
When the project was first created each class was handcoded from the CloudFormation documentation.
19+
Thus the code base grew organically as new validation routines and features were added. This was a
20+
bit challenging to know what new resources and properties were added periodically from AWS.
21+
22+
Eventually AWS added the Resource Specification and started publishing machine readable versions but
23+
initially it had quite a few errors and inconsistensies. An early code generator was used for adding
24+
new resources and delta changes to existing code but still needed hand tweaking.
25+
26+
An updated code generator is now available but may require tweaks to handle inconsistencies or backward compatibility.
27+
There is use of jsonpatch to handle some of these changes within the Resource Specification and the validation code has
28+
been moved into separate code files to allow the code generator to more easily update the generated classes.
29+
30+
## About backward compatibility
31+
32+
The troposphere authors strive to maintain backward compatibility within
33+
a single major versions of this library (i.e., 2.1.0 to 2.2.0 would be backward compatible but not for 2.1.0 to 3.0.0).
34+
However, there may be some minor breaks that occur in minor versions to correct errors, fix CloudFormation compatibility, and/or clarify usage.
35+
36+
## Generating troposphere code
37+
38+
The code that gets generated is for the Resources and Properties associated with CloudFormation to help determine the type of each
39+
property and whether it is a *required* field. There is other code to perform more thorough class or property validation.
40+
41+
To download a new Resource Specification:
42+
```
43+
make spec
44+
```
45+
To generate code, the current process is roughly, scan the CloudFormation history to identify changes and then run (using S3 as an example):
46+
47+
```
48+
python3 scripts/gen.py --stub --name s3 CloudFormationResourceSpecification.json > troposphere/s3.py
49+
```
50+
Use the auto-formatters to clean up the generated code using:
51+
```
52+
make fix
53+
```
54+
55+
Verify the changes using:
56+
```
57+
git diff
58+
```
59+
Further verification can be done via:
60+
```
61+
make lint test
62+
```
63+
64+
If everything looks ok, further tests can be run prior to a PR such as:
65+
```
66+
make lint
67+
```
68+
69+
## Handling errors in the Resource Specification
70+
71+
Let's walk through some of the issues that may need to be tweaked in the Resource Specification.
72+
The application of jsonpatch changes are done by the code generator by applying all of the changes
73+
locationed in `scripts/patches` looking for a `patch` list in each file.
74+
75+
### Resource and Properties using the same name
76+
77+
The Python classes used by troposphere must have unique names. But occaisionally CloudFormation services will reuse the same name.
78+
Here is one example of a Resource and Property needing to be renamed:
79+
80+
```
81+
# Rename AWS::IoTSiteWise::AccessPolicy.Portal to AWS::IoTSiteWise::AccessPolicy.PortalProperty due to conflict with Portal resource name
82+
{
83+
"op": "move",
84+
"from": "/PropertyTypes/AWS::IoTSiteWise::AccessPolicy.Portal",
85+
"path": "/PropertyTypes/AWS::IoTSiteWise::AccessPolicy.PortalProperty",
86+
},
87+
{
88+
"op": "replace",
89+
"path": "/PropertyTypes/AWS::IoTSiteWise::AccessPolicy.AccessPolicyResource/Properties/Portal/Type",
90+
"value": "PortalProperty",
91+
},
92+
```
93+
The first patch will move (rename) the Property from Portal to PortalProperty.
94+
The second patch will adjust the usage of this new name within the Property that contains it.
95+
96+
The above example replaces the *Type* field. But sometimes there is a need to use *ItemType* in cases where there is a List or Map of a type.
97+
98+
```
99+
# Rename AWS::Lightsail::Instance.Disk to AWS::Lightsail::Instance.DiskProperty
100+
{
101+
"op": "move",
102+
"from": "/PropertyTypes/AWS::Lightsail::Instance.Disk",
103+
"path": "/PropertyTypes/AWS::Lightsail::Instance.DiskProperty",
104+
},
105+
{
106+
"op": "replace",
107+
"path": "/PropertyTypes/AWS::Lightsail::Instance.Hardware/Properties/Disks/ItemType",
108+
"value": "DiskProperty",
109+
},
110+
```
111+
112+
### Backward compatibility
113+
114+
Early on it was not always clear what AWS wanted Resources and Properties named. These
115+
names have been kept historically for backward compatibility (although these might change
116+
in a future release). Thus, the names used in code generatio must be maintained.
117+
An S3 example to maintain the seage of `S3Key` instead of the current name `S3KeyFilter`.
118+
119+
```
120+
# Rename AWS::S3::Bucket.S3KeyFilter to AWS::S3::Bucket.S3Key - backward compatibility
121+
{
122+
"op": "move",
123+
"from": "/PropertyTypes/AWS::S3::Bucket.S3KeyFilter",
124+
"path": "/PropertyTypes/AWS::S3::Bucket.S3Key",
125+
},
126+
# backward compatibility
127+
{
128+
"op": "replace",
129+
"path": "/PropertyTypes/AWS::S3::Bucket.NotificationFilter/Properties/S3Key/Type",
130+
"value": "S3Key",
131+
},
132+
```
133+
134+
### Same name, different property
135+
136+
Usually the same name for a property will be the same within the same service But occasionally this is not the case.
137+
Thus names need to be made unique within a given service (file). In this example, *FieldToMatch* is used 3 times within
138+
WAFv2 with 2 different Property contents. This renames the LoggingConfiguration *FieldToMatch* to *LoggingConfigurationFieldToMatch*.
139+
140+
```
141+
{
142+
"op": "move",
143+
"from": "/PropertyTypes/AWS::WAFv2::LoggingConfiguration.FieldToMatch",
144+
"path": "/PropertyTypes/AWS::WAFv2::LoggingConfiguration.LoggingConfigurationFieldToMatch",
145+
},
146+
{
147+
"op": "replace",
148+
"path": "/ResourceTypes/AWS::WAFv2::LoggingConfiguration/Properties/RedactedFields/ItemType",
149+
"value": "LoggingConfigurationFieldToMatch",
150+
},
151+
```
152+
153+
## Validating types and classes
154+
155+
One of the core reasons to use troposphere is to help with the validation of the CloudFormation template prior to applying it into AWS.
156+
The code generator will usually do type validation to ensure the correct type is used for a property and tje *required* field which will
157+
warn if thereare missing required fields. The other validators allow for functions to do further type and value validation along with
158+
class validation.
159+
160+
### Property type function validators
161+
162+
For some primitive types (Boolean, Integer, Double, String), the code generator will insert a type validator automatically.
163+
This helps ensure the given value can be coerced into a the correct type. Here are two examples for boolean and integer validation:
164+
165+
```
166+
def boolean(x):
167+
if x in [True, 1, "1", "true", "True"]:
168+
return True
169+
if x in [False, 0, "0", "false", "False"]:
170+
return False
171+
raise ValueError
172+
173+
174+
def integer(x):
175+
try:
176+
int(x)
177+
except (ValueError, TypeError):
178+
raise ValueError("%r is not a valid integer" % x)
179+
else:
180+
return x
181+
```
182+
183+
Another one is to verfify a network port is an integer and checks for either -1 or the port to be between 0 and 65535:
184+
185+
```
186+
def network_port(x):
187+
from .. import AWSHelperFn
188+
189+
# Network ports can be Ref items
190+
if isinstance(x, AWSHelperFn):
191+
return x
192+
193+
i = integer(x)
194+
if int(i) < -1 or int(i) > 65535:
195+
raise ValueError("network port %r must been between 0 and 65535" % i)
196+
return x
197+
```
198+
199+
Most properties can have a helper function (If, FindInMap, Ref, etc.) so there is a check for those included.
200+
201+
### Property value function validators
202+
203+
Another use of validators is to ensure the value is correct. This is usually fields that accept a limited set
204+
of strings for their value. Here is an example for
205+
[AWS::S3::Bucket AccelerateConfiguration](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-accelerateconfiguration.html)
206+
which requires the field to be either "Enabled" or "Suspended":
207+
208+
```
209+
def s3_transfer_acceleration_status(value):
210+
"""
211+
Property: AccelerateConfiguration.AccelerationStatus
212+
"""
213+
valid_status = ["Enabled", "Suspended"]
214+
if value not in valid_status:
215+
raise ValueError(
216+
'AccelerationStatus must be one of: "%s"' % (", ".join(valid_status))
217+
)
218+
return value
219+
```
220+
221+
In the past these validators were included in the main code base (`troposphere/*.py`) but are now located in
222+
`troposphere/validators` directory with corresponding names (`troposphere/validators/s3.py`) in the case of the above.
223+
Note the docstring contains `Property: AccelerateConfiguration.AccelerationStatus` which is parse by the code generator
224+
to apply this validator to the correct Property. This can be used multiple times since the same validator could apply in
225+
several different places.
226+
227+
### Class function validators
228+
229+
A class function validator will usually look at several different properties to determine if the class is valid.
230+
As mentioned above, these used to be in the main code but are now in the validation directory.
231+
232+
Some simple examples from CodeDeploy:
233+
234+
The LoadBalancerInfo property must have either an ElbInfoList or TargetGroupInfoList defined.
235+
```
236+
def validate_load_balancer_info(self):
237+
"""
238+
Class: LoadBalancerInfo
239+
"""
240+
conds = ["ElbInfoList", "TargetGroupInfoList"]
241+
exactly_one(self.__class__.__name__, self.properties, conds)
242+
```
243+
244+
This shows where fields must be mutually exclusive.
245+
```
246+
def validate_deployment_group(self):
247+
"""
248+
Class: DeploymentGroup
249+
"""
250+
ec2_conds = ["EC2TagFilters", "Ec2TagSet"]
251+
onPremises_conds = ["OnPremisesInstanceTagFilters", "OnPremisesTagSet"]
252+
mutually_exclusive(self.__class__.__name__, self.properties, ec2_conds)
253+
mutually_exclusive(self.__class__.__name__, self.properties, onPremises_conds)
254+
```

‎docs/conf.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
4040
# ones.
4141
extensions = [
42+
"myst_parser",
4243
"sphinx.ext.autodoc",
4344
"sphinx.ext.coverage",
4445
"sphinx.ext.mathjax",

‎docs/contribute.rst‎

Lines changed: 0 additions & 3 deletions
This file was deleted.

‎docs/contributing.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
```{include} ../CONTRIBUTING.md
2+
```

‎docs/index.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
apis/tests_toc
1616

1717
changelog
18-
contribute
18+
contributing
1919
code_of_conduct
2020
license
2121

0 commit comments

Comments
 (0)