-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Automated generation of the Asc types
#3722
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
74 commits
Select commit
Hold shift + click to select a range
9b43fe0
init import
soleinik-figment c6be7fe
recovered after merge, to some degree
soleinik-figment e613550
WIP
soleinik-figment 7e9b811
WIP
soleinik-figment 2b91a7d
WIP Add AscFromRust trait
roignpar 894ee44
Merge branch 'GRAPH-361' of github.com:figment-networks/graph-node in…
soleinik-figment 6b64895
WIP
soleinik-figment 23ef119
WIP
soleinik-figment 5b9c775
WIP
soleinik-figment 0b97eb6
WIP
soleinik-figment 96aebec
WIP
soleinik-figment ea1bbf7
WIP
soleinik-figment f4b769d
very little left
soleinik-figment 98e3f63
very little left
soleinik-figment 35cd850
very little left
soleinik-figment 7b79648
very little left
soleinik-figment 14fd248
very little left
soleinik-figment 1488b8d
version fix
soleinik-figment 4ed5d3d
Merge pull request #36 from figment-networks/GRAPH-360-ENUMS
soleinik-figment fb21e7f
imports fixed
soleinik-figment 6af26f9
enums done
soleinik-figment 050ca1e
enums done
soleinik-figment 01f43f0
Remove impossible Debug derive
roignpar e03c7f1
Add padding logic to AscType derive macro
roignpar 8b9abc0
Re-enable cosmos asc memory alignment unit tests
roignpar 5fddfa0
merged with GRAPH-360
soleinik-figment e4e0046
formatting
soleinik-figment ed8636d
unit tests fixed
soleinik-figment a6f044a
AscType derive: return early when no fields
roignpar 9b95710
Merge pull request #37 from figment-networks/asc_padding
soleinik-figment 3f3a315
padded fields unit tests
soleinik-figment 239ddfb
Commented
soleinik-figment ad52dc4
merged
soleinik-figment 64db289
merged
soleinik-figment ff69a34
firehose.required proto attribute introduced
soleinik-figment 06c0b55
firehose.required proto attribute introduced
soleinik-figment 14923c6
firehose.required proto attribute introduced
soleinik-figment c1dd899
firehose.required proto attribute introduced
soleinik-figment 2a9e68f
firehose.required proto attribute introduced
soleinik-figment bdf2b5c
firehose.required proto attribute introduced
soleinik-figment fbfc801
Merge pull request #38 from figment-networks/GRAPH-360_rename
soleinik-figment 32564ab
Graph 360 tests (#39)
soleinik-figment 8962201
merged wit upstream master
soleinik-figment e287e88
clean up
soleinik-figment 1c9dc9d
Fix the case conversion
pienkowb 5a5835d
Clean up the code
pienkowb 5d16bf9
Fix the field name
pienkowb b8254bc
Merge pull request #40 from figment-networks/GRAPH-360-cleanup
pienkowb 24c2a83
Merge branch 'master' into GRAPH-360
pienkowb 0a7e40e
Spell mistakes and Cargo.lock - syn upgrade (#41)
soleinik-figment bdbdfe1
Graph 485 (#42)
soleinik-figment 384d350
merge upstream
soleinik-figment bc3eaae
minor clean up
soleinik-figment a25d2ca
Update chain/cosmos/src/runtime/mod.rs
soleinik-figment a708e22
PR comments
soleinik-figment 2ff2ba4
Merge branch 'GRAPH-360' of github.com:figment-networks/graph-node in…
soleinik-figment 98529fe
PR comments
soleinik-figment 1520e88
WIP
soleinik-figment dc7405b
WIP
soleinik-figment 111fd86
WIP
soleinik-figment a300f94
WIP
soleinik-figment 99a1029
WIP
soleinik-figment d2e6c59
WIP
soleinik-figment 4bdbae1
WIP
soleinik-figment ee9a8e1
WIP
soleinik-figment 4e60ada
WIP
soleinik-figment 2fd8e49
v4 added
soleinik-figment 0e15f0f
Merge branch 'master' into GRAPH-362
soleinik-figment 67dcdc0
merge from master
soleinik-figment 257e7ee
formatting
soleinik-figment 82fe57f
formatting
soleinik-figment 8b4b5bf
Merge branch 'master' into GRAPH-361
soleinik-figment ae6bec8
merge from upstream & address PR comments
soleinik-figment 1d13873
merge from upstream & address PR comments
soleinik-figment File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| [package] | ||
| name = "graph-chain-common" | ||
| version = "0.27.0" | ||
| edition = "2021" | ||
|
|
||
| # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html | ||
|
|
||
| [dependencies] | ||
| protobuf = "3.0.2" | ||
| protobuf-parse = "3.0.2" | ||
| anyhow = "1" | ||
| heck = "0.4" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,218 @@ | ||
| use std::collections::HashMap; | ||
| use std::fmt::Debug; | ||
|
|
||
| use anyhow::Error; | ||
| use protobuf::descriptor::field_descriptor_proto::Label; | ||
| use protobuf::descriptor::field_descriptor_proto::Type; | ||
| use protobuf::descriptor::DescriptorProto; | ||
| use protobuf::descriptor::FieldDescriptorProto; | ||
| use protobuf::descriptor::OneofDescriptorProto; | ||
| use protobuf::Message; | ||
| use protobuf::UnknownValueRef; | ||
| use std::convert::From; | ||
| use std::path::Path; | ||
|
|
||
| const REQUIRED_ID: u32 = 66001; | ||
|
|
||
| #[derive(Debug, Clone)] | ||
| pub struct Field { | ||
| pub name: String, | ||
| pub type_name: String, | ||
| pub required: bool, | ||
| pub is_enum: bool, | ||
| pub is_array: bool, | ||
| pub fields: Vec<Field>, | ||
| } | ||
|
|
||
| #[derive(Debug, Clone)] | ||
| pub struct PType { | ||
| pub name: String, | ||
| pub fields: Vec<Field>, | ||
| pub descriptor: DescriptorProto, | ||
| } | ||
|
|
||
| impl PType { | ||
| pub fn fields(&self) -> Option<String> { | ||
| let mut v = Vec::new(); | ||
| if let Some(vv) = self.req_fields_as_string() { | ||
| v.push(vv); | ||
| } | ||
| if let Some(vv) = self.enum_fields_as_string() { | ||
| v.push(vv); | ||
| } | ||
|
|
||
| if v.len() < 1 { | ||
| None | ||
| } else { | ||
| Some(v.join(",")) | ||
| } | ||
| } | ||
|
|
||
| pub fn has_req_fields(&self) -> bool { | ||
| self.fields.iter().any(|f| f.required) | ||
| } | ||
|
|
||
| pub fn req_fields_as_string(&self) -> Option<String> { | ||
| if self.has_req_fields() { | ||
| Some(format!( | ||
| "__required__{{{}}}", | ||
| self.fields | ||
| .iter() | ||
| .filter(|f| f.required) | ||
| .map(|f| format!("{}: {}", f.name, f.type_name)) | ||
| .collect::<Vec<String>>() | ||
| .join(",") | ||
| )) | ||
| } else { | ||
| None | ||
| } | ||
| } | ||
|
|
||
| pub fn has_enum(&self) -> bool { | ||
| self.fields.iter().any(|f| f.is_enum) | ||
| } | ||
|
|
||
| pub fn enum_fields_as_string(&self) -> Option<String> { | ||
| if !self.has_enum() { | ||
| return None; | ||
| } | ||
|
|
||
| Some( | ||
| self.fields | ||
| .iter() | ||
| .filter(|f| f.is_enum) | ||
| .map(|f| { | ||
| let pairs = f | ||
| .fields | ||
| .iter() | ||
| .map(|f| format!("{}: {}", f.name, f.type_name)) | ||
| .collect::<Vec<String>>() | ||
| .join(","); | ||
|
|
||
| format!("{}{{{}}}", f.name, pairs) | ||
| }) | ||
| .collect::<Vec<String>>() | ||
| .join(","), | ||
| ) | ||
| } | ||
| } | ||
|
|
||
| impl From<&FieldDescriptorProto> for Field { | ||
| fn from(fd: &FieldDescriptorProto) -> Self { | ||
| let options = fd.options.unknown_fields(); | ||
|
|
||
| let type_name = if let Some(type_name) = fd.type_name.as_ref() { | ||
| type_name.to_owned() | ||
| } else { | ||
| if let Type::TYPE_BYTES = fd.type_() { | ||
| "Vec<u8>".to_owned() | ||
| } else { | ||
| use heck::ToUpperCamelCase; | ||
| fd.name().to_string().to_upper_camel_case() | ||
| } | ||
| }; | ||
|
|
||
| Field { | ||
| name: fd.name().to_owned(), | ||
| type_name: type_name.rsplit(".").next().unwrap().to_owned(), | ||
| required: options | ||
| .iter() | ||
| //(firehose.required) = true, UnknownValueRef::Varint(0) => false, UnknownValueRef::Varint(1) => true | ||
| .find(|f| f.0 == REQUIRED_ID && UnknownValueRef::Varint(1) == f.1) | ||
| .is_some(), | ||
| is_enum: false, | ||
| is_array: Label::LABEL_REPEATED == fd.label(), | ||
| fields: vec![], | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl From<&OneofDescriptorProto> for Field { | ||
| fn from(fd: &OneofDescriptorProto) -> Self { | ||
| Field { | ||
| name: fd.name().to_owned(), | ||
| type_name: "".to_owned(), | ||
| required: false, | ||
| is_enum: true, | ||
| is_array: false, | ||
| fields: vec![], | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl From<&DescriptorProto> for PType { | ||
| fn from(dp: &DescriptorProto) -> Self { | ||
| let mut fields = dp | ||
| .oneof_decl | ||
| .iter() | ||
| .enumerate() | ||
| .map(|(index, fd)| { | ||
| let mut fld = Field::from(fd); | ||
|
|
||
| fld.fields = dp | ||
| .field | ||
| .iter() | ||
| .filter(|fd| fd.oneof_index.is_some()) | ||
| .filter(|fd| *fd.oneof_index.as_ref().unwrap() as usize == index) | ||
| .map(|fd| Field::from(fd)) | ||
| .collect::<Vec<Field>>(); | ||
|
|
||
| fld | ||
| }) | ||
| .collect::<Vec<Field>>(); | ||
|
|
||
| fields.extend( | ||
| dp.field | ||
| .iter() | ||
| .filter(|fd| fd.oneof_index.is_none()) | ||
| .map(|fd| Field::from(fd)) | ||
| .collect::<Vec<Field>>(), | ||
| ); | ||
|
|
||
| PType { | ||
| name: dp.name().to_owned(), | ||
| fields, | ||
| descriptor: dp.clone(), | ||
| } | ||
| } | ||
| } | ||
|
|
||
| pub fn parse_proto_file<'a, P>(file_path: P) -> Result<HashMap<String, PType>, Error> | ||
| where | ||
| P: 'a + AsRef<Path> + Debug, | ||
| { | ||
| let dir = if let Some(p) = file_path.as_ref().parent() { | ||
| p | ||
| } else { | ||
| return Err(anyhow::anyhow!( | ||
| "Unable to derive parent path for {:?}", | ||
| file_path | ||
| )); | ||
| }; | ||
|
|
||
| let fd = protobuf_parse::Parser::new() | ||
| .include(dir) | ||
| .input(&file_path) | ||
| .file_descriptor_set()?; | ||
|
|
||
| assert!(fd.file.len() == 1); | ||
| assert!(fd.file[0].has_name()); | ||
|
|
||
| let file_name = file_path | ||
| .as_ref() | ||
| .clone() | ||
| .file_name() | ||
| .unwrap() | ||
| .to_str() | ||
| .unwrap(); | ||
| assert!(fd.file[0].name() == file_name); | ||
|
|
||
| let ret_val = fd | ||
| .file | ||
| .iter() //should be just 1 file | ||
| .flat_map(|f| f.message_type.iter()) | ||
| .map(|dp| (dp.name().to_owned(), PType::from(dp))) | ||
| .collect::<HashMap<String, PType>>(); | ||
|
|
||
| Ok(ret_val) | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.