TryParse support for request delegate source generator #46696

mitchdenny · 2023-02-16T03:28:03Z

Partially addresses: #46335

This is still a work in progress but I wanted to share the approach early to get feedback. Here I am primarily focused on getting an end to end scenario working (that is being able to call TryParse on a broad selection of types). So far the PR is supporting:

Not calling TryParse on strings ;)
Calling TryParse on Int32
Calling TryParse on enums.
Calling TryParse on complex types
Various DateTime, DateTimeOffset, DateOnly, TimeOnly types

For our contrived test cases it works, but I need to add the following to make it more bulletproof:

Validates that the TryParse method exists, I'll be pulling across ParsabilityHelper (to shared source) from our analyzer to help with this.
Selecting the right TryParse to use based on precedence rules (this will be a separate method I call which returns a Func<string,string,string>.
Make sure generated code indentation gods don't smite me.

src/Http/Http.Extensions/gen/StaticRouteHandlerModel/Emitters/EndpointParameterEmitter.cs

halter73

Looking good so far. What are we waiting for to undraft this? Fixing the indentation? If so, I agree we could probably save this for a follow up PR. Getting string and nullablity support would be great, but those could probably also be follow-ups if you want.

src/Http/Http.Extensions/gen/StaticRouteHandlerModel/EndpointParameter.cs

mitchdenny · 2023-02-22T02:14:50Z

@captainsafia and @halter73 ... ready for review. Note that I made some more changes to the ParsabilityHelper that we have from the analyzers to allow me to switch on what kind of parsing method it found.

src/Http/Http.Extensions/test/RequestDelegateGenerator/RequestDelegateGeneratorTests.cs

src/Shared/RoslynUtils/ParsabilityHelper.cs

src/Http/Http.Extensions/test/RequestDelegateGenerator/RequestDelegateGeneratorTestBase.cs

src/Http/Http.Extensions/gen/StaticRouteHandlerModel/Emitters/EndpointParameterEmitter.cs

src/Http/Http.Extensions/gen/StaticRouteHandlerModel/EndpointParameter.cs

captainsafia

LGTM! Some non-blocking things to note:

Do we support the same DateTime styles options that the RDF currently does? If not, we might want to track this as a follow-up issue.
I'm not sure what refactoring was tried here but I think some split up will be necessary when #46715 is rebased.

cc: @halter73 for sanity checking the way TryParse is invoked here

halter73 · 2023-02-22T22:11:18Z

cc: @halter73 for sanity checking the way TryParse is invoked here

I discussed this a bit with @mitchdenny yesterday. I think we tried to use the "default" NumberStyles where we could because before IParsable support, a lot of the numeric types required passing NumberStyles to TryParse if you also wanted to pass an IFormatProvider which we did for the invariant culture. Now that there's IParsable support, I think we get the same behavior by just calling that. We should verify by looking at the source code. For instance here's the NumberStyles we use for decimal:

aspnetcore/src/Shared/ParameterBindingMethodCache.cs

Lines 634 to 642 in 33e7a9a

    
           else if (type == typeof(decimal)) 
        
           { 
        
               method = typeof(decimal).GetMethod( 
        
                         nameof(decimal.TryParse), 
        
                         BindingFlags.Public | BindingFlags.Static, 
        
                         new[] { typeof(string), typeof(NumberStyles), typeof(IFormatProvider), typeof(decimal).MakeByRefType() }); 
        
               numberStyles = NumberStyles.Number; 
        
           }

And here's the IParsable implementation which also uses NumberStyles.Number:

https://github.com/dotnet/runtime/blob/86bfe2a0e2dc95eb15c124a8d18f4f8ac2bca3f5/src/libraries/System.Private.CoreLib/src/System/Decimal.cs#L1815-L1816

We should verify this for all the numeric types we special case in ParameterBindingMethodCache.TryGetNumberStylesTryGetMethod like I did for decimal. There aren't that many types we hard-code NumberStyles for in RDF, so it shouldn't be too hard to do manually. But maybe we could update the ParameterBindingMethodCache to prefer the IParsable implementation and see if all the tests still pass (other than some overly specific ones in ParameterBindingMethodCacheTests.

We should also add test cases for all the built-in types that we know support TryParse. I see there are some existing theories but they seem to be incomplete. I came up with the original test cases prior to .NET 6 by using reflection to scan the runtime for public TryParse methods. There could even be more now:

aspnetcore/src/Http/Http.Extensions/test/RequestDelegateFactoryTests.cs

Lines 579 to 609 in 33e7a9a

    
           // string is not technically "TryParsable", but it's the special case. 
        
           new object[] { (Action<HttpContext, string>)Store, "plain string", "plain string" }, 
        
           new object[] { (Action<HttpContext, int>)Store, "-42", -42 }, 
        
           new object[] { (Action<HttpContext, uint>)Store, "42", 42U }, 
        
           new object[] { (Action<HttpContext, bool>)Store, "true", true }, 
        
           new object[] { (Action<HttpContext, short>)Store, "-42", (short)-42 }, 
        
           new object[] { (Action<HttpContext, ushort>)Store, "42", (ushort)42 }, 
        
           new object[] { (Action<HttpContext, long>)Store, "-42", -42L }, 
        
           new object[] { (Action<HttpContext, ulong>)Store, "42", 42UL }, 
        
           new object[] { (Action<HttpContext, IntPtr>)Store, "-42", new IntPtr(-42) }, 
        
           new object[] { (Action<HttpContext, char>)Store, "A", 'A' }, 
        
           new object[] { (Action<HttpContext, double>)Store, "0.5", 0.5 }, 
        
           new object[] { (Action<HttpContext, float>)Store, "0.5", 0.5f }, 
        
           new object[] { (Action<HttpContext, Half>)Store, "0.5", (Half)0.5f }, 
        
           new object[] { (Action<HttpContext, decimal>)Store, "0.5", 0.5m }, 
        
           new object[] { (Action<HttpContext, Uri>)Store, "https://example.org", new Uri("https://example.org") }, 
        
           new object[] { (Action<HttpContext, DateTime>)Store, now.ToString("o"), now.ToUniversalTime() }, 
        
           new object[] { (Action<HttpContext, DateTimeOffset>)Store, "1970-01-01T00:00:00.0000000+00:00", DateTimeOffset.UnixEpoch }, 
        
           new object[] { (Action<HttpContext, TimeSpan>)Store, "00:00:42", TimeSpan.FromSeconds(42) }, 
        
           new object[] { (Action<HttpContext, Guid>)Store, "00000000-0000-0000-0000-000000000000", Guid.Empty }, 
        
           new object[] { (Action<HttpContext, Version>)Store, "6.0.0.42", new Version("6.0.0.42") }, 
        
           new object[] { (Action<HttpContext, BigInteger>)Store, "-42", new BigInteger(-42) }, 
        
           new object[] { (Action<HttpContext, IPAddress>)Store, "127.0.0.1", IPAddress.Loopback }, 
        
           new object[] { (Action<HttpContext, IPEndPoint>)Store, "127.0.0.1:80", new IPEndPoint(IPAddress.Loopback, 80) }, 
        
           new object[] { (Action<HttpContext, AddressFamily>)Store, "Unix", AddressFamily.Unix }, 
        
           new object[] { (Action<HttpContext, ILOpCode>)Store, "Nop", ILOpCode.Nop }, 
        
           new object[] { (Action<HttpContext, AssemblyFlags>)Store, "PublicKey,Retargetable", AssemblyFlags.PublicKey | AssemblyFlags.Retargetable }, 
        
           new object[] { (Action<HttpContext, int?>)Store, "42", 42 }, 
        
           new object[] { (Action<HttpContext, MyEnum>)Store, "ValueB", MyEnum.ValueB }, 
        
           new object[] { (Action<HttpContext, MyTryParseRecord>)Store, "https://example.org", new MyTryParseRecord(new Uri("https://example.org")) }, 
        
           new object?[] { (Action<HttpContext, int?>)Store, null, null },

I think the DateTimeStyles stuff still has impact. We have pretty special logic about adjusting/assuming UTC which I doubt is default.

aspnetcore/src/Shared/ParameterBindingMethodCache.cs

Lines 136 to 141 in 33e7a9a

    
           var dateTimeStyles = type switch 
        
           { 
        
               Type t when t == typeof(DateTime) => DateTimeStyles.AdjustToUniversal | DateTimeStyles.AllowWhiteSpaces, 
        
               Type t when t == typeof(DateTimeOffset) => DateTimeStyles.AssumeUniversal | DateTimeStyles.AllowWhiteSpaces, 
        
               _ => DateTimeStyles.AllowWhiteSpaces 
        
           };

We should add the equivalent following test cases for dates to the source generator tests:

aspnetcore/src/Http/Http.Extensions/test/RequestDelegateFactoryTests.cs

Lines 4253 to 4260 in 33e7a9a

    
           new object?[] { (Func<DateTime, string>)dateTimeParsing, "9/20/2021 4:18:44 PM", "Time: 2021-09-20T16:18:44.0000000, Kind: Unspecified" }, 
        
           new object?[] { (Func<DateTime, string>)dateTimeParsing, "2021-09-20 4:18:44", "Time: 2021-09-20T04:18:44.0000000, Kind: Unspecified" }, 
        
           new object?[] { (Func<DateTime, string>)dateTimeParsing, "   9/20/2021    4:18:44 PM  ", "Time: 2021-09-20T16:18:44.0000000, Kind: Unspecified" }, 
        
           new object?[] { (Func<DateTime, string>)dateTimeParsing, "2021-09-20T16:28:02.000-07:00", "Time: 2021-09-20T23:28:02.0000000Z, Kind: Utc" }, 
        
           new object?[] { (Func<DateTime, string>)dateTimeParsing, "  2021-09-20T 16:28:02.000-07:00  ", "Time: 2021-09-20T23:28:02.0000000Z, Kind: Utc" }, 
        
           new object?[] { (Func<DateTime, string>)dateTimeParsing, "2021-09-20T23:30:02.000+00:00", "Time: 2021-09-20T23:30:02.0000000Z, Kind: Utc" }, 
        
           new object?[] { (Func<DateTime, string>)dateTimeParsing, "     2021-09-20T23:30: 02.000+00:00 ", "Time: 2021-09-20T23:30:02.0000000Z, Kind: Utc" }, 
        
           new object?[] { (Func<DateTime, string>)dateTimeParsing, "2021-09-20 16:48:02-07:00", "Time: 2021-09-20T23:48:02.0000000Z, Kind: Utc" },

aspnetcore/src/Http/Http.Extensions/test/RequestDelegateFactoryTests.cs

Lines 4297 to 4300 in 33e7a9a

    
           new object?[] { (Func<DateTimeOffset, string>)dateTimeOffsetParsing, "09/20/2021 16:35:12 +00:00", "Time: 2021-09-20T16:35:12.0000000+00:00, Offset: 00:00:00" }, 
        
           new object?[] { (Func<DateTimeOffset, string>)dateTimeOffsetParsing, "09/20/2021 11:35:12 +07:00", "Time: 2021-09-20T11:35:12.0000000+07:00, Offset: 07:00:00" }, 
        
           new object?[] { (Func<DateTimeOffset, string>)dateTimeOffsetParsing, "09/20/2021 16:35:12", "Time: 2021-09-20T16:35:12.0000000+00:00, Offset: 00:00:00" }, 
        
           new object?[] { (Func<DateTimeOffset, string>)dateTimeOffsetParsing, " 09/20/2021 16:35:12 ", "Time: 2021-09-20T16:35:12.0000000+00:00, Offset: 00:00:00" },

aspnetcore/src/Http/Http.Extensions/test/RequestDelegateFactoryTests.cs

Lines 4337 to 4338 in 33e7a9a

    
           new object?[] { (Func<DateOnly, string>)dateOnlyParsing, "9/20/2021", "Time: 2021-09-20" }, 
        
           new object?[] { (Func<DateOnly, string>)dateOnlyParsing, "9 /20 /2021", "Time: 2021-09-20" },

aspnetcore/src/Http/Http.Extensions/test/RequestDelegateFactoryTests.cs

Lines 4375 to 4376 in 33e7a9a

    
           new object?[] { (Func<TimeOnly, string>)timeOnlyParsing, "4:34 PM", "Time: 16:34:00.0000000" }, 
        
           new object?[] { (Func<TimeOnly, string>)timeOnlyParsing, "    4:34 PM   ", "Time: 16:34:00.0000000" },

src/Http/Http.Extensions/gen/StaticRouteHandlerModel/EndpointParameter.cs

src/Shared/RoslynUtils/ParsabilityHelper.cs

mitchdenny · 2023-02-22T22:51:55Z

We should verify this for all the numeric types we special case in ParameterBindingMethodCache.TryGetNumberStylesTryGetMethod like I did for decimal. There aren't that many types we hard-code NumberStyles for in RDF, so it shouldn't be too hard to do manually. But maybe we could update the ParameterBindingMethodCache to prefer the IParsable implementation and see if all the tests still pass (other than some overly specific ones in ParameterBindingMethodCacheTests.

I just went and compared all the IParsable<T>.TryParse implementations with what we have in the method cache and they all line up. Given this I think we should go ahead and rely on the IParsable<T> implementation for these types in RDF like we are here in RDG.

#46788

captainsafia · 2023-02-22T22:53:40Z

Thanks for the test pointers, @halter73. I'll try to get my test consolidation branch in shape so we can take advantage of that...

halter73 · 2023-02-22T23:42:35Z

I'll try to get my test consolidation branch in shape so we can take advantage of that...

👍 In the meantime, I think it should be very quick to add all the types from RDFTests.TryParsableParameters to the new RDGTests.MapAction_SingleNumericParam_StringReturn and similar. We've already defined all the source strings and what they should be parsed to. Doing this would have caught both the Uri issue and the UTC issue with DateTime/DateTimeOffset.

src/Shared/RoslynUtils/ParsabilityMethod.cs

src/Http/Http.Extensions/gen/StaticRouteHandlerModel/EndpointParameter.cs

halter73 · 2023-02-22T23:45:02Z

We still need to use the DateTimeStyles to AllowWhiteSpaces and AdjustToUniversal/AssumeUniversal.

mitchdenny · 2023-02-24T02:26:06Z

We still need to use the DateTimeStyles to AllowWhiteSpaces and AdjustToUniversal/AssumeUniversal.

Done. Found quite a few bugs picking up these test cases. Definitely worth the journey.

Co-authored-by: Stephen Halter <[email protected]>

…arameter.cs Co-authored-by: Stephen Halter <[email protected]>

ghost added the area-runtime label Feb 16, 2023

brunolins16 reviewed Feb 16, 2023

View reviewed changes

src/Http/Http.Extensions/gen/StaticRouteHandlerModel/Emitters/EndpointParameterEmitter.cs Outdated Show resolved Hide resolved

mitchdenny force-pushed the rdg-try-parse branch from ceb0e1d to 7bd0546 Compare February 20, 2023 01:26

build-analysis bot mentioned this pull request Feb 20, 2023

Request failed "500 Internal Server Error" dotnet/arcade#11737

Open

5 tasks

halter73 reviewed Feb 21, 2023

View reviewed changes

src/Http/Http.Extensions/gen/StaticRouteHandlerModel/EndpointParameter.cs Outdated Show resolved Hide resolved

mitchdenny marked this pull request as ready for review February 22, 2023 02:13

mitchdenny requested review from Tratcher and BrennanConroy as code owners February 22, 2023 02:13

mitchdenny self-assigned this Feb 22, 2023

mitchdenny requested review from halter73 and captainsafia February 22, 2023 02:13

captainsafia reviewed Feb 22, 2023

View reviewed changes

captainsafia approved these changes Feb 22, 2023

View reviewed changes

halter73 reviewed Feb 22, 2023

View reviewed changes

src/Shared/RoslynUtils/ParsabilityMethod.cs Outdated Show resolved Hide resolved

src/Http/Http.Extensions/gen/StaticRouteHandlerModel/EndpointParameter.cs Outdated Show resolved Hide resolved

captainsafia mentioned this pull request Feb 23, 2023

Expand FromX support and add route vs. query param inference #46715

Merged

captainsafia self-requested a review February 24, 2023 16:11

mitchdenny added 5 commits February 27, 2023 10:22

WIP: End-to-end TryParse.

fcb1a8d

Handle more parsing scenarios.

dbe320d

Handle TryParse method precedence and reused more of ParsabilityHelper.

24f5574

Remove unnecessary lambda.

c840b04

PR feedback from Safia.

5e8617d

mitchdenny and others added 6 commits February 27, 2023 10:24

Apply suggestions from code review

1548ffc

Co-authored-by: Stephen Halter <[email protected]>

Update src/Http/Http.Extensions/gen/StaticRouteHandlerModel/EndpointP…

70b8732

…arameter.cs Co-authored-by: Stephen Halter <[email protected]>

WIP: Working on reusing test cases from RDF.

aa5d905

Handle RDF TryParse binding test cases.

8a1a3c9

Fix compiler errors.

2495d89

Rebase on Safia's changes.

8ba7574

mitchdenny force-pushed the rdg-try-parse branch from 7de4bfe to 8ba7574 Compare February 27, 2023 01:12

mitchdenny merged commit 446e27e into dotnet:main Feb 27, 2023

mitchdenny deleted the rdg-try-parse branch February 27, 2023 04:48

ghost added this to the 8.0-preview3 milestone Feb 27, 2023

mitchdenny mentioned this pull request Mar 3, 2023

Set up TryParse blocks for explicit header parameters. #47017

Merged

amcasey added area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions and removed area-runtime labels Aug 25, 2023

TryParse support for request delegate source generator #46696

TryParse support for request delegate source generator #46696

Uh oh!

Conversation

mitchdenny commented Feb 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

halter73 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mitchdenny commented Feb 22, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

captainsafia left a comment

Choose a reason for hiding this comment

Uh oh!

halter73 commented Feb 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mitchdenny commented Feb 22, 2023

Uh oh!

captainsafia commented Feb 22, 2023

Uh oh!

halter73 commented Feb 22, 2023

Uh oh!

Uh oh!

Uh oh!

halter73 commented Feb 22, 2023

Uh oh!

mitchdenny commented Feb 24, 2023

Uh oh!

Uh oh!

mitchdenny commented Feb 16, 2023 •

edited

Loading

halter73 commented Feb 22, 2023 •

edited

Loading