Enhance and refactor Add-Type cmdlet#6141
Conversation
57c8af9 to
e34192f
Compare
daxian-dbw
left a comment
There was a problem hiding this comment.
Haven't finished all but want to share the feedback early. Will continue review from CheckDuplicateTypes.
There was a problem hiding this comment.
It allows compile a source code on PowerShell Core and Windows PowerShell with different set of usings. Sometimes .Net Core and .Net Framework is different in that.
There was a problem hiding this comment.
But the Add-Type in Windows PowerShell doesn't support this parameter in FromSource set, and thus it won't bring us this benefit. I would rather have UsingNamespace only belong to FromMember set, just like today.
There was a problem hiding this comment.
I believe the scenario can be important if someone has a portable code template and have to compile in Windows PowerShell and PowerShell Core with different usings. If the scenario is important we should enhance Windows PowerShell too. If no I'll remove this.
There was a problem hiding this comment.
Yes, will do. (I wanted reduce size of the PR)
There was a problem hiding this comment.
Is -PassThru available in all parameters set? If so, then the original [Parameter()] is more preferred as it's easy to tell that a parameter is in all parameter sets.
There was a problem hiding this comment.
The membership is under question. I want to get feedback. If no one objects to this '-PassThru available in all parameters set ' then I will use [Parameter()].
There was a problem hiding this comment.
Currently it is not in "FromAssemblyName".
I think it makes sense to add?
There was a problem hiding this comment.
Add-Type -AssemblyName F:\pscore\Microsoft.PowerShell.Commands.Utility.dll -PassThru returns all available types from the DLL, so I think we should keep the same.
There was a problem hiding this comment.
I think currently Add-Type -TypeDefinition $code -OutputAssembly f:\test.dll only produces the assembly without loading it. So is this new parameter really needed?
There was a problem hiding this comment.
We should always load assembly by default otherwise it is a breaking change.
There was a problem hiding this comment.
Add-Type -TypeDefinition $code -OutputAssembly f:\test.dll doesn't load the assembly today, so why would it be a breaking change?
I tried the following in PSCore:
[System.AppDomain]::CurrentDomain.GetAssemblies() | Out-File F:\old.txt
Add-Type -TypeDefinition $code -OutputAssembly F:\test.dll
[System.AppDomain]::CurrentDomain.GetAssemblies() | Out-File F:\new.txtOne assembly is loaded when calling Add-Type, and it's System.ValueTuple.dll. I don't see the test.dll being loaded.
There was a problem hiding this comment.
Will this handle the case that the code in those files references to symbols defined in other files?
There was a problem hiding this comment.
My understanding is yes - it is the same parser/compiler session. But I don't test this.
There was a problem hiding this comment.
Can you please verify it? ParseText is a static method, so it's hard to say for sure that one signle parser session is used for all those files.
There was a problem hiding this comment.
Done. Works well.
Add-Type -Path C:\tmp\1.cs,c:\tmp\2.cs
[Test.AddType.CSharpTest2]::Add2(1,2)
31.cs
namespace Test.AddType
{
public class CSharpTest1
{
public static int Add1(int a, int b)
{
return (a + b);
}
}
}2.cs
namespace Test.AddType
{
public class CSharpTest2
{
public static int Add2(int a, int b)
{
return CSharpTest1.Add1(a, b);
}
}
}There was a problem hiding this comment.
nit: By looking at this assertion here, I feel maybe arrange this if {} else if {} else if {} else {} in a switch of parameter name would be cleaner.
There was a problem hiding this comment.
I think it would be better if you make _syntaxTree also a local variable. It can be passed to IsSourceCodeUpdated and CSharpCompileToAssembly.
I think it's better to make it clear what a method consumes and produces in its signature, and having the arguments like compilationOptions, emitOptions, syntaxTree passed into methods helps it.
There was a problem hiding this comment.
Similar comments as in CSharpSourceCodeProcessing.
There was a problem hiding this comment.
Do we honor -PassThru in this case?
There was a problem hiding this comment.
Yes, see below in DoEmitAndLoadAssemply
There was a problem hiding this comment.
It allows compile a source code on PowerShell Core and Windows PowerShell with different set of usings. Sometimes .Net Core and .Net Framework is different in that.
There was a problem hiding this comment.
Yes, will do. (I wanted reduce size of the PR)
There was a problem hiding this comment.
errorPosition is not used anywhere. Do you intend to use it to replace lineSpan.StartLinePosition.Character at line 1397?
There was a problem hiding this comment.
Yes, thanks - it's more readable.
Done.
There was a problem hiding this comment.
I don't see the change that addresses this comment in your new commit.
There was a problem hiding this comment.
Sorry, lost commit.
Fixed.
There was a problem hiding this comment.
Maybe change to if (!char.IsWhiteSpace(errorLineString[i]))? It's more readable.
There was a problem hiding this comment.
I thought about it but there's a lot of other characters
https://msdn.microsoft.com/en-us/library/t809ektx%28v=vs.110%29.aspx
Is it ok?
There was a problem hiding this comment.
As long as they are whitespace characters, I don't see a difference. Do you?
There was a problem hiding this comment.
I fixed this but ...
This come from original code and I'm afraid the code isn't exactly correct because we're replacing one tab with a single space. Perhaps it was necessary for CodeDom. I hope that Roslyn don't use tabs in error messages. Although these whitespaces can come from the code that it compiles. Therefore we have to clean the error message too or remove this cleanup at all. Thoughts?
There was a problem hiding this comment.
Math.Max(0, lineSpan.StartLinePosition.Character - i)
Is the Math.Max call really needed? It seems to me that errorPosition would be -ge i.
There was a problem hiding this comment.
When I researched how to make the error report I was looking for code examples. Now I do not remember where I saw this pattern.
I think this is because the lineSpan and the error position are completely different Roslyn code paths - so we should protect.
There was a problem hiding this comment.
VisualBasic
Fixed. #Closed.
There was a problem hiding this comment.
I guess this applies to both csharp and vb. If that's the case, for CSharp should be removed.
There was a problem hiding this comment.
I don't found the option for VB https://github.com/dotnet/roslyn/blob/master/docs/compilers/Visual%20Basic/CommandLine.md
There was a problem hiding this comment.
OK, then never mind. #Closed.
There was a problem hiding this comment.
But the Add-Type in Windows PowerShell doesn't support this parameter in FromSource set, and thus it won't bring us this benefit. I would rather have UsingNamespace only belong to FromMember set, just like today.
There was a problem hiding this comment.
https://github.com/PowerShell/PowerShell/pull/6141/files#r171029964
I want to point it out again that this is a breaking change in that System and System.Runtime.InteropServices are not included when -UsingNamespace is not specified after this change.
There was a problem hiding this comment.
If we add this we get errors:
> $guid = [Guid]::NewGuid().ToString().Replace("-","")
> $CSharpCode1 = @"
>> using System;
>> using System.Runtime.InteropServices;
>> namespace Test.AddType
>> {
>> public class CSharpTest1$guid
>> {
>> public static int Add1(int a, int b)
>> {
>> return (a + b);
>> }
>> }
>> }
>> "@
PS C:\Users\sie\Documents\GitHub\iSazonov\PowerShell> Add-type -TypeDefinition $CSharpCode1
Add-type : (2,1): hidden CS8019: Unnecessary using directive.
using System.Runtime.InteropServices;
^
At line:1 char:1
+ Add-type -TypeDefinition $CSharpCode1
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: ((2,1): hidden C...sing directive.:CSDiagnostic) [Add-Type], Exception
+ FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand
Add-type : (1,1): hidden CS8019: Unnecessary using directive.
using System;
^
At line:1 char:1
+ Add-type -TypeDefinition $CSharpCode1
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: ((1,1): hidden C...sing directive.:CSDiagnostic) [Add-Type], Exception
+ FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand
There was a problem hiding this comment.
Currently Appveyor CI failed with the same error in log line 3827. 😕
Can I rebase to fix an error with File.wxs?
There was a problem hiding this comment.
The error seems caused by a warning that more than one using system; is specified. I'm sure there is a way to deal with it.
Feel free to rebase.
There was a problem hiding this comment.
There are two problems here:
- The assert at this location doesn't do anything useful because when the list is initially created, its count will be 0.
- I don't know how many people uses debug mode binaries. It could happen that this assert fails without us knowing. This happened before to an assert in
NativeComamndParameterBinder.
There was a problem hiding this comment.
What if -CompileOnly and -PassThru are both specified?
As mentioned in some other places, I don't think we need this parameter.
There was a problem hiding this comment.
Currently -PassThru will be ignored.
The switch came from #5725 (comment)
There was a problem hiding this comment.
As I stated in comments above #6141 (comment), when using -OutputAssembly, Add-Type only compiles the assembly without loading it. So I don't think -CompileOnly is really needed.
There was a problem hiding this comment.
Currently we always load generated assembly. So with -CompileOnly we address new scenario.
There was a problem hiding this comment.
Based on the comments in SyntaxTree, you can directly call ToString() without having to call GetText() first.
There was a problem hiding this comment.
IAssemblySymbol has a property TypeNames, of the type ICollection<string>. Maybe we can directly use that property to avoid a visitor (simplify the code)?
There was a problem hiding this comment.
Seems we have to use the recursion dotnet/roslyn#6138
There was a problem hiding this comment.
The linked issue is quite old (2015), and it doesn't mention the IAssemblySymbol.TypeNames property. If it were me, I would try this property to see if it works.
There was a problem hiding this comment.
Tested - IAssemblySymbol.TypeNames doesn't return full type names.
There was a problem hiding this comment.
AsParallel() is likely to be less performant when it comes to a small collection. In the case of Add-Type, most usages involve only a few types being compiled, so AsParallel might not serve our interest here.
If we use the property IAssemblySymbol.TypeNames, then we can decide whether to use AsParallel based on the Count (the property returns an ICollection of string).
There was a problem hiding this comment.
I'd remove the AsParallel at all but but don't found how.
There was a problem hiding this comment.
If IAssemblySymbol.TypeNames works, we won't need the visitor code :)
There was a problem hiding this comment.
AsParallel() was replaced with foreach.
Fixed.
There was a problem hiding this comment.
Question: Is the name here a namespace-fully-qualified name?
There was a problem hiding this comment.
Yes, I checked this in debuger (but I don't found implementation in Roslyn).
From our tests: "Test.AddType.CSharpTest1ed31e6c201cc4ca9b1ed0493e79434f7".
I renamed a variable in the visitor and add new comment.
There was a problem hiding this comment.
It looks to me the VisualBasicSourceCodeProcessing and CSharpSourceCodeProcessing can be merged into one method.
For example:
private CommandLineArguments ParseOptions(Language language, IEnumerable<string> args)
{
var sdkDirectory = sdkDirectory ?? s_defaultSdkDirectory;
var baseDirectory = this.SessionState.Path.CurrentLocation.Path;
switch (language)
{
case CSharp:
return CSharpCommandLineParser.Default.Parse(...);
case VB:
return VisualBasicCommandLineParser.Default.Parse(...)
}
}
private CompilationOptions GetDefaultCompilationOptions(language, outputKind)
{
switch (language)
{
case CSharp:
new CSharpCompilationOptions(OutputAssemblyTypeToOutputKind(OutputType));
case VB:
return new VisualBasicCompilationOptions(outputKind: OutputAssemblyTypeToOutputKind(OutputType));
}
}
private SyntaxTree ParseText(language, sourceText, parseOptions, file)
{
switch (lanaguage)
{
case CSharp:
return CSharptSyntextTree.ParseText(sourceText, (CSharpParseOptions) parseOptions)
case VB:
return VisualBasicSyntaxTree.ParseText(sourceText, (VBParserParseOptions) parseOptions)
}
}
private void CompileToAssembly(language, syntaxTrees, compilationOptions, emitOptions)
{
IEnumerable<PortableExecutableReference> references = GetPortableExecutableReferences();
switch (language)
{
case CSharp:
compilation = CSharpCompilation.Create(..., (CSharpCompilationOptions) compilationOptions, ...)
case VB:
compilation = VisualBasicCompilation.Create(..., (VBCompilationOptions) compilationOptions, ...)
}
DoEmitAndLoadAssemply(compilation, emitOptions);
}Then we can have a method named CompileSourceCode to replace the VisualBasicSourceCodeProcessing and CSharpSourceCodeProcessing, like this:
private void CompileSourceCode(language)
{
if (ExtendedOptions != null)
{
var arguments = ParseOptions(language, args);
...
}
else
{
compilationOptions = GetDefaultCompilationOptions(langauge);
}
...
}There was a problem hiding this comment.
Maybe the check of diagnisticRecord.IsSuppressed should be put at the outmost of this method. If isSuppressed is true, no need to go through the loop.
There was a problem hiding this comment.
Sorry, I don't understand - we get diagnisticRecord in the loop - how we can move it out of the loop?
There was a problem hiding this comment.
My bad. Never mind this comment. #Closed.
|
@iSazonov I left more comments in many of my existing comments to reply to your responses. I add |
|
@daxian-dbw I'm trying to address your comments but some might miss because Github hides them. 😄 Also the browser freezes on his too large PR. |
00d380b to
09cae71
Compare
Every time I revisit this PR, I open every hidden comment left by myself and see if you respond to my last reply. For comments that is closed, a |
|
@daxian-dbw I have collected almost all open comments for a quicker response.
I believe the scenario can be important if someone has a portable code template and have to compile in Windows PowerShell and PowerShell Core with different usings. If the scenario is important we should enhance Windows PowerShell too. If no I'll remove this. [daxian] I don't see such a change can meet the bar for Windows PowerShell. I think you can remove this for now. If a demand comes up at a later time, we can revisit this idea. [iSazonov] Fixed.
Sorry, I already forget my thoughts - Add-Type -TypeDefinition $code -OutputAssembly f:\test.dll -PassTru > $nullSo the idea is to add new parameter for better UX. [daxian] IMO, [daxian] If we don't break the current behavior, then the [daxian] For [iSazonov] Fixed.
In master branch code What is fix we want get now? Should we keep CodeDom (Windows PowerShell) default? It is main question. If so it will be again a breaking change with 6.0.0 GA. Roslyn has opposite default. I'd prefer use new parameter and new default. I assumed we could keep the
[daxian] It's broken and needs to be fixed. That parameter should have the same behavior as in Windows PowerShell. The behavior of [iSazonov] Fixed.
I fixed this but ...
[daxian] What code does [iSazonov] Thanks! Clear. Fixed.
Not addressed still [daxian] I don't see your response on another big item -- merging the VisualBasicSourceCodeProcessing and CSharpSourceCodeProcessing into one method (see my comment here). Hope you can think about it. [iSazonov] Yes, of course. I was hoping to avoid |
029802e to
5b0093a
Compare
|
CI Appveyor failed on |
Why can you not fix this? |
|
I have to delete 30 old comments in order to make the web browser able to open this PR page. Wait for the @PowerShell/powershell-committee's final call to merge this PR. |
|
@daxian-dbw this is a long PR, can you summarize what the committee needs to review? |
|
@SteveL-MSFT I need the committee's approval to merge the PR as is (with the VB related code). |
|
@PowerShell/powershell-committee is ok with taking this PR as-is and having a separate discussion about removing VB support |
|
@iSazonov Thanks for the hard work to get |
|
@daxian-dbw Thanks for great comments and help! |
|
Remove |
|
Add back the |
|
🔨 |
|
We have never considered changing FullyQualifiedName as breaking change. However, I agree to leave this label because updates to the cmdlet are significant. |
PR Summary
Fix #5515
Fix #4814
Fix #5158
Close #5725
-IgnoreWarnings- ignore warnings as errors. By default the cmdlet consider warnings as errors.-CompilerOptionsparameter allow set Roslyn command line parameters including:ATTENTION: The
CompilerOptionscan be specified along with other options like-OutputAssembly,-Languageand-IgnoreWarnings. The explicit setting parameters will take precedence over the same settings specified in-CompileOptions.See docs about the compiler options:
https://github.com/dotnet/roslyn/blob/master/docs/compilers/CSharp/CommandLine.md
https://github.com/dotnet/roslyn/blob/master/docs/compilers/Visual%20Basic/CommandLine.md
ATTENTION:
-OutputTypedefault isLibrary. If-OutputTypeis absent the-OutputTypedefault overlaps a value inCompileOptions. In other words output type ("target" ot "t" in command line) is always ignored inCompileOptions. We have to use-OutputTypeto set an output type.@TravisEz13 @PaulHigin Please audit security of the
-ExtendedOptionsparameter.PR Checklist
Note: Please mark anything not applicable to this PR
NA.[feature]if the change is significant or affects feature testsWIP:to the beginning of the title and remove the prefix when the PR is ready.