.NET 10 support - #222
Merged
Merged
.NET 10 support#222
Conversation
The four C# projects were fixed-framework MSBuild files, three of them on v4.5 and Boo.Lang on v3.5, which no current SDK can build. They become SDK-style projects on net10.0, with the settings they shared moved into Directory.Build.props, and Boo.slnx names them. This does not make the core compile yet: on net10.0 it reports 21 errors across five files, all in the emit backend and the code access security it used. Those are the next commits. Nothing regresses, because nothing built on .NET before this. - signing stays, against src/boo.snk, and each project keeps its own AssemblyInfo rather than generating one - Boo.Lang.Compiler defines NET_40_OR_GREATER outright; the old project derived it by comparing TargetFrameworkVersion, which net10.0 breaks - NoWarn covers CA2200 and the SYSLIB obsoletions on the [Serializable] surface, both pre-existing and neither affecting what is emitted default.build is untouched and keeps its own source lists: NAnt never read these project files, so the two build systems do not interact.
Two of the five files that fail to compile on net10.0, neither of them part of the emit backend. Code access security is gone: EnvironmentPermission, FileIOPermission and SecurityPermission no longer exist, and there is nothing to demand against. The three Permissions helpers keep their signatures so callers read the same, but now just run the work, still swallowing a failure the way a refused permission was. CompilerOutputType derived its values from System.Reflection.Emit.PEFileKinds, which .NET does not have. They are spelled out. Every use of the enum is by name, so the numbering only has to stay self-consistent.
The CLR implemented both on every delegate, over remoting. .NET dropped remoting and left them throwing PlatformNotSupportedException, and the step that rewrote EndInvoke recovered the delegate through System.Runtime.Remoting.Messaging.AsyncResult, which is also gone. That import is the third of the five files that fail to compile. Boo.Lang.Runtime.AsyncCall runs the call on the thread pool and carries the result, and InjectCallableConversions rewrites BeginInvoke and EndInvoke onto it. A delegate type cannot carry a method body of its own, so the calls have to be rewritten rather than implemented. - ref and out arguments are written back after the wait, since the values do not exist until the call finishes - the IAsyncResult members are public rather than explicit implementations: Boo reaches IsCompleted by name through duck typing, which only sees a type's own public members - EndInvokeCalled is carried over from the remoting AsyncResult, which exposed it and which callers read
AssemblyBuilder.Save is gone, AppDomain no longer defines assemblies, and a runtime builder cannot be written out at all. EmitAssembly and SaveAssembly are the last two files that fail to compile; with this the core builds and booc emits again. EmitAssembly always builds a PersistedAssemblyBuilder. AssemblyImage turns it into a PE image with ManagedPEBuilder, cached on the context because generating metadata twice renumbers it. GenerateInMemory loads that image, so GeneratedAssembly is a real assembly with a working entry point; SaveAssembly writes the same bytes. Three things the builder will not do for us, each in its own file: - assembly attributes whose constructor lives in the assembly being built land as a nil MethodDef, because the rows are written before the module's own handles exist. DeferredAssemblyAttributes adds them after GenerateMetadata - a StructLayout naming a Size on a sequential struct is dropped, since only explicit layouts are recorded. DeferredTypeLayouts adds those rows the same way - an assembly loaded from an image is not bound by name, so a generated assembly cannot reference another one. GeneratedAssemblies registers them and answers the load context's Resolving event Libraries carry ExecutableImage as well as Dll. Windows refuses to load an image without it while Linux and macOS never look, so omitting it fails only on Windows, and only once something references the output. Default references now come from the runtime's trusted platform assemblies, scoped to the directory holding the core library: mscorlib, System and System.Core are empty type-forwarding facades on .NET, and the host's own assemblies must not leak into a compilation. Signing is reported rather than attempted. AssemblyName.KeyPair throws here instead of being ignored, so a request to sign warns and the assembly comes out unsigned. Abstract methods no longer get an ILGenerator: the runtime builder tolerated the stray call, the persisted one rejects it with "Method body should not exist".
Independent fixes, each surfaced by a testcase once the compiler could emit and run again. - an assembly's forwarded types are catalogued alongside its own. System.Xml, System.Drawing and the rest define nothing on .NET and forward the old names, so an import naming one found nothing - ExceptionDispatchInfo.Throw resolves to the instance overload. .NET added a static Throw(Exception) beside it, and the async rewrite wants the instance one, called on a captured info - a for loop disposes only the enumerator it obtained itself. One handed in belongs to the caller, who may still be using it - a struct's size comes from its StructLayout rather than from adding up its fields - the parser stops calling its own deprecated APIs: ParseExpression builds ParserSettings directly, BooParsingStep installs the error handler once around all inputs, and WSABooParsingStep overrides the surviving ParseModule so it keeps being called - a misspelt name suggests the nearest spelling among the members that sound alike, not whichever the runtime enumerated first. Member order changed, so a misspelt CursorLeft came back as CursorSize The deprecated three-argument ParseModule stays. It is protected and virtual, so subclasses outside this repository may still override it.
Boo.Boo.targets replaces CoreCompile with a booc invocation, so references, output paths and project ordering keep coming from the .NET SDK. The .booproj files stay .booproj and import it. This is a two stage build: the C# core produces booc, which then compiles Boo.Lang.Extensions, Boo.Lang.PatternMatching and Boo.Lang.Useful from source. Nothing needs a prebuilt compiler. - the SDK's reference pack is filtered out of the booc command line. booc loads the framework from the running runtime, and the ref assemblies would collide with the implementation ones - Convert.ChangeType is qualified: System.Convert and System.Text.Encoding.Convert are both in scope, and .NET made that ambiguous - .slnx needs an explicit project type for the .booproj extension
booi, booish, Boo.Lang.Interpreter, Boo.Lang.CodeDom and Boo.Microsoft.Build.Tasks, so the solution covers the whole tree rather than the compiler alone. Two source changes, both from APIs that moved rather than went: - Path.GetDirectoryName is qualified in booi. .NET added a ReadOnlySpan overload beside the string one and Boo cannot choose between them - booish uses Trace.Listeners. Debug.Listeners is gone, and Trace captures what Debug writes Boo.Lang.CodeDom needs the System.CodeDom package, which is where System.CodeDom.Compiler went when it left the framework. The Booc task derived from ManagedCompiler, which has left the public MSBuild API, and ToolTaskExtension below it cannot be derived from outside MSBuild either. It derives from ToolTask and declares the properties it used to inherit, keeping their names so project files still bind. booc ships as a managed dll now, so the task runs it through the dotnet host rather than looking for booc.exe. booi runs a script through the in-memory pipeline, which exercises emit and entry point resolution end to end.
The test projects were fixed-framework MSBuild files referencing NUnit 2 by assembly name. They become SDK-style projects on NUnit 3 with NUnit3TestAdapter and Microsoft.NET.Test.Sdk, and Boo.slnx names them, so dotnet test runs the suite rather than finding nothing. Source changes are the NUnit 2 to 3 renames: Assert.IsInstanceOfType becomes Assert.IsInstanceOf and [TestFixtureSetUp]/[TestFixtureTearDown] become [OneTimeSetUp]/[OneTimeTearDown], across eight files. - BooSupportingClasses and BooModules get .booproj files; the testcases compile against them - BooTestCaseUtil finds the repository root by walking up to tests/testcases rather than assuming one shared output directory - tests/Directory.Build.props hands the test packages only to the C# projects. The other .booproj files under tests/ are support libraries, and the test SDK would force OutputType to Exe 2530 pass, 12 fail, 15 are skipped with their existing reasons. The 12 are testcases whose expected output or scaffolding needs updating for .NET, not compiler defects; they follow.
Every one is the testcase catching a genuine .NET behaviour change, not a compiler defect. - single.MaxValue and double.MaxValue print every digit needed to read the value back, which .NET changed - Point now reports System.Drawing.Primitives, the assembly it really lives in, rather than the facade named in the import - Debug.Listeners is gone; Trace.Listeners captures what Debug writes - BOO-707-1 built its delegate through AppDomain.DefineDynamicAssembly with AssemblyBuilderAccess.Save, and uses PersistedAssemblyBuilder now. The UIntPtr delegate under test is unchanged - RealProxy-1 is rewritten on DispatchProxy: .NET dropped remoting, so RealProxy and MarshalByRefObject transparent proxies are gone - BCE0136-1 and BCW0008-1 leaned on System.Web.Services and System.Drawing.Common for scaffolding rather than for what they test, and use XmlRoot and Point instead. Each still fails only on the mistake it was written for - interfaces-22 wants a concrete IDbCommand and takes SQLite The two WinForms cases are ignored. WinForms still ships, in Microsoft.WindowsDesktop.App, but reaching it needs a net10.0-windows target and these projects are net10.0. They would not pass on a Windows runner either. This is the only coverage the port gives up. 2538 pass, none fail, 19 are skipped with their reasons.
Travis and AppVeyor both target the NAnt build on Mono and neither service still runs for this repository. This adds a workflow that builds and tests what the SDK build produces, on the three platforms the compiler now emits for. The old .travis.yml, appveyor.yml, ci and ci.ps1 are left in place. - fail-fast is off, so one platform failing does not hide the others - test results upload as trx on every run, including failures - the actions are pinned to their current majors - TestResults, where dotnet test writes those trx files, is ignored. The existing test-results rule covers NAnt's output, not this Windows coverage is the point rather than a bonus. The emitted PE header has to carry ExecutableImage on libraries or Windows refuses to load them, and nothing on Linux or macOS notices when it is missing.
Microsoft.Build.Utilities.Core and Microsoft.Build.Tasks.Core 17.11.4 carry two high severity advisories, GHSA-h4j7-5rxr-p4wc and GHSA-w3q9-fxm7-j8fq, which NuGet reports as 44 NU1903 warnings across the solution. 18.9.6 is clear of both. The task only uses ToolTask and CommandLineBuilder, neither of which changed.
The C# build reported 30 warnings that say nothing about this code. .editorconfig marks the vendored ANTLR runtime and the five files the grammar generates as generated, and silences the three codes they produce: unreachable code, locals assigned and never read, and a call to an API since marked obsolete. Silenced by path rather than in the project, so the hand-written parser files still report them. It also records what the sources already do, so an editor stops fighting them: tabs for C# and Boo, two spaces for the XML project and config files. Two more suppressions in Directory.Build.props: - CS8981, the all-lowercase type names Boo.Lang uses by design - CS1616, which reports that AssemblyOriginatorKeyFile overrides the AssemblyKeyFile attribute in the AssemblyInfo sources. It does, and should: that attribute names ../src/boo.snk, which resolves to src/src/boo.snk and does not exist. The property is what signs, and the assemblies come out with a public key The Win32 RESOURCE structs get a scoped pragma rather than deletion. Nothing populates them yet, but they are the reference for wiring up win32 resource emission later. CS0659 stays. GenericConstructedType overrides Equals without GetHashCode, so two equal instances can land in different buckets of any dictionary keyed on IType. That is a real defect and worth leaving visible. Warnings: 74 to 2, both of them CS0659.
Windows refuses to load an image whose header omits ExecutableImage, while the loader on Linux and macOS never looks. A library missing the flag therefore fails only on Windows, and only once a later compilation references it, so nothing else in the suite catches it. Verified by reverting the flag: the library case fails with "Windows will not load an image without ExecutableImage" and passes again once restored.
booc and booi were `env mono build/booc.exe`, pointing at NAnt's output directory through Mono. Neither exists any more. dotnet build produces a real executable for each tool, so the scripts run that, honour BOO_CONFIGURATION for a release build, and say what to do when it is missing rather than failing inside Mono. booish gets one too; it built all along and had no script.
booish read `keyChar in Environment.NewLine` to detect Enter. Console.ReadKey reports Enter as CR on every platform, but Environment.NewLine is LF everywhere except Windows, so the test never matched and typing a line did nothing. It checks ConsoleKey.Enter, the way the surrounding code tests every other key.
Adds a section for dotnet build and dotnet test ahead of the existing instructions, which are left as they are, and points the badge at the CI workflow instead of Travis. Also fixes three paths that no longer resolve: testcases/integration is under tests/, and examples/hw.boo and examples/replace.boo moved into examples/misc.
PEVerify shelled out to peverify.exe or Mono's pedump. The first ships only with the .NET Framework SDK; the second fails every .NET image on System.Private.CoreLib. Nothing was checking the IL. ilverify replaces both, from the dotnet-ilverify tool. ILVerification, the library behind it, is not on nuget.org, so this stays a subprocess. The suite queues each generated assembly and verifies the lot in one invocation once every fixture has finished, naming each after the test that produced it so a failure points back at it. Assemblies go over by wildcard: Windows caps a command line at 32767 characters and the queue holds about 1400 paths. A missing verifier is reported rather than passed over, and BOO_REQUIRE_ILVERIFY=1 turns it into a failure. CI sets that and installs the tool. 1423 assemblies verify with no errors.
masonwheeler
left a comment
masonwheeler
left a comment
Contributor
There was a problem hiding this comment.
Excellent work on this PR! You clearly have a deep understanding of .NET Core and of this project, and also the patience required to make good use of that understanding. I have just a few minor requests here, but this is very, very good work overall.
| } | ||
|
|
||
| [Category("FailsOnMono")][Test] | ||
| [Ignore("System.Windows.Forms needs a net10.0-windows target; this one is net10.0")][Test] |
Contributor
There was a problem hiding this comment.
Why ignore here? It should be possible to build this project as a multi-target net10.0 and net10.0-windows, and mark these two tests as [SupportedOSPlatform("windows")].
Contributor
Author
There was a problem hiding this comment.
You're right - I am developing on a Mac so I needed to rely on CI to do the Windows testing. I put this in place, but did it a different way than you suggested.
[SupportedOSPlatform] is analyzer-only, so NUnit would still run the test on Linux. I changed it so that tests can be marked with a #platform Win comment to only include them on specific platforms.
RegressionTestFixture.cs is generated by tests/generate_regression.boo, so the marker lives in the testcase: #ignore <reason> became #platform Win, and the generator now emits [Platform] alongside [Ignore] and [Category].
I think that should be a nicer way to deal with platform specific tests going forward, but you tell me if you have a different idea that you'd prefer to see.
- No need to match the 3 part Microsoft.CSharp.targets style naming - Nothing imported it by convention, so the repetition bought nothing
- file-scoped namespaces in the files this branch adds - null-conditional and not patterns in CallTargetFor - collapse the GeneratedAssemblies lookup to TryGetValue's out value
- 10.0.0 pulled SQLitePCLRaw.lib.e_sqlite3 2.1.11, which NU1903 flags; 10.0.11 pulls 2.1.12 and the pin in unnecessary
- the test assembly targets net10.0-windows on a Windows host and picks up Microsoft.WindowsDesktop.App, so System.Windows.Forms is loadable - a #platform marker in a testcase becomes NUnit's [Platform], which skips with a reason off Windows rather than ignoring everywhere - ilverify gets every shared framework directory, not just the core one, or it cannot resolve the WinForms references
- it references BooCompiler.Tests for BooTestCaseUtil, so a net10.0 project cannot consume it once that one goes net10.0-windows
Contributor
Author
|
@masonwheeler - all commits are up from your review and CI in my fork confirms everything passes (https://github.com/mattmc3/boo/actions). The only one I left unresolved was the winforms feedback because I implemented it different than you suggested, so I wanted you to see that before it got marked resolved. |
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build and test Boo on .NET 10
Boo still builds with NAnt on Mono. The
nantscript runsmono --runtime=v4.0 NAnt.exe -t:mono-4.5, which needs Mono with the .NET 4.5 profile; the README asks for Mono 4.2.x, released in 2015.A new user likely cannot build Boo today as it stands. This PR adds a .NET 10 build alongside the existing one and gets the toolchain and test suite working again.
With this PR,
dotnet build Boo.slnxanddotnet test Boo.slnxall work on Linux, macOS and Windows. 2540 tests pass, 19 are skipped with reasons, none fail, and ilverify checks the IL of all 1423 assemblies the testcases generate.Nothing is removed
Legacy stuff like NAnt, Gradle, autotools,
bin/,extras/,build-tools/, the vendored libraries, the examples,.travis.ymlandappveyor.ymlare all untouched, though many of them won't work in a modern environment. Due to the size of this PR however, these are left untouched and can be removed in a subsequent PR.default.buildnever reads.csprojor.booproj- it has its own<sources basedir=...>lists - so converting the project files cannot affect it. The two build systems are independent.The
boocandbooiscripts did change. They ranmono build/booc.exe, and neither Mono nor that directory is around any more. These scripts now run thedotnetcommand.The emit backend
AssemblyBuilder.Saveis gone,AppDomainno longer defines assemblies, and a runtime builder cannot be written out.EmitAssemblynow builds aPersistedAssemblyBuilder, and a newAssemblyImagestep turns it into a PE image withManagedPEBuilder, cached on the compiler context because generating metadata twice renumbers it.GenerateInMemoryloads that image;SaveAssemblywrites the same bytes.Three things the builder will not do, each in its own file:
DeferredAssemblyAttributes— assembly attributes whose constructor lives inthe assembly being built land as a nil
MethodDef, because those rows arewritten before the module's own handles exist
DeferredTypeLayouts— aStructLayoutnaming aSizeon a sequential structis dropped; only explicit layouts are recorded
GeneratedAssemblies— an assembly loaded from an image is not bound by name,so a generated assembly could not reference another one
Two things that only fail on Windows
Neither is visible on Linux or macOS, which is why CI covers three platforms.
A library's PE header must carry
ExecutableImageas well asDll. Windows refuses to load an image without it and the other loaders never look, so omitting it fails only on Windows, and only once something references the output.tests/booc.Tests/EmittedImageTest.csguards this.boocruns macros and AST attributes at compile time, so it loads what it references rather than reading metadata. The SDK resolves a package reference to theref/assembly, which has no method bodies.Boo.Boo.targetspasses the implementation assembly instead.Known gaps
-embedres,-resource) and-iconthrow on the newbackend.
boocemits an assembly and nothing else. Running it needs aruntimeconfig.jsonandBoo.Lang.dllbeside it, both by hand. There is nopackaging story: no apphost, no
dotnet pack, no tool manifest. This might be worth a future PR.Microsoft.WindowsDesktop.App, but reaching it needs anet10.0-windowstarget and these projects are
net10.0. This is the only coverage given up.GenericConstructedTypeoverridesEqualswithoutGetHashCode, the oneremaining warning. Equal instances can land in different buckets of a
dictionary keyed on
IType. This warning was pre-existing, and left visible.Not in this change
There is an .editorconfig added for the purpose of defining a standard formatting, but I did not run
dotnet formatto normalize whitespace/line endings. That would create a lot of diff noise, and isn't worth it in this PR, but the .editorconfig is a helpful way to codify the standards going forward and allows ignoring certain warnings.Discord
@masonwheeler and I discussed this PR on the Boo discord. If there's other changes needed, I'm happy to help.