zio-test: make `assert` capture expression and source location using macro #4427

dkarlinsky · 2020-11-21T20:30:26Z

Most tests verifying assert output are failing and need to be fixed.

adamgfraser · 2020-11-21T20:33:43Z

@dkarlinsky Great! How does this interrelate with #4242 by @hmemcpy?

dkarlinsky · 2020-11-21T20:37:07Z

It doesn't yet. We talked about it and agreed that I'd rebase on top of @hmemcpy 's PR, once it's merged.

adamgfraser · 2020-11-24T04:28:57Z

@dkarlinsky @hmemcpy Can we circle up on the two open PRs we have on this? It seems like this PR has a way of capturing the source information that results in fewer changes to the public API. @hmemcpy Does the way the information is being captured here give you the ability to access it and do what you want with it?

hmemcpy · 2020-11-24T07:14:58Z

I need to take a better look at this, but I think this could work for my case too! It seems to be extracting the location from the tree rather than capturing it in a separate object... Might work! This would mean I'd have to have to change the implementation of test and testM to point to the private macro-based implementation, but their public signature wouldn't change!

I'll experiment with it. Thanks!

dkarlinsky · 2020-11-24T18:10:53Z

It doesn't yet. We talked about it and agreed that I'd rebase on top of @hmemcpy 's PR, once it's merged.

Looking at @hmemcpy 's PR again, there doesn't seem to be an actual overlap in code.
So maybe they can proceed in parallel

hmemcpy · 2020-11-24T18:12:59Z

I want to give it a try your way, capture the Tree of the label element of test/testM, so I don't have to introduce the implicit parameter. I can piggyback off your changes, so if you can finish this PR, I'll rebase mine on top instead!

adamgfraser · 2020-11-24T18:22:47Z

@dkarlinsky You've definitely tackled it slightly different ways but are trying to address similar goals. Anyway, I think the approach suggested by @hmemcpy makes sense.

hmemcpy · 2020-11-24T21:34:22Z

So good news - it indeed works! I was able to capture the location info of the test/testM inside a macro without needing to change the public API!

This means that similarly to assert, the test methods will move into the CompileVariant interface, implemented by a macro that will call into the original implementation, but augmenting it with the location.

I'd still like to put my changes in next, are you close to finishing the PR, @dkarlinsky?

dkarlinsky · 2020-11-24T21:59:26Z

So good news - it indeed works! I was able to capture the location info of the test/testM inside a macro without needing to change the public API!

This means that similarly to assert, the test methods will move into the CompileVariant interface, implemented by a macro that will call into the original implementation, but augmenting it with the location.

I'd still like to put my changes in next, are you close to finishing the PR, @dkarlinsky?

Just need to fix all the broken testis in DefaultTestReporterSpec and SummaryBuilderSpec that verify assert output.
Might be a little tricky, as now the output includes the absolute path to the source file.
(I would really prefer to make the path relative to the source root, but can't find anything in macro reflect api that helps figure out what the root is)

hmemcpy · 2020-11-24T22:04:14Z

I think if you keep just the filename, IntelliJ will be smart enough to hyperlink it anyway. Not sure this would work for the console output. But I agree, relative path to the source would be better.

dkarlinsky · 2020-11-24T22:13:30Z

I think if you keep just the filename, IntelliJ will be smart enough to hyperlink it anyway. Not sure this would work for the console output. But I agree, relative path to the source would be better.

Didn't work for me in console output - if it's just the file name, no hyperlink :(

…rivate

ignoring line numbers in test output - otherwise the tests get too brittle.

hmemcpy · 2020-11-26T17:07:29Z

Hallelujah 🤣

dkarlinsky · 2020-11-26T17:09:47Z

Hallelujah 🤣

Yay! Finally! 😅

hmemcpy · 2020-11-27T10:59:59Z

@adamgfraser would love if you could take a look.

adamgfraser

I think this looks really good!

For the rendering of the results, right now we're also labeling the assert with the left hand side of the assertion. Do you think that is necessary or maybe we can just render the line of the failing test as a new line? Just wondering if it could be confusing if users can add their own labeling and now we are adding this labeling? Also happy to address the rendering in a follow up PR.

dkarlinsky · 2020-11-27T19:04:27Z

I think this looks really good!

For the rendering of the results, right now we're also labeling the assert with the left hand side of the assertion. Do you think that is necessary or maybe we can just render the line of the failing test as a new line? Just wondering if it could be confusing if users can add their own labeling and now we are adding this labeling? Also happy to address the rendering in a follow up PR.

@adamgfraser
I'm not crazy about the way it is rendered either. Adding a label to the assert was just the easiest way to do this.
Ideally I think the label (expression + source location) should go into AssertionValue as a new field, not on the Assertion that it contains.
And yes, I would prefer to address this as a separate PR.

adamgfraser · 2020-11-27T20:07:30Z

@dkarlinsky Sounds good to me!

…macro (zio#4427) * zio-test|assert captures expression and source location using macro * prepend `_root_` in macros; fix; fmt * rename `zio.test.assertRuntime` to `assertImpl` and made it package private * DefaultTestReporterSpec passes for 2.x and dotty * fixing line numbers after fix and fmt * fixed ZTestFrameworkSpec; ignoring line numbers in test output - otherwise the tests get too brittle. * fixed SummaryBuilderSpec * fixed MavenJunitSpec * fix + fmt * fixing fatal "unused import" in 2.13 * dropping lihaoyi sourcecode dependency in favor of small local macro * fixing ZTestFrameworkSpec for scala 3 * fix linesIterator deprecation in 2.11

dkarlinsky changed the title ~~zio-test: make assert captures expression and source location using macro~~ zio-test: make assert capture expression and source location using macro Nov 21, 2020

dkarlinsky added 7 commits November 26, 2020 11:30

zio-test|assert captures expression and source location using macro

a421a01

prepend _root_ in macros; fix; fmt

06c5c88

rename zio.test.assertRuntime to assertImpl and made it package p…

96501be

…rivate

DefaultTestReporterSpec passes for 2.x and dotty

ba13309

fixing line numbers after fix and fmt

03cee20

fixed ZTestFrameworkSpec;

8d8d101

ignoring line numbers in test output - otherwise the tests get too brittle.

fixed SummaryBuilderSpec

051cd77

dkarlinsky force-pushed the zio-test-assert-macro branch from baebdd3 to 051cd77 Compare November 26, 2020 09:43

dkarlinsky marked this pull request as ready for review November 26, 2020 09:45

dkarlinsky requested review from mijicd and softinio as code owners November 26, 2020 09:45

dkarlinsky added 2 commits November 26, 2020 13:38

fixed MavenJunitSpec

66076e4

fix + fmt

99ff836

hmemcpy mentioned this pull request Nov 26, 2020

ZIO Test: include source location traces for all test executions #3152

Closed

dkarlinsky added 4 commits November 26, 2020 15:22

fixing fatal "unused import" in 2.13

f280cda

dropping lihaoyi sourcecode dependency in favor of small local macro

d478294

fixing ZTestFrameworkSpec for scala 3

87ec114

fix linesIterator deprecation in 2.11

55cd2f6

adamgfraser approved these changes Nov 27, 2020

View reviewed changes

adamgfraser merged commit 94b896f into zio:master Nov 27, 2020

dkarlinsky mentioned this pull request Jan 5, 2021

zio-test: improvements for assert capturing expression and location #4533

Merged

dkarlinsky deleted the zio-test-assert-macro branch January 6, 2021 11:46

Uh oh!

zio-test: make assert capture expression and source location using macro #4427

zio-test: make assert capture expression and source location using macro #4427

Uh oh!

Conversation

dkarlinsky commented Nov 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamgfraser commented Nov 21, 2020

Uh oh!

dkarlinsky commented Nov 21, 2020

Uh oh!

adamgfraser commented Nov 24, 2020

Uh oh!

hmemcpy commented Nov 24, 2020

Uh oh!

dkarlinsky commented Nov 24, 2020

Uh oh!

hmemcpy commented Nov 24, 2020

Uh oh!

adamgfraser commented Nov 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hmemcpy commented Nov 24, 2020

Uh oh!

dkarlinsky commented Nov 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hmemcpy commented Nov 24, 2020

Uh oh!

dkarlinsky commented Nov 24, 2020

Uh oh!

hmemcpy commented Nov 26, 2020

Uh oh!

dkarlinsky commented Nov 26, 2020

Uh oh!

hmemcpy commented Nov 27, 2020

Uh oh!

adamgfraser left a comment

Choose a reason for hiding this comment

Uh oh!

dkarlinsky commented Nov 27, 2020

Uh oh!

adamgfraser commented Nov 27, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zio-test: make `assert` capture expression and source location using macro #4427

zio-test: make `assert` capture expression and source location using macro #4427

dkarlinsky commented Nov 21, 2020 •

edited

Loading

adamgfraser commented Nov 24, 2020 •

edited

Loading

dkarlinsky commented Nov 24, 2020 •

edited

Loading