Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@hillarymarler
Copy link
Collaborator

This pull requests fixes bug with USGS speciation occurring as part of units in TADA_ComparableDataIdentifier. Units of "NA" may still appear in data frame if TADA_SimpleCensoredMethods has not yet been run. Once TADA_SimpleCensoredMethods is run, a new TADA.ComparableDataIdentifier is created which includes units for any result values that have been estimated from detection limits.

added missing TADA_CreateComparableID
Created Add_USGS() internal function to make sure all possible TADA.ResultMeasure.MeasureUnitCodes are included without user having to enter all the possible USGS combinations
@hillarymarler
Copy link
Collaborator Author

I need to update branch refs in vignettes in order to pass checks. I will do that on Monday,

@hillarymarler
Copy link
Collaborator Author

@wokenny13 and @cefergus - this is now passing checks and ready for review.

@wokenny13
Copy link
Collaborator

Using the example with loading the example dataset

data(Data_Nutrients_UT)

does the ExampleData.R re-run the example datasets with any of the changes that are made with applyautoclean done to TADA_SimpleCensoredMethods and TADA_ConvertResultUnit?

After running

`Data_Nutrients_UT2 <- TADA_SimpleCensoredMethods(Data_Nutrients_UT)
TADA_FieldValuesTable(Data_Nutrients_UT2, field = 'TADA.ComparableDataIdentifier')'

I get two fields that would still have NA for the units, is this expected as a potential output?:
image

Or is it required to re-run the example data pull to reflect the changes?

@hillarymarler
Copy link
Collaborator Author

hillarymarler commented Jul 22, 2024

Using the example with loading the example dataset

data(Data_Nutrients_UT)

does the ExampleData.R re-run the example datasets with any of the changes that are made with applyautoclean done to TADA_SimpleCensoredMethods and TADA_ConvertResultUnit?

After running

`Data_Nutrients_UT2 <- TADA_SimpleCensoredMethods(Data_Nutrients_UT) TADA_FieldValuesTable(Data_Nutrients_UT2, field = 'TADA.ComparableDataIdentifier')'

I get two fields that would still have NA for the units, is this expected as a potential output?: image

Or is it required to re-run the example data pull to reflect the changes?

@wokenny13 - the missing units are related to results where there is no unit (or value) provided for the detection quantitation limit. So even when TADA_SimpleCensoredMethods is run, there is no unit/value to pull. It is strange to see - maybe something we should bring up with WQP team.

Working off the example you provided, see:
subset_na_unitcode <- Data_Nutrients_UT2 %>% dplyr::filter(is.na(TADA.ResultMeasure.MeasureUnitCode))

@wokenny13
Copy link
Collaborator

I figured something along the lines of that was the case, but just wanted to confirm. Do these results get flagged at all? It represents a small portion of the dataset, but might be good to bring up to the WQP team.

@wokenny13
Copy link
Collaborator

I ran a test dataset, but I seem to still be seeing speciation name appearing at the end where the units should be.

datatest <- TADA_DataRetrieval(statecode = 'FL', startDate = '2024-01-01', endDate = '2024-06-30', characteristicName = c('Ammonia','Nitrate','Nitrogen'))
unique(datatest$TADA.ComparableDataIdentifier)
datatest2 <- TADA_SimpleCensoredMethods(datatest)
datatest3 <- TADA_ConvertResultUnits(datatest)
unique(datatest2$TADA.ComparableDataIdentifier)
unique(datatest3$TADA.ComparableDataIdentifier)

@hillarymarler
Copy link
Collaborator Author

Thanks for catching this - I had made some minor changes related to issues happening with NAs in an example data set. Now if there are NOT NAs in certain columns there are issues it seems. I'll work on an update to fix this new bug.

@hillarymarler
Copy link
Collaborator Author

I made some major changes to TADA_CreateUnitRef and TADA_ConvertResultUnits. I believe both are working as expected now and as a bonus, the code for those two functions is now much more concise.

I re-ran your FL example and this was the result for unique TADA.ComparableDataIdentifier:
image

@wokenny13
Copy link
Collaborator

I made some major changes to TADA_CreateUnitRef and TADA_ConvertResultUnits. I believe both are working as expected now and as a bonus, the code for those two functions is now much more concise.

I re-ran your FL example and this was the result for unique TADA.ComparableDataIdentifier: image

This is good to hear! I will take a look and review tomorrow

@hillarymarler
Copy link
Collaborator Author

@wokenny13 - looks like I've got a couple of check errors to work through first, so you can wait until I've got those resolved.

Addressed test failure
@hillarymarler
Copy link
Collaborator Author

@wokenny13 - check errors have been resolved.

Copy link
Collaborator

@wokenny13 wokenny13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran through some sample datasets. They seem to be functioning correctly. I am good with merging the changes in.

@hillarymarler hillarymarler merged commit e9c0df4 into develop Jul 26, 2024
@hillarymarler hillarymarler deleted the 502-tadacomparabledataidentifier-column-showing-na-for-the-tadaresultmeasuremeasureunitcode-portion-of-its-string-pasting branch July 29, 2024 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TADA.ComparableDataIdentifier column showing NA for the TADA.ResultMeasure.MeasureUnitCode portion of its string pasting

3 participants