Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add detection of text missing values in validation_report() #242

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 12 commits into from
May 28, 2025
Merged

Add detection of text missing values in validation_report() #242

merged 12 commits into from
May 28, 2025

Conversation

Copy link
Contributor

Copilot AI commented May 28, 2025 •
edited by martinctc
Loading

This PR enhances the validation_report() function to detect and report on text strings that likely represent missing values but aren't actual NA values in R.

This PR also bumps versions to v1.9.2 and is submitted to CRAN.

Changes:

  1. Added new na_values parameter to hrvar_count_all() function with default value c("NA", "N/A", "#N/A", " ")
  2. Updated the missing value calculations to identify both:
    • Actual NA values (using is.na())
    • Text strings that represent missing values (like "NA", "N/A", "#N/A", spaces)
  3. Added reporting of these potential missing values in the output message with a count and the specific values found
  4. Added the na_values parameter to validation_report() so it can be customized when calling the report function
  5. Added a test case to verify the text missing values detection works correctly

Example:

When data contains text strings like "NA" or "N/A" in HR attributes, the validation report will now include a message like:

There are 15 values which may potentially represent missing values: NA, N/A, #N/A.

These values are now also properly counted in the missing values percentage calculations.

Fixes #35.

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Feature: check for spaces and "NA" in validation_report() Add detection of text missing values in validation_report() May 28, 2025
Copilot AI requested a review from martinctc May 28, 2025 08:50
Copilot finished work on behalf of martinctc May 28, 2025 08:50
@martinctc martinctc marked this pull request as ready for review May 28, 2025 10:32
@martinctc
Copy link
Member

@copilot - tried submitting this package to CRAN, but returned the following issues in the incoming checks:

  Found the following (possibly) invalid URLs:
    URL: https://docs.microsoft.com/en-us/learn/modules/workplace-analytics-r-package/ (moved to https://learn.microsoft.com/en-us/learn/modules/workplace-analytics-r-package/)
      From: README.md
      Status: 301
      Message: Moved Permanently
    URL: https://docs.microsoft.com/en-us/workplace-analytics/ (moved to https://learn.microsoft.com/en-us/workplace-analytics/)
      From: README.md
      Status: 301
      Message: Moved Permanently
    URL: https://stackoverflow.com/questions/34522732/changing-fonts-in-ggplot2
      From: man/theme_wpa.Rd
      Status: 403
      Message: Forbidden
  For content that is 'Moved Permanently', please change http to https,
  add trailing slashes, or replace the old by the new URL.

Please remove the offending links from the above so that we can pass the checks.

Copy link
Contributor Author

Copilot AI commented May 28, 2025 •
edited
Loading

@copilot - tried submitting this package to CRAN, but returned the following issues in the incoming checks:

  Found the following (possibly) invalid URLs:

...

I've fixed the invalid URLs flagged by the CRAN checks in commit d0b620b:

  1. Updated URLs in README.md from docs.microsoft.com to learn.microsoft.com
  2. Removed the Stack Overflow URL from man/theme_wpa.Rd that was returning a 403 Forbidden status

These changes should resolve the reported issues with the invalid URLs.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

  • CRAN.R-project.org
    • Triggering command: /usr/lib/R/bin/exec/R --no-restore --no-echo --args nextArg--as-crannextArgwpa_1.9.2.tar.gz (dns block)
  • bioconductor.org
    • Triggering command: /usr/lib/R/bin/exec/R --no-restore --no-echo --args nextArg--as-crannextArgwpa_1.9.2.tar.gz (dns block)
  • cloud.r-project.org
    • Triggering command: /usr/lib/R/bin/exec/R --no-restore --no-echo --args nextArg--as-crannextArgwpa_1.9.2.tar.gz (dns block)
  • esm.ubuntu.com
    • Triggering command: /usr/lib/apt/methods/https (dns block)
  • igraph.org
    • Triggering command: /usr/lib/R/bin/exec/R --no-restore --no-echo --args nextArg--as-crannextArgwpa_1.9.2.tar.gz (dns block)
  • lifecycle.r-lib.org
    • Triggering command: /usr/lib/R/bin/exec/R --no-restore --no-echo --args nextArg--as-crannextArgwpa_1.9.2.tar.gz (dns block)
  • martinctc.github.io
    • Triggering command: /usr/lib/R/bin/exec/R --no-restore --no-echo --args nextArg--as-crannextArgwpa_1.9.2.tar.gz (dns block)
  • microsoft.github.io
    • Triggering command: /usr/lib/R/bin/exec/R --no-restore --no-echo --args nextArg--as-crannextArgwpa_1.9.2.tar.gz (dns block)
  • rdrr.io
    • Triggering command: /usr/lib/R/bin/exec/R --no-restore --no-echo --args nextArg--as-crannextArgwpa_1.9.2.tar.gz (dns block)
  • www.fstpackage.org
    • Triggering command: /usr/lib/R/bin/exec/R --no-restore --no-echo --args nextArg--as-crannextArgwpa_1.9.2.tar.gz (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of martinctc May 28, 2025 11:07
@martinctc
Copy link
Member

package wpa_1.9.2.tar.gz is on its way to CRAN - merging now

@martinctc martinctc merged commit be75742 into main May 28, 2025
3 checks passed
@martinctc martinctc deleted the copilot/fix-35 branch May 28, 2025 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@martinctc martinctc Awaiting requested review from martinctc

Assignees

@martinctc martinctc

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Feature: check for spaces and "NA" in validation_report()

2 participants

@martinctc