-
Notifications
You must be signed in to change notification settings - Fork 3
Add writers for xdr and ascii files #40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
124 commits
Select commit
Hold shift + click to select a range
d43adb7
Add xdr and ascii writers
trossi 846817e
Assert that there is no unexpected value
trossi 2331898
Include non-zero gp value in representation
trossi ad13676
Add initial Python to R converter
trossi ee7e635
Simplify string ndarray handling
trossi a17e1ad
Add converter for dict
trossi bac485e
Add converter for matrices
trossi 816a2a4
Add function for building r lists
trossi 1bdaad8
Convert tuple to list
trossi fc88b62
Use consistent function naming
trossi 7a87f0f
Convert arrays of any dimensionality
trossi e632d87
Add converter for basic types
trossi 9a539f4
Add compression
trossi a34f71c
Use functions instead of a class
trossi bbdb899
Add docstrings
trossi 19a073c
Add Python-to-R converter functions
trossi 7be83bf
Add docstring
trossi ac3fde5
Rename write_all() to write_r_data()
trossi 328fb39
Fix flake8 errors
trossi 41e3834
Update docstrings
trossi 5631e4b
Fix ruff
trossi f15c495
Add support for RDA files (in addition to RDS)
trossi 295d29c
Add initial test for writing
trossi c975802
Write RDA magic with other magic
trossi 89bd71f
Avoid temporary files in test
trossi 9453eee
Include test for ascii files
trossi 192362c
Do not include trailing .0 for doubles
trossi 98c29b0
Loop over all test files
trossi bd0550f
Mark not-implemented features xfail
trossi cb15e4c
Fix testing Windows-generated ascii files
trossi 6532e38
Fix altreps
trossi 646b32d
Fix writing NA string
trossi 09aa9dd
Include filenames in pytest output
trossi ab5915a
Add debugging output
trossi ebef7d1
Remove redundant writer functions
trossi f58532d
Enable expression type
trossi e68536f
Enable builtin type
trossi cd30d46
Fix empty (None) string
trossi b153a3d
Add test for conversion
trossi 5a9994b
Add option to change version information
trossi 70187e2
Fix rda files
trossi dea4548
Set gp flag as plain int
trossi c1f5647
Use tuple
trossi 4bace9f
Fix extra info in version 2 format
trossi 6b5247b
Skip altreps
trossi 65d1468
Generalize list builder
trossi b97d650
Use same encoding as in input
trossi 499f706
Skip ambiguous tests
trossi fe7580e
Add support for R expression and language
trossi d99a003
Add equality operator for comparing numpy arrays
trossi dc0141f
Skip more ambiguous test files
trossi 0ae79e0
Print also python data for debugging
trossi fb48601
Remove unused variable
trossi 0d8d96a
Compare string presentations too
trossi 534fa47
Allow debug printing in tests
trossi 7d15bc9
Fix ruff
trossi b82aa04
Check that integers are small enough
trossi 2bbf4eb
Fix mypy
trossi 902a1fa
Add missing parameters to docstring
trossi cd5c6e7
Use RDS by default
trossi 8aec705
Simplify loops
trossi 05c4eac
Fix typos
trossi c0a118e
Remove debug prints from tests
trossi e0d5e6d
Add test for writing files with compression
trossi 9bba69b
Use binary mode for writing ascii files
trossi 733d6ca
Fix mypy
trossi 386f907
Do not open temporary file twice
trossi fbdbbe0
Add high-level writer functions
trossi 14c7f94
Rename format to file_format
trossi e8a5fad
Use high-level interface
trossi ad5ffc9
Add test for bad RDA creation
trossi 5c36c73
Add tests for bad encodings
trossi b2b9b2b
Include testing None values
trossi 0829615
Improve error messages
trossi 5c27b04
Raise NotImplementedError on untested code
trossi df5b463
Remove unused function
trossi 4beb10c
Add test for writing too large integers
trossi 5e51d1f
Raise error instead of warning for unwritten tag
trossi 3d815f1
Exclude type checking blocks from coverage report
trossi aa06572
Update authors
trossi 76c13ab
Rename writer files to unparser
trossi a6af5fd
Rename writing to unparsing
trossi 1c1cbfb
Add unparse_data() function returning bytes
trossi c129729
Fix function names in docstring
trossi 6e9b022
Use Literal type for file format
trossi 6d504d7
Use Literal type for compression
trossi 07f4393
Use None for no compression
trossi f94996a
Use Literal type for file type
trossi 746848d
Simplify handling RDA magic
trossi d834a8a
Simplify handling RDA vs RDS difference
trossi 35c7ba8
Simplify type names
trossi 3b201d2
Add test for unparsing bad rda
trossi 86958d6
Simplify handling version integers in write
trossi 49f70a1
Use hex representation for R versions
trossi 7509ecb
Use callback protocol for type hint
trossi 6347d17
Fix docstring style
trossi 7eb953b
Fix mypy in Python 3.12
trossi b421ed0
Fix mypy
trossi 6c72967
Fix mypy
trossi 9b3093c
Remove unnecessary casting
trossi 84cff21
Add test file for empty list
trossi eb57e98
Add test file for empty named list
trossi d539483
Fix parsing file format
trossi d82995f
Fix converting empty dict
trossi 1fef633
Use file type parser function
trossi b7fdb39
Clarify variable names
trossi 08c513f
Add tests for parsing empty lists
trossi 0e001ea
Allow only non-empty data
trossi c35c0e5
Add test for failing to create empty RDA file
trossi 5bc262f
Fix style
trossi f84b4aa
Include received type in the error message
trossi 7aa1d31
Clarify the conversion of np.array([None])
trossi 38c1ff6
Simplify no-error context
trossi cf78843
Rewrite file checker with existing functions
trossi 53f6864
Fix mypy
trossi d4c961f
Clarify compression handling
trossi 4f45cba
Add test for parsing NaN and Inf
trossi 2bc73ce
Fix comparing NaN values
trossi 5ab4cd5
Fix writing NaN and Inf
trossi 565b383
Fix formatting alignment
trossi 68b4e55
Simplify packing bits
trossi 442fd3a
Fix error message
trossi 42ac9b8
Use Literal type and lower case for encodings
trossi 7cf643f
Fix integer range check in NumPy 2.0
trossi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,7 +10,8 @@ keywords = [ | |
| "dataset", | ||
| ] | ||
| authors = [ | ||
| {name = "Carlos Ramos Carreño", email = "[email protected]"}, | ||
| {name = "Carlos Ramos Carreño"}, | ||
| {name = "Tuomas Rossi"}, | ||
| ] | ||
| maintainers = [ | ||
| {name = "Carlos Ramos Carreño", email = "[email protected]"}, | ||
|
|
@@ -103,6 +104,11 @@ addopts = "--doctest-modules --doctest-glob='*.rst'" | |
| doctest_optionflags = "NORMALIZE_WHITESPACE ELLIPSIS" | ||
| norecursedirs = ".* build dist *.egg venv .svn _build docs/auto_examples examples asv_benchmarks" | ||
|
|
||
| [tool.coverage.report] | ||
| exclude_also = [ | ||
| "if TYPE_CHECKING:", | ||
| ] | ||
|
|
||
| [tool.ruff.lint] | ||
| select = [ | ||
| "ALL", | ||
|
|
@@ -144,4 +150,4 @@ max-args = 7 | |
| include = ["rdata*"] | ||
|
|
||
| [tool.setuptools.dynamic] | ||
| version = {attr = "rdata.__version__"} | ||
| version = {attr = "rdata.__version__"} | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| """Functions to perform conversion and unparsing in one step.""" | ||
| from __future__ import annotations | ||
|
|
||
| from typing import TYPE_CHECKING | ||
|
|
||
| from .conversion import build_r_data, convert_to_r_object, convert_to_r_object_for_rda | ||
| from .conversion.to_r import DEFAULT_FORMAT_VERSION | ||
| from .unparser import unparse_file | ||
|
|
||
| if TYPE_CHECKING: | ||
| import os | ||
| from typing import Any | ||
|
|
||
| from .conversion.to_r import Encoding | ||
| from .unparser import Compression, FileFormat | ||
|
|
||
|
|
||
| def write_rds( | ||
| path: os.PathLike[Any] | str, | ||
| data: Any, # noqa: ANN401 | ||
| *, | ||
| file_format: FileFormat = "xdr", | ||
| compression: Compression = "gzip", | ||
| encoding: Encoding = "utf-8", | ||
| format_version: int = DEFAULT_FORMAT_VERSION, | ||
| ) -> None: | ||
| """ | ||
| Write an RDS file. | ||
|
|
||
| This is a convenience function that wraps | ||
| :func:`rdata.conversion.convert_to_r_object`, | ||
| :func:`rdata.conversion.build_r_data`, | ||
| and :func:`rdata.unparser.unparse_file`, | ||
| as it is the common use case. | ||
|
|
||
| Args: | ||
| path: File path to be written. | ||
| data: Python data object. | ||
| file_format: File format. | ||
| compression: Compression. | ||
| encoding: Encoding to be used for strings within data. | ||
| format_version: File format version. | ||
|
|
||
| See Also: | ||
| :func:`write_rda`: Similar function that writes an RDA or RDATA file. | ||
|
|
||
| Examples: | ||
| Write a Python object to an RDS file. | ||
|
|
||
| >>> import rdata | ||
| >>> | ||
| >>> data = ["hello", 1, 2.2, 3.3+4.4j] | ||
| >>> rdata.write_rds("test.rds", data) | ||
| """ | ||
| r_object = convert_to_r_object( | ||
| data, | ||
| encoding=encoding, | ||
| ) | ||
| r_data = build_r_data( | ||
| r_object, | ||
| encoding=encoding, | ||
| format_version=format_version, | ||
| ) | ||
| unparse_file( | ||
| path, | ||
| r_data, | ||
| file_type="rds", | ||
| file_format=file_format, | ||
| compression=compression, | ||
| ) | ||
|
|
||
|
|
||
| def write_rda( | ||
| path: os.PathLike[Any] | str, | ||
| data: dict[str, Any], | ||
| *, | ||
| file_format: FileFormat = "xdr", | ||
| compression: Compression = "gzip", | ||
| encoding: Encoding = "utf-8", | ||
| format_version: int = DEFAULT_FORMAT_VERSION, | ||
| ) -> None: | ||
| """ | ||
| Write an RDA or RDATA file. | ||
|
|
||
| This is a convenience function that wraps | ||
| :func:`rdata.conversion.convert_to_r_object_for_rda`, | ||
| :func:`rdata.conversion.build_r_data`, | ||
| and :func:`rdata.unparser.unparse_file`, | ||
| as it is the common use case. | ||
|
|
||
| Args: | ||
| path: File path to be written. | ||
| data: Python dictionary with data and variable names. | ||
| file_format: File format. | ||
| compression: Compression. | ||
| encoding: Encoding to be used for strings within data. | ||
| format_version: File format version. | ||
|
|
||
| See Also: | ||
| :func:`write_rds`: Similar function that writes an RDS file. | ||
|
|
||
| Examples: | ||
| Write a Python dictionary to an RDA file. | ||
|
|
||
| >>> import rdata | ||
| >>> | ||
| >>> data = {"name": "hello", "values": [1, 2.2, 3.3+4.4j]} | ||
| >>> rdata.write_rda("test.rda", data) | ||
| """ | ||
| r_object = convert_to_r_object_for_rda( | ||
| data, | ||
| encoding=encoding, | ||
| ) | ||
| r_data = build_r_data( | ||
| r_object, | ||
| encoding=encoding, | ||
| format_version=format_version, | ||
| ) | ||
| unparse_file( | ||
| path, | ||
| r_data, | ||
| file_type="rda", | ||
| file_format=file_format, | ||
| compression=compression, | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.