Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@Mikejmnez
Copy link
Collaborator

@Mikejmnez Mikejmnez commented Nov 18, 2024

This is a quick patchwork so that pydap CAN parse DAP4 responses from TDS. See #401

There are a couple of issues related to parsing DAP responses from a TDS. This PR solves:

a) Correctly split DMR from binary data in the response. There are very small differences in how the 2 servers split the dmr from the binary data:

Hyrax

"... </Dataset>\n\r\n<binary data>"

TDS

"... </Dataset>\r\n<binary data>"

DAP4 support within pydap has been written to accomodate Hyrax. The vary small difference between the two servers means that returning data from a TDS via DAP4 results in error.

With this PR, now there is no error but there is a "UserWarning" to inform users to use DAP2 protocol with TDS. The reason for that is this PR still does not fix another discrepancy between TDS and Hyrax servers. The discrepancy may be small, but relates to at least partially with how endianness is specified in the response.

For example, when reading from this data_URL on Hyrax you CORRECTLY get:

ds = open_url(data_url, protocol='dap4')
ds['y'][:]
>>> <BaseType with data array([   0,    1,    2, ..., 5421, 5422, 5423], dtype=int16)>

The same data_url running on TDS gets you:

ds = open_url(data_url, protocol='dap4')
ds['y'][:]
UserWarning: Full DAP4 support for TDS data servers remains under development for pydap. We recommend to continue to use DAP2 protocol instead. See: https://github.com/pydap/pydap/issues/401
>>> <BaseType with data array([     0,    256,    512, ..., -26107, -25851, -25595], dtype='>i2')

The issue is NOT JUST endianness. And so for now passing a useful UserWarning is enough, while we spend some time working with the tds folks on resolving this issue.

@Mikejmnez Mikejmnez marked this pull request as ready for review November 18, 2024 17:05
@Mikejmnez Mikejmnez merged commit bc24d8e into pydap:main Nov 19, 2024
7 checks passed
@Mikejmnez
Copy link
Collaborator Author

merging

@Mikejmnez Mikejmnez deleted the tds_patch branch December 7, 2024 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant