It may be squashed into a single commit by me, but rest assured @PPMo did all the work 😉

Can you add some information to the documentation regarding the units of the vectors. Does the function assume the units are the same in each coordinate system (e.g. the vectors are desired to be in m/s in both) or does it assume the vectors are based on the distance metric in each individual projection? I'm assuming it is the former because that is by far the most common case, but you need to say so because the two are obviously not the same thing.

Can you comment about not utilizing the existing cartopy.crs.CRS.transform_vectors method? I'm not saying you should have, but it would be good to understand what the differences between the two methods are, and if there is common ground that should be exploited. I'm sure you have good reason for making methodological choices, and perhaps some of this work could feed back into cartopy if the approach proves to be better.

Can you comment about not utilizing the existing cartopy.crs.CRS.transform_vectors method? I'm not saying you should have, but it would be good to understand what the differences between the two methods are, and if there is common ground that should be exploited. I'm sure you have good reason for making methodological choices, and perhaps some of this work could feed back into cartopy if the approach proves to be better.

I did have a version of the code that wrapped the cartopy rotate_vectors call with before-and-after distance scaling, but then I found I had all the bits in place to just do it entirely in Iris..

I came to the conclusion that the existing cartopy transform_vectors is actually "wrong", because although it preserves magnitudes it doesn't take account of unequal scaling of X and Y w.r.t. actual distances on the ground.
This emerged from testing the new code against the analytic solution for the rotated-pole case...

An example
Consider a south-west wind of 14m/sec at the zero meridian on the equator.
In a true-latlon system, this is at (0,0), and follows a track at 45 degrees from East, with equal eastwards and northward components (approx 10 m/sec).
In terms of the coordinate differentials, the X and Y components, in radians/sec, are both equal to 10/R (where R is earth radius) because the X and Y scaling (metres/radian) is equal at the equator.

Now consider the same wind in a rotated system with the pole at (say) lat=30, long=180 (pole is rotated "backwards" by 60 deg).

• The point is now at (lat'=-60, lon'=0).
• The North and East directions are the same as before.
• dY/dT = 10/R as before,
• but dX/dT = 10/Rcos(60) = 20 radians/sec.

So, the problem with the existing routine is : in this case the code will take the vector (10, 10) and transform it. However, a vector with equal X and Y steps does not follow a 45 degree path on the ground. As the calculation is the correct one for coordinate vectors, this doesn't work right (of course, a similar distance-scaling factor is also needed after the transform step).

Another way of looking at this is that the units of an actual coordinate time differential would be (e.g.) degrees/second, but winds are in metres/second. Even if the projection has metres as its units, these can't possibly match real metres everywhere.

So, what I think the existing routine is actually doing is transforming coordinate vectors, not wind-type vectors -- except that it adjusts the magnitudes to keep them the same.
After some head-scratching, @esc24 + I came to the conclusion that this probably needs fixing in cartopy, ultimately.
Arguably we could have two routines, one to transform coordinate vectors (which does not preserve magnitude), and another to do wind-type values (or "distance differential vectors").

Can you add some information to the documentation regarding the units of the vectors. Does the function assume the units are the same in each coordinate system (e.g. the vectors are desired to be in m/s in both) or does it assume the vectors are based on the distance metric in each individual projection? I'm assuming it is the former because that is by far the most common case, but you need to say so because the two are obviously not the same thing.

Your assumption is 100% correct, that is what is meant.
The problem is to state it clearly -- see how long my previous comment is !
I originally wrote more, but @esc24 was strongly for keeping the docstring simpler.

In fact, I was originally were intending to name the operation in a more general fashion, something like "transform_distance_differentials" or "project_displacement_vectors"... In the end, I was persuaded that simply calling it "rotate_winds" was the least confusing way of explaining what is meant, even though the concept is in fact more general, and more subtle.

@ajdawson if you really want more explanation, can you suggest how we might phrase it ?

+1 for a statement on the units. One of my early versions mentioned that they are not modified.

@pp-mo - I don't think I agree with your assertion that the cartopy code is wrong. However, your explanation is not clear to me either. There is plenty of room for misunderstanding here so I'll be clear about how I view the cartopy methodology:

Cartopy transform vectors assuming vectors are defined as components in projection coordinates, i.e. they are grid relative rather than north/east. It assumes these are vectors of a quantity such as wind speed. It transforms to a set of vectors that are grid relative in the target projection (i.e. u and v are in the same directions as the strictly orthogonal x and y projection coordinates). The transformed vectors are constructed so that they point in the same direction (north/east) in the target projection as in the source projection, and the magnitude of the vector in terms of the wind speed will be conserved. I'm not really sure how this doesn't fit the bill, are you saying the vector directions are incorrect with cartopy? At least visually I can't see any issue, but perhaps it is more subtle? For example:

I'd appreciate your feedback on this. In particular it'd be great to see a reproduction of this example using your methodology (u=v=10; x=0,30,60,90; y=0,45; all expressed in Plate Carree).

Hi @ajdawson. When I began this work I convinced myself that the cartopy code fitted the bill perfectly. The code is clear and the logic is well explained. After a lot of thinking I've become convinced that it doesn't quite do what's needed in this case. Whether the code is cartopy needs changing is another question that my brain couldn't take on at the time. I now think it does, but I was hoping we could get this functionality into Iris and then have the debate on cartopy. My thinking as to why cartopy.crs.CRS.transform_vectors() is not appropriate here concerns the code on the following lines: https://github.com/SciTools/cartopy/blob/master/lib/cartopy/_crs.pyx#L401-L409

This is calculating an angle in terms of u and v (in m/s) at a location where u and v are components in the directions defined by the grid directions in the source coordinate system at the location in question. In my mind this angle points in the physical direction that the wind is pointing in the real world. The issue is that a perturbation in grid coordinates (e.g. deg lat and deg lon) is then calculated using this angle in a way that assumes the scaling of the x and y axes is the same i.e. delta degrees lat and delta degrees lon is the same distance on the ground. This scaling varies with location (latitude), which manifests itself as a systematic error as you move away from the equator.

Another way to imagine this (bear with me, this might sound very silly 😉) is to consider a short stick lying on the ground pointing in the direction of the wind. Imagine that the physical length of this stick (say 20cm) represents the wind speed (20 m/s), and the windspeed is 20m/s everywhere. As the wind direction changes we rotate the stick to point in the right direction. As we move around the globe, we take our stick with us and provided the wind speed is 20m/s the stick is the correct length. Pick a location and for simplicity, consider a non-rotated lot-lon where x and y are measured in degrees longitude and latitude. The location is easy with my garmin GPS and the x and y directions are well defined as they happen to point east and north. Now the tricky bit. Let the wind (and the stick) point NE. Look down at the stick from above and imagine grid lines of lat and lon. The stick will be at 45 degrees to these axes and the u and v components will be equal. Using the cartopy code the vector_angles will also be 45 deg. So good so far. Now, consider lines 408 and 409. The x_perturbation and y_perturbation will also be equal as cos(45) = sin(45) = sqrt(2)/2. Imagine delta happened to be roughly the same scale as the stick, and place a pebble at the new location. The problem is a small shift of delta*sqrt(2)/2 in degrees longitude and the same in degrees latitude will be a position that doesn't lie long the direction of the stick. In the northern hemisphere the pebble will lie slightly north of stick, with the discrepancy getting greater the further north you move. This is because as we get closer to the pole a small shift in longitude gets smaller compared to the same shift in latitude.

Does any of this make sense?

Thanks @esc24. I think that's a very cogent account, and assuming I've understood it right I think this agrees with my view of it.
I'm working on a better example / explanation myself, to respond to #1546 (comment) as @ajdawson asked. But it's not ready yet ! ...

Meanwhile, though, I want to make one important point right now : I think we need to be very careful about plots.

For example, consider @ajdawson 's first plot in the above comment..

This appears to show a constant "SW" wind, i.e. U=V; or "bearing from 135 deg"; or arrows at 45deg everywhere.
It is plotted, I assume, on PlateCarree.
However, if so then the displayed arrows do not point along the line of an actual NE-ly movement from the base points -- because those lines are only 45deg on the map at the equator, and everywhere else the angle will be flatter. In fact, the only cylindrical map on which all those arrows will be at 45deg is a Mercator plot -- that's the whole point of the Mercator projection.

I'm still not sure whether I think the cartopy quiver is "correct" in this. However there's another even bigger question here, which is what a wind plot "should" look like.
It seems very reasonable to me that a SW wind arrow should always be plotted on the map with a 45deg angle, because that is how the user wants to interpret it. If so, the arrows aren't really to be plotted on the map projection at all, but only at the given points on the map. So this is really quite different from streamline arrows, which would point along the actual ground path.

@esc24 - It does make sense to me. I understand the different scaling with respect to to coordinate system. I've no problem with the functionality in Iris. The thing I'm trying to think over is whether this is actually wrong or not in cartopy, which is tangential to the discussion of this PR. I guess in a lot of plotting software you actually get to choose this behaviour (e.g. Reading IDL meteorology library's /MAP keyword).

I'm excited that this could lead to better handling of vectors in Cartopy too, the current implementation is definitely a bit rough around the edges to say the least. For instance if you want "standard" vectors from a lat/lon grid you have to define them on a PlateCarree() projection then they are treated as being in the projection coordinates, so equal u and v components will be at 45 degrees everywhere, which is correct given the definition of the problem. However, perhaps it would be better to be able to express them in a lat/lon CRS and transform to a Plate Carree projection which would then exhibit an angle change away from the equator. I'm not sure the current framework allows for this, but perhaps this new work would.

If the functionality required here and that in Cartopy are the same (solving the same problem) then it would be good to eventually do the work in Cartopy and leave a thin wrapper over it in Iris. Of course implementing it solely in Iris is a good first start.

@pp-mo - The definition of the quiver function in cartopy specifies that wind components be grid relative. In the example I chose I defined the vectors on a Plate Carree projection (not a lat/lon spherical) so in this case it is absolutely correct that they are at 45 degrees everywhere. However, users may want to transform the data in spherical lat/lon to Plate Carree and see the variation in angle away from the equator. This is currently not supported.

However there's another even bigger question here, which is what a wind plot "should" look like.
It seems very reasonable to me that a SW wind arrow should always be plotted on the map with a 45deg angle, because that is how the user wants to interpret it. If so, the arrows aren't really to be plotted on the map projection at all, but only at the given points on the map. So this is really quite different from streamline arrows, which would point along the actual ground path.

Good point, other software I've worked with often gives you the choice of how to scale, and matplotlib can do this too (with respect to the actual projection coordinates though, so won't understand the nature of the projection of the sphere). We should definitely aim to be able to plot as you describe in cartopy, especially for streamlines as you described, but it may also be desirable to retain the present functionality as many users will actually want to use it.

@ajdawson I guess in a lot of plotting software you actually get to choose this behaviour (e.g. Reading IDL meteorology library's /MAP keyword).

Thanks for this hint.
Did you mean this... ?

;+
; NAME:pp_windy
; TAG: Vector plot for pp fields
; PURPOSE: Plot a two-dimensional vector field, indicating magnitude and direction
...

; KEYWORD PARAMETERS:
...
;   /noscale -- draw arrows which indicate only direction, not magnitude.
;   /onescale,/draw_one (synonymous) -- Use the same scaling for x- and
;     y-components of the vector. Ignored if max_arrow is specified.
;   /twoscales -- Use separate scalings for the x- and y-components.
;     Ignored if max_arrow is specified.
;   /trueangle -- This is the default unless the vector components are of such
;     different size that different scaling is chosen for them. It has no
;     effect if separate scalings for the two components have been requested
;     with /twoscales or by specifying a two-element max_arrow. It causes the
;     arrows to be drawn such that equal vector components are represented by
;     equal distances on the plot. This means that, for instance, a NEward wind
;     will be drawn at 45 degrees to the vertical.
;   /mapangle -- This option is available only for lat-long fields, and has no
;     effect if separate scalings for the two components have been requested
;     with /twoscales or by specifying a two-element max_arrow. It causes the
;     arrows to be drawn such that equal vector components are represented by
;     equal lengths north and east on the surface of the earth, rather than by
;     equal distances on the plot. This means that, for instance, a NEward wind
;     will be drawn with an aspect ratio of 1 km eastward by 1 km northward, so
;     it will point in a NEward direction on the map, even if the proportions
;     of the map are distorted. By default it would point at an angle of 45
;     degrees, half-way between horizontal and vertical.  This is only the same
;     on the equator and if the axes have been drawn proportionately.
;   /axisangle -- This option has no effect if separate scalings for the two
;     components have been requested with /twoscales or by specifying a
;     two-element max_arrow. It causes the arrows to be drawn such that equal
;     vector components are represented by equal lengths in data coordinates on
;     the x- and y-axes. On a lat-long plot, this mode is less useful than
;     /mapangle, because vector components usually relate to distances on the
;     Earth's surface, rather than to degrees of latitude and longitude on the
;     axes. However, for a vector plot where the axes have the same physical
;     units, this mode may be useful.

This is actually from our own older codebase (the IDL "UKMO library").
I just hadn't thought to look there !
I think this may provide an interesting take on the issues.

@pp-mo - Not specifically, I've never used the UKMO PP library. I was referring to the guide and accompanying software Using IDL In Meteorology (pdf). Other tools like NCL have this type of feature too.

Hi @ajdawson
And thanks for your linked pdf, I can see I'm a bit behind on understanding these things.
Anyway, big thanks for your engagement on this !!

I have just made a worked example of my above confusing+wordy example.
See : http://nbviewer.ipython.org/github/pp-mo/wind_rotate_example/blob/master/rotate_winds_compare_cartopy.ipynb

I think this shows how the cartopy calculation differs, and what I thought was the "wrong" answer.
I've found it very hard to come up with an example for which the right answer is "obvious", so this was my best shot.
What do you think to this @ajdawson ?

Ok, I'm good with those changes, and I also checked I'm happy with the other tests I inherited from Ed's code before.
Good to go by my reckoning.

Lots of useful info here to think about from a cartopy perspective, but I'm happy with iris.analysis.cartography.rotate_winds. A big thank you to everyone involved.

Lots of useful info here to think about from a cartopy perspective

Not just cartopy, either, as the tested result accuracies are still rather questionable.
For background :

• there are quite significant errors visible, even for the plain ordinary rotated-pole conversion. That appears both by comparison with the "exact calculation", and in @esc24 's latest preservation-of-magnitudes test, as now used to apply accuracy masking.
• though errors can be seen to get bigger as you approach the poles of either source or destination projections, they are significant even quite a long way away
  • For example, see @esc24 's recent "test_rotated_to_osgb" example : https://github.com/SciTools/iris/blob/master/lib/iris/tests/unit/analysis/cartography/test_rotate_winds.py#L376. Here we have 0.1% magnitude errors within +/-20deg of the 'equator' of a transverse mercator projection. I find that plain surprising -- I'd expected it to be substantially better.
• I think we've now shown that fiddling with the "delta fineness parameter" doesn't actually have much effect unless it is grossly too small or too big.

So the big remaining questions are :-

• where do the errors come from -- is it PROJ4, or general accuracy issues, or what ?
• are these results actually "good enough" for users' needs ?

It occurs to me we may be able to investigate the role of PROJ4 by performing the numerical calculation, but with "exact" rotated-pole equations for the point transformations.