Fixed path downlinks using multiple group_index via gRPC schedule multiple times to the same downlink path #7919

Summary

When scheduling a downlink using a fixed downlink path via the gRPC API, providing an array of $N$ gateways with distinct group_index values results in incorrect routing. Instead of distributing the downlink across the groups and their specified gateways, the message is scheduled $N$ times to the exact same gateway/group. The other gateways defined in the fixed path array are discarded/ignored.

Steps to Reproduce

1. Initiate downlink request via the gRPC API.
2. Specify a fixed downlink path that includes an array of multiple gateways.
3. Assign a distinct group_index to each gateway in the array.

Example Fixed Path Payload:

"gateways": [
  {
    "gateway_ids": {
      "gateway_id": "gateway-alpha-obfuscated"
    },
    "group_index": 1
  },
  {
    "gateway_ids": {
      "gateway_id": "gateway-beta-obfuscated"
    },
    "group_index": 2
  },
  {
    "gateway_ids": {
      "gateway_id": "gateway-gamma-obfuscated"
    },
    "group_index": 3
  }
]

4. Send the downlink request.
5. Observe the routing logs or gateway traffic.

Current Result

The network server discards the group_index distinct routing paths. It correctly schedules $N$ separate messages, but all $N$ are sent to only one single of the gateways in the array (e.g., gateway-alpha-obfuscated receives all messages). The other groups in the array are entirely ignored.

Expected Result

The network server should schedule exactly one downlink message per distinct group_index. Picking one of the gateways specified in the fixed path array, honoring the assigned group_index fand availabilty or each distinct gateway (i.e., one message to gateway-alpha-obfuscated, one to gateway-beta-obfuscated, and one to gateway-gamma-obfuscated).

Relevant Logs

URL

No response

Deployment

The Things Stack Cloud

The Things Stack Version

3.36.0

Client Name and Version

**Interface:** gRPC API

Other Information

No response

Proposed Fix

i suppose the problem is located in https://github.com/TheThingsNetwork/lorawan-stack/blob/v3.36/pkg/networkserver/downlink.go scheduleDownlinkByPaths. Lines 1043 to 1048.

ctx := events.ContextWithCorrelationID(ctx, fmt.Sprintf("ns:transmission:%s", events.NewCorrelationID()))
		for _, a := range attempts {
			req.TxRequest.DownlinkPaths = a.paths
			down := &ttnpb.DownlinkMessage{
				RawPayload: req.RawPayload,
				Payload:    req.Payload,
				Settings: &ttnpb.DownlinkMessage_Request{
					Request: req.TxRequest,
				},
				CorrelationIds: events.CorrelationIDsFromContext(ctx),
			}
			queuedEvents = append(queuedEvents, attemptEvent.New(ctx, eventIDOpt, events.WithData(down)))

As req seems to be a shared pointer and each schedule is not receiving its distinct copy, I assume any downlink not yet processed is receiving an updated downlink path by the next iteration of the loop. while it remains in the queue. A solution could be to create a copy of req.TxRequest using proto.clone(), and changing the copies downlinkpath.

Like this:

 txReq := proto.Clone(req.TxRequest).(*ttnpb.TxRequest)
    txReq.DownlinkPaths = a.paths

    down := &ttnpb.DownlinkMessage{
        RawPayload: req.RawPayload,
        Payload:    req.Payload,
        Settings: &ttnpb.DownlinkMessage_Request{
            Request: txReq,
        },
        CorrelationIds: events.CorrelationIDsFromContext(ctx),
    }

Contributing

• I can help by doing more research.
• I can help by implementing a fix after the proposal above is approved.
• I can help by testing the fix before it's released.

Validation

• The fix is tested in a staging environment.
• The fix is documented in The Things Stack Documentation

Code of Conduct

• I agree to follow TTN's Community Code of Conduct.