Do we need to use the offset descriptor even for STATE_PREVIOUS_MINIBATCH variable ? Isn't its value constant at all time steps.

If the lstm delay value is larger than -1, e.g. -3, then we need to add 3 previous states at t=-3, -2, -1 for frames at t=0, 1, 2 respectively, in which case we need the offset descriptor with value -3

I don't think it is possible to add the previous state from multiple time steps in the same variable in the current framework ? Did you try using this network with larger offsets and check it works. I think we need to have separate variables to store each time offset from the previous minibatch.

In my implementation, I add to each example an additional input io with multiple time steps of the same variable (Ln 108-130 in nnet3-add-recurrent-io-to-egs.cc), just the same way as ordinary input. I think as long as the Index of those additional input io is correct (in terms of computability of all the component nodes after building the computation graph).

nice that you managed to get this done with minimal variables.