It is not clear to me what is the internal structure of the pixel decoder.

It is clear that it uses the Multi-Scale Deformable Attention and an encoder-only transformer, but what are the query points and features? Is some upsampling performed? How many transformer layers are used and what are their characteristics in terms of input and outputs?