It might be good to note that headers can't be actually received in this PR, so this TODO is safe to be left for future.

I added this in the comment.

Replace Fn.const () with ignore

Replaced

Note on producer_transition_writer_ref: type of this parameter is changed from _ Strict_pipe.Writer.t ref to _ Strict_pipe.Writer.t option ref.

This makes sense because no block production happens when bootstrap is running. This pipe was created just to substitute a value for the type and was never actually used. It could have been read only by the transition frontier controller, and it's not running when bootstrap controller is active.

I added this as a comment in the code for the future.

This code has been moved to start_bootstrap_controller to highlight that we don't really depend on best seen transition itself, but only on the knowledge of which peer shared it.

Note: big change in this module is that transition cache may hold not only blocks, but headers too.

Also validation callbacks are not discarded any more

Note: instead of just checking whether a state hash already was among children, we'll merge an existing child with the new transition.

It's needed to maintain consistency about transitions that may come from gossip or other ways (not sure if there could be some, but interfaces seem to allow that from the first glance), we want to preserve some validation callback (and we don't expect two gossips for the same).

Also if we received a block after receiving a header (definitely possible when we'll simultaneously support both old and new gossip topics), we want to preserve a block because it's more than a header.

I added this as a comment in the code for the future.

Consider removing the type containing Block constructor and just use:

[ `Block of Mina_block.t | `Header of Mina_block.Header.t ]

Then we'll no longer need dependency on the bootstrap controller here

Done in this commit

Note: code below moved here from bootstrap controller.

New function combines writing to boostrap controller writer (which was here already) and extracting preferred peers (freshly moved).

Arguably change of semantics behind transition_writer_ref is the most non-trivial change of the PR.

I strongly advise to extract this change as a separate commit.

Also, I think there is a bug here.

Original intention seemed to have been to remove the need to pass transition_reader_ref everywhere (because where it's used what's inside is always a reader created locally, hence we don't need a ref to use the correct value). And also to remove redundant pipe creation to start initialization (a pipe that gets discarded immediately). For this reason the transition_writer_ref was made an optional argument (and no argument in case of initialize function).

So the code below does it right that it uses ref transition_frontier_controller_writer as the default value. It does it wrong in not setting ref to transition_frontier_controller_writer when transition_writer_ref is not None.

So the code I think should be here is Option.value_map ~f:(fun r -> r := transition_frontier_controller_writer; r) transition_writer_ref ~default:(ref transition_frontier_controller_writer)

And same in start_bootstrap_controller.

Isn't Option.value_map ~f:(fun r -> r := transition_frontier_controller_writer; r) transition_writer_ref ~default:(ref transition_frontier_controller_writer) the same as ref transition_frontier_controller_writer ?

I think I’m missing something because to me this makes the ?transition_writer_ref argument given to the start_transition_frontier_controller function useless, as it is just replaced with transition_frontier_controller_writer

A very good question!

When ?transition_writer_ref:None is passed, following codes are identical by definition:

(* code 1 *)
Option.value_map ~f:(fun r -> r := transition_frontier_controller_writer; r) transition_writer_ref ~default:(ref transition_frontier_controller_writer)

(* code 2 *)
ref transition_frontier_controller_writer

However when ?transition_writer_ref:(Some myref) is passed, behavior changes. Code that calls start_transition_frontier_controller may use myref in some other thread. In particular we use it the following way:

don't_wait_for
@@ Strict_pipe.Reader.iter_without_pushback valid_transition_reader2
  ~f:(fun (`Block enveloped_transition, `Valid_cb vc) ->
   don't_wait_for @@
   let%map () = (* some code removed for clarity *) in
   Strict_pipe.Writer.write !transition_writer_ref
      (`Block enveloped_transition, `Valid_cb (Some vc)) ) )

We have an infinite loop of start_transition_frontier_controller transitioning to start_bootstrap_controller and then back, depending on state of sync. However the async thread above is started once with some ref. It's important that we write to that same variable instead of creating a new one as the code let transition_writer_ref = ref transition_frontier_controller_writer would do.

Thanks a lot for this explaination, it makes more sense now.

I made a separate commit including your new version : 0cbd947

So, this line removal wasn't part of original fbc8bf0

So, this line removal wasn't part of original fbc8bf0

I think I removed it because I was confused that the new code was redundant :

  let transition_writer_ref =
    Option.value transition_writer_ref
      ~default:(ref bootstrap_controller_writer)
  in
  transition_writer_ref := bootstrap_controller_writer ;

Was it a mistake ?

Nevermind, I had not read the previous comment about transition_writer_ref.

If I understood correctly, whatever the value in transition_writer_ref is when given as an argument, we want to replace it with ref transition_frontier_controller_writer. Is that correct ?

I'll reply to https://github.com/MinaProtocol/mina/pull/16145/files#r1781433238, there you formulated your concern excellently!