Right now, if I'm not mistaken, we can only add one SymbolManager (and others) to a map.
The issue for our case is that we have a big numbers of annotations (> 1000) that have different type and different ways to be added/removed to the map.
Example of performance issues (I think at least) we have is we have 8000 symbols, we need to remove 1000 from this, which are removed from the main thread and is costly.
Example of issue can also appears in click because there is no way to add a tag to a symbol, so if two type of symbol have different click listeners, we need to use "hacks" to handle them.

My suggestion here is to allow multiple managers to be added (as different layers I suppose) to the map, so we can easily have one manager for a type, which greatly reduce the hassle (we can remove all symbols in one manager but not the others, handle click only etc...)
Right now (I think) it's not possible because a SymbolManager is created with a fixed layer id :
public static final String ID_GEOJSON_LAYER = "mapbox-android-symbol-layer";
and we can't change it because the constructor is package protected.