Added tests, which revealed an odd case:

        ens = nengo.Ensemble(10, 1, neuron_type=nengo.LIF())
        old_neurons = ens.neurons
        ens.neuron_type = nengo.RectifiedLinear()
        assert ens.neurons == old_neurons

This assertion passes at the moment, but should we consider two neurons objects with not only different parameterizations, but completely different neuron types to be equal?

1. Appears to be a bug to me. In LearningRule.__eq__ we check if the learning rule types are equal, which should be false if they have different learning rates. So seems like a bug in LearningRuleType.__eq__
2. All that Neurons references is the ensemble. So when you change the neuron type on the ensemble, it will automatically change for every instance of ens.neurons that has ever been made.

EDIT: What you pointed out in 2 would actually happen in the case of learning_rule, because it stores references to both the connection and the learning rule type. But, since we check whether the learning rule types are equal as part of LearningRule.__eq__, then it should return false if the learning rule type has been changed.

EDIT2: I ran 1 on my machine, and I get them not equal.

Yeah 1. was a mistake, I mixed up the cases I was testing. For 2, we could change the behaviour (if we wanted) by storing the neuron_type in the Neurons object (analogously to how we store learning_rule_type in LearningRules). Just depends on what we want the semantics of the equality to be (i.e. is it checking that these two neurons objects belong to the same ensemble, or is it checking that they belong to the same ensemble and have the same type).

ens.neurons is just supposed to be a way to refer to the ensemble's neurons, so the less state it can have the better in my opinion. i.e., if I do something like:

        ens = nengo.Ensemble(10, 1, neuron_type=nengo.LIF())
        nengo.Connection(ens.neurons, ens.neurons)
        ens.neuron_type = nengo.RectifiedLinear()

I wouldn't want that connection to break because the Neurons object is storing the old neuron type. So I like the way that it's currently set up.

If I could, I would change LearningRule to act the same (i.e. just reference its connection, so if the associated learning rule type changes on the connection, it will change automatically on the LearningRule). Wouldn't be too hard; LearningRule would just have to store an index or key of which learning rule type it's associated with in the connection.

I remember us talking about removing the multiple-learning-rules-on-one-connection functionality (since it isn't really used), which would make that even easier. I'll just leave all the current behaviour as-is for now, and we can worry about it if/when the time comes.