Pybrain3 compatability fixes for Python 3.9-based environments - #256

update for working with python3.9 environment

update for working with python3.9 environment

SciPy 1.12 removed the deprecated numpy aliases from the scipy namespace
(scipy/scipy#19067), so `from scipy import array, zeros, ...` fails outright
on any current SciPy. It was used on 165 lines across 144 files, plus the
`scipy.array(...)` attribute form in 9 more.

Mechanical rewrite, plus the cases that are not a straight rename:

  - rand/randn came from numpy.random, not numpy, and now import from there
  - np.product and np.mat were removed in numpy 2.0 -> prod / asmatrix,
    call sites renamed accordingly
  - the three `from scipy import *` wildcards resolved to explicit imports
    (all three only needed `array`)
  - dropped MultiDimHash._findLocalBall_inline: it ran through scipy.weave,
    gone since SciPy 1.0, and was already dead code because the class binds
    _findLocalBall to the _noinline variant

Genuine scipy imports (scipy.linalg, scipy.stats, ...) are untouched.

Each of these is a hard removal upstream, not a deprecation:

  - numpy.Infinity (removed in numpy 2.0) -> numpy.inf
  - scipy.linalg.pinv2 (removed in SciPy 1.9) -> pinv, which has used the
    same SVD-based algorithm since 1.7 -- that was the reason for merging it
  - matplotlib.mlab.rk4 (removed in matplotlib 3.1) -> pybrain.utilities.rk4,
    a verbatim copy of the original. Deliberately not scipy.integrate: the
    cartpole environments depend on this exact integrator
  - PIL: `import Image` -> `from PIL import Image`, and Image.fromstring ->
    Image.frombytes (removed in Pillow 3.0)
  - imp (removed in Python 3.12) -> os.path.dirname(pybrain.__file__)
  - the Python 2 `exceptions` module -> the OSError builtin

Also fixes three Python 2 integer divisions that are now type errors, in
index, shape and zeros() position -- found by walking the AST for divisions
reaching a context that requires an int. The float-index cast in
SequentialDataSet.getField, previously uncommitted in the working tree,
belongs to the same class and is included here.

The matplotlib guard in cartpole/__init__.py now tests pylab, the dependency
the package actually imports, rather than rk4.

competitivecoevolution and multipopulationcoevolution still imported
Coevolution from pybrain.optimization.coevolution, a location that no longer
exists -- the package sits under pybrain.optimization.populationbased. This
predates the numpy/scipy work and fails on the old stack too; it surfaced
because the modernised imports let the walk reach these modules.

The suite is doctest-driven (38 modules, ~640 examples) and had no working
runner: pybrain/tests/runtests.py references IGNORE_EXCEPTION_DETAIL without
importing it, and passes optionflags to TestSuite.addTest, which takes no such
argument. Neither bug surfaced because nothing invoked it.

pytest.ini collects the same set runtests.py intends -- test_*.py only, so the
deliberately disabled _test_*.py stay out -- and sets the doctest flags the old
runner meant to set. IGNORE_EXCEPTION_DETAIL is what makes the traceback
doctests pass: they expect a bare `NetworkConstructionException:` where Python
3 prints the fully qualified name.

conftest.py restores numpy's pre-2.0 repr for the duration of the suite. The
doctests were written when numpy printed `6.0`, `True` and `array([ 0.,  3.])`;
numpy 2 prints `np.float64(6.0)`, `np.True_` and `array([0., 3.])`. Measured
against the suite, legacy="1.13" passes 32 tests and "1.25" passes 25, so
"1.13" it is -- and it beats rewriting ~75 expected values by hand.

Four defects the doctests reach as soon as the suite runs again:

  - permuteToBlocks2d computed its block coordinates with `/`, integer
    division under Python 2. The results feed the index j, so every call
    raised IndexError. These divisions reach the subscript through local
    variables, which is why an AST scan for divisions in index position
    does not find them.

  - triu2flat sized its output with `zeros(dim * (dim + 1) / 2)`, a float.

  - fListToString formatted each element with "%g". Callers pass column
    vectors such as an LSTM state, and numpy 2 no longer converts a
    one-element array to a scalar for that. Raveling keeps the output
    identical for the flat lists the function already handled.

  - dictCombinations built its result with dict(d, **{k: v}) although the
    keys need not be strings, which ** requires in Python 3, and ordered
    values with sorted(), which Python 3 refuses for mixed types. The new
    _sortedMixed helper only falls back to a type-name key when a plain
    sort raises, so numeric lists keep their numeric order -- sorting
    everything by str() would reorder [2, 10] to [10, 2].

These are defects in the tests, not the library:

  - cStringIO -> io.BytesIO in test_utilities and test_datasets_datasets.
    Bytes, not text: saveToFileLike pickles.
  - `>>> s.seek(0)` now returns the offset and needs its result bound,
    otherwise doctest reports the 0 as unexpected output.
  - test_utilities_reachable passed a map() object to reachable(), which
    calls len() on it. In Python 2 map returned a list; a comprehension
    restores that intent.
  - test_utilities_dictionaries sorted dicts with mixed int/str keys and
    now uses the same _sortedMixed helper as the library.
  - test_datasets_datasets mixed tabs and spaces inside the docstring, so
    doctest refused to parse the module at all -- it has been uncollectable
    for years. Also expected the Python 2 `<type 'numpy.ndarray'>`.

numpy 2 removed it, and PyBrain passes states, actions and rewards around as
one-element arrays -- Task.denormalize even returns a one-element list. The
result is TypeError or an inhomogeneous-array ValueError deep inside training
loops, none of which an import check would reveal. Found by driving the
library rather than importing it.

Adds pybrain.utilities.asScalar, routed through numpy.asarray so array, list,
tuple and scalar are all handled and anything with more than one element
raises, and applies it where the values are consumed as scalars: Q and SARSA
(state, action, reward), ActionValue tables, and the cart-pole force term,
where a one-element action made two of the four derivatives arrays while the
other two stayed scalars.

ClassificationDataSet._convertToOneOfMany also asked for dtype='Int32'. The
capitalised dtype aliases are gone in numpy 2; the lower-case spelling is the
same type.

NetworkWriter serialised parameters with str(list(params)). list() over an
array keeps numpy scalars, which numpy 2 reprs as "np.float64(1.5)", and
NetworkReader evaluates that text -- so every network written under numpy 2
failed to load again with NameError. tolist() yields plain Python floats and
the same output on every numpy version.

The reader also gained a numpy import so files already written in the broken
format still load.

Five NameError/TypeError sites that predate this work -- none of them are
reachable at import time, which is why they survived the 2to3 pass. Found by
walking the AST for names that are read but never bound, and by exercising the
code paths.

  - DataSet.reconstruct called the Python 2 `file` builtin
  - functions.ranking used sorted(cmp=...) and the cmp builtin, both removed
    in Python 3; sorting by the second element gives the same ordering
  - ClassificationDataSetPlot used `ceil` without importing it, and passed a
    0-based index to plt.subplot, which numbers from 1
  - GA.initPopulation referenced `rd` with no such import; the call is
    rd.random(n), so it means numpy.random
  - VanillaGradientEvolutionStrategy sized its distribution parameters with
    xdim*(xdim+1)/2, a float, which zeros() rejects

Also fixes the two bugs in the old test runner: testsuites.py used
IGNORE_EXCEPTION_DETAIL without importing it and passed optionflags to
TestSuite.addTest, which takes no such argument -- they belong to
DocTestSuite.

rlgluebridge imported the RL-Glue codec at module level, so it failed with a
bare ModuleNotFoundError. It now raises the same kind of explanatory
ImportError the ode and libsvm modules already used, naming why the dependency
is unavailable rather than what is missing.

conftest honours the _dependencies convention that pybrain/tests/runtests.py
has always looked for, so a test module declaring _dependencies = ['rlglue']
is skipped with a reason instead of failing.

The legacy printing option is now set unconditionally: numpy 1.14 already
changed the array formatting, so guarding on the major version left eight
doctests red on numpy 1.x. Every numpy since 1.14 accepts "1.13".

The shebang still said python2.5, and install_requires listed only scipy
although numpy is imported directly throughout. Adds python_requires and
extras for the optional plotting, environment and test dependencies.

No move to pyproject.toml -- setup.py installs fine with current setuptools
and that is a separate decision.

Both predate the fork -- they are already missing at dcdf32b, the last
upstream commit:

  - autodoc_hack.prepare_docstring calls textwrap.dedent without importing
    textwrap. The original even carries an #@undefinedvariable marker.
  - the bicycle example uses np.abs and np.arctan but only imports individual
    names from numpy.

Found by walking the AST for names that are read but never bound.

The sources are verified against several stacks (3.10/numpy 1.26, 3.13 and
3.14/numpy 2.x, and a replica of what Ubuntu 26.04 ships), each in its own
venv. A glob keeps them all out of the tree.

Restores the intent of the scipy.weave routine removed earlier in this branch.
That routine existed because scanning the grid balls one at a time in Python
is slow; weave has been gone since SciPy 1.0, so the class fell back to the
plain loop it had been bound to by hand.

Testing all balls in one einsum is both faster and needs no compiler:

    dim=2,  160 balls, 200 queries:  21.2 ms -> 3.6 ms   (5.9x)
    dim=3, 2463 balls, 200 queries: 279.7 ms -> 19.6 ms  (14.3x)

The advantage grows with the number of grid balls, which grows steeply with
the dimension. Verified identical against the previous loop over 9000 points
including the no-match case, where argmax on an all-False array returns 0 and
the guard turns it into None.

First step of moving the physics environments off ODE. They depend on two dead
packages, not one: py-ode (last released 2012) and xode, the parser of their
scene format, which is not on PyPI at all any more. Both are replaced by
translating the scene files, since XODE and MuJoCo's MJCF are both declarative
XML over bodies, joints and geoms.

Two things are more than renaming:

  - XODE lists bodies flat and relates them through a joint's <link1>/<link2>;
    MJCF wants a tree. _buildTree derives one, rooted at whatever the file
    affixes to the environment. A joint named in <affixToEnvironment> is
    deliberately excluded from the body graph: ODEEnvironment.loadConfig
    re-attached it to the static world, so it stops relating two bodies. Doing
    otherwise picked acrobot's fixed joint as the tree edge and dropped the
    hinge that makes it a pendulum.
  - Positions are absolute in XODE, parent-relative in MJCF.

Masses are computed here rather than left to MuJoCo's density attribute. A
body may declare a <mass_shape> that differs from its collision <geom> --
acrobot's leg1 has its box dimensions transposed between the two -- so
deriving the inertia from the geom would quietly change the physics.

Gravity keeps ODE's negative-y convention instead of rotating the scenes into
MuJoCo's negative-z world, which leaves the numbers comparable to the .xode
sources.

All 14 models convert, compile under MuJoCo 3.11 and run 200 steps with finite
state; total mass agrees with MuJoCo's own accounting to 1e-6. As an
independent check, the number of actuated joints derived from the geometry
matches the hand-written torque lists in the ODE instances exactly: 11 for
johnnie, 16 for ccrl.

Verified in a Linux container -- neither MuJoCo nor PyBullet ships an
Intel-macOS wheel, so the host cannot run the engine.

MuJoCoEnvironment keeps ODEEnvironment's pybrain-facing surface -- loadXODE,
reset, step, performAction, addSensor, addActuator, getSensors, indim/outdim --
so the instances and tasks written against it need no rework.

The implementations shrink rather than grow. Where the ODE sensors walked the
XODE tree and switched on the joint class to flatten degrees of freedom into a
vector, MuJoCo already stores that layout in qpos and qvel. Collision handling,
contact joints and the passpairs bookkeeping disappear into model compilation.

Two behaviours are carried over on purpose:

  - a slider joint reports a value through JointSensor but consumes none of
    the action vector, because the ODE actuator counted it as zero. Changing
    that would shift every action index behind it.
  - gravity stays along negative y, the convention the XODE scenes use.

mujoco is imported lazily rather than at module scope, so xodeconvert stays
usable where the engine has no wheel -- Intel macOS, for one -- and the failure
is an explanatory ImportError in the style the rest of the library uses.

Verified on the acrobot in the container: indim 1, outdim 2, torque moves the
joint in the commanded direction and back, reset restores the initial state,
and two runs from the same start produce identical trajectories.

15 bodies, 11 actuated hinges. indim matches the hand-written tourqueList
exactly, which is the same cross-check the converter passed.

Verified in the container over 2000 steps under gravity with no action: the
robot falls, lands on the floor at y=-12.7 and comes to rest at a maximum
velocity of 1.3e-4, with every state finite. All 11 hinges keep their XODE
LowStop/HiStop limits to within 0.023 rad, which is MuJoCo's soft constraint
solver rather than a conversion error. Driving all 11 joints moves all 11.

All three scenes -- glass, plate and table -- with 16 actuated joints each,
matching the hand-written tourqueList exactly.

CCRL needs more than the plain environment. It tallies contacts every step so
the task can tell whether the hand holds the object and what rests on the
table. ODE collected those while building contact joints in its _near_callback;
MuJoCo has solved the contacts by the time a step returns, so they are read
out of data.contact instead. The name matching is the original's.

That matching is why the converter now names a geom after its body when the
XODE geom is unnamed: ODEEnvironment._parseBodies did the same ("if geom
doesn't have own name, use the name of its body"), and CCRL keys on names like
plate, objectP00 and pressLeft1 that only exist on bodies in the scene files.

Verified over 2000 steps: all three compile, stay finite, register contacts,
and all 16 joints respond to actuation.

The tasks needed almost nothing: they read the environment through
getSensorByName and EpisodicTask, so only the sensor import moved.

Two corrections to the converter fell out of running the whole chain:

  - The axis's FMax no longer becomes an actuator forcerange. In ODE FMax caps
    the joint *motor* (dParamFMax), while pybrain's JointActuator drove joints
    with addTorque, which FMax never limited. Clamping it left the acrobot
    unable to swing: every policy scored the same, so nothing could be learnt.
  - Geom names are made unique. ODE tolerated duplicates -- sphere-walker,
    crawler and octacrawl name every geom "boxGeom" -- where MJCF rejects
    them. The first use keeps its spelling so CCRL's contact matching still
    works.

ode/__init__.py now points at the replacement instead of only naming the
missing dependency.

Verified in the container: all three environments build with the expected
action widths, run 1000 steps finite, reset reproducibly, and respond on every
joint -- 21 checks. End to end, an OptimizationAgent with a HillClimber over
the acrobot improves from 0.0 with an idle policy to 2473.6 across 60
episodes, with best-so-far monotone. All 14 scene files still convert and
compile.

No regression: 37 tests pass on Python 3.13, 3.14, a replica of Ubuntu 26.04
and inside the container; 374 of 379 modules import, the five remaining being
the optional dependencies that stay dead.

cartpolecompile.py was a Pyrex build script. Pyrex was abandoned around 2010
and distutils left the standard library in Python 3.12, so neither half of it
worked. Cython and setuptools replace them; the .pyx source is untouched,
since Cython still accepts the Pyrex syntax it grew out of. The old Windows
mingw32 and macOS deployment-target workarounds are gone -- setuptools handles
both now.

Compiling was only half the problem. __init__.py and cartpoleenv.py imported
the extension as a bare `import cartpolewrap`, which relied on Python 2's
implicit relative imports, so the module stayed unimportable even after a
successful build. They now import it from the package.

Missing setuptools or Cython gives a one-line message instead of a traceback:
slim Python images ship neither, and Cython is not a runtime dependency.

Worth having: on the same machine the extension runs 20000 task steps at
162,000 per second against 3,600 for the pure-Python BalanceTask -- 46x, and
150x measured at the wrapper without pybrain's overhead. That matters for
reinforcement learning, where runs are millions of steps.

Verified on both platforms: builds, imports and runs, with one pole and with
two (outdim 6). Build artefacts were already covered by .gitignore.

The old file advised `python setup.py install` and pointed at a GitHub wiki
URL that has returned HTTP 400 for years. Markdown so GitHub renders it.

It now says what this fork is and why it exists, gives the supported version
ranges, covers the MuJoCo environments that replaced the ODE ones, the compile
step for the C++ cart-pole, and names the modules that stay unavailable. Both
code examples are run and their output checked, not written from memory.

Also drops the pybrain.org URL from setup.py. The domain is no longer the
project's -- it now serves an unrelated casino site -- so the package metadata
was pointing everyone who installed it there. It points at this repository
instead, and the README warns against following old links.

Names the Junior Chair of Business Information Systems, especially AI-Based
Application Systems, Prof. Dr.-Ing. Marcus Grum at the University of Potsdam,
and what was done here, without touching the credit to the original IDSIA and
CogBotLab developers or their funding acknowledgements.

BSD-3 requires the original notice to be retained, so it stays; this adds a
second line for the work done in this fork, which began in 2022.

Appended to the 201 files touched in this branch, keeping every existing
author entry. Thirteen files that carried no __author__ at all got one.

Three files declare __author__ as a multi-line tuple and were edited by hand
so their formatting survives: auxiliary/gradientdescent.py,
optimization/populationbased/pso.py and unsupervised/trainers/rbm.py.

SVMUnit and SVMTrainer wrapped the LIBSVM Python bindings directly. Those
exposed svm_model, svm_parameter and svm_problem as module-level names; current
LIBSVM moved the parameters into an option string, so the wrapper stopped
fitting and both modules refused to import. scikit-learn provides the same
algorithms -- it builds on LIBSVM itself -- behind an interface that is
maintained.

The PyBrain-facing interface is unchanged, down to the base-2 logarithms the
old parameters used: train(log2C=5, log2g=-3) still works, as do kernel_type,
setParams, save and load. Two things differ underneath and are documented as
such: saveModel writes a pickle rather than LIBSVM's text format, since a
scikit-learn estimator carries state that format cannot hold; and forwardPass
with values='raw' returns an array of decision values where LIBSVM returned a
dictionary keyed by class pair.

GridSearch and GridSearchDOE survive as names for a trainer that always
searches. The hand-written grid they implemented, its design-of-experiments
variant and its plotting are now GridSearchCV, which cross-validates,
parallelises and refits on the best parameters.

Two further defects surfaced on the way:

  - matplotlib 3.0 removed pylab.hold, which five files still imported and
    called: gaussprocess, three examples and the feed-forward tutorial. All of
    them raised ImportError on the first plot. Overlaying is the default now,
    so the calls simply go.
  - Adding __author__ to every file put the line ahead of `from __future__
    import print_function` in two of them, which must come first. That broke
    example_tools and the fnn tutorial. My check at the time used ast.parse,
    which does not enforce the rule -- only compile() does, and the check now
    uses it.

Verified: the SVM trains, classifies at 88.9% on generated data and 90.0%
after a search, saves and reloads to identical predictions, produces
probabilities, and honours the old parameter names. The worked example under
examples/supervised/neuralnets+svm runs end to end at 1.67% train and 7.33%
test error.

Documented in the API pages, the README and the status page, with the modules
removed from the list of what is unavailable, and an [svm] extra in setup.py.
The documentation now carries 167 executable examples, all passing.

xodetools generates the .xode scene files the physics environments read, and
the documentation tells you to use it. It sat under rl/environments/ode, whose
__init__ raises ImportError to point at the MuJoCo replacement -- which made
the tools unreachable. That was a hole I opened when the ODE package became a
signpost, and the documentation walked readers straight into it.

They now live under rl/environments/mujoco/tools, which is where they belong:
the scenes they write are read by the MuJoCo converter.

Two Python 2 leftovers had to go before they would run at all, neither ever
reachable to fail before now:

  - configgrab called string.strip(), a module-level function removed in
    Python 3
  - xodetools opened its output in binary mode and wrote strings to it, which
    Python 2 allowed. newline='\n' keeps the unix line endings the binary mode
    was there for.

Verified end to end: XODEJohnnie writes a scene, the converter reads it, and
MuJoCo runs it for 300 steps -- 15 bodies and 11 actuators, matching the
model that ships with the library.

pybrain.tools.rlgluebridge existed to run PyBrain learners on environments
maintained elsewhere. RL-Glue, the protocol it spoke, has had no maintained
implementations for over a decade, so restoring that bridge would have
connected to nothing. Gymnasium is where that ecosystem went, and it is where
the capability comes back: classic control, Box2D, MuJoCo, Atari and the
third-party environments beyond them.

Two models had to be reconciled. Gymnasium's step returns observation, reward,
terminated, truncated and info together, where PyBrain splits the same
information between an Environment that observes and acts and a Task that
judges reward and decides when an episode ends. GymnasiumEnvironment keeps
what the task needs and GymnasiumTask reads it back; terminated and truncated
both end the episode while staying distinguishable.

Action spaces map both ways. A discrete space takes one value per action with
the largest winning, which is what ActionValueNetwork already emits; a
continuous one takes the vector and clips it to the space rather than
rejecting it. Observations are scaled to (-1, 1) where the space states finite
bounds, and passed through where it does not -- CartPole's pole velocity is
unbounded, for one.

Verified by learning rather than by importing: an OptimizationAgent with a
HillClimber over CartPole-v1 takes a network of zero weights from 11 steps to
59 across 60 episodes. Both space types were exercised, on CartPole and
Pendulum.

Documented in advanced/gymnasium with nine executable examples, referenced
from the entry that used to say only that rlgluebridge was unavailable, and
carried into the README with a worked example under
examples/rl/environments/gymnasium.

Says updated, further developed and maintained rather than only maintained,
and marks the list of work as not exhaustive.

Serializable.loadFromFile opened its file with mode 'rbU'. The 'U' flag asked
for universal newline translation, which never applied to binary mode in the
first place, and Python 3.11 removed it: the call raises ValueError, so
anything saved with saveToFile could not be read back. Only Python 3.10, the
oldest version supported here, still accepted it.

The test covered only saveToFileLike and loadFromFileLike, over a BytesIO,
which is why this passed unnoticed on every run. It now exercises the
file-based pair as well.

The README and the status page said "Verified on Ubuntu 22.04, 24.04 and 26.04
and on macOS", which claimed more than was done: those distributions were
represented by their Python, numpy and SciPy versions rather than run on, and
"macOS" read as covering Apple Silicon, which was never touched. Windows was
never touched either.

What was run is Linux x86_64 and macOS x86_64, across Python 3.10, 3.13 and
3.14. Both pages now say that, and say plainly that Windows and arm64 are
untested -- not known to fail, simply never tried -- with what to expect on
each: the Visual C++ build tools for the cart-pole extension on Windows, and a
MuJoCo wheel on Apple Silicon, which Intel Macs do not get.

HTML and PDF rebuilt from the changed source.

The PDF reached 82 pages when the platform section went in, and the Gymnasium
bridge brought the module count to 404. Both were still quoting the earlier
figures. Checked against the repository rather than adjusted by memory: the
line count, the 25 optimisers, the module totals and the example count all
hold as stated.

buildNetwork sets opt['recurrent'] = True whenever the hidden or output class
is sequential, because such a network has to be a RecurrentNetwork to carry
state from one timestep to the next. That is a statement about the network
type, not about the topology -- but the LSTM branch further down read the same
flag to decide whether to wire the hidden layer back into itself.

The two questions had collapsed into one. buildNetwork(2, 3, 1,
hiddenclass=LSTMLayer, recurrent=False) got the self-connection anyway, and
there was no way to ask for an LSTM layer without it: 76 parameters where 40
were wanted.

The originally requested value is now kept separately and used for the
topology decision, while the network type keeps following the class. Asking
for recurrent=True still yields one recurrent connection and 76 parameters;
recurrent=False now yields none and 40.

LSTMLayer(n) is n blocks of one cell each, not one block of n cells: its
forward pass is elementwise throughout, so every cell carries its own input,
forget and output gate and can be written, kept and read independently of the
others. That is the S = 1 special case of the memory block, and it is the only
case the library offered -- there was no way to build a block of several cells
under shared gating.

MultiCellLSTMLayer(cells, blocks) fills that gap. One input gate, one forget
gate and one output gate govern all cells of a block, so a block of S cells
takes 3 + S inputs instead of 4S, and the layer costs 55 % of the parameters
at equal cell count. The consequence for the backward pass is that a shared
gate receives the sum of what each of its cells sends back, where the
elementwise version passes a single term through; peepholes likewise let each
gate see every state in its block, one weight per cell per gate.

Gradients are checked against numerical ones at several block and cell counts,
with and without peepholes. gradientCheck runs on random data and trips its own
tolerance now and then, so each configuration is given several seeds and has to
succeed on most -- the standard LSTMLayer meets the same bar and no better.

Both classes now open with a sentence saying which of the two they are, since
the names alone do not distinguish them.

A tutorial page setting LSTMLayer against MultiCellLSTMLayer: what a block is
in each, what the sharing costs and buys, and when to reach for which. The
figures come from examples/supervised/lstm_block_comparison.py, added here,
which runs both at three cells over eight timesteps, three runs each, 120
epochs of RPROP, on two tasks built to separate the cases.

Where the remembered values are gated together -- shown at once, asked for at
once -- the multi-cell block reaches a slightly lower error, 0.01229 against
0.01295, on 51 parameters against 93. Where each value arrives at its own
timestep, a shared input gate cannot write them apart and it comes out about
6 % worse, 0.01686 against 0.01585, still on 55 % of the parameters. That is
the whole trade, and the page says so rather than claiming a winner.

The page closes by placing the class historically: blocks of S cells sharing
gates are the original 1997 formulation, not a new architecture. What is new
here is only that this library now has the general case.

HTML and PDF regenerated from the changed sources; the PDF goes from 82 to 84
pages with the memory-block tutorial. The README gains a paragraph on the two
LSTM layers and pointing at that tutorial, and its counts are refreshed
against the repository: 389 of 392 modules import from a fresh clone, 38 tests
pass, 182 documentation examples execute.

multicelllstm.py sat between mdlstm.py and neuronlayer.py in a directory
listing, which is the wrong place for it: someone scanning the modules
directory for the available LSTM variants would pass over it. Renamed to
lstm_multi_cell.py, which puts it directly under lstm.py where it belongs.

The class name is unchanged, so no import of MultiCellLSTMLayer through
pybrain.structure or pybrain.structure.modules is affected; only the module
path moves. The cross-references in lstm.py and in the memory-block tutorial
follow it, and HTML and PDF are rebuilt.

While in the API reference: MultiCellLSTMLayer was missing from it entirely.
That page lists its classes explicitly rather than sweeping the module, so a
new class does not appear by itself. It is now listed after LSTMLayer.

…aimed

The comparison in the memory-block tutorial reported final training error over
three runs. On that measure the multi-cell block came out slightly ahead on the
jointly gated task, and the page said so. The measure was the wrong one: it
records how closely a network fits sequences it has already been trained on,
not whether it remembers them.

Repeated on held-out sequences over ten seeds, the picture is different. The
ordering reverses, and more importantly every gap between the two layers is
smaller than the spread across seeds -- at three and at five cells, on both
tasks, including the separately gated task built specifically to be hard for a
shared gate. The restriction we expected to cost something does not cost a
detectable amount at this scale.

What survives is the part that never depended on training at all: gating is
paid for once per block rather than once per cell, so the multi-cell layer uses
55 % of the parameters at three cells and 46 % at five, exactly and for any
task. The tutorial now leads with that, gives the new table with its spreads,
and states plainly that the error differences are not measurable. A note
records the earlier figures and why they were replaced, rather than quietly
substituting the new ones.

The example script is reworked to match: it evaluates on held-out sequences,
prints the seed spread beside every mean, and says outright when a gap is
smaller than that spread. It also says how it differs from the longer run the
tutorial quotes, so nobody reads its three seeds as the same evidence.

README, HTML and PDF follow.

… bars

The previous commit measured held-out error and then asked, for each
condition, whether the gap between the layers exceeded the standard deviation
across seeds. It never did, so the tutorial concluded that the two are
indistinguishable at this scale.

That test threw away the design. Seed k gives both layers the same training
and held-out sequences, so the difference can be taken seed by seed, and the
paired form of the same data is far more sensitive: the multi-cell block is
behind in all four conditions, and in one of them -- separately gated, five
cells -- at p = 0.037 by a paired t-test and p = 0.027 by Wilcoxon. That one
does not survive Holm correction across the comparisons made, so on its own it
would prove little.

The evidence is in the consistency. Extending to the block-size sweep gives ten
paired comparisons, and sharing is the worse arrangement in nine of them, which
a fair coin does with probability 0.021. Ten individually inconclusive
comparisons are not ten null results.

So the honest reading is a small penalty for sharing gates, present across both
tasks, both cell counts and every block size, too small for ten seeds per
condition to resolve, against a parameter count reduced by up to 56 %. A trade,
mild, rather than the free choice claimed before. The tutorial now says that,
and the guidance is adjusted: the grouping argument for the multi-cell layer is
marked as a hypothesis about the reader's data, because the tasks here did not
show it paying off even where the grouping was true by construction.

The example script gains the same caveat, since three seeds cannot show any of
this and should not be read as if they could. README, HTML and PDF follow.

Everything measured so far held the cell count equal and let the parameter
count fall, which isolates what gate sharing costs: a little, consistently.
That is the scientific comparison. It is not the one a practitioner faces, who
has a parameter, memory or energy budget and gets to choose the cell count.

Asked the second way the answer changes sign, because the saving can be spent.
LSTMLayer(5) costs 245 parameters on this task; a shared-gate block of nine
cells costs 225, so 8 % less money buys nearly twice the memory. On the jointly
gated task that block reaches 0.0234 against 0.0423, a 45 % lower held-out
error, with the paired confidence interval entirely below zero (p = 0.0008).
On the separately gated task it is ahead as well, there within the noise, and
the two-block arrangement at 236 parameters is ahead significantly.

Run the other way -- how few cells suffice -- a single block of six cells at
138 parameters is the point from which nothing larger is measurably behind the
reference, on both tasks. That is 56 % of the budget, so roughly 44 % of the
parameters can be dropped before this experiment can detect a difference.

The criterion for that number is stricter than the obvious one, and the code
says why: a plain "cheapest interval containing zero" would have answered four
cells on the jointly gated task, purely because that configuration is noisy
enough for its interval to cover zero while the more expensive five-cell block
is judged behind. Requiring the property to hold from that point upward makes
the answer a crossing point rather than the noisiest member of the sweep.

The tutorial now carries both comparisons and says plainly that they do not
contradict each other: at equal cells sharing costs a little, at equal cost it
buys cells, and the cells are worth more than the sharing costs. The guidance
is rewritten around which quantity the reader actually holds fixed, with the
warning that sizing a shared-gate block to the same cell count as the layer it
replaces gives away its only advantage.

Two caveats are stated wherever the numbers are: an interval containing zero
means equality was not ruled out rather than shown, and a parameter count is a
proxy for energy, not a measurement of it.

The new data lives in its own file. The earlier study is not superseded --
it answers a different question -- and nothing in it was overwritten.

…ts own

A literature check turned up prior art for the central empirical claim, and it
is in the paper that introduced LSTM. Table 1 of Hochreiter and Schmidhuber
(1997), on the embedded Reber grammar, compares four blocks of one cell at 264
weights against three blocks of two cells at 276, and the multi-cell
arrangement reaches the criterion in 21 730 sequence presentations against
39 740 -- roughly 1.8 times faster at nearly the same weight count. The same
paper states outright that its experiments use blocks of various sizes.

That is the equal-budget comparison, published twenty-nine years ago, pointing
the same way as the measurements on this page. The tutorial's closing note
already said the architecture was not new; it now says the finding is not new
either, and quotes the figures.

This does not weaken the numbers here -- an independent replication across a
gap of that size makes the effect more credible, not less. It does mean the
page should not read as though the question had been open.

The most-asked comparison in recurrent networks is LSTM against GRU, and the
library could not make it. GRULayer fills that in: an update gate deciding how
much of the previous output survives, a reset gate deciding how much of it the
candidate is allowed to see, and no separate cell state -- three inputs per
unit where an LSTM cell needs four.

One design point is worth recording because it breaks with the rest of the
library. Every other recurrent layer here takes its recurrence from an
explicit addRecurrentConnection and receives the feed-forward and recurrent
parts of its input already summed. A GRU cannot work that way: the reset gate
has to act on the recurrent contribution alone, before the candidate sees it,
and a summed buffer cannot be taken apart again. GRULayer therefore carries its
three recurrent matrices itself and needs no recurrent connection. Adding one
anyway is harmless -- it gives the gates a second, unreset view of the previous
output -- but the layer is complete without it, and the module docstring says
so.

Carrying its own recurrence also means carrying its own backward accumulation:
the error arriving from the next timestep has no connection to travel along, so
the layer keeps it in a buffer of its own, the way LSTMLayer already does for
its cell state.

Gradients check against numerical ones on 8 of 8 seeds with and without an
added recurrent connection, where LSTMLayer on the same harness manages 6 of 8.

A standard cell decides separately what to write into the state and what to
keep in it, which allows the two to disagree: a cell can be left unwritten and
also never cleared. Coupling them ties the keep-fraction to the write-fraction
as f = 1 - i, so whatever is overwritten is exactly what is forgotten.

That is one gate fewer, hence three inputs per cell rather than four, and two
peephole weights per cell rather than three, since the forget gate is now the
input gate and shares its view of the state. Greff et al. (2017) report the
variant performing on par with the full formulation, which makes it a
reasonable default rather than a curiosity.

It also belongs next to MultiCellLSTMLayer for a reason beyond arithmetic. Both
remove a degree of freedom by making one gate serve two purposes -- the
multi-cell block shares a gate across cells, the coupled gate shares one across
the write and forget decision. Having both in the library is what lets the two
kinds of sharing be set against each other.

The backward pass differs from LSTMLayer in one substantive place: the gate now
moves the state in two directions at once, so its error carries the difference
between what is written and what it replaces, and the state error inherited
from the next timestep is scaled by 1 - i rather than by an independent f.

Gradients check on 6 of 8 seeds at three cells and 8 of 8 at one, matching
LSTMLayer's own 6 of 8 on the same harness.

The forget gate is not in the original publication; Gers, Schmidhuber and
Cummins added it in 2000. Without it the state is a pure accumulator: what is
written stays, further writes add to it, and nothing ever decays. That is worth
being able to build, both for teaching and because it is the baseline against
which the forget gate was argued for.

LSTMLayer(dim, forgetgate=False) does it. The layer then takes 3n inputs
instead of 4n, in the order input gate, cell input, output gate, and peepholes
cost two weights per cell instead of three, a gate that does not exist needing
no view of the state. The default is unchanged, so nothing that already used
this class behaves differently.

One implementation note that cost a debugging pass. The flag cannot be stored
as self.forgetgate: a buffer of that name is created during construction and
overwrites it. But the name has to exist on the class anyway, because setArgs
only records keywords the class declares and otherwise prints a warning that
lands in the output of every doctest constructing an LSTM. Both names are
therefore present, with a comment saying which one is live.

The test shows the accumulation directly, and then shows why it matters: after
three writes the state is at 1.99 and tanh of that is 0.963, most of the way to
the saturation that the forget gate exists to prevent.

Gradients check on 8 of 8 seeds without the gate, against 6 of 8 with it.

…layers

Dropout sets units to zero, which in a recurrent layer also cuts the path
gradients travel back along. Zoneout instead makes a unit keep its previous
value: with probability p the new value is discarded and the old one carried
forward, so a zoned-out unit still passes gradient backwards through time
undamaged. Cell state and output get separate rates, and the defaults follow
Krueger et al. (2017) in putting a high rate on the state and a low one on the
output.

Two things needed care.

The output path. Zoning out the output makes h_t depend directly on h_{t-1},
which is a recurrent path with no connection to carry its error. The layer
therefore keeps that error itself, the way GRULayer does -- and the state path
now reaches the next timestep twice, carried by the mask and through the forget
gate on the fraction that was not carried, so the inherited state error is
scaled by m + (1 - m) f rather than by f alone.

The masks. They are stored per timestep rather than resampled, because the
backward pass has to see the same coin flips the forward pass did. Getting that
wrong would produce gradients that are quietly wrong rather than obviously so.

Training and evaluation genuinely differ here, and nothing in PyBrain toggles
the mode, so the caller must: layer.training = False before measuring. The
evaluation path uses the expectation and is deterministic, which is also what
makes the gradient checkable at all -- under sampling the function changes
between the two evaluations a numerical gradient needs.

With both rates at zero the layer reproduces LSTMLayer step for step, which the
test asserts directly and is the strongest evidence available that the rest is
right.

An LSTM cell has three gates, a GRU two, and the minimal gated unit of Zhou et
al. (2016) one, used for both jobs a GRU splits: it decides how much of the
previous value survives and how much of it the candidate is allowed to see.
Two inputs per unit, against a GRU's three and an LSTM cell's four.

With GRULayer, CoupledLSTMLayer, LSTMLayer with and without its forget gate,
and MultiCellLSTMLayer, the library now spans four gates down to one, which is
what makes any of them informative: the interesting question is not how a
single variant performs but what each gate is buying, and that needs the series
rather than the points.

The backward pass has one place worth naming. The gate appears twice in the
forward computation -- once interpolating between old value and candidate, once
masking what the candidate sees -- so both uses contribute to its error, and
dropping either term produces a gradient that is close enough to look plausible
and still wrong. The test pins the layer against a GRU whose two gates are
given the same value, which is the configuration the two architectures share.

Gradients check on 8 of 8 seeds. Like GRULayer, the layer carries its own
recurrent weights, for the same reason: the gate has to act on the recurrent
contribution before the candidate sees it.

The LSTM-side relative of this idea is JANET, which drops the input and output
gates instead and keeps a cell state. The module docstring records the
relationship and why the GRU-side version is the one implemented here.

A plain LSTM layer sends all its cell outputs back to its own gates, so the
recurrent matrix costs 4c by c and grows quadratically in the cell count. Sak
et al. (2014) put a linear projection in between: c cells produce c values, a
matrix reduces them to r, and it is those r that go round. Memory can then be
made large without the recurrence becoming the dominant expense.

The test states the saving concretely rather than describing it: six cells fed
back directly need 144 recurrent weights, projected to two they need 48, with
the same six cells of memory.

This is the first layer here whose input and output sizes come apart -- 4c in,
r out -- which is worth knowing when wiring one up, and is noted in the module
docstring. The projection is deliberately linear; Sak et al. leave it so, and
squashing it would make the layer a different thing.

The backward pass goes through the projection first, which also collects its
own derivative, and the cells then see the mapped-back error in place of the
outer one. Gradients check on 8 of 8 seeds, 6 of 8 with peepholes, against
LSTMLayer's own 6 of 8.

One assumption the test heads off: setting the projection equal to the cell
count does not give back a plain LSTMLayer. The matrix is still there, still
learnt, and still initialised at random.

Batch normalisation does not transfer to recurrent networks: the statistics
depend on the batch, and a recurrent layer sees a different distribution at
every timestep. Layer normalisation normalises across the units of a layer
instead, which makes it a per-timestep, per-example operation and therefore
usable here.

Five vectors are normalised -- the four gate pre-activations and the cell state
before it is squashed for output -- each with its own learnt gain and bias, so
ten parameters per cell. That is the first layer in this library whose
parameters are not peepholes or a projection.

Two departures from Ba et al. (2016), both stated in the module docstring
rather than left to be discovered. They normalise the input and recurrent
contributions separately, before adding; this layer receives them already
summed, as every gate-input layer in PyBrain does, and normalises the sum.
Separating them would mean the layer owning its recurrent weights, which is a
bigger change than the normalisation warrants. And peepholes are not offered,
because the interaction between a peephole reading the raw state and a
normalisation rescaling it is not something to invent here.

The normalisation has a backward pass of its own, and it is the part worth
checking: it couples every unit of a vector to every other, so an error in it
would not show up as an obviously wrong number. Gradients check on 7 of 8 seeds
at three and five cells and 8 of 8 at one.

The test also records a consequence of normalising across units that is easy to
walk into: with a single cell there is nothing to normalise across, the
normalised vector is zero whatever the input, and only the bias survives.

An LSTM cell mixes its input and its previous output only through the gate
weights, and only additively. Melis et al. (2020) let them modulate each other
multiplicatively first, in alternating rounds: odd rounds scale the input by a
gate read off the previous output, even rounds scale the previous output by a
gate read off the input. The factor of two in each round makes the identity the
expected starting point, so the modulation does not halve the signal every time.

This one breaks the library's convention, and there was no way around it. Every
other LSTM variant here receives 4c pre-summed gate inputs and leaves the
weights to connections. A mogrifier has to see the raw input and the raw
previous output separately, before any weight touches them, so this layer owns
its input weights, its recurrent weights and the mogrifier matrices. Its indim
is the input width, not four times the cell count, and buildNetwork cannot
construct one. All of that is in the module docstring, with a worked wiring
example, because a layer that breaks the convention silently would be worse
than one that does not exist.

The backward pass is the most involved here: errors run back through the rounds
newest first, each round contributing to the other vector's error through its
gate matrix. Gradients check on 6 to 7 of 8 seeds at zero, one, two and four
rounds, against LSTMLayer's own 6 of 8 -- zero rounds exercising the plain cell
underneath and four the full chain.

The test also pins down the factor of two: at the first timestep the previous
output is zero, every odd gate sits at sigmoid(0) = 0.5, and the round leaves
the input exactly unchanged.

Ordinary recurrent layers update at every timestep, forcing irregular or
multi-rate signals onto one clock. Neil et al. (2016) give each cell a rhythm:
a time gate opens for a short window of a learnt cycle, the cell is updated
only while it is open, and between openings the state is held exactly rather
than decayed. Cells with different periods then cover different timescales.
The period and the offset are learnt, one of each per cell, with the period
held as its logarithm so a perturbation cannot drive it negative.

The layer needs a clock. Rather than extend every dataset class in the library
with timestamps, it takes the time as one extra input -- indim is 4*dim + 1 --
fed from a bias unit for a regular clock or from an input unit for an irregular
one. The cost is that buildNetwork cannot construct one, which the docstring
says.

Verifying this took three wrong turns, and the test records the outcome of each.

The gradient with respect to the time input was initially set to zero, on the
grounds that time is data. That is wrong as soon as a learnable connection
feeds it, which is exactly what the test harness does; the phase depends on
time as 1/tau per cell and the error now says so.

The standard gradientCheck helper then still failed almost everywhere, and the
reason turned out not to be the layer. With the gate shut for most of a cycle
the layer attenuates nearly everything, so most parameters end up with true
gradients around 1e-9, and the helper's relative measure compares two numbers
that are both rounding error. Every gradient above a noise floor agrees to
better than 2e-4; the test checks those and reports how many there were, so it
cannot pass by testing nothing.

The gate's closing ramp reaches zero exactly where the leak takes over, so the
gate steps up at that point rather than continuing down. That discontinuity is
in the published formulation, and the test asserts it rather than smoothing it
away.

Until now the library had two gradient trainers: BackpropTrainer, which is
stochastic gradient descent with momentum and a decaying learning rate, and
RPropMinusTrainer, which discards the gradient's magnitude and adapts a step
width per parameter from the sign alone. RProp is a full-batch method and does
not tolerate the noise of small batches; plain backprop needs its learning rate
chosen for the problem.

Adam sits between them, and its absence was the largest gap on the training
side. It keeps a running mean of the gradient and a running mean of its square
and steps proportionally to the first over the root of the second, so the step
size is scale-free per parameter while the direction still uses the magnitude
rather than only the sign.

Both trainers change nothing but the update rule; the forward pass,
backpropagation through time and the dataset handling stay BackpropTrainer's.
RPropMinusTrainer had already established that shape, so this follows it, and
the arithmetic lives in a descender class next to GradientDescent. RMSProp is
the same class with the first moment switched off, which is what it is.

On XOR over three seeds and 400 epochs, Adam at alpha=0.05 reaches zero error,
matching RProp, where plain backpropagation at its default rate does not. The
test asserts that, and also asserts the less flattering fact that Adam at its
own default of 0.001 does *not* get there in that budget -- the default is
meant for larger problems, and someone trying it on a four-sample task should
know that before concluding it is broken.

The test also pins down where the scale-freeness stops. The first step is
exactly alpha in magnitude for any gradient well above epsilon, but at a
gradient of epsilon itself it is half of alpha. That is intended -- it keeps a
parameter with no real gradient from being moved a full step by noise -- but it
means epsilon has to sit below the gradients that matter.

Two new tutorial pages. One sets all nine gated recurrent layers against each
other on a single recall task at equal unit count, with the parameter counts
beside the errors; the other explains when to reach for which trainer.

The comparison table is worth reading for what it does not say. Most of the
gaps between layers are smaller than the spread across the three seeds beside
them, and the page says so rather than presenting a ranking. Where the gaps are
real the direction is not the expected one: the two cheapest layers, MGU with a
single gate and the coupled-gate cell, both beat the full LSTM. On a task this
small and a budget this short, the extra gates are mostly extra things to fit.
That is a fact about this task, and the page says that too.

Zoneout comes last, which is correct behaviour rather than a defect -- it is
regularisation, and a task with nothing to overfit is where regularisation
costs without paying. Layer normalisation is also behind, which is what to
expect when normalising across four units.

The generating script is examples/supervised/recurrent_layer_comparison.py.

README gains a paragraph on the layers and one on the trainers, and its counts
are refreshed: 408 of 411 modules import from a fresh clone, 48 tests pass, 203
documentation examples run. HTML and PDF rebuilt, the PDF going from 84 to 92
pages.

Two tables in the tutorials were three unpaired seeds with no test, and one of
them carried a directional claim: that MGU, with its single gate, and the
coupled-gate cell both beat the full LSTM "by more than their spread". Nine
layers means eight comparisons against the reference, and eight uncorrected
tests at the 5 % level turn up at least one difference by chance about a third
of the time. That is the trap this repository's own documentation warns about
elsewhere.

Re-run over twenty seeds, every layer on the same seeds so the differences can
be taken seed by seed, and Holm-corrected together. Half the claim survives and
half does not.

Coupling the input and forget gates does beat the standard cell: 0.0132 against
0.0278, at 77 % of the parameters, p = 0.0019 after correction. MGU does not:
0.0286 against 0.0278, p = 0.88. At three seeds it had come out at 0.0051 --
what a small sample does when it is allowed to pick the winner from nine
candidates. The tutorial now says both things, and a note records the wrong
version rather than quietly replacing it.

Two further results the small sample had missed. The multi-cell block uses 49 %
of the parameters for no measurable difference, which is a cleaner statement
than the ranking it previously appeared in. And three layers are genuinely
worse here -- zoneout, layer normalisation, the projection -- each for a reason
about the task rather than the layer, which the page now gives.

The trainer table gets the same treatment and gains a second task, the recall
problem with an LSTM layer, since the update rule does not care what produced
the gradient and a library used mostly for sequences should be measured on one.
All four trainers beat plain backpropagation on both, all four survive
correction. That surfaced a claim of mine that was wrong: RPROP- was described
as hard to beat in the full-batch setting it assumes, and on the recurrent task
Adam reaches 0.0040 against its 0.0245, interval well clear of zero. The
guidance now leads with what RPROP- actually offers -- no learning rate to tune
-- rather than with a superiority it does not have.

The example script keeps its three seeds, since it is a demonstration and has
to finish in minutes, but now says outright what three unpaired seeds can and
cannot support.

The credits section went straight from the licence to the original authors,
which left the maintenance of the fork to be inferred from acknowledgements.txt.
It now says plainly who keeps it running, before the paragraph on where the
library came from. The original attribution and the pointer to the full
acknowledgements are unchanged.