Refactor downbeats and add bar tracking #316

superbock · 2017-07-25T10:34:14Z

This PR moves all downbeat & pattern tracking functionality into a separate downbeats module. It additionally adds bar tracking (i.e. track the downbeats given the beat positions) functionality.

~~Although this PR is more or less ready, there are a couple of things to be discussed:~~

Should RNNBarProcessor return a single 2D numpy array containing the beat times as the first column and the probability to be a downbeat position as the second column (eventually returning NaN as its last (or: first, see next point) value. EDIT: returns a 2D array with NaN as the last downbeat probability. This also enables us to simply save the bar activations.
If we return the same number of downbeat probabilities as beat positions, should these probabilities be based on the features of the following beat period (as it is now) or on the one which just passed? Or: should these be aligned differently, e.g. centred? EDIT: features are centred around the beat (or interpolated subdivision) and num_subdivision features (for the current) beat are returned. We might change this for online mode, however.
Should the position of the window to aggregate/summarise the features around the (sub-)beat positions/divisions be coupled with the origin parameter of FramedSignal? EDIT: deferred for later (i.e. when implementing online downbeat trackers).
Check the impact/usefulness of interpolating missing values when synchronising features. EDIT: removed, since no difference was observed.
Decide how pattern transitions in beats_hmm.MultiPatternTransitionModel should be parametrised, right now the parameters are kind of overlapping. EDIT: use transition_prob to be able to model transitions between patterns.
Fix inability to transition between patterns with more than 1 entry state.
Decide which classes to import into the features-namespace.
Find and fix memory leak (suspect: CLPChromaProcessor). EDIT: (at least partially) fixed in prevent memory leak when accessing signal frames #322.

superbock · 2017-08-11T10:50:39Z

On the Ballroom set, interpolation has exactly 0 impact, thus I think we can safely remove it completely.

superbock · 2017-08-11T10:52:02Z

Rebased on master after merging #322 solves the memory issue. CLPChromaProcessor makes heavy use of FramedSignal which in turn triggered the slice memory/reference leak.

superbock · 2017-08-14T14:13:22Z

MultiPatternStateSpace and MultiPatternTransitionModel use different arguments than their beat and bar counterparts. Should we change this to be more consistent with the others or keep it as it is?

fdlm

I think this is fine, although to be honest, I did not go through each change in detail (e.g. I did not check, if moving the Downbeat classes to a different module breaks something - I hope the tests catch that).

Besides that - good work 👍

fdlm · 2017-09-19T08:57:35Z

madmom/ml/hmm.pyx

        return np.log(self.probabilities)

+    @staticmethod
+    def make_dense(states, pointers, probabilities):


This does not relate to this PR directly, but I'm not sure if make_dense and make_sparse are good names here, because imho they just convert to/from different sparse representations of the transition matrix (CCO to CSR, see https://en.wikipedia.org/wiki/Sparse_matrix)...

Yes, this is something which bothered me as well, especially since it doesn't become really dense, just a bit 'denser'. How about compress/de-compress instead? We can deprecate make_sparse then -- although I don't think a lot of people use it, since this is a rather low-level routine.

I will still merge this PR as it is now, since there exists already a from_dense method, so this should be addressed jointly. Created issue #335.

fdlm · 2017-09-19T08:59:08Z

madmom/ml/nn/layers.py


    def __setstate__(self, state):
+        # TODO: old models have a 'hid_init' instead of an 'init' attribute
+        #       remove this unpickling code after updating all models


I think we should track such ToDos in the issue tracker. ToDo items in code tend to be forgotten after some time... 😄

I am not sure if all of these TODOs should go into the issue tracker, since there are plenty and some are simply not really urgent. However, I still mark them with TODO, since then at least Pylint lists them and they won't be forgotten completely.

superbock force-pushed the refactor_downbeats branch 9 times, most recently from 26ae1d2 to 087c8c2 Compare July 27, 2017 13:41

superbock force-pushed the refactor_downbeats branch 4 times, most recently from 2222f6a to 66cc302 Compare August 4, 2017 15:56

superbock requested a review from flokadillo August 4, 2017 16:45

superbock changed the title ~~[WIP] Refactor downbeats~~ Refactor downbeats and add bar tracking Aug 4, 2017

superbock force-pushed the refactor_downbeats branch 4 times, most recently from 84bc0ef to b0f698e Compare August 6, 2017 14:41

superbock mentioned this pull request Aug 6, 2017

[WIP] BarTracker #258

Closed

superbock force-pushed the refactor_downbeats branch from b0f698e to eb6abd4 Compare August 10, 2017 13:06

superbock mentioned this pull request Aug 10, 2017

Memory leak #321

Closed

superbock force-pushed the refactor_downbeats branch 2 times, most recently from 70192c5 to 9d24194 Compare August 11, 2017 10:45

superbock force-pushed the refactor_downbeats branch from 9d24194 to 4e80be8 Compare August 14, 2017 14:09

superbock force-pushed the refactor_downbeats branch 2 times, most recently from 730f7d2 to 48a9177 Compare August 16, 2017 08:28

superbock force-pushed the refactor_downbeats branch from 48a9177 to d8eb403 Compare August 16, 2017 08:29

superbock force-pushed the refactor_downbeats branch 7 times, most recently from 3691d5b to d833ec7 Compare September 18, 2017 19:38

superbock requested a review from fdlm September 19, 2017 08:34

fdlm reviewed Sep 19, 2017

View reviewed changes

Sebastian Böck and others added 13 commits September 19, 2017 12:33

refactored downbeat and pattern stuff into features.downbeats

cf51399

add pattern transitions to MultiPatternTransitionModel

304ac45

add CLPChromaProcessor

c237f0d

compute energy on the filtered signal, not on the FramedSignal

5ecd9bb

fix loading and resetting of old GRU models with hid_init attributes

3de64c7

add bar tracking functionality

59af3b5

refactor downbeat reporting to programs

b3d7e6c

remove feature interpolation

d4a27a7

RNNBarProcessor returns 2D activations (beats, probabilities)

8d08a00

add BarTracker program

d13c4d4

cover more test cases

927982d

add downbeat docs

32628be

disable batch processing test on Windows

10596e0

superbock force-pushed the refactor_downbeats branch from 0af2cba to 10596e0 Compare September 19, 2017 10:53

superbock mentioned this pull request Sep 19, 2017

Naming of make_sparse / make_dense in HMM module #335

Open

superbock merged commit b8306ed into master Sep 19, 2017

superbock deleted the refactor_downbeats branch September 19, 2017 11:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Refactor downbeats and add bar tracking #316

Refactor downbeats and add bar tracking #316

Uh oh!

superbock commented Jul 25, 2017 •

edited

Loading

Uh oh!

superbock commented Aug 11, 2017 •

edited

Loading

Uh oh!

superbock commented Aug 11, 2017

Uh oh!

superbock commented Aug 14, 2017 •

edited

Loading

Uh oh!

fdlm left a comment •

edited

Loading

Uh oh!

fdlm Sep 19, 2017

Uh oh!

superbock Sep 19, 2017 •

edited

Loading

Uh oh!

superbock Sep 19, 2017

Uh oh!

fdlm Sep 19, 2017

Uh oh!

superbock Sep 19, 2017 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Refactor downbeats and add bar tracking #316

Refactor downbeats and add bar tracking #316

Uh oh!

Conversation

superbock commented Jul 25, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

superbock commented Aug 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

superbock commented Aug 11, 2017

Uh oh!

superbock commented Aug 14, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fdlm left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fdlm Sep 19, 2017

Choose a reason for hiding this comment

Uh oh!

superbock Sep 19, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

superbock Sep 19, 2017

Choose a reason for hiding this comment

Uh oh!

fdlm Sep 19, 2017

Choose a reason for hiding this comment

Uh oh!

superbock Sep 19, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

superbock commented Jul 25, 2017 •

edited

Loading

superbock commented Aug 11, 2017 •

edited

Loading

superbock commented Aug 14, 2017 •

edited

Loading

fdlm left a comment •

edited

Loading

superbock Sep 19, 2017 •

edited

Loading

superbock Sep 19, 2017 •

edited

Loading