Skip to main content

Yevgeny Seldin

University of Copenhagen, Department of Computer Science (DIKU), Faculty Member

Followers

15

Following

6

Co-authors

6

Public Views

Universite Jean Monnet - Saint-Etienne

François Laviolette

University of East Anglia

Delft University of Technology

Rajiv Sambasivan

University of Sidi Mohammed Ben Abdellah

Georgia Institute of Technology

Interests

Uploads

Papers by Yevgeny Seldin

On PAC-Bayesian bounds for random forests

Machine Learning

Existing guarantees in terms of rigorous upper bounds on the generalization error for the origina... more Existing guarantees in terms of rigorous upper bounds on the generalization error for the original random forest algorithm, one of the most frequently used machine learning methods, are unsatisfying. We discuss and evaluate various PAC-Bayesian approaches to derive such bounds. The bounds do not require additional hold-out data, because the out-of-bag samples from the bagging in the training process can be exploited. A random forest predicts by taking a majority vote of an ensemble of decision trees. The first approach is to bound the error of the vote by twice the error of the corresponding Gibbs classifier (classifying with a single member of the ensemble selected at random). However, this approach does not take into account the effect of averaging out of errors of individual classifiers when taking the majority vote. This effect provides a significant boost in performance when the errors are independent or negatively correlated, but when the correlations are strong the advantage from taking the majority vote is small. The second approach based on PAC-Bayesian C-bounds takes dependencies between ensemble members into account, but it requires estimating correlations between the errors of the individual classifiers. When the correlations are high or the estimation is poor, the bounds degrade. In our experiments, we compute generalization bounds for random forests on various benchmark data sets. Because the individual decision trees already perform well, their predictions are highly correlated and the C-bounds do not lead to satisfactory results. For the same reason, the bounds based on the analysis of Gibbs classifiers are typically superior and often reasonably tight. Bounds based on a validation set coming at the cost of a smaller training set gave better performance guarantees, but worse performance in most experiments.

Information Bottleneck for Non Co-Occurence Data Supplementary Material

This technical report provides supplementary material for the paper ìInformation Bottleneck for N... more This technical report provides supplementary material for the paper ìInformation Bottleneck for Non Co-Occurrence Dataî (1). The report includes run time analy- sis for the algorithm proposed in the ìTheoryî section of the paper; detailed results of the algorithm application to the ESR dataset reported in the ìApplicationsî sec- tion of the paper; visualization of the solution of MovieLens data

Unsupervised Sequence Segmentation by a Mixture of Switching Variable Memory Markov Sources

International Conference on Machine Learning, 2001

We present a novel information theoretic algorithm for unsupervised segmentation of sequences int... more We present a novel information theoretic algorithm for unsupervised segmentation of sequences into alternating Variable Memory Markov sources. The algorithm is based on competitive learning between Markov models, when implemented as Prediction Suffix Trees (Ron et al., 1996) using the MDL principle. By applying a model clustering procedure, based on rate distortion theory combined with deterministic annealing, we obtain a

PAC-Bayesian Analysis of Martingales and Multiarmed Bandits

Eprint Arxiv 1105 2416, May 12, 2011

We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random v... more We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This lemma provides an alternative tool to Hoeffding-Azuma inequality to bound concentration of martingale values. Our second approach is based on integration of Hoeffding-Azuma inequality with PAC-Bayesian analysis. We also introduce a way to apply PAC-Bayesian analysis in situation of limited feedback. We combine the new tools to derive PAC-Bayesian generalization and regret bounds for the multiarmed bandit problem. Although our regret bound is not yet as tight as state-of-the-art regret bounds based on other well-established techniques, our results significantly expand the range of potential applications of PAC-Bayesian analysis and introduce a new analysis tool to reinforcement learning and many other fields, where martingales and limited feedback are encountered.

PAC-Bayesian Analysis of the Exploration-Exploitation Trade-off

Corr Abs 1105 4585, May 20, 2011

We develop a coherent framework for integrative simultaneous analysis of the explorationexploitat... more We develop a coherent framework for integrative simultaneous analysis of the explorationexploitation and model order selection tradeoffs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a combination is also of independent interest for studies of multiple simultaneously evolving martingales.

Markovian domain fingerprinting in statistical segmentation of protein sequences

PAC-Bayesian Analysis of Contextual Bandits

We derive an instantaneous (per-round) data-dependent regret bound for stochastic multiarmed band... more We derive an instantaneous (per-round) data-dependent regret bound for stochastic multiarmed bandits with side information (also known as contextual bandits). The scaling of our regret bound with the number of states (contexts) N goes as p NI ⇢t (S; A), where I ⇢t (S; A) is the mutual information between states and actions (the side information) used by the algorithm at round t. If the algorithm uses all the side information, the regret bound scales as p N ln K, where K is the number of actions (arms). However, if the side information I ⇢t (S; A) is not fully used, the regret bound is significantly tighter. In the extreme case, when I ⇢t (S; A) = 0, the dependence on the number of states reduces from linear to logarithmic. Our analysis allows to provide the algorithm large amount of side information, let the algorithm to decide which side information is relevant for the task, and penalize the algorithm only for the side information that it is using de facto. We also present an algorithm for multiarmed bandits with side information with O(K) computational complexity per game round.

PAC-Bayesian Analysis of Co-clustering and Beyond

Journal of Machine Learning Research - JMLR, 2010

We derive PAC-Bayesian generalization bounds for supervised and unsupervised learning models base... more We derive PAC-Bayesian generalization bounds for supervised and unsupervised learning models based on clustering, such as co-clustering, matrix tri-factorization, graphical models, graph clustering, and pairwise clustering. 1 We begin with the analysis of co-clustering, which is a widely used approach to the analysis of data matrices. We distinguish among two tasks in matrix data analysis: discriminative prediction of the missing entries in data matrices and estimation of the joint probability distribution of row and column variables in co-occurrence matrices. We derive PAC-Bayesian generalization bounds for the expected out-of-sample performance of co-clustering-based solutions for these two tasks. The analysis yields regularization terms that were absent in the previous formulations of co-clustering. The bounds suggest that the expected performance of co-clustering is governed by a trade-off between its empirical performance and the mutual information preserved by the cluster variables on row and column IDs. We derive an iterative projection algorithm for finding a local optimum of this trade-off for discriminative prediction tasks. This algorithm achieved stateof-the-art performance in the MovieLens collaborative filtering task. Our co-clustering model can also be seen as matrix tri-factorization and the results provide generalization bounds, regularization terms, and new algorithms for this form of matrix factorization. The analysis of co-clustering is extended to tree-shaped graphical models, which can be used to analyze high dimensional tensors. According to the bounds, the generalization abilities of treeshaped graphical models depend on a trade-off between their empirical data fit and the mutual information that is propagated up the tree levels. We also formulate weighted graph clustering as a prediction problem: given a subset of edge weights we analyze the ability of graph clustering to predict the remaining edge weights. The analysis of co-clustering easily extends to this problem and suggests that graph clustering should optimize the trade-off between empirical data fit and the mutual information that clusters preserve on graph nodes.

PAC-Bayesian Analysis of Co-clustering with Extensions to Matrix Tri-factorization, Graph Clustering, Pairwise Clustering, and Graphical Models

Citeseer

... first non-trivial application. 2. The PAC-Bayesian bound for classification (5) is a direct c... more

Unsupervised sequence segmentation by a mixture of switching variable memory Markov sources

… WORKSHOP THEN CONFERENCE …, 2001

PAC-Bayesian analysis of contextual bandits

We derive an instantaneous (per-round) data-dependent regret bound for stochastic multiarmed band... more We derive an instantaneous (per-round) data-dependent regret bound for stochastic multiarmed bandits with side information (also known as contextual bandits). The scaling of our regret bound with the number of states (contexts) N goes as p NI ⇢t (S; A), where I ⇢t (S; A) is the mutual information between states and actions (the side information) used by the algorithm at round t. If the algorithm uses all the side information, the regret bound scales as p N ln K, where K is the number of actions (arms). However, if the side information I ⇢t (S; A) is not fully used, the regret bound is significantly tighter. In the extreme case, when I ⇢t (S; A) = 0, the dependence on the number of states reduces from linear to logarithmic. Our analysis allows to provide the algorithm large amount of side information, let the algorithm to decide which side information is relevant for the task, and penalize the algorithm only for the side information that it is using de facto. We also present an algorithm for multiarmed bandits with side information with O(K) computational complexity per game round.

PAC-Bayesian analysis of martingales and multiarmed bandits

We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random v... more We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This lemma provides an alternative tool to Hoeffding-Azuma inequality to bound concentration of martingale values. Our second approach is based on integration of Hoeffding-Azuma inequality with PAC-Bayesian analysis. We also introduce a way to apply PAC-Bayesian analysis in situation of limited feedback. We combine the new tools to derive PAC-Bayesian generalization and regret bounds for the multiarmed bandit problem. Although our regret bound is not yet as tight as state-of-the-art regret bounds based on other well-established techniques, our results significantly expand the range of potential applications of PAC-Bayesian analysis and introduce a new analysis tool to reinforcement learning and many other fields, where martingales and limited feedback are encountered.

PAC-Bayes-Bernstein Inequality for Martingales and its Application to Multiarmed Bandits

We combine PAC-Bayesian analysis with a Bernstein-type inequality for martingales to obtain a res... more We combine PAC-Bayesian analysis with a Bernstein-type inequality for martingales to obtain a result that makes it possible to control the concentration of multiple (possibly uncountably many) simultaneously evolving and interdependent martingales. We apply this result to derive a regret bound for the multiarmed bandit problem. Our result forms a basis for integrative simultaneous analysis of exploration-exploitation and model order selection trade-offs. It also opens a way for applying PAC-Bayesian analysis in other fields, where sequentially dependent samples and limited feedback are encountered.

Method for content presentation

Enhanced electronic program guides

by Yevgeny Seldin and Miriam Goldenberg

On Unsupervised Learning of Mixtures of Markov Sources

Abstract: Unsupervised classication, or clustering, is one of the basic problems in data analysis... more Abstract: Unsupervised classication, or clustering, is one of the basic problems in data analysis. Whilethe problem of unsupervised classication of independent random variables has been deeplyinvestigated, the problem of unsupervised classication of dependent random variables, andin particular the problem of segmentation of mixtures of Markov sources, has been hardly addressed.At the same time supervised classication of Markov sources has become

PAC-Bayesian Generalization Bound for Density Estimation with Application to Co-clustering

We derive a PAC-Bayesian generalization bound for density estimation. Similar to the PAC-Bayesian... more We derive a PAC-Bayesian generalization bound for density estimation. Similar to the PAC-Bayesian generalization bound for clas- sication, the result has the appealingly sim- ple form of a tradeo between empirical per- formance and the KL-divergence of the pos- terior from the prior. Moreover, the PAC- Bayesian generalization bound for classica- tion can be derived as a special case of

Extraction of Protein Domains and Signatures through Unsupervised Statistical Sequence Segmentation

We present a novel information theoretic method for protein domain and statistical signature extr... more We present a novel information theoretic method for protein domain and statistical signature extraction. We apply a new algorithm for unsupervised segmentation of sequences into alternating Variable Memory Markov sources, to families of protein sequences. The algorithm is based on competitive learning between Markov models, implemented as Prediction Suffix Trees , which we have shown in earlier works to model well protein families . By clustering the statistical models themselves, based on rate distortion theory combined with deterministic annealing, we obtain a hierarchical segmentation of sequences between alternating Markov sources that seems to automatically avoid over segmentation. This paper demonstrates the potential of this method in protein sequences analysis. We analyze in detail the resulting segmentation achieved for several diverse protein families, and demonstrate the automatic differentiation between different known domains within the sequences. The method also has the potential to pinpoint exact domain boundaries or sub-divide a family into biological sub-families where other computational tools do not.

Online Learning in Markov Decision Processes with Adversarially Chosen Transition Probability Distributions

We study the problem of learning Markov decision processes with finite state and action spaces wh... more We study the problem of learning Markov decision processes with finite state and action spaces when the transition probability distributions and loss functions are chosen adversarially and are allowed to change with time. We introduce an algorithm whose regret with respect to any policy in a comparison class grows as the square root of the number of rounds of the game, provided the transition probabilities satisfy a uniform mixing condition. Our approach is efficient as long as the comparison class is polynomial and we can compute expectations over sample paths for each policy. Designing an efficient algorithm with small regret for the general case remains an open problem.

One Practical Algorithm for Both Stochastic and Adversarial Bandits

We present an algorithm for multiarmed bandits that achieves almost optimal performance in both s... more We present an algorithm for multiarmed bandits that achieves almost optimal performance in both stochastic and adversarial regimes without prior knowledge about the nature of the environment. Our algorithm is based on augmentation of the EXP3 algorithm with a new control lever in the form of exploration parameters that are tailored individually for each arm. The algorithm simultaneously applies the "old" control lever, the learning rate, to control the regret in the adversarial regime and the new control lever to detect and exploit gaps between the arm losses. This secures problem-dependent "logarithmic" regret when gaps are present without compromising on the worst-case performance guarantee in the adversarial regime. We show that the algorithm can exploit both the usual expected gaps between the arm losses in the stochastic regime and deterministic gaps between the arm losses in the adversarial regime. The algorithm retains "logarithmic" regret guarantee in the stochastic regime even when some observations are contaminated by an adversary, as long as on average the contamination does not reduce the gap by more than a half. Our results for the stochastic regime are supported by experimental validation.

On PAC-Bayesian bounds for random forests

Machine Learning

Existing guarantees in terms of rigorous upper bounds on the generalization error for the origina... more Existing guarantees in terms of rigorous upper bounds on the generalization error for the original random forest algorithm, one of the most frequently used machine learning methods, are unsatisfying. We discuss and evaluate various PAC-Bayesian approaches to derive such bounds. The bounds do not require additional hold-out data, because the out-of-bag samples from the bagging in the training process can be exploited. A random forest predicts by taking a majority vote of an ensemble of decision trees. The first approach is to bound the error of the vote by twice the error of the corresponding Gibbs classifier (classifying with a single member of the ensemble selected at random). However, this approach does not take into account the effect of averaging out of errors of individual classifiers when taking the majority vote. This effect provides a significant boost in performance when the errors are independent or negatively correlated, but when the correlations are strong the advantage from taking the majority vote is small. The second approach based on PAC-Bayesian C-bounds takes dependencies between ensemble members into account, but it requires estimating correlations between the errors of the individual classifiers. When the correlations are high or the estimation is poor, the bounds degrade. In our experiments, we compute generalization bounds for random forests on various benchmark data sets. Because the individual decision trees already perform well, their predictions are highly correlated and the C-bounds do not lead to satisfactory results. For the same reason, the bounds based on the analysis of Gibbs classifiers are typically superior and often reasonably tight. Bounds based on a validation set coming at the cost of a smaller training set gave better performance guarantees, but worse performance in most experiments.

Information Bottleneck for Non Co-Occurence Data Supplementary Material

This technical report provides supplementary material for the paper ìInformation Bottleneck for N... more This technical report provides supplementary material for the paper ìInformation Bottleneck for Non Co-Occurrence Dataî (1). The report includes run time analy- sis for the algorithm proposed in the ìTheoryî section of the paper; detailed results of the algorithm application to the ESR dataset reported in the ìApplicationsî sec- tion of the paper; visualization of the solution of MovieLens data

Unsupervised Sequence Segmentation by a Mixture of Switching Variable Memory Markov Sources

International Conference on Machine Learning, 2001

We present a novel information theoretic algorithm for unsupervised segmentation of sequences int... more We present a novel information theoretic algorithm for unsupervised segmentation of sequences into alternating Variable Memory Markov sources. The algorithm is based on competitive learning between Markov models, when implemented as Prediction Suffix Trees (Ron et al., 1996) using the MDL principle. By applying a model clustering procedure, based on rate distortion theory combined with deterministic annealing, we obtain a

PAC-Bayesian Analysis of Martingales and Multiarmed Bandits

Eprint Arxiv 1105 2416, May 12, 2011

We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random v... more We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This lemma provides an alternative tool to Hoeffding-Azuma inequality to bound concentration of martingale values. Our second approach is based on integration of Hoeffding-Azuma inequality with PAC-Bayesian analysis. We also introduce a way to apply PAC-Bayesian analysis in situation of limited feedback. We combine the new tools to derive PAC-Bayesian generalization and regret bounds for the multiarmed bandit problem. Although our regret bound is not yet as tight as state-of-the-art regret bounds based on other well-established techniques, our results significantly expand the range of potential applications of PAC-Bayesian analysis and introduce a new analysis tool to reinforcement learning and many other fields, where martingales and limited feedback are encountered.

PAC-Bayesian Analysis of the Exploration-Exploitation Trade-off

Corr Abs 1105 4585, May 20, 2011

We develop a coherent framework for integrative simultaneous analysis of the explorationexploitat... more We develop a coherent framework for integrative simultaneous analysis of the explorationexploitation and model order selection tradeoffs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a combination is also of independent interest for studies of multiple simultaneously evolving martingales.

Markovian domain fingerprinting in statistical segmentation of protein sequences

PAC-Bayesian Analysis of Contextual Bandits

We derive an instantaneous (per-round) data-dependent regret bound for stochastic multiarmed band... more We derive an instantaneous (per-round) data-dependent regret bound for stochastic multiarmed bandits with side information (also known as contextual bandits). The scaling of our regret bound with the number of states (contexts) N goes as p NI ⇢t (S; A), where I ⇢t (S; A) is the mutual information between states and actions (the side information) used by the algorithm at round t. If the algorithm uses all the side information, the regret bound scales as p N ln K, where K is the number of actions (arms). However, if the side information I ⇢t (S; A) is not fully used, the regret bound is significantly tighter. In the extreme case, when I ⇢t (S; A) = 0, the dependence on the number of states reduces from linear to logarithmic. Our analysis allows to provide the algorithm large amount of side information, let the algorithm to decide which side information is relevant for the task, and penalize the algorithm only for the side information that it is using de facto. We also present an algorithm for multiarmed bandits with side information with O(K) computational complexity per game round.

PAC-Bayesian Analysis of Co-clustering and Beyond

Journal of Machine Learning Research - JMLR, 2010

We derive PAC-Bayesian generalization bounds for supervised and unsupervised learning models base... more We derive PAC-Bayesian generalization bounds for supervised and unsupervised learning models based on clustering, such as co-clustering, matrix tri-factorization, graphical models, graph clustering, and pairwise clustering. 1 We begin with the analysis of co-clustering, which is a widely used approach to the analysis of data matrices. We distinguish among two tasks in matrix data analysis: discriminative prediction of the missing entries in data matrices and estimation of the joint probability distribution of row and column variables in co-occurrence matrices. We derive PAC-Bayesian generalization bounds for the expected out-of-sample performance of co-clustering-based solutions for these two tasks. The analysis yields regularization terms that were absent in the previous formulations of co-clustering. The bounds suggest that the expected performance of co-clustering is governed by a trade-off between its empirical performance and the mutual information preserved by the cluster variables on row and column IDs. We derive an iterative projection algorithm for finding a local optimum of this trade-off for discriminative prediction tasks. This algorithm achieved stateof-the-art performance in the MovieLens collaborative filtering task. Our co-clustering model can also be seen as matrix tri-factorization and the results provide generalization bounds, regularization terms, and new algorithms for this form of matrix factorization. The analysis of co-clustering is extended to tree-shaped graphical models, which can be used to analyze high dimensional tensors. According to the bounds, the generalization abilities of treeshaped graphical models depend on a trade-off between their empirical data fit and the mutual information that is propagated up the tree levels. We also formulate weighted graph clustering as a prediction problem: given a subset of edge weights we analyze the ability of graph clustering to predict the remaining edge weights. The analysis of co-clustering easily extends to this problem and suggests that graph clustering should optimize the trade-off between empirical data fit and the mutual information that clusters preserve on graph nodes.

PAC-Bayesian Analysis of Co-clustering with Extensions to Matrix Tri-factorization, Graph Clustering, Pairwise Clustering, and Graphical Models

Citeseer

... first non-trivial application. 2. The PAC-Bayesian bound for classification (5) is a direct c... more

Unsupervised sequence segmentation by a mixture of switching variable memory Markov sources

… WORKSHOP THEN CONFERENCE …, 2001

PAC-Bayesian analysis of contextual bandits

We derive an instantaneous (per-round) data-dependent regret bound for stochastic multiarmed band... more We derive an instantaneous (per-round) data-dependent regret bound for stochastic multiarmed bandits with side information (also known as contextual bandits). The scaling of our regret bound with the number of states (contexts) N goes as p NI ⇢t (S; A), where I ⇢t (S; A) is the mutual information between states and actions (the side information) used by the algorithm at round t. If the algorithm uses all the side information, the regret bound scales as p N ln K, where K is the number of actions (arms). However, if the side information I ⇢t (S; A) is not fully used, the regret bound is significantly tighter. In the extreme case, when I ⇢t (S; A) = 0, the dependence on the number of states reduces from linear to logarithmic. Our analysis allows to provide the algorithm large amount of side information, let the algorithm to decide which side information is relevant for the task, and penalize the algorithm only for the side information that it is using de facto. We also present an algorithm for multiarmed bandits with side information with O(K) computational complexity per game round.

PAC-Bayesian analysis of martingales and multiarmed bandits

We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random v... more We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This lemma provides an alternative tool to Hoeffding-Azuma inequality to bound concentration of martingale values. Our second approach is based on integration of Hoeffding-Azuma inequality with PAC-Bayesian analysis. We also introduce a way to apply PAC-Bayesian analysis in situation of limited feedback. We combine the new tools to derive PAC-Bayesian generalization and regret bounds for the multiarmed bandit problem. Although our regret bound is not yet as tight as state-of-the-art regret bounds based on other well-established techniques, our results significantly expand the range of potential applications of PAC-Bayesian analysis and introduce a new analysis tool to reinforcement learning and many other fields, where martingales and limited feedback are encountered.

PAC-Bayes-Bernstein Inequality for Martingales and its Application to Multiarmed Bandits

We combine PAC-Bayesian analysis with a Bernstein-type inequality for martingales to obtain a res... more We combine PAC-Bayesian analysis with a Bernstein-type inequality for martingales to obtain a result that makes it possible to control the concentration of multiple (possibly uncountably many) simultaneously evolving and interdependent martingales. We apply this result to derive a regret bound for the multiarmed bandit problem. Our result forms a basis for integrative simultaneous analysis of exploration-exploitation and model order selection trade-offs. It also opens a way for applying PAC-Bayesian analysis in other fields, where sequentially dependent samples and limited feedback are encountered.

Method for content presentation

Enhanced electronic program guides

by Yevgeny Seldin and Miriam Goldenberg

On Unsupervised Learning of Mixtures of Markov Sources

Abstract: Unsupervised classication, or clustering, is one of the basic problems in data analysis... more Abstract: Unsupervised classication, or clustering, is one of the basic problems in data analysis. Whilethe problem of unsupervised classication of independent random variables has been deeplyinvestigated, the problem of unsupervised classication of dependent random variables, andin particular the problem of segmentation of mixtures of Markov sources, has been hardly addressed.At the same time supervised classication of Markov sources has become

PAC-Bayesian Generalization Bound for Density Estimation with Application to Co-clustering

We derive a PAC-Bayesian generalization bound for density estimation. Similar to the PAC-Bayesian... more We derive a PAC-Bayesian generalization bound for density estimation. Similar to the PAC-Bayesian generalization bound for clas- sication, the result has the appealingly sim- ple form of a tradeo between empirical per- formance and the KL-divergence of the pos- terior from the prior. Moreover, the PAC- Bayesian generalization bound for classica- tion can be derived as a special case of

Extraction of Protein Domains and Signatures through Unsupervised Statistical Sequence Segmentation

We present a novel information theoretic method for protein domain and statistical signature extr... more We present a novel information theoretic method for protein domain and statistical signature extraction. We apply a new algorithm for unsupervised segmentation of sequences into alternating Variable Memory Markov sources, to families of protein sequences. The algorithm is based on competitive learning between Markov models, implemented as Prediction Suffix Trees , which we have shown in earlier works to model well protein families . By clustering the statistical models themselves, based on rate distortion theory combined with deterministic annealing, we obtain a hierarchical segmentation of sequences between alternating Markov sources that seems to automatically avoid over segmentation. This paper demonstrates the potential of this method in protein sequences analysis. We analyze in detail the resulting segmentation achieved for several diverse protein families, and demonstrate the automatic differentiation between different known domains within the sequences. The method also has the potential to pinpoint exact domain boundaries or sub-divide a family into biological sub-families where other computational tools do not.

Online Learning in Markov Decision Processes with Adversarially Chosen Transition Probability Distributions

We study the problem of learning Markov decision processes with finite state and action spaces wh... more We study the problem of learning Markov decision processes with finite state and action spaces when the transition probability distributions and loss functions are chosen adversarially and are allowed to change with time. We introduce an algorithm whose regret with respect to any policy in a comparison class grows as the square root of the number of rounds of the game, provided the transition probabilities satisfy a uniform mixing condition. Our approach is efficient as long as the comparison class is polynomial and we can compute expectations over sample paths for each policy. Designing an efficient algorithm with small regret for the general case remains an open problem.

One Practical Algorithm for Both Stochastic and Adversarial Bandits

We present an algorithm for multiarmed bandits that achieves almost optimal performance in both s... more We present an algorithm for multiarmed bandits that achieves almost optimal performance in both stochastic and adversarial regimes without prior knowledge about the nature of the environment. Our algorithm is based on augmentation of the EXP3 algorithm with a new control lever in the form of exploration parameters that are tailored individually for each arm. The algorithm simultaneously applies the "old" control lever, the learning rate, to control the regret in the adversarial regime and the new control lever to detect and exploit gaps between the arm losses. This secures problem-dependent "logarithmic" regret when gaps are present without compromising on the worst-case performance guarantee in the adversarial regime. We show that the algorithm can exploit both the usual expected gaps between the arm losses in the stochastic regime and deterministic gaps between the arm losses in the adversarial regime. The algorithm retains "logarithmic" regret guarantee in the stochastic regime even when some observations are contaminated by an adversary, as long as on average the contamination does not reduce the gap by more than a half. Our results for the stochastic regime are supported by experimental validation.

An Improved Multileaving Algorithm for Online Ranker Evaluation

by C. Lioma, Brian Brost, and Yevgeny Seldin

Online ranker evaluation is a key challenge in information retrieval. An important task in the on... more Online ranker evaluation is a key challenge in information retrieval. An important task in the online evaluation of rankers is using implicit user feedback for inferring preferences between rankers. Interleaving methods have been found to be efficient and sensitive, i.e. they can quickly detect even small differences in quality. It has recently been shown that multileaving methods exhibit similar sensitivity but can be more efficient than interleaving methods. This paper presents empirical results demonstrating that existing multileaving methods either do not scale well with the number of rankers, or, more problematically, can produce results which substantially differ from evaluation measures like NDCG. The latter problem is caused by the fact that they do not correctly account for the similarities that can occur between rankers being multileaved. We propose a new multileaving method for handling this problem and demonstrate that it substantially outperforms existing methods, in some cases reducing errors by as much as 50%.

Multi-Dueling Bandits and Their Application to Online Ranker Evaluation

by C. Lioma, Brian Brost, and Yevgeny Seldin

New ranking algorithms are continually being developed and refined, necessitating the development... more New ranking algorithms are continually being developed and refined, necessitating the development of efficient methods for evaluating these rankers. Online ranker evaluation focuses on the challenge of efficiently determining, from implicit user feedback, which ranker out of a finite set of rankers is the best. Online ranker evaluation can be modeled by dueling ban- dits, a mathematical model for online learning under limited feedback from pairwise comparisons. Comparisons of pairs of rankers is performed by interleaving their result sets and examining which documents users click on. The dueling bandits model addresses the key issue of which pair of rankers to compare at each iteration, thereby providing a solution to the exploration-exploitation trade-off. Recently, methods for simultaneously comparing more than two rankers have been developed. However, the question of which rankers to compare at each iteration was left open. We address this question by proposing a generalization of the dueling bandits model that uses simultaneous comparisons of an unrestricted number of rankers. We evaluate our algorithm on synthetic data and several standard large-scale online ranker evaluation datasets. Our experimental results show that the algorithm yields orders of magnitude improvement in performance compared to stateof- the-art dueling bandit algorithms.