Geometric interpretation of probability distributions — Bayesian inference is based on the fact that we often don’t know the underlying distribution of data, so we need to build a model and then iteratively adjust it as we get more data. In parametric Bayesian inference you start with picking a general form of the probability distribution f(x;θ)…

Using causal modeling for feature selection and creation — Model Robustness We would all like our Machine Learning models generalize on the unseen data, but often find that our model performance drops when the new data do not look like the old data, that is have a different distribution. For example, a medical diagnostics system we trained on the data from…

Using Legendre polynomials as the features can significantly improve convergence — TL;DR In this article, we revisit an apparent implicit regularization effect of Gradient Descent-based optimization algorithms. We consider it on a textbook example of overfitting that you can find on scikit-learn website. …

A tutorial on deep signal processing — In signal processing, it is sometimes necessary to measure horizontal distance between some features of the signal, for example, the peaks. A good example of this could be interpreting an electrocardiogram (ECG), which relies on measuring distances for much of its interpretation. …

Small synthetic data set can be enough for training a model. — The topic of data-efficient learning an important topic in Data Science and is an active area of research. Training large models on big data could take a lot of time and resources, so the question is can we replace a large data set with a smaller one, that will nevertheless…

When all you have are categorical variables — I have spent some time studying data with categorical variables trying to explore many ways to encode them into numeric features. What if all your variables are categorical? One of the mechanism to describe this scenario known as contingency tables. Contingency tables in their essence are (potentially multidimensional) tables where…

A deep learning algorithm for real world time series data — As a data scientist working primarily with business data (sometimes also called “tabular data”), I’m always looking for latest development in the areas of data science that helps work with the more realistic data. One of this area addresses the fact that business data are rarely “tabular”, but usually are…

Predict a distribution of the target variable, not just point estimate — While looking through the ICML 2020 accepted papers, I found an interesting paper: NGBoost: Natural Gradient Boosting for Probabilistic Prediction We present Natural Gradient Boosting (NGBoost), an algorithm for generic probabilistic prediction via gradient…arxiv.org You may ask, how many more papers we need on Gradient Boosting? But in fact, the GBT family of algorithms works really well on tabular data, consistently taking the first places in the Kaggle leaderboards. …