This category groups articles dealing with supervised problems. Each post focuses on either a specific supervised algorithm, or a tool used when tackling a supervised problem. The emphasis here is on understanding these models and techniques at a technical level. Here you will learn to build supervised models in Python from scratch.
Unbalanced data is a common occurrence for classification problems, with significant implications for model performance. In this post, we’ll compare 4 different techniques for treating unbalanced data.
4 Techniques to Handle Unbalanced Data Read More »
We will outline 8 key advantages and disadvantages of Decision Trees in this post. Both classification and regression Decision Trees will considered.
8 Key Advantages and Disadvantages of Decision Trees Read More »
Pruning Decision Trees involves a set of techniques that can be used to simplify a Decision Tree, and enable it to generalise better.
Pruning Decision Trees in 3 Easy Examples Read More »
Build a Decision Tree in Python from Scratch Yes, Decision Trees handle categorical features naturally. Often these features are treated by first one-hot-encoding (OHE) in a preprocessing step. However, it is straightforward to extend the CART algorithm to make use of categorical features without such preprocessing. In this post, I will implement classification and regression Decision
Can Decision Trees Handle Categorical Features? Read More »
Build a Decision Tree in Python from Scratch Yes, Decision Trees handle missing values naturally. It is straightforward to extend the CART algorithm to support the handling of missing values. However, attention needs to be made regarding how the algorithm is implemented in code. In this post, I will implement classification and regression Decision Trees capable
Can Decision Trees Handle Missing Values? Read More »
Can Decision Trees Handle Missing Values? In general, Decision Trees are quite robust to the presence of outliers in the data. This is true for both training and prediction. However, care needs to be taken to ensure the Decision Tree has been adequately regularised. An overfitted Decision Tree will show sensitivity to outliers. Why are
Are Decision Trees Robust to Outliers? Read More »
Can Decision Trees Handle Missing Values? We can interpret Decision Trees as a sequence of simple questions for our data, with yes/no answers. One starts at the root node, where the first question is asked. Based upon the answer, we navigate to one of two child nodes. Each child node asks an additional question, and based
How to Interpret Decision Trees with 1 Simple Example Read More »
Build a Decision Tree in Python from Scratch We can tune hyperparameters in Decision Trees by comparing models trained with different parameter configurations, on the same data. An optimal model can then be selected from the various different attempts, using any relevant metrics. There are several different techniques for accomplishing this task. Three of the
3 Methods to Tune Hyperparameters in Decision Trees Read More »
How Neural Networks Learn, with 1 Complete Example In this post, we will cover the K Nearest Neighbours algorithm: how it works and how it can be used. We will work through implementing this algorithm in Python from scratch, and verify that our model works as expected. What is the KNN Algorithm? K Nearest Neighbours
Implement the KNN Algorithm in Python from Scratch Read More »
Can Decision Trees Handle Missing Values? In this post, we will build a CART Decision Tree model in Python from scratch. We will start with the foundational principals, and work straight through to implementation in code. Both classification and regression examples will be included. Note: for those who prefer video content, you can watch me work
Build a Decision Tree in Python from Scratch Read More »