Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 2: Core Machine Learning Algorithms

Capstone Practice: Choosing an Algorithm for Your Problem

Print / Save as PDF

Capstone Practice: Choosing an Algorithm for Your Problem

This lesson pulls together the algorithms you have learned — Linear Regression, Logistic Regression, Decision Tree, Random Forest and SVM — into one algorithm-selection reasoning exercise. Do it after you have completed units 2.1 through 2.8.

The task is in the Assignment section below.

Study notes

By the end of this lesson you can:

  • Decide whether a real-world problem is regression or classification from its target column
  • Match a problem to a suitable Module 2 algorithm and justify the choice in writing
  • Use explainability, data shape and data size as criteria when comparing algorithms
  • Plan a simple baseline before trying a more complex model

Key idea: Choosing an algorithm is a reasoned argument, not a guess: start from the type of target, then weigh explainability, the shape of the relationship, the size of the data and the preprocessing each algorithm needs.

What this practice asks you to do

This capstone brings together the five algorithms from units 2.1 to 2.8: Linear Regression, Logistic Regression, decision trees, Random Forest and the Support Vector Machine. The task itself is in the Assignment section of the lesson. These notes give you a checklist for writing a well-argued answer.

Step 1: identify the type of problem

Look at the column you need to predict.

TargetProblem typeCandidate algorithms
a continuous number (salary, house price)regressionLinear Regression, Random Forest (RandomForestRegressor)
a category (Junior/Senior, approve/reject, spam/normal)classificationLogistic Regression, decision tree, Random Forest, SVM

Remember the naming trap from unit 2.2: Logistic Regression is a classification algorithm.

Step 2: recall what each algorithm is good at

  • Linear Regression: fits a straight line or hyperplane; simple and easy to interpret (each coefficient says how much the target changes per unit of a feature); cannot capture curved relationships.
  • Logistic Regression: sigmoid plus threshold gives a probability and a label; simple, fast, a natural baseline for classification.
  • Decision tree: a flowchart of yes/no questions chosen by Gini impurity; very easy to explain, but a single tree is unstable and overfits easily.
  • Random Forest: many trees on bootstrap samples, combined by majority vote or averaging; robust and accurate, but much harder to explain than one tree.
  • SVM: maximum margin boundary decided by the support vectors; the kernel trick handles curved boundaries; works well on small datasets but needs standardised features.

Step 3: ask the three key questions

  1. Must the decision be explained? A bank rejecting a loan or a hospital suggesting a diagnosis often has to justify the decision. Prefer a decision tree or a linear/logistic model, even if accuracy is a little lower.
  2. Is the relationship linear? Plot the data. A clear curve, or one class surrounded by another, rules out a plain linear model; consider trees, Random Forest, or SVM with an RBF kernel.
  3. How much data is there? SVM shines on small datasets (a few hundred samples) but becomes slow on very large ones; linear models and tree ensembles scale better.

Step 4: plan preprocessing and a baseline

  • Distance-based SVM requires standardisation; Linear and Logistic Regression benefit from it; trees do not need it.
  • Always begin with a simple baseline (Linear or Logistic Regression). If a complex model does not clearly beat it, keep the simple one.

Worked example

Scenario: a small clinic has 300 patient records with 6 numeric measurements and wants to predict whether a patient needs a follow-up visit (yes/no). Doctors want to see why a patient was flagged.

Reasoning step by step:

  1. Target is yes/no, so this is binary classification. Linear Regression is out.
  2. Doctors want explanations, so a decision tree (readable path of questions) or Logistic Regression (readable weights) should be the main candidates.
  3. Only 300 records: a single deep tree may overfit, so limit its depth (for example max_depth=3). An SVM would also suit the small size, but it is hard to explain to doctors.
  4. Plan: Logistic Regression as baseline, a shallow decision tree as the explainable candidate, and Random Forest only as an accuracy reference. Standardise features for Logistic Regression.
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

candidates = {
    "baseline": make_pipeline(StandardScaler(), LogisticRegression()),
    "tree": DecisionTreeClassifier(max_depth=3, random_state=100),
    "forest": RandomForestClassifier(random_state=100),
}
for name, model in candidates.items():
    model.fit(x_train, y_train)
    print(name, model.score(x_test, y_test))

Expected result: three accuracy scores. If the shallow tree is close to the forest, choose the tree because it can be explained; if the baseline is as good as both, choose the baseline.

Common mistakes

  • Choosing by popularity. "Random Forest is the strongest" is not a justification; link the choice to the problem's requirements.
  • Using Linear Regression for a yes/no target. Categories need a classifier.
  • Forgetting preprocessing requirements. An SVM without standardised features can perform very badly.
  • Skipping the baseline. Without it you cannot tell whether a complex model adds anything.

Summary

  • Start from the target: a number means regression, a category means classification.
  • Each Module 2 algorithm has strengths and costs: interpretability, flexibility, stability, data size, scaling needs.
  • Ask: must it be explained, is it linear, how large is the data?
  • Plan preprocessing (standardise for SVM) and always begin with a simple baseline.
  • Write your choice as an argument that connects these criteria to the scenario.

Flashcards · Capstone Practice: Choosing an Algorithm for Your Problem

7 words
WordMeaningExample
  • Regression problem

    A task whose target is a continuous number.

    Predicting house price in billion VND.

  • Classification problem

    A task whose target is a category or class label.

    Approve or reject a loan application.

  • Interpretability

    How easily people can understand why a model made a particular prediction.

    A decision tree path explains a loan rejection.

  • Baseline model

    A simple first model that gives a reference score for judging more complex models.

    LogisticRegression before trying Random Forest.

  • Algorithm selection criteria

    Factors used to choose a model: target type, explainability, linearity, data size and preprocessing needs.

    Small, explainable classification task -> shallow decision tree.

  • Shallow tree

    A decision tree with limited depth, which is easier to explain and less likely to overfit.

    DecisionTreeClassifier(max_depth=3)

  • Accuracy reference model

    A powerful but less explainable model used to see how much accuracy an explainable model gives up.

    Compare a shallow tree with RandomForestClassifier.