Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 3: Model Evaluation and Optimization

3.P3 random_state and Reproducibility

Print / Save as PDF

3.P3 random_state and Reproducibility

After this unit you will: be able to explain why we set random_state when splitting data.

Key Points

  • Splitting train/test is a random split; random_state fixes that random number generator.
  • With the same random_state=1009, running the cell twice gives an identical x_train (the first rows have indices 498, 705, 36, 199, 309).
  • When you comment out # random_state=1009, the next run produces a completely different set (first rows 627, 506, 429, 661, 54).
  • random_state does not make the model better or worse; it only makes the experiment reproducible so comparisons are fair.
  • After the experiment, set random_state=1009 back so later steps use exactly one split.

Study notes

By the end of this lesson you can:

  • Read the row index of x_train to see that the split is random
  • Show that the same random_state reproduces exactly the same split
  • Show that removing random_state gives a different split on every run
  • Explain, with the exam analogy, why a changing test set makes comparisons unfair
  • Explain that random_state affects reproducibility, not model quality

Key idea: random_state does not make a model better or worse; it freezes the random split so every experiment is evaluated on the same test set and results can be compared fairly.

Reading the index of x_train

The lecture adds a cell that displays x_train. The first, unnamed column is the index: the row number from the original data.

  • In the original x, the index increases one by one: 0, 1, 2, 3, ...
  • In x_train the index is no longer in order: 498, 705, 36, 199, 309, ... because train_test_split picked the rows randomly.

Experiment 1: same random_state, same split

With random_state=1009, running the split cell again and printing x_train gives exactly the same first rows (498, 705, 36, ...). The random choice is repeated identically. And since x_train is the same, x_test is the same too.

Experiment 2: no random_state, a new split every time

Next, the random_state=1009 line is commented out and the cell is run again. Now the index changes on every run: a different set of rows appears at the top each time, and x_test changes with it.

For x_train alone that is not a disaster. For x_test it is a real problem: if each training run is tested on a different test set, you cannot tell whether a higher accuracy comes from a better model or from an easier test.

The exam analogy

To compare two students fairly, they must sit the same test. If student A scores 9 on one test and student B scores 8 on a different test, you cannot conclude that A is better, because different tests have different levels of difficulty. Test sets work the same way.

Back to a fixed split

After the experiment, the line random_state=1009 is put back. Running the cell several times again shows the same indices every time: the test set is fixed, and all later steps (scaling, training, evaluation) use exactly one split.

Note what random_state does not do: changing it from 1009 to another number does not improve or worsen the model. It only selects a different (but repeatable) split. If two people use different random_state values, their test scores come from different test sets and should not be compared directly.

Worked example

Check reproducibility in code instead of by eye.

from sklearn.model_selection import train_test_split

a = train_test_split(x, y, test_size=0.2, random_state=1009)[0]
b = train_test_split(x, y, test_size=0.2, random_state=1009)[0]
c = train_test_split(x, y, test_size=0.2)[0]          # no random_state

print(a.index[:5].tolist())
print(a.index.equals(b.index))
print(a.index.equals(c.index))

Expected output, step by step:

  1. The first line prints the first five row indices of x_train; in the lecture's setup they are [498, 705, 36, 199, 309].
  2. True: the two splits with random_state=1009 are identical.
  3. False (almost certainly): without random_state, the split is a new random draw.

[0] takes the first returned object, x_train. Comparing the index is a quick way to check that two DataFrames contain the same rows in the same order.

Common mistakes

  • Thinking a "good" random_state improves the model. It only fixes which rows are in the test set.
  • Comparing scores from different random_state values. Different seeds mean different test sets; the comparison is not fair.
  • Leaving random_state commented out after experimenting. Every later step then works on a different split each run.
  • Worrying only about x_train. A changing training set is less of a problem; a changing test set breaks comparisons.

Summary

  • train_test_split picks rows randomly, which is visible in the shuffled index of x_train.
  • The same random_state (1009) reproduces exactly the same split on every run.
  • Without random_state, each run gives a different x_train and x_test.
  • Like students sitting the same exam, models must be compared on the same test set.
  • random_state gives reproducibility, not better accuracy; put it back before continuing.

Flashcards · 3.P3 random_state and Reproducibility

7 words
WordMeaningExample
  • random_state

    A seed that fixes random operations so they give the same result on every run.

    train_test_split(x, y, random_state=1009)

  • Reproducibility

    Getting exactly the same results when an experiment is run again.

    Same seed -> same x_train and x_test.

  • DataFrame index

    The row labels of a DataFrame; after a random split they show which original rows were picked.

    x_train.index[:5] -> 498, 705, 36, 199, 309

  • Random split

    Assigning rows to train and test by chance; without a seed it changes on every call.

    Commenting out random_state gives new indices each run.

  • Fixed test set

    A test set that stays identical across experiments, so score differences reflect model changes.

    Keep random_state=1009 for all runs.

  • Same-exam analogy

    Students must take the same test to be compared; models must be evaluated on the same test set.

    9 on test A vs 8 on test B proves nothing.

  • index.equals()

    pandas method that checks whether two indexes contain the same labels in the same order.

    a.index.equals(b.index) -> True