Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 3: Model Evaluation and Optimization

3.P1 Loading Data and Splitting x, y

Print / Save as PDF

3.P1 Loading Data and Splitting x, y

After this unit you will: load a CSV file with pandas and separate the feature matrix x from the label y.

Key Points

  • The diabetes.csv dataset has 9 columns: 8 feature columns (Pregnancies, Glucose, BloodPressure, SkinThickness, Insulin, BMI, DiabetesPedigreeFunction, Age) and the label column Outcome (0/1).
  • pd.read_csv("diabetes.csv") produces a DataFrame of 768 rows × 9 columns.
  • x = data.drop(target, axis=1) drops the Outcome column (axis=1 = along columns), leaving the 8 feature columns.
  • y = data[target] takes only the label column Outcome.

Study notes

By the end of this lesson you can:

  • Describe the diabetes.csv dataset: its 8 feature columns, the Outcome label and its size
  • Load a CSV file into a DataFrame with pd.read_csv and display it in Jupyter
  • Explain the axis parameter of drop (0 = rows, 1 = columns)
  • Split a DataFrame "vertically" into the feature matrix x and the label y

Key idea: Before any modelling, load the data and separate it vertically: x holds every feature column, and y holds only the target column the model must predict.

The diabetes prediction problem

The practical section builds, trains and evaluates a model that predicts whether a patient has diabetes. Each row of diabetes.csv is one patient. The columns are:

ColumnRole
Pregnancies, Glucose, BloodPressure, SkinThickness, Insulin, BMI, DiabetesPedigreeFunction, Age8 features
Outcomelabel: 1 = has diabetes, 0 = does not

Because Outcome has only two values, this is a binary classification problem. The other columns are the inputs used to predict it.

Loading the data with pandas

The first notebook cell has two lines:

import pandas as pd
data = pd.read_csv("diabetes.csv")
  • pandas is a library every data scientist should know. It makes it easy to work with structured data such as CSV, Excel, JSON or HTML tables.
  • pd is the conventional short alias for pandas.
  • pandas has a family of reader functions (read_csv, read_excel, read_json, read_html, ...). The file is a CSV, so we use read_csv and store the result in the variable data.

Displaying data in a new cell shows the table. Jupyter shows only the first five and last five rows by default, plus the size: 768 rows x 9 columns (8 features + Outcome).

Splitting the data vertically into x and y

"Splitting vertically" means splitting the columns: features go into x, the target into y.

target = "Outcome"
x = data.drop(target, axis=1)
y = data[target]
  • First define which column is the target.
  • drop removes the target column to create x. The axis parameter says what to drop: axis=0 drops rows, axis=1 drops columns. We want to remove a column, so axis=1.
  • data[target] selects only the Outcome column for y.

Displaying them confirms the result: x has the 8 feature columns (no Outcome), and y is a single column of 0s and 1s. drop returns a new DataFrame; the original data is unchanged.

Why keep x and y separate? Every scikit-learn model is trained with the same pattern, model.fit(x, y): the features and the answers are passed as two separate arguments. Splitting them now also makes the next step, the train/test split, straightforward, because train_test_split receives x and y and keeps their rows aligned.

Worked example

Load the file, split it and check the shapes. (Run in the folder that contains diabetes.csv.)

import pandas as pd

data = pd.read_csv("diabetes.csv")
print(data.shape)

target = "Outcome"
x = data.drop(target, axis=1)
y = data[target]

print(x.shape, y.shape)
print(list(x.columns))
print(y.value_counts())

Expected output, step by step:

  1. data.shape is (768, 9).
  2. x.shape is (768, 8) and y.shape is (768,): same number of rows, y is a single column (a Series).
  3. The column list shows the 8 features from Pregnancies to Age, without Outcome.
  4. y.value_counts() shows how many patients have each label. Checking it early tells you whether the classes are imbalanced, which matters when you read accuracy later.

An equivalent, more explicit way to drop the column is data.drop(columns=target).

Common mistakes

  • Forgetting axis=1. Without it pandas looks for a row called "Outcome", fails, and raises an error.
  • Leaving the label inside x. If Outcome stays in the features, the model can "cheat" by reading the answer.
  • Confusing the feature count. The file has 9 columns; only 8 are features.
  • Expecting drop to change data. It returns a new DataFrame; assign it to x.

Summary

  • diabetes.csv has 768 patients and 9 columns: 8 features and the binary label Outcome.
  • pd.read_csv("diabetes.csv") loads it into a DataFrame.
  • x = data.drop(target, axis=1) keeps all feature columns; axis=1 means columns.
  • y = data[target] keeps only the label.
  • Check shapes and class counts right after splitting.

Flashcards · 3.P1 Loading Data and Splitting x, y

9 words
WordMeaningExample
  • pandas (pd)

    A Python library for working with structured data such as CSV, Excel and JSON; usually imported as pd.

    import pandas as pd

  • pd.read_csv()

    Reads a CSV file into a pandas DataFrame.

    data = pd.read_csv('diabetes.csv')

  • DataFrame

    A pandas table with labelled rows and columns.

    data has 768 rows x 9 columns.

  • Outcome

    The label column of diabetes.csv: 1 = has diabetes, 0 = does not.

    y = data['Outcome']

  • Feature matrix x

    The table of input columns the model uses to make predictions.

    x = data.drop('Outcome', axis=1) -> 8 columns

  • Label y

    The target column the model learns to predict.

    y.shape -> (768,)

  • axis parameter

    In drop, axis=0 removes rows and axis=1 removes columns.

    data.drop('Outcome', axis=1)

  • data.shape

    A DataFrame attribute giving (number of rows, number of columns).

    (768, 9)

  • value_counts()

    Counts how many times each value appears in a column; useful to check class balance.

    y.value_counts()