Customer Segmentation Using Machine Learning (K-Means Clustering) – Full Python Data Science Project

This video introduces a customer segmentation project using the K-Means machine learning algorithm. The project will involve data analysis, cleaning, feature engineering, clustering model creation, and a Streamlit web application for predictions. The dataset used, 'customer segmentation.csv', focuses on customer personality analysis and includes 29 columns with various customer attributes like ID, birth year, education, marital status, income, and spending habits.

The project begins by importing necessary libraries such as pandas, numpy, matplotlib, and seaborn in a Jupyter Notebook. The 'customer segmentation.csv' dataset is loaded into a pandas DataFrame. Initial exploration includes checking the DataFrame's head, column names, shape (2240 rows, 29 columns), and data types using `df.info()`. The dataset contains one float, 25 integers, and three object type columns.

A crucial step in data cleaning is identifying and addressing missing values. The `df.isna().sum()` command reveals 24 missing values, all in the 'Income' column. These missing values are dropped from the dataset using `df.dropna(inplace=True)`, ensuring a clean dataset for further analysis. After dropping, verifying `df.isna().sum()` shows zero missing values.

Categorical variables are analyzed for relationships. Box plots compare income by education level, revealing higher incomes for PhD holders and lower for basic education. Spending by marital status is also examined, showing widows and singles with higher total spending. A correlation heatmap is generated for key numerical features like income, age, total spending, web purchases, and store purchases. This visual representation highlights strong positive correlations, such as between income and total spending (0.79) and total spending with web/store purchases.

Several new features are engineered to enhance the dataset. The 'Dt_Customer' column is converted to datetime objects. 'Age' is calculated (2025 - Birth Year), 'TotalChildren' combines 'Kidhome' and 'Teenhome', and 'TotalSpending' sums spending across various product categories. 'CustomerSince' calculates the tenure in days as an indicator of customer loyalty. These engineered features provide more relevant insights for segmentation.

The distribution of key numerical features is visualized using histograms. The 'Age' distribution centers around 50, with a range from 20 to 85. 'Income' exhibits a right-skewed distribution, with most customers having lower to medium incomes. 'TotalSpending' also shows a right-skewed pattern, indicating that a majority of customers engage in lower spending. These visualizations help understand the overall customer base.

A pivot table is created to analyze the average income by education and marital status, visualized as a heatmap showing income variations across these demographic groups. Bar charts illustrate average spending by education level, again highlighting PhD holders as highest spenders. Campaign acceptance rates are calculated and visualized by marital status. A new 'AcceptedCmpAny' column indicates if a customer accepted at least one campaign, showing varying acceptance rates across different marital statuses.

For segmentation, specific features are selected: 'Age', 'Income', 'TotalSpending', 'NumWebPurchases', 'NumStorePurchases', 'NumWebVisitsMonth', and 'Recency'. These features are scaled using `StandardScaler` to normalize their ranges. The optimal number of clusters for K-Means is determined using the elbow method, with an inertia plot suggesting 6 as a suitable number of clusters. The K-Means model is then initialized and fitted to the scaled data, assigning a cluster label to each customer.

The assigned cluster labels are added back to the original DataFrame. A cluster summary is generated by grouping the data by cluster and calculating the mean of the selected features for each cluster. This helps characterize each customer segment (e.g., 'high income, high spending'). Additionally, Principal Component Analysis (PCA) is used to reduce the dimensionality of the scaled features to two components for visualization. A scatter plot of these PCA components, colored by cluster, provides a visual representation of the distinct customer segments.

To make the model accessible, the trained K-Means model and the `StandardScaler` object are saved using `joblib.dump()`. A Streamlit web application (`segmentation.py`) is then developed to allow users to input customer details and predict their segment. The app loads the saved model and scaler, creates input fields for each feature, and a 'Predict Segment' button. Upon clicking, the app scales the user input, makes a prediction using the K-Means model, and displays the predicted segment.

The Streamlit application is launched in a local browser. Users can input values for age, income, total spending, web purchases, store purchases, web visits, and recency. By changing these inputs and clicking 'Predict Segment', the application provides an instant prediction of the customer's cluster. This interactive interface demonstrates the practical utility of the segmentation model for business users, enabling them to understand and target different customer groups effectively.