{ "cells": [ { "cell_type": "code", "execution_count": null, "id": "dffa60e8", "metadata": {}, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "id": "3340eabf", "metadata": { "cell_marker": "\"\"\"" }, "source": [ "Inclass-Lab\n", "\n", "Automatically generated by Colab.\n", "\n", "Original file is located at\n", " https://colab.research.google.com/drive/13N7lQHvv4_LxKgcerumJ-18e5GfdBggm" ] }, { "cell_type": "code", "execution_count": null, "id": "34271feb", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "9491847c", "metadata": { "cell_marker": "\"\"\"" }, "source": [ "In-Class Lab: Comparing Classification Models (Breast Cancer Dataset)\n", "\n", "Learning Objectives\n", "\n", "---\n", "\n", "\n", "By the end of this lab, you should be able to:\n", "\n", "\n", "\n", "* Train multiple models (KNN, Logistic Regression, Decision Tree)\n", "* Compare performance across various scenarios\n", "* Understand the effect of scaling and evaluation methods\n", "* Interpret confusion matrix & classification report\n", "\n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "id": "cceb43dc", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 1:" ] }, { "cell_type": "markdown", "id": "818bfdad", "metadata": { "cell_marker": "'''", "lines_to_next_cell": 0 }, "source": [ "Load dataset\n", "Explain features/target\n", "Do train_test_split" ] }, { "cell_type": "markdown", "id": "6de94b5d", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [] }, { "cell_type": "code", "execution_count": 1, "id": "a8899ca0", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['malignant' 'benign']\n", "(569, 30)\n" ] } ], "source": [ "from sklearn.datasets import load_breast_cancer\n", "data = load_breast_cancer()\n", "X = data.data\n", "y = data.target\n", "\n", "print(data.target_names)\n", "print(X.shape)\n", "#0 - malignant\n", "#1 - benigh" ] }, { "cell_type": "markdown", "id": "ae1b0d61", "metadata": { "cell_marker": "###############################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 2: IMPORT LIBRARIES FOR CLASSIFIERS\n", "##############################" ] }, { "cell_type": "code", "execution_count": 9, "id": "8db6dfd6", "metadata": { "lines_to_next_cell": 0 }, "outputs": [], "source": [ "from sklearn.neighbors import KNeighborsClassifier\n", "from sklearn.linear_model import LogisticRegression\n", "from sklearn.tree import DecisionTreeClassifier\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "a2301b40", "metadata": { "cell_marker": "#########################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 3: IMPORT MODUL FOR DATA SPLIT\n", "########################################" ] }, { "cell_type": "code", "execution_count": 10, "id": "75fe2532", "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split" ] }, { "cell_type": "markdown", "id": "fb69921d", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 4: SPLIT THE DATASET\n", "###################################" ] }, { "cell_type": "code", "execution_count": 11, "id": "6eb66fc0", "metadata": {}, "outputs": [], "source": [ "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)" ] }, { "cell_type": "markdown", "id": "b182e930", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 5\n", "TASK 1: Train 3 models (no scaling)\n", "###################################" ] }, { "cell_type": "code", "execution_count": 12, "id": "f31f0262", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
DecisionTreeClassifier()
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "DecisionTreeClassifier()" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "knn = KNeighborsClassifier(n_neighbors=5)\n", "logreg = LogisticRegression(max_iter=5000)\n", "tree = DecisionTreeClassifier()\n", "\n", "knn.fit(X_train, y_train)\n", "logreg.fit(X_train, y_train)\n", "tree.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "id": "d3e0088e", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 6\n", "Task 2: Evaluate accuracy" ] }, { "cell_type": "markdown", "id": "3a8845a1", "metadata": { "cell_marker": "'''", "lines_to_next_cell": 0 }, "source": [ "* Which model performs best?\n", "* Are the results very different?" ] }, { "cell_type": "markdown", "id": "d49b62b3", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [] }, { "cell_type": "code", "execution_count": 6, "id": "395d887c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "KNN: 0.956140350877193\n", "LogReg: 0.956140350877193\n", "Tree: 0.9298245614035088\n" ] } ], "source": [ "print(\"KNN:\", knn.score(X_test, y_test))\n", "print(\"LogReg:\", logreg.score(X_test, y_test))\n", "print(\"Tree:\", tree.score(X_test, y_test))" ] }, { "cell_type": "markdown", "id": "9de74c34", "metadata": { "cell_marker": "####################################" }, "source": [ "BLOCK 7\n", "Part 2: Evaluation Metrics\n", "Task 3: Confusion Matrix" ] }, { "cell_type": "code", "execution_count": 13, "id": "deec64c6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[38 5]\n", " [ 0 71]]\n" ] } ], "source": [ "from sklearn.metrics import confusion_matrix\n", "print(confusion_matrix(y_test, knn.predict(X_test)))" ] }, { "cell_type": "markdown", "id": "18bc0c0e", "metadata": { "cell_marker": "#####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 8: CLASSIFICATION FREPORT" ] }, { "cell_type": "markdown", "id": "a2e8747f", "metadata": { "cell_marker": "'''", "lines_to_next_cell": 0 }, "source": [ "* Which model has better recall?\n", "* Which is better for detecting cancer?" ] }, { "cell_type": "markdown", "id": "88303af7", "metadata": { "cell_marker": "#####################################", "lines_to_next_cell": 0 }, "source": [] }, { "cell_type": "code", "execution_count": 14, "id": "e9ef7bea", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " 0 1.00 0.88 0.94 43\n", " 1 0.93 1.00 0.97 71\n", "\n", " accuracy 0.96 114\n", " macro avg 0.97 0.94 0.95 114\n", "weighted avg 0.96 0.96 0.96 114\n", "\n" ] } ], "source": [ "from sklearn.metrics import classification_report\n", "print(classification_report(y_test, knn.predict(X_test)))" ] }, { "cell_type": "markdown", "id": "c1d8978a", "metadata": { "cell_marker": "########################################" }, "source": [ "BLOCK 9:\n", "Part 3: Scaling Effect\n", "Task 5: Apply scaling using pipeline\n", "#######################################" ] }, { "cell_type": "code", "execution_count": 15, "id": "84e1f5d4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== RESULTS WITH SCALING ===\n", "KNN Accuracy: 0.9473684210526315\n", "Logistic Regression Accuracy: 0.9736842105263158\n", "Decision Tree Accuracy: 0.9473684210526315\n" ] } ], "source": [ "from sklearn.pipeline import Pipeline\n", "from sklearn.preprocessing import StandardScaler\n", "from sklearn.metrics import accuracy_score, confusion_matrix, classification_report\n", "\n", "knn_scaled = Pipeline([\n", " ('scaler', StandardScaler()),\n", " ('knn', KNeighborsClassifier(n_neighbors=5))\n", "])\n", "\n", "logreg_scaled = Pipeline([\n", " ('scaler', StandardScaler()),\n", " ('logreg', LogisticRegression(max_iter=5000))\n", "])\n", "\n", "tree_scaled = Pipeline([\n", " ('scaler', StandardScaler()),\n", " ('tree', DecisionTreeClassifier(random_state=42))\n", "])\n", "\n", "knn_scaled.fit(X_train, y_train)\n", "logreg_scaled.fit(X_train, y_train)\n", "tree_scaled.fit(X_train, y_train)\n", "\n", "knn_pred_scaled = knn_scaled.predict(X_test)\n", "logreg_pred_scaled = logreg_scaled.predict(X_test)\n", "tree_pred_scaled = tree_scaled.predict(X_test)\n", "\n", "print(\"\\n=== RESULTS WITH SCALING ===\")\n", "print(\"KNN Accuracy:\", accuracy_score(y_test, knn_pred_scaled))\n", "print(\"Logistic Regression Accuracy:\", accuracy_score(y_test, logreg_pred_scaled))\n", "print(\"Decision Tree Accuracy:\", accuracy_score(y_test, tree_pred_scaled))" ] }, { "cell_type": "markdown", "id": "d5caa8ec", "metadata": { "cell_marker": "#####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 10: CLASSIFICATION REPORT\n", "####################################" ] }, { "cell_type": "code", "execution_count": 16, "id": "6643f0af", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== CLASSIFICATION REPORTS (WITH SCALING) ===\n", "\n", "KNN Report:\n", " precision recall f1-score support\n", "\n", " 0 0.93 0.93 0.93 43\n", " 1 0.96 0.96 0.96 71\n", "\n", " accuracy 0.95 114\n", " macro avg 0.94 0.94 0.94 114\n", "weighted avg 0.95 0.95 0.95 114\n", "\n", "\n", "Logistic Regression Report:\n", " precision recall f1-score support\n", "\n", " 0 0.98 0.95 0.96 43\n", " 1 0.97 0.99 0.98 71\n", "\n", " accuracy 0.97 114\n", " macro avg 0.97 0.97 0.97 114\n", "weighted avg 0.97 0.97 0.97 114\n", "\n", "\n", "Decision Tree Report:\n", " precision recall f1-score support\n", "\n", " 0 0.93 0.93 0.93 43\n", " 1 0.96 0.96 0.96 71\n", "\n", " accuracy 0.95 114\n", " macro avg 0.94 0.94 0.94 114\n", "weighted avg 0.95 0.95 0.95 114\n", "\n" ] } ], "source": [ "print(\"\\n=== CLASSIFICATION REPORTS (WITH SCALING) ===\")\n", "\n", "print(\"\\nKNN Report:\")\n", "print(classification_report(y_test, knn_pred_scaled))\n", "\n", "print(\"\\nLogistic Regression Report:\")\n", "print(classification_report(y_test, logreg_pred_scaled))\n", "\n", "print(\"\\nDecision Tree Report:\")\n", "print(classification_report(y_test, tree_pred_scaled))" ] }, { "cell_type": "markdown", "id": "c38c51d0", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 11: CONFUSION MATRIX\n", "###################################" ] }, { "cell_type": "code", "execution_count": 17, "id": "444fbb13", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== CONFUSION MATRICES (WITH SCALING) ===\n", "\n", "KNN Confusion Matrix:\n", "[[40 3]\n", " [ 3 68]]\n", "\n", "Logistic Regression Confusion Matrix:\n", "[[41 2]\n", " [ 1 70]]\n", "\n", "Decision Tree Confusion Matrix:\n", "[[40 3]\n", " [ 3 68]]\n" ] } ], "source": [ "print(\"\\n=== CONFUSION MATRICES (WITH SCALING) ===\")\n", "\n", "print(\"\\nKNN Confusion Matrix:\")\n", "print(confusion_matrix(y_test, knn_pred_scaled))\n", "\n", "print(\"\\nLogistic Regression Confusion Matrix:\")\n", "print(confusion_matrix(y_test, logreg_pred_scaled))\n", "\n", "print(\"\\nDecision Tree Confusion Matrix:\")\n", "print(confusion_matrix(y_test, tree_pred_scaled))\n", "\n" ] }, { "cell_type": "markdown", "id": "c5d82a08", "metadata": { "cell_marker": "###############################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 12: MODWLS WITH CROSS-VALIDATION\n", "##############################################" ] }, { "cell_type": "code", "execution_count": 18, "id": "17493cc3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "=== Cross-Validation Results ===\n", "\n", "KNN\n", "CV Scores: [0.96491228 0.95614035 0.98245614 0.95614035 0.96460177]\n", "Mean Accuracy: 0.9648501785437045\n", "Standard Deviation: 0.009609970350036127\n", "\n", "Logistic Regression\n", "CV Scores: [0.98245614 0.98245614 0.97368421 0.97368421 0.99115044]\n", "Mean Accuracy: 0.9806862288464524\n", "Standard Deviation: 0.006539441283506109\n", "\n", "Decision Tree\n", "CV Scores: [0.9122807 0.90350877 0.92982456 0.95614035 0.88495575]\n", "Mean Accuracy: 0.9173420276354604\n", "Standard Deviation: 0.02419491828674519\n" ] } ], "source": [ "from sklearn.model_selection import cross_val_score\n", "from sklearn.preprocessing import StandardScaler\n", "\n", "knn_pipeline = Pipeline([\n", " ('scaler', StandardScaler()),\n", " ('knn', KNeighborsClassifier(n_neighbors=5))\n", "])\n", "\n", "logreg_pipeline = Pipeline([\n", " ('scaler', StandardScaler()),\n", " ('logreg', LogisticRegression(max_iter=5000))\n", "])\n", "\n", "tree_model = DecisionTreeClassifier(random_state=42)\n", "\n", "knn_cv_scores = cross_val_score(knn_pipeline, X, y, cv=5, scoring='accuracy')\n", "logreg_cv_scores = cross_val_score(logreg_pipeline, X, y, cv=5, scoring='accuracy')\n", "tree_cv_scores = cross_val_score(tree_model, X, y, cv=5, scoring='accuracy')\n", "\n", "print(\"=== Cross-Validation Results ===\")\n", "\n", "print(\"\\nKNN\")\n", "print(\"CV Scores:\", knn_cv_scores)\n", "print(\"Mean Accuracy:\", np.mean(knn_cv_scores))\n", "print(\"Standard Deviation:\", np.std(knn_cv_scores))\n", "\n", "print(\"\\nLogistic Regression\")\n", "print(\"CV Scores:\", logreg_cv_scores)\n", "print(\"Mean Accuracy:\", np.mean(logreg_cv_scores))\n", "print(\"Standard Deviation:\", np.std(logreg_cv_scores))\n", "\n", "print(\"\\nDecision Tree\")\n", "print(\"CV Scores:\", tree_cv_scores)\n", "print(\"Mean Accuracy:\", np.mean(tree_cv_scores))\n", "print(\"Standard Deviation:\", np.std(tree_cv_scores))" ] }, { "cell_type": "markdown", "id": "94d94dd2", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 13: CONFUSION MATRIX\n", "###################################" ] }, { "cell_type": "code", "execution_count": 19, "id": "c706abc3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== Confusion Matrices ===\n", "\n", "KNN\n", "[[198 14]\n", " [ 6 351]]\n", "\n", "Logistic Regression\n", "[[204 8]\n", " [ 3 354]]\n", "\n", "Decision Tree\n", "[[193 19]\n", " [ 28 329]]\n", "\n", "=== Classification Reports ===\n", "\n", "KNN\n", " precision recall f1-score support\n", "\n", " 0 0.97 0.93 0.95 212\n", " 1 0.96 0.98 0.97 357\n", "\n", " accuracy 0.96 569\n", " macro avg 0.97 0.96 0.96 569\n", "weighted avg 0.96 0.96 0.96 569\n", "\n", "\n", "Logistic Regression\n", " precision recall f1-score support\n", "\n", " 0 0.99 0.96 0.97 212\n", " 1 0.98 0.99 0.98 357\n", "\n", " accuracy 0.98 569\n", " macro avg 0.98 0.98 0.98 569\n", "weighted avg 0.98 0.98 0.98 569\n", "\n", "\n", "Decision Tree\n", " precision recall f1-score support\n", "\n", " 0 0.87 0.91 0.89 212\n", " 1 0.95 0.92 0.93 357\n", "\n", " accuracy 0.92 569\n", " macro avg 0.91 0.92 0.91 569\n", "weighted avg 0.92 0.92 0.92 569\n", "\n" ] } ], "source": [ "from sklearn.model_selection import cross_val_predict\n", "from sklearn.metrics import confusion_matrix, classification_report\n", "\n", "knn_pred = cross_val_predict(knn_pipeline, X, y, cv=5)\n", "logreg_pred = cross_val_predict(logreg_pipeline, X, y, cv=5)\n", "tree_pred = cross_val_predict(tree_model, X, y, cv=5)\n", "\n", "print(\"\\n=== Confusion Matrices ===\")\n", "print(\"\\nKNN\")\n", "print(confusion_matrix(y, knn_pred))\n", "\n", "print(\"\\nLogistic Regression\")\n", "print(confusion_matrix(y, logreg_pred))\n", "\n", "print(\"\\nDecision Tree\")\n", "print(confusion_matrix(y, tree_pred))\n", "\n", "print(\"\\n=== Classification Reports ===\")\n", "print(\"\\nKNN\")\n", "print(classification_report(y, knn_pred))\n", "\n", "print(\"\\nLogistic Regression\")\n", "print(classification_report(y, logreg_pred))\n", "\n", "print(\"\\nDecision Tree\")\n", "print(classification_report(y, tree_pred))" ] }, { "cell_type": "markdown", "id": "34354ec9", "metadata": { "cell_marker": "###########################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 14: LETS GET THE BEST VALUE FOR K\n", "task: use the k value to recompute the previous KNN model and evaluate the performance\n", "#########################################" ] }, { "cell_type": "code", "execution_count": 20, "id": "93fa45e5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Best k: 13\n", "Best CV Accuracy: 0.9332401800962584\n" ] } ], "source": [ "from sklearn.model_selection import cross_val_score\n", "import numpy as np\n", "\n", "k_values = range(1, 21)\n", "scores = []\n", "\n", "for k in k_values:\n", " knn = KNeighborsClassifier(n_neighbors=k)\n", " cv_scores = cross_val_score(knn, X, y, cv=5)\n", " scores.append(np.mean(cv_scores))\n", "\n", "best_k = k_values[np.argmax(scores)]\n", "\n", "print(\"Best k:\", best_k)\n", "print(\"Best CV Accuracy:\", max(scores))\n" ] } ], "metadata": { "jupytext": { "cell_metadata_filter": "-all", "encoding": "# -*- coding: utf-8 -*-", "main_language": "python", "notebook_metadata_filter": "-all" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.14.3" } }, "nbformat": 4, "nbformat_minor": 5 }